hydra
Search…
Hydra Indexer
Hydra Indexer is a daemon that ingests raw substrate data such as events and extrinsics from a substrate chain and saves it into a Postgres database. It is typically used in conjunction with Hydra Indexer Gateway providing a GraphQL API interface.

Installation

The indexer exits if an unrecoverable error occurs. It is therefore advisable to run the indexer with a process manager, such as PM2 or Docker.

Prerequisites

    Node 12.x
    Postgres database
    Redis instances
    (Optional) Docker

Environment variables

The indexer is set up using the following environment variables
Variable
Default
Required
Description
WS_PROVIDER_ENDPOINT_URI
-
Yes
Substrate API endpoint to connect
REDIS_URI
-
Yes
Redis instance URI
DB_NAME
-
Yes
Database name
DB_PORT
-
Yes
Database port
DB_HOST
-
Yes
Database host
DB_USER
-
Yes
Database user
DB_PASS
-
Yes
Database password
TYPES_JSON
-
No
Path to a JSON type definition with custom Substrate types
BLOCK_HEIGHT
0
No
Block height to start indexing. Ignored if the database already contains indexed blocks

Manual setup

Run
1
yarn && yarn build
Copied!
For setting up the database and running the migrations, run yarn db:bootstrap. Make sure the environment variables DB_* are set.
For starting the indexer, run yarn start:prod.

Docker image

There are pre-built runnable docker images in joystream/hydra-indexer docker repo. The default command is yarn start:prod.
First, bootstrap the database:
1
docker run -e DB_HOST=... -e DB_PORT=... -e DB_NAME=... -e DB_PASS=... -e DB_USER=... joystream/hydra-indexer sh -c 'yarn db:bootstrap'
Copied!
Then run the indexer (make sure that all the required environment variables are set)
1
docker run -e ... joystream/hydra-indexer
Copied!

Advanced environment variables

Some optional environment variables are available for fine-tuning.
Variable
Default
Description
BLOCK_CACHE_TTL_SEC
60*60
TTL for processed blocks in the Redis cache
INDEXER_HEAD_TTL_SEC
60*15
TTL for the indexer head block entry
WORKERS_NUMBER
5
Number of concurrent workers fetching the blocks
BLOCK_PRODUCER_FETCH_RETRIES
3
Number of attempts fetching each a block before throwing an error. Set to -1 for indefinite attempts
SUBSTRATE_API_TIMEOUT
1000 * 60 * 5
Timeout in (milliseconds) for API calls
NEW_BLOCK_TIMEOUT_MS
60 * 10 * 1000
Panic if no blockchain blocks have been received within this time
Last modified 5mo ago