hydra
Search…
Hydra Indexer
Hydra Indexer is a daemon that ingests raw substrate data such as events and extrinsics from a substrate chain and saves it into a Postgres database. It is typically used in conjunction with Hydra Indexer Gateway providing a GraphQL API interface.

Installation

The indexer exits if an unrecoverable error occurs. It is therefore advisable to run the indexer with a process manager, such as PM2 or Docker.

Prerequisites

  • Node 12.x
  • Postgres database
  • Redis instances
  • (Optional) Docker

Environment variables

The indexer is set up using the following environment variables
Variable
Default
Required
Description
WS_PROVIDER_ENDPOINT_URI
-
Yes
Substrate API endpoint to connect
REDIS_URI
-
Yes
Redis instance URI
DB_NAME
-
Yes
Database name
DB_PORT
-
Yes
Database port
DB_HOST
-
Yes
Database host
DB_USER
-
Yes
Database user
DB_PASS
-
Yes
Database password
TYPES_JSON
-
No
Path to a JSON type definition with custom Substrate types
BLOCK_HEIGHT
0
No
Block height to start indexing. Ignored if the database already contains indexed blocks

Manual setup

Run
1
yarn && yarn build
Copied!
For setting up the database and running the migrations, run yarn db:bootstrap. Make sure the environment variables DB_* are set.
For starting the indexer, run yarn start:prod.

Docker image

There are pre-built runnable docker images in joystream/hydra-indexer docker repo. The default command is yarn start:prod.
First, bootstrap the database:
1
docker run -e DB_HOST=... -e DB_PORT=... -e DB_NAME=... -e DB_PASS=... -e DB_USER=... joystream/hydra-indexer sh -c 'yarn db:bootstrap'
Copied!
Then run the indexer (make sure that all the required environment variables are set)
1
docker run -e ... joystream/hydra-indexer
Copied!

Advanced environment variables

Some optional environment variables are available for fine-tuning.
Variable
Default
Description
BLOCK_CACHE_TTL_SEC
60*60
TTL for processed blocks in the Redis cache
INDEXER_HEAD_TTL_SEC
60*15
TTL for the indexer head block entry
WORKERS_NUMBER
5
Number of concurrent workers fetching the blocks
BLOCK_PRODUCER_FETCH_RETRIES
3
Number of attempts fetching each a block before throwing an error. Set to -1 for indefinite attempts
SUBSTRATE_API_TIMEOUT
1000 * 60 * 5
Timeout in (milliseconds) for API calls
NEW_BLOCK_TIMEOUT_MS
60 * 10 * 1000
Panic if no blockchain blocks have been received within this time
Last modified 8mo ago