hydra
Search…
Quickstart
Build a Hydra Indexer and GraphQL server from scratch under five minutes
Before starting, make surehydra-cliis installed on your machine together with all the prerequisites. Use @dzlzv/[email protected] version to use the latest features.

0. Hello Hydra!

Start off by setting up a project folder
1
mkdir hello-hydra && cd hello-hydra
Copied!

1. From zero to one

Run the scaffold command, which generates all the required files in a new folder hydra-sample
1
hydra-cli scaffold -d hydra-sample
Copied!
Answer the prompts and the scaffolder will generate a sample backbone for our Hydra project. This includes:
    Sample GraphQL data schema in schema.graphql describing proposals in the Kusama network
    Sample mapping scripts in the ./mapping folder translating substrate events into the Proposal entity CRUD operations
    docker folder with scripts for running a Hydra Indexer and Hydra Processor locally
    .env with all the necessary environment variables. It is pre-populated with the prompt answers but can be edited at any time.
    package.json with a few utility yarn scripts to be used later on.

2. Codegen

Make sure a Postgres database is up and running in the background and is accessible with the credentials provided during the scaffolding. Run
1
yarn && yarn bootstrap
Copied!
It will generate the model files as defined in schema.graphql, create the database schema and run all the necessary migrations.
NB! Use with caution in production, as it will delete all the existing records in the processor database.
Under the fold, yarn booststrap creates generated/graphql-server with a ready-to-use Apollo GraphQL server powering the query node API.

3. Typegen for events and extrinsics

Now let's inspect manifest.yml which defines which events and extrinsics are going to be processed by Hydra Processor. Two most important sections are typegen and mappings
hydra-typegen is an auxiliary tool for generating typesafe event and extrinsic classes from the on-chain metadata. It is not strictly necessary to use it, but type safety significantly simplifies the development of the event and extrinsic handlers.
The typegen section of the manifest lists the events and extrinsics for which typescript classes will be generated together with the metadata source and the output directory.
1
typegen:
2
typegen:
3
metadata:
4
source: wss://rpc.polkadot.io
5
blockHash: '0xab5c9230a7dde8bb90a6728ba4a0165423294dac14336b1443f865b796ff682c'
6
events:
7
- balances.Transfer
8
calls:
9
- timestamp.set
10
outDir: ./mappings/generated/types
Copied!
Typegen fetches the metadata from the chain from the block with a given hash (or from the top block if no hash is provided). For chains with non-standard types one should additionally provide custom type definitions, as below:
1
typegen:
2
metadata:
3
source: ws://arch.subsocial.network:9944
4
# add hash of the block if the metadata from a specific block
5
# should be used by typegen
6
# blockHash: 0x....
7
events:
8
- posts.PostCreated
9
calls:
10
- posts.CreatePost
11
customTypes:
12
lib: '@subsocial/types/substrate/interfaces'
13
typedefsLoc: typedefs.json
14
outDir: ./mappings/generated/types
Copied!
Run
1
yarn typegen
Copied!
and inspect mappings/generated/types where the newly created classes for the declared events and extrinsics will be generated.

4. Mappings

Mapping are defined in the mappings section of the manifest file and reside in the mappings folder.
1
mappings:
2
# the transpiled js module with the mappings
3
mappingsModule: mappings/lib/mappings
4
imports:
5
# generated types to be loaded by the processor
6
- mappings/lib/mappings/generated/types
7
eventHandlers:
8
# event to handle
9
- event: posts.PostCreated
10
# handler function with argument types
11
handler: postCreated
12
extrinsicHandlers:
13
# extrinsic to handle
14
- extrinsic: timestamp.set
15
handler: timestampCall
Copied!
Run
1
yarn mappings:build
Copied!
to build the mappings into a js module. Make sure the mappings are rebuilt after each change.

5. Run Hydra Indexer locally

Hydra's two-tier architecture separates data ingesting and indexing (done by Hydra Indexer) and processing (done by Hydra Processor, of course). Hydra Indexer + API gateway is a set-and-forget service which requires maintainance only when there is a major runtime upgrade. The scaffolder conveniently creatres a stub for running the indexer stack with docker-compose, as defined in docker-compose-indexer.yml
The WS_PROVIDER_ENDPOINT_URIenvironment variable defines the node to connect. Additionally, one can map volumes as json files with runtime type definitions. The following environment variables
    TYPES_JSON
    SPEC_TYPES
    CHAIN_TYPES
    BUNDLE_TYPES
can be used to inject custom types and type overrides for spec, chain and bundle definitions. For more info, consult polkadot.js docs
Let's run a local indexer against a Polkadot chain. Since all Polkadot type definitions are already included in polkadot.js library, there is no need to add type definition and the only change is to set WS_PROVIDER_ENDPOINT_URI=wss://rpc.polkadot.io together with the database variables and run
1
docker-compose -f docker-compose-indexer.yml up -d
Copied!
Check the status of the indexer by navigating to the indexer playground at localhost:4001/graphql and querying
1
query {
2
indexerStatus {
3
chainHeight # current chain height
4
head # last indexed block
5
inSync # if the processor is fully in sync
6
hydraVersion # processor version
7
}
8
}
Copied!
Make sure the major hydraVersion matches the one of hydra-cli and declared in manifest.yml

6. Run Hydra Processor locally

Hydra Processor connects to a Hydra Indexer gateway for sourcing the indexed block, event and extrinsic data for processing.
Set INDEXER_ENDPOINT_URL in .env to the local indexer http://localhost:4001/graphql and run
1
yarn processor:run
Copied!

7. Run Query Node API

Run
1
yarn query-node:start:dev
Copied!
The query node API is now available at http://localhost:4000/graphql and you can find some transfers:
1
query {
2
transfers(orderBy:block_ASC) {
3
from
4
to
5
block
6
}
7
}
Copied!

7. Dockerize & deploy

Among other things, the scaffolder generates a docker folder with Dockerfiles.
First, build the builder image:
1
$ docker build . -f docker/Dockerfile.builder -t builder
Copied!
Now the images for the GraphQL query node and the processor can be built (they use the builder image under the hood)
1
$ docker build . -f docker/Dockerfile.query-node -t query-node:latest
2
$ docker build . -f docker/Dockerfile.processor -t processor:latest
Copied!
In order to run the docker-compose stack, we need to create the schema and run the database migrations.
1
$ docker-compose up -d db
2
$ yarn docker:db:migrate
Copied!
The last command runs yarn db:bootstrap in the builder image. A similar setup strategy may be used for Kubernetes (with builderas a starter container).
Now everything is ready:
1
$ docker-compose up
Copied!

What to do next?

    Explore more examples
    Describe your own schema in schema.graphql
    Write your indexer mappings
    Push your Hydra indexer and GraphQL Docker images to Docker Hub and deploy
Last modified 4mo ago