Self-Hosted EVM Chain

Run the Tycho Indexer on an EVM chain with no hosted Substreams endpoint

Overview

Tycho indexes a chain by streaming blocks from a Substreams endpoint. For most chains you point the indexer at a hosted StreamingFast or Pinax endpoint and never run any Firehose infrastructure yourself — see Hosted Endpoints.

Some EVM chains have no hosted Substreams endpoint. To index one of those, run your own Firehose + Substreams stack next to the indexer. The substreams-endpoint Docker Compose profile does exactly that: it spins up a single-container Firehose that polls blocks from an EVM JSON-RPC node and serves them over the same gRPC interface the indexer expects.

Choose self-hosted when no hosted Substreams endpoint exists for your chain. If a hosted endpoint exists, prefer it — it needs no extra infrastructure.

The RPC poller produces base blocks (no contract storage — VM protocols and DCI do not work). Read Limitations first.

Limitations

Base block model — VM protocols and DCI do not work

The poller reads blocks over plain JSON-RPC, which exposes logs and receipts but not storage writes or internal calls. It therefore produces base blocks — the protobuf fields for storage and call data arrive empty. Hosted endpoints backed by instrumented nodes serve extended blocks that carry all of it.

To overcome it, run your own instrumented node: a Firehose-patched build of the chain's execution client (available for geth, op-geth, BNB, Polygon, Arbitrum) under fireeth start reader-node, in place of the poller — see the Firehose EVM node setup guide.

Single machine — meant for dev/test usage

This limitation comes from the compose profile, not Firehose: it starts every Firehose component in one container against one RPC endpoint — no high availability, no failover. For production, replace the profile with a distributed topology following the Firehose deployment and scaling guide.

Prerequisites

• Docker and Docker Compose.

• The tycho-indexer image, built locally. From the repo root, build it and tag it to match TYCHO_IMAGE:

  docker build -f docker/tycho-indexer.Dockerfile -t tycho-indexer:local .

• An EVM JSON-RPC endpoint for your chain. The poller fetches every block from it, so prefer a low-latency node — RPC latency directly caps the fetch rate.

• A compiled Substreams package (.spkg) for each protocol you want to index, built for your chain. A chain Tycho has never indexed has no ready-made package, so you build one — see Building a Substreams package for your chain. Place each .spkg under docker/substreams/; the compose file mounts that directory into the indexer at /opt/tycho-indexer/substreams/.

Configuration

Configure the stack through docker/.env. The compose file reads it for both the indexer and the Firehose service.

Environment variable reference

Variable	Service	Required	Default	Purpose
RPC_URL	poller + indexer	Yes	—	EVM JSON-RPC endpoint. The poller fetches blocks from it; the indexer reads token metadata from it.
SUBSTREAMS_ENDPOINT	indexer	Yes	https://mainnet.eth.streamingfast.io:443	Substreams tier1 gRPC. Self-hosted: http://substreams-endpoint:10016.
TYCHO_IMAGE	indexer	Yes	—	tycho-indexer image tag.
START_BLOCK	poller	No	0	First block the poller fetches.
CHAIN_NAME	poller	No	mainnet	Chain name the Firehose advertises (--advertise-chain-name).
CHAINS	indexer	No	ethereum	Active chain to index. The indexer uses only the first value (multichain is not yet supported). Name a built-in chain, or a custom chain you declare in chains.yaml (see below).
RETENTION_HORIZON	indexer	No	2000-01-01T00:00:00	Earliest version history the indexer keeps. Use a future date to keep no historical state (recommended) — see the note below. The 2000-01-01 default keeps all history and fails a from-scratch historical backfill.
EXTRACTORS_CONFIG	indexer	No	/opt/tycho-indexer/extractors.yaml	Path to the extractors config inside the container.
TYCHO_CHAINS_CONFIG	indexer + consumers	No	/opt/tycho-indexer/chains.yaml	Path to the custom-chains config. The indexer and every consumer (tycho-simulation, tycho-client, tycho-execution) read the same variable. Only needed for a non-built-in chain.
SUBSTREAMS_API_TOKEN	indexer	No	readme	Auth token for a hosted Substreams endpoint; unused self-hosted.
TRACE_RPC_URL	indexer	For DCI	readme (placeholder)	Trace-capable RPC for dynamic contract indexing.
OTLP_EXPORTER_ENDPOINT	indexer	No	empty (disabled)	OpenTelemetry collector. Set http://lgtm:4317 with the observability profile.
AUTH_API_KEY	indexer	No	local-dev-key	Tycho RPC API key.
RUST_LOG	indexer	No	info	Log level.

A self-hosted docker/.env for the Tempo chain looks like this:

TYCHO_IMAGE=tycho-indexer:local

RPC_URL=https://rpc.tempo.xyz
TRACE_RPC_URL=https://rpc.tempo.xyz
CHAIN_NAME=tempo
START_BLOCK=6455886

# Self-hosted Firehose, reachable by its docker service name
SUBSTREAMS_ENDPOINT=http://substreams-endpoint:10016

# Custom chain — defined in chains.yaml (see below)
CHAINS=tempo

# Keep no historical state (recommended); the 2000-01-01 default keeps all history and breaks a from-scratch backfill (see "Retention horizon")
RETENTION_HORIZON=2100-01-01T00:00:00
SUBSTREAMS_API_TOKEN=local

Writing your extractors.yaml

Edit crates/tycho-indexer/extractors.yaml — the compose file mounts it into the container at /opt/tycho-indexer/extractors.yaml. Each entry under extractors: configures one protocol:

The indexer builds every entry in this file at startup and fails if any referenced .spkg is missing. The shipped file lists Ethereum protocols whose packages are not in docker/substreams/. Replace them with only the extractors for the chain and protocols you are indexing, each pointing at a .spkg you have placed under docker/substreams/.

extractors:
  uniswap_v3:
    name: "uniswap_v3"
    chain: "tempo"
    implementation_type: "Custom"
    sync_batch_size: 1000
    start_block: 6455886
    protocol_types:
      - name: "uniswap_v3_pool"
        financial_type: "Swap"
    spkg: "substreams/tempo-uniswap-v3-v0.1.0.spkg"
    module_name: "map_protocol_changes"

Field	Purpose
name	Unique extractor name; also the protocol system name exposed over the RPC.
chain	Chain this extractor runs on — a built-in chain name, or a custom chain defined in chains.yaml (see below). The indexer rejects an unknown chain name at startup.
implementation_type	Custom for natively integrated protocols — the Substreams emits protocol attributes and the pool maths are implemented natively in tycho-simulation. Vm for protocols whose full contract state is indexed and whose logic runs in a local VM inside tycho-simulation. The indexer stores this as metadata; downstream consumers (e.g. tycho-simulation) act on it.
sync_batch_size	How many blocks the indexer buffers before flushing them to Postgres in a single write, during the initial catch-up sync. Substreams delivers messages as a continuous stream — this only tunes DB write batching while syncing.
start_block	Block at which the protocol was deployed; the indexer starts streaming here.
spkg	Path to the compiled .spkg, relative to /opt/tycho-indexer/ (i.e. under docker/substreams/).
module_name	Substreams output module to consume (e.g. map_protocol_changes).
protocol_types	Protocol component types this extractor produces, each with a name and a financial_type (Swap, Psm, Debt, or Leverage).

Declaring a custom chain

Built-in chains (ethereum, base, unichain, …) need no extra config. To index a chain Tycho does not know, define it in a separate chains.yaml file. Copy crates/tycho-indexer/chains.example.yaml to crates/tycho-indexer/chains.yaml and edit it — the compose file mounts it into the container at /opt/tycho-indexer/chains.yaml, and the indexer reads it via TYCHO_CHAINS_CONFIG (the --chain-config flag). Each extractor's chain: field and CHAINS resolve against these entries; the indexer fails fast at startup if an extractor references a chain that is neither built-in nor defined here.

chains:
  - name: tempo                       # the value used as `chain:` on extractors and in CHAINS
    chain_id: 12345                   # EVM chain id
    block_time_secs: 1
    native:                           # native gas token
      address: "0x0000000000000000000000000000000000000000"
      symbol: "ETH"
      decimals: 18
    wrapped_native:                   # wrapped native token (e.g. WETH)
      address: "0x0000000000000000000000000000000000000000"
      symbol: "WETH"
      decimals: 18
    default_tvl_thresholds:           # TVL gates (in native token units) for component tracking
      low: 1000
      medium: 10000

Field	Purpose
name	Chain identifier; reference it from an extractor's chain: field and from CHAINS.
chain_id	EVM chain id.
block_time_secs	Average block time in seconds.
native	Native gas token: address, symbol, decimals.
wrapped_native	Wrapped native token (e.g. WETH): address, symbol, decimals.
default_tvl_thresholds	Liquidity gates in native-token units (e.g. ETH), read by downstream consumers (solvers, tycho-simulation) to decide which components to track — not by the indexer. Size low/medium to the USD floor you want at the native token's price; the Ethereum defaults target roughly $20k / $200k.

Building a Substreams package

Tycho ships the Substreams module sources in protocols/substreams/. Each directory holds one WASM module plus a manifest per chain and fork that reuses it — for example ethereum-uniswap-v2/ also carries the PancakeSwap manifests and the Arbitrum, BSC, Base, and Unichain variants. The WASM logic is chain-agnostic; each manifest pins the chain-specific factory address and start block. A chain Tycho has never indexed has no prebuilt .spkg, so you build one — from a logs-only module such as ethereum-uniswap-v2 or ethereum-uniswap-v3-logs-only (see Limitations).

To index a Uniswap-V2-style DEX on a new chain, reuse the ethereum-uniswap-v2 module and add a manifest for your chain:

1. Copy an existing manifest, e.g. protocols/substreams/ethereum-uniswap-v2/ethereum-uniswap-v2.yaml, to <chain>-<dex>.yaml in the same directory.

2. Edit the copy: set the package name/version, set every module's initialBlock to the DEX factory's deployment block, and set the params line to factory_address=<your factory>&protocol_type_name=<your pool type>.

3. Build the WASM and pack the package (some protocols build with --profile substreams instead of --release — check that protocol's Makefile or README):

   cd protocols/substreams/ethereum-uniswap-v2
   cargo build --target wasm32-unknown-unknown --release
   substreams pack <chain>-<dex>.yaml

4. Copy the resulting .spkg into docker/substreams/, then point your extractor's spkg: field at it.

substreams pack may warn that network is not set. This is harmless for the self-hosted stack — the Firehose advertises the chain through CHAIN_NAME, not the package.

Running the stack

Start everything — Postgres, the self-hosted Firehose, and the indexer — with the substreams-endpoint profile:

cd docker
docker compose --profile substreams-endpoint up

On a cold start the poller begins at START_BLOCK and streams forward. The indexer only commits a block once it sits behind the finality horizon, so expect a delay before committed state appears — on a fresh chain the first cold start takes a while to reach the deployment block of your protocols.

Retention horizon

Set RETENTION_HORIZON to a future date (the example uses 2100-01-01T00:00:00) so the indexer keeps no historical state — only the current state of every component. This is the simplest setup and sidesteps a partition crash during backfill.

The 2000-01-01T00:00:00 default keeps the full version history. When you then backfill a chain from a deployment block more than about a month old, the indexer crashes on the first historical state update it writes:

duplicate key value violates unique constraint "component_balance_default_unique_pk"

pg_partman partitions the component_balance, protocol_state, and contract_storage tables by day on valid_to, keeping a one-month retention window. A superseded historical version carries a valid_to older than any existing partition, so it lands in the default partition and violates its uniqueness constraint. A future RETENTION_HORIZON drops those old versions before they are written. Use a recent date instead only if you genuinely need a short window of version history.

Resuming after a restart

Both halves of the stack resume on their own:

• The poller auto-detects the highest stored one-block file under /data/storage/one-blocks/ and resumes from there, so it skips already-fetched blocks.

• The indexer resumes from the cursor stored in its Postgres database.

docker compose --profile substreams-endpoint down followed by up without -v preserves both the Firehose data volume and the database, so the stack picks up where it left off. Passing -v deletes the volumes and forces a full cold start.

When you switch the chain you index (change CHAIN_NAME, START_BLOCK, or CHAINS), bring the stack down with -v first:

docker compose --profile substreams-endpoint down -v

The Firehose data volume is chain-specific. Reusing it for a different chain makes the Firehose advertise a first-streamable block from the previous chain, and the indexer fails with initial block N smaller than first streamable block M.

Connecting to a hosted endpoint instead

To use a hosted Substreams endpoint rather than the self-hosted Firehose, omit the profile and set the endpoint and token:

cd docker
# in .env:
#   SUBSTREAMS_ENDPOINT=https://<hosted-endpoint>:443
#   SUBSTREAMS_API_TOKEN=<your-token>
docker compose up

Without --profile substreams-endpoint, the substreams-endpoint service never starts and the indexer streams straight from the hosted endpoint.

Consuming the custom chain

The indexer serves your custom chain over RPC and WebSocket, but a consumer (tycho-simulation, tycho-client, tycho-execution) resolves a chain name against its own copy of the chain config. Point it at the same chains.yaml through the TYCHO_CHAINS_CONFIG environment variable — the same variable the indexer uses:

export TYCHO_CHAINS_CONFIG=crates/tycho-indexer/chains.yaml

With the variable set, a chain name like tempo resolves to its full config on first use. Leave it unset and the consumer resolves only built-in chains and rejects the custom name. When the variable points at a missing or malformed file, the stream builder rejects it at startup and returns a set-up error naming the config — the consumer never starts against a half-configured chain.

Monitoring sync progress

Follow the logs to watch the stack catch up:

docker compose --profile substreams-endpoint logs -f

Signals to watch, in the order blocks flow through the stack:

• Poller — block-fetch rate. A healthy poller logs a steady stream of fetched blocks; a stalled or slow rate points at the RPC.

• Merger — bundles one-block files into merged-block segments.

• Substreams tier2 — produces the state segments tier1 serves to the indexer.

• tycho-indexer — the committed block height climbs as finalized blocks reach the database.

Both services expose TCP healthchecks you can probe:

Endpoint	Port	Service
Firehose / Substreams tier1 gRPC	10016	substreams-endpoint
Tycho RPC	4242	tycho-indexer

For dashboards, logs, and traces, enable the observability profile alongside substreams-endpoint and set OTLP_EXPORTER_ENDPOINT=http://lgtm:4317. See Observability.

Troubleshooting

• Cold start takes a long time. START_BLOCK is the Firehose poller's start block (where it begins fetching from the RPC), separate from each extractor's start_block. A fresh chain must fetch and merge every block from START_BLOCK before your protocols' deployment blocks appear. Set START_BLOCK at or just below your earliest protocol start_block so the poller skips irrelevant history.

Performance tuning

• --interval-between-fetch (poller) — delay between RPC fetches. The compose file sets 0ms (no delay) for maximum throughput. Raise it to throttle a rate-limited RPC.

• RPC latency — the poller fetches blocks sequentially, so round-trip latency directly bounds the fetch rate. A nearby, low-latency node is the single biggest lever on cold-start speed.

• Chain block time vs fetch rate — once caught up, the poller only needs to keep pace with the chain's block time. Cold start is fetch-bound; steady state is block-time-bound.

• --substreams-state-bundle-size (tier1/tier2) — number of blocks per state segment, 1000 by default. Larger bundles cut per-segment overhead at the cost of coarser caching granularity.