Testing

We provide a comprehensive testing suite for the whole Tycho stack. The suite facilitates end-to-end testing and ensures your protocol integration behaves as expected. For unit tests, please use standard Rust unit testing practices.

Find the suite in /protocol_testing.

What does the suite test?

There are two test modes:

  • range — indexes and validates test cases defined in integration_test.tycho.yaml for specific block ranges.

  • full — indexes and validates the entire protocol history from creation to the latest block, without comparing specific component information.

Here's what the testing suite does:

  1. Runs Tycho Indexer with your Substreams implementation for a specific block range. If running on the range test mode, it also verifies that the components' state matches the expected states specified by the testing YAML file. This confirms that your Substreams package is indexable and that it outputs what you expect.

  2. Retrieves swap quotes using Tycho Simulation. This verifies that all necessary data for simulation is indexed and, for VM implementations, that the provided SwapAdapter contract works. It is important to know that the simulation engine runs entirely off-chain and only accesses the data and contracts you index (token contracts are mocked and don't need to be indexed)

  3. Encodes and simulates transactions using Tycho Execution against an RPC on an historical block. This ensures that your protocol swaps can be executed on chain and that the indexed data and quotes match onchain state and logic.

How to run

Prerequisites

Archive node

You need an EVM Archive node to fetch the state from a previous block (you can use Alchemy for example). If you index only with Substreams, as in Tycho's production mode, you must sync blocks since the protocol's deployment date, which can take a long time. The archive node skips this requirement by fetching all the required account's storage slots on the block you specify in the testing yaml file.

The node also needs to support the debug_storageRangeAt method, which is required for our Token Quality Analysis.

Test Configuration

Range Mode

Use range mode to test specific block intervals and verify that your Substreams implementation indexes and outputs the expected component states.

You'll need an integration_test.tycho.yaml file inside your Substreams directory. The configuration file should include:

  • The target Substreams config file;

  • The corresponding SwapAdapter and args to build it (if it's a vm integration);

  • The protocol system;

  • The expected protocol types;

  • The tests to be run.

Each test will:

  1. index all blocks between start-block and stop-block;

  2. verify that the indexed state matches the expected state (creation of the expected components);

  3. simulate get amount out (if vm implementation, it uses the provided SwapAdapter);

  4. encode a single swap and simulate its execution;

  5. ensure that the amount out value from the execution matches the one from simulation.

Full Mode

Use full mode to validate the entire protocol lifecycle — from creation to the latest indexed block.

It will:

  1. index all blocks between initial-block and the current block;

  2. simulate get amount out in the current block (if vm implementation, it uses the provided SwapAdapter);

  3. encode a single swap and simulate its execution in the current block;

  4. ensure that the amount out value from the execution matches the one from simulation.

Test Parameters

Here are the test parameters that you need to set:

1. initialized_accounts

This is a list of contract addresses that simulation requires, although their creation is not indexed within the test block range. Leave empty if not required.

Importantly, you use this config during testing only. Your Substreams package should still properly initialise the accounts listed here. This configuration only eliminates the need to include historical blocks that contain the initialisation events in your test data. This ensures tests are targeted and quick to run.

You can use the initialized_accounts config at two levels in the test configuration file:

  • global: accounts listed here are used for all tests in this suite;

  • test level: accounts listed here are scoped to that test only.

2. expected_components (for range mode)

This is a list of components whose creation you are testing. It includes all component data (tokens, static attributes, etc.). You don't need to include all components created within your test block range – only those on which the test should focus.

3. skip_balance_check

By default, this should be false. Testing verifies the balances reported for the component by comparing them to the on-chain balances of the Component.id . This should be false if:

  1. the Component.id does not correlate to a contract address;

  2. balances are not stored on the component's contract (i.e. they're stored on a vault).

If this skip is set to true, you must comment on why.

4. skip_simulation

By default this should be false . It should only be true temporarily if you want to isolate testing the indexing phase only. If set to true, you must comment on why.

5. skip_execution

By default this should be false . It should only be true temporarily if you want to isolate testing the indexing and simulation phases only. If set to true, you must comment on why.

Code changes

If the protocol you are integrating is not a vm integration, to be able to test simulation, you need to register it in register_decoder_for_protocol (here). This is to match your protocol system name with the State that is used in Tycho Simulation.

Running Tests

We offer two approaches for running tests: local run and Docker run.

Local run works best when you're actively developing your integration. You can test individual phases (indexing, simulation, execution) in isolation and get faster iteration cycles for debugging. However, you'll need to handle additional setup and prerequisites yourself.

Docker run suits CI environments and final validation. You run the complete end-to-end test suite in an encapsulated environment, which eliminates the setup complexity you'd face otherwise. The actual test execution is fast once you have the images built, but every time you change something in your package, you'll need to rebuild the images—and that's the slow part. This approach makes most sense once your package is stable.

Here is how you can run the tests with each approach:

Prerequisites:

Before continuing, ensure the following tools and libraries are installed on your system:

  • Docker: Containerization platform for running applications in isolated environments.

  • Git: Version control tool

  • Rust: Programming language and toolchain

  • GCC: GNU Compiler Collection

  • libpq: PostgreSQL client library

  • OpenSSL (libssl): OpenSSL development library

  • pkg-config: Helper tool for managing compiler flags

  • Substreams CLI: Indexing tool that uses Rust modules to process blockchain data.

  • Tycho Indexer: The testing module runs a minified version of Tycho Indexer. You need to ensure that the latest version is correctly setup in your PATH and if it isn't you need to (re)install Tycho. Run the following command on your terminal to check the version: 

    > tycho-indexer --version
tycho-indexer 0.88.0 # should match the latest version published on GitHub

Step 1: Export Environment Variables

  • RPC_URL: The URL for the Ethereum RPC endpoint. This fetches the storage data.

  • SUBSTREAMS_API_TOKEN: The JWT token for accessing Substreams services. This token is necessary for authentication. Please refer to the Substreams Authentication guide to set up and validate your token.

  • RUST_LOG to define the log level you want to see. For enhanced debugging, we recommend running the testing module with Tycho indexer logs.

export RPC_URL="https://ethereum-mainnet.core.chainstack.com/123123123123"
export SUBSTREAMS_API_TOKEN=eyJhbGci...
export RUST_LOG=protocol_testing=info,tycho_client=error,tycho_indexer=error

Step 2: Build the substreams wasm file

If you do not have one already, you must build the wasm file of the substreams package you wish to test. This can be done by navigating to the substreams package directory and running:

cargo build --target wasm32-unknown-unknown --release

Step 3: Run a local Postgres test database using docker-compose.

In /protocol-testing , run:

docker compose -f ./docker-compose.yaml up -d db

Step 4: Run tests

In /protocol-testing , run:

cargo run -- range/full --package <package-name>

Select range or full depending on your test mode.

These are the optional arguments:

Options:
# Range tests
--package <PACKAGE>         # Name of the package to test
--match-test <MATCH_TEST>   # Run only tests matching name
--db-url <DB_URL>           # Database URL (default: postgres://postgres:mypassword@localhost:5431/tycho_indexer_0)
--rpc-url                   # RPC endpoint with trace support (required)

# Full tests
--initial-block    # Start block (default: protocol creation block)
--stop-block       # End block (default: latest)

# Debugging
--vm-simulation-traces    # Enable VM simulation traces
--execution-traces        # Enable execution traces

Complete example

If you want to run the range tests for ethereum-balancer-v2, use the following:

# Setup Environment Variables
export RPC_URL="https://ethereum-mainnet.core.chainstack.com/123123123123"
export SUBSTREAMS_API_TOKEN=eyJhbGci...
export RUST_LOG=info,protocol_testing=info,tycho_client=error

# Build Substreams wasm for BalancerV2
cd substreams
cargo build --release --package ethereum-balancer-v2 --target wasm32-unknown-unknown
cd ../protocol-testing

# Run Postgres DB using Docker compose
docker compose -f ./docker-compose.yaml up -d db

# Run test
cargo run -- range range --package ethereum-balancer-v2

Installing or updating the Tycho Indexer version (Optional)

Troubleshooting

Slow tests

An integration test should take a maximum 5–10 minutes. If the tests take longer, here are key things you can explore:

  1. Ensure you have no infinite loops within your code.

  2. Ensure you're using a small block range for your test, ideally below 1,000 blocks. The blocks in your test only need to cover the creation of the component you are testing. Optionally, they can extend to blocks with changes for the component you want the test to cover. To help limit the test block range, you could explore the initialized_accounts config.

  3. Ensure you are not indexing tokens. Token contracts use a lot of storage, so fetching their historical data is slow. Instead, they are mocked on the simulation engine and don't have to be explicitly indexed. Make an exception if they have unique behavior, like acting as both a token and a pool, or if they are rebasing tokens that provide a getRatemethod.

Note: Substreams uses cache to improve the speed of subsequent runs of the same module. A test's first run is always slower than subsequent runs, unless you change the Substreams module's code.

Account not initialised

There are two main causes for this error:

  1. Your Substreams package is not indexing a contract that is necessary for simulations.

  2. Your test begins at a block that is later than the block on which the contract was created. To fix this, add the missing contract to the initialized_accounts test config.

Dev Cluster Tests

After your protocol is moved to our dev environment, it will be subject to constant indexing, simulation, and execution testing via a constant running pod in our cluster. If we find problems in any of these areas, we may reach out to you for help debugging.

PreviousCode ArchitectureNextRequest for Quote Protocols

Last updated