Testing

We provide a comprehensive testing suite for the whole Tycho stack. The suite facilitates end-to-end testing and ensures your protocol integration behaves as expected. For unit tests, please use standard Rust unit testing practices.

Find the suite in /protocol_testing.

What does the suite test?

Here's what the testing suite does:

  1. Runs Tycho Indexer with your Substreams implementation for a specific block range. It verifies that the components' state matches the expected states specified by the testing YAML file. This confirms that your Substreams package is indexable and that it outputs what you expect.

  2. Retrieves swap quotes using Tycho Simulation. This verifies that all necessary data for simulation is indexed and, for VM implementations, that the provided SwapAdapter contract works. It is important to know that the simulation engine runs entirely off-chain and only accesses the data and contracts you index (token contracts are mocked and don't need to be indexed)

  3. Encodes and simulates transactions using Tycho Execution against an RPC on an historical block. This ensures that your protocol swaps can be executed on chain and that the indexed data and quotes match onchain state and logic.

How to run

Prerequisites

Archive node

You need an EVM Archive node to fetch the state from a previous block (you can use Alchemy for example). If you index only with Substreams, as in Tycho's production mode, you must sync blocks since the protocol's deployment date, which can take a long time. The archive node skips this requirement by fetching all the required account's storage slots on the block you specify in the testing yaml file.

The node also needs to support the debug_storageRangeAt method, which is required for our Token Quality Analysis.

Test Configuration

You'll need an integration_test.tycho.yaml file inside your Substreams directory. The configuration file should include:

  • The target Substreams config file;

  • The corresponding SwapAdapter and args to build it (if it's a vm integration);

  • The protocol system;

  • The expected protocol types;

  • The tests to be run.

Each test will:

  1. index all blocks between start-block and stop-block;

  2. verify that the indexed state matches the expected state (creation of the expected components);

  3. simulate transactions (if vm implementation, it uses the provided SwapAdapter);

  4. encode a single swap and simulate its execution;

  5. ensure that the amount out value from the execution match the one from simulation.

Test Parameters

Here are the test parameters that you need to set:

1. initialized_accounts

This is a list of contract addresses that simulation requires, although their creation is not indexed within the test block range. Leave empty if not required.

Importantly, you use this config during testing only. Your Substreams package should still properly initialise the accounts listed here. This configuration only eliminates the need to include historical blocks that contain the initialisation events in your test data. This ensures tests are targeted and quick to run.

You can use the initialized_accounts config at two levels in the test configuration file:

  • global: accounts listed here are used for all tests in this suite;

  • test level: accounts listed here are scoped to that test only.

2. expected_components

This is a list of components whose creation you are testing. It includes all component data (tokens, static attributes, etc.). You don't need to include all components created within your test block range – only those on which the test should focus.

3. skip_balance_check

By default, this should be false. Testing verifies the balances reported for the component by comparing them to the on-chain balances of the Component.id . This should be false if:

  1. the Component.id does not correlate to a contract address;

  2. balances are not stored on the component's contract (i.e. they're stored on a vault).

If this skip is set to true, you must comment on why.

4. skip_simulation

By default this should be false . It should only be true temporarily if you want to isolate testing the indexing phase only. If set to true, you must comment on why.

5. skip_execution

By default this should be false . It should only be true temporarily if you want to isolate testing the indexing and simulation phases only. If set to true, you must comment on why.

Code changes

If the protocol you are integrating is not a vm integration, to be able to test simulation, you need to register it in register_decoder_for_protocol (here). This is to match your protocol system name with the State that is used in Tycho Simulation.

Running Tests

We offer two approaches for running tests: local run and Docker run.

Local run works best when you're actively developing your integration. You can test individual phases (indexing, simulation, execution) in isolation and get faster iteration cycles for debugging. However, you'll need to handle additional setup and prerequisites yourself.

Docker run suits CI environments and final validation. You run the complete end-to-end test suite in an encapsulated environment, which eliminates the setup complexity you'd face otherwise. The actual test execution is fast once you have the images built, but every time you change something in your package, you'll need to rebuild the images—and that's the slow part. This approach makes most sense once your package is stable.

Here is how you can run the tests with each approach:

Prerequisites:

Before continuing, ensure the following tools and libraries are installed on your system:

  • Docker: Containerization platform for running applications in isolated environments.

  • Git: Version control tool

  • Rust: Programming language and toolchain

  • GCC: GNU Compiler Collection

  • libpq: PostgreSQL client library

  • OpenSSL (libssl): OpenSSL development library

  • pkg-config: Helper tool for managing compiler flags

  • Substreams CLI: Indexing tool that uses Rust modules to process blockchain data.

  • Tycho Indexer: The testing module runs a minified version of Tycho Indexer. You need to ensure that the latest version is correctly setup in your PATH and if it isn't you need to (re)install Tycho. Run the following command on your terminal to check the version: 

    > tycho-indexer --version
tycho-indexer 0.88.0 # should match the latest version published on GitHub

Step 1: Export Environment Variables

  • RPC_URL: The URL for the Ethereum RPC endpoint. This fetches the storage data.

  • SUBSTREAMS_API_TOKEN: The JWT token for accessing Substreams services. This token is necessary for authentication. Please refer to the Substreams Authentication guide to set up and validate your token.

  • RUST_LOG to define the log level you want to see. For enhanced debugging, we recommend running the testing module with Tycho indexer logs.

export RPC_URL="https://ethereum-mainnet.core.chainstack.com/123123123123"
export SUBSTREAMS_API_TOKEN=eyJhbGci...
export RUST_LOG=protocol_testing=info,tycho_client=error

Step 2: Build the substreams wasm file

If you do not have one already, you must build the wasm file of the substreams package you wish to test. This can be done by navigating to the substreams package directory and running:

cargo build --target wasm32-unknown-unknown --release

Step 3: Run a local Postgres test database using docker-compose.

In /protocol-testing , run:

docker compose -f ./docker-compose.yaml up -d db

Step 4: Run tests

In /protocol-testing , run:

cargo run -- --package <package-name>

These are the optional arguments:

Options:
      --root-path <ROOT_PATH>    Path to the root directory containing all packages. If not provided, it will look for packages in the current working directory
      --package <PACKAGE>        Name of the package to test
      --match-test <MATCH_TEST>  If provided, only run the tests with a matching name
      --db-url <DB_URL>          Postgres database URL for the Tycho indexer [default: postgres://postgres:mypassword@localhost:5431/tycho_indexer_0]
      --vm-simulation-traces     Enable tracing during vm simulations
      --execution-traces         Enable tracing during execution simulations

Complete example

If you want to run tests for ethereum-balancer-v2, use the following:

# Setup Environment Variables
export RPC_URL="https://ethereum-mainnet.core.chainstack.com/123123123123"
export SUBSTREAMS_API_TOKEN=eyJhbGci...
export RUST_LOG=protocol_testing=info,tycho_client=error

# Build Substreams wasm for BalancerV2
cd substreams
cargo build --release --package ethereum-balancer-v2 --target wasm32-unknown-unknown
cd ../protocol-testing

# Run Postgres DB using Docker compose
docker compose -f ./docker-compose.yaml up -d db

# Run test
cargo run -- --package ethereum-balancer-v2

Installing or updating the Tycho Indexer version (Optional)

Troubleshooting

Slow tests

An integration test should take a maximum 5–10 minutes. If the tests take longer, here are key things you can explore:

  1. Ensure you have no infinite loops within your code.

  2. Ensure you're using a small block range for your test, ideally below 1,000 blocks. The blocks in your test only need to cover the creation of the component you are testing. Optionally, they can extend to blocks with changes for the component you want the test to cover. To help limit the test block range, you could explore the initialized_accounts config.

  3. Ensure you are not indexing tokens. Token contracts use a lot of storage, so fetching their historical data is slow. Instead, they are mocked on the simulation engine and don't have to be explicitly indexed. Make an exception if they have unique behavior, like acting as both a token and a pool, or if they are rebasing tokens that provide a getRatemethod.

Note: Substreams uses cache to improve the speed of subsequent runs of the same module. A test's first run is always slower than subsequent runs, unless you change the Substreams module's code.

Account not initialised

There are two main causes for this error:

  1. Your Substreams package is not indexing a contract that is necessary for simulations.

  2. Your test begins at a block that is later than the block on which the contract was created. To fix this, add the missing contract to the initialized_accounts test config.

PreviousCode ArchitectureNextRequest for Quote Protocols

Last updated