Tycho Protocol SDK is a library to integrate liquidity layer protocols (DEXs, Staking, Lending etc.) into Tycho.

Integration Process

Integrating with Tycho requires three components:

1. Indexing: Provide the protocol state/data needed for simulation and execution

2. Simulation: Implement the protocol's logic for simulations

3. Execution: Define how to encode and execute swaps against your protocol

Indexing

Provide a substreams package that emits a specified set of messages. If your protocol already has a substreams package, you can adjust it to emit the required messages.

Important: Simulation happens entirely off-chain. This means everything needed during simulation must be explicitly indexed.

Simulation

Tycho offers two integration modes:

• VM Integration: Implement an adapter interface in a language that compiles to VM bytecode. This SDK provides a Solidity interface (read more here). Simulations run in an empty VM loaded only with the indexed contracts, storage and token balances.

• Native Rust Integration: Implement a Rust trait that defines the protocol logic. Values used in this logic must be indexed as state attributes.

Execution

To enable swap execution, implement:

1. SwapEncoder: A Rust struct that formats input/output tokens, pool addresses, and other parameters correctly for the Executor contract.

2. Executor: A Solidity contract that handles the execution of swaps over your protocol's liquidity pools.

Integration Criteria

Tycho supports many protocol designs, however certain architectures present indexing challenges.

Before integrating, consider these limitations:

Soon to be supported:

• Protocols that interface with external contracts during operations Tycho should support (i.e swap, price and limit calculations etc.). External contracts are those not deployed by the protocol's factories. ERC20 token contracts are exempt from this restriction.

Not supported:

• Protocols where any operation that Tycho should support requires off-chain data, such as signed prices.

Our indexing integrations require a Substreams SPKG to transform raw blockchain data into structured data streams. These packages enable our indexing integrations to track protocol state changes with low latency.

What is Substreams?

Substreams is a new indexing technology that uses Rust modules to process blockchain data. These modules, along with protobuf definitions and a manifest, are packaged into an SPKG file which can be run on the Substreams server.

Learn more:

• Quick explanation

• SPKGs

• Full documentation

Integration Modes

VM Integration

VM integrations primarily track contract storage associated with the protocol’s behavior. A key limitation in Substreams to keep in mind is that you must witness a contract’s creation to access its full storage. Most integrations will likely use the VM method due to its relative simplicity, so this guide focuses on VM-based integrations.

It's important to keep in mind that simulations are run in an empty VM loaded only with the indexed contracts and storage. If your protocol calls external contracts during any simulation (swaps, price calculations etc), those contracts must also be indexed.

Native Integration

Native integrations follow a similar approach with one main difference: instead of emitting changes in contract storage slots, they should emit values for all created and updated attributes relevant to the protocol’s behavior.

Understanding the Data Model

The Tycho Indexer ingests all data versioned by block and transaction. This approach helps maintain a low-latency feed and correctly handles chains that may undergo reorgs. The key requirements for the data emitted are:

1. Each state change must include the transaction that caused it

2. Each transaction must be paired with its corresponding block

3. All changes must be absolute values (final state), not deltas

Details of the data model that encodes these changes, transactions, and blocks in messages are available here. These models facilitate communication between Substreams and the Tycho Indexer, as well as within Substreams modules. Tycho Indexer expects to receive a BlockChanges output from your Substreams package.

Changes must be aggregated at the transaction level; it is considered an error to emit BlockChanges with duplicate transactions in the changes attributes.

Data Encoding

To ensure compatibility across blockchains, many of the data types used are encoded as variable-length bytes. This flexible approach requires an informal interface so that consuming applications can interpret these bytes consistently:

• Integers: When encoding integers, particularly those representing balances, always use unsigned big-endian format. Balances are referenced at multiple points within the system and need to be consistently decoded along their entire journey.

• Strings: Use UTF-8 encoding for any string data stored as bytes.

• Attributes: Attribute encoding is variable and depends on the specific use case. However, whenever possible, follow the encoding standards mentioned above for integers and string

Reserved Attributes

Some attribute names are reserved for specific functions in our simulation process. Use these names only for their intended purposes. See list of reserved attributes.

Changes of interest

Tycho Protocol Integrations should communicate the following changes:

1. New Protocol Components: Notify any newly added protocol components, such as pools, pairs, or markets—essentially, anything that indicates a new operation can now be executed using the protocol.

2. ERC20 Balances: Whenever the balances of any contracts involved with the protocol change, report these changes in terms of absolute balances.

3. Protocol State Changes: For VM integrations, this typically involves reporting contract storage changes for all contracts whose state may be accessed during a swap operation (except token contracts).

For a hands-on integration guide, refer to the following pages:

Install Rust

Install Rust. You can do so with the following command:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Install Substreams

You can do so with any of the following:

Using Homebrew:

brew install streamingfast/tap/substreams

Using precompiled binaries

# Use correct binary for your platform
LINK=$(curl -s https://api.github.com/repos/streamingfast/substreams/releases/latest | awk '/download.url.*linux/ {print $2}' | sed 's/"//g')
curl -L  $LINK  | tar zxf -

Compiling from source:

git clone https://github.com/streamingfast/substreams
cd substreams
go install -v ./cmd/substreams

Install Buf

Using Homebrew:

$ brew install bufbuild/buf/buf

For other installation methods, see the official buf website

Fork the SDK repo

1. Start by making a fork of the Tycho Protocol SDK repository

2. Clone the fork you just created

3. Make sure everything compiles fine

   Copy

   cd substreams
   cargo check --all

1. Understanding the Protocol

Before integrating, ensure you have a thorough understanding of the protocol’s structure and behavior. Key areas to focus on include:

1. Contracts and their roles: Identify the contracts involved in the protocol and the specific roles they play. Understand how they impact the behavior of the component you're integrating.

2. Conditions for State Changes: Determine which conditions, such as oracle updates or particular method calls, trigger state changes (e.g. price updates) in the protocol.

3. Component Addition and Removal: Check how components are added or removed within the protocol. Many protocols either use a factory contract to deploy new components or provision new components directly through specific method calls.

Once you have a clear understanding of the protocol's mechanics, you can proceed with the implementation.

2. Choosing a template

We provide two templates that outline all necessary implementation steps to get you started:

• ethereum-template-factory: Use when the protocol deploys one contract per pool (e.g., UniswapV2, UniswapV3)

• ethereum-template-singleton: Use when the protocol uses a fixed set of contracts (e.g., UniswapV4)

If you are unsure which one to choose please ask in the tycho.build group for support.

Once you have chosen a template:

1. Create a new directory for your integration by copying the template, rename all the references to ethereum-template-[factory|singleton] to [CHAIN]-[PROTOCOL_SYSTEM](please use lowercase letters):

   Copy

   cp -r ./substreams/ethereum-template-factory ./substreams/[CHAIN]-[PROTOCOL_SYSTEM]

2. Now, generate the required protobuf code by running:

   Copy

   substreams protogen substreams.yaml --exclude-paths="google"

3. Next, register the new package within the workspace by adding it to the members list in substreams/Cargo.toml.

4. Add any ABIs specific to your protocol under [CHAIN]-[PROTOCOL-SYSTEM]/abi/

5. Your project should now compile and be runnable with substreams:

   Copy

   cd [CHAIN]-[PROTOCOL-SYSTEM]
   cargo build --release --target wasm32-unknown-unknown
   substreams gui substreams.yaml map_protocol_changes

3. Implementation

If you're using a template, at minimum you'll need to implement three key sections to ensure proper functionality:

1. Identify newly created ProtocolComponents and Metadata

   Extract relevant protocol components and attach all necessary metadata needed for encoding swaps (or other actions) or filtering components. Examples of such attributes could be: pool identifier, pool keys, swap fees, pool_type and any other relevant static properties. Note that some attribute names are reserved. They may not always be needed but must be respected for compatibility.

2. Emit balances for ProtocolComponents Tycho tracks TVL per component, so you must emit a BalanceChange whenever an event impacts the balances associated with a component. Absolute balances are expected here. Often protocols are only able to identify balance deltas - to handle these effectively please refer to our page detailing handling relative balances.

3. Track relevant storage slot changes [VM implementations only] For factory-like protocols: the template covers this automatically as long as the ProtocolComponent.id is equivalent with the contract address. For singleton contracts: you'll have to collect the contract changes for the contracts you need tracked. To handle these effectively please refer to our page detailing tracking of contract storage.

Some protocols may require additional customisation based on their specific architecture. See Common Problems & Patterns for how to handle these cases.

We provide a comprehensive testing suite for Substreams modules. The testing suite facilitates end-to-end testing, ensuring your Substreams modules function as expected. For unit tests, please use standard Rust unit testing practices.

What is tested?

The testing suite runs Tycho Indexer with your provided Substream implementation for a specific block range and verifies that the end state matches the expected state specified on the testing YAML file. This confirms that your substreams package is indexable and outputs what you're expecting.

Next it simulates transactions using Tycho Simulation engine. This will verify that the that all necessary data was indexed as well as the functionality of the provided SwapAdapter contract.

Test Configuration

Inside your substreams directory you'll need an integration_test.tycho.yaml file. This test template file already outlines everything you need, however for clarity some test configs are expanded upon here:

1. skip_balance_check

By default this should be false. During testing the balances reported for the component are verified by comparing them to the onchain balances of the Component.id. This should be set to false if:

1. the Component.id does not correlate to a contract address

2. balances are not stored on the component's contract (i.e. they're stored on a vault)

If this skip is set to true, it is required that you comment the reason why.

2. initialized_accounts

Set to a list of addresses of contracts that are required during simulation, but their creation is not indexed within the test block range. Leave empty if not required.

It is important to note that this config is used during testing only. It is expected that the accounts listed here are still properly initialised by your substreams package. This configuration only eliminates the need to include historical blocks containing the initialisation events in your test data. This is useful for ensuring tests are targeted and quick to run.

The initialized_accounts config can be used on 2 levels in the test configuration file:

• global: the accounts listed here are used for all tests in this suite

• test level: the accounts listed here are scoped to that test only

5. expected_components

A list of the components whose creation you are testing, including all component data (tokens, static attributes etc). You do not need to include all components created within your test block range, only the ones you wish to focus on in the test.

4. skip_simulation

By default this should be set to false and should only be set to true temporarily if you want to isolate testing the indexing phase only, or for extenuating circumstances (such as you are testing indexing a pool type that simulation does not support yet). If set to true, it is required to comment the reason why.

Troubleshooting

Slow tests

At the most, an integration test is expected to take 5-10 minutes to run. If the tests take noticeably longer than that, there are a few key things you can look into:

1. ensure you have no infinite loops within your code.

2. ensure you are using a small block range for your test, ideally below 1000 blocks. The blocks you include in your test need to cover only the creation of the component you are testing and optionally extend to blocks with changes for that component that you want the test to cover. To help keep the test block range small, it might be useful for you to look into the initialized_accounts config.

3. make sure you are not indexing tokens. Token contracts tend to use a lot of storage, so are slow to fetch historical data for. Instead they are mocked on the simulation engine and do not need to be explicitly indexed. Exceptions include if they have unique behavior, such as can act as both a token and a pool, or rebasing tokens that provide a getRatemethod.

Note: substreams uses cache to improve the speed of subsequent runs of the same module. The first run of a test will always be slower than subsequent runs (unless you adjust the substreams module).

Account not initialised

There are 2 main causes for this error:

1. your substream package is not indexing a contract that is needed for simulations

2. your test starts at a block that is later than the block the contract is created on. To fix this, add the missing contract to the initialized_accounts test config.

Debugging

For enhanced debugging, running the testing module with the --tycho-logs flag is recommended. It will enable Tycho-indexer logs.

Prerequisites

Before continuing, ensure the following tools and libraries are installed on your system:

• Docker: Containerization platform for running applications in isolated environments.

• Conda: Package and environment manager for Python and other languages.

• AWS CLI: Tool to manage AWS services from the command line.

• Git: Version control tool

• Rust: Programming language and toolchain

• GCC: GNU Compiler Collection

• libpq: PostgreSQL client library

• OpenSSL (libssl): OpenSSL development library

• pkg-config: Helper tool for managing compiler flags

• Conda: Python package manager

• pip: Python package installer

Archive node

The testing system relies on an EVM Archive node to fetch the state from a previous block. Indexing only with Substreams, as done in Tycho's production mode, requires syncing blocks since the protocol's deployment date, which can take a long time. The node skips this requirement by fetching all the required account's storage slots on the block specified in the testing yamlfile.

The node also needs to support the debug_storageRangeAt method, as it's a requirement for our Token Quality Analysis.

Tycho Indexer

Verify the current version

The testing module runs a minified version of Tycho Indexer. You can ensure that the latest version is correctly setup in your PATH by running the following command on your terminal:

> tycho-indexer --version
tycho-indexer 0.62.0 # should match the latest version published on GitHub

Installing or updating the version (Optional)

If the command above does not provide the expected output, you need to (re)install Tycho.

git clone git@github.com:propeller-heads/tycho-indexer.git
cd tycho-indexer

cargo build --release --bin tycho-indexer

sudo ln -s $(pwd)/target/release/tycho-indexer /usr/local/bin/tycho-indexer

> tycho-indexer --version
tycho-indexer 0.54.0 # should match the latest version published on GitHub

tar -xvzf tycho-indexer-x86_64-unknown-linux-gnu-{version}.tar.gz

// Ensure the binary is executable:
sudo chmod +x tycho-indexer
// Create symlink
sudo ln -s $(pwd)/tycho-indexer /usr/local/bin/tycho-indexer

> tycho-indexer --version
tycho-indexer 0.54.0 # should match the latest version published on GitHub

Test Configuration

Tests are defined in a yaml file. A documented template can be found at substreams/ethereum-template/integration_test.tycho.yaml. The configuration file should include:

• The target Substreams config file.

• The corresponding SwapAdapter and args to build it.

• The expected protocol types.

• The tests to be run.

Each test will index all blocks between start-block and stop-block, verify that the indexed state matches the expected state, and optionally simulate transactions using the provided SwapAdapter.

You will also need the VM Runtime file for the adapter contract. Our testing script should be able to build it using your test config. The script to generate this file manually is available under evm/scripts/buildRuntime.sh.

Setup testing environment

To set up your test environment, run the setup environment script. It will create a Conda virtual env and install all the required dependencies.

./setup_env.sh

This script must be run from within the tycho-protocol-sdk/testing directory.

Lastly, you need to activate the conda env:

conda activate tycho-protocol-sdk-testing

Running Tests

Step 1: Export Environment Variables

Export the required environment variables for the execution. You can find the available environment variables in the .env.default file. Please create a .env file in the testing directory and set the required environment variables.

Environment Variables

RPC_URL

• Description: The URL for the Ethereum RPC endpoint. This is used to fetch the storage data.

• Example: export RPC_URL="https://ethereum-mainnet.core.chainstack.com/123123123123"

SUBSTREAMS_API_TOKEN

• Description: The JWT token for accessing Substreams services. This token is required for authentication. Please refer to Substreams Authentication guide to setup and validate your token.

• Example: export SUBSTREAMS_API_TOKEN=eyJhbGci...

Step 2: Set up tests

If you do not have one already, you must build the wasm file of the package you wish to test. This can be done by navigating to the package directory and running:

cargo build --target wasm32-unknown-unknown --release

Then, run a local Postgres test database using docker-compose.

docker compose -f ./testing/docker-compose.yaml up -d db

Step 3: Run tests

Run tests for your package. This must be done from the main project directory.

python ./testing/src/runner/cli.py --package "your-package-name"

Example

If you want to run tests for ethereum-balancer-v2, use:

// Activate conda environment
conda activate tycho-protocol-sdk-testing

// Setup Environment Variables
export RPC_URL="https://ethereum-mainnet.core.chainstack.com/123123123123"
export SUBSTREAMS_API_TOKEN=eyJhbGci...

// Build BalancerV2's Substreams wasm
cd substreams
cargo build --release --package ethereum-balancer-v2 --target wasm32-unknown-unknown
cd ..

// Run Postgres DB using Docker compose
docker compose -f ./testing/docker-compose.yaml up -d db

// Run the testing file
python ./testing/src/runner/cli.py --package "ethereum-balancer-v2"

Testing CLI args

A list and description of all available CLI args can be found using:

python ./testing/src/runner/cli.py --help

Some protocol design choices follow a common pattern. Instructions on how to handle these cases are provided. Such cases include:

• Factory contracts

• Tracking contract storage [VM implementations]

• Using relative component balances

• Vaults/singleton contracts

• Persisting data between modules

Factory contracts

A common protocol design is to use factories to deploy components. In this case it is recommended to detect the creation of these components and store their contract addresses (an potentially other metadata) to track them for use later in the module. See Tracking Components.

Tracking contract storage

For VM implementations it is essential that the contract code and storage of all involved contracts are tracked. See Tracking Contract Storage.

Using relative component balances

For some protocols, absolute component balances are not easily obtainable. Instead, balance deltas/changes are observed. Since absolute balances are expected by Tycho, it is recommended to use a balance store to track current balances and apply deltas as the occur. See Normalizing relative ERC20 Balances.

Vaults/Singleton contracts

For protocols that store balances in an a-typical way (not on dedicated pool contracts), a special approach to balance tracking must be used. See Tracking Contract Balances.

When a contract change is indexed, consumers of the indexed data typically trigger recalculating prices on all pools marked as associated with that contract (the contract is listed in the ProtocolComponent's contracts field). In the case where multiple components are linked to a single contract, such as a vault, this may cause excessive and unnecessary simulations on components that are unaffected by a specific change on the linked contract. In this case it is recommended to use 'manual update' triggers. See Reserved Attributes for more details.

Persisting data between modules

It is often the case where data needs to be persisted between modules in your substream package. This may be because components and their metadata (such as their tokens, or pool type) are needed when handling state changes downstream, or could be because the protocol reports relative changes instead of absolute values and the relative changes must be compounded to reach an absolute value. For this, substream Stores and Custom Protobuf Models are recommended.

If protocols use factories to deploy components, a common pattern used during indexing is to detect the creation of these new components and store their contract addresses to track them downstream. Later, you might need to emit balance and state changes based on the current set of tracked components.

Implementation Steps

1. Implement logic to identify newly created components. A recommended approach is to create a factory.rs module to facilitate the detection of newly deployed components.

2. Use the logic/helper module from step 1 in a map handler that consumes substreams_ethereum::pb::eth::v2::Block models and outputs a message containing all available information about the component at the time of creation, along with the transaction that deployed it. The recommended output model for this initial handler is BlockTransactionProtocolComponents. Note that a single transaction may create multiple components. In such cases, TransactionProtocolComponents.components should list all newly created ProtocolComponents.

3. After emitting, store the protocol components in a Store. This you will use later in the module to detect relevant balance changes and to determine whether a contract is relevant for tracking.

In VM implementations, accurately identifying and extracting relevant contract changes is essential.

The tycho_substreams::contract::extract_contract_changes helper function simplifies this process significantly.

Factory protocols

In factory-based protocols, each contract typically corresponds to a unique component, allowing its hex-encoded address to serve as the component ID, provided there is a one-to-one relationship between contracts and components.

The example below shows how to use a component store to define a predicate. This predicate filters for contract addresses of interest:

use tycho_substreams::contract::extract_contract_changes;

// all changes on this block, aggregated by transaction
let mut transaction_changes: HashMap<_, TransactionChanges> = HashMap::new();

extract_contract_changes(
    &block,
    |addr| {
        components_store
            .get_last(format!("pool:{0}", hex::encode(addr)))
            .is_some()
    },
    &mut transaction_changes,
);

Other protocols

For protocols where contracts aren't necessarily pools themselves, you'll need to identify specific contracts to track. These addresses can be:

1. Hard-coded (for single-chain implementations)

2. Configured via parameters in your substreams.yaml file (for chain-agnostic implementations)

3. Read from the storage of a known contract (hardcoded or configured)

Here's how to extract changes for specific addresses using configuration parameters:

// substreams.yaml
...

networks:
  mainnet:
    params:
      map_protocol_changes: "vault_address=0000,swap_helper_address=0000"

...

// map_protocol_changes
use tycho_substreams::contract::extract_contract_changes;

// all changes on this block, aggregated by transaction
let mut transaction_changes: HashMap<_, TransactionChanges> = HashMap::new();

// *params* is a module input var
let config: DeploymentConfig = serde_qs::from_str(params.as_str())?;
extract_contract_changes_builder(
    &block,
    |addr| {
        addr == config.vault_address
        || addr == config.swap_helper_address
    },
    &mut transaction_changes,
);

Tracking balances is complex if only relative values are available. If the protocol provides absolute balances (e.g., through logs), you can skip this section and simply emit the absolute balances.

To derive absolute balances from relative values, you’ll need to aggregate by component and token, ensuring that balance changes are tracked at the transaction level within each block.

Implementation Steps:

1. Index relative balance changes

To accurately process each block and report balance changes, implement a handler that returns the BlockBalanceDeltas struct. Each BalanceDelta for a component-token pair must be assigned a strictly increasing ordinal to preserve transaction-level integrity. Incorrect ordinal sequencing can lead to inaccurate balance aggregation.

Example interface for a handler that uses an integer, loaded from a store to indicate if a specific address is a component:

#[substreams::handlers::map]
pub fn map_relative_balances(
    block: eth::v2::Block,
    components_store: StoreGetInt64,
) -> Result<BlockBalanceDeltas, anyhow::Error> {
    todo!()
}

Use the tycho_substream::balances::extract_balance_deltas_from_tx function from our Substreams SDK to extract BalanceDelta data from ERC20 Transfer events for a given transaction, as in the Curve implementation.

2. Aggregate balances with an additive store

To efficiently convert BlockBalanceDeltas messages into absolute values while preserving transaction granularity, use the StoreAddBigInt type with a store module. The tycho_substream::balances::store_balance_changes helper function simplifies this task.

Typical usage of this function:

#[substreams::handlers::store]
pub fn store_balances(deltas: BlockBalanceDeltas, store: StoreAddBigInt) {
    tycho_substreams::balances::store_balance_changes(deltas, store);
}

3. Combine absolute values with component and address

Finally, associate absolute balances with their corresponding transaction, component, and token. Use the tycho_substream::balances::aggregate_balances_changes helper function for the final aggregation step. This function outputs BalanceChange structs for each transaction, which can then be integrated into map_protocol_changes to retrieve absolute balance changes per transaction.

Example usage:

#[substreams::handlers::map]
pub fn map_protocol_changes(
    block: eth::v2::Block,
    grouped_components: BlockTransactionProtocolComponents,
    deltas: BlockBalanceDeltas,
    components_store: StoreGetInt64,
    balance_store: StoreDeltas,
) -> Result<BlockChanges> {
    let mut transaction_contract_changes: HashMap<_, TransactionChanges> = HashMap::new();

    aggregate_balances_changes(balance_store, deltas)
        .into_iter()
        .for_each(|(_, (tx, balances))| {
            transaction_contract_changes
                .entry(tx.index)
                .or_insert_with(|| TransactionChanges::new(&tx))
                .balance_changes
                .extend(balances.into_values());
        });
}

Each step ensures accurate tracking of balance changes, making it possible to reflect absolute values for components and tokens reliably.

Sometimes the balances a component uses is stored on a contract that is not a dedicated single pool contract. During Tycho VM simulations, token contracts are mocked and any balances checked or used during a swap need to be overwritten for a simulation to succeed. Default behavior is for the component balances reported to be used to overwrite the pool contract balances. This assumes 2 things: there is a one-to-one relationship between contracts and components, and the hex-encoded contract address serves as the component ID.

If a protocol deviates from this assumption, the balances for each appropriate contract needs to be tracked for that contract. All contracts that have their balances checked/accessed during a simulation need to be tracked in this way.

Implementation Steps:

1. Implement logic/a helper function to extract the absolute balances of the contract. This is protocol specific and might be obtained from an event, or extracted from a storage slot if an appropriate one is identified.

2. Create an InterimContractChange for the contract and add the contract balances using upsert_token_balance.

3. Add these contract changes to the appropriate TransactionChangesBuilder using add_contract_changes.

An example for a protocol that uses a single vault contract is as follows:

use tycho_substreams::models::{InterimContractChange, TransactionChangesBuilder};

// all changes on this block, aggregated by transaction
let mut transaction_changes: HashMap<_, TransactionChanges> = HashMap::new();

// Extract token balances for vault contract
block
    .transaction_traces
    .iter()
    .for_each(|tx| {
        // use helper function to get absolute balances at this transaction
        let vault_balance_change = get_vault_reserves(tx, &components_store, &tokens_store);

        if !vault_balance_change.is_empty() {
            let tycho_tx = Transaction::from(tx);
            let builder = transaction_changes
                .entry(tx.index.into())
                .or_insert_with(|| TransactionChangesBuilder::new(&tycho_tx));

            let mut vault_contract_changes = InterimContractChange::new(VAULT_ADDRESS, false);
            for (token_addr, reserve_value) in vault_balance_change {
                vault_contract_changes.upsert_token_balance(
                    token_addr.as_slice(),
                    reserve_value.value.as_slice(),
                );
            }
            builder.add_contract_changes(&vault_contract_changes);
        }
    });

In some cases, you may need to create custom intermediate protobuf messages, especially when facilitating communication between Substreams handler modules or storing additional data in stores.

Place these protobuf files within your Substreams package, such as ./substreams/ethereum-template/proto/custom-messages.proto. Be sure to link them in the substreams.yaml file. For more details, refer to the substreams manifest documentation or review the official Substreams UniswapV2 example integration.

Some best practices we encourage on all integrations are:

• Clear Documentation: Write clear, thorough comments. Good documentation:

  • Helps reviewers understand your logic and provide better feedback

  • Serves as a guide for future developers who may adapt your solutions

  • Explains why you made certain decisions, not just what they do

• Module Organisation: For complex implementations it is recommended to:

  • Break large module.rs files into smaller, focused files

  • Place these files in a modules directory

  • Name files clearly with numerical prefixes indicating execution order (e.g., 01_parse_events.rs, 02_process_data.rs)

  • Use the same number for parallel modules that depend on the same previous module

  A good example of this done well is in the uniswap-v4 implementation.

• Substream Initial Block: Your package will work just fine setting the initial block in your manifest file to 1, however it means anyone indexing your protocol has to wait for it to process an excessive number of unnecessary blocks before it reaches the first relevant block. This increases substream costs and causes long wait times for the protocol to reach the current block.

  A good rule of thumb is to identify the earliest deployed contract that you index and set this config to that block.

• Performance Considerations:

  • Minimize use of .clone(), especially in loops or on complex/nested data structures. Instead use references (&) when possible.

Certain attribute names are reserved exclusively for specific purposes. Please use them only for their intended applications. Attribute names are unique: if the same attribute is set twice, the value will be overwritten.

Static Attributes

The following attributes names are reserved and must be given using ProtocolComponent.static_att. These attributes MUST be immutable.

1. manual_updates

Description

Determines whether the component updates should be manually triggered using the update_marker state attribute. By default, updates occur automatically whenever there is a change indexed for any of the required contracts. For contracts with frequent changes, automatic updates may not be desirable. For instance, a change in Balancer Vault storage should only trigger updates for the specific pools affected by the change, rather than for all pools indiscriminately. The manual_updates field helps to control and prevent unnecessary updates in such cases.

If it's enable, updates on this component are only triggered by emitting an update_marker state attribute (described below).

Type

Set to [1u8]to enable manual updates.

Example Usage

Attribute {
    name: "manual_updates".to_string(),
    value: [1u8],
    change: ChangeType::Creation.into(),
}

2. pool_id

Description

The pool_id static attribute is used to specify the identifier of the pool when it differs from the ProtocolComponent.id. For example, Balancer pools have a component ID that corresponds to their contract address, and a separate pool ID used for registration on the Balancer Vault contract (needed for swaps and simulations).

Notice: In most of the cases, using ProtocolComponent.id is preferred over pool_id and pool_id should only be used if a special identifier is strictly necessary.

Type

This attribute value must be provided as a UTF-8 encoded string in bytes.

Example Usage

Attribute {
    name: "pool_id".to_string(),
    value: format!("0x{}", hex::encode(pool_registered.pool_id)).as_bytes(),
    change: ChangeType::Creation.into(),
}

State Attributes

The following attributes names are reserved and must be given using EntityChanges. Unlike static attributes, state attributes are updatable.

1. stateless_contract_addr

Description

The stateless_contract_addr_{index} field specifies the address of a stateless contract required by the component. Stateless contracts are those where storage is not accessed for the calls made to it during swaps or simulations.

This is particularly useful in scenarios involving DELEGATECALL. If the contract's bytecode can be retrieved in Substreams, provide it using the stateless_contract_code attribute (see below).

Note: If no contract code is given, the consumer of the indexed protocol has to access a chain node to fetch the code. This is considered non-ideal and should be avoided where possible.

An index is used if multiple stateless contracts are needed. This index should start at 0 and increment by 1 for each additional stateless_contract_addr.

The value for stateless_contract_addr_{index} can be provided in two ways:

1. Direct Contract Address: A static contract address can be specified directly.

2. Dynamic Address Resolution: Alternatively, you can define a function or method that dynamically resolves and retrieves the stateless contract address at runtime. This can be particularly useful in complex contract architectures, such as those using a dynamic proxy pattern. It is important to note that the called contract must be indexed by the Substreams module.

Type

This attribute value must be provided as a UTF-8 encoded string in bytes.

Example Usage

1. Direct Contract Address

To specify a direct contract address:

Attribute {
    name: "stateless_contract_addr_0".into(),
    value: format!("0x{}", hex::encode(address)).into_bytes(),
    change: ChangeType::Creation.into(),
}
Attribute {
    name: "stateless_contract_addr_1".into(),
    value: format!("0x{}", hex::encode(other_address)).into_bytes(),
    change: ChangeType::Creation.into(),
}

2. Dynamic Address Resolution

To specify a function that dynamically resolves the address:

Attribute {
    name: "stateless_contract_addr_0".into(),
    // Call views_implementation() on TRICRYPTO_FACTORY
    value: format!("call:0x{}:views_implementation()", hex::encode(TRICRYPTO_FACTORY)).into_bytes(),
    change: ChangeType::Creation.into(),
}

2. stateless_contract_code

Description

The stateless_contract_code_{index} field is used to specify the bytecode for a given stateless_contract_addr. The index used here must match with the index of the related address.

Type

This attribute value must be provided as bytes.

Example Usage

Attribute {
    name: "stateless_contract_code_0".to_string(),
    value: code.to_vec(),
    change: ChangeType::Creation.into(),
}

3. update_marker

Description

The update_marker field is used to indicate that a pool has changed, thereby triggering an update on the protocol component. This is particularly useful for when manual_updates is enabled.

Type

Set to [1u8]to trigger an update.

Example Usage

Attribute {
    name: "update_marker".to_string(),
    value: vec![1u8],
    change: ChangeType::Update.into(),
};

4. balance_owner[deprecated]

Description

The balance_owner field specifies the address of the account that owns the protocol component tokens, when tokens are not owned by the protocol component itself or the multiple contracts are involved. This is particularly useful for protocols that use a vault, for example Balancer.

Type

This attribute value must be provided as bytes.

Example Usage

Attribute {
    name: "balance_owner".to_string(),
    value: VAULT_ADDRESS.to_vec(),
    change: ChangeType::Creation.into(),
}

To enable simulations for a newly added protocol, it must first be integrated into the Tycho Simulation repository. Please submit a pull request to the repository to include it.

Native Integration

In order to add a new native protocol, you will need to complete the following high-level steps:

1. Create a protocol state struct that contains the state of the protocol, and implements the ProtocolSim trait (see here).

2. Create a tycho decoder for the protocol state: i.e. implement TryFromWithBlock for ComponentWithState to your new protocol state.

Each native protocol should have its own module under tycho-simulation/src/evm/protocol.

VM Integration

To create a VM integration, provide a manifest file and an implementation of the corresponding adapter interface. Tycho Protocol SDK is a library to integrate DEXs and other onchain liquidity protocols into Tycho.

Example Implementations

The following exchanges are integrated with the VM approach:

• Balancer V2 (see code here)

Install prerequisites

1. Install Foundry, start by downloading and installing the Foundry installer:

   Copy

   curl -L https://foundry.paradigm.xyz | bash

   then start a new terminal session and run

   Copy

   foundryup

2. Clone the Tycho Protocol SDK:

   Copy

   git clone https://github.com/propeller-heads/tycho-protocol-lib

3. Install dependencies:

   Copy

   cd ./tycho-protocol-lib/evm/
   forge install

Understanding the ISwapAdapter

Read the documentation of the Ethereum Solidity interface. It describes the functions that need to be implemented and the manifest file.

Additionally, read through the docstring of the ISwapAdapter.sol interface and the ISwapAdapterTypes.sol interface, which defines the data types and errors the adapter interface uses. You can also generate the documentation locally and look at the generated documentation in the ./docs folder:

cd ./evm/
forge doc

Implementing the ISwapAdapter interface

Your integration should be in a separate directory in the evm/src folder. Start by cloning the template directory:

cp ./evm/src/template ./evm/src/<your-adapter-name>

Implement the ISwapAdapter interface in the ./evm/src/<your-adapter-name>.sol file. See Balancer V2 implementation for reference.

Testing your implementation

1. Set up test files:

   • Copy evm/test/TemplateSwapAdapter.t.sol

   • Rename to <your-adapter-name>.t.sol

2. Write comprehensive tests:

   • Test all implemented functions.

   • Use fuzz testing (see Foundry test guide, especially the chapter for Fuzz testing)

   • Reference existing test files: BalancerV2SwapAdapter.t.sol

3. Configure fork testing (run a local mainnet fork against actual contracts and data):

   • Set ETH_RPC_URL environment variable

   • Use your own Ethereum node or services like Infura

4. Run the tests with

   Copy

   cd ./evm
   forge test

Add implementation to Tycho simulation

Once you have the swap adapter implemented for the new protocol, you will need to:

1. Generate the adapter runtime file by running the evm/scripts/buildRuntime.sh script in our SDK repository with the proper input parameters. For example, in order to build the Balancer V2 runtime, the following command can be run:

   Copy

   >>> cd evm
   >>> ./scripts/buildRuntime.sh -c “BalancerV2SwapAdapter” -s “constructor(address)” -a “0xBA12222222228d8Ba445958a75a0704d566BF2C8”

2. Add the associated adapter runtime file to tycho-simulations/src/protocol/vm/assets. Make sure to name the file according to the protocol name used by Tycho Indexer in the following format: <Protocol><Version>Adapter.evm.runtime. For example: vm:balancer_v2 will be BalancerV2Adapter.evm.runtime. Following this naming format is important as we use an automated name resolution for these files.

Filtering

If your implementation does not support all pools indexed for a protocol, you can create a filter function to handle this. This filter can then be used when registering an exchange in the ProtocolStreamBuilder. See here for example implementations.

Swap/Exchange Protocol Guide

Implementing the Protocol

To integrate an EVM exchange protocol:

1. Implement the ISwapAdapter.sol interface.

2. Create a manifest file summarizing the protocol's metadata.

The Manifest File

The manifest file contains author information and additional static details about the protocol and its testing. Here's a list of all valid keys:

yamlCopy# Author information helps us reach out in case of issues
author:
  name: Propellerheads.xyz
  email: alan@propellerheads.xyz

# Protocol Constants
constants:
  # Minimum gas usage for a swap, excluding token transfers
  protocol_gas: 30000
  # Minimum expected capabilities (individual pools may extend these)
  # To learn about Capabilities, see ISwapAdapter.sol)
  capabilities:
    - SellSide
    - BuySide
    - PriceFunction

# Adapter contract (byte)code files
contract: 
  # Contract runtime (deployed) bytecode (required if no source is provided)
  runtime: UniswapV2SwapAdapter.bin
  # Source code (our CI can generate bytecode if you submit this)
  source: UniswapV2SwapAdapter.sol

# Deployment instances for chain-specific bytecode
# Used by the runtime bytecode build script
instances:
  - chain:
      name: mainnet
      id: 1
    # Constructor arguments for building the contract
    arguments:
      - "0x5C69bEe701ef814a2B6a3EDD4B1652CB9cc5aA6f"

# Automatic test cases (useful if getPoolIds and getTokens aren't implemented)
tests:
  instances:
    - pool_id: "0xB4e16d0168e52d35CaCD2c6185b44281Ec28C9Dc"
      sell_token: "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2"
      buy_token: "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"
      block: 17000000
      chain:
        name: mainnet
        id: 1

Key Functions

Price (optional)

Calculates marginal prices for specified amounts.

function price(
    bytes32 poolId,
    IERC20 sellToken,
    IERC20 buyToken,
    uint256[] memory sellAmounts
) external returns (Fraction[] memory prices);

• Return marginal prices in buyToken/sellToken units.

• Include all protocol fees (use minimum fee for dynamic fees).

• If you don't implement this function, flag it accordingly in capabilities and make it revert using the NotImplemented error.

• While optional, we highly recommend implementing this function. If unavailable, we'll numerically estimate the price function from the swap function.

Swap

Simulates token swapping on a given pool.

function swap(
    bytes32 poolId,
    IERC20 sellToken,
    IERC20 buyToken,
    OrderSide side,
    uint256 specifiedAmount
) external returns (Trade memory trade);

• Execute the swap and change the VM state accordingly.

• Include a gas usage estimate for each amount (use gasleft() function).

• Return a Trade struct with a price attribute containing price(specifiedAmount).

• If the price function isn't supported, return Fraction(0, 1) for the price (we'll estimate it numerically).

GetLimits

Retrieves token trading limits.

function getLimits(bytes32 poolId, address sellToken, address buyToken)
    external
    returns (uint256[] memory limits);

• Return the maximum tradeable amount for each token.

• The limit is reached when the change in received amounts is zero or close to zero.

• Overestimate the limit if in doubt.

• Ensure the swap function doesn't error with LimitExceeded for amounts below the limit.

getCapabilities

Retrieves pool capabilities.

function getCapabilities(bytes32 poolId, IERC20 sellToken, IERC20 buyToken)
    external
    returns (Capability[] memory);

getTokens (optional)

Retrieves tokens for a given pool.

function getTokens(bytes32 poolId)
    external
    returns (IERC20[] memory tokens);

• We mainly use this for testing, as it's redundant with the required substreams implementation.

getPoolIds (optional)

Retrieves a range of pool IDs.

function getPoolIds(uint256 offset, uint256 limit)
    external
    returns (bytes32[] memory ids);

• We mainly use this for testing. It's okay not to return all available pools here.

• This function helps us test against the substreams implementation.

• If you implement it, it saves us time writing custom tests.

To integrate a new protocol into Tycho, you need to implement two key components:

1. SwapEncoder (Rust struct) – Handles swap encoding.

2. Executor (Solidity contract) – Executes the swap on-chain.

See more about our code architecture here.

Main Encoder Interface

Each new protocol requires a dedicated SwapEncoder that implements the SwapEncoder trait. This trait defines how swaps for the protocol are encoded into calldata.

fn encode_swap(
    &self,
    swap: Swap,
    encoding_context: EncodingContext,
) -> Result<Vec<u8>, EncodingError>;

This function encodes a swap and its relevant context information into calldata that is compatible with the Executor contract. The output of the SwapEncoder is the input of the Executor (see next section). See current implementations here.

Main Swap Interface

Every integrated protocol requires its own swap executor contract. This contract must conform to the IExecutor interface, allowing it to interact with the protocol and perform swaps. See currently implemented executors here.

It has the main method:

function swap(uint256 givenAmount, bytes calldata data)
    external
    payable
    returns (uint256 calculatedAmount)
{

This function:

• Accepts the input amount (givenAmount).

• Processes the swap using the provided calldata (data) which is the output of the SwapEncoder.

• Returns the final output amount (calculatedAmount).

Ensure that the implementation supports transferring received tokens to a designated receiver address, either within the swap function or through an additional transfer step.

If the protocol requires token approvals (allowances) before swaps can occur, manage these approvals within the implementation to ensure smooth execution of the swap.

Make sure to have an integration test that uses the calldata from the SwapEncoder as input.

Protocols Supporting Consecutive Swap Optimizations

As described in the Swap Group section in our solver encoding docs, our swap strategies support protocols which save token transfers between consecutive swaps using systems such as flash accounting. In such cases, as shown in the diagram below using Uniswap V4 as an example, the SwapEncoder is still only in charge of encoding a single swap. These swaps will then be concatenated at the SwapStrategy level as a single executor call.

Depending on the index of the swap in the swap group, the executor may be responsible for adding additional information which is not necessary in other swaps of the sequence (see the first swap in the diagram below).

Callbacks

Some protocols require a callback during swap execution. In these cases, the executor contract must inherit from ICallback and implement the necessary callback functions.

Required Methods

function handleCallback(
    bytes calldata data
) external returns (bytes memory result);

function verifyCallback(bytes calldata data) external view;

• handleCallback: The main entry point for handling callbacks.

• verifyCallback: Should be called within handleCallback to ensure that the msg.sender is a valid pool from the expected protocol.

Deployment and Whitelisting

Once your implementation is approved:

1. Deploy the executor contract on the appropriate network.

2. Contact us to whitelist the new executor address on our main router contract.

3. Update the configuration by adding the new executor address to executor_addresses.json and register the SwapEncoder within the SwapEncoderBuilder .

By following these steps, your protocol will be fully integrated with Tycho, enabling it to execute swaps seamlessly.

The following diagram summarizes the code architecture:

Encoding

The TychoEncoder is responsible for validating the solutions of orders and providing the user with a list of transactions that you must execute against the TychoRouter or Executors.

At initialization, you can choose which SwapStrategyEncoder should the TychoEncoder use:

• SplitSwapStrategyEncoder:executes the transaction through the TychoRouter

• ExecutorStrategyEncoder: bypasses the TychoRouter and executes directly against our executors.

Internally, the user-selected SwapStrategyEncoder chooses the appropriate SwapEncoder(s) to encode the individual swaps, which depend on the protocols used in the solution.

Execution

The TychoRouter calls one or more Executors (corresponding with the output of the SwapEncoders) to interact with the correct protocol and perform each swap of the solution. The TychoRouter optionally verifies that the user receives a minimum amount of the output token.

If you select the ExecutorStrategyEncoder during setup, you must execute the outputted calldata directly against the Executor which corresponds to the solution’s swap’s protocol. Beware that you are responsible for performing any necessary output amount checks. This strategy is useful if you want to call Tycho executors through your own router. For more information direct execution, see here.

Local Development

Changing Rust Code

Please make sure that the following commands pass if you have changed the code:

cargo check --all
cargo test --all --all-features
cargo +nightly fmt -- --check
cargo +nightly clippy --workspace --all-features --all-targets -- -D warnings

We are using the stable toolchain for building and testing, but the nightly toolchain for formatting and linting, as it allows us to use the latest features of rustfmt and clippy.

If you are working in VSCode, we recommend you install the rust-analyzer extension, and use the following VSCode user settings:

"editor.formatOnSave": true,
"rust-analyzer.rustfmt.extraArgs": ["+nightly"],
"rust-analyzer.check.overrideCommand": [
"cargo",
"+nightly",
"clippy",
"--workspace",
"--all-features",
"--all-targets",
"--message-format=json"
],
"[rust]": {
"editor.defaultFormatter": "rust-lang.rust-analyzer"
}

Changing Solidity code

Setup

Install foudryup and foundry

curl -L https://foundry.paradigm.xyz | bash
foundryup

Running tests

export ETH_RPC_URL=<url>
forge test

Code formatting

forge fmt

Contract Analysis

We use Slither to detect any potential vulnerabilities in our contracts.

To run locally, simply install Slither in your conda env and run it inside the foundry directory.

conda create --name tycho-execution python=3.10
conda activate tycho-execution

pip install slither-analyzer
cd foundry
slither .

Creating a Pull Request

We use conventional commits as our convention for formatting commit messages and PR titles.