Indexing
Last updated
Last updated
Our indexing integrations require a Substreams SPKG to transform raw blockchain data into structured data streams. These packages enable our indexing integrations to track protocol state changes with low latency.
Substreams is a new indexing technology that uses Rust modules to process blockchain data. These modules, along with protobuf definitions and a manifest, are packaged into an SPKG file which can be run on the Substreams server.
Learn more:
VM integrations primarily track contract storage associated with the protocol’s behavior. A key limitation in Substreams to keep in mind is that you must witness a contract’s creation to access its full storage. Most integrations will likely use the VM method due to its relative simplicity, so this guide focuses on VM-based integrations.
It's important to keep in mind that simulations are run in an empty VM loaded only with the indexed contracts and storage. If your protocol calls external contracts during any simulation (swaps, price calculations etc), those contracts must also be indexed.
Native integrations follow a similar approach with one main difference: instead of emitting changes in contract storage slots, they should emit values for all created and updated attributes relevant to the protocol’s behavior.
The Tycho Indexer ingests all data versioned by block and transaction. This approach helps maintain a low-latency feed and correctly handles chains that may undergo reorgs. The key requirements for the data emitted are:
Each state change must include the transaction that caused it
Each transaction must be paired with its corresponding block
All changes must be absolute values (final state), not deltas
Changes must be aggregated at the transaction level; it is considered an error to emit BlockChanges with duplicate transactions in the changes attributes.
To ensure compatibility across blockchains, many of the data types used are encoded as variable-length bytes. This flexible approach requires an informal interface so that consuming applications can interpret these bytes consistently:
Integers: When encoding integers, particularly those representing balances, always use unsigned big-endian format. Balances are referenced at multiple points within the system and need to be consistently decoded along their entire journey.
Strings: Use UTF-8 encoding for any string data stored as bytes.
Attributes: Attribute encoding is variable and depends on the specific use case. However, whenever possible, follow the encoding standards mentioned above for integers and string
Tycho Protocol Integrations should communicate the following changes:
New Protocol Components: Notify any newly added protocol components, such as pools, pairs, or markets—essentially, anything that indicates a new operation can now be executed using the protocol.
ERC20 Balances: Whenever the balances of any contracts involved with the protocol change, report these changes in terms of absolute balances.
Protocol State Changes: For VM integrations, this typically involves reporting contract storage changes for all contracts whose state may be accessed during a swap operation (except token contracts).
For a hands-on integration guide, refer to the following pages:
Details of the data model that encodes these changes, transactions, and blocks in messages are available . These models facilitate communication between Substreams and the Tycho Indexer, as well as within Substreams modules. Tycho Indexer expects to receive a BlockChanges output from your Substreams package.
Some attribute names are reserved for specific functions in our simulation process. Use these names only for their intended purposes. .