The Graph is a protocol for indexing and querying Blockchain data. Currently, The Graph has the legacy version and the decentralized version. The legacy version is a centralized and managed service hosted by The Graph, and it will eventually be shutdown in the future. The decentralized version (aka The Graph Network) consists of 4 major roles: Developers, Indexers, Curators, and Delegators.
ref:
https://thegraph.com/
Video from Finematics
https://finematics.com/the-graph-explained/
Terminologies
- Subgraph: defines which data should be indexed and how they will be stored and queried.
- Developer: designs and implements subgraphs.
- Indexer: stakes GRT and operates Graph Nodes to index data and serve queries.
- Delegator: stakes GRT to indexers to earn rewards, so a delegator doesn't have to run a Graph Node.
- Curator: stakes GRT to signal which subgraphs should be indexed.
- Consumer: pays GRT for querying data from indexers.
Define a Subgraph
A subgraph defines one or more entities of indexed data: what kind of data on the blockchain you want to index for faster querying. Once deployed, subgraphs could be queried by dApps to fetch blockchain data to power their frontend interfaces. Basically, a subgraph is like a database, and an entity (a GraphQL type) is like a table in RDBMS.
A subgraph definition consists of 3 files:
subgraph.yaml: a YAML file that defines the subgraph manifest and metadata.schema.graphql: a GraphQL schema that defines what data (entities) are stored, and how to query it via GraphQL.mappings.ts: AssemblyScript code that translates blockchain data (events, blocks, or contract calls) to GraphQL entities.
GraphQL Schema
First, we need to design our GraphQL entity schemas (the data model) which mainly depends on how you want to query the data instead of how the data are emitted from the blockchain. GraphQL schemas are defined using the GraphQL schema language.
Here're some notes about subgraph's GraphQL schema:
- For each entity in a subgraph, a
id field is required and it must be a string.
- A field with a
! suffix means it's required, otherwise it's optional.
- There is a special field type
Bytes for storing Ethereum hashes and addresses.
- An entity may have relationships to one or more other entities (one-to-one, one-to-many, or many-to-many) which is similar to how you design tables in a relational database.
- Subgraphs also support full-text search (since the storage backend is PostgreSQL). Though, it might only has few use cases in the world of blockchains.
An example of schema.graphql:
type Market @entity {
id: ID!
baseToken: Bytes!
pool: Bytes!
feeRatio: BigInt!
tradingFee: BigDecimal!
tradingVolume: BigDecimal!
blockNumberAdded: BigInt!
timestampAdded: BigInt!
}
type Trader @entity {
id: ID!
realizedPnl: BigDecimal!
fundingPayment: BigDecimal!
tradingFee: BigDecimal!
badDebt: BigDecimal!
totalPnl: BigDecimal!
positions: [Position!]! @derivedFrom(field: "traderRef")
}
type Position @entity {
id: ID!
trader: Bytes!
baseToken: Bytes!
positionSize: BigDecimal!
openNotional: BigDecimal!
openPrice: BigDecimal!
realizedPnl: BigDecimal!
tradingFee: BigDecimal!
badDebt: BigDecimal!
totalPnl: BigDecimal!
traderRef: Trader!
}
ref:
https://thegraph.com/docs/developer/create-subgraph-hosted#the-graphql-schema
https://thegraph.com/docs/developer/graphql-api
https://graphql.org/learn/schema/
It's also worth noting that The Graph supports Time-travel queries. We can query the state of your entities for an arbitrary block in the past:
{
positions(
block: {
number: 1234567
},
where: {
trader: "0x5abfec25f74cd88437631a7731906932776356f9"
}
) {
id
trader
baseToken
positionSize
openNotional
openPrice
realizedPnl
fundingPayment
tradingFee
badDebt
totalPnl
blockNumber
timestamp
}
}
ref:
https://thegraph.com/docs/developer/graphql-api#time-travel-queries
Subgraph Manifest
Second, we must provide a manifest to tell The Graph which contracts we would like to listen to, which contract events we want to index. Also a mapping file that instructs The Graph on how to transform blockchain data into GraphQL entities.
A template file of subgraph.yaml:
specVersion: 0.0.2
description: Test Subgraph
repository: https://github.com/vinta/my-subgraph
schema:
file: ./schema.graphql
dataSources:
- kind: ethereum/contract
name: ClearingHouse
network: {{ network }}
source:
abi: ClearingHouse
address: {{ clearingHouse.address }}
startBlock: {{ clearingHouse.startBlock }}
mapping:
kind: ethereum/events
apiVersion: 0.0.4
language: wasm/assemblyscript
file: ./src/mappings/clearingHouse.ts
entities:
- Protocol
- Market
- Trader
- Position
abis:
- name: ClearingHouse
file: ./abis/ClearingHouse.json
eventHandlers:
- event: PoolAdded(indexed address,indexed uint24,indexed address)
handler: handlePoolAdded
- event: PositionChanged(indexed address,indexed address,int256,int256,uint256,int256,uint256)
handler: handlePositionChanged
Since we would usually deploy our contracts to multiple chains (at least one for mainnet and one for testnet), so we could use a template engine (like mustache.js) to facilate deployment.
$ cat configs/arbitrum-rinkeby.json
{
"network": "arbitrum-rinkeby",
"clearingHouse": {
"address": "0xYourContractAddress",
"startBlock": 1234567
}
}
# generate the subgraph manifest for different networks
$ mustache configs/arbitrum-rinkeby.json subgraph.template.yaml > subgraph.yaml
$ mustache configs/arbitrum-one.json subgraph.template.yaml > subgraph.yaml
It's worth noting that The Graph Legacy (the Hosted Service) supports most of common networks, for instance, mainnet, rinkeby, bsc, matic, arbitrum-one, and optimism. However, The Graph Network (the decentralized version) only supports Ethereum mainnet and rinkeby.
You could find the full list of supported networks on the document:
https://thegraph.com/docs/developer/create-subgraph-hosted#from-an-existing-contract
Mappings
Mappings are written in AssemblyScript and will be compiled to WebAssembly (WASM) when deploying. AssemblyScript's syntax is similar to TypeScript, but it's actually a completely different language.
For each event handler defined in subgraph.yaml under mapping.eventHandlers, we must create an exported function of the same name. What we do in a event handler is basically:
- Creating new entities or loading existed ones by
id.
- Updating fields of entities from a blockchain event.
- Saving entities to The Graph.
- It's not necessary to load an entity before updating it. It's fine to simply create the entity, set properties, then save. If the entity already exists, changes will be merged automatically.
export function handlePoolAdded(event: PoolAdded): void {
// upsert Protocol
const protocol = getOrCreateProtocol()
protocol.publicMarketCount = protocol.publicMarketCount.plus(BI_ONE)
// upsert Market
const market = getOrCreateMarket(event.params.baseToken)
market.baseToken = event.params.baseToken
market.pool = event.params.pool
market.feeRatio = BigInt.fromI32(event.params.feeRatio)
market.blockNumberAdded = event.block.number
market.timestampAdded = event.block.timestamp
// commit changes
protocol.save()
market.save()
}
export function handlePositionChanged(event: Swapped): void {
// upsert Market
const market = getOrCreateMarket(event.params.baseToken)
market.tradingFee = market.tradingFee.plus(swappedEvent.fee)
market.tradingVolume = market.tradingVolume.plus(abs(swappedEvent.exchangedPositionNotional))
...
// upsert Trader
const trader = getOrCreateTrader(event.params.trader)
trader.tradingFee = trader.tradingFee.plus(swappedEvent.fee)
trader.realizedPnl = trader.realizedPnl.plus(swappedEvent.realizedPnl)
...
// upsert Position
const position = getOrCreatePosition(event.params.trader, event.params.baseToken)
const side = swappedEvent.exchangedPositionSize.ge(BD_ZERO) ? Side.BUY : Side.SELL
...
// commit changes
market.save()
trader.save()
position.save()
}
We can also access contract states and call contract functions at the current block (even ). Though the functionality of calling contract functions is limited by @graphprotocol/graph-ts, it's not as powerful as libraries like ethers.js. And no, we cannot import ethers.js in mappings, as mappings are written in AssemblyScript. However, contract calls are quite "expensive" in terms of indexing performance. In extreme cases, some indexers might avoid syncing a very slow subgraph, or charge a premium for serving queries.
export function handlePoolAdded(event: PoolAdded): void {
...
const pool = UniswapV3Pool.bind(event.params.pool)
market.poolTickSpacing = pool.tickSpacing()
...
}
ref:
https://thegraph.com/docs/developer/create-subgraph-hosted#writing-mappings
https://thegraph.com/docs/developer/assemblyscript-api
In addition to event handlers, we're also able to define call handlers and block handlers. A call handler listens to a specific contract function call, and receives input and output of the call as the handler argument. On the contrary, a block handler will be called after every block or after blocks that match a predefined filter - for every block which contains a call to the contract listed in dataSources.
ref:
https://thegraph.com/docs/developer/create-subgraph-hosted#defining-a-call-handler
https://thegraph.com/docs/developer/create-subgraph-hosted#block-handlers
Here're references to how other projects organize their subgraphs:
https://github.com/Uniswap/uniswap-v3-subgraph
https://github.com/Synthetixio/synthetix-subgraph
https://github.com/mcdexio/mai3-perpetual-graph
Deploy a Subgraph
Deploy to Legacy Explorer
Before deploying your subgraph to the Legacy Explorer (the centralized and hosted version of The Graph), we need to create it on the Legacy Explorer dashboard.
Then run the following commands to deploy:
$ mustache configs/arbitrum-rinkeby.json subgraph.template.yaml > subgraph.yaml
$ graph auth --product hosted-service <YOUR_THE_GRAPH_ACCESS_TOKEN>
$ graph deploy --product hosted-service <YOUR_GITHUB_USERNAME>/<YOUR_SUBGRAPH_REPO>
ref:
https://thegraph.com/docs/developer/deploy-subgraph-hosted
Deploy to Subgraph Studio
When we deploy a subgraph to Subgraph Studio (the decentralized version of The Graph), we just push it to the Studio where we're able to test it. Versus, when we "publish" a subgraph in Subgraph Studio, we are publishing it on-chain. Unfortunately, Subgraph Studio only supports Ethereum Mainnet and Rinkeby testnet currently.
$ graph auth --studio <YOUR_SUBGRAPH_DEPLOY_KEY>
$ graph deploy --studio <YOUR_SUBGRAPH_SLUG>
ref:
https://thegraph.com/docs/developer/deploy-subgraph-studio
Token Economics
Before we talk about the token economics of GRT token, it's important to know that the following description only applies to The Graph Network, the decentralized version of The Graph. Also, the name of The Graph Network is a bit ambiguous, it is not a new network or a new blockchain, instead, it is a web service that charges HTTP/WebSocket API calls in GRT token.
To make GRT token somehow valuable, when you query data (through GraphQL APIs) from The Graph Network, you need to pay for each query in GRT. First, you have to connect your wallet and create an account on Subgraph Studio to obtain an API key, then you deposit some GRT tokens into the account's billing balance on Polygon since their billing contract is built on Polygon. At the end of each week, if you used your API keys to query data, you will receive an invoice based on the query fees you have generated during this period. This invoice will be paid using GRT available in your balance.
ref:
https://thegraph.com/docs/studio/billing
When it comes to token economics:
- Indexers earn query fees and indexing rewards. GRT would be slashed if indexers are malicious or serve incorrect data. Though, there's no documentation about how exactly slashing works.
- Delegators earn a portion of query fees and indexing rewards by delegating GRT to existing indexers.
- Curators earn a portion of query fees for the subgraphs they signal on by depositing GRT into a bonding curve of a specific subgraph.
ref:
https://thegraph.com/blog/the-graph-grt-token-economics
https://thegraph.com/blog/the-graph-network-in-depth-part-1
https://thegraph.com/blog/the-graph-network-in-depth-part-2
Query Fee
The price of queries will be set by indexers and vary based on cost to index the subgraph, the demand for queries, the amount of curation signal and the market rate for blockchain queries. Though querying data from the hosted version of The Graph is free now.
The Graph has developed a Cost Model (Agora) for pricing queries, and there is also a microtransaction system (Scalar) that uses state channels to aggregate and compress transactions before being finalized on-chain.
ref:
https://github.com/graphprotocol/agora
https://thegraph.com/blog/scalar
