Compound analysis of DeFi lending products: an overview

DeFi loan product Compound: Contract chapter

What is a Subgraph

Subgraph is a technology provided by The Graph. The Graph is a decentralized indexing protocol for querying data on blockchain networks like Ethereum and IPFS. It currently supports The Ethereum main network and several test networks, as well as BSC, Matic, Fantom, and others. Anyone can create and deploy an open API based on it, known as the Subgraph, making blockchain data more accessible.

That said, Subgraph is an open indexing service created by developers and deployed to The Graph.

Based on The Graph and subgraph, The data flow is roughly as follows:

  1. A DApp executes a transaction through a contract that emits one or more events.
  2. The Graph’s nodes continuously scan blocks of data, and The subgraph you define sets The events to listen for;
  3. The Graph node listens to events of interest to your subgraph and creates or updates the data of the entity object according to the processing method defined by your subgraph. This data is stored in the Graph node.
  4. The front-end framework connects to the Graph node and writes GraphQL to query the corresponding entity object data and display it on the page.

Simply put, subgraph allows you to reencapsulate on-chain data into data that is easier to query. Because some of the data is hard to query directly on the chain.

The official document gives an example, such as: Who is the owner of a CryptoKitties born between January and February 2018? To answer this question, you must process all the Birth events and then call the ownerOf method for each born CryptoKitty. Even for this relatively simple question, dapps running in browsers can take hours or even days to answer. Indexing blockchain data is difficult. Blockchain properties, such as finality, chain refactoring, or unprocessed blocks, further complicate the process, making it not only time consuming, but also conceptually difficult to retrieve the correct query results from blockchain data. The Graph solves this problem with a hosted service that indexes blockchain data. You can then query these indexes (subgraphs) using the standard GraphQL API.

Graph Node

The Graph Node is the environment for Subgraph to run, and there are actually three ways to run the Subgraph:

  • Hosted Service: Hosted Service is The most widely used and convenient centralized hosting Service provided by The Official team of The Graph.
  • Private Node: Private Node built by the Dapp team according to the open source Graph-node project.
  • The Graph Network: Decentralized Network. The main Network is online, but it is not stable yet. The development team cannot deploy Subgraph directly on The main Network.

Using Hosted Service is the most convenient and by far the most recommended approach. The deployment process of the Subgraph based on Hosted Service is as follows:

1. Sign up for an account on the website

First, you need to sign up with a Github account at The Graph website (thegraph.com/explorer/). If you don’t have a Github account, you need to sign up for one first.

2. Add Subgraph to the official website

Once you’ve created your account and logged in, you can go to Dashboard and start creating subgraphs through the interface. Go to your Dashboard and click Add Subgraph. In addition, each account has an Access token, which is needed when deploying the Subgraph to the Graph node.

You need to fill in the following information:

Except for Subgraph Name and Account, other fields can be changed later. Also, the Subgraph Name is case sensitive, so you’ll need to specify it later when you create a local project, keep that in mind.

3. Install Graph CLI locally

To install Graph CLI, the Graph client tool, you can use NPM or YARN:

$ npm install -g @graphprotocol/graph-cli
$ yarn global add @graphprotocol/graph-cli
Copy the code

In the process of installation, there may be failure due to domestic network problems, which can be solved by using the wall climbing tool and setting the agent, and may also need to set the socks5 agent. During my installation, I used to only set the HTTP proxy, but there was a dependent library that always reported ESOCKETTIMEOUT.

In addition, NPM and YARN only support the HTTP proxy, but do not support the Socks5 proxy. Therefore, you need a tool to convert the HTTP proxy to the Socks5 proxy. The tool is http-proxy-to-socks.

#Assume that the local socks5 proxy port is 1086
#First install the conversion tool
npm install -g http-proxy-to-socks
#Then use this tool to listen on port 8002 and support HTTP proxy, and all HTTP proxy data for 8002 will be converted into PROXY data for SOCKS and sent to 1086HPTS -s 127.0.0.1:1086 -p 8002#Finally, set up the NPM agentNPM config set proxy http://127.0.0.1:8002 NPM config set HTTPS -proxy http://127.0.0.1:8002Copy the code

The same applies to yarn. The corresponding NPM commands can be changed to:

#Installing the Conversion Tool
yarn global add http-proxy-to-socks
#Set the proxy toYarn config set proxy http://127.0.0.1:8002 yarn config set HTTPS -proxy http://127.0.0.1:8002Copy the code

In addition, HPTS -s 127.0.0.1:1086 -p 8002 needs to be on at all times during use.

Of course, this is my situation, some people may have different problems, and I’m not sure if my solution will solve your problem, but at least give it a try.

Finally, if you have a network problem because of Git, you can set Git to use a proxy as well.

4. Create a local Subgraph project

To initialize a subgraph project in the local environment, use the graph init command. You can specify –from-contract or –from-example. –from-example — contract — contract — contract — contract — contract — contract — contract — contract — contract — contract — contract — contract — contract

–from-contract is a more practical way to use the command format:

graph init \
  --from-contract <CONTRACT_ADDRESS> \
  [--network <ETHEREUM_NETWORK>] \
  [--abi <FILE>] \
  <GITHUB_USER>/<SUBGRAPH_NAME> [<DIRECTORY>]
Copy the code
  • –from-contract: Specifies the contract address
  • – the network: Specifies the network to be used. Currently, a variety of networks are supported, including Mainnet, Kovan, Rinkeby, Ropsten, Goerli, POA-Core, XDAI, POA-Sokol, MATIC, Mumbai, Fantom, BSC, clover
  • — ABI: Specifies the ABI file to be used. If not, it will be obtained using etherscan, which is rarely available due to network reasons, so it is better to specify a local file

  • /

    : this is the subgraph I created earlier on the official website, such as keeganlee/First
  • [DIRECTORY] : specifies the DIRECTORY in which the subgraph project is to be created. If not specified, it is the same as SUBGRAPH_NAME

For example, the example project I created uses the following parameters:

graph init \
  --from-contract 0x3d9819210A31b4961b30EF54bE2aeD79B9c9Cd3B \
  --network mainnet \
  --abi Comptroller.json \
  keeganlee/First
Copy the code

During the creation process, you will also download some dependent libraries and may also fail due to domestic network reasons, so you can set up the proxy using the methods mentioned above.

After the project is successfully created, the directory structure of the project is as follows:

subgraph.yaml     The manifest file, which is also the entry point to the Subgraph, is where much of the information is configured
schema.graphql    # Schema mainly defines the data structure of the various entity data stored in the Graph node
package.json
yarn.lock
abis              The abi files for the contract are stored here
  - Contract.json
src               The process logic is to create or update the entity data
  - mapping.ts
generated         # This is automatically generated, and can be regenerated using graph codeGen command after modifying the above files
  - schema.ts
  - Contract
    - Contract.ts
Copy the code

5. Coding

Code the Subgraph project according to the actual business needs. The main files that need to be modified and written are:

  • subgraph.yaml
  • schema.graphql
  • mapping.ts

There’s a lot of knowledge involved in this, and we’ll talk about it later.

Deployment of 6.

Once The coding is done and there are no problems, you can deploy it to The Graph’s Hosted Service.

Before deployment, the first step is to authorize the deployment by executing the following command:

graph auth https://api.thegraph.com/deploy/ <access-token>
Copy the code

The access-token is the access token shown in your Dashboard, as mentioned earlier.

You can then call the following command to deploy from the root of your subgraph project:

yarn deploy
Copy the code

Once deployed, you can go to The Graph website to view your subgraph. For example, after deploying my subgraph, it looks like this:

Inside, there is a progress bar that shows the Syncing(4.6%), which is the synchronizing block data.

On the right side of Playground, the schema defined in the project is displayed. On the left side, the query statement can be written with GraphQL. Click the Play button, and the data of the query result will be displayed in the blank area in the middle.

Private node

To build your own private nodes, follow the instructions for the Graph-Node project on Github to deploy them. Its Github address is:

  • Github.com/graphprotoc…

There are also two ways to deploy a Graph-node, either by following the steps described on the README or by using Docker. I have tried both methods, but the first method failed and many attempts to solve the problem were still fruitless, while the second method succeeded very quickly, so I strongly recommend the Docker method for deployment.

First, install Docker and Docker-compose on the server that needs to be deployed.

Second, open the graph-node/docker/docker-compose. Yml file and modify one line:

ethereum: 'mainnet:http://host.docker.internal:8545'
Copy the code

This row specifies the network and node to be used. For example, if I deploy access to the Kovan network and the node uses Infura, then the value set is:

ethereum: 'kovan:https://kovan.infura.io/v3/<PROJECT_ID>'
Copy the code

Where <PROJECT_ID> is the project ID assigned when infura registered the project.

Finally, in the graph-node/docker directory, run the following command:

docker-compose up -d
Copy the code

– The IPFS, Postgres, and Graph Node Docker images will be automatically downloaded and the container will run in a backend manner.

At this point, the private Graph node is started and you can access the following:

  • Graph Node:
    • GraphiQL: http://localhost:8000/
    • HTTP: http://localhost:8000/subgraphs/name/<subgraph-name>
    • WebSockets: ws://localhost:8001/subgraphs/name/<subgraph-name>
    • Admin: http://localhost:8020/
  • IPFS:
    • 127.0.0.1:5001 or / ip4/127.0.0.1 / TCP / 5001
  • Postgres:
    • postgresql://graph-node:let-me-in@localhost:5432/graph-node

Configure the external ports and permissions. External users can also access the node.

For the local subgraph project to deploy to this private node, run the following command locally:

graph create <SUBGRAPH_NAME> --node http://<NODE_IP>:8020
Copy the code

Where, <SUBGRAPH_NAME> replaces the name of your subgraph, and <NODE_IP> replaces the address of your node.

Next, you need to modify the script configuration of package.json in the Subgraph project. For example, using The Graph’s official Hosted Service, The scripts configuration in my package.json looks like this:

"scripts": {
    "codegen": "graph codegen --output-dir src/types/"."build": "graph build --ipfs https://api.staging.thegraph.com/ipfs/"."create-local": "Graph create Keeganlee /First --node http://127.0.0.1:8020"."deploy-local": "Graph deploy Keeganlee /First --debug --ipfs http://localhost:5001 --node http://127.0.0.1:8020/"."deploy": "graph deploy keeganlee/First --debug --ipfs https://api.thegraph.com/ipfs/ --node https://api.thegraph.com/deploy/"."deploy-staging": "graph deploy --debug --ipfs https://api.staging.thegraph.com/ipfs/ --node https://api.staging.thegraph.com/deploy/ keeganlee/First"."prettier": ". / node_modules/bin/prettier - write '/ *. * * ts'"
  },
Copy the code

The URL after the –ipfs and –node parameters of the build, deploy, and deploy-staging commands must be changed to the IP address of a private node. The following uses deploy as an example:

"deploy": "graph deploy keeganlee/First --debug --ipfs http://<NODE_IP>:5001 --node http://<NODE_IP>:8020".Copy the code

Then you can run the yarn deploy command to deploy the yarn.

The Graph Network

As for The decentralized Graph Network, although The main Network is online, there is no documentation yet on how to deploy Subgraph to The Network. All subgraphs that have migrated to The Network have been co-migrated by The official technical team. The team also said that they will wait for the mainnet to stabilize in the near future before moving other subgraphs to the mainnet.

At present, the most detailed introduction to Network is the official document:

Thegraph.com/docs/networ…

As for the detailed content, I have not studied it in depth, so I will not carry it out, and I will explain this piece in another post after in-depth research.

Subgraph coding

Let’s get the source code for the Compound Subgraph project. There are actually two different sources. Note the difference between the two addresses:

  • Github.com/compound-fi…
  • Github.com/graphprotoc…

The first is The Compound’s official Github address, and The second is The Graph’s. The first one has not submitted any code for more than two years, the second one has been forked since the first one, adding many extensions, and the last one was submitted 8 months ago. The second is also deployed on The Graph’s Hosted Service and displayed publicly:

  • Thegraph.com/explorer/su…

Therefore, from the point of view of the results, the source code project of the second address is worth studying. After downloading the source code and eliminating some irrelevant files, the project structure is as follows:

│ ├── ctoken. Json │ ├─ ctoken. Json │ ├─ erc20 PriceOracle. Json │ └ ─ ─ priceOracle2. Json ├ ─ ─ the SRC │ ├ ─ ─ the mappings │ │ ├ ─ ─ comptroller. Ts │ │ ├ ─ ─ ctoken. Ts │ │ ├ ─ ─ Hels.ts │ │ ├── markets.ts ├── Package.json ├── schema.graphql ├── subgraph.yaml ├─ yarnCopy the code

Abis contains the ABI files for the contracts we need to use, and this is where we will put the ABI files we need when developing our subgraph project. Package. json is the project description file, and the most important thing we need to care about is the script commands that are wrapped in it, which we’ve already covered. The yarn.lock file lists all of the project’s dependencies. It is automatically generated and we do not need to change it.

The rest of the files, under subgraph.yaml, schema.graphql and SRC /mappings, are the core coding work.

Manifest

The Manifest file is the subgraph.yaml file and is the core entry point for defining contracts to index, contract events to listen for, and functions to handle the mapping between event data and the entity data saved to the Graph node, and so on. The core configuration is dataSources. You can configure multiple dataSources. Each data source specifies the name, target network, corresponding contract information, mapping information, and so on. In addition, templates can be used to set up contracts that support dynamic creation.

The following documentation has a complete description of the manifest file:

  • Github.com/graphprotoc…

Compound subgraph. Yaml set only a data source and a template, the data source is Comptroller, the template is CToken. It is important to note that because the Comptroller contract uses the agent mode, the contract address bound to the Comptroller data source is actually that of its agent contract, Unitroller.

The Comptroller data source is set to listen for the MarketListed(Address) event, which is also triggered when the CToken contract is added to the market, so each CToken data source can be initialized using the CToken template in the handler of this event.

Schema

GraphQL Schemas are defined in the schema. GraphQL file, using GraphQL IDL (Interface Definition Language) to define the schema. If you are not familiar with GraphQL Schema, you can learn the following two documents:

  • Graphql.org/learn/schem…
  • Thegraph.com/docs/define…

Schema.graphql defines entities, as shown in the following example:

type Account @entity {
    "User ETH address"
    id: ID!
    "Array of CTokens user is in"
    tokens: [AccountCToken!] !@derivedFrom(field: "account")
    "Count user has been liquidated"
    countLiquidated: Int!
    "Count user has liquidated others"
    countLiquidator: Int!
    "True if user has ever borrowed"
    hasBorrowed: Boolean!
}
Copy the code

Each entity can be likened to a table in a MySQL database, and the above code can be simply interpreted as defining the table structure of the Account.

Below are all the schemas defined in the Compound subgraph:

Among them, the last two, CTokenTransfer and UnderlyingTransfer, are interfaces, while the others are all Entities.

Mappings

“Mappings” is the module that does the most coding in a subgraph project and is written in a subset of TypeScript called AssemblyScript, which can be compiled to WASM (WebAssembly). AssemblyScript is more stringent than regular TypeScript, but provides a common syntax.

“Mappings” refers to in-chain data and schema entities. The “mappings” function maps to in-chain data and schema entities. “Mappings” refers to in-chain data and schema entities, and “mappings” refers to in-chain data and schema entities, and “mappings” refers to in-chain data and schema entities.

For example, as mentioned earlier, the Comptroller data source listens for a MarketListed(address) event and specifies the handler that handles it:

eventHandlers:
  - event: MarketListed(address)
    handler: handleMarketListed
Copy the code

In addition, mapping is specified in the Comptroller data source:

mapping:
  kind: ethereum/events
  apiVersion: 0.04.
  language: wasm/assemblyscript
  file: ./src/mappings/comptroller.ts
  entities:
    - Comptroller
    - Market
Copy the code

Among them, the file specified as. / SRC/the mappings comptroller. Ts, that said comptroller all handler functions defined in the data source will be found in the file. In addition, entities specify Comptroller and Market, indicating that both entities will be used in comptroller.ts. However, in fact, comptroller.ts also uses the Account entity, which is missing from the configuration.

However, before writing the actual mappings, you need to execute Graph CodeGen to automatically generate some TS code. The command can also specify the path where the generated code will be stored, as follows:

graph codegen --output-dir src/types/
Copy the code

The generated TS code is stored in the SRC /types/ directory of the current project. If no directory is specified, the generated directory of the current project is used by default.

Compent-v2-subgraph generates the following directory:

Types │ ├── cto.ts │ ├─ cto.ts │ ├─ erc20.ts │ ├── priceoracle.ts │ ├─ priceoracle.ts │ ├─ Exercises │ ├── CToken │ ├── CToken. Ts │ ├─ erc20. ts │ ├── priceoracle.ts │ ├─ priceoracle.ts │ ├─.ts templates.tsCopy the code

The Comptroller directory corresponds to the Comptroller data source, which is generated by the ABIS set under the data source, converting the ABI file of the contract into a TS file.

Schema. ts is the result of all the schemas defined in the schema.graphql file.

The Templates directory is for each template defined, because only one CToken template is defined, so there is only one CToken subdirectory, and the TS files in this directory are mappings to the ABI defined in the CToken template.

Templates. Ts is simple, with the following code:

import {
  Address,
  DataSourceTemplate,
  DataSourceContext
} from "@graphprotocol/graph-ts";

export class CToken extends DataSourceTemplate {
  static create(address: Address): void {
    DataSourceTemplate.create("CToken", [address.toHex()]);
  }

  static createWithContext(address: Address, context: DataSourceContext): void {
    DataSourceTemplate.createWithContext("CToken", [address.toHex()], context); }}Copy the code

It is a function encapsulation that creates a CToken contract object based on the template.

Then, let’s take a look at. / SRC/the mappings comptroller. Ts specific handleMarketListed handler function implementation:

export function handleMarketListed(event: MarketListed) :void {
  // Dynamically index all new listed tokens
  CToken.create(event.params.cToken)
  // Create the market for this token, since it's now been listed.
  let market = createMarket(event.params.cToken.toHexString())
  market.save()
}
Copy the code

The first line creates a CToken contract object, where the CToken is actually the CToken class defined above in templates.

The second and third rows create a market object and save it. This is an entity instance, which can also be interpreted as generating a new record in the Market table.

In addition, every event object actually inherits from Ethereum. event, which encapsulates some basic properties like this:

export class Event {
  address: Address
  logIndex: BigInt
  transactionLogIndex: BigInt
  logType: string | null
  block: Block
  transaction: Transaction
  parameters: Array<EventParam>
}
Copy the code

Address, Block, and Transaction encapsulate some of the basic attributes for easy invocation.

Of course, the most important thing is to get the parameters of event, which can be read from event.params.xxx.

Finally, look at the following documentation to learn how to write “Mappings” :

  • Thegraph.com/docs/define…
  • Thegraph.com/docs/assemb…

GraphQL API

Once the full subgraph has been developed and successfully deployed to the Graph node, query operations can be implemented. The query is done using the GraphQL API. Most of the data displayed on the front page of Dapp can be realized by writing corresponding query statements using GraphQL API.

The GraphQL API queries are also based primarily on entities defined in the Schema. For example, take the following entity:

type Market @entity {
  id: ID!
  name: String!
  symbol: String!
  borrowRate: BigDecimal!
  supplyRate: BigDecimal!
}
Copy the code

Now, to find out all the market data, the query would read:

{
  markets {
    id
    name
    symbol
    borrowRate
    supplyRate
  }
}
Copy the code

If you want to do a conditional query, the GraphQL API provides a where parameter. For example, if borrowRate is 15, you can say:

{
  markets(where: {borrowRate: 15}) {
    id
    name
    symbol
    borrowRate
    supplyRate
  }
}
Copy the code

If borrowRate is greater than 15, it says:

{
  markets(where: {borrowRate_gt: 15}) {
    id
    name
    symbol
    borrowRate
    supplyRate
  }
}
Copy the code

That is, add the _gt suffix after the query field name. The GraphQL API provides several such suffixes:

  • _not
  • _gt
  • _lt
  • _gte
  • _lte
  • _in
  • _not_in
  • _contains
  • _not_contains
  • _starts_with
  • _ends_with
  • _not_starts_with
  • _not_ends_with

In addition to the WHERE argument, the GraphQL API provides other parameters, including:

  • Id: Specifies the ID to be queried
  • OrderBy: Specifies the sorted field
  • OrderDirection: sort direction, asc | desc
  • First: indicates the number of records to be queried. For example, if the value is set to 10, a maximum of 10 records will be found
  • Skip: Indicates the number of items that are not queried
  • Block: Specifies a block query. You can specify a block number or hash
  • Text: Full-text search

Finally, the GraphQL API also has learning documentation:

  • Thegraph.com/docs/graphq…

conclusion

There are a lot of details involved in Subgraph, but I don’t have the space to cover all of them. This article is more of a guide for those who don’t know much about Subgraph. To learn more about Subgraph, you need to learn more about the documentation and the source code of the actual project.

In the next post, I’ll talk about how to design the Compound Clearing service, which is not an open source project, so I’ll use my own design experience.

Scan the following QR code to follow the public account (public account name: Keegan Xiaosteel)