Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Breaking up creating subgraph flow #811

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion website/next-env.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
/// <reference types="next/image-types/global" />

// NOTE: This file should not be edited
// see https://nextjs.org/docs/basic-features/typescript for more information.
// see https://nextjs.org/docs/pages/building-your-application/configuring/typescript for more information.
2 changes: 1 addition & 1 deletion website/pages/en/developing/_meta.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
export default {
'supported-networks': '',
'creating-a-subgraph': '',
'creating-a-subgraph': 'Creating a Subgraph',
'graph-ts': 'AssemblyScript API',
'unit-testing-framework': '',
'developer-faqs': '',
Expand Down
8 changes: 8 additions & 0 deletions website/pages/en/developing/creating-a-subgraph/_meta.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
export default {
'creating-a-subgraph': '',
'install-the-cli': '',
'subgraph-manifest': '',
'ql-schema': '',
'assemblyscript-mappings': '',
advanced: '',
}
555 changes: 555 additions & 0 deletions website/pages/en/developing/creating-a-subgraph/advanced.mdx

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
---
title: Writing AssemblyScript Mappings
---

## Writing Mappings

The mappings take data from a particular source and transform it into entities that are defined within your schema. Mappings are written in a subset of [TypeScript](https://www.typescriptlang.org/docs/handbook/typescript-in-5-minutes.html) called [AssemblyScript](https://github.com/AssemblyScript/assemblyscript/wiki) which can be compiled to WASM ([WebAssembly](https://webassembly.org/)). AssemblyScript is stricter than normal TypeScript, yet provides a familiar syntax.

For each event handler that is defined in `subgraph.yaml` under `mapping.eventHandlers`, create an exported function of the same name. Each handler must accept a single parameter called `event` with a type corresponding to the name of the event which is being handled.

In the example subgraph, `src/mapping.ts` contains handlers for the `NewGravatar` and `UpdatedGravatar` events:

```javascript
import { NewGravatar, UpdatedGravatar } from '../generated/Gravity/Gravity'
import { Gravatar } from '../generated/schema'

export function handleNewGravatar(event: NewGravatar): void {
let gravatar = new Gravatar(event.params.id)
gravatar.owner = event.params.owner
gravatar.displayName = event.params.displayName
gravatar.imageUrl = event.params.imageUrl
gravatar.save()
}

export function handleUpdatedGravatar(event: UpdatedGravatar): void {
let id = event.params.id
let gravatar = Gravatar.load(id)
if (gravatar == null) {
gravatar = new Gravatar(id)
}
gravatar.owner = event.params.owner
gravatar.displayName = event.params.displayName
gravatar.imageUrl = event.params.imageUrl
gravatar.save()
}
```

The first handler takes a `NewGravatar` event and creates a new `Gravatar` entity with `new Gravatar(event.params.id.toHex())`, populating the entity fields using the corresponding event parameters. This entity instance is represented by the variable `gravatar`, with an id value of `event.params.id.toHex()`.

The second handler tries to load the existing `Gravatar` from the Graph Node store. If it does not exist yet, it is created on-demand. The entity is then updated to match the new event parameters before it is saved back to the store using `gravatar.save()`.

### Recommended IDs for Creating New Entities

It is highly recommended to use `Bytes` as the type for `id` fields, and only use `String` for attributes that truly contain human-readable text, like the name of a token. Below are some recommended `id` values to consider when creating new entities.

- `transfer.id = event.transaction.hash`

- `let id = event.transaction.hash.concatI32(event.logIndex.toI32())`

- For entities that store aggregated data, for e.g, daily trade volumes, the `id` usually contains the day number. Here, using a `Bytes` as the `id` is beneficial. Determining the `id` would look like

```typescript
let dayID = event.block.timestamp.toI32() / 86400
let id = Bytes.fromI32(dayID)
```

- Convert constant addresses to `Bytes`.

`const id = Bytes.fromHexString('0xdead...beef')`

There is a [Graph Typescript Library](https://github.com/graphprotocol/graph-tooling/tree/main/packages/ts) which contains utilities for interacting with the Graph Node store and conveniences for handling smart contract data and entities. It can be imported into `mapping.ts` from `@graphprotocol/graph-ts`.

### Handling of entities with identical IDs

When creating and saving a new entity, if an entity with the same ID already exists, the properties of the new entity are always preferred during the merge process. This means that the existing entity will be updated with the values from the new entity.

If a null value is intentionally set for a field in the new entity with the same ID, the existing entity will be updated with the null value.

If no value is set for a field in the new entity with the same ID, the field will result in null as well.

## Code Generation

In order to make it easy and type-safe to work with smart contracts, events and entities, the Graph CLI can generate AssemblyScript types from the subgraph's GraphQL schema and the contract ABIs included in the data sources.

This is done with

```sh
graph codegen [--output-dir <OUTPUT_DIR>] [<MANIFEST>]
```

but in most cases, subgraphs are already preconfigured via `package.json` to allow you to simply run one of the following to achieve the same:

```sh
# Yarn
yarn codegen

# NPM
npm run codegen
```

This will generate an AssemblyScript class for every smart contract in the ABI files mentioned in `subgraph.yaml`, allowing you to bind these contracts to specific addresses in the mappings and call read-only contract methods against the block being processed. It will also generate a class for every contract event to provide easy access to event parameters, as well as the block and transaction the event originated from. All of these types are written to `<OUTPUT_DIR>/<DATA_SOURCE_NAME>/<ABI_NAME>.ts`. In the example subgraph, this would be `generated/Gravity/Gravity.ts`, allowing mappings to import these types with.

```javascript
import {
// The contract class:
Gravity,
// The events classes:
NewGravatar,
UpdatedGravatar,
} from '../generated/Gravity/Gravity'
```

In addition to this, one class is generated for each entity type in the subgraph's GraphQL schema. These classes provide type-safe entity loading, read and write access to entity fields as well as a `save()` method to write entities to store. All entity classes are written to `<OUTPUT_DIR>/schema.ts`, allowing mappings to import them with

```javascript
import { Gravatar } from '../generated/schema'
```

> **Note:** The code generation must be performed again after every change to the GraphQL schema or the ABIs included in the manifest. It must also be performed at least once before building or deploying the subgraph.

Code generation does not check your mapping code in `src/mapping.ts`. If you want to check that before trying to deploy your subgraph to Graph Explorer, you can run `yarn build` and fix any syntax errors that the TypeScript compiler might find.
119 changes: 119 additions & 0 deletions website/pages/en/developing/creating-a-subgraph/install-the-cli.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
---
title: Install the Graph CLI
---

> In order to use your subgraph on The Graph's decentralized network, you will need to [create an API key](/deploying/subgraph-studio-faqs/#2-how-do-i-create-an-api-key) in [Subgraph Studio](https://thegraph.com/studio/apikeys/). It is recommended that you add signal to your subgraph with at least 3,000 GRT to attract 2-3 Indexers. To learn more about signaling, check out [curating](/network/curating/).

## Getting Started

### Install the Graph CLI

To build and deploy a subgraph, you will need the [Graph CLI](https://github.com/graphprotocol/graph-tooling/tree/main/packages/cli).

The Graph CLI is written in TypeScript, and you must have `node` and either `npm` or `yarn` installed to use it. Check for the [most recent](https://github.com/graphprotocol/graph-tooling/releases?q=%40graphprotocol%2Fgraph-cli&expanded=true) CLI version.

On your local machine, run one of the following commands:

#### Using [npm](https://www.npmjs.com/)

```bash
npm install -g @graphprotocol/graph-cli@latest
```

#### Using [yarn](https://yarnpkg.com/)

```bash
yarn global add @graphprotocol/graph-cli
```

- The `graph init` command can be used to set up a new subgraph project, either from an existing contract or from an example subgraph.

- This `graph init` command can also create a subgraph in Subgraph Studio by passing in `--product subgraph-studio`.

- If you already have a smart contract deployed to your preferred network, you can bootstrap a new subgraph from that contract to get started.

## Create a Subgraph

### From an Existing Contract

The following command creates a subgraph that indexes all events of an existing contract:

```sh
graph init \
--product subgraph-studio
--from-contract <CONTRACT_ADDRESS> \
[--network <ETHEREUM_NETWORK>] \
[--abi <FILE>] \
<SUBGRAPH_SLUG> [<DIRECTORY>]
```

- The command tries to retrieve the contract ABI from Etherscan.

- The Graph CLI relies on a public RPC endpoint. While occasional failures are expected, retries typically resolve this issue. If failures persist, consider using a local ABI.

- If any of the optional arguments are missing, it guides you through an interactive form.

- The `<SUBGRAPH_SLUG>` is the ID of your subgraph in [Subgraph Studio](https://thegraph.com/studio/). It can be found on your subgraph details page.

### From an Example Subgraph

The following command initializes a new project from an example subgraph:

```sh
graph init --studio <SUBGRAPH_SLUG> --from-example=example-subgraph
```

- The [example subgraph](https://github.com/graphprotocol/example-subgraph) is based on the Gravity contract by Dani Grant, which manages user avatars and emits `NewGravatar` or `UpdateGravatar` events whenever avatars are created or updated.

- The subgraph handles these events by writing `Gravatar` entities to the Graph Node store and ensuring these are updated according to the events.

### Add New `dataSources` to an Existing Subgraph

Since `v0.31.0`, the Graph CLI supports adding new `dataSources` to an existing subgraph through the `graph add` command:

```sh
graph add <address> [<subgraph-manifest default: "./subgraph.yaml">]

Options:

--abi <path> Path to the contract ABI (default: download from Etherscan)
--contract-name Name of the contract (default: Contract)
--merge-entities Whether to merge entities with the same name (default: false)
--network-file <path> Networks config file path (default: "./networks.json")
```

#### Specifics

The `graph add` command will fetch the ABI from Etherscan (unless an ABI path is specified with the `--abi` option) and creates a new `dataSource`, similar to how the `graph init` command creates a `dataSource` `--from-contract`, updating the schema and mappings accordingly. This allows you to index implementation contracts from their proxy contracts.

- The `--merge-entities` option identifies how the developer would like to handle `entity` and `event` name conflicts:

- If `true`: the new `dataSource` should use existing `eventHandlers` & `entities`.

- If `false`: a new `entity` & `event` handler should be created with `${dataSourceName}{EventName}`.

- The contract `address` will be written to the `networks.json` for the relevant network.

> Note: When using the interactive CLI, after successfully running `graph init`, you'll be prompted to add a new `dataSource`.

### Getting The ABIs

The ABI file(s) must match your contract(s). There are a few ways to obtain ABI files:

- If you are building your own project, you will likely have access to your most current ABIs.
- If you are building a subgraph for a public project, you can download that project to your computer and get the ABI by using [`npx hardhat compile`](https://hardhat.org/hardhat-runner/docs/guides/compile-contracts#compiling-your-contracts) or using `solc` to compile.
- You can also find the ABI on [Etherscan](https://etherscan.io/), but this isn't always reliable, as the ABI that is uploaded there may be out of date. Make sure you have the right ABI, otherwise running your subgraph will fail.

## SpecVersion Releases

| Version | Release notes |
| :-: | --- |
| 1.2.0 | Added support for [Indexed Argument Filtering](/#indexed-argument-filters--topic-filters) & declared `eth_call` |
| 1.1.0 | Supports [Timeseries & Aggregations](#timeseries-and-aggregations). Added support for type `Int8` for `id`. |
| 1.0.0 | Supports [`indexerHints`](/developing/creating-a-subgraph/#indexer-hints) feature to prune subgraphs |
| 0.0.9 | Supports `endBlock` feature |
| 0.0.8 | Added support for polling [Block Handlers](/developing/creating-a-subgraph/#polling-filter) and [Initialisation Handlers](/developing/creating-a-subgraph/#once-filter). |
| 0.0.7 | Added support for [File Data Sources](/developing/creating-a-subgraph/#file-data-sources). |
| 0.0.6 | Supports fast [Proof of Indexing](/network/indexing/#what-is-a-proof-of-indexing-poi) calculation variant. |
| 0.0.5 | Added support for event handlers having access to transaction receipts. |
| 0.0.4 | Added support for managing subgraph features. |
Loading