plugin-paginate-graphql.js

Octokit plugin to paginate GraphQL API endpoint responses

Usage

1<script type="module">
2  import { Octokit } from "https://esm.sh/@octokit/core";
3  import { paginateGraphQL } from "https://esm.sh/@octokit/plugin-paginate-graphql";
4</script>

1import { Octokit } from "@octokit/core";
2import { paginateGraphQL } from "@octokit/plugin-paginate-graphql";

[!IMPORTANT] As we use conditional exports, you will need to adapt your tsconfig.json by setting "moduleResolution": "node16", "module": "node16".

See the TypeScript docs on package.json "exports".
See this helpful guide on transitioning to ESM from @sindresorhus

1const MyOctokit = Octokit.plugin(paginateGraphQL);
2const octokit = new MyOctokit({ auth: "secret123" });
3
4const { repository } = await octokit.graphql.paginate(
5  `query paginate($cursor: String) {
6    repository(owner: "octokit", name: "rest.js") {
7      issues(first: 10, after: $cursor) {
8        nodes {
9          title
10        }
11        pageInfo {
12          hasNextPage
13          endCursor
14        }
15      }
16    }
17  }`,
18);
19
20console.log(`Found ${repository.issues.nodes.length} issues!`);

There are two conventions this plugin relies on:

1. The name of the cursor variable must be $cursor
2. You must include a valid pageInfo object in the paginated resource (see Pagination Direction for more info on what is considered valid)

octokit.graphql.paginate()

The paginateGraphQL plugin adds a new octokit.graphql.paginate() method which accepts a query with a single $cursor variable that is used to paginate.

The query gets passed over to the octokit.graphql()-function. The response is then scanned for the required pageInfo-object. If hasNextPage is true, it will automatically use the endCursor to execute the next query until hasNextPage is false.

While iterating, it continually merges all nodes and/or edges of all responses and returns a combined response in the end.

Warning Please note that this plugin only supports pagination of a single resource - so you can not execute queries with parallel or nested pagination. You can find more details in the chapter below.

octokit.graphql.paginate.iterator()

If your target runtime environments supports async iterators (such as most modern browsers and Node 10+), you can iterate through each response:

1const pageIterator = octokit.graphql.paginate.iterator(
2  `query paginate($cursor: String) {
3    repository(owner: "octokit", name: "rest.js") {
4      issues(first: 10, after: $cursor) {
5        nodes {
6          title
7        }
8        pageInfo {
9          hasNextPage
10          endCursor
11        }
12      }
13    }
14  }`,
15);
16
17for await (const response of pageIterator) {
18  const issues = response.repository.issues;
19  console.log(`${issues.length} issues found.`);
20}

Variables

Just like with octokit/graphql.js, you can pass your own variables as a second parameter to the paginate or iterator function.

1await octokit.graphql.paginate(
2  `
3      query paginate($cursor: String, $organization: String!) {
4        repository(owner: $organization, name: "rest.js") {
5          issues(first: 10, after: $cursor) {
6            nodes {
7              title
8            }
9            pageInfo {
10              hasNextPage
11              endCursor
12            }
13          }
14        }
15      }
16    `,
17  {
18    organization: "octokit",
19  },
20);

You can also use this to pass a initial cursor value:

1await octokit.graphql.paginate(
2  `
3      query paginate($cursor: String, $organization: String!) {
4        repository(owner: $organization, name: "rest.js") {
5          issues(first: 10, after: $cursor) {
6            nodes {
7              title
8            }
9            pageInfo {
10              hasNextPage
11              endCursor
12            }
13          }
14        }
15      }
16    `,
17  {
18    organization: "octokit",
19    cursor: "initialValue",
20  },
21);

Pagination Direction

You can control the pagination direction by the properties defined in the pageInfo resource.

For a forward pagination, use:

1pageInfo {
2  hasNextPage
3  endCursor
4}

For a backwards pagination, use:

1pageInfo {
2  hasPreviousPage
3  startCursor
4}

If you provide all 4 properties in a pageInfo, the plugin will default to forward pagination.

Unsupported: Nested pagination

Nested pagination with GraphQL is complicated, so the following is not supported:

1await octokit.graphql.paginate((cursor) => {
2  const issuesCursor = cursor.create("issuesCursor");
3  const commentsCursor = cursor.create("issuesCursor");
4  return `{
5    repository(owner: "octokit", name: "rest.js") {
6      issues(first: 10, after: ${issuesCursor}) {
7        nodes {
8          title,
9          comments(first: 10, after: ${commentsCursor}) {
10            nodes: {
11              body
12            }
13            pageInfo {
14              hasNextPage
15              endCursor
16            }
17          }
18        }
19        pageInfo {
20          hasNextPage
21          endCursor
22        }
23      }
24    }
25  }`;
26});

There is a great video from GitHub Universe 2019 Advanced patterns for GitHub's GraphQL API by @ReaLoretta that goes into depth why this is so hard to achieve and patterns and ways around it.

TypeScript Support

You can type the response of the paginateGraphQL() and iterator() functions like this:

1await octokit.graphql.paginate<RepositoryIssueResponseType>((cursor) => {
2  return `{
3      repository(owner: "octokit", name: "rest.js") {
4        issues(first: 10, after: ${cursor.create()}) {
5          nodes {
6            title
7          }
8          pageInfo {
9            hasNextPage
10            endCursor
11          }
12        }
13      }
14    }`;
15});

You can utilize the PageInfoForward and PageInfoBackward-Interfaces exported from this library to construct your response-types:

1import { PageInfoForward } from "@octokit/plugin-paginate-graphql";
2
3type Issues = {
4  title: string;
5};
6
7type IssueResponseType = {
8  repository: {
9    issues: {
10      nodes: Issues[];
11      pageInfo: PageInfoForward;
12    };
13  };
14};
15
16// Response will be of type IssueResponseType
17const response = await octokit.graphql.paginate<IssueResponseType>((cursor) => {
18  return `{
19      repository(owner: "octokit", name: "rest.js") {
20        issues(first: 10, after: ${cursor.create()}) {
21          nodes {
22            title
23          }
24          pageInfo {
25            hasNextPage
26            endCursor
27          }
28        }
29      }
30    }`;
31});

The PageInfoBackward contains the properties hasPreviousPage and startCursor and can be used accordingly when doing backwards pagination.

Contributing

See CONTRIBUTING.md

License

MIT