My Pagination Art

I've have been spending some time using and reading github api through their official JavaScript SDK octokit.

One interesting thing I stumbled upon today was their .paginate.iterator api (implementation), wherein you can provide this function any other listing api’s fetching function ( say listCommits ), and provide options to tell how many to fetch in one time ( and optionally takes a page ).

The nice thing about implementing this iterator function is that we can just use a normal for await loop and it’ll keep fetching next paginated page’s data. Something like

const mainCommits: Endpoints["GET /repos/{owner}/{repo}/commits"]["response"]["data"] = []
const mainCommitsIterator = octokit.paginate.iterator(octokitRest.repos.listCommits, {
  ...githubDetails,
  sha: mainCommitSHA,
  per_page: 30, // Set the number of commits per page
});

for await (const { data: commits } of mainCommitsIterator) {
  mainCommits.push(...commits);
  if (mainCommits.length >= 150) break;
}

This provided me a very easy option to ‘not fetch after 150 commits have been fetched’ functionality by just breaking the for loop. Internally any loop structure which supports iterating over iterators, calls the iterator’s next function internally, automatically. More on JavaScript iterators