The API as a data source

Every earlier chapter treated GitHub as somewhere work happens: repos hold code, PRs gate changes, Actions run jobs. This chapter treats it as something else entirely, a database you already have. Every repo, every commit, every language byte-count is a row you can query with gh api, and if you've been running your shipping operation on GitHub for any length of time, that database has more signal in it than most people realise. The homepage feature that reads your own repos' language stats and the inventory sweep that reads their trees and READMEs are the same move applied twice: point an API call at your own history and let it answer a question you'd otherwise answer by memory or by guessing.

GitHub as a database you already have

The habit worth building here is small: before you reach for a spreadsheet, a manual count, or "I think it's roughly X repos", ask whether the answer already lives in the API. It usually does. Repo lists, language breakdowns, commit counts, file trees, workflow run histories: all of it is one gh api call away, and none of it requires you to maintain a separate source of truth. The repo is already the system of record (chapter 1); the API is how you read it back out as structured data instead of as a page you scroll.

This matters more on a personal operation than it sounds like it should. A team with a data warehouse and a BI tool would never think to query GitHub directly for a homepage stat. A solo developer running forty-odd repos doesn't have a warehouse, and building one to answer "what language have I actually written the most of this year" would be absurd. The API is the warehouse. You just have to treat it as one.

The endpoint: repo languages, in bytes

The concrete example is a homepage feature on this site called Workshop XP: a ranked list of languages, driven entirely by real repo data rather than a hand-maintained skills list. The data source is one endpoint, documented plainly:

GET /repos/{owner}/{repo}/languages

GitHub's own docs describe the response as an object mapping each language name to "the number of bytes of code written in that language." Not lines, not files, not commits: bytes. That single fact shapes everything downstream. A verbose language inflates its own byte-count relative to a terse one for the same amount of actual logic, so any feature built on this endpoint is measuring code mass, not code quality, and it should say so or correct for it rather than pretend otherwise.

Called against one repo, it looks like this:

gh api repos/CaptainRandom00/captainrandom.co.uk/languages

{
  "MDX": 2509206,
  "TypeScript": 459821,
  "CSS": 149009,
  "JavaScript": 79592,
  "PHP": 57823,
  "Shell": 11541,
  "HTML": 6006,
  "Python": 4392,
  "Dockerfile": 3134
}

Nine languages, nine byte-counts, nothing else. Call it across every repo an account owns and you have raw material for a real feature.

Turning bytes into something a homepage can render

fetch-workshop-xp.mjs, the script behind the Workshop section, does exactly that: it walks every repo under one account with gh repo list, calls /languages on each, and sums the byte-counts by language across the whole account. Two decisions in that script are worth calling out because they're the difference between a number and a claim you'd defend.

First, it excludes a fixed set of languages before ranking: HTML, CSS, SCSS, Dockerfile, Makefile, Batchfile, PowerShell. These are structural or generated noise. HTML and CSS get emitted by frameworks regardless of what you actually chose to learn; a Dockerfile is thirty lines that say nothing about skill depth. Filtering them out before summing means the byte-share weighting later in the pipeline is computed over "real tech" only, not diluted by markup the tooling wrote for you.

Second, because bytes aren't lines and a stat panel reading "184,213 bytes of TypeScript" means nothing to a human, the script documents an explicit conversion constant right at the top of the file:

/** Bytes-per-line approximation. GitHub's /languages returns byte-counts,
 *  not LOC; this ratio converts. 35 chars/line is a typical source-code
 *  average across the languages we deal with. */
const BYTES_PER_LINE = 35

That's an approximation, named as one, sitting next to the code that uses it. Anyone reading the script six months later doesn't have to reverse-engineer where the "lines of code" number on the homepage came from; the comment says it plainly. That's the standard worth holding yourself to whenever you convert a raw API number into a display number: write down the conversion, don't bury it.

Commits as a second dataset: the pagination trick

Bytes tell you what's in a repo. They don't tell you how much work went into it. For that, fetch-workshop-xp.mjs also pulls total commit counts, and it does it in a way that's worth stealing regardless of what you're building: instead of paging through every commit to count them, it asks for one commit and reads the answer out of a response header.

gh api "repos/CaptainRandom00/captainrandom.co.uk/commits?per_page=1" -i

The -i flag includes response headers. Buried in them, when a repo has more than one commit, is a Link header with a rel="last" entry pointing at the final page:

Link: <https://api.github.com/repositories/123/commits?per_page=1&page=1847>; rel="last"

Page 1847 at one commit per page means 1,847 commits, discovered without fetching a single one of them. Pagination metadata, meant to help you walk a result set, repurposed as a counting shortcut. It's a small trick but it changes the cost of the operation from "one request per commit" to "one request per repo", which is the difference between a script that runs in seconds and one that burns your rate limit finding out how many commits you've made.

Attribution without lying: weighting commits by byte share

Once you have both datasets (bytes per language, commits per repo), the naive move is to attribute all of a repo's commits to its "primary language" and call it done. The script deliberately doesn't do that, because it produces a specific, documented artefact: a repo that's 60% TypeScript, 30% Python, 10% MDX by bytes would, under primary-only attribution, credit 100% of its commits to TypeScript and none to Python or MDX. Any language that's never a repo's single largest language accumulates a permanent zero commit count across the entire account, which is false: work happened in that language, it just never happened to be the plurality in any one repo.

The fix is to weight: a repo's commits get split across its languages in exactly the proportion of its byte-share.

// Attribute the repo's commit history across its languages weighted
// by byte share. So a repo with 60% TS / 30% Python / 10% MDX bytes
// contributes 60% / 30% / 10% of its commits to those languages.

It's not a perfect model of where effort went; a one-line MDX typo fix and a two-hundred-line TypeScript refactor are still both "one commit" under this scheme. But it's honest about what it's approximating, and it avoids the specific lie that primary-only attribution tells. When you build a derived stat from two API datasets, ask the same question this script answers: what artefact does the naive combination produce, and is it one you're willing to publish.

Auth, rate limits, and failing open

None of this works without authentication, and the limits are exact and worth knowing before you build anything that fans out across dozens of repos. GitHub documents authenticated REST requests as counting toward "your personal rate limit of 5,000 requests per hour", while unauthenticated requests are capped at "60 requests per hour." Sixty an hour is nothing: a repo-list call plus a /languages call plus a commit-count call across even ten repos would exhaust it before you'd looked at a third of your account. Authentication isn't optional for this pattern, it's the entire ballgame.

fetch-workshop-xp.mjs handles auth as a documented split rather than an afterthought:

/**
 * Auth:
 *   - Local: uses the `gh` CLI (already authenticated as the account owner).
 *   - CI:    set GH_LOC_TOKEN (PAT with read access to the listed repos).
 */

Locally, gh is already logged in as the account owner, so calls ride that session. In CI, the default GITHUB_TOKEN every workflow gets can't read other private repos on the same account, so the script needs a classic PAT (GH_LOC_TOKEN) with the repo scope, set as a repository secret and never written to a file. That split is the same lesson from chapter 7's fallback token problem, applied here to reads instead of pushes: the default token is scoped to the one repo running the workflow, and any script that needs to see other repos needs a token that says so explicitly.

The other detail worth copying is what happens when the repo-list call itself fails, the very first step in the pipeline:

try {
  repos = JSON.parse(gh(['repo', 'list', ACCOUNT, '--limit', '200', '--json', 'name,primaryLanguage']))
} catch (e) {
  console.error(`[workshop-xp] failed to list repos: ${e.message}`)
  process.exit(0) // fail-open: keep whatever JSON exists
}

It exits 0, not 1. A failed data-refresh doesn't fail the build; it leaves the previously-generated JSON in place and lets a stale homepage stat ship rather than a broken one. That's a deliberate trade, not an oversight: a wrong-but-plausible number that's a day old is a better failure mode for a decorative homepage feature than a build that won't ship at all over a transient API blip.

The build-time-fetch pattern

Step back from the specifics and there's a repeatable shape here, and it's the same one this site's static-export architecture forces on everything else: generate JSON at build time (or on a schedule, per chapter 7), commit or cache the result, render statically. fetch-workshop-xp.mjs runs as a prebuild hook so a production build gets fresh numbers, and the generated JSON is also committed to the repo, so a build that runs without API access (a fresh clone, a CI outage) still ships with recent-ish data rather than nothing. Fresh when it can be, stale rather than broken when it can't.

That pattern generalises past language stats. Anything you'd otherwise hardcode, a project list, a "repos I'm active in" widget, a stat about how many PRs shipped this month, can be generated the same way: a script that calls the API, writes a JSON file into src/data/, and a component that renders that file at build time. No API route, no client-side fetch, no runtime dependency on GitHub being up when a visitor loads the page. It fits a statically-exported site exactly because the fetch happens once, at build, and the output is a plain file the static export already knows how to bundle.

Project-inventory sweeps: reading trees and READMEs

The same pattern shows up outside of homepage features. This week's project-inventory sweep across the account's repos used the identical shape for a different question: not "what languages have I written" but "what state is each repo actually in." A gh api repos/{owner}/{repo}/contents/README.md call per repo pulls back the README (base64-encoded; decode before you read it), and a gh api repos/{owner}/{repo}/git/trees/{sha}?recursive=true call pulls back the full file tree without cloning anything. Across forty repos, that's forty small API calls that answer "does this have a README, does it have tests, is there a docs/roadmap.md" without opening a single one of them in an editor.

It's the same three-step shape every time: point an API call at your own account, shape the response into the question you're actually asking, decide what happens when a call fails. The workshop XP feature and the inventory sweep are the same tool used twice.