Querying Google Search Console

Chapter 6 of 8. Prerequisite: Chapter 5 — you should have a telemetry/auth.py returning a Credentials object and telemetry/gsc.py with subcommand registrations declared. This chapter fills in the bodies of the GSC subcommands.

Three endpoints, one library

The Search Console API surfaces enough functionality to support both an indexation dashboard and a cannibalisation detector — both of which I have built and shipped against this exact surface. The endpoints you need, all documented under developers.google.com/webmaster-tools:

• Sites — list the properties your authenticated account has access to. Read-only confirmation step; rarely the headline call but useful for sanity checks.
• URL Inspection — per-URL state. Returns the bucket Google has assigned the URL, the date of the last crawl, the canonical Google has chosen, and so on. This is the endpoint behind the Index Coverage report.
• Search Analytics — query-and-page-level performance data. Impressions, clicks, CTR, position, broken down by query, page, country, device, search appearance.

All three are exposed via the searchconsole service built by the google-api-python-client:

from googleapiclient.discovery import build
service = build("searchconsole", "v1", credentials=creds)

Sites — the sanity check

def cmd_sites(args: argparse.Namespace, creds: Credentials) -> int:
    """List GSC properties the authenticated account has access to."""
    service = build("searchconsole", "v1", credentials=creds)
    response = service.sites().list().execute()
    for entry in response.get("siteEntry", []):
        print(f"{entry['permissionLevel']:>12}  {entry['siteUrl']}")
    return 0

This is the call to make when you first wire the CLI up. If it returns an empty list, the OAuth client is authenticated but the account does not have any GSC properties. If it returns a property you expected but with permissionLevel: 'siteUnverifiedUser', the verification has lapsed and other endpoints will return 403s.

URL Inspection — what state is this URL in?

The URL Inspection endpoint takes one URL at a time and returns the indexation state. There is no batch variant; if you want to inspect twenty URLs you make twenty calls.

def cmd_indexation(args: argparse.Namespace, creds: Credentials) -> int:
    """Pull index-coverage state for every URL in a sitemap.

    Walks a sitemap URL, inspects each entry, groups by verdict, prints
    a one-line summary per bucket.  Detail in the inline comments."""
    service = build("searchconsole", "v1", credentials=creds)
    urls = _fetch_sitemap(args.sitemap)

    buckets: dict[str, list[str]] = {}
    for url in urls:
        body = {
            "inspectionUrl": url,
            "siteUrl": args.property,
        }
        response = service.urlInspection().index().inspect(body=body).execute()
        # The verdict field maps directly to the Index Coverage UI buckets
        # listed in the GSC help article.  See the bucket taxonomy callout below.
        verdict = response["inspectionResult"]["indexStatusResult"]["verdict"]
        buckets.setdefault(verdict, []).append(url)

    for verdict, items in sorted(buckets.items()):
        print(f"{verdict:>20}  {len(items):>4}")
    return 0

The _fetch_sitemap helper does an HTTP GET on the sitemap URL and parses out the <loc> entries. I am keeping that out-of-band of this chapter because it is incidental; the GSC-specific work is the inspect call.

Search Analytics — what queries are landing where?

The Search Analytics endpoint is where the cannibalisation detector lives. The shape is one HTTP call with a query body specifying the date range, dimensions, and row limit.

def cmd_queries(args: argparse.Namespace, creds: Credentials) -> int:
    """Pull top queries for the property over a date window."""
    service = build("searchconsole", "v1", credentials=creds)
    end = date.today()
    start = end - timedelta(days=args.days)

    body = {
        "startDate": start.isoformat(),
        "endDate": end.isoformat(),
        "dimensions": ["query"],
        "rowLimit": 1000,
    }
    response = service.searchanalytics().query(siteUrl=args.property, body=body).execute()

    for row in response.get("rows", []):
        query = row["keys"][0]
        clicks = row["clicks"]
        impressions = row["impressions"]
        ctr = row["ctr"]
        position = row["position"]
        print(f"{clicks:>5}  {impressions:>7}  {ctr:>6.2%}  {position:>5.1f}  {query}")
    return 0

For cannibalisation detection, the change is one line:

body["dimensions"] = ["query", "page"]

Now each row carries both the query and the URL it matched. Group by keys[0] (the query) in your client code and count distinct keys[1] (the page). Any query landing on two or more pages is a cannibalisation candidate. The cannibalisation article walks through what you do with that list once you have it.

Quota — what counts and what does not

The GSC API has a per-project per-day quota cap. The exact number changes; check the current value in the API reference. The practical number to remember is that you can comfortably support:

• A daily indexation pull across ≤200 URLs per property.
• A daily Search Analytics pull with rowLimit: 1000 across ≤10 properties.
• A few ad-hoc CLI invocations on top.

Where you run out of quota first is the URL Inspection endpoint, because every URL is a separate call. The mitigation is to inspect only URLs that changed — diff the sitemap day-over-day, inspect only the new entries, and rely on yesterday's bucket assignment for everything else. That cuts the daily inspect count from len(sitemap) to len(new_urls).

What this gives you

With sites, indexation, and queries wired, you have most of what the Index Coverage report and the Performance report surface in the UI — but as data, in your CLI, run from a cron. Chapter 7 does the same for GA4. Chapter 8 wraps both into a daily digest the Telegram bot DMs you.

The two articles linked above are the production case studies — both used exactly this CLI shape and exactly these three endpoints to surface signal the UI was burying under clicks.