A pre-publish style audit you can run

Chapter 7 of 7. Prerequisites: Chapter 3 and Chapter 5. You can hear the rhythm tells and you know what first-hand specifics look like. This closing chapter turns all of it into a gate that runs before every article ships, because catching tells by feel, in the words of the article this chapter is built on, is not a process. "It's a mood."

The pain this chapter closes: a writer shipping on "good enough" eventually ships a tell, and one detected tell taxes every article before and after it. That is a stance rather than a measurement, but it is the stance this whole course runs on: treat a generated-sounding paragraph as a trust problem, not a style note. More care during the read is not the fix. Chapter 4 showed a careful human read approving three over-budget articles in a single day. The fix is a check that runs the same way every time, and this chapter is that check, built from two published post-mortems on this site plus two incidents from the same toolchain in July.

How to write a technical blog that doesn't sound like AI

Stop trusting your read and start measuring the draft. Six chapters of this course named the tells: vocabulary in Chapter 2, rhythm in Chapter 3, structure in Chapter 4, missing experience in Chapter 5. Naming them is necessary and insufficient, because a human reading for content cannot simultaneously count surface patterns. The author put it plainly in the post-mortem of that incident, after approving three flawed articles himself: "A human can't reliably count em-dash density across a long-form article while reading for content."

The gate on this site is a shell script, audit-ai-tells.sh, introduced in the article that shipped it. Given a file path, it counts pattern instances against word count and exits non-zero when any threshold is breached. Output names the pattern, the count, the threshold, the word count and the offending lines:

em-dash density: 11.2 per 1k words (threshold: 3)
...
FAIL (2 patterns exceeded threshold)

The non-zero exit is wired into the publish command, so a breach blocks the ship rather than annotating it. Every run also logs to a SQLite events table with a timestamp and a verdict, and the trail matters more than it looks. It captures "the things that almost shipped as well as the things that did".

Why go this far for a blog? Google's helpful-content guidance is blunt about the incentive structure: "People-first content means content that's created primarily for people, and not to manipulate search engine rankings." The same page states the policy edge: "If you use automation, including AI-generation, to produce content for the primary purpose of manipulating search rankings, that's a violation of our spam policies." Google's dedicated generative-AI guidance draws the identical line, warning that "using generative AI tools or other similar tools to generate many pages without adding value for users may violate Google's spam policy". Prose that reads as generated raises exactly the value-for-users question those policies ask. The audit is how you ask it on record, before anyone else does.

Calibrate the thresholds against your own archive

The numbers do not come from a paper. The em-dash budget of 3 per 1,000 words came from counting this site's four published articles, all LLM-assisted, at 8 to 13 per 1,000, then counting pieces written with no model in the loop. Calibration was empirical: "The articles that felt clean passed; the ones that had needed post-draft editing flagged." The article calls its threshold a "working figure, not a derived constant", and that honesty is load-bearing. A number you calibrated is a number you can defend revising.

The first retroactive run earned its keep immediately. Two patterns surfaced in the already-published articles that repeated human re-reads had missed:

false continuity markers   ("of course", "naturally")  clustered in the second article,
                           three instances each in a ~900-word piece
transitional padding       ("in order to")              a run of instances, each
                           shortenable to "to"

Both had survived, in the article's words, "because they're quiet". Loud tells get caught on re-read. Quiet ones need a counter.

The script then audited its own announcement. The article describing it measured 6 em-dashes before the final pass, roughly 3.8 per 1,000. The script flagged its own launch post. The author cut six down to one, "keeping the one above because it earned the beat."

Two refusals are designed into the script, and both are worth copying. It does not auto-rewrite: "Automated rewrites on flagged sentences produce their own tells, often subtler ones, so the rewrite step stays manual by design." And it does not attempt semantic tells. A fabricated benchmark or an invented customer quote sails straight through a surface-pattern counter. Chapter 5's discipline covers that layer, and nothing in this chapter replaces it.

Never let a model count its own tells

In July this rule stopped being a preference and became doctrine. A multi-agent workflow was authoring a course for this site with three adversarial verifiers on every chapter. One verifier was an LLM voice checker carrying the exact budget in its instructions: at most 3 em-dashes per 1,000 words. It passed chapters 6 and 7 of that course. A deterministic recount after the workflow found chapter 6 at 6.61 per 1,000, more than double the budget the checker had just signed off, and chapter 7 over as well.

The recount is three lines. Strip the fenced code, then count the character against the prose word count:

prose = strip_code_fences(body)   # quoted source is exempt from the budget
words = len(prose.split())
density = prose.count("—") / (words / 1000)

The devlog's conclusion from that run is the sentence this whole chapter hangs on: "a next-token predictor can write to a budget it cannot measure, so a measurable property of AI output must be verified with a measurement, not with another AI." Chapter 6 made the drafting half of that argument: no prompt holds a density budget. The July run closed the checking half: no second model can enforce one either, because the checker shares the drafter's blindness to running totals.

The fix respected a boundary the comment in that snippet names. The over-budget chapters quoted a real .htaccess ErrorDocument block and a real deploy.yml verbatim, and some em-dashes lived inside those quotes. Those stayed byte-identical, because "those code-block em-dashes are the source, not style". Only prose em-dashes were restructured, landing the chapters at 2.6 and 2.0 per 1,000.

When the metric passes and the tell survives

Four days after the recount incident, the audit failed in the opposite direction. One chapter of the next course came back with a perfect em-dash score, 0.00 per 1,000. Auditors read it anyway and found fourteen trailing appositives at 6.6 per 1,000, all one shape:

Example of the migrated tell (not this course's prose):

The gate runs in CI, which is the safest place for it.
The token lives in the Keychain, which is the whole fix.
The check exits non-zero, which is what makes it a gate.

The em-dash cadence had been re-punctuated, not removed. The parenthetical pivot Chapter 3 trained your ear on was fully intact; only the glyph had changed, so the glyph counter read clean. A deterministic recount cannot see a tell that no longer contains the character it counts. Chapter 2 warned that tells migrate and that a vocabulary list is a snapshot of the model's current habits. This is what the migration looks like in production: ban one punctuation mark and the same cadence resurfaces wearing another.

That incident forced a second layer into the audit. The counters stay, because they are the only honest arbiter for anything countable. On top of them sits a cadence read: a pass over the prose listening for the pivot rhythm regardless of punctuation. The deterministic side then grew to match what the ear found. Later runs added a cadence-tails counter, and a chapter now has to land at 0 prose em-dashes per 1,000 with at most one trailing-appositive tail.

A check that can only pass is checking nothing

How does an audit stay honest for months? This site's diff-check post-mortem supplies the frame. A guard test named test_verify_py_not_modified_by_wiring_pr existed to prove a wiring PR left a verification script untouched. The moment PR #43 squash-merged on 12 June, the two things it compared became permanently identical and the test became trivially true. "The test was checking nothing… A green result that reported nothing true and prevented something useful. That is the shape of a shelf-life test."

Style audits inherit the shape exactly. A tells list written once and never re-audited is a shelf-life check: the model's habits drift underneath it, the budget keeps passing, and the prose reads as AI behind a green dashboard. The appositive incident above is that drift caught live. The post-mortem's remedies transpose directly:

1. Pre-register the pass criteria. "A criterion registered after you have seen the result is a description. A criterion registered before is a constraint." Set the thresholds in the style guide before the draft exists, not after you have seen what the draft needs forgiven.
2. Sabotage-test the gate. Feed the script a draft deliberately salted with tells and confirm it fails. "A gate whose sabotage cycle has never run has not been tested; it has been installed and assumed."
3. Judge the audit by its reds. The measure "is not whether the gates are green; it is whether they can go red." An events table with no FAIL ever recorded is evidence about the audit, not about the prose.

The audit, end to end

1. Pre-register the thresholds in a canonical file

   A repo-root style guide, one numbered entry per tell: the pattern, the mechanism that makes it read as generated, the threshold, the fix. On this site the em-dash guidance started as an inline note in a drafting doc and was invisible at publish time. "Moving it into a canonical reference that the toolchain can point at is the difference between a note-to-self and a constraint."

2. Run the deterministic counters at article scope

   Whole-document scope, never per paragraph. Chapter 4 documented the proxy failure: a dozen paragraphs at two em-dashes each never trips a per-paragraph flag while the article lands far over budget. Strip fenced code first. Non-zero exit blocks the publish command.

3. Do a cadence read for migrated tells

   Read the prose for the pivot rhythm from Chapter 3 while ignoring punctuation. When a familiar cadence shows up at density under a new glyph, add a counter for the new form and log the migration. The counter set is a living file, not a finished one.

4. Fix flagged lines by hand

   The script prints the offending lines; rewrite them yourself. An automated rewrite trades a counted tell for an uncounted one. For each em-dash, apply the style guide's actual remedy: check whether the aside is load-bearing. Cut it if not. If it is, rewrite so the aside becomes the main clause.

5. Log every run, sabotage on a schedule

   Timestamp and verdict into an events table, passes and failures alike. Periodically push a salted known-bad draft through and confirm a FAIL. A gate with no recorded red has never been tested.

Nothing above came out of a planning document. The audit accreted, one incident at a time. The anomalous density was flagged by the author's writing advisor on 22 May, and a manual count found 8 to 13 per 1,000 across everything published. The style guide and the script shipped on 28 May. Three articles cleared a proxy-scoped gate and went live over budget on 11 and 12 June, so the gate moved to article scope. The shelf-life doctrine landed on 28 June after a guard test went permanently green. Between 9 and 14 July, LLM checkers were proven unable to count, a deterministic recount became the arbiter, and then re-punctuation gamed the count and the cadence layer went in. Each incident hardened the audit. None of them ended it.

One honest limit from the original article belongs in yours too: the four articles that started all this were never retroactively edited. "Retroactive edits to published pieces create a different kind of trust problem. The record stops being reliable." The audit gates what ships next; it does not launder what already shipped.

Where does that leave the course? Back at credibility, with receipts. The helpful-content page carries one more verbatim line worth acting on: "Sharing details about the processes involved can help readers and visitors better understand any unique and useful role automation may have served." A logged audit trail is that disclosure in its strongest form: not a badge claiming the prose is human, but a record of what was measured and rewritten before you ever saw it. This chapter was drafted with a model in the loop, held to the budget it describes, and counted before it shipped. That is the whole method. Count what can be counted, and read for what cannot. The log does the rest.