Day 3, 08:21 BST. The cron fires clean, produces one draft. It's the same mortgage rate-spike cluster published the day before: same BBC, Zoopla, Moneyfacts, Bank of England ground; same US-Iran framing; same £232 and £66 regional figures. A repetition, not a new article.
Day 4: zero drafts. One hundred and fifty clusters ranked; mortgage scores 20 at the top; candidates fetched. Nothing comes out.
Both facts trace to the same gap. It took ten cycles to close the diagnosis.
The assumption that broke the cadence
The zombie IHT draft and the growing dead zone shared one underlying assumption: that any presence in the index signals completion. It doesn't. "Attempted" and "shipped" are different terminal states. "Attempted and failed" isn't a terminal state at all. It's a retry position.
draft.py could not tell them apart. Every entry in the index, including entries prefixed failed-*, was treated as permanently consumed. That single equivalence broke a 14-day cadence in the ways described below.
The 14-day clock
The streak started 1 July. Days 1 and 2 ran clean. By day 3, the highest-ranked cluster was one already published. By day 4, the top candidate had no path to a draft despite scoring 20 and reaching the ground-fetch stage.
On 7–9 July, three consecutive weekdays produced zero publishes. 7 July alone: eight clusters triaged, candidates fetched, zero ready articles. The cron exited zero and the pipeline completed its cycle cleanly. The log showed nothing worth flagging. No exception, no exit code, no flag to indicate the working set had contracted to the point where drafting was structurally impossible.
The cadence clock stopped. It restarted on 10 July with two hand-published articles: a stamp-duty cron draft rescued from the queue, a Lifetime ISA pillar pushed directly. Operator publishes. The counter reset; the failure mode didn't.
Across ten cycles from 30 June to 13 July, one ready draft emerged. Zero auto-publishes. The skipped counter went from 2 to 7. The schedule held. The pipeline was consuming its own eligible pool.
The zombie
On 6 July, the long-slug IHT file was on its third PR. First: squash-and-delete within the same pipeline run. Second: merged live. Third: the pipeline regenerating an article that already existed in production.
The regeneration source, whatever was feeding the IHT topic back into candidates each cycle, hadn't been root-caused. The index had no record of "shipped and merged" in a form the drafting stage could act on. The cluster kept arriving as a candidate. The drafting process kept running.
Two PRs wasted. One duplicate briefly live before cleanup. The pipeline had no memory of the completed state that would have stopped it.
The dead zone
Seven clusters had accumulated permanent skipped status. Each had failed to produce a draft on some earlier cycle: bad ground, failed verification, empty candidate set. draft.py had written each into the index. On every subsequent cycle, the entry was found and the cluster was skipped.
Not because the articles existed. Because the drafting attempts had.
The root cause, confirmed on 13 July: draft.py treated any index entry, including failed-* entries, as permanently consumed. A cluster that failed to draft was indexed identically to a cluster whose article had shipped. Both were "seen." Both were skipped on every cycle thereafter.
Seven clusters. Never retried. Their articles never written.
The index had one field
"Shipped" and "failed last time" require opposite treatment on the next cycle.
A shipped cluster exits the queue permanently. The topic is exhausted for this ground set. Permanent skip: correct.
A failed cluster re-enters the queue on the next cycle. The failure may have been transient: a sourcing timeout, a verification that would clear on retry, fresher ground now available. Permanent skip: incorrect.
draft.py's index encoded one bit: seen or not seen. The skip logic ran identically on both states, even though attempted, shipped, and failed are three distinct situations. The index represented two of them.
The same gap in both directions
The zombie over-retried because the index lacked the signal "shipped to production." The dead zone under-retried because the index lacked the signal "eligible for retry." Both errors exist because the index tracked presence, not meaning.
Adding the production-merge signal fixes the zombie. Adding the retry-eligible state fixes the dead zone. One schema change covers both directions, because the index needs to encode why an entry is there, not just that it is.
What a recurring pipeline requires
A content pipeline on a daily schedule is a retry system with a graduation condition. Clusters cycle through until one of two things happens: a shipped article, or an explicit decision to retire the cluster. "Was attempted" is neither.
The working set each morning should contain clusters never attempted and clusters that failed and remain eligible. It should exclude clusters whose articles are live. It should not exclude clusters that failed yesterday and might succeed today.
By 13 July, the working set was contracting by one entry every time a cluster failed. Ten cycles. Seven contractions. The eligible pool was smaller than at the start of the streak, with no corresponding growth in the published count.
The cron fires on schedule. The log looks clean. Cadence dies anyway.
The fix
The index needs three states: not attempted, attempted-and-failed, and shipped. Possibly a fourth if a cluster should exit after N failed retries rather than retrying indefinitely.
The cron schedule doesn't change. The drafting logic doesn't change. The index stops treating a failed attempt as evidence of completion.
The seven clusters in permanent skipped need manual triage before the queue is meaningful again. Some may have shipped via other routes. A few will be genuinely unviable. Others belong back in the working set. That's the backlog ten cycles of silent failure built.
Regularity is a scheduling property. Reliability is a data model property. The cron had one; the pipeline needed both.
