The failure was assert 0 == 2. Three tests in tests/dvlaw/test_post.py, all green the day before. The only thing that had changed between the last green run and the failure was a flag value in the production database: auto_mode turned back on on 2026-09-17, a routine re-enablement after the duplicate-post fix shipped.
No code changed. The tests hadn’t been touched. The production change wasn’t touching the code path the tests were supposed to cover. The tests broke anyway.
An earlier post traced the causal chain: flag turned on, tests broke, tests weren’t testing the flag. This one names what the tests were actually doing wrong. The earlier article got the incident right. This one gets to the principle: the tests weren’t just testing the wrong thing. They had been reading the wrong database for months.
The setup
dvlaw is the article-pipeline subsystem for this brain. post_today is one of its core functions. It reads triage state and configuration, then decides what to post. It has a test suite. The suite uses a temp_db fixture that creates a clean SQLite database for each test run, populates the rows under test, and tears it down afterwards. Standard pattern.
The three failing tests each followed the same shape: create a temp_db, populate it with controlled data, call post_today, assert over the result. The fixture was correct. The populated data matched what the tests intended to verify. The assertions were consistent with the test’s intent.
What post_today does internally
post_today is not a pure function. Somewhere during its execution, it opens its own connection to the database to read the auto_mode flag. That connection is not a parameter the test controls. It goes to dvlaw.db by default. That is the operator’s real database, the production file.
The temp_db fixture creates a controlled database. It covers the arguments passed to post_today, but nothing beyond them. post_today’s internal connection is out of reach and bypasses the fixture entirely.
When production auto_mode was off, post_today read False from the real database. The tests, which expected behaviour consistent with auto_mode=False, passed. Not because the fixture was controlling the flag. It wasn’t. Production happened to agree with the tests’ implicit assumption.
This had been the case for months.
What hermeticity actually means
A hermetic test is commonly understood as a test that doesn’t depend on external state. The temp_db fixture is one implementation of that idea: give the function a controlled database, remove the dependency on the real database.
The fixture only controls what you route through it. The temp_db pattern assumes the function reads from the database it’s given. If the function opens its own connection (to a hard-coded path, a default argument, whatever resolves at runtime outside the test boundary), the fixture has a gap.
Hermeticity is a property of connections, not fixtures. The fixture controls one connection path; the internal open() call is another path entirely, uncontrolled. A hermetic test either removes or controls every path from the function to external state. These are different things, and the fixture pattern only addresses one of them.
The months of green tests
The breakage was informative. What it also revealed, retroactively, is that every green run before 2026-09-17 was a coincidence. The tests were passing not because they demonstrated that post_today behaved correctly under a controlled auto_mode value, but because the real database happened to contain the flag value the tests implicitly expected.
A coincidentally-passing test is worse than a failing test. A failing test tells you something is wrong. A coincidentally-passing test tells you nothing is wrong, and you believe it.
The coverage signal was false for months. The tests reported green while concealing an uncontrolled dependency on production state. The only reason it surfaced was that production state finally changed.
The fix
post_today is updated to accept the database path as a parameter rather than resolving it internally. Default: the standard production path, so existing call sites are unaffected. The temp_db fixture passes its own path. post_today now reads auto_mode from the fixture’s database.
Three tests. A handful of lines. Green.
Making the dependency a parameter is preferable to patching it at the test layer via monkeypatching. A parameter surfaces the connection in the function’s signature where it’s visible and can be reasoned about. A monkeypatch treats the symptom without changing the function’s contract.
What to look for
The pattern generalises beyond SQLite connections. Others in the same class:
- Reading from a config file at a hard-coded path
- Checking an environment variable with a non-test-safe default
- Accessing a module-level connection pool initialised at import time
- Calling
datetime.now()orrandom.random()internally without injection
In each case, a clean fixture can be constructed and the test will still have a hidden dependency on ambient state. The fixture covers what you route through it. The rest is ambient.
The signal: a test that fails immediately after an unrelated production change, with no code modifications, against an assertion value that matches live data the test shouldn’t be touching. That is coincidence expiring.
The principle
Hermeticity is a claim about connections, not fixtures. The fixture controls what you route through it, but an unparameterised internal connection routes around it. Both paths contribute to the test’s result; only one is under test control.
The temp_db pattern is right for what it does. It isn’t sufficient on its own when the function under test opens its own connections.
The post_today path in tests/dvlaw/test_post.py is closed now. The article pipeline suite has 599 passing tests. The question worth asking of any function that opens its own connections: how long has production been agreeing with the tests?
