The 2026-09-17 production change was routine: turn auto_mode back on. The next test run was not. Three failures in tests/dvlaw/test_post.py, all returning assert 0 == 2. Nothing in the codebase had changed between the previous green run and this one.
The instinct is to go looking for a regression. There isn’t one.
The split
post_today accepts a database connection as a parameter. The three failing tests supply a hermetic in-memory fixture, a controlled SQLite database with auto_mode = 0. Standard practice: inject the dependency, control the state, verify the behaviour.
But post_today also reads auto_mode. And it reads it from a connection it opens itself, not the parameter the caller passes in.
So the test was running with two active database connections simultaneously. The fixture was one: controlled, hermetic, auto_mode = 0. The second was post_today’s own connection, opened independently against the operator’s real dvlaw.db. The live value.
Both readings agreed while auto_mode sat at False in production. Tests passed. Once it flipped to True, the live read returned True and the tests collapsed. Expected zero messages posted, got two.
No regression. The fixture was working. The function had reached past it.
What the fixture controls
A test fixture controls what you give it. No more.
Pass a connection parameter and the fixture owns that connection for the test. Every database access that flows through that parameter is under test control. The fixture’s state is what the function sees.
This pattern assumes that all database access flows through the parameter. Fixture isolation is complete while that assumption holds. Once the function also opens connections independently, whether by reading a path resolved at call time, calling a helper that reaches into the environment, or constructing its own connection against an os.environ path, the fixture no longer covers those reads.
post_today violated the assumption in exactly that way. The parameter connection was covered. The auto_mode read was not.
This is not an argument against fixture-based testing. It’s an observation about what fixtures cover.
Finding the gap
The question to ask when writing a fixture test is not “what do I need to supply?” The more useful question is: what connections does this code path open, wherever they come from?
Parameter injection is the obvious answer and the one the fixture covers. Beyond that:
- Module-level globals that resolve a path
os.environ.get(...)calls that produce a db path- Helper functions that open their own connections
- Configuration reads from a second table in a second file
- Singleton-pattern connection managers the function reaches into directly
Each of these is a connection the test doesn’t control unless it explicitly patches the resolution. A function that accepts a connection parameter and also calls any of the above has split hermeticity: partially controlled by the fixture, partially determined by the runtime environment.
The symptom is consistent: tests that pass when production configuration matches the test’s implicit assumptions, and fail when production diverges. The failure is config-triggered rather than code-triggered, which is why the first instinct (regression) is wrong.
Why it stays invisible
The test suite reports green. Every line of the function is exercised. All declared test inputs are under control. By every normal signal, coverage is adequate.
The gap is invisible because passing the fixture isn’t the same as covering all the function’s state reads. Test suites track line coverage or branch coverage; the test runs those lines. A line that reads from an independent connection is “covered” in the coverage sense while remaining uncontrolled in the hermeticity sense. Coverage tells you the code ran. It doesn’t tell you the code ran with controlled inputs.
For post_today, every line in the function executed during the test. The auto_mode read happened. The connection it read from was just the wrong one.
Production configuration diverging from the test’s unspoken assumptions is the only signal you get. The tests are not obviously broken until that happens; they pass. Only an environment shift surfaces the hermeticity hole.
The fix
For post_today, the fix was ensuring the auto_mode read resolved to the test fixture’s database, not the operator’s live one. The specific mechanism depends on how the independent connection is opened.
If the function reads from an environment variable to get the db path, the test sets that variable to point at the test fixture. If the function calls a module-level helper that resolves the path, the test patches that helper. If the function directly constructs a path from a known constant, the test patches the constant.
In each case: trace how the independent connection resolves to a file path, then intercept that resolution in the test. Nothing needs to change in the production code. There’s nothing wrong with a function opening a second connection. What has to change is the test’s ability to cover it.
Once both connections resolve to the fixture, the test is hermetic. The production state of dvlaw.db is irrelevant to the test outcome.
Hermeticity at the connection level
“Hermetic” is usually applied at the level of the test: a hermetic test doesn’t depend on external state. But the mechanism is connection-level.
A test is hermetic when every connection the code path opens is under test control for the duration of the run. Not only the connections the test explicitly supplies. Every connection. The function has to be unable to reach outside the test’s controlled environment for any of its reads.
post_today had two connections. The test controlled one. That left the test non-hermetic, not because the fixture was wrong, but because the coverage was incomplete.
The production change made it visible. That’s the legitimate diagnostic role of a production config change against a test suite: if the suite is hermetic, production state is irrelevant to it. If a production change breaks a test, the test had an uncontrolled dependency. Here, the dependency was a database connection.
What to look for in review
The pattern to flag: a function that accepts a connection parameter and also calls any helper that could independently resolve a database path. Together those are the signature of split hermeticity. The risk isn’t visible in the function signature alone; you have to trace the helpers.
In practice this shows up most often in functions that take a primary db connection for writes but read configuration or feature flags from a secondary connection managed elsewhere. The write path is covered by the test. The config read is not.
The test suite will catch it the first time production configuration diverges from the test’s implicit assumptions. By then the tests have been falsely green for however long the configuration happened to agree. For post_today, that was as long as auto_mode had been False.
Three tests, assert 0 == 2, fixed by making the second connection hermetic. Straightforward once the mechanism was clear.
