The guard checked whether the installed SDK satisfied the pin. Nobody had
ever checked whether anyone had READ it. Those are different questions, and
the gap between them was a whole version range: pinned >=0.2.111,<0.3,
premises source-verified through 0.2.110, installed 0.2.120. Every build in
between was admissible and unexamined — `uv sync --upgrade` would have kept
806 tests green on an SDK no one had opened. Written red first: a guard
handed 0.2.140 returned it without complaint.
_VERIFIED_THROUGH is the ratchet. It records the newest build actually read
at source, and a newer one fails naming the five premises to re-check. The
pin is untouched and was never the defect — measurement dissolved the
premise that it needed lifting. It was not too narrow but too wide, and a
wider permission is not repaired by widening it further.
The premises themselves were prose the failure message recited. Nothing
tested them, so one that stopped being true would have surfaced on the one
live paid run (S10, D6). They are now a table introspected against the
installed package, with the printed prose derived from that same table so a
checked attribute cannot go unreported or a reported one unchecked. The
premise introspection structurally cannot see — that query() yields an
AssistantMessage then a closing ResultMessage — is named apart, and is the
honest reason the human reading still has to happen.
Value-proved, not merely named: disabling the ratchet reds 1 test, stubbing
the inventory to "no gaps" reds 3, re-hardcoding the prose reds 1, and
lowering _VERIFIED_THROUGH below the installed build reds the real
installed-version test rather than only a monkeypatched one.
0.2.139 read at source (0.2.120 -> 0.2.139, latest on PyPI today; STATE said
0.2.134, measured 08-09 and stale). The public query.py is byte-identical,
every premise field keeps its type and default, and the parser changes are
additive. One needed a look: 0.2.139 added a skills path defaulting
setting_sources to ["user", "project"], which would have undone the S10
isolation fix — it fires only on None, so the explicit [] is out of reach.
Prose carrying stale version claims moved with the reading, never ahead of
it: each was re-verified at 0.2.139 before being restated.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014dKDjVG7qrBh9NkAAxutqN
Point 2 of the sweep, enumerated rather than assumed. STATE's total was right and
its distribution was not: 86 hits confirmed (`assert not X` 41 / `== []` 42 /
`== {}` 2 / `== set()` 1), but per file measured `test_cli_paritet` 13 (STATE said
19), `test_preflight` 10 (11), `test_step7` 4 (6).
AST triage split the 86: 55 hits sit in 50 tests whose assertions are ALL
negative; the other 31 already have a positive sibling assert in the same test.
Two negative results worth recording, because they bound the remaining work:
- The `test_preflight` "clears" family (`_check_credentials(...) == []` and
friends) is NOT vacuous. Each sits beside a sibling in the same class that
asserts refusals are non-empty, so a no-op checker turns the sibling red.
Class-level pairing is a real control; these need no change.
- `test_method_spec_loadbearing.py` already models the right pattern for
detectors — explicit `test_guard_red_when_*` red-proofs against a mutated COPY.
This commit fixes the class that had no control at all: static/AST guards that
assert an absence without ever showing the scanner can detect a presence.
1. TAUTOLOGICAL RED-PROOFS (both spec guards). `test_guard_red_when_spec_missing`
asserted a file is absent from a fresh `tmp_path` — true by construction of the
fixture, and it never called the guard it is named for. It would have stayed
green with `test_spec_is_present` deleted outright. Both now exercise the same
`_spec_is_present` predicate the guard calls, in both directions.
2. MISSING RED-PROOF. `test_spec_keeps_structure_markers` had none, unlike its
toolkit and contract-field siblings: with `_STRUCTURE_MARKERS` emptied or
`_missing_markers` stubbed to `[]` it reported green forever. Added
`test_guard_red_when_marker_removed`, parametrized over all 21 markers.
3. BLIND IMPORT SCANNERS (costsim x2, okf, preflight, notify). Every one asserted
`not names & {forbidden}` or `outside == set()` with nothing showing `names`
was non-empty — an empty scan satisfies them exactly as well as real purity.
`test_okf_is_pure_stdlib`'s subset check is likewise trivially true of the
empty set, so it did not guard its neighbour either. Each now asserts a
known-present module first. The notify guard gets the strongest form
available: it proves the detector DOES match a network import inside the seam,
so the matcher itself is shown to work rather than only its silence.
Value-proved, not merely detach-proved. Seven vacuity mutations run against the
NEW tests: all seven RED, each dying on the intended control line. The same
mutations run against the PRE-CHANGE tests (session edits stashed): all five
applicable ones GREEN — blind to the vacuity they were meant to catch. Green
before, red after, same mutation, is the value-proof.
Harness held original bytes in memory, restored in `finally`, sha256-verified
every restore, and checked each run ACTUALLY RAN (a wrong test id yields rc!=0
and mimics red). `git status` clean before and after.
Remaining in the class and NOT closed here: ~45 all-negative tests, mostly CLI
refusal (`calls == []` after a refused invocation) and empty-default
(`missing dir -> []`). Listed in STATE, not silently dropped.
Suite 690 -> 711.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DJmse16bEkaSBtvXhncEUc
Oekt 17 found the class on four named files. This sweep ENUMERATES it: 42 negative
substring assertions across 21 test files (STATE's "~34 across 23" was a premise --
measured, it is 42/21). Sixteen of them measured an absence without ever having
shown presence; all sixteen now carry a positive control asserting the searched-for
string PRESENT in the source artifact, in EXACTLY the form the negative looks for.
Files touched: test_costsim, test_loop, test_okf (3 sites), test_preflight,
test_run_entrance, test_s10_run_layer, test_sdk_version_guard, test_simulation
(2 sites), test_step1_expel, test_step5_refine, test_step7_async_loop,
test_step8_promotion, test_valuereport.
VALUE-PROOF (green-without / red-with, per the oekt-17 rule that a detach proof is
not a value proof). Seven source/fixture mutations, each making the negative vacuous:
M1 verdict fixture loses the realization signal VALUE-PROVEN
M2 decoy fixture loses its text VALUE-PROVEN
M3 renderer stops emitting typed section headings VALUE-PROVEN
M4 promotion stops writing the marker VALUE-PROVEN (pass 2)
M5 fold stops rendering the realization surface VALUE-PROVEN
M6 report stops labelling the cost section VALUE-PROVEN
M7 preflight stops importing the SDK VALUE-PROVEN
M4 needed pass 2: a PRECEDING assertion caught the same mutation, hiding the new
control behind it -- the oekt-17 lesson reproduced. The remaining nine controls are
vacuity guards (non-emptiness / form-presence) whose mutation would have to break
the source artificially; they are stated as guards, not claimed as value-proven.
MEASURED FINDING (test_loop): the FIRST-RUN-MARKER negative cannot be given a
positive control at all. Within a run only the CHECKER's critique is fed back --
the proposer's own prior reasoning crosses no prompt boundary, not even within a
run. So that negative holds trivially. Left in place with the limitation stated in
the test rather than dressed up as a controlled seam; the CRITIQUE negative beside
it IS controlled and is the real seam.
Mutations were in-place on src/ and shared/ with original bytes restored and
sha-verified; git status clean before and after. Suite 688 -> 688 (assertions added
inside existing tests, no new test cases). ruff + mypy --strict green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Vc5PmZGjwuJypdhzKnJa5
Everything that CAN be validated WITHOUT a model call is validated BEFORE the
operator pays for one (S4.1-analog, SDK-native — Foundry-auth is MAF-specific,
not mirrored). The preflight IS the boundary: it never calls query(), never
validates a credential VALUE online, never touches the network. It returns a
list of structured, actionable Refusals; the CLI exits non-zero on any, so a
broken config stops cheaply instead of on the first billed call.
- preflight.py: run_preflight + `python -m …preflight --profile anthropic`.
Four offline checks:
* credential — ANTHROPIC_API_KEY set + not a placeholder form (the value is
NEVER checked online, only presence, §1); the bundled CLI's own
CLAUDE_CODE_OAUTH_TOKEN also satisfies it (run_s10 relies on it — refusing
would be a false alarm).
* model_map — the requested profile exists and every id it resolves to,
THROUGH resolve_model (so the default fall-through is covered), is real,
not a placeholder left in config.
* sdk — claude_agent_sdk imports (a missing install is a structured refusal
naming `uv sync`, never an ImportError out of the preflight), the run-path
symbols exist, and the bundled Claude Code CLI is present on disk — located
OFFLINE via the SDK package's own files, mirroring the SDK's _find_cli
order (bundled first, then a claude on PATH). Verified against installed
0.2.120 (bundled binary present; the STATE 0.2.110 note was stale).
* termination — the §8 stop contract constructs with positive caps and the
per-call USD belt is positive.
- tests/test_preflight.py: credential contract, placeholder detector, model_map
incl. unknown-profile-without-raising, SDK + bundled-CLI offline probe (with a
monkeypatched missing-SDK refusal), termination, run_preflight aggregation,
CLI both paths, and the offline guards (no network import, no query() call —
AST-based so prose stays green). THREE seams detach-proven RED: credential
branch, placeholder model-id guard, no-network grep-guard.
478→514 green, golden byte-exact, full gate clean (ruff+format+mypy strict,
24 src files), run_s10.py/runs/ byte-untouched. README test-count sync ×2 +
preflight.py module note + load-bearing mention. IKKE-scope (held): the actual
API call (ALDRI — the preflight IS the boundary) and Foundry/Azure auth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RiTwaKLesgcwXx2mDviqpt
