Point 2 of the sweep, enumerated rather than assumed. STATE's total was right and
its distribution was not: 86 hits confirmed (`assert not X` 41 / `== []` 42 /
`== {}` 2 / `== set()` 1), but per file measured `test_cli_paritet` 13 (STATE said
19), `test_preflight` 10 (11), `test_step7` 4 (6).
AST triage split the 86: 55 hits sit in 50 tests whose assertions are ALL
negative; the other 31 already have a positive sibling assert in the same test.
Two negative results worth recording, because they bound the remaining work:
- The `test_preflight` "clears" family (`_check_credentials(...) == []` and
friends) is NOT vacuous. Each sits beside a sibling in the same class that
asserts refusals are non-empty, so a no-op checker turns the sibling red.
Class-level pairing is a real control; these need no change.
- `test_method_spec_loadbearing.py` already models the right pattern for
detectors — explicit `test_guard_red_when_*` red-proofs against a mutated COPY.
This commit fixes the class that had no control at all: static/AST guards that
assert an absence without ever showing the scanner can detect a presence.
1. TAUTOLOGICAL RED-PROOFS (both spec guards). `test_guard_red_when_spec_missing`
asserted a file is absent from a fresh `tmp_path` — true by construction of the
fixture, and it never called the guard it is named for. It would have stayed
green with `test_spec_is_present` deleted outright. Both now exercise the same
`_spec_is_present` predicate the guard calls, in both directions.
2. MISSING RED-PROOF. `test_spec_keeps_structure_markers` had none, unlike its
toolkit and contract-field siblings: with `_STRUCTURE_MARKERS` emptied or
`_missing_markers` stubbed to `[]` it reported green forever. Added
`test_guard_red_when_marker_removed`, parametrized over all 21 markers.
3. BLIND IMPORT SCANNERS (costsim x2, okf, preflight, notify). Every one asserted
`not names & {forbidden}` or `outside == set()` with nothing showing `names`
was non-empty — an empty scan satisfies them exactly as well as real purity.
`test_okf_is_pure_stdlib`'s subset check is likewise trivially true of the
empty set, so it did not guard its neighbour either. Each now asserts a
known-present module first. The notify guard gets the strongest form
available: it proves the detector DOES match a network import inside the seam,
so the matcher itself is shown to work rather than only its silence.
Value-proved, not merely detach-proved. Seven vacuity mutations run against the
NEW tests: all seven RED, each dying on the intended control line. The same
mutations run against the PRE-CHANGE tests (session edits stashed): all five
applicable ones GREEN — blind to the vacuity they were meant to catch. Green
before, red after, same mutation, is the value-proof.
Harness held original bytes in memory, restored in `finally`, sha256-verified
every restore, and checked each run ACTUALLY RAN (a wrong test id yields rc!=0
and mimics red). `git status` clean before and after.
Remaining in the class and NOT closed here: ~45 all-negative tests, mostly CLI
refusal (`calls == []` after a refused invocation) and empty-default
(`missing dir -> []`). Listed in STATE, not silently dropped.
Suite 690 -> 711.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DJmse16bEkaSBtvXhncEUc
Oekt 17 found the class on four named files. This sweep ENUMERATES it: 42 negative
substring assertions across 21 test files (STATE's "~34 across 23" was a premise --
measured, it is 42/21). Sixteen of them measured an absence without ever having
shown presence; all sixteen now carry a positive control asserting the searched-for
string PRESENT in the source artifact, in EXACTLY the form the negative looks for.
Files touched: test_costsim, test_loop, test_okf (3 sites), test_preflight,
test_run_entrance, test_s10_run_layer, test_sdk_version_guard, test_simulation
(2 sites), test_step1_expel, test_step5_refine, test_step7_async_loop,
test_step8_promotion, test_valuereport.
VALUE-PROOF (green-without / red-with, per the oekt-17 rule that a detach proof is
not a value proof). Seven source/fixture mutations, each making the negative vacuous:
M1 verdict fixture loses the realization signal VALUE-PROVEN
M2 decoy fixture loses its text VALUE-PROVEN
M3 renderer stops emitting typed section headings VALUE-PROVEN
M4 promotion stops writing the marker VALUE-PROVEN (pass 2)
M5 fold stops rendering the realization surface VALUE-PROVEN
M6 report stops labelling the cost section VALUE-PROVEN
M7 preflight stops importing the SDK VALUE-PROVEN
M4 needed pass 2: a PRECEDING assertion caught the same mutation, hiding the new
control behind it -- the oekt-17 lesson reproduced. The remaining nine controls are
vacuity guards (non-emptiness / form-presence) whose mutation would have to break
the source artificially; they are stated as guards, not claimed as value-proven.
MEASURED FINDING (test_loop): the FIRST-RUN-MARKER negative cannot be given a
positive control at all. Within a run only the CHECKER's critique is fed back --
the proposer's own prior reasoning crosses no prompt boundary, not even within a
run. So that negative holds trivially. Left in place with the limitation stated in
the test rather than dressed up as a controlled seam; the CRITIQUE negative beside
it IS controlled and is the real seam.
Mutations were in-place on src/ and shared/ with original bytes restored and
sha-verified; git status clean before and after. Suite 688 -> 688 (assertions added
inside existing tests, no new test cases). ruff + mypy --strict green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Vc5PmZGjwuJypdhzKnJa5
Everything that CAN be validated WITHOUT a model call is validated BEFORE the
operator pays for one (S4.1-analog, SDK-native — Foundry-auth is MAF-specific,
not mirrored). The preflight IS the boundary: it never calls query(), never
validates a credential VALUE online, never touches the network. It returns a
list of structured, actionable Refusals; the CLI exits non-zero on any, so a
broken config stops cheaply instead of on the first billed call.
- preflight.py: run_preflight + `python -m …preflight --profile anthropic`.
Four offline checks:
* credential — ANTHROPIC_API_KEY set + not a placeholder form (the value is
NEVER checked online, only presence, §1); the bundled CLI's own
CLAUDE_CODE_OAUTH_TOKEN also satisfies it (run_s10 relies on it — refusing
would be a false alarm).
* model_map — the requested profile exists and every id it resolves to,
THROUGH resolve_model (so the default fall-through is covered), is real,
not a placeholder left in config.
* sdk — claude_agent_sdk imports (a missing install is a structured refusal
naming `uv sync`, never an ImportError out of the preflight), the run-path
symbols exist, and the bundled Claude Code CLI is present on disk — located
OFFLINE via the SDK package's own files, mirroring the SDK's _find_cli
order (bundled first, then a claude on PATH). Verified against installed
0.2.120 (bundled binary present; the STATE 0.2.110 note was stale).
* termination — the §8 stop contract constructs with positive caps and the
per-call USD belt is positive.
- tests/test_preflight.py: credential contract, placeholder detector, model_map
incl. unknown-profile-without-raising, SDK + bundled-CLI offline probe (with a
monkeypatched missing-SDK refusal), termination, run_preflight aggregation,
CLI both paths, and the offline guards (no network import, no query() call —
AST-based so prose stays green). THREE seams detach-proven RED: credential
branch, placeholder model-id guard, no-network grep-guard.
478→514 green, golden byte-exact, full gate clean (ruff+format+mypy strict,
24 src files), run_s10.py/runs/ byte-untouched. README test-count sync ×2 +
preflight.py module note + load-bearing mention. IKKE-scope (held): the actual
API call (ALDRI — the preflight IS the boundary) and Foundry/Azure auth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RiTwaKLesgcwXx2mDviqpt
