test(loadbearing): positive controls for the static-guard half of the sibling-vacuity class
Point 2 of the sweep, enumerated rather than assumed. STATE's total was right and
its distribution was not: 86 hits confirmed (`assert not X` 41 / `== []` 42 /
`== {}` 2 / `== set()` 1), but per file measured `test_cli_paritet` 13 (STATE said
19), `test_preflight` 10 (11), `test_step7` 4 (6).
AST triage split the 86: 55 hits sit in 50 tests whose assertions are ALL
negative; the other 31 already have a positive sibling assert in the same test.
Two negative results worth recording, because they bound the remaining work:
- The `test_preflight` "clears" family (`_check_credentials(...) == []` and
friends) is NOT vacuous. Each sits beside a sibling in the same class that
asserts refusals are non-empty, so a no-op checker turns the sibling red.
Class-level pairing is a real control; these need no change.
- `test_method_spec_loadbearing.py` already models the right pattern for
detectors — explicit `test_guard_red_when_*` red-proofs against a mutated COPY.
This commit fixes the class that had no control at all: static/AST guards that
assert an absence without ever showing the scanner can detect a presence.
1. TAUTOLOGICAL RED-PROOFS (both spec guards). `test_guard_red_when_spec_missing`
asserted a file is absent from a fresh `tmp_path` — true by construction of the
fixture, and it never called the guard it is named for. It would have stayed
green with `test_spec_is_present` deleted outright. Both now exercise the same
`_spec_is_present` predicate the guard calls, in both directions.
2. MISSING RED-PROOF. `test_spec_keeps_structure_markers` had none, unlike its
toolkit and contract-field siblings: with `_STRUCTURE_MARKERS` emptied or
`_missing_markers` stubbed to `[]` it reported green forever. Added
`test_guard_red_when_marker_removed`, parametrized over all 21 markers.
3. BLIND IMPORT SCANNERS (costsim x2, okf, preflight, notify). Every one asserted
`not names & {forbidden}` or `outside == set()` with nothing showing `names`
was non-empty — an empty scan satisfies them exactly as well as real purity.
`test_okf_is_pure_stdlib`'s subset check is likewise trivially true of the
empty set, so it did not guard its neighbour either. Each now asserts a
known-present module first. The notify guard gets the strongest form
available: it proves the detector DOES match a network import inside the seam,
so the matcher itself is shown to work rather than only its silence.
Value-proved, not merely detach-proved. Seven vacuity mutations run against the
NEW tests: all seven RED, each dying on the intended control line. The same
mutations run against the PRE-CHANGE tests (session edits stashed): all five
applicable ones GREEN — blind to the vacuity they were meant to catch. Green
before, red after, same mutation, is the value-proof.
Harness held original bytes in memory, restored in `finally`, sha256-verified
every restore, and checked each run ACTUALLY RAN (a wrong test id yields rc!=0
and mimics red). `git status` clean before and after.
Remaining in the class and NOT closed here: ~45 all-negative tests, mostly CLI
refusal (`calls == []` after a refused invocation) and empty-default
(`missing dir -> []`). Listed in STATE, not silently dropped.
Suite 690 -> 711.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DJmse16bEkaSBtvXhncEUc
This commit is contained in:
parent
d691510163
commit
fae5b22578
6 changed files with 64 additions and 4 deletions
|
|
@ -115,9 +115,14 @@ def _undocumented_fields(text: str) -> list[str]:
|
|||
# --- The guard itself (against the real spec) ---------------------------------------
|
||||
|
||||
|
||||
def _spec_is_present(path: Path) -> bool:
|
||||
"""The presence predicate itself, so the red-proof can exercise THE SAME one."""
|
||||
return path.is_file()
|
||||
|
||||
|
||||
def test_spec_is_present() -> None:
|
||||
# RED if the spec goes missing (the method stops being implementable from spec alone).
|
||||
assert SPEC.is_file(), "method-spec.md missing — subtree pull the commons contract"
|
||||
assert _spec_is_present(SPEC), "method-spec.md missing — subtree pull the commons contract"
|
||||
|
||||
|
||||
def test_spec_keeps_structure_markers() -> None:
|
||||
|
|
@ -142,7 +147,26 @@ def test_spec_documents_contract_field(field: str) -> None:
|
|||
|
||||
|
||||
def test_guard_red_when_spec_missing(tmp_path: Path) -> None:
|
||||
assert not (tmp_path / "method-spec.md").is_file()
|
||||
# Was VACUOUS: it asserted a file is absent from a fresh ``tmp_path`` — true by
|
||||
# construction of the fixture, and it never touched the guard it is named for.
|
||||
# It would have stayed green with ``test_spec_is_present`` deleted outright.
|
||||
# Now it exercises THE SAME predicate the guard calls, both directions.
|
||||
# Positive control first: without it, "False for a missing path" would also
|
||||
# hold for a predicate that is False for everything.
|
||||
assert _spec_is_present(SPEC)
|
||||
assert not _spec_is_present(tmp_path / "method-spec.md")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("marker", _STRUCTURE_MARKERS)
|
||||
def test_guard_red_when_marker_removed(tmp_path: Path, marker: str) -> None:
|
||||
# The structure-marker guard had NO red-proof, unlike its toolkit and
|
||||
# contract-field siblings: with ``_STRUCTURE_MARKERS`` emptied or
|
||||
# ``_missing_markers`` stubbed to ``[]``, it would report green forever.
|
||||
# Positive control: the REAL spec is missing no marker, so the detection
|
||||
# below is the mutation being caught, not a spec that was already broken.
|
||||
assert _missing_markers(SPEC.read_text(encoding="utf-8")) == []
|
||||
mutated = SPEC.read_text(encoding="utf-8").replace(marker, "")
|
||||
assert marker in _missing_markers(mutated)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("toolkit", _FORBIDDEN_TOOLKITS)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue