test(accounting): the two forged routes become mutants, and src/ becomes measurable
killed 39 of 39, exit 0 (36 before this round). Three new mutants, one per check that closes PM's measured forgery, each felled by its own arm: - X3 restores the pre-round `_conversions` verbatim -- the clause read from anywhere in the bundle text. Felled by the body-text and table-cell arms. - X4 drops the tie between the clause and the asset its block points at. Felled by the arm where the clause stands in another asset's block. - X5 drops `_inline`'s checksum disarming. Felled by the end-to-end arm where an `<img alt>` carries the clause. X5 IS WHY THE HARNESS CHANGED, and it is a finding about the harness rather than about the code: it SURVIVED 112 green tests on the first run, and it had never been applied. The copy is run with the venv's interpreter, which carries an editable install pointing at the working tree, so `import llm_ingestion_okf` in the copy resolved to the original `src/`. The subprocess now gets the copy's own `src/` on `PYTHONPATH`, which wins over the editable finder, and X5 dies on the first run under it -- the survivor/killed pair is the measurement. The gate and the witness were never affected; the suite reaches those through the copy's own `tools/`. The docstring's "the twenty-six mutants" now says "twenty-six of them", since the list has been 39 for a while and the sentence is about their provenance. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
579cb1c83c
commit
beef436cbb
2 changed files with 51 additions and 4 deletions
|
|
@ -622,9 +622,7 @@ def test_a_document_cannot_forge_a_conversion_claim(tmp_path: Path) -> None:
|
|||
# KNOWN-POSITIVE on the same bytes: the clause where the code writes it,
|
||||
# in the pointer block for the asset it names. Without this the arms above
|
||||
# would pass on a route that had simply stopped working.
|
||||
honest_text = (
|
||||
f"\nImage: figur.bmp (1x1 px) -- {clause}\n"
|
||||
)
|
||||
honest_text = f"\nImage: figur.bmp (1x1 px) -- {clause}\n"
|
||||
honest = _build(assets=assets, bundle_text=honest_text)
|
||||
assert gate.asset_holds(honest, never_carried) is True
|
||||
|
||||
|
|
|
|||
|
|
@ -5,12 +5,22 @@ textual edit that makes the instrument weaker in a way a reader would call a
|
|||
defect, and the question is whether the suite goes red. A mutant that survives
|
||||
names a check nothing holds.
|
||||
|
||||
The twenty-six mutants are an independent review's, ported to the code as it
|
||||
Twenty-six of them are an independent review's, ported to the code as it
|
||||
stands rather than re-invented: at `0b00de4` twelve of them survived the
|
||||
forty-two tests, among them `main` always returning 0 and six of the seven
|
||||
witness mutants -- which had no fixture that could exercise the element they
|
||||
removed.
|
||||
|
||||
A MUTANT MAY EDIT `src/`, and until 2026-09-19 one could not. The copy is run
|
||||
with the venv's interpreter, which carries an EDITABLE install pointing at the
|
||||
original tree, so `import llm_ingestion_okf` in the copy resolved to the
|
||||
working tree and a `src/` mutant was reported as a survivor without ever
|
||||
having been applied -- measured on X5, which survived 112 green tests and then
|
||||
died on the first run with `PYTHONPATH` set. The subprocess now gets the
|
||||
copy's own `src/` on `PYTHONPATH`, which wins over the editable finder. The
|
||||
gate and the witness were never affected: the suite reaches those through
|
||||
`sys.path.insert(0, TOOLS)` off its own location, which is already the copy.
|
||||
|
||||
python3 tools/okf_gate_mutants.py
|
||||
|
||||
Runs on a COPY of the tree in a temporary directory: the working tree is never
|
||||
|
|
@ -26,6 +36,7 @@ could fail on the finding this harness exists to produce (H4).
|
|||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
|
@ -284,6 +295,41 @@ MUTANTS: tuple[Mutant, ...] = (
|
|||
' if status == "rejected" and persisted:\n invalid += 1',
|
||||
" if False:\n invalid += 1",
|
||||
),
|
||||
# PM's sjekkpunkt 2026-09-19: the conversion route the viewable-asset round
|
||||
# added reads two digests out of the bundle, which put an untrusted document
|
||||
# inside the judge's own input. Three edits, one per check that closes it.
|
||||
# X5 edits `src/`, which no other mutant here does: the door that keeps a
|
||||
# LABEL from emitting the judge's grammar lives in the build, and the test
|
||||
# that fells it lives in this gate's suite, so it is measured from here.
|
||||
Mutant(
|
||||
"X3 the conversion claim is read from anywhere in the bundle text",
|
||||
GATE,
|
||||
""" found: dict[str, str] = {}
|
||||
for pointer in _POINTER.finditer(bundle_text):
|
||||
clause = _CONVERSION.search(pointer.group("detail"))
|
||||
if clause is None:
|
||||
continue
|
||||
after = clause.group("after")
|
||||
# The claim has to be about the picture the block points at. A clause
|
||||
# standing in one asset's block while naming another's digest is a
|
||||
# sentence nothing in this build writes.
|
||||
if pointer.group("asset").startswith(after[:12]):
|
||||
found[clause.group("before")] = after
|
||||
return found""",
|
||||
""" return {m.group("before"): m.group("after") for m in _CONVERSION.finditer(bundle_text)}""",
|
||||
),
|
||||
Mutant(
|
||||
"X4 the claim need not be about the asset its block points at",
|
||||
GATE,
|
||||
'if pointer.group("asset").startswith(after[:12]):',
|
||||
"if True:",
|
||||
),
|
||||
Mutant(
|
||||
"X5 a document-supplied label may emit a checksum field",
|
||||
"src/llm_ingestion_okf/assets.py",
|
||||
'return _CHECKSUM_FIELD.sub("sha256 ", collapsed.replace("[", "(").replace("]", ")"))',
|
||||
'return collapsed.replace("[", "(").replace("]", ")")',
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -328,12 +374,15 @@ def main(argv: list[str] | None = None) -> int:
|
|||
)
|
||||
continue
|
||||
path.write_text(_apply(original, mutant), encoding="utf-8")
|
||||
environment = dict(os.environ)
|
||||
environment["PYTHONPATH"] = str(root / "src")
|
||||
try:
|
||||
run = subprocess.run(
|
||||
[sys.executable, "-m", "pytest", SUITE, "-q", "-x", "-p", "no:cacheprovider"],
|
||||
cwd=root,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=environment,
|
||||
)
|
||||
finally:
|
||||
path.write_text(original, encoding="utf-8")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue