test(accounting): the two forged routes become mutants, and src/ becomes measurable

killed 39 of 39, exit 0 (36 before this round). Three new mutants, one per
check that closes PM's measured forgery, each felled by its own arm:

- X3 restores the pre-round `_conversions` verbatim -- the clause read from
  anywhere in the bundle text. Felled by the body-text and table-cell arms.
- X4 drops the tie between the clause and the asset its block points at.
  Felled by the arm where the clause stands in another asset's block.
- X5 drops `_inline`'s checksum disarming. Felled by the end-to-end arm where
  an `<img alt>` carries the clause.

X5 IS WHY THE HARNESS CHANGED, and it is a finding about the harness rather
than about the code: it SURVIVED 112 green tests on the first run, and it had
never been applied. The copy is run with the venv's interpreter, which carries
an editable install pointing at the working tree, so `import llm_ingestion_okf`
in the copy resolved to the original `src/`. The subprocess now gets the copy's
own `src/` on `PYTHONPATH`, which wins over the editable finder, and X5 dies on
the first run under it -- the survivor/killed pair is the measurement. The gate
and the witness were never affected; the suite reaches those through the copy's
own `tools/`.

The docstring's "the twenty-six mutants" now says "twenty-six of them", since
the list has been 39 for a while and the sentence is about their provenance.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 09:47:46 +02:00
commit beef436cbb
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
2 changed files with 51 additions and 4 deletions

View file

@ -622,9 +622,7 @@ def test_a_document_cannot_forge_a_conversion_claim(tmp_path: Path) -> None:
# KNOWN-POSITIVE on the same bytes: the clause where the code writes it,
# in the pointer block for the asset it names. Without this the arms above
# would pass on a route that had simply stopped working.
honest_text = (
f"![Ekte](/assets/{after[:12]}-ekte.png)\nImage: figur.bmp (1x1 px) -- {clause}\n"
)
honest_text = f"![Ekte](/assets/{after[:12]}-ekte.png)\nImage: figur.bmp (1x1 px) -- {clause}\n"
honest = _build(assets=assets, bundle_text=honest_text)
assert gate.asset_holds(honest, never_carried) is True

View file

@ -5,12 +5,22 @@ textual edit that makes the instrument weaker in a way a reader would call a
defect, and the question is whether the suite goes red. A mutant that survives
names a check nothing holds.
The twenty-six mutants are an independent review's, ported to the code as it
Twenty-six of them are an independent review's, ported to the code as it
stands rather than re-invented: at `0b00de4` twelve of them survived the
forty-two tests, among them `main` always returning 0 and six of the seven
witness mutants -- which had no fixture that could exercise the element they
removed.
A MUTANT MAY EDIT `src/`, and until 2026-09-19 one could not. The copy is run
with the venv's interpreter, which carries an EDITABLE install pointing at the
original tree, so `import llm_ingestion_okf` in the copy resolved to the
working tree and a `src/` mutant was reported as a survivor without ever
having been applied -- measured on X5, which survived 112 green tests and then
died on the first run with `PYTHONPATH` set. The subprocess now gets the
copy's own `src/` on `PYTHONPATH`, which wins over the editable finder. The
gate and the witness were never affected: the suite reaches those through
`sys.path.insert(0, TOOLS)` off its own location, which is already the copy.
python3 tools/okf_gate_mutants.py
Runs on a COPY of the tree in a temporary directory: the working tree is never
@ -26,6 +36,7 @@ could fail on the finding this harness exists to produce (H4).
from __future__ import annotations
import os
import shutil
import subprocess
import sys
@ -284,6 +295,41 @@ MUTANTS: tuple[Mutant, ...] = (
' if status == "rejected" and persisted:\n invalid += 1',
" if False:\n invalid += 1",
),
# PM's sjekkpunkt 2026-09-19: the conversion route the viewable-asset round
# added reads two digests out of the bundle, which put an untrusted document
# inside the judge's own input. Three edits, one per check that closes it.
# X5 edits `src/`, which no other mutant here does: the door that keeps a
# LABEL from emitting the judge's grammar lives in the build, and the test
# that fells it lives in this gate's suite, so it is measured from here.
Mutant(
"X3 the conversion claim is read from anywhere in the bundle text",
GATE,
""" found: dict[str, str] = {}
for pointer in _POINTER.finditer(bundle_text):
clause = _CONVERSION.search(pointer.group("detail"))
if clause is None:
continue
after = clause.group("after")
# The claim has to be about the picture the block points at. A clause
# standing in one asset's block while naming another's digest is a
# sentence nothing in this build writes.
if pointer.group("asset").startswith(after[:12]):
found[clause.group("before")] = after
return found""",
""" return {m.group("before"): m.group("after") for m in _CONVERSION.finditer(bundle_text)}""",
),
Mutant(
"X4 the claim need not be about the asset its block points at",
GATE,
'if pointer.group("asset").startswith(after[:12]):',
"if True:",
),
Mutant(
"X5 a document-supplied label may emit a checksum field",
"src/llm_ingestion_okf/assets.py",
'return _CHECKSUM_FIELD.sub("sha256 ", collapsed.replace("[", "(").replace("]", ")"))',
'return collapsed.replace("[", "(").replace("]", ")")',
),
)
@ -328,12 +374,15 @@ def main(argv: list[str] | None = None) -> int:
)
continue
path.write_text(_apply(original, mutant), encoding="utf-8")
environment = dict(os.environ)
environment["PYTHONPATH"] = str(root / "src")
try:
run = subprocess.run(
[sys.executable, "-m", "pytest", SUITE, "-q", "-x", "-p", "no:cacheprovider"],
cwd=root,
capture_output=True,
text=True,
env=environment,
)
finally:
path.write_text(original, encoding="utf-8")