test(retrieval-gate): three fixtures make M06, M07 and M10 decide a delivery — row 7 14 of 14

Row 7 left three mutants standing after v1.1 (11 of 14, bar 13): the passage
signal reading no body, the field signal weighing no title and no path, and a
flattened fusion (bm25.RRF_K = 10 000). Each switches off a mechanism the
default ranking runs, and each moved 0 ranks, because every synthetic concept
was short and opened with its own title -- none of the three ever decided a
delivery here.

Three bundles, one per mechanism, and one pinned set (set-mechanisms.json):

- PASSAGE: a long concept answered in one window of its body, against ten
  short concepts whose titles carry the question's words. Rank 1; with no
  body windows it falls out of k (class b).
- PATH: a concept named by its path alone, against ten decoys denser in the
  body. Rank 4 at k = 6; with no title/path weight it falls out of k.
- FUSION: gold 1st in the passage signal and 21st in the field signal, one
  decoy 10th and 11th (rank sum 19 < 20). At RRF_K = 60 the gold leads, and
  it keeps the lead through K = 180 (measured in steps of 10); flattened, the
  decoy wins k = 1.

Chose separate fixtures over one combined one because each fixture's comment
can then name the single mechanism it pins. Verified per question against all
14 mutants: no previously felled mutant is lost. src/ is untouched, MUTANT_BAR
and the roster are unchanged, the corpus pin moved with the corpus.

Gate: rows 1 and 6 go 10/10 -> 13/13, row 7 11/14 -> 14/14 GREEN, rows 2/3/4
unchanged, GATE RED: rows 5, 8. Suite: 2444 passed, 1 skipped (+3), measured
with FORCE_COLOR unset -- with FORCE_COLOR=3 in the environment Python 3.14
colours argparse help and test_each_arm_flag_carries_its_attribution fails,
independently of this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-22 22:41:01 +02:00
commit e503f6abd7
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
5 changed files with 234 additions and 48 deletions

View file

@ -632,14 +632,15 @@ def test_the_gate_is_red_today_and_says_which_rows(tmp_path: Path) -> None:
by_number = {row.number: row for row in rows}
assert sorted(by_number) == [1, 2, 3, 4, 5, 6, 7, 8]
# Rows 2 and 3 are green again since the synthetic corpus was re-measured
# for BM25 (2026-09-21). Row 7 is red on three survivors, each printed with
# what it moved.
assert [row.number for row in rows if row.fails] == [5, 7, 8]
assert (by_number[1].k, by_number[1].m) == (10, 10)
# for BM25 (2026-09-21). Row 7 fells all 14 since 2026-09-22, when the three
# v1.1 survivors each got a fixture that makes their mechanism decide a
# delivery (`set-mechanisms.json`, three more row-1 units).
assert [row.number for row in rows if row.fails] == [5, 8]
assert (by_number[1].k, by_number[1].m) == (13, 13)
assert (by_number[2].k, by_number[2].m) == (7, 7)
assert (by_number[3].k, by_number[3].m) == (5, 5)
assert (by_number[6].k, by_number[6].m) == (10, 10)
assert (by_number[7].k, by_number[7].m) == (11, 14)
assert (by_number[6].k, by_number[6].m) == (13, 13)
assert (by_number[7].k, by_number[7].m) == (14, 14)
def test_the_same_tree_measures_the_same_twice(tmp_path: Path) -> None:
@ -655,7 +656,7 @@ def test_the_command_exits_one_and_prints_every_row(
printed = capsys.readouterr().out
for number in range(1, 9):
assert f"\n{number} " in f"\n{printed}"
assert "GATE RED: rows 5, 7, 8" in printed
assert "GATE RED: rows 5, 8" in printed
def test_the_json_form_carries_the_same_rows(capsys: pytest.CaptureFixture[str]) -> None: