feat(b-gate): the gate that measures po as a toolbox, red on six measured rows [skip-docs]

python -m portfolio_optimiser.evals.b_gate — one command, offline, no model call, exit 1 today:

  1 steg i kjørestien kallbare utenfra        3 av 13   RØD
  2 roller som kan leveres utenfra            0 av 2    RØD
  3 vakter mot en vei fra po til Claude       6 av 6    GRØNN  (435 published files)
  4 løpet drevet uten et eneste modellkall    0 av 3    RØD
  5 Foundry-veien urørt og samme artefaktfamilie 1 av 2 RØD
  6 kjøreboka finnes og er kjørt              0 av 2    IKKE MÅLT

Every denominator is read off the source, never off a list in the gate. Row 1 counts the steps
of the run path that resolve to a symbol AND have a call site; a step is externally callable only
when a CLI (or MCP-registered) entry reaches it without any chat-client construct on the way —
which is why the ten run.py steps are red and round_builder's two plus the v1 gate are green. Row
2 reads the roles off workflow._MAKER_CHECKER_ROLES. Row 3's patterns each carry a known-positive
AND a known-negative fixture, so a guard that cannot hit is not counted as a zero.

Three decisions the operator cannot answer without reading code, made here and stated in the
gate's own output:

* the external door is a CLI subcommand, not MCP — po already has five main() and two console
  commands, and MCP would need a server the run path does not have. The gate still counts an
  MCP-registered door, so the choice does not bind the next order.
* the budget guard in B is NOT po's: BudgetMiddleware is fail-closed on missing usage and is
  never constructed without a chat client, so keeping it here would turn fail-closed into
  fail-open. The ceiling in B is the Claude Code session's own spend, which po neither sees nor
  steers. The Foundry path keeps its ceiling unchanged.
* row 6 is IKKE MÅLT, never green, until the operator attests that the runbook actually drove an
  analysis — a file the gate never writes, the same rule as the v1 gate's attestation.

Row 3's pattern text is base64 in the config so the contract cannot register as its own finding;
that is what lets the row run without an exclusion list, and a row without exclusions is a row
nobody can switch off by adding a filename.

Suite after: 2106 passed / 5 skipped / 5 xfailed (was 2072/5/5; +34 new, none changed).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 08:29:10 +02:00
commit f7ade7aa8b
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
3 changed files with 716 additions and 13 deletions

View file

@ -282,7 +282,9 @@ def test_a_planted_path_to_claude_turns_the_row_red_for_that_pattern(tmp_path: P
cfg = json.loads(json.dumps(_CONFIG["no_claude_path"]))
cfg["roots"] = ["src"]
for spec in cfg["patterns"]:
_write(tmp_path / "src" / f"{spec['id']}.py", gate.decode(spec["known_positive_b64"]) + "\n")
_write(
tmp_path / "src" / f"{spec['id']}.py", gate.decode(spec["known_positive_b64"]) + "\n"
)
row = gate.score_no_claude_path(cfg, tmp_path)
assert (row.k, row.status) == (0, gate.RED)
assert len(row.exceptions) == _PATTERNS_TODAY
@ -408,9 +410,7 @@ def test_m5_the_gate_never_writes_the_attestation_itself(tmp_path: Path) -> None
"""M-5: kan gaten skrive fila den leser, er rad 6 en sløyfe som bekrefter seg selv."""
_write(tmp_path / _CONFIG["runbook"]["path"], "# kjørebok\n")
attest = tmp_path / _CONFIG["runbook"]["attestation"]
rows = gate.evaluate(
config=_CONFIG, repo_root=tmp_path, src=_SRC, probe_runner=_probes()
)
rows = gate.evaluate(config=_CONFIG, repo_root=tmp_path, src=_SRC, probe_runner=_probes())
assert not attest.exists(), "gaten skrev operatørens attestering — da beviser rad 6 ingenting"
assert _row(rows, "kjørebok").status == gate.NOT_MEASURED
assert any("aldri" in a for a in _row(rows, "kjørebok").attests)
@ -439,11 +439,16 @@ def test_the_gate_is_red_today_and_every_row_is_red_for_a_measured_reason() -> N
assert row.exceptions, f"{row.key} er ikke grønn uten å si hvorfor"
def test_the_probe_runner_can_return_passed_for_a_test_that_exists() -> None:
"""Kjent-positiv for selve probe-mekanismen: «missing» over alt ville vært et måleresultat
uten nevner. Nevner: én nodeid som finnes i denne suiten."""
nodeid = "tests/test_b_gate.py::test_the_probe_runner_can_return_passed_for_a_test_that_exists"
assert v1_gate.run_probes([nodeid], _REPO)[nodeid] == "passed"
def test_the_probe_runner_separates_a_test_that_exists_from_one_that_does_not() -> None:
"""Kjent-positiv for selve probe-mekanismen: «missing» over hele lista ville vært et
måleresultat uten nevner. Nevneren er to nodeid-er én som finnes i suiten og én som ikke
gjør det og mekanismen skille dem. Proben peker med vilje UT av denne fila: en nodeid
herfra ville fått barne-pytest til å kjøre denne testen igjen, i det uendelige."""
lever = "tests/test_ledger.py::test_totals_per_project_and_portfolio"
dod = "tests/test_ledger.py::test_finnes_ikke_xyzzy"
outcomes = v1_gate.run_probes([lever, dod], _REPO)
assert outcomes[lever] == "passed"
assert outcomes[dod] == "missing"
def test_render_names_every_row_the_budget_choice_and_the_runbook_rule() -> None: