feat(b-gate): the gate that measures po as a toolbox, red on six measured rows [skip-docs]

python -m portfolio_optimiser.evals.b_gate — one command, offline, no model call, exit 1 today:

  1 steg i kjørestien kallbare utenfra        3 av 13   RØD
  2 roller som kan leveres utenfra            0 av 2    RØD
  3 vakter mot en vei fra po til Claude       6 av 6    GRØNN  (435 published files)
  4 løpet drevet uten et eneste modellkall    0 av 3    RØD
  5 Foundry-veien urørt og samme artefaktfamilie 1 av 2 RØD
  6 kjøreboka finnes og er kjørt              0 av 2    IKKE MÅLT

Every denominator is read off the source, never off a list in the gate. Row 1 counts the steps
of the run path that resolve to a symbol AND have a call site; a step is externally callable only
when a CLI (or MCP-registered) entry reaches it without any chat-client construct on the way —
which is why the ten run.py steps are red and round_builder's two plus the v1 gate are green. Row
2 reads the roles off workflow._MAKER_CHECKER_ROLES. Row 3's patterns each carry a known-positive
AND a known-negative fixture, so a guard that cannot hit is not counted as a zero.

Three decisions the operator cannot answer without reading code, made here and stated in the
gate's own output:

* the external door is a CLI subcommand, not MCP — po already has five main() and two console
  commands, and MCP would need a server the run path does not have. The gate still counts an
  MCP-registered door, so the choice does not bind the next order.
* the budget guard in B is NOT po's: BudgetMiddleware is fail-closed on missing usage and is
  never constructed without a chat client, so keeping it here would turn fail-closed into
  fail-open. The ceiling in B is the Claude Code session's own spend, which po neither sees nor
  steers. The Foundry path keeps its ceiling unchanged.
* row 6 is IKKE MÅLT, never green, until the operator attests that the runbook actually drove an
  analysis — a file the gate never writes, the same rule as the v1 gate's attestation.

Row 3's pattern text is base64 in the config so the contract cannot register as its own finding;
that is what lets the row run without an exclusion list, and a row without exclusions is a row
nobody can switch off by adding a filename.

Suite after: 2106 passed / 5 skipped / 5 xfailed (was 2072/5/5; +34 new, none changed).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 08:29:10 +02:00
commit f7ade7aa8b
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
3 changed files with 716 additions and 13 deletions

View file

@ -286,10 +286,14 @@
}
},
"foundry": {
"profiles": [
"AZURE",
"LOCAL"
],
"profile": {
"module": "backends.py",
"scope": "Profile",
"members": [
"AZURE",
"LOCAL"
]
},
"factory": {
"module": "run.py",
"symbol": "_default_factory"