portfolio-optimiser/src/portfolio_optimiser/evals
Kjell Tore Guttormsen 6375ce5af3
feat(toolbox): the seven outbox writers get their own door -- row 1 moves 8 -> 15 of 17
Every one of the run path's seven `outbox.write_*` steps was reachable only through
`run.main`, and every path through that builds a chat client. The steps need no model:
they take already-rendered data and put it on disk in a byte-deterministic form.

Seven subcommands, seven thin adapters. The outbox directory is always the caller's to
name -- never a default, never the repository's own, because a step that wrote into a
folder the framework also reads as an inbox would bypass the Step-8 promotion gate.

`write-outbox` is the one that is not purely mechanical: `outbox.write_outbox` branches
on the outcome TYPE, so a door that took the outcome as an argument would let anyone
author an outbox of claims and hand it to Step 8 as results. The door DERIVES it through
`validate_proposal` -- the run path's own composition -- and a blocked proposal exits 3
with the artefacts still written, since that is where the rejection is recorded.
`verdict_id` stays an argument: `verdict-key` already owns that minting.

`--stop-reason` is required rather than defaulted to the empty string, inheriting the
core writer's measured reason: "the run finished" and "we never found out" must not be
the same value.

Eight probes, each a subprocess with the subcommand in argv, each asserting on the FILE
the command wrote. The ground truth is composed in the test -- the payload it wrote and
counted itself, and the byte form the contract requires -- never `outbox._dump`, which
would have measured the module against itself. The refusal arm carries its rc-0 control.

Measured, own run of the B gate: row 1 8 of 17 -> 15 of 17, exit 1 unchanged, no other
row moved. 0 chat-client names reachable from the toolbox (known-positive control: 24 in
run.py).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-20 10:32:12 +02:00
..
__init__.py test(v1-gate): the v1 gate, written RED 2026-09-17 15:31:48 +02:00
b_gate.json feat(toolbox): the seven outbox writers get their own door -- row 1 moves 8 -> 15 of 17 2026-09-20 10:32:12 +02:00
b_gate.py fix(b-gate): a heading is not a section -- the rest of N3, and one rewrapped ledger line [skip-docs] 2026-09-20 09:05:08 +02:00
round_builder.py fix(round-builder): a link is refused with a reason, an identical quote is said once, a shared cost line is named on both sides [skip-docs] 2026-09-19 10:20:30 +02:00
v1_gate.json test(v1-gate): harden the gate against a handwritten green 2026-09-17 17:53:40 +02:00
v1_gate.py feat(round-builder): one command turns a run's outbox into a round the gate can read [skip-docs] 2026-09-19 06:09:12 +02:00