feat(outbox): every evaluated approach becomes something an expert can judge
A run commissioned to evaluate three approaches wrote ONE proposal artefact, so
only the approach it selected could ever receive a verdict. The other two were
evaluated, reported in the settlement, and then taught the learning loop nothing.
The defect class is a key collapse, and it had two halves — fixing either alone
leaves it intact:
* the WRITER wrote one pair per run, so the non-selected approaches never existed
on disk;
* the READER (hitl._read_outbox_proposals) joins proposal to outcome on the
run_id FIELD read from file CONTENT, never the filename. Three files sharing
one run_id collapse onto one dict key, last write wins — so widening only the
filename would have produced three artefacts and still one pending row. This is
the S3.2 collision class: two rows under one key silently become one.
Artefacts are now keyed {run_id}-{approach_id}-*.json AND carry approach_id in the
payload; the join key is (run_id, approach_id). Two properties make them genuinely
judgeable rather than merely present:
* verdict_id is minted per approach (verdicts.verdict_key, the S3.2 content hash)
— reusing the run's single id would let one delivered verdict clear all three
from the queue;
* provenance.validator_decision follows ITS OWN approach — the run's stamp would
report a rejected candidate as validated, and nothing downstream could correct it.
verdicts.verdict_key is public so a run can stamp the key a verdict WILL arrive
under without capturing a decision nobody has made; it delegates to _mint_id
rather than restating the hash (the (p) rule: one keying rule, one copy).
The per-approach set REPLACES the run-level pair rather than joining it — the
selected approach is already among them, and writing both would count it twice in
hitl pending. The selected one carries the run's final outcome, so the outbox can
never disagree with the RunResult; the others carry the validator's verdict, the
only falsifier that ran on them.
mandate.py is deliberately untouched: hanging a ValidatedProposal off a coverage
row would drag validator — and pulp — into a module kept to pydantic+stdlib for
D7 portability, so _evaluate_mandate returns the evaluated outcomes alongside.
Ran it, not just tested it: a real CLI run wrote six artefacts and hitl pending
listed three rows. It also showed the honest edge — three approaches that produce
an identical candidate share one content-hash key, so one verdict settles all
three. That is correct (they were one candidate), and it is now documented.
Load-bearing MEASURED (tests/test_a5_per_approach_artifacts_loadbearing.py) against
the whole 750-test suite, five mutations all red: detach the per-approach writer ·
drop approach_id from the join key · reuse the run's verdict id · reuse the run's
provenance stamp · widen the filename but not the payload. Control: on a full
detach exactly the 5 new tests fail and 745 pre-existing ones stay green — the
no-mandate path is inert, and writes neither the filename segment nor the field.
Docs: bestille-en-kjoring.md (what the commissioner gets) + ekspert-svar.md (what
the expert's queue looks like, and that "rejected" is the validator's verdict on
the numbers, never a professional judgement of the idea).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VtRd8y1PDPGwkrRXFhubqr
This commit is contained in:
parent
9668e17f2f
commit
455d93d33e
7 changed files with 443 additions and 33 deletions
|
|
@ -98,6 +98,18 @@ def _mint_id(features: ProposalFeatures) -> str:
|
|||
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()[:16]
|
||||
|
||||
|
||||
def verdict_key(features: ProposalFeatures) -> str:
|
||||
"""The id a verdict on a proposal with these features keys under — the learning-loop key.
|
||||
|
||||
Public because a run must be able to stamp an artefact with the key an expert verdict on THAT
|
||||
candidate will arrive under, WITHOUT capturing a decision nobody has made yet (A5: every
|
||||
evaluated approach gets its own judgeable artefact, but only one of them is the run's outcome).
|
||||
One key, one minting rule: this delegates to ``_mint_id`` rather than restating the hash, so a
|
||||
caller can never drift from the id ``capture_verdict`` actually assigns (the ``(p)`` precedent
|
||||
— a second private copy of a keying rule is the defect, not the convenience)."""
|
||||
return _mint_id(features)
|
||||
|
||||
|
||||
def capture_verdict(features: ProposalFeatures, decision: str, rationale: str) -> Verdict:
|
||||
"""Layer-2 out-of-band verdict constructor: mint a stable content-hash id (the
|
||||
learning-loop key) and build the ``Verdict`` to persist in the store."""
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue