open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(verdicts): key each verdict on its own candidate, not the bundle's one IR projection (S3.2)

Browse source 
seed_store_from_bundle keyed EVERY `type: verdict` file on bundle_candidate_features — the single
candidate the bundle's validator-input.json describes. A bundle carrying verdicts about several
candidates collapsed them onto one key, so a verdict about candidate B scored a perfect structural
match against candidate A's query and could be folded into A's hypothesis prompt. The ExpeL
substrate was single-candidate by construction.

A verdict file may now carry its own structural key in frontmatter (affected_codes / measure_type /
claimed_saving_nok); absent, keying falls back to the bundle candidate, so every pre-S3.2 seed keeps
working unchanged. promote_verdict writes the three fields, so a promoted verdict — frequently about
a different candidate than the target bundle's projection — does not impersonate that candidate.

Semantics decided HERE, not pulled: commons' seeding rule (method-spec §3 Steg 1 + bundle example)
has not arrived; we said we would build locally first. D7 mirroring stays open.

- ALL THREE fields or none. A partial declaration raises VerdictFrontmatterError rather than merging
  with the bundle candidate, which would mint a key belonging to NEITHER candidate. Validation,
  never repair (mirrors write_concept_file); the tolerant-skip rule belongs to the RAW inbox layer.
- claimed_saving_nok parses via json.loads — the SAME literal rule the IR projection went through —
  and is written back with str() of the raw value. _mint_id hashes that value, so 30000 and 30000.0
  are different keys; a normalising writer would split one candidate's signal across two ids.
- The structural key is signal-free, so it does not weaken the Step-8 no-leak property (Test C green).

Load-bearing MEASURED, five mutations all red: detach per-verdict keying · detach the fields
promote_verdict writes · make a partial/unparseable key tolerant · normalise the magnitude on write ·
remove the fallback (control — breaks the step1 suite at collection, proving the fallback bears load).

589 -> 597 tests. Full gate green (pytest, ruff, mypy).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QkjvTTxrg9LTrmghebfiij
This commit is contained in:
Kjell Tore Guttormsen 2026-08-03 16:44:58 +02:00
commit 012adc0a3c
10 changed files with 496 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

139
src/portfolio_optimiser/verdicts.py
View file

@ -474,6 +474,79 @@ def bundle_candidate_features(bundle_dir: str) -> ProposalFeatures:
return _features_from_ir(okf.load_ir_projection(bundle_dir))
# S3.2: a verdict file MAY carry its own structural key. All three fields or none — see
# ``_features_from_verdict_frontmatter``.
_STRUCTURAL_FRONTMATTER_KEYS = ("affected_codes", "measure_type", "claimed_saving_nok")
class VerdictFrontmatterError(ValueError):
"""A ``type: verdict`` file declares its structural key PARTIALLY or unparseably. Fail-fast
(validation, never repair — mirroring ``write_concept_file``): the curated context layer is
hand-written or written by ``promote_verdict``, and the silent alternative — falling back to the
bundle candidate — keys the verdict to the WRONG candidate, which is the exact defect S3.2
closes. Contrast the tolerant RAW inbox layer (``load_verdicts_from_dir``), which skips
malformed files because anyone may drop anything there."""
def _unquote(raw: str) -> str:
"""``parse_frontmatter`` preserves quotes (OKF SPEC §4); the structural fields are compared and
hashed against the IR projection's RAW JSON values, so the quotes have to come off here."""
return raw.strip().strip('"').strip("'").strip()
def _parse_affected_codes(raw: str) -> frozenset[str]:
"""Parse ``affected_codes`` — written by ``promote_verdict`` as ``[A, B]``, and accepted bare
(``A, B``) for hand-authored files. Empty is an error: a declared-but-contentless code set
Jaccard-matches every other empty set, which is a silent mis-key rather than a key."""
codes = {_unquote(part) for part in _unquote(raw).strip("[]").split(",")}
codes.discard("")
if not codes:
raise VerdictFrontmatterError("'affected_codes' is declared but empty")
return frozenset(codes)
def _parse_claimed_saving(raw: str) -> float:
"""Parse ``claimed_saving_nok`` with ``json.loads`` — deliberately the SAME literal rule the IR
projection went through, so ``18000`` stays an int and ``18000.0`` a float. ``_mint_id`` hashes
the raw value, so a parser that normalised the type would mint a different id for a promoted
verdict than for the bundle-keyed seed describing the same candidate."""
try:
value = json.loads(_unquote(raw))
except ValueError as exc:
raise VerdictFrontmatterError(f"'claimed_saving_nok' is not a number: {raw!r}") from exc
if not isinstance(value, (int, float)) or isinstance(value, bool):
raise VerdictFrontmatterError(f"'claimed_saving_nok' is not a number: {raw!r}")
return value
def _features_from_verdict_frontmatter(fm: dict[str, str]) -> ProposalFeatures | None:
"""The verdict's OWN structural key, or ``None`` when it declares none (caller falls back to the
bundle candidate — how every pre-S3.2 seed keeps working).
ALL THREE fields or none. A partial declaration is refused rather than merged with the bundle
candidate, because the merge would mint a key belonging to NEITHER candidate — a synthetic third
proposal that retrieves for nothing. Both fully-present and fully-absent are honest; half is not.
"""
present = [key for key in _STRUCTURAL_FRONTMATTER_KEYS if fm.get(key, "").strip()]
if not present:
return None
if len(present) != len(_STRUCTURAL_FRONTMATTER_KEYS):
missing = [k for k in _STRUCTURAL_FRONTMATTER_KEYS if k not in present]
raise VerdictFrontmatterError(
f"a verdict file declares {present} but not {missing}; a verdict carries its whole "
"structural key or none of it (a partial key belongs to no candidate)"
)
measure_type = _unquote(fm["measure_type"])
if not measure_type:
raise VerdictFrontmatterError("'measure_type' is declared but empty")
return ProposalFeatures(
affected_codes=_parse_affected_codes(fm["affected_codes"]),
measure_type=measure_type,
claimed_saving_nok=_parse_claimed_saving(fm["claimed_saving_nok"]),
description=measure_type,
)
def _verdict_rationale(fm: dict[str, str]) -> str:
"""Build the few-shot rationale from a ``type: verdict`` file's frontmatter, carrying the
learning signal the deterministic validator cannot compute (the realization rate + expected
@ -491,20 +564,34 @@ def _verdict_rationale(fm: dict[str, str]) -> str:
def seed_store_from_bundle(bundle_dir: str) -> VerdictStore:
"""Build a ``VerdictStore`` from an OKF bundle's ``type: verdict`` files. Each verdict is keyed
on the bundle's candidate features (so it retrieves for that measure) and carries the
realization signal in its rationale. The seed verdict stands in for the durable HITL verdict a
real expert would supply via the same folder interface (målbilde §3)."""
features = bundle_candidate_features(bundle_dir)
"""Build a ``VerdictStore`` from an OKF bundle's ``type: verdict`` files. Each verdict carries
the realization signal in its rationale, and stands in for the durable HITL verdict a real
expert would supply via the same folder interface (målbilde §3).
KEYING (S3.2): a verdict is keyed on ITS OWN candidate when its frontmatter declares the
structural fields (``affected_codes`` / ``measure_type`` / ``claimed_saving_nok``), and on the
bundle's IR-projection candidate otherwise. The bundle key alone was single-candidate by
construction: a bundle holding verdicts about several candidates collapsed them onto one key, so
a verdict about candidate B scored a perfect match against candidate A's query and could be
folded into A's hypothesis prompt. The fallback is what keeps every pre-S3.2 seed working
unchanged; the fields are OPTIONAL, never required."""
bundle = okf.navigate_bundle(bundle_dir)
verdicts = [
capture_verdict(
features,
vf.frontmatter.get("decision", "approved"),
_verdict_rationale(vf.frontmatter),
fallback: ProposalFeatures | None = None
verdicts = []
for vf in bundle.verdicts:
features = _features_from_verdict_frontmatter(vf.frontmatter)
if features is None:
# Read the IR projection lazily: a bundle whose verdicts all carry their own key does
# not need one, and this keeps the fallback path's behaviour byte-identical.
fallback = fallback if fallback is not None else bundle_candidate_features(bundle_dir)
features = fallback
verdicts.append(
capture_verdict(
features,
vf.frontmatter.get("decision", "approved"),
_verdict_rationale(vf.frontmatter),
)
)
for vf in bundle.verdicts
]
return VerdictStore(verdicts=verdicts)
@ -555,11 +642,19 @@ def promote_verdict(
wiki. Provenance-stamped (who/which-experiment/when). ``timestamp`` is a required keyword (no
wall-clock default) so promotion is deterministic and the stamp reproducible.
The promoted file is MINIMAL: it does NOT reproduce the hand-authored seed's structured fields
(``realization_rate`` etc.) — the raw ``Verdict`` model carries the learning signal only as
``rationale`` prose, which becomes the ``description`` frontmatter ``seed_store_from_bundle``
folds into ExpeL. The index link uses a NEUTRAL label (``_PROMOTED_LINK_LABEL``), so the signal
reaches a prompt only via the gated fold, never via ``bundle_context`` (§3/§6).
The promoted file is MINIMAL as to the LEARNING SIGNAL: it does NOT reproduce the hand-authored
seed's structured signal fields (``realization_rate`` etc.) — the raw ``Verdict`` model carries
that only as ``rationale`` prose, which becomes the ``description`` frontmatter
``seed_store_from_bundle`` folds into ExpeL. It DOES carry its own structural KEY (S3.2:
``affected_codes`` / ``measure_type`` / ``claimed_saving_nok``), so the next run keys it on the
candidate it is about — a promoted verdict is frequently about a different candidate than the
one that bundle's IR projection describes. The index link uses a NEUTRAL label
(``_PROMOTED_LINK_LABEL``), so the signal reaches a prompt only via the gated fold, never via
``bundle_context`` (§3/§6) — and the structural key is signal-free by construction.
Round-trip caveat: ``render_frontmatter`` single-lines every value, so a ``measure_type``
containing newlines is re-read with those collapsed to spaces and would re-mint a different id.
Measure strings are single-line in practice; this is stated rather than defended against.
Known limitation (mirrors ``write_verdict``): ``_mint_id`` keys on the candidate features, so two
approved verdicts about the SAME candidate share an id -> share a filename -> last-write-wins;
@ -575,6 +670,16 @@ def promote_verdict(
"type": "verdict",
"decision": verdict.decision,
"description": verdict.rationale,
# S3.2: the promoted file carries its OWN structural key, so the next run's
# seed_store_from_bundle keys it on the candidate it is actually about rather than on
# whichever candidate that bundle's IR projection happens to describe. Written as the three
# fields _features_from_verdict_frontmatter reads back — all three, never a partial key.
"affected_codes": "[" + ", ".join(sorted(f.affected_codes)) + "]",
"measure_type": f.measure_type,
# ``str`` of the raw value, NOT a normalised format: it round-trips through
# ``_parse_claimed_saving``'s ``json.loads`` back to the same int/float, and ``_mint_id``
# hashes that raw value (``18000`` and ``18000.0`` are different keys).
"claimed_saving_nok": str(f.claimed_saving_nok),
"verdict_id": verdict.id,
"provenance": f"godkjent av {approver}; eksperiment {experiment}; {timestamp}",
"timestamp": timestamp,