feat(verdicts): key each verdict on its own candidate, not the bundle's one IR projection (S3.2)
seed_store_from_bundle keyed EVERY `type: verdict` file on bundle_candidate_features — the single candidate the bundle's validator-input.json describes. A bundle carrying verdicts about several candidates collapsed them onto one key, so a verdict about candidate B scored a perfect structural match against candidate A's query and could be folded into A's hypothesis prompt. The ExpeL substrate was single-candidate by construction. A verdict file may now carry its own structural key in frontmatter (affected_codes / measure_type / claimed_saving_nok); absent, keying falls back to the bundle candidate, so every pre-S3.2 seed keeps working unchanged. promote_verdict writes the three fields, so a promoted verdict — frequently about a different candidate than the target bundle's projection — does not impersonate that candidate. Semantics decided HERE, not pulled: commons' seeding rule (method-spec §3 Steg 1 + bundle example) has not arrived; we said we would build locally first. D7 mirroring stays open. - ALL THREE fields or none. A partial declaration raises VerdictFrontmatterError rather than merging with the bundle candidate, which would mint a key belonging to NEITHER candidate. Validation, never repair (mirrors write_concept_file); the tolerant-skip rule belongs to the RAW inbox layer. - claimed_saving_nok parses via json.loads — the SAME literal rule the IR projection went through — and is written back with str() of the raw value. _mint_id hashes that value, so 30000 and 30000.0 are different keys; a normalising writer would split one candidate's signal across two ids. - The structural key is signal-free, so it does not weaken the Step-8 no-leak property (Test C green). Load-bearing MEASURED, five mutations all red: detach per-verdict keying · detach the fields promote_verdict writes · make a partial/unparseable key tolerant · normalise the magnitude on write · remove the fallback (control — breaks the step1 suite at collection, proving the fallback bears load). 589 -> 597 tests. Full gate green (pytest, ruff, mypy). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QkjvTTxrg9LTrmghebfiij
This commit is contained in:
parent
e8cec2e2c0
commit
012adc0a3c
10 changed files with 496 additions and 17 deletions
|
|
@ -0,0 +1,34 @@
|
|||
---
|
||||
type: index
|
||||
okf_version: 0.1
|
||||
title: "Multi-kandidat mikro — repo-lokal fixture (S3.2 per-dom-noekling)"
|
||||
description: "Minimal repo-lokal OKF-bundle under data/ med dommer om TO ULIKE kandidater, mens IR-projeksjonen beskriver kandidat A alene. Fixturen gjoer forskjellen mellom per-dom-noekling og bundle-noekling maalbar."
|
||||
tags: [fixture, S3.2, multi-kandidat]
|
||||
timestamp: 2026-08-03
|
||||
---
|
||||
|
||||
# Multi-kandidat mikro (repo-lokal test-fixture)
|
||||
|
||||
En **repo-lokal mini OKF-bundle** under pakkens `data/` (ALDRI `shared/` — subtree er PULL-ONLY).
|
||||
Den finnes for ett formaal: bundelen bærer dommer om **to ulike kandidater**, mens
|
||||
`validator-input.json` (ExpeL-query-noekkelen) beskriver **kandidat A alene**.
|
||||
|
||||
Foer S3.2 noeklet `seed_store_from_bundle` HVER dom paa bundelens ene IR-projeksjon, saa dommen om
|
||||
kandidat B ble uskillelig fra dommen om kandidat A og kunne naa kandidat As hypotese-prompt. Med
|
||||
per-dom-noekling leser hver dom sine EGNE strukturelle felt fra frontmatter.
|
||||
|
||||
Rekkefoelgen under er bevisst: **kandidat B er lenket foerst**, saa en detachet per-dom-noekling
|
||||
(begge dommer faar kandidat As noekkel → uavgjort likhet, uavgjort id) rangerer B-dommen oeverst.
|
||||
|
||||
## Innhold (progressiv disclosure)
|
||||
|
||||
- [verdict-b-asfalt.md](verdict-b-asfalt.md) — `type: verdict` — dom om **kandidat B**
|
||||
(asfalttykkelse, kode `05.2`). Bærer sine egne `affected_codes`/`measure_type`/
|
||||
`claimed_saving_nok`. Skal ALDRI naa kandidat As hypotese-prompt.
|
||||
- [verdict-a-led.md](verdict-a-led.md) — `type: verdict` — dom om **kandidat A**
|
||||
(LED-retrofit, kode `ENERGI-TOTAL-EL`). Dette er dommen kandidat As query skal hente.
|
||||
- [prosjekt-multi.md](prosjekt-multi.md) — `type: project` — bygget/strekningen begge kandidatene
|
||||
hoerer til. Concept-filen OKF-navigasjonen leser inn som lese-kontekst.
|
||||
|
||||
`validator-input.json` er IR-projeksjonen den deterministiske validatoren konsumerer, og kilden for
|
||||
ExpeL-query-noekkelen (`bundle_candidate_features`) — kandidat A.
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
---
|
||||
type: project
|
||||
title: "Kombinert bygg + veistrekning (syntetisk fixture-prosjekt)"
|
||||
description: "Syntetisk prosjekt som baerer to uavhengige kandidat-tiltak — ett energitiltak i kontorfloy A og ett dekketiltak paa veistrekning B."
|
||||
resource: MULTI-KANDIDAT-MIKRO
|
||||
tags: [fixture, project]
|
||||
timestamp: 2026-08-03
|
||||
---
|
||||
|
||||
# Kombinert bygg + veistrekning
|
||||
|
||||
Syntetisk fixture-prosjekt. To uavhengige tiltak er identifisert:
|
||||
|
||||
- **Kandidat A — LED-retrofit kontorfloy A.** Beroerer kostkode `ENERGI-TOTAL-EL`.
|
||||
Modellert besparelse 18 000 NOK/aar.
|
||||
- **Kandidat B — redusert asfalttykkelse veistrekning B.** Beroerer kostkode `05.2`.
|
||||
Modellert besparelse 900 000 NOK.
|
||||
|
||||
Kandidatene deler verken kostkode, tiltakstype eller stoerrelsesorden. Det er med vilje: en dom om
|
||||
den ene sier ingenting om den andre, og skal derfor ikke hentes for den andre.
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
{
|
||||
"_note": "SYNTHETIC repo-local multi-candidate mini-bundle IR-projeksjon (Fase 2a S3.2-fixture) — ikke ekte data. Bundelen bærer dommer om TO kandidater, men IR-projeksjonen (og dermed bundle_candidate_features) er kandidat A alene — nettopp asymmetrien per-verdict-noekling maa haandtere.",
|
||||
"project_id": "MULTI-KANDIDAT-MIKRO",
|
||||
"measure": "LED-retrofit kontorfloy A",
|
||||
"affected_items": [
|
||||
{
|
||||
"code": "ENERGI-TOTAL-EL",
|
||||
"quantity": 180000,
|
||||
"unit_cost": 1.0
|
||||
}
|
||||
],
|
||||
"claimed_saving_nok": 18000,
|
||||
"assumptions": {
|
||||
"ENERGI-TOTAL-EL": [0.70, 1.40]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
---
|
||||
type: verdict
|
||||
title: "Ekspert-dom (froe) — LED-retrofit kontorfloy A"
|
||||
description: "Froesatt realiseringskorreksjon for KANDIDAT A (energitiltak). Dette er dommen kandidat As hypotese-prompt skal hente."
|
||||
resource: MULTI-KANDIDAT-MIKRO
|
||||
measure_id: LED-RETROFIT-A
|
||||
decision: approved
|
||||
affected_codes: [ENERGI-TOTAL-EL]
|
||||
measure_type: "LED-retrofit kontorfloy A"
|
||||
claimed_saving_nok: 18000
|
||||
realization_rate: 0.80
|
||||
expected_actual_saving_nok: 14400
|
||||
gap_source: hours-of-use-overestimation
|
||||
provenance: "froe — AI-forfattet test-fixture for S3.2; ikke ekte data"
|
||||
tags: [verdict, fixture, kandidat-A]
|
||||
timestamp: 2026-08-03
|
||||
---
|
||||
|
||||
# Ekspert-dom (froe) — kandidat A
|
||||
|
||||
**Beslutning:** godkjent, med realiseringskorreksjon.
|
||||
|
||||
Den modellerte besparelsen paa 18 000 NOK/aar er teknisk korrekt, men i drift realiseres
|
||||
erfaringsvis ~80 % av en timeplan-stipulert LED-besparelse i kontorbygg.
|
||||
|
||||
Dette er dommen som ER relevant for LED-hypotesen — samme kostkode, samme tiltakstype, samme
|
||||
stoerrelsesorden.
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
---
|
||||
type: verdict
|
||||
title: "Ekspert-dom (froe) — redusert asfalttykkelse veistrekning B"
|
||||
description: "Froesatt dom om KANDIDAT B (dekketiltak). Baerer sine egne strukturelle felt, saa den noekles paa sin egen kandidat og ikke paa bundelens IR-projeksjon."
|
||||
resource: MULTI-KANDIDAT-MIKRO
|
||||
measure_id: ASFALT-B
|
||||
decision: rejected
|
||||
affected_codes: [05.2]
|
||||
measure_type: "Redusert asfalttykkelse veistrekning B"
|
||||
claimed_saving_nok: 900000
|
||||
realization_rate: 0.31
|
||||
expected_actual_saving_nok: 279000
|
||||
gap_source: levetidsforkortelse-gir-tidligere-reasfaltering
|
||||
provenance: "froe — AI-forfattet test-fixture for S3.2; ikke ekte data"
|
||||
tags: [verdict, fixture, kandidat-B]
|
||||
timestamp: 2026-08-03
|
||||
---
|
||||
|
||||
# Ekspert-dom (froe) — kandidat B
|
||||
|
||||
**Beslutning:** avvist.
|
||||
|
||||
Den modellerte besparelsen paa 900 000 NOK er regnemessig korrekt, men tynnere dekke forkorter
|
||||
levetiden saa mye at reasfaltering kommer tidligere. Livsloepsregnskapet blir negativt.
|
||||
|
||||
Denne dommen handler om **dekketiltaket**, ikke om energitiltaket. Den skal ikke hentes naar
|
||||
hypotesen gjelder LED-retrofit — kostkode, tiltakstype og stoerrelsesorden er alle ulike.
|
||||
|
|
@ -474,6 +474,79 @@ def bundle_candidate_features(bundle_dir: str) -> ProposalFeatures:
|
|||
return _features_from_ir(okf.load_ir_projection(bundle_dir))
|
||||
|
||||
|
||||
# S3.2: a verdict file MAY carry its own structural key. All three fields or none — see
|
||||
# ``_features_from_verdict_frontmatter``.
|
||||
_STRUCTURAL_FRONTMATTER_KEYS = ("affected_codes", "measure_type", "claimed_saving_nok")
|
||||
|
||||
|
||||
class VerdictFrontmatterError(ValueError):
|
||||
"""A ``type: verdict`` file declares its structural key PARTIALLY or unparseably. Fail-fast
|
||||
(validation, never repair — mirroring ``write_concept_file``): the curated context layer is
|
||||
hand-written or written by ``promote_verdict``, and the silent alternative — falling back to the
|
||||
bundle candidate — keys the verdict to the WRONG candidate, which is the exact defect S3.2
|
||||
closes. Contrast the tolerant RAW inbox layer (``load_verdicts_from_dir``), which skips
|
||||
malformed files because anyone may drop anything there."""
|
||||
|
||||
|
||||
def _unquote(raw: str) -> str:
|
||||
"""``parse_frontmatter`` preserves quotes (OKF SPEC §4); the structural fields are compared and
|
||||
hashed against the IR projection's RAW JSON values, so the quotes have to come off here."""
|
||||
return raw.strip().strip('"').strip("'").strip()
|
||||
|
||||
|
||||
def _parse_affected_codes(raw: str) -> frozenset[str]:
|
||||
"""Parse ``affected_codes`` — written by ``promote_verdict`` as ``[A, B]``, and accepted bare
|
||||
(``A, B``) for hand-authored files. Empty is an error: a declared-but-contentless code set
|
||||
Jaccard-matches every other empty set, which is a silent mis-key rather than a key."""
|
||||
codes = {_unquote(part) for part in _unquote(raw).strip("[]").split(",")}
|
||||
codes.discard("")
|
||||
if not codes:
|
||||
raise VerdictFrontmatterError("'affected_codes' is declared but empty")
|
||||
return frozenset(codes)
|
||||
|
||||
|
||||
def _parse_claimed_saving(raw: str) -> float:
|
||||
"""Parse ``claimed_saving_nok`` with ``json.loads`` — deliberately the SAME literal rule the IR
|
||||
projection went through, so ``18000`` stays an int and ``18000.0`` a float. ``_mint_id`` hashes
|
||||
the raw value, so a parser that normalised the type would mint a different id for a promoted
|
||||
verdict than for the bundle-keyed seed describing the same candidate."""
|
||||
try:
|
||||
value = json.loads(_unquote(raw))
|
||||
except ValueError as exc:
|
||||
raise VerdictFrontmatterError(f"'claimed_saving_nok' is not a number: {raw!r}") from exc
|
||||
if not isinstance(value, (int, float)) or isinstance(value, bool):
|
||||
raise VerdictFrontmatterError(f"'claimed_saving_nok' is not a number: {raw!r}")
|
||||
return value
|
||||
|
||||
|
||||
def _features_from_verdict_frontmatter(fm: dict[str, str]) -> ProposalFeatures | None:
|
||||
"""The verdict's OWN structural key, or ``None`` when it declares none (caller falls back to the
|
||||
bundle candidate — how every pre-S3.2 seed keeps working).
|
||||
|
||||
ALL THREE fields or none. A partial declaration is refused rather than merged with the bundle
|
||||
candidate, because the merge would mint a key belonging to NEITHER candidate — a synthetic third
|
||||
proposal that retrieves for nothing. Both fully-present and fully-absent are honest; half is not.
|
||||
"""
|
||||
present = [key for key in _STRUCTURAL_FRONTMATTER_KEYS if fm.get(key, "").strip()]
|
||||
if not present:
|
||||
return None
|
||||
if len(present) != len(_STRUCTURAL_FRONTMATTER_KEYS):
|
||||
missing = [k for k in _STRUCTURAL_FRONTMATTER_KEYS if k not in present]
|
||||
raise VerdictFrontmatterError(
|
||||
f"a verdict file declares {present} but not {missing}; a verdict carries its whole "
|
||||
"structural key or none of it (a partial key belongs to no candidate)"
|
||||
)
|
||||
measure_type = _unquote(fm["measure_type"])
|
||||
if not measure_type:
|
||||
raise VerdictFrontmatterError("'measure_type' is declared but empty")
|
||||
return ProposalFeatures(
|
||||
affected_codes=_parse_affected_codes(fm["affected_codes"]),
|
||||
measure_type=measure_type,
|
||||
claimed_saving_nok=_parse_claimed_saving(fm["claimed_saving_nok"]),
|
||||
description=measure_type,
|
||||
)
|
||||
|
||||
|
||||
def _verdict_rationale(fm: dict[str, str]) -> str:
|
||||
"""Build the few-shot rationale from a ``type: verdict`` file's frontmatter, carrying the
|
||||
learning signal the deterministic validator cannot compute (the realization rate + expected
|
||||
|
|
@ -491,20 +564,34 @@ def _verdict_rationale(fm: dict[str, str]) -> str:
|
|||
|
||||
|
||||
def seed_store_from_bundle(bundle_dir: str) -> VerdictStore:
|
||||
"""Build a ``VerdictStore`` from an OKF bundle's ``type: verdict`` files. Each verdict is keyed
|
||||
on the bundle's candidate features (so it retrieves for that measure) and carries the
|
||||
realization signal in its rationale. The seed verdict stands in for the durable HITL verdict a
|
||||
real expert would supply via the same folder interface (målbilde §3)."""
|
||||
features = bundle_candidate_features(bundle_dir)
|
||||
"""Build a ``VerdictStore`` from an OKF bundle's ``type: verdict`` files. Each verdict carries
|
||||
the realization signal in its rationale, and stands in for the durable HITL verdict a real
|
||||
expert would supply via the same folder interface (målbilde §3).
|
||||
|
||||
KEYING (S3.2): a verdict is keyed on ITS OWN candidate when its frontmatter declares the
|
||||
structural fields (``affected_codes`` / ``measure_type`` / ``claimed_saving_nok``), and on the
|
||||
bundle's IR-projection candidate otherwise. The bundle key alone was single-candidate by
|
||||
construction: a bundle holding verdicts about several candidates collapsed them onto one key, so
|
||||
a verdict about candidate B scored a perfect match against candidate A's query and could be
|
||||
folded into A's hypothesis prompt. The fallback is what keeps every pre-S3.2 seed working
|
||||
unchanged; the fields are OPTIONAL, never required."""
|
||||
bundle = okf.navigate_bundle(bundle_dir)
|
||||
verdicts = [
|
||||
capture_verdict(
|
||||
features,
|
||||
vf.frontmatter.get("decision", "approved"),
|
||||
_verdict_rationale(vf.frontmatter),
|
||||
fallback: ProposalFeatures | None = None
|
||||
verdicts = []
|
||||
for vf in bundle.verdicts:
|
||||
features = _features_from_verdict_frontmatter(vf.frontmatter)
|
||||
if features is None:
|
||||
# Read the IR projection lazily: a bundle whose verdicts all carry their own key does
|
||||
# not need one, and this keeps the fallback path's behaviour byte-identical.
|
||||
fallback = fallback if fallback is not None else bundle_candidate_features(bundle_dir)
|
||||
features = fallback
|
||||
verdicts.append(
|
||||
capture_verdict(
|
||||
features,
|
||||
vf.frontmatter.get("decision", "approved"),
|
||||
_verdict_rationale(vf.frontmatter),
|
||||
)
|
||||
)
|
||||
for vf in bundle.verdicts
|
||||
]
|
||||
return VerdictStore(verdicts=verdicts)
|
||||
|
||||
|
||||
|
|
@ -555,11 +642,19 @@ def promote_verdict(
|
|||
wiki. Provenance-stamped (who/which-experiment/when). ``timestamp`` is a required keyword (no
|
||||
wall-clock default) so promotion is deterministic and the stamp reproducible.
|
||||
|
||||
The promoted file is MINIMAL: it does NOT reproduce the hand-authored seed's structured fields
|
||||
(``realization_rate`` etc.) — the raw ``Verdict`` model carries the learning signal only as
|
||||
``rationale`` prose, which becomes the ``description`` frontmatter ``seed_store_from_bundle``
|
||||
folds into ExpeL. The index link uses a NEUTRAL label (``_PROMOTED_LINK_LABEL``), so the signal
|
||||
reaches a prompt only via the gated fold, never via ``bundle_context`` (§3/§6).
|
||||
The promoted file is MINIMAL as to the LEARNING SIGNAL: it does NOT reproduce the hand-authored
|
||||
seed's structured signal fields (``realization_rate`` etc.) — the raw ``Verdict`` model carries
|
||||
that only as ``rationale`` prose, which becomes the ``description`` frontmatter
|
||||
``seed_store_from_bundle`` folds into ExpeL. It DOES carry its own structural KEY (S3.2:
|
||||
``affected_codes`` / ``measure_type`` / ``claimed_saving_nok``), so the next run keys it on the
|
||||
candidate it is about — a promoted verdict is frequently about a different candidate than the
|
||||
one that bundle's IR projection describes. The index link uses a NEUTRAL label
|
||||
(``_PROMOTED_LINK_LABEL``), so the signal reaches a prompt only via the gated fold, never via
|
||||
``bundle_context`` (§3/§6) — and the structural key is signal-free by construction.
|
||||
|
||||
Round-trip caveat: ``render_frontmatter`` single-lines every value, so a ``measure_type``
|
||||
containing newlines is re-read with those collapsed to spaces and would re-mint a different id.
|
||||
Measure strings are single-line in practice; this is stated rather than defended against.
|
||||
|
||||
Known limitation (mirrors ``write_verdict``): ``_mint_id`` keys on the candidate features, so two
|
||||
approved verdicts about the SAME candidate share an id -> share a filename -> last-write-wins;
|
||||
|
|
@ -575,6 +670,16 @@ def promote_verdict(
|
|||
"type": "verdict",
|
||||
"decision": verdict.decision,
|
||||
"description": verdict.rationale,
|
||||
# S3.2: the promoted file carries its OWN structural key, so the next run's
|
||||
# seed_store_from_bundle keys it on the candidate it is actually about rather than on
|
||||
# whichever candidate that bundle's IR projection happens to describe. Written as the three
|
||||
# fields _features_from_verdict_frontmatter reads back — all three, never a partial key.
|
||||
"affected_codes": "[" + ", ".join(sorted(f.affected_codes)) + "]",
|
||||
"measure_type": f.measure_type,
|
||||
# ``str`` of the raw value, NOT a normalised format: it round-trips through
|
||||
# ``_parse_claimed_saving``'s ``json.loads`` back to the same int/float, and ``_mint_id``
|
||||
# hashes that raw value (``18000`` and ``18000.0`` are different keys).
|
||||
"claimed_saving_nok": str(f.claimed_saving_nok),
|
||||
"verdict_id": verdict.id,
|
||||
"provenance": f"godkjent av {approver}; eksperiment {experiment}; {timestamp}",
|
||||
"timestamp": timestamp,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue