portfolio-optimiser/tests/test_step8_promotion_loadbearing.py
Kjell Tore Guttormsen 6b645ad32a feat(fase6): gate-promote approved verdicts back into the OKF wiki (Steg 8)
Close the last agentic-loop seam (målbilde §3/§6/§7/§11 step 6): an
APPROVED verdict is promoted from the raw output layer into the context
layer (the OKF bundle) as a navigable `type: verdict` concept file, so
human/persona-approved knowledge reaches the next run's hypothesis.

- okf.py (pure stdlib, MAF-free): render_frontmatter / write_concept_file
  / link_in_index — the D7-portable OKF write counterpart of navigate.
- verdicts.py: promote_verdict + PromotionRefused gate (fail-closed; only
  approved decisions enter the wiki, never raw agent output), provenance
  stamp (who/experiment/when; timestamp a required kwarg), neutral index
  label (signal reaches a prompt only via the gated ExpeL fold, never
  bundle_context), _safe_filename_token (id sanitised for path/link).
- R4 = optional+gated: a public opt-in primitive, NOT wired into
  run_project (mirrors write_verdict — the system reads, the gate promotes).
- Load-bearing trio (test_step8_promotion_loadbearing.py): gate refuses a
  non-approved verdict, approved verdict is navigable, promoted signal
  stays out of the read-context — all proven RED-on-detach. Suite 144->148.

Design hardened by an adversarial plan-critic (12 findings; the BLOCKER —
index-link leak into bundle_context via index_summary — closed by the
neutral label + a no-leak test). Honesty limits documented: promoted file
is minimal (signal as prose only), and the learning-key id means
same-candidate approvals share a filename (last-write-wins).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MHR8iKxJRxDiDfNw8HZmWE
2026-06-30 11:06:28 +02:00

174 lines
8.1 KiB
Python

"""Step-8 load-bearing seam (målbilde §3 / §6 / §7 / §11 step 6): GATED wiki-promotion — when an
expert/persona APPROVES an outcome, it is promoted from the raw output layer into the context layer
(the OKF bundle) as a ``type: verdict`` concept file; a NON-approved verdict MUST NOT reach the wiki.
The gap (fot-i-bakken): an approved verdict never re-entered the context layer, so the loop learned
only via injected/dropped seeds. Step 8 adds ``promote_verdict`` — the public, opt-in (R4) promotion
primitive, deliberately NOT wired into ``run_project`` (the system reads context; the gate promotes).
Three load-bearing tests (per the Fase-2 green-but-dead trap):
- Test A (GATE, §7 mandatory): a rejected verdict — otherwise fully promotable — raises
``PromotionRefused`` and writes/links NOTHING. Goes RED the moment the gate (the ``raise``) is
detached: the rejected verdict's file + index link then appear (self-contamination, §6).
- Test B (NAVIGABLE): an approved verdict is written AND linked so ``navigate_bundle`` reaches it —
proving promotion is non-decorative (a file the loop cannot navigate never reaches the next run).
Goes RED if ``link_in_index`` is detached (the promoted file ∉ ``bundle.verdicts``).
- Test C (NO-LEAK): the promoted verdict's realization marker (0.57 — absent from every bundle file)
stays OUT of ``bundle_context`` — it reaches a prompt only via the gated ExpeL fold, never the
read-context. Goes RED if ``promote_verdict`` passes the rationale (not the neutral label) into
the index, leaking the signal into ``index_summary`` -> context (the BLOCKER the design closes).
Transitive coverage: ``test_step1_expel_loadbearing.py`` already proves
``seed_store_from_bundle`` -> ExpeL fold -> prompt; Test B proves the promoted verdict ∈
``bundle.verdicts`` (what that seeder reads). Together: promoted approved knowledge reaches the next
run's hypothesis — proven without re-running ``run_project``.
"""
from __future__ import annotations
import shutil
from pathlib import Path
import pytest
from portfolio_optimiser import okf
from portfolio_optimiser.verdicts import (
ProposalFeatures,
PromotionRefused,
Verdict,
bundle_candidate_features,
promote_verdict,
)
BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
# A realization marker absent from EVERY bundle file (the seed is 0.82). It can reach a prompt only
# via the ingest -> ExpeL fold path; in the read-context it must NEVER appear (Test C).
_MARKER = "realiseringsgrad=0.57"
_APPROVED_ID = "STEG8-APPROVED"
def _copy_bundle(tmp_path: Path) -> str:
"""Promotion WRITES into the bundle, so every test works on a throwaway copy — the shared
framework-neutral fixture is never mutated."""
dst = tmp_path / "bundle"
shutil.copytree(BUNDLE_DIR, dst)
return str(dst)
def _approved_verdict(bundle_dir: str, *, vid: str = _APPROVED_ID) -> Verdict:
"""An approved verdict keyed on the bundle candidate, carrying the realization marker in its
rationale — the same key/payload shape the Step-1 fold consumes."""
return Verdict(
id=vid,
proposal_features=bundle_candidate_features(bundle_dir),
decision="approved",
rationale=f"LED-retrofit godkjent med realiseringskorreksjon ({_MARKER})",
)
def _promoted_files(bundle_dir: str) -> list[str]:
return [p.name for p in Path(bundle_dir).glob("promoted-verdict-*.md")]
# --- Test A: the gate ---------------------------------------------------------------------------
def test_rejected_verdict_is_not_promoted(tmp_path) -> None:
"""LOAD-BEARING GATE (§7): a NON-approved verdict — otherwise fully promotable (valid id,
writable bundle) so detaching the gate WOULD produce a file+link — raises ``PromotionRefused``
and leaves the wiki untouched. RED if the gate is removed."""
bundle_dir = _copy_bundle(tmp_path)
index_before = (Path(bundle_dir) / "index.md").read_text(encoding="utf-8")
rejected = Verdict(
id="STEG8-REJECTED",
proposal_features=ProposalFeatures(
affected_codes=frozenset({"ENERGI-TOTAL-EL"}),
measure_type="LED-retrofit",
claimed_saving_nok=30000,
),
decision="rejected",
rationale="expert rejected: realiseringsgapet for stort i denne konteksten",
)
with pytest.raises(PromotionRefused):
promote_verdict(
bundle_dir, rejected, approver="persona", experiment="exp-A", timestamp="2026-06-30"
)
assert _promoted_files(bundle_dir) == [], "a rejected verdict must NOT be written to the wiki"
assert (Path(bundle_dir) / "index.md").read_text(encoding="utf-8") == index_before, (
"a rejected verdict must NOT be linked into the index"
)
# --- Test B: positive promotion is navigable ----------------------------------------------------
def test_approved_verdict_is_promoted_and_navigable(tmp_path) -> None:
"""LOAD-BEARING NAVIGABILITY: an approved verdict is written as a ``type: verdict`` file AND
linked so ``navigate_bundle`` reaches it (the seeder ``seed_store_from_bundle`` reads exactly
``bundle.verdicts``). RED if ``link_in_index`` is detached."""
bundle_dir = _copy_bundle(tmp_path)
path = promote_verdict(
bundle_dir,
_approved_verdict(bundle_dir),
approver="persona",
experiment="exp-B",
timestamp="2026-06-30",
)
assert path.name == "promoted-verdict-STEG8-APPROVED.md"
fm = okf.parse_frontmatter(path)
assert fm["type"] == "verdict"
assert fm["decision"] == "approved"
assert fm["verdict_id"] == _APPROVED_ID
assert "persona" in fm["provenance"] and "exp-B" in fm["provenance"]
verdict_names = {f.name for f in okf.navigate_bundle(bundle_dir).verdicts}
assert path.name in verdict_names, (
"the promoted verdict is not navigable — link_in_index did not make it reachable, so it "
"could never reach a later run (decorative promotion)"
)
def test_promote_sanitizes_unsafe_verdict_id(tmp_path) -> None:
"""A verdict id is read verbatim and can be an arbitrary author string; it must be sanitized
before it becomes a filename/link, or a ``/`` makes the link unnavigable and ``..`` escapes the
bundle. The original id is preserved in the ``verdict_id`` frontmatter."""
bundle_dir = _copy_bundle(tmp_path)
nasty = _approved_verdict(bundle_dir, vid="../../etc/evil id")
path = promote_verdict(bundle_dir, nasty, approver="p", experiment="e", timestamp="2026-06-30")
assert path.parent == Path(bundle_dir) # written INSIDE the bundle, not escaped
assert "/" not in path.name.replace("promoted-verdict-", "").replace(".md", "")
assert path.name in {f.name for f in okf.navigate_bundle(bundle_dir).verdicts}
assert okf.parse_frontmatter(path)["verdict_id"] == "../../etc/evil id" # original preserved
# --- Test C: the promoted signal does NOT leak into the read-context -----------------------------
def test_promoted_signal_stays_out_of_bundle_context(tmp_path) -> None:
"""LOAD-BEARING NO-LEAK (§3/§6): after promotion the realization marker is in the bundle (in the
promoted ``type: verdict`` file) but ABSENT from ``bundle_context`` — it reaches a prompt only
via the gated ExpeL fold, never the read-context. RED if ``promote_verdict`` passes the rationale
(not the neutral label) into the index link, leaking the marker into ``index_summary``."""
bundle_dir = _copy_bundle(tmp_path)
path = promote_verdict(
bundle_dir,
_approved_verdict(bundle_dir),
approver="persona",
experiment="exp-C",
timestamp="2026-06-30",
)
bundle = okf.navigate_bundle(bundle_dir)
context = okf.bundle_context(bundle)
assert _MARKER not in context, (
"the promoted verdict's realization signal leaked into the read-context — promotion must "
"pass a NEUTRAL index label, so the signal reaches a prompt only via the gated ExpeL fold"
)
# though it IS present in the bundle (just excluded from context, like the seed verdict):
assert _MARKER in okf.parse_frontmatter(path)["description"]
assert path.name in {f.name for f in bundle.verdicts}