open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(simulation): the demo now RUNS the Step-7 file inbox it narrates (P1/S1.a)

Browse source 
The Step-7 trace line said "lang fil-løkke" while the verdict arrived as a
function argument (`verdict_input`) — the short, in-run capture. The long loop
was tested but never exercised by the thing on stage.

An expert now drops a real verdict FILE (`write_verdict`) into an inbox between
the runs, and Run B is given `verdict_dir=`, so `run_project` merges it into the
store before the Step-1 fold.

Not done as the plan point was worded, and the difference is load-bearing:
routing the PERSONA verdict through the inbox would have put ONE marker on two
paths — Step 7 (inbox) and Step 8 (promotion) both end in Run B's prompt, so
either could carry it alone and `test_simulation_loadbearing.py`'s promotion
assertion would have stayed green with promotion detached. A second verdict with
its own marker keeps both seams independently red-able; `simulate_learning_loop`
raises when the two markers are equal. The inbox sits beside the bundle copy,
never inside it, and the id is an explicit sentinel (a minted id would collide
with the promoted verdict's, and `VerdictStore.add` is first-write-wins).

766 -> 769 passed (773 collected). Criterion 6 re-measured: stdout byte-identical
across two runs; stderr unchanged at 6 lines. Mutations measured against the full
suite, four red + a green control: detach `verdict_dir=` · point Run B at an empty
folder while the file is still written · marker set to `realization_rate: 0.82`
(measured present in the verdict seed) · marker set to `energy performance gap`
(measured present in a navigated concept file) · benign rename of the inbox dir.

Honesty limit found while measuring: the last two mutations fell on the causality
assertion, not the Run A control — generation prompts carry the debate output, not
the bundle context. The pair holds, but each assert defends a different property.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FVYDeJ9evZicgU5r3roZVW
This commit is contained in:
Kjell Tore Guttormsen 2026-08-09 13:13:21 +02:00
commit 1e11dcb96c
5 changed files with 252 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

113
src/portfolio_optimiser/simulation.py
View file

@ -41,10 +41,34 @@ from portfolio_optimiser.persona import load_persona_example
from portfolio_optimiser.run import RunResult, run_project
from portfolio_optimiser.shared_root import shared_root
from portfolio_optimiser.validator import ValidatedProposal
from portfolio_optimiser.verdicts import VerdictStore, promote_verdict, seed_store_from_bundle
from portfolio_optimiser.verdicts import (
Verdict,
VerdictStore,
bundle_candidate_features,
promote_verdict,
seed_store_from_bundle,
write_verdict,
)
_PROJECT_ID = "BYGG-KONTOR-NORD"
# --- Step 7, the LONG loop: what an expert drops into the inbox between the two runs -------------
# A SECOND marker, deliberately distinct from the persona's (Step 8, promotion). The two mechanisms
# both end in Run B's hypothesis prompt, so a single shared marker would let either path carry it
# alone — and detaching one seam would then go unnoticed by both load-bearing tests. Absence from
# the bundle is asserted by tests/test_step7_demo_inbox_loadbearing.py, not assumed here.
_INBOX_MARKER = "realiseringsgrad=0.66"
# An explicit sentinel id, never a minted one: ``_mint_id`` hashes the candidate FEATURES, so a
# minted id would collide with the promoted verdict's — and ``VerdictStore.add`` is first-write-wins
# per id, which would silently drop whichever arrived second.
_INBOX_VERDICT_ID = "STEG7-EKSPERT-DRIFTSNOTAT"
_INBOX_RATIONALE = (
"Ettersendt driftsnotat fra energiingeniør, lagt i innboksen etter kjøringen: målingene "
f"fra fyringssesongen viser at realiseringen faller til {_INBOX_MARKER} når "
"tilstedeværelsesstyringen står på fabrikkinnstilling. Godkjent, med den korreksjonen."
)
_INBOX_DIR_NAME = "verdict-innboks"
def _default_bundle_dir() -> Path:
"""The demo bundle under the shared core, resolved at CALL time via ``shared_root()`` (env
@ -256,9 +280,13 @@ def scripted_factory(
@dataclass(frozen=True)
class LearningSimulationResult:
"""The trace of one two-run learning simulation. ``marker_in_run_b_prompt`` true while
``marker_in_run_a_prompt`` false is the closed loop: the persona knowledge approved in Run A
reached Run B's hypothesis only via promotion into the wiki."""
"""The trace of one two-run learning simulation, carrying BOTH feedback timescales.
``marker_in_run_b_prompt`` true while ``marker_in_run_a_prompt`` false is the Step-8 loop: the
persona knowledge approved in Run A reached Run B's hypothesis via promotion into the wiki.
``inbox_marker_*`` is the same shape for the Step-7 LONG loop: a verdict file an expert dropped
into a folder after Run A, merged into Run B's store before the fold. The two are kept apart on
purpose — see ``tests/test_step7_demo_inbox_loadbearing.py``."""
run_a: RunResult
run_b: RunResult
@ -266,6 +294,10 @@ class LearningSimulationResult:
marker: str
marker_in_run_a_prompt: bool
marker_in_run_b_prompt: bool
inbox_path: Path
inbox_marker: str
inbox_marker_in_run_a_prompt: bool
inbox_marker_in_run_b_prompt: bool
run_a_generation_prompts: list[str]
run_b_generation_prompts: list[str]
@ -283,6 +315,8 @@ async def simulate_learning_loop(
project_id: str = _PROJECT_ID,
persona_rationale: str | None = None,
marker: str | None = None,
inbox_rationale: str = _INBOX_RATIONALE,
inbox_marker: str = _INBOX_MARKER,
timestamp: str = "2026-06-30",
max_rounds: int = 3,
) -> LearningSimulationResult:
@ -311,6 +345,16 @@ async def simulate_learning_loop(
persona_rationale = example.rationale
if marker not in persona_rationale:
raise ValueError("marker must be a substring of persona_rationale (the carried payload)")
if inbox_marker not in inbox_rationale:
raise ValueError(
"inbox_marker must be a substring of inbox_rationale (the carried payload)"
)
if inbox_marker == marker:
raise ValueError(
"inbox_marker must differ from marker: the Step-7 inbox and the Step-8 promotion are "
"two mechanisms that both end in Run B's prompt, so a shared marker would let either "
"path carry it alone and make both load-bearing tests vacuous"
)
copy = Path(work_dir) / "bundle"
shutil.copytree(bundle_dir, copy)
@ -346,6 +390,23 @@ async def simulate_learning_loop(
timestamp=timestamp,
)
# Steg 7, the LONG loop: an expert drops a verdict FILE into an inbox AFTER Run A. This is the
# OTHER timescale — no promotion gate, no wiki, just a folder the next run reads. The inbox sits
# beside the bundle copy, never inside it: a verdict file within the bundle would reach Run B as
# navigable context, which is a different mechanism wearing this one's label. ``write_verdict``
# is the same public primitive a human expert would use (målbilde §3: the system READS, the
# expert WRITES) — which is why ``run_project`` is never handed the writing job.
inbox = Path(work_dir) / _INBOX_DIR_NAME
inbox_path = write_verdict(
str(inbox),
Verdict(
id=_INBOX_VERDICT_ID,
proposal_features=bundle_candidate_features(copy_s),
decision="approved",
rationale=inbox_rationale,
),
)
# Re-seed the wiki: the promoted verdict is now navigable and folds into the next run.
store_b = seed_store_from_bundle(copy_s)
@ -360,6 +421,7 @@ async def simulate_learning_loop(
bundle_dir=copy_s,
verdict_input=verdict_input,
store=store_b,
verdict_dir=str(inbox),
client_factory=scripted_factory(replies, sink_b),
max_rounds=max_rounds,
),
@ -374,6 +436,10 @@ async def simulate_learning_loop(
marker=marker,
marker_in_run_a_prompt=any(marker in p for p in gen_a),
marker_in_run_b_prompt=any(marker in p for p in gen_b),
inbox_path=inbox_path,
inbox_marker=inbox_marker,
inbox_marker_in_run_a_prompt=any(inbox_marker in p for p in gen_a),
inbox_marker_in_run_b_prompt=any(inbox_marker in p for p in gen_b),
run_a_generation_prompts=gen_a,
run_b_generation_prompts=gen_b,
)
@ -475,7 +541,10 @@ def _run_trace_lines(result: RunResult, *, marker: str, marker_in_prompt: bool)
lines.extend(_step5_lines(result))
lines.append(" Steg 6 — FORKAST ELLER FORESLÅ (typet utfall forlater kjøringen)")
lines.append(f" {_decision_line(result)}")
lines.append(" Steg 7 — SVAR PÅ TILBAKEMELDING (ekspert-persona, lang fil-løkke)")
# Step 7 has TWO timescales, and only the short one happens INSIDE a run: the expert verdict
# captured while the run is live. The long one — a verdict file dropped into a folder after the
# run — is printed by ``main`` between the two runs, because that is when it happens.
lines.append(" Steg 7 — SVAR PÅ TILBAKEMELDING (ekspert-persona, kort løkke i kjøringen)")
lines.append(f" dom: {result.verdict.decision}")
lines.append(f" begrunnelse: {_clip(result.verdict.rationale, 300)}")
return lines
@ -507,26 +576,48 @@ def main(argv: list[str] | None = None) -> int: # pragma: no cover - console tr
print(line)
print(" (forventet: markøren er FRAVÆRENDE her — dommen finnes ikke i wikien ennå)")
print("\n MELLOM KJØRINGENE — to uavhengige tilbakemeldings-veier tas i bruk")
print("\n Steg 7 (lang løkke) — EN EKSPERT LEGGER EN DOM I INNBOKSEN, ETTER KJØRINGEN")
print(f" fil: {result.inbox_path.parent.name}/{result.inbox_path.name}")
print(f" bærer: {result.inbox_marker} (ettersendt driftsmåling — ny kunnskap)")
print(" rollene byttes aldri: systemet LESER denne mappa, eksperten SKRIVER den")
print(" neste kjøring merger fila inn i minnet FØR hypotesen formes — dager kan gå")
print("\n Steg 8 — PROMOTER GODKJENT KUNNSKAP (gatet wiki-promotering)")
print(f" skrev: {result.promoted_path.name} (lenket i index.md, nøytral etikett)")
print(f" bærer: {result.marker} (personaens dom fra kjøring A)")
print(" gaten er fail-closed: kun en godkjent dom promoteres — rå agent-output aldri")
print("\nKJØRING B (re-seedet kunnskapsbase — leser den promoterte dommen)")
print("\nKJØRING B (re-seedet kunnskapsbase + innboksen lest)")
print(" samme åtte steg kjøres igjen; her vises kun det som ENDRET seg:")
print(f" tidligere dommer hentet for kandidaten: {len(result.run_b.retrieved)}")
print(
f" markør '{result.marker}' i hypotese-prompten: "
f" markør '{result.marker}' (Steg 8, wiki) i hypotese-prompten: "
f"{result.marker_in_run_b_prompt} (forventet True)"
)
print(
f" markør '{result.inbox_marker}' (Steg 7, innboks) i hypotese-prompten: "
f"{result.inbox_marker_in_run_b_prompt} (forventet True)"
)
print(f" utfall: {_decision_line(result.run_b)}")
closed = result.marker_in_run_b_prompt and not result.marker_in_run_a_prompt
# Both paths must close, and neither may have been present in Run A — two markers means the two
# mechanisms are checked separately here, exactly as the load-bearing tests check them.
closed = (
result.marker_in_run_b_prompt
and not result.marker_in_run_a_prompt
and result.inbox_marker_in_run_b_prompt
and not result.inbox_marker_in_run_a_prompt
)
print("\n" + "-" * 78)
if closed:
print("LÆRINGSSLØYFA ER LUKKET: kunnskapen eksperten godkjente i kjøring A nådde kjøring")
print("B's hypotese utelukkende via den fil-baserte wikien (promoter -> re-seed -> fold).")
print("LÆRINGSSLØYFA ER LUKKET, PÅ BEGGE TIDSSKALAER: kunnskapen eksperten godkjente i")
print("kjøring A nådde kjøring B's hypotese via den fil-baserte wikien (promoter ->")
print("re-seed -> fold), OG driftsnotatet som ble lagt i innboksen etterpå nådde den via")
print("fil-innboksen (skriv fil -> merge -> fold). Ingen av dem gikk gjennom minnet.")
else:
print("LÆRINGSSLØYFA ER IKKE LUKKET — markøren krysset ikke kjøringene som forventet.")
print("LÆRINGSSLØYFA ER IKKE LUKKET — en av markørene krysset ikke kjøringene som ventet.")
print("-" * 78)
# The throwaway copy's path is the ONE non-deterministic value here (``mkdtemp``), so it goes to
# stderr: stdout is then byte-identical across runs, which is what the dress rehearsal's