feat(simulation): the demo now RUNS the Step-7 file inbox it narrates (P1/S1.a)
The Step-7 trace line said "lang fil-løkke" while the verdict arrived as a function argument (`verdict_input`) — the short, in-run capture. The long loop was tested but never exercised by the thing on stage. An expert now drops a real verdict FILE (`write_verdict`) into an inbox between the runs, and Run B is given `verdict_dir=`, so `run_project` merges it into the store before the Step-1 fold. Not done as the plan point was worded, and the difference is load-bearing: routing the PERSONA verdict through the inbox would have put ONE marker on two paths — Step 7 (inbox) and Step 8 (promotion) both end in Run B's prompt, so either could carry it alone and `test_simulation_loadbearing.py`'s promotion assertion would have stayed green with promotion detached. A second verdict with its own marker keeps both seams independently red-able; `simulate_learning_loop` raises when the two markers are equal. The inbox sits beside the bundle copy, never inside it, and the id is an explicit sentinel (a minted id would collide with the promoted verdict's, and `VerdictStore.add` is first-write-wins). 766 -> 769 passed (773 collected). Criterion 6 re-measured: stdout byte-identical across two runs; stderr unchanged at 6 lines. Mutations measured against the full suite, four red + a green control: detach `verdict_dir=` · point Run B at an empty folder while the file is still written · marker set to `realization_rate: 0.82` (measured present in the verdict seed) · marker set to `energy performance gap` (measured present in a navigated concept file) · benign rename of the inbox dir. Honesty limit found while measuring: the last two mutations fell on the causality assertion, not the Run A control — generation prompts carry the debate output, not the bundle context. The pair holds, but each assert defends a different property. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FVYDeJ9evZicgU5r3roZVW
This commit is contained in:
parent
d0571ca408
commit
1e11dcb96c
5 changed files with 252 additions and 15 deletions
15
CLAUDE.md
15
CLAUDE.md
|
|
@ -267,6 +267,21 @@ Python ≥3.10. MAF (`agent-framework-core` 1.9.0). Pakkehåndtering: `uv`. To b
|
|||
lever på Claude-SDK-siden (minimal API-kjøring). Skriptet klient = MAF-side stillas, IKKE delt
|
||||
(`shared/` forblir framework-nøytralt). Kjøres `uv run python -m portfolio_optimiser.simulation`.
|
||||
Load-bearing: `tests/test_simulation_loadbearing.py` blir RØD når promoteringen detaches.
|
||||
- **Demoen KJØRER begge tidsskalaer, og de bæres av HVER SIN markør (P1/S1.a):** Steg 7-linja sa
|
||||
«lang fil-løkke», men `simulate_learning_loop` kalte `run_project` UTEN `verdict_dir` — dommen kom
|
||||
som funksjonsargument (`verdict_input`, den KORTE i-kjøring-fangsten). Nå skriver en ekspert en
|
||||
faktisk fil (`write_verdict`) i en innboks MELLOM kjøringene, og Run B får `verdict_dir=`.
|
||||
**Hvorfor en ANDRE markør og ikke persona-dommen gjennom innboksen:** Steg 7 (innboks) og Steg 8
|
||||
(promotering) er to ULIKE mekanismer som begge ender i Run B's hypotese-prompt — med én delt
|
||||
markør kunne hver av dem båret den alene, og `test_simulation_loadbearing.py`s promoterings-assert
|
||||
ville stått GRØNN med promoteringen detached, altså blitt vakuøs. `simulate_learning_loop` raiser
|
||||
derfor `ValueError` når `inbox_marker == marker`. Innboksen ligger VED SIDEN AV bundle-kopien,
|
||||
aldri inni: en dom-fil inne i bundelen når neste kjøring som navigerbar kontekst, som er en annen
|
||||
mekanisme i denne sin forkledning. Sentinel-`id` (aldri myntet) — `_mint_id` hasher kandidat-
|
||||
featurene, så en myntet id kolliderer med den promoterte dommens, og `VerdictStore.add` er
|
||||
first-write-wins per id. Load-bearing MÅLT (`tests/test_step7_demo_inbox_loadbearing.py`), tre
|
||||
mutasjoner røde + grønn kontroll: detach `verdict_dir=` · la Run B lese en TOM mappe · sett
|
||||
innboks-markøren til en verdi som FINNES i bundelen.
|
||||
- **Det skriptede manuset nøkles på PROSJEKT-ID-en, og det er MÅLT:** `scripted_proposer(candidates)`
|
||||
bygger simuleringens proposer fra et `ScriptedCandidate`-register, så et nytt prosjekt er en
|
||||
data-oppføring (demo-uke-plan §4 risiko 2) — ikke et andre håndskrevet manus. **Hvorfor ikke
|
||||
|
|
|
|||
10
README.md
10
README.md
|
|
@ -58,9 +58,13 @@ uv run python -m portfolio_optimiser.simulation
|
|||
```
|
||||
|
||||
Each run shows the refinement step: the proposer's first claim is falsified by the deterministic
|
||||
validator, and the corrected claim validates. The trace then ends with the approved verdict's marker
|
||||
present in Run B's prompt and absent from Run A's — knowledge crossing runs purely through the
|
||||
file-backed wiki (promote → re-seed → fold).
|
||||
validator, and the corrected claim validates. Between the runs, knowledge travels back on **both
|
||||
feedback timescales, each carrying its own marker** so neither can stand in for the other: the
|
||||
persona's approved verdict is promoted into the file-backed wiki (promote → re-seed → fold), and a
|
||||
second verdict — an operations note an expert drops into an inbox folder *after* the run, the way a
|
||||
reviewer would days later — is merged from disk into the next run's store before its hypothesis is
|
||||
formed (write file → merge → fold). Both markers are present in Run B's prompt and absent from
|
||||
Run A's. Nothing crosses in memory.
|
||||
|
||||
**3 — Run the loop over a knowledge base, with answers you supply.** Write the stand-in replies,
|
||||
then point the CLI at the bundle:
|
||||
|
|
|
|||
|
|
@ -135,7 +135,7 @@ det. Stderr-støyen (to warnings + to «forcing completion») er det første pub
|
|||
|
||||
## 2. Plan FØR demoen (P1–P4, i utførelsesrekkefølge)
|
||||
|
||||
### P1 — Steg 7-innboksen kobles på i demoen ☐
|
||||
### P1 — Steg 7-innboksen kobles på i demoen ✔ (2026-08-09)
|
||||
**[1 økt · Opus 5/high · TDD direkte · MÅ lande før onsdags-frysen]**
|
||||
Rute persona-dommen gjennom en faktisk `verdict_dir`-katalog i `simulate_learning_loop` —
|
||||
sømmene finnes allerede (`run_project(verdict_dir=…)` + `write_verdict` er offentlig primitiv).
|
||||
|
|
@ -144,6 +144,33 @@ Ny load-bearing-test: detach innboks-lesingen → markør-/Steg 7-linja endres
|
|||
**Faller den på tid:** minimumsvarianten = ærlig etikett i `_run_trace_lines` («dom levert
|
||||
direkte her; fil-innboksen er samme søm, bevist i test») + én muntlig setning. 0,1 økt.
|
||||
|
||||
**UTFØRT — men IKKE slik punktet var formulert, og forskjellen er bærende.** Å rute
|
||||
*persona-dommen* gjennom innboksen ville gitt ÉN markør på to veier: Steg 7 (innboks) og Steg 8
|
||||
(promotering) ender begge i Run B's hypotese-prompt, så hver av dem kunne båret markøren alene —
|
||||
og `test_simulation_loadbearing.py`s promoterings-assert ville stått GRØNN med promoteringen
|
||||
detached. Punktet ville altså gjort en eksisterende load-bearing test vakuøs for å lukke seg selv.
|
||||
Utført i stedet: en ANDRE dom, med sin egen markør (`realiseringsgrad=0.66`), skrives som fil med
|
||||
`write_verdict` MELLOM kjøringene, og Run B får `verdict_dir=`. `simulate_learning_loop` raiser
|
||||
`ValueError` hvis de to markørene er like. 766 → 769 grønne (773 kollektert).
|
||||
- **Kriterium 6 re-målt:** stdout byte-identisk over to kjøringer (`diff` tomt). Stderr uendret
|
||||
6 linjer. *(Lærdom: første stderr-måling ga 62 linjer — `2>&1 >/dev/null` under zsh MULTIOS
|
||||
blander stdout inn. Formen som holder er `>/dev/null 2>fil`. Målefeilen, ikke koden.)*
|
||||
- **Mutasjoner MÅLT mot hele suiten (~110 s hver), fire røde + grønn kontroll:** detach
|
||||
`verdict_dir=` (2 røde) · la Run B lese en TOM mappe mens fila fortsatt skrives (2 røde) ·
|
||||
markør = `realization_rate: 0.82`, målt til stede i dom-frøet (1 rød) · markør =
|
||||
`energy performance gap`, målt til stede i en navigert konseptfil (1 rød) · godartet omdøping av
|
||||
innboks-katalogen (grønn kontroll).
|
||||
- **Ærlighets-grense funnet UNDER målingen:** de to siste mutasjonene felte
|
||||
kausalitets-asserten (markøren finnes i bundelen), IKKE Run A-kontrollen — fordi
|
||||
genererings-prompten bærer `Project: {id} - {name}` + debatt-outputen, ikke bundle-konteksten.
|
||||
Run A-kontrollen kan altså ikke alene fange en bundle-tilstedeværende markør; det er
|
||||
kausalitets-asserten som lukker det hullet. Paret holder, men det er verdt å vite hvilken av
|
||||
dem som faktisk bærer hvilken egenskap.
|
||||
- **Lærdom, samme klasse som 08-06:** første markør-mutasjon satte `realiseringsgrad=0.82` og gikk
|
||||
GRØNN — bundelen bærer `realization_rate: 0.82`, ikke den strengen. Mutasjonen endret ingen
|
||||
betingelse og beviste ingenting. En mutasjon må måles mot hva fila FAKTISK inneholder, ikke mot
|
||||
hva STATE kaller verdien.
|
||||
|
||||
### P2 — Spor B: innholdsgaten ☐
|
||||
**[1–2 økter · Opus 5/high · TDD direkte · følger innholdsgate-planen §3–§4 uendret]**
|
||||
Reviewens skjerpelse, ellers ingen endring: §4-beslutning 2 (avvisning per dokument eller per
|
||||
|
|
|
|||
|
|
@ -41,10 +41,34 @@ from portfolio_optimiser.persona import load_persona_example
|
|||
from portfolio_optimiser.run import RunResult, run_project
|
||||
from portfolio_optimiser.shared_root import shared_root
|
||||
from portfolio_optimiser.validator import ValidatedProposal
|
||||
from portfolio_optimiser.verdicts import VerdictStore, promote_verdict, seed_store_from_bundle
|
||||
from portfolio_optimiser.verdicts import (
|
||||
Verdict,
|
||||
VerdictStore,
|
||||
bundle_candidate_features,
|
||||
promote_verdict,
|
||||
seed_store_from_bundle,
|
||||
write_verdict,
|
||||
)
|
||||
|
||||
_PROJECT_ID = "BYGG-KONTOR-NORD"
|
||||
|
||||
# --- Step 7, the LONG loop: what an expert drops into the inbox between the two runs -------------
|
||||
# A SECOND marker, deliberately distinct from the persona's (Step 8, promotion). The two mechanisms
|
||||
# both end in Run B's hypothesis prompt, so a single shared marker would let either path carry it
|
||||
# alone — and detaching one seam would then go unnoticed by both load-bearing tests. Absence from
|
||||
# the bundle is asserted by tests/test_step7_demo_inbox_loadbearing.py, not assumed here.
|
||||
_INBOX_MARKER = "realiseringsgrad=0.66"
|
||||
# An explicit sentinel id, never a minted one: ``_mint_id`` hashes the candidate FEATURES, so a
|
||||
# minted id would collide with the promoted verdict's — and ``VerdictStore.add`` is first-write-wins
|
||||
# per id, which would silently drop whichever arrived second.
|
||||
_INBOX_VERDICT_ID = "STEG7-EKSPERT-DRIFTSNOTAT"
|
||||
_INBOX_RATIONALE = (
|
||||
"Ettersendt driftsnotat fra energiingeniør, lagt i innboksen etter kjøringen: målingene "
|
||||
f"fra fyringssesongen viser at realiseringen faller til {_INBOX_MARKER} når "
|
||||
"tilstedeværelsesstyringen står på fabrikkinnstilling. Godkjent, med den korreksjonen."
|
||||
)
|
||||
_INBOX_DIR_NAME = "verdict-innboks"
|
||||
|
||||
|
||||
def _default_bundle_dir() -> Path:
|
||||
"""The demo bundle under the shared core, resolved at CALL time via ``shared_root()`` (env
|
||||
|
|
@ -256,9 +280,13 @@ def scripted_factory(
|
|||
|
||||
@dataclass(frozen=True)
|
||||
class LearningSimulationResult:
|
||||
"""The trace of one two-run learning simulation. ``marker_in_run_b_prompt`` true while
|
||||
``marker_in_run_a_prompt`` false is the closed loop: the persona knowledge approved in Run A
|
||||
reached Run B's hypothesis only via promotion into the wiki."""
|
||||
"""The trace of one two-run learning simulation, carrying BOTH feedback timescales.
|
||||
|
||||
``marker_in_run_b_prompt`` true while ``marker_in_run_a_prompt`` false is the Step-8 loop: the
|
||||
persona knowledge approved in Run A reached Run B's hypothesis via promotion into the wiki.
|
||||
``inbox_marker_*`` is the same shape for the Step-7 LONG loop: a verdict file an expert dropped
|
||||
into a folder after Run A, merged into Run B's store before the fold. The two are kept apart on
|
||||
purpose — see ``tests/test_step7_demo_inbox_loadbearing.py``."""
|
||||
|
||||
run_a: RunResult
|
||||
run_b: RunResult
|
||||
|
|
@ -266,6 +294,10 @@ class LearningSimulationResult:
|
|||
marker: str
|
||||
marker_in_run_a_prompt: bool
|
||||
marker_in_run_b_prompt: bool
|
||||
inbox_path: Path
|
||||
inbox_marker: str
|
||||
inbox_marker_in_run_a_prompt: bool
|
||||
inbox_marker_in_run_b_prompt: bool
|
||||
run_a_generation_prompts: list[str]
|
||||
run_b_generation_prompts: list[str]
|
||||
|
||||
|
|
@ -283,6 +315,8 @@ async def simulate_learning_loop(
|
|||
project_id: str = _PROJECT_ID,
|
||||
persona_rationale: str | None = None,
|
||||
marker: str | None = None,
|
||||
inbox_rationale: str = _INBOX_RATIONALE,
|
||||
inbox_marker: str = _INBOX_MARKER,
|
||||
timestamp: str = "2026-06-30",
|
||||
max_rounds: int = 3,
|
||||
) -> LearningSimulationResult:
|
||||
|
|
@ -311,6 +345,16 @@ async def simulate_learning_loop(
|
|||
persona_rationale = example.rationale
|
||||
if marker not in persona_rationale:
|
||||
raise ValueError("marker must be a substring of persona_rationale (the carried payload)")
|
||||
if inbox_marker not in inbox_rationale:
|
||||
raise ValueError(
|
||||
"inbox_marker must be a substring of inbox_rationale (the carried payload)"
|
||||
)
|
||||
if inbox_marker == marker:
|
||||
raise ValueError(
|
||||
"inbox_marker must differ from marker: the Step-7 inbox and the Step-8 promotion are "
|
||||
"two mechanisms that both end in Run B's prompt, so a shared marker would let either "
|
||||
"path carry it alone and make both load-bearing tests vacuous"
|
||||
)
|
||||
|
||||
copy = Path(work_dir) / "bundle"
|
||||
shutil.copytree(bundle_dir, copy)
|
||||
|
|
@ -346,6 +390,23 @@ async def simulate_learning_loop(
|
|||
timestamp=timestamp,
|
||||
)
|
||||
|
||||
# Steg 7, the LONG loop: an expert drops a verdict FILE into an inbox AFTER Run A. This is the
|
||||
# OTHER timescale — no promotion gate, no wiki, just a folder the next run reads. The inbox sits
|
||||
# beside the bundle copy, never inside it: a verdict file within the bundle would reach Run B as
|
||||
# navigable context, which is a different mechanism wearing this one's label. ``write_verdict``
|
||||
# is the same public primitive a human expert would use (målbilde §3: the system READS, the
|
||||
# expert WRITES) — which is why ``run_project`` is never handed the writing job.
|
||||
inbox = Path(work_dir) / _INBOX_DIR_NAME
|
||||
inbox_path = write_verdict(
|
||||
str(inbox),
|
||||
Verdict(
|
||||
id=_INBOX_VERDICT_ID,
|
||||
proposal_features=bundle_candidate_features(copy_s),
|
||||
decision="approved",
|
||||
rationale=inbox_rationale,
|
||||
),
|
||||
)
|
||||
|
||||
# Re-seed the wiki: the promoted verdict is now navigable and folds into the next run.
|
||||
store_b = seed_store_from_bundle(copy_s)
|
||||
|
||||
|
|
@ -360,6 +421,7 @@ async def simulate_learning_loop(
|
|||
bundle_dir=copy_s,
|
||||
verdict_input=verdict_input,
|
||||
store=store_b,
|
||||
verdict_dir=str(inbox),
|
||||
client_factory=scripted_factory(replies, sink_b),
|
||||
max_rounds=max_rounds,
|
||||
),
|
||||
|
|
@ -374,6 +436,10 @@ async def simulate_learning_loop(
|
|||
marker=marker,
|
||||
marker_in_run_a_prompt=any(marker in p for p in gen_a),
|
||||
marker_in_run_b_prompt=any(marker in p for p in gen_b),
|
||||
inbox_path=inbox_path,
|
||||
inbox_marker=inbox_marker,
|
||||
inbox_marker_in_run_a_prompt=any(inbox_marker in p for p in gen_a),
|
||||
inbox_marker_in_run_b_prompt=any(inbox_marker in p for p in gen_b),
|
||||
run_a_generation_prompts=gen_a,
|
||||
run_b_generation_prompts=gen_b,
|
||||
)
|
||||
|
|
@ -475,7 +541,10 @@ def _run_trace_lines(result: RunResult, *, marker: str, marker_in_prompt: bool)
|
|||
lines.extend(_step5_lines(result))
|
||||
lines.append(" Steg 6 — FORKAST ELLER FORESLÅ (typet utfall forlater kjøringen)")
|
||||
lines.append(f" {_decision_line(result)}")
|
||||
lines.append(" Steg 7 — SVAR PÅ TILBAKEMELDING (ekspert-persona, lang fil-løkke)")
|
||||
# Step 7 has TWO timescales, and only the short one happens INSIDE a run: the expert verdict
|
||||
# captured while the run is live. The long one — a verdict file dropped into a folder after the
|
||||
# run — is printed by ``main`` between the two runs, because that is when it happens.
|
||||
lines.append(" Steg 7 — SVAR PÅ TILBAKEMELDING (ekspert-persona, kort løkke i kjøringen)")
|
||||
lines.append(f" dom: {result.verdict.decision}")
|
||||
lines.append(f" begrunnelse: {_clip(result.verdict.rationale, 300)}")
|
||||
return lines
|
||||
|
|
@ -507,26 +576,48 @@ def main(argv: list[str] | None = None) -> int: # pragma: no cover - console tr
|
|||
print(line)
|
||||
print(" (forventet: markøren er FRAVÆRENDE her — dommen finnes ikke i wikien ennå)")
|
||||
|
||||
print("\n MELLOM KJØRINGENE — to uavhengige tilbakemeldings-veier tas i bruk")
|
||||
|
||||
print("\n Steg 7 (lang løkke) — EN EKSPERT LEGGER EN DOM I INNBOKSEN, ETTER KJØRINGEN")
|
||||
print(f" fil: {result.inbox_path.parent.name}/{result.inbox_path.name}")
|
||||
print(f" bærer: {result.inbox_marker} (ettersendt driftsmåling — ny kunnskap)")
|
||||
print(" rollene byttes aldri: systemet LESER denne mappa, eksperten SKRIVER den")
|
||||
print(" neste kjøring merger fila inn i minnet FØR hypotesen formes — dager kan gå")
|
||||
|
||||
print("\n Steg 8 — PROMOTER GODKJENT KUNNSKAP (gatet wiki-promotering)")
|
||||
print(f" skrev: {result.promoted_path.name} (lenket i index.md, nøytral etikett)")
|
||||
print(f" bærer: {result.marker} (personaens dom fra kjøring A)")
|
||||
print(" gaten er fail-closed: kun en godkjent dom promoteres — rå agent-output aldri")
|
||||
|
||||
print("\nKJØRING B (re-seedet kunnskapsbase — leser den promoterte dommen)")
|
||||
print("\nKJØRING B (re-seedet kunnskapsbase + innboksen lest)")
|
||||
print(" samme åtte steg kjøres igjen; her vises kun det som ENDRET seg:")
|
||||
print(f" tidligere dommer hentet for kandidaten: {len(result.run_b.retrieved)}")
|
||||
print(
|
||||
f" markør '{result.marker}' i hypotese-prompten: "
|
||||
f" markør '{result.marker}' (Steg 8, wiki) i hypotese-prompten: "
|
||||
f"{result.marker_in_run_b_prompt} (forventet True)"
|
||||
)
|
||||
print(
|
||||
f" markør '{result.inbox_marker}' (Steg 7, innboks) i hypotese-prompten: "
|
||||
f"{result.inbox_marker_in_run_b_prompt} (forventet True)"
|
||||
)
|
||||
print(f" utfall: {_decision_line(result.run_b)}")
|
||||
|
||||
closed = result.marker_in_run_b_prompt and not result.marker_in_run_a_prompt
|
||||
# Both paths must close, and neither may have been present in Run A — two markers means the two
|
||||
# mechanisms are checked separately here, exactly as the load-bearing tests check them.
|
||||
closed = (
|
||||
result.marker_in_run_b_prompt
|
||||
and not result.marker_in_run_a_prompt
|
||||
and result.inbox_marker_in_run_b_prompt
|
||||
and not result.inbox_marker_in_run_a_prompt
|
||||
)
|
||||
print("\n" + "-" * 78)
|
||||
if closed:
|
||||
print("LÆRINGSSLØYFA ER LUKKET: kunnskapen eksperten godkjente i kjøring A nådde kjøring")
|
||||
print("B's hypotese utelukkende via den fil-baserte wikien (promoter -> re-seed -> fold).")
|
||||
print("LÆRINGSSLØYFA ER LUKKET, PÅ BEGGE TIDSSKALAER: kunnskapen eksperten godkjente i")
|
||||
print("kjøring A nådde kjøring B's hypotese via den fil-baserte wikien (promoter ->")
|
||||
print("re-seed -> fold), OG driftsnotatet som ble lagt i innboksen etterpå nådde den via")
|
||||
print("fil-innboksen (skriv fil -> merge -> fold). Ingen av dem gikk gjennom minnet.")
|
||||
else:
|
||||
print("LÆRINGSSLØYFA ER IKKE LUKKET — markøren krysset ikke kjøringene som forventet.")
|
||||
print("LÆRINGSSLØYFA ER IKKE LUKKET — en av markørene krysset ikke kjøringene som ventet.")
|
||||
print("-" * 78)
|
||||
# The throwaway copy's path is the ONE non-deterministic value here (``mkdtemp``), so it goes to
|
||||
# stderr: stdout is then byte-identical across runs, which is what the dress rehearsal's
|
||||
|
|
|
|||
100
tests/test_step7_demo_inbox_loadbearing.py
Normal file
100
tests/test_step7_demo_inbox_loadbearing.py
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
"""Step-7 in the DEMO: the simulation must actually USE the async file inbox it claims to show.
|
||||
|
||||
The gap (egnethetsreview Funn 2, measured): ``simulation._run_trace_lines`` labels Step 7 "lang
|
||||
fil-løkke", but ``simulate_learning_loop`` called ``run_project`` WITHOUT ``verdict_dir`` — the
|
||||
persona verdict reached the run as a function argument (``verdict_input``, the SHORT in-run
|
||||
capture). The long loop was tested (``test_step7_async_loop_loadbearing.py``) but never exercised
|
||||
by the thing on stage. Same class as Step 5 before 2026-08-07: a step that is *narrated* is not
|
||||
thereby *shown*.
|
||||
|
||||
The fix routes a SECOND verdict through a real folder: after Run A an expert drops a verdict FILE
|
||||
into an inbox (``write_verdict``), and Run B is given ``verdict_dir=`` so ``run_project`` merges it
|
||||
into the store BEFORE the Step-1 fold.
|
||||
|
||||
Why a second, separately-keyed marker instead of routing the persona verdict through the inbox:
|
||||
Step 7 (file inbox) and Step 8 (wiki promotion) are two DIFFERENT mechanisms that both end in Run
|
||||
B's hypothesis prompt. Had one marker travelled both paths, either path could carry it alone —
|
||||
``test_simulation_loadbearing.py``'s promotion assertion would then stay green with promotion
|
||||
detached, i.e. it would become vacuous. Two markers keep each seam independently red-able, which is
|
||||
the whole point of a load-bearing test.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from portfolio_optimiser.simulation import simulate_learning_loop
|
||||
from portfolio_optimiser.validator import ValidatedProposal
|
||||
|
||||
BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
|
||||
|
||||
|
||||
async def test_dropped_inbox_verdict_reaches_run_b_in_the_simulation(tmp_path) -> None:
|
||||
"""LOAD-BEARING: a verdict file dropped into the inbox AFTER Run A reaches Run B's hypothesis
|
||||
prompt in the SIMULATION — the demo's Step-7 claim, executed rather than narrated. Goes RED the
|
||||
moment ``verdict_dir=`` is detached from the simulation's Run B (the marker never arrives).
|
||||
|
||||
The Run A control is causality, not decoration: the file does not exist during Run A, so its
|
||||
marker must be absent there. If it were present in both, the positive assertion would be
|
||||
measuring the bundle, not the loop."""
|
||||
result = await simulate_learning_loop(str(BUNDLE_DIR), str(tmp_path))
|
||||
|
||||
assert isinstance(result.run_a.outcome, ValidatedProposal)
|
||||
assert isinstance(result.run_b.outcome, ValidatedProposal)
|
||||
|
||||
assert not result.inbox_marker_in_run_a_prompt, (
|
||||
"Run A carried the inbox marker before the file existed — the positive result below would "
|
||||
"not be caused by the file loop"
|
||||
)
|
||||
assert result.inbox_marker_in_run_b_prompt, (
|
||||
"the verdict dropped into the inbox after Run A did not reach Run B's hypothesis prompt — "
|
||||
"the demo narrates a long file loop it does not run"
|
||||
)
|
||||
|
||||
|
||||
async def test_the_inbox_verdict_is_a_real_file_outside_the_wiki(tmp_path) -> None:
|
||||
"""The medium is the point: the payload must be a FILE an expert could have written by hand,
|
||||
and it must live OUTSIDE the bundle copy. The role split (målbilde §3) is that the system READS
|
||||
the inbox — the expert/persona writes it — and the wiki is the separate, gated Step-8 layer.
|
||||
An inbox nested inside the bundle would be navigable context, which is a different mechanism."""
|
||||
result = await simulate_learning_loop(str(BUNDLE_DIR), str(tmp_path))
|
||||
|
||||
assert result.inbox_path.is_file(), "the dropped verdict is not a file on disk"
|
||||
payload = json.loads(result.inbox_path.read_text(encoding="utf-8"))
|
||||
assert result.inbox_marker in payload["rationale"], (
|
||||
"the marker is not carried by the file's own rationale — it did not travel through the file"
|
||||
)
|
||||
assert payload["decision"] == "approved"
|
||||
|
||||
bundle_copy = Path(tmp_path) / "bundle"
|
||||
assert bundle_copy.is_dir()
|
||||
assert bundle_copy not in result.inbox_path.parents, (
|
||||
"the inbox lies inside the bundle copy — a verdict there would reach the next run as "
|
||||
"navigable context, not through the Step-7 file loop"
|
||||
)
|
||||
|
||||
|
||||
async def test_the_two_learning_paths_stay_separately_observable(tmp_path) -> None:
|
||||
"""CONTROL for the pair: Step 7 (file inbox) and Step 8 (wiki promotion) must carry DISTINCT
|
||||
markers and both must cross into Run B. With one shared marker, detaching either seam would
|
||||
still leave the other carrying it — and both load-bearing tests would pass while a mechanism
|
||||
was gone. This test is what makes the other two, and
|
||||
``test_simulation_loadbearing.py``, mean what they say."""
|
||||
result = await simulate_learning_loop(str(BUNDLE_DIR), str(tmp_path))
|
||||
|
||||
assert result.marker != result.inbox_marker, (
|
||||
"the promotion and inbox paths share a marker — neither seam can then be detached "
|
||||
"observably"
|
||||
)
|
||||
assert result.marker_in_run_b_prompt, "the Step-8 promotion path stopped crossing"
|
||||
assert result.inbox_marker_in_run_b_prompt, "the Step-7 file path stopped crossing"
|
||||
|
||||
# The marker can only come from the file: it appears nowhere in the source bundle. (The
|
||||
# promotion marker's own absence is asserted by test_simulation_loadbearing.py.)
|
||||
for path in BUNDLE_DIR.rglob("*"):
|
||||
if path.is_file():
|
||||
assert result.inbox_marker not in path.read_text(encoding="utf-8"), (
|
||||
f"the inbox marker occurs in the bundle ({path.name}) — the positive assertion "
|
||||
f"could pass for the wrong reason"
|
||||
)
|
||||
Loading…
Add table
Add a link
Reference in a new issue