Kriterium 6 er selv-identitet: to kjøringer av en regredert demo er like enige som
to av en riktig. Fasiten forlater derfor prosessen. stdout pinnes ORDRETT (og er
dermed demoens abortsti); stderr normaliseres på nøyaktig to MÅLTE miljø-spann —
site-packages-prefikset og temp-katalogen — med po-sim- holdt synlig, fordi det er
en egenskap ved programmet og ikke ved miljøet. Pinnet stderr = fire linjer.
Kontrollen som forbyr at masken vokser er load-bearing: en droppende normaliserer
med fasiten regenerert under seg holder BEGGE likhets-testene grønne.
Pkt. 4: planens forhåndsskrevne frø-setning sa «én av de TO tidligere dommene».
Målt mot levert VEGLYS-bundle henter Kjøring B TRE — én fulgte med kunnskapsbasen,
to er demoens egne, én per tidsskala. Splitten avledes derfor fra kjøringen; en
håndskrevet «én av tre» ville vært den andre kopien som drifter.
Fem mutasjoner alle røde + grønn kontroll (hele suiten hver gang): ett byte i en
stdout-linje · detach dempingen · over-normaliser stderr · literal splitt · detach
frø-setningens print. Byte- og detach-mutasjonene ble fanget av KUN golden-testen;
den literale splitten av KUN skille-testen.
793 -> 801 passed / 4 skipped.
P3 a-c lukket. Pullen hentet commons 002f000+27cdce9: kriterium 8 grønt (nav-goldens
byte-uendret, målt både med git diff og shasum) og suiten uendret 785 — ingen abort.
Retningen på tallene er SNUDD (P3 b): manus-registerets kostlinjer er skrevet FRA
shared/examples/veglys-fv-soer/cost-baseline.json, ikke ved siden av den, og
baseline_from_scripted_candidate brukes ikke på denne stien — main() leser levert fil.
Overdrivelsen 2100000 er valgt utenfor commons' tall-inventar: målt fraværende fra
bundelen og over målt P90 1769915 (deres anslag var ~1770000). 600000/900000 ville
klarert gaten og aldri utløst Steg 5.
Målingen felte en defekt reserven skjulte: :g slår over i eksponentform ved 7. signifikante
siffer, så levert baseline printet 4.38615e+06. Reservens 300000 har seks siffer og nådde
aldri overgangen. _num erstatter :g begge steder.
Load-bearing MÅLT mot hele suiten, fem mutasjoner alle røde + grønn kontroll:
detach main-wiringen · reverter _num til :g · drift registeret ETT siffer (4386151 —
innenfor 5 %-toleransen, fanget av ingenting i 792 tester bortsett fra den nye) ·
sett flip_key til et token som finnes i bundelen · detach forankringen på bundle-stien.
785 -> 793 passed / 4 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BUjfw4eJdwwnqHSXhfcY6i
Two commands are now part of the install surface a fresh clone gets from `uv sync`:
`portfolio-optimiser` (run:main) and `portfolio-optimiser-demo` (simulation:main).
Deliberately two of five main()s — costsim/hitl/preflight stay module-invoked; every
name here is a name the freeze has to carry. Pinned against the INSTALLED distribution's
metadata, not the TOML: a [project.scripts] line that has never been synced is a claim,
not a command. Measured: stdout is byte-identical across both invocation forms.
stderr (P4 pkt. 2), the session's open decision, resolved by measurement rather than by
preference. Damped: the round-cap notice only, via a filter on the emitting logger, keyed
on the message and installed by main() — never at import, so a library consumer keeps its
own logging config. NOT damped: the two ExperimentalWarnings. They fire while the package
__init__ imports run -> agent_framework, always before simulation's own imports and under
both invocation forms, so silencing them would mean filtering warnings inside the library
package on every consumer's behalf; they are pinned in pkt. 3 instead. A console-script
wrapper was rejected for a second reason: the two forms would then write different stderr,
and a byte-fasit would pin the command rather than the program. stderr 6 -> 4 lines.
A first implementation wrapped simulation's own agent_framework import in a scoped mute.
Measurement showed it can never fire — the package __init__ has already imported
agent_framework by then — so it was removed rather than left as a green-but-dead seam.
Load-bearing MEASURED against the whole suite, five mutations all red + green control:
remove [project.scripts] · typo the target · detach the main() call · make the filter drop
everything · install the filter at import time. The typo mutation also felled a test: the
resolve-assert re-checked the expected constant against itself, and now resolves what the
distribution actually installs. 775 -> 785 passed / 4 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C2bxLcCRguxXzpM4priTMn
The validator can reconcile a proposal against the project's actual cost lines
(S4.0 stage 0), but only when the knowledge base ships a cost-baseline.json —
and no bundle under shared/ has one. So on stage the gate reasoned only about
numbers the proposal supplied itself.
The reserve can never receive the file in shared/ (pull-only subtree, and demo
criterion 8 requires the goldens byte-unchanged). That is a placement
constraint, not an impossibility: materialize_anchored_bundle copies the bundle
and adds the file outside shared/, and the run path reads it through exactly
the seam a delivered bundle would use.
The baseline is DERIVED IN CODE from the scripted register, never typed beside
it — two sources of the same numbers drift, and drift is precisely what the
10 % probe models. On GO day the direction reverses (plan P3 b). Both scripted
replies must state the same cost lines or ValueError: were they to differ,
hypothesis #1 would be falsified by stage 0 instead of by P90 — the same
REJECTED line on screen, a different mechanism behind it.
10 % probe, measured: baseline x 1.10 -> FORKASTET at stage 0, before the
solver; corrected -> FORESLÅTT. Criterion 6 re-measured (stdout byte-identical
across two runs); stderr unchanged at 6 lines. The ONLY diff against the
un-anchored demo is the new KUNNSKAPSBASE block — everything else is
byte-identical, which is the problem: an anchoring nobody can see is one nobody
can check. Hence it is printed, and hence `provenance` is a required argument.
769 -> 775 passed.
Five mutations red + green control. The measurement failed the TEST first:
"ingen kostbaseline erklært" CONTAINS "kostbaseline erklært", and
ENERGI-TOTAL-EL already appears in the Step-2 line, so both assertions survived
the detach mutation. The two branches now share no wording.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GD6Y2Y23NZZxPYtSRoCmst
The Step-7 trace line said "lang fil-løkke" while the verdict arrived as a
function argument (`verdict_input`) — the short, in-run capture. The long loop
was tested but never exercised by the thing on stage.
An expert now drops a real verdict FILE (`write_verdict`) into an inbox between
the runs, and Run B is given `verdict_dir=`, so `run_project` merges it into the
store before the Step-1 fold.
Not done as the plan point was worded, and the difference is load-bearing:
routing the PERSONA verdict through the inbox would have put ONE marker on two
paths — Step 7 (inbox) and Step 8 (promotion) both end in Run B's prompt, so
either could carry it alone and `test_simulation_loadbearing.py`'s promotion
assertion would have stayed green with promotion detached. A second verdict with
its own marker keeps both seams independently red-able; `simulate_learning_loop`
raises when the two markers are equal. The inbox sits beside the bundle copy,
never inside it, and the id is an explicit sentinel (a minted id would collide
with the promoted verdict's, and `VerdictStore.add` is first-write-wins).
766 -> 769 passed (773 collected). Criterion 6 re-measured: stdout byte-identical
across two runs; stderr unchanged at 6 lines. Mutations measured against the full
suite, four red + a green control: detach `verdict_dir=` · point Run B at an empty
folder while the file is still written · marker set to `realization_rate: 0.82`
(measured present in the verdict seed) · marker set to `energy performance gap`
(measured present in a navigated concept file) · benign rename of the inbox dir.
Honesty limit found while measuring: the last two mutations fell on the causality
assertion, not the Run A control — generation prompts carry the debate output, not
the bundle context. The pair holds, but each assert defends a different property.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FVYDeJ9evZicgU5r3roZVW
The scripted proposer answered one hard-coded pair of proposals. A second project meant a second
hand-written selector, written under demo-week time pressure -- the risk the week plan names
explicitly (§4, risk 2). It is now a registry: `ScriptedCandidate` entries selected by
`scripted_proposer`, plus `project_id` as an argument to `simulate_learning_loop`.
The open decision was WHAT identifies the candidate in the prompt blob; the plan flagged it as
unverified, so it was measured. Two prompt shapes reach the selector: the debate prompt carries the
whole bundle context, the generation prompt carries `Project: {id} - {name}` plus -- as its context
-- the debate output, which is the selector's own earlier reply. So the cost code and the measure
name are present in the generation prompt only because the script put them there; keying on them
would key the script on its own output. The project id is the one identifier both shapes carry and
the framework stamps.
Validation, never repair: no match, or more than one, raises `ScriptedCandidateError`. A default
reply would answer an unregistered project with another project's numbers, which on screen is
indistinguishable from a correct run; an ambiguous blob is a data problem that must surface at the
rehearsal rather than be decided by registry order.
Load-bearing MEASURED against the whole suite, five mutations all red plus a green control: detach
the project keying - one global flip key - fall back on an unknown project - first-match on an
ambiguous prompt - detach the `project_id` argument. The flip-key test was rewritten mid-measurement
because its first form asserted on the FIRST registry entry, where "the matched candidate's key" and
"candidates[0]'s key" coincide -- it could not separate the two implementations, and proved nothing.
766 passed / 4 skipped. Simulation still exits 0, still prints eight labelled steps, still
byte-identical across two runs.
[skip-docs] README is deliberately untouched: O4 defers the README rewrite to 14-15 August, after
the demo has produced the evidence for the level-2 claim. CLAUDE.md carries the invariant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XoHJCKBTjFKcjsfEQyGbzh
The simulation proved the loop but printed only four of its eight steps, so a
listener could not follow what they were looking at without narration. This is
presentation only: every value printed is read off the RunResult the run already
returned -- nothing is recomputed against the bundle, nothing is inferred, and
the run path is untouched. The week plan's assumption ("seven of eight steps are
pure presentation") therefore held; no new logic was needed.
Run A walks steps 1-7, the promotion between the runs IS step 8, and Run B is
not re-numbered -- it shows only what changed, which is the marker reaching the
hypothesis prompt. Two honesty limits are visible in what is printed rather than
papered over: `retrieved` is the post-hoc proposal-keyed retrieval, not the
Step-1 fold (the marker line is what evidences the fold reaching the prompt),
and the run carries the checker's DECISION, not its prose -- the decision is
what gates, so it is what is shown.
The working-copy path moves to stderr: mkdtemp is the one non-deterministic
value in the output, and stdout must be byte-identical across runs for the dress
rehearsal's diff check. Status tokens stay VALIDATED/REJECTED in English on
purpose -- the same vocabulary as provenance.validator_decision, which the
Step-6 line prints verbatim.
Verified: `... | grep -cE "^ *Steg [1-8]"` -> 8; two runs byte-identical on
stdout; both a REJECTED and a VALIDATED line for the same candidate; suite
759 passed / 4 skipped; ruff + mypy clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XoHJCKBTjFKcjsfEQyGbzh
generate_via_llm consumed each validator Rejection internally (`last`), fed it into the
next attempt's prompt, and dropped it. So Step 5 was real but unobservable: a caller could
see THAT a proposal validated, never that it validated on attempt 2 after the deterministic
validator falsified attempt 1. It was the one step of the eight with no output to show.
The seam is a typed return value -- GenerationResult(outcome, refinements) -- rather than an
out-parameter or a callback: a returned value cannot be silently lost by a caller that forgets
to pass a collector, and mypy forces every call site to acknowledge it.
refinements carries ONLY rejections that were actually fed back. When the attempt budget runs
out the final rejection IS outcome; counting it here would be double-counting, and the bounded
control test goes red on the collect-everything implementation that gets this wrong.
The loop's bound is untouched: max_attempts and meter.tick_round stand, and `last` still drives
the prompt alone, so prompt growth is unchanged. run.py accumulates across _evaluate calls, so
_evaluate_mandate is untouched; RunResult.refinements defaults (the coverage precedent) and is
concatenated across approaches rather than keyed per approach -- stated as an honesty limit.
The simulation now shows it: the scripted proposer overclaims 250000, which the validator
falsifies against P90 = 90000, and the corrected 30000 validates. Only the overclaim is
scripted -- the rejection is computed. scripted_factory takes a per-role reply selector so this
needs no second scripted client body.
README records the two accuracy changes only (Step 5 is now inspectable; the simulation trace
shows the correction). The level-2 publishing claim stays deferred until after the demo (O4).
Load-bearing MEASURED against the full suite with a control, four mutations all red:
detach the returned history (4 tests) - collect-everything (control only) - detach the run
wiring (2 tests) - revert the simulation's proposer to a constant (the demo-protection test).
Control: 759 passed / 4 skipped; ruff, format and mypy clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017CcWFcREUi6YPjEpN3ACDP
One call-time resolver (env PORTFOLIO_SHARED_ROOT, default the in-repo
shared/) consumed by both MAF-side readers of the shared core:
persona._example_path() (the _EXAMPLE_PATH monkeypatch seam is kept) and
simulation._default_bundle_dir() (replaces the _BUNDLE_DIR module global).
De-risks the S4 extraction: re-pointing the commons becomes an env var,
not a code change. Override test proves the marker follows a tmp copy of
the whole shared tree; both detach points proven RED. Suite 155->157.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AaQCFnfsh3tfq1VfzdJpoi
The expert reviewer was only a hardcoded verdict_input dict inside the offline
simulation. Build it as the real, shared artifact target picture §8 calls for:
shared/skills/expert-reviewer/ — a SKILL.md persona prompt (energy-advisor / M&V
role + the realization-gap methodology the validator cannot compute) plus a
canonical references/example-verdict.json. shared/ stays pure data; the MAF side
reads it via portfolio_optimiser.persona.load_persona_example (call-time,
fail-fast) and the Claude-SDK sibling reads the same JSON with its own loader.
This de-stubs the simulation: its persona judgement (decision + rationale + traced
marker) is now sourced from the artifact at call time, not an inline literal — so
the shared persona is genuinely consumed and cannot rot silently. decision is
binary (approved/rejected, the FeedbackContract the run path accepts);
approved_with_adjustment is rejected there and lives only in the bundle seed
frontmatter + the promotion gate, so the realization correction is carried in the
rationale prose.
Load-bearing trio (tests/test_persona_skill_loadbearing.py), each proven RED on its
own detach: structure + framework-neutrality, the example is valid pipeline input
(incl. FeedbackContract, on a throwaway copy), and the simulation's marker follows
the artifact file. Suite 149->152.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MHR8iKxJRxDiDfNw8HZmWE
The primary method proof, offline — a deliberate, cost-driven substitution
for målbilde §11.8's real-model run (the operator runs MAF against no real
model; API for both repos is too costly privately).
`portfolio_optimiser.simulation` drives `run_project` with a scripted
synthetic chat client across two runs separated by a promotion, and shows
the learning loop close end to end:
- ScriptedChatClient subclasses the LAYERED OpenAIChatCompletionClient (not
bare BaseChatClient — else the always-attached BudgetMiddleware no-ops),
constructs offline (loopback url + dummy key), role-keys proposer/checker
replies, and records every prompt into a shared sink.
- simulate_learning_loop: Run A (fresh wiki) -> validated, persona-approved
verdict carrying a realization marker absent from the bundle -> promote_verdict
into the OKF wiki -> seed_store_from_bundle re-reads it -> Run B's hypothesis
prompt carries the marker. An empty-wiki control on Run A proves causality.
- `python -m portfolio_optimiser.simulation` prints an honest trace.
Honesty (§1): this proves the plumbing, the deterministic spine, and that the
learning dataflow closes — NOT that a live LLM would produce the proposal or
verdict (scripted stand-ins). The genuine model-behaviour comparison lives on
the Claude-SDK side (a minimal API run); the scripted client is MAF-side
scaffolding, not part of the framework-neutral shared/ core.
Load-bearing: tests/test_simulation_loadbearing.py goes red when promotion is
detached (the marker never crosses into Run B). Suite 148->149.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MHR8iKxJRxDiDfNw8HZmWE
