The scripted proposer answered one hard-coded pair of proposals. A second project meant a second
hand-written selector, written under demo-week time pressure -- the risk the week plan names
explicitly (§4, risk 2). It is now a registry: `ScriptedCandidate` entries selected by
`scripted_proposer`, plus `project_id` as an argument to `simulate_learning_loop`.
The open decision was WHAT identifies the candidate in the prompt blob; the plan flagged it as
unverified, so it was measured. Two prompt shapes reach the selector: the debate prompt carries the
whole bundle context, the generation prompt carries `Project: {id} - {name}` plus -- as its context
-- the debate output, which is the selector's own earlier reply. So the cost code and the measure
name are present in the generation prompt only because the script put them there; keying on them
would key the script on its own output. The project id is the one identifier both shapes carry and
the framework stamps.
Validation, never repair: no match, or more than one, raises `ScriptedCandidateError`. A default
reply would answer an unregistered project with another project's numbers, which on screen is
indistinguishable from a correct run; an ambiguous blob is a data problem that must surface at the
rehearsal rather than be decided by registry order.
Load-bearing MEASURED against the whole suite, five mutations all red plus a green control: detach
the project keying - one global flip key - fall back on an unknown project - first-match on an
ambiguous prompt - detach the `project_id` argument. The flip-key test was rewritten mid-measurement
because its first form asserted on the FIRST registry entry, where "the matched candidate's key" and
"candidates[0]'s key" coincide -- it could not separate the two implementations, and proved nothing.
766 passed / 4 skipped. Simulation still exits 0, still prints eight labelled steps, still
byte-identical across two runs.
[skip-docs] README is deliberately untouched: O4 defers the README rewrite to 14-15 August, after
the demo has produced the evidence for the level-2 claim. CLAUDE.md carries the invariant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XoHJCKBTjFKcjsfEQyGbzh
generate_via_llm consumed each validator Rejection internally (`last`), fed it into the
next attempt's prompt, and dropped it. So Step 5 was real but unobservable: a caller could
see THAT a proposal validated, never that it validated on attempt 2 after the deterministic
validator falsified attempt 1. It was the one step of the eight with no output to show.
The seam is a typed return value -- GenerationResult(outcome, refinements) -- rather than an
out-parameter or a callback: a returned value cannot be silently lost by a caller that forgets
to pass a collector, and mypy forces every call site to acknowledge it.
refinements carries ONLY rejections that were actually fed back. When the attempt budget runs
out the final rejection IS outcome; counting it here would be double-counting, and the bounded
control test goes red on the collect-everything implementation that gets this wrong.
The loop's bound is untouched: max_attempts and meter.tick_round stand, and `last` still drives
the prompt alone, so prompt growth is unchanged. run.py accumulates across _evaluate calls, so
_evaluate_mandate is untouched; RunResult.refinements defaults (the coverage precedent) and is
concatenated across approaches rather than keyed per approach -- stated as an honesty limit.
The simulation now shows it: the scripted proposer overclaims 250000, which the validator
falsifies against P90 = 90000, and the corrected 30000 validates. Only the overclaim is
scripted -- the rejection is computed. scripted_factory takes a per-role reply selector so this
needs no second scripted client body.
README records the two accuracy changes only (Step 5 is now inspectable; the simulation trace
shows the correction). The level-2 publishing claim stays deferred until after the demo (O4).
Load-bearing MEASURED against the full suite with a control, four mutations all red:
detach the returned history (4 tests) - collect-everything (control only) - detach the run
wiring (2 tests) - revert the simulation's proposer to a constant (the demo-protection test).
Control: 759 passed / 4 skipped; ruff, format and mypy clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017CcWFcREUi6YPjEpN3ACDP
STATE said the release text was already written and only needed dating. Measured:
CHANGELOG.md was last touched at 9e149c6, 53 commits back, so [Unreleased]
described the repo as of S3.1 — and claimed 512 passing tests where the suite
measures 755.
For a FIRST tag there is no predecessor, so the section is not "what changed" but
"what this version is". A description that stops at S3.1 does not under-report a
delta; it misrepresents the artefact being tagged, on a public mirror.
Audited the existing bullets for claims that had gone FALSE rather than merely
stale — the class a test-count catch is only a sample of. Four checked, all still
true at HEAD: numpy confined to semretrieval.py; value_report deliberately not
wired into costsim; the simulation module and knowledge-base recipe present; the
two run modes still two.
Added, grouped by seam rather than by commit: S4.0 cost-baseline anchoring, S2.7,
S3.2 per-candidate verdict keying, S3.3 wave concurrency, S3.4 global token cap,
S2.2/S2.4 ingest transports, --scripted-replies, --mandate, --mcp-config, the
per-approach outbox, and external-call provenance. New Fixed section for the
seven correctness fixes, with a lead-in saying plainly that none of them ever
shipped. Notes gained the deployer-owns-DPIA scope boundary and the pull-only
subtree rule.
The test-suite bullet now records TWO cases where "every seam is load-bearing"
did not hold until measured, not one: the S3.1 CLI-level gap, and four of five
BudgetExceeded raise sites free to report any `observed`.
No code change. Suite measured green at 755 passed / 4 skipped after the edit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVekyKcvT4ah6jnCT8R8fe
README: the stale one-line CLI mention replaced by the two-mode flag matrix (single-project vs
--portfolio), runnable 'uv run python -m portfolio_optimiser.run ...' examples, the --outbox-dir
!= --verdict-dir self-contamination warning (documented, not enforced), and the --decision/
--rationale inert-in-portfolio note. Honesty scoping: the prior-verdict fold (the learning step)
is stated to happen ONLY on the --bundle-dir path; a --docs-dir-only run is single-shot (no fold).
CHANGELOG [Unreleased]/Added: S5.3 CLI-parity entry (six flags + portfolio mode + load_dimension +
recipe doc) plus catch-up for the shipped-but-undocumented S4.1 (preflight), S4.2 (--live-dry-run),
S5.1 (hitl CLI), S5.2 (notify); stale test count 237 -> 431. extending.md verified accurate (B11
notifier note stands verbatim — main() auto-wires no notifier; no CLI section to sync), left
unchanged.
SC4 honesty grep clean (each hit in bundle-path context, none on a fold-less path):
grep -rniE 'learning loop|learns from|self-improv' README.md docs/extending.md CHANGELOG.md
README:11 (system-level) :56 (wiki substrate) :67 (8-step bundle loop);
CHANGELOG:13 (gated ExpeL fold) :17 (offline simulation). Full suite 431 passed (no code touched).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KNNiJRk1sSwxgVLS5AobT1
Findings 4-7 from the 2026-07-02 status analysis, per the session plan (S1):
- CHANGELOG rewritten truthfully (was: 'Plan phase - no framework code yet')
- README stack line names the split GA packages, not the agent-framework meta-package
- CLAUDE.md: MCP downgraded to extension point (in-process FunctionTool is the default seam)
- Verdict conflict semantics documented as chosen (store first-write-wins per id,
disk/wiki last-write-wins per file; full B10 taxonomy deliberately deferred)
- docs/extending.md: explicit 90%-principle cut-list (B10, B11, U12, U14, concurrent fan-out)
- .gitignore covers .trekexecute-progress-* (docs/.DS_Store was already untracked/ignored -
the plan's git rm --cached assumption was stale; no-op)
No code behavior changed (docstring only in verdicts.py). Suite 152/4 green, mypy clean,
ruff format --check clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AaQCFnfsh3tfq1VfzdJpoi
