open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(sim): the demo script is keyed on the project the prompt names, so a new project is data

Browse source 
The scripted proposer answered one hard-coded pair of proposals. A second project meant a second
hand-written selector, written under demo-week time pressure -- the risk the week plan names
explicitly (§4, risk 2). It is now a registry: `ScriptedCandidate` entries selected by
`scripted_proposer`, plus `project_id` as an argument to `simulate_learning_loop`.

The open decision was WHAT identifies the candidate in the prompt blob; the plan flagged it as
unverified, so it was measured. Two prompt shapes reach the selector: the debate prompt carries the
whole bundle context, the generation prompt carries `Project: {id} - {name}` plus -- as its context
-- the debate output, which is the selector's own earlier reply. So the cost code and the measure
name are present in the generation prompt only because the script put them there; keying on them
would key the script on its own output. The project id is the one identifier both shapes carry and
the framework stamps.

Validation, never repair: no match, or more than one, raises `ScriptedCandidateError`. A default
reply would answer an unregistered project with another project's numbers, which on screen is
indistinguishable from a correct run; an ambiguous blob is a data problem that must surface at the
rehearsal rather than be decided by registry order.

Load-bearing MEASURED against the whole suite, five mutations all red plus a green control: detach
the project keying - one global flip key - fall back on an unknown project - first-match on an
ambiguous prompt - detach the `project_id` argument. The flip-key test was rewritten mid-measurement
because its first form asserted on the FIRST registry entry, where "the matched candidate's key" and
"candidates[0]'s key" coincide -- it could not separate the two implementations, and proved nothing.

766 passed / 4 skipped. Simulation still exits 0, still prints eight labelled steps, still
byte-identical across two runs.

[skip-docs] README is deliberately untouched: O4 defers the README rewrite to 14-15 August, after
the demo has produced the evidence for the level-2 claim. CLAUDE.md carries the invariant.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XoHJCKBTjFKcjsfEQyGbzh
This commit is contained in:
Kjell Tore Guttormsen 2026-08-06 16:43:29 +02:00
commit 688ee24973
4 changed files with 278 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

6
CHANGELOG.md
View file

@ -19,6 +19,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
bound is unchanged (`max_attempts` + token meter).
- `simulation.scripted_factory` accepts a per-role reply *selector* over `(prompt, role)` as well as
a constant reply, so a scripted role can answer differently on a later attempt.
- The offline simulation's scripted proposer is now a candidate **registry** rather than a
hand-written reply: `simulation.scripted_proposer(candidates)` builds the selector from
`ScriptedCandidate` entries keyed on the project id the prompt names, and
`simulate_learning_loop` takes `project_id` alongside `bundle_dir`. Adding a project to the
walkthrough is a data entry. A prompt matching no entry — or more than one — raises
`ScriptedCandidateError` rather than answering with another project's numbers.
## [0.1.0] - 2026-08-06