ktg
released this
2026-08-12 18:17:43 +00:00
| 65 commits to main since this release
Added
- Step 5 is now observable:
generate_via_llm returns a GenerationResult carrying the validator
falsifications that informed a later attempt, surfaced on RunResult.refinements. The offline
simulation exercises it — the scripted proposer overclaims, the deterministic validator falsifies
the number, and the refined proposal validates.
- Two console entry points ship with
uv sync: portfolio-optimiser (the CLI) and
portfolio-optimiser-demo (the offline walkthrough). Deliberately two of the package's five
main() functions — costsim, hitl and preflight stay operator tools invoked as modules, and
every name here is one a release has to carry. Both invocation forms write byte-identical stdout.
The entry points are tested against the INSTALLED distribution's metadata rather than the TOML: a
[project.scripts] line that has never been synced is a claim, not a command.
- The offline walkthrough's transcript is checked in as a golden fixture
(tests/golden/demo-transcript.stdout and .stderr). Self-identity across two runs cannot detect
a regression — two runs of a regressed walkthrough agree exactly as well as two runs of a correct
one — so the fixture leaves the process. stdout is pinned verbatim; stderr is normalised on exactly
two measured environment spans, the site-packages prefix and the temporary directory, leaving the
po-sim- prefix visible because that belongs to the program rather than the environment. A
companion control forbids the mask from widening: a normaliser that dropped whole lines, with the
fixture regenerated beneath it, would keep both equality tests green.
Changed
- Breaking (library API):
generate_via_llm returns GenerationResult instead of
ValidatedProposal | Rejection; read .outcome for the previous value. The refinement loop's
bound is unchanged (max_attempts + token meter).
simulation.scripted_factory accepts a per-role reply selector over (prompt, role) as well as
a constant reply, so a scripted role can answer differently on a later attempt.- The offline simulation's scripted proposer is now a candidate registry rather than a
hand-written reply: simulation.scripted_proposer(candidates) builds the selector from
ScriptedCandidate entries keyed on the project id the prompt names, and
simulate_learning_loop takes project_id alongside bundle_dir. Adding a project to the
walkthrough is a data entry. A prompt matching no entry — or more than one — raises
ScriptedCandidateError rather than answering with another project's numbers.
- The offline simulation now EXERCISES the Step-7 file inbox it narrates. The verdict previously
arrived as a function argument — the short, in-run capture — while the trace line described the
long file loop. An expert now writes a real verdict file into an inbox between the two runs, and
the second run is given verdict_dir=, so run_project merges it before the Step-1 fold. The
inbox sits beside the bundle copy and never inside it: a verdict file within the bundle would reach
the next run as navigable context, which is a different mechanism wearing this one's clothes. The
two time-scales carry SEPARATE markers by construction, since one marker on both paths would let
either seam alone satisfy the assertion and leave the other free to rot; simulate_learning_loop
refuses equal markers.
- The offline walkthrough runs ANCHORED. Its deterministic gate reconciles each proposal against the
project's real cost lines, which activates only when the knowledge base ships a cost-baseline.json;
without one the gate reasoned solely about numbers the proposal supplied itself. The walkthrough now
reads that file through exactly the seam a delivered knowledge base would use. For the synthetic
fallback bundle, which cannot receive the file inside the pull-only shared/ subtree, the baseline
is DERIVED IN CODE from the scripted register rather than typed beside it — two sources of the same
numbers drift, and drift is precisely what the walkthrough's own 10 % probe models. The declared
baseline is printed, because an anchoring nobody can see is one nobody can check.
- The walkthrough's stderr is quieter. The expected round-cap notice is dropped by a filter on the
emitting logger, keyed on the message and installed by main() — never at import, so a library
consumer keeps its own logging configuration. The two ExperimentalWarning lines are deliberately
NOT damped: they fire while the package __init__ imports the agent framework, always before the
simulation's own imports and under both invocation forms, so silencing them would mean filtering
warnings inside the library on every consumer's behalf. They are pinned in the golden fixture
instead. stderr went from six lines to four.
- The walkthrough DERIVES its provenance sentence for prior verdicts instead of stating it. The line
above already computes the count, so a hand-written split would be a second copy of the same fact,
free to drift the moment a knowledge base ships another seeded verdict.
- The shared expert-reviewer persona's canonical example verdict is worded domain-neutrally
("i tilsvarende anlegg" rather than "i kontorbygg"), pulled from the upstream commons repository.
The walkthrough prints that rationale verbatim, so the wording was a building-type justification
read out over a road-lighting project; it could not be fixed downstream, because overriding the
text locally would re-stub the very artifact the shared skill exists to make load-bearing. The
marker value is byte-unchanged, and the pinned transcript fixture was regenerated against a
prediction written before the pull — the printed line is clipped at a fixed width, so the swap
moves the tail as well, and a regeneration without a written prediction could not tell that
expected shift apart from drift.
Security
- Door A — the ingest path that materialises externally sourced documents into a knowledge base — can
now scan generated content before it is published, through ingest.materialize_gated. The gate is
opt-in and requested by name: materialize itself stays ungated by design, because golden
suites pin its bytes and a caller that wants the gate asks for it.
The seam sits around materialisation rather than inside it. The pinned upstream stages in memory
and then performs its own disk phase, with no callback between the two, so a gate placed "at the
write point" could only have run after the bytes had landed — a cleanup, not a gate. Instead the
bundle is COPIED, materialised into the copy, scanned, and then published or discarded as a whole.
The copy is load-bearing rather than convenient: the upstream's ownership scan, its collision gate
against curated content, and its index merge all read the EXISTING bundle, so staging into an empty
directory would publish a bundle stripped of its curated neighbours and their index links — data
loss dressed as a security fix.
Trust follows ORIGIN, never channel. The outcome is per BUNDLE, since partial publication would
leave a bundle and index answering to no manifest, while diagnostics are per DOCUMENT so a single
run reports every finding rather than only the first. Findings are written to the bundle's log.md
and never to concept frontmatter, where four golden suites pin the bytes.
Notes
- The
1.0.0 version signals a stable public surface, not a finished research programme. Two
boundaries are open and named rather than implied: the ingest stamp predicate has diverged from the
upstream specification (a value literal here, a structured field upstream) and does not touch the
run path, and the mirroring of several seams to the sibling implementation is outstanding.
Downloads