The context sets, the packaged knowledge bases and the example bundles are
replaced by one fictitious example set about IT operations in an invented
organisation: three context sets (serverrom-2027, driftsavtale-2027 and the
two-base drift-og-avtale-2027), two synthetic knowledge bases under
src/portfolio_optimiser/data/kunnskapsbaser and two example bundles under
src/portfolio_optimiser/data/bundles. Numbers, codes and structural values in
tests and fixtures are kept; names, ids and wording change. Dated measurement
documents that only recorded runs on the replaced material are deleted.
Gate figures measured on the new set are not comparable with earlier ones.
The exclusion gate from the previous commit is green: 0 tracked files hit
outside the shared/ subtree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An independent reviewer found it and the claim was verified by measurement
before being accepted, not taken on trust.
test_portfolio_scripted_pass_makes_no_real_client raised from the patched
_default_factory to prove the scripted portfolio pass never builds a production
client. It cannot: the factory is called inside run_project's coroutine, and
run_portfolio gathers with return_exceptions=True, so the AssertionError was
collected into a RunFailure and never escaped. Measured directly -- with BOTH
the client_factory wiring and the rc rule detached, the blade stayed GREEN. It
was going red on rc alone, which means it was testing defect B while claiming
to test defect A. The original seven-mutation sweep did not catch this because
each mutation was applied singly, and dropping the wiring alone still flips rc.
Replaced with a call sentinel: a list appended inside the factory and asserted
in the test body, which the wave handler cannot swallow. Re-measured -- red on
the wiring detach alone, and red on both detaches together.
The docstring now also states what the sweep could not: blades 1 and 8 are
environment-conditional. The local profile points at loopback, so on a machine
running a local model server the wiring detach would make real calls and could
complete the pass. Their red was real on the machine it was measured on and is
not portable; blade 2's is.
Separately, the budget-stop print is marked as defensive and currently
unreachable from main(), because main() never constructs a PortfolioMeter and
every write to budget_stop is gated on one -- the strict=True precedent
directly above says untested future-proofing must be labelled as such. The
README claim that a portfolio pass reports a cap stop is corrected to say the
cap has no CLI flag yet. Noted for whoever wires that door: BudgetRefused is a
RuntimeError and the existing except clause would not catch it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0118noV9rCfrdREH26XqZB5z
Walking --portfolio end-to-end as a downloader would -- which no session had
done -- surfaced two defects of a class this repo already legislates against.
(A) --scripted-replies was silently DROPPED in portfolio mode. main()'s
portfolio dispatch returned before the block that builds the scripted client
factory, and the flag was absent from the single_only refusal set: neither
honoured nor refused. Measured against the shipped reference portfolio: no
banner, four real model calls attempted, four APIConnectionError. The previous
session joined this flag to the --report allowlist and missed the portfolio
partition. Resolved by WIRING rather than refusing -- run_portfolio already
exposes the same client_factory seam, and refusing would have left portfolio
mode with no offline door at all for an adopter without a model budget. The
scripted block is hoisted above the dispatch; the single-project required-arg
and semantic-retrieval refusals are hoisted with it so an incomplete argv is
still refused BEFORE the honesty banner could claim a scripted run happened,
and the refusal order within single-project mode is unchanged.
(B) A portfolio pass reported one of its four outcome channels. failures
(S3.3 collect-and-continue) and budget_stop (S3.4 global cap) never reached the
operator and rc was unconditionally 0, so the four-failure pass above printed
NOTHING and exited 0 -- silence read as success. BudgetStop is a separate field
precisely so exhaustion can be told from success; the CLI showed neither.
Failures now print to stderr with project id, error type and message; the
budget stop prints its four numbers; rc is 1 iff something raised. A budget
stop alone stays rc 0: exhaustion is a structured stop the operator asked for
by setting a cap, not a crash. Completed runs still print, so the non-zero rc
does not undo collect-and-continue.
Load-bearing MEASURED against the whole 662-test suite, seven mutations all
red, including both controls: detach the client_factory wiring - make the
banner a single-project courtesy again - detach the failure print - revert rc
to 0 - detach the budget-stop print - print the failure line unconditionally
(control) - print the budget-stop line unconditionally (control).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0118noV9rCfrdREH26XqZB5z
