README: the stale one-line CLI mention replaced by the two-mode flag matrix (single-project vs
--portfolio), runnable 'uv run python -m portfolio_optimiser.run ...' examples, the --outbox-dir
!= --verdict-dir self-contamination warning (documented, not enforced), and the --decision/
--rationale inert-in-portfolio note. Honesty scoping: the prior-verdict fold (the learning step)
is stated to happen ONLY on the --bundle-dir path; a --docs-dir-only run is single-shot (no fold).
CHANGELOG [Unreleased]/Added: S5.3 CLI-parity entry (six flags + portfolio mode + load_dimension +
recipe doc) plus catch-up for the shipped-but-undocumented S4.1 (preflight), S4.2 (--live-dry-run),
S5.1 (hitl CLI), S5.2 (notify); stale test count 237 -> 431. extending.md verified accurate (B11
notifier note stands verbatim — main() auto-wires no notifier; no CLI section to sync), left
unchanged.
SC4 honesty grep clean (each hit in bundle-path context, none on a fold-less path):
grep -rniE 'learning loop|learns from|self-improv' README.md docs/extending.md CHANGELOG.md
README:11 (system-level) :56 (wiki substrate) :67 (8-step bundle loop);
CHANGELOG:13 (gated ExpeL fold) :17 (offline simulation). Full suite 431 passed (no code touched).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KNNiJRk1sSwxgVLS5AobT1
SC2 second half: --verdict-dir had no main()-level test (exploration gap). New test drops one
valid verdict into a tmp inbox and drives main([pid, --docs-dir, --bundle-dir, --verdict-dir,
--live-dry-run]) -> rc 0: the inbox ingestion (load_verdicts_from_dir, run.py:287) runs before the
dry-run cut (run.py:335), so the flag's wiring is exercised offline without raising. --bundle-dir's
main()-level coverage already exists in test_live_dry_run.py and is referenced, not duplicated.
run.py untouched (never re-wired). 11 passed in test_run_cli.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KNNiJRk1sSwxgVLS5AobT1
After parse_args, validate mode consistency with structured refusals (rc 1, not argparse.error):
--portfolio + any single-project-only flag (--docs-dir/--bundle-dir/--verdict-dir/--outbox-dir/
--run-id/--live-dry-run) is refused naming the offending flag; --goals/--ledger outside --portfolio
is refused. --decision/--rationale are EXCLUDED (non-None defaults make explicit-vs-default
indistinguishable — Pass-2 #2; inert in portfolio mode, README says so). --dimension-config is
valid in both modes. Validation precedes the portfolio dispatch, so refusals fire before any load.
RED-first: (a)/(a')/(c) failed offline (rc 0 fall-through) before the check, green after; (b)/(b')
single-project guard + legacy backward-compat pin were already green post-Step-3. All refusal-arm
RED fall-throughs held OFFLINE (met-goal / --live-dry-run) — no socket, per brief NFR. 15 passed,
ruff + mypy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KNNiJRk1sSwxgVLS5AobT1
main() gains a --portfolio mode flag dispatching to the EXISTING run_portfolio (not modified):
loads --goals/--ledger/--dimension-config via the fail-fast loaders, passes project_ids=(pid,)
or None (all reference projects), and prints a deterministic goal-stop line
'goal reached: scope=... project=... observed_ore=... limit_ore=... stopped_early=...' from the
returned GoalReached/PortfolioResult. Positional project_id relaxed to nargs='?' and --docs-dir to
optional, with a compensating single-project-mode guard (no pid/no --docs-dir -> rc 1 refusal) so
the legacy contract still fails loudly. Dispatch + guard sit BEFORE the live_dry_run branch
(Pass-2 #3 — appended after, they'd be dead code). Loader failures use the same structured refusal
('portfolio run refused: ...'). Load-bearing pair (RED-first): portfolio-hard goal already met
breaks offline before any client (scope=portfolio, stopped_early=True); per-project-hard control
skips the only pid (scope=project, stopped_early=False) — the two arms force the printed fields to
derive from run_portfolio's real values. Marker 13731 øre. 9 passed, ruff + mypy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KNNiJRk1sSwxgVLS5AobT1
main() single-project path now parses --dimension-config (fail-fast via load_dimension),
--outbox-dir, and --run-id (deliberate 7th companion flag: determinism invariant forbids a
wall-clock run_id default). All three threaded into BOTH run_project call sites. Full-run
branch wrapped in a structured-refusal (catch ValueError/FileNotFoundError/ValidationError ->
'run refused: {exc}' on stderr, rc 1, no traceback); the EXISTING dry-run handler widened to
the same tuple (pydantic ValidationError is not a ValueError subclass; load_dimension's
FileNotFoundError would otherwise traceback — Pass-2 #1). --outbox-dir carries a loud help=
note it must differ from --verdict-dir (self-contamination footgun; documented, not enforced).
RED-first: 4 CLI tests failed on unrecognized args, green after. 7 passed (incl. live-dry-run
regression). ruff + mypy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KNNiJRk1sSwxgVLS5AobT1
New load_dimension(str | Path) in dimension.py: is_file() -> FileNotFoundError,
then Dimension.model_validate_json -> pydantic.ValidationError on malformed shape.
Fail-fast because dimension config is authoritative startup input (contrast the
tolerant verdict-inbox RAW layer). Stdlib + pydantic only — stays MAF-free
(test_okf_is_maf_free AST guard green). Exported from __init__.py in both the
import block and __all__. RED-first: 3 loader tests failed on the missing import,
green after. tests/test_dimension.py + test_okf.py: 29 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KNNiJRk1sSwxgVLS5AobT1
pkt.2 — coverage gap (green-but-dead): link_in_index preserves the existing
index byte-for-byte and in order on the SUCCESS path by construction, but
nothing asserted it. Byte-equality was checked only on the REFUSAL path
(test_step8_promotion_loadbearing.py Test A); other tests check mere line
MEMBERSHIP. A writer that kept every link but reordered/rewrote the existing
body would pass the whole suite — the exact ingest-spec §6 / Step-8 promotion
invariant we rely on (promoted verdict links survive re-ingest byte-for-byte).
New test uses deliberately non-sorted existing links; proven load-bearing (a
temporary `sorted()` reorder mutation flips it RED, then reverted). Test-only,
no production change.
pkt.4 — shared_root.py docstring overclaimed "Every MAF-side consumer resolves
through this ONE seam". Verified: both runtime consumers (persona, simulation)
do route through shared_root(); the 15 test modules hardcode the in-repo
fixture path deliberately (a test needing the real fixture must not be
redirected by a production env var). Scoped the claim to "runtime consumer"
rather than churning 15 test files to make a false claim true.
Full suite 415 passed, ruff + mypy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVHLJuwBzp7MXkrUXJYARS
is_within_dir called os.path.realpath OUTSIDE its try block, so a path with
an embedded null byte (which makes realpath raise ValueError, not OSError)
leaked that ValueError to callers. Via safe_resolve -> okf._load_file (which
catches only PathSecurityError) it propagated uncaught out of
navigate_bundle, breaking the OKF SPEC §4 guarantee (okf.py:8-9) that a
broken cross-link is silently skipped, never raised: an index link like
`](a\x00b.md)` has no `/`, slips past the same-dir pre-filter, and reached
path resolution.
Move both realpath calls inside the existing try so an uncanonicalisable
path is treated as not-within (fail-closed): safe_resolve raises
PathSecurityError -> _load_file returns None -> the link is skipped. Fixes
the class for both callers of the seam (navigate_bundle + retrieve).
Tests (load-bearing — RED before, GREEN after):
- test_navigate_skips_null_byte_link: the reported regression.
- test_null_byte_path_rejected: the seam directly (is_within_dir/safe_resolve).
- test_navigate_skips_bundle_escaping_symlink_link: closes the previously
untested path-safety-at-link-resolution branch of navigate_bundle.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVHLJuwBzp7MXkrUXJYARS
Door A (manifest -> connector -> deterministic materialization -> index) is no
longer implemented here. src/portfolio_optimiser/ingest.py becomes a thin
consumer seam over the shared library, git-pinned to v0.3.1 on the same Forgejo
channel portfolio-optimiser-claude uses. Net -626/+385; ingest.py 599 -> 145 lines.
shared/ingest-spec.md remains the normative spec: the library implements it, it
does not replace it. Spec changes continue to go via commons.
Acceptance criterion met and proven: all three golden bundles (file/sql/http)
are byte-exact before and after, including the idempotence re-run. examples/ and
shared/ carry ZERO modifications -- the fasit was not adjusted to fit.
The rejection set was verified equivalent, not assumed: all 22 malformations the
repo's pydantic models refused are refused by the library, with typed codes
(okf_type_reserved, credential_embedded, extraction_id_duplicate, ...).
Test rebinding (invariants preserved, vehicle changed): the library has zero
runtime dependencies by design, so pydantic is unavailable to it.
ManifestV1.model_validate(dict) -> load_manifest_bytes(bytes); ValidationError ->
ManifestError; model_fields -> dataclasses.fields; PathSecurityError ->
SourceError(path_escape); ValueError -> MaterializationError(ingested_at_invalid).
Tests now also pin the refusal `code`, the library's documented stability
contract -- a sharper assertion than "some validation error was raised".
Two accepted behavioural deltas, recorded rather than silently dropped:
- Title whitespace is stored verbatim instead of collapsed at validation, so the
frontmatter title and the index label are no longer guaranteed identical for
irregular whitespace. Both behaviours are spec-conformant (the spec is SILENT;
the old one was a repo-local pinned decision). Queued as a commons-amendment
candidate so both stacks pin the same answer. Goldens unaffected.
- The section 8 audit log moves to logger llm_ingestion_okf.materialize. Nothing
in the repo consumed the old channel.
Also: the `type` discriminator is no longer a dataclass field, so the spec
cross-check asserts it explicitly -- without that line the swap would have
silently narrowed the test.
New tests/test_ingest_library_seam.py pins the seam itself: the restated section 5
stamp formula against the stamp the library actually writes (the one place the
adapter does not purely delegate, since v0.3.1 exposes no stamp helper), the
local-only allow_network default, the list[Path] unwrapping, and a guard that the
adapter never regrows local Door A machinery. All four verified RED when detached,
as were both golden regressions under a byte-level render mutation.
Door A is UNGATED: it calls no guard before writing to disk. Gating untrusted
content remains the caller's responsibility (guard wiring still planned).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B4jNN186eVqfe1x5DnTU6r
Map untrusted-ingest surface (ingest.materialize http/I6, verdict-inbox
load, promote_verdict, future received-bundle) vs first-party paths;
verdict = planned, wire scan/sanitize before M3 as S2.4/S2.5 extension.
Plan only — guard not wired. shared/ hardening owned by commons session.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0145ZKPLMVeqM47z2jxxokym
Both live in the tolerant inbox-read path and both contradicted the module's
own "skipped, never raised" / loader-parity docstrings (S5.1 review: ALLOW,
2 MINOR, non-gating):
- UnicodeDecodeError (a *.json hand-saved in Latin-1 with Norwegian æ/ø/å is
invalid UTF-8) now SKIPPED in hitl._load_json_dict AND
verdicts.load_verdicts_from_dir — was: crashed pending/route with a raw
traceback (a ValueError subclass, caught by neither OSError nor
JSONDecodeError). Fix symmetric across both readers.
- hitl._inbox_verdict_ids now skips a non-iterable affected_codes exactly as
load_verdicts_from_dir does (frozenset() raises TypeError) — was: marked the
proposal judged on key-presence alone → a silent false-negative in the
operator pending queue. hitl-only: verdicts is the correct reference.
TDD: 3 RED-then-green (both UnicodeDecodeError twins + the parity gap) + 1
ground-truth pin locking the loader side so the parity cannot rot.
Gate: pytest → 389 passed, 4 skipped (was 385); ruff check + format clean; mypy clean.
Post-hoc /trekreview of S4.2 surfaced two confirmed findings; both closed via TDD.
S42-001 (MAJOR): the new --live-dry-run CLI tests read PORTFOLIO_MODEL_MAP /
PORTFOLIO_FOUNDRY_PROJECT_ENDPOINT via resolve_model/AzureFoundryBackend but did
not isolate them, so both arms inverted their rc in a Foundry-configured env.
Add an autouse fixture that delenvs both, mirroring test_backends/test_preflight.
S42-002 (MINOR): the --live-dry-run except ValueError attached the azure-preflight
remediation to every offline-path ValueError (unknown project_id, empty docs_dir,
bundle mismatch). Scope the hint to args.profile == "azure"; structured refusal +
rc 1 preserved for all. New test proves a LOCAL unknown-project refusal carries no
azure hint.
Gate: pytest 358 passed / 4 skipped, ruff check + format clean, mypy 24 files.
Post-hoc /trekreview (WARN, scope 0670b2a..b513e64) surfaced 2 findings, both closed:
- MAJOR MISSING_TEST: the non-https scheme guard (preflight.py:61) and non-azure
profile refusal (:82-86) had zero test coverage — added
test_non_https_endpoint_refused + test_non_azure_profile_refused so an inverted
condition can't regress silently.
- MINOR MISSING_ERROR_HANDLING: an existing-but-unreadable PORTFOLIO_MODEL_MAP
raised PermissionError (an OSError, not ValueError) past the except at :95 →
widened to OSError (subsumes FileNotFoundError). test_unreadable_override_is_
structured_refusal proves the traceback-free invariant (red pre-fix).
Suite 348->351/4, ruff+format+mypy clean.
Closes the one MAJOR from /trekreview of S3.6 (verdict WARN): the
user-facing NOK-string formatter had only an isinstance(...,str) check,
so a regression (wrong divmod, dropped padding, øre/kr transposition,
or non-breaking-space → ASCII) would ship a wrong money figure with the
suite green. Adds a load-bearing value-level test (thousands grouping,
sub-100-øre padding, negative-sign branch) pinning the actual Norwegian
\xa0-separated formatting; verified red under a divmod(...,1000) mutation
and reverted. Suite 333 passed / 4 skipped, ruff+mypy clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0145ZKPLMVeqM47z2jxxokym
