open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(navigation,validator): read_dir names the rung that reads a document, and a run can require its anchoring

Browse source 
F3 and F4, the two findings the S7 acid test (session 98) reported and left. The order required
both descriptions to be treated as PREMISES. One held; the other was felled before anything was
built on it.

F3 -- premise FELLED, asymmetry real. The order read arm C's two refused calls as "the path names
a document that EXISTS". Measured against the base that ran: its root holds 27 directories named
del-ii-bilag-N-... and 12 documents named inbox-del-ii-bilag-N-....md, and the requested path
matches NEITHER -- it is the directory naming convention applied to a document whose real name
carries an inbox- prefix. So the two live rounds were the UNKNOWN-path class, and this delivery
does NOT recover them (gated). What IS real: read_file on a directory has named read_dir since
session 95, while read_dir on a document named neither the rung nor the path.
okf.DocumentPathRefused closes that one direction -- a ValueError, a SIBLING of BundlePathNotFound
rather than a subclass, built from context_files (never files) and through the same in_dimension
predicate the listing uses, quoting the document's REAL name so what it hands back resolves.

F4 -- premise HELD, option (c) felled by measurement. All four live artefacts stamped
cost_baseline_anchored: False and each arm invented its cost codes. derive_cost_baseline refuses
against the delivered base: K2's price schedule is a pandoc SIMPLE table with ONE column header,
so making --derive-cost-baseline reachable there would mean inventing a rule for an unmeasured
form -- MAJOR-4's own honesty limit. Chose (b) over (a): --require-cost-baseline /
run_project(require_cost_baseline=...), OPT-IN and never default, so every bundle without a
cost-baseline.json runs unchanged. The gate sits where both branches have bound baseline and ABOVE
the dry-run cut, so it fires on the free trip too and, on the paid one, before the first model
call. Three CLI refusals by name, each with an rc-0 control.

12 mutations, all red against the WHOLE suite. Green control 1493/5 -> 1511/5 (+18 node-ids, 0
removed); golden demo-transcript.stdout BYTE-UNCHANGED (shasum -a 1 of the CONTENT =
ea8c534773acdbe41ae68f2c55724d69aaf8be4f). No paid run: both findings measured offline.

Measurement: docs/2026-09-08-f3-f4-nekten-og-forankringen.md
Order: 20260908T020419Z-5837110336-from-portfolio-optimiser

Co-Authored-By: Claude <Opus 5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 05:32:46 +02:00
commit 9232f94041
8 changed files with 759 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

8
tests/test_hierarchical_navigation_loadbearing.py
View file

@ -265,10 +265,14 @@ def test_a_nested_document_is_reachable_through_the_rung_below() -> None:
def test_an_unknown_directory_is_refused_by_name(tmp_path: Path) -> None:
"""(g) Fail-closed, and it is the vacuity trap in its own right: an unknown path rendered as an
empty listing is indistinguishable from a directory that exists and holds nothing. Validation,
never invention (``write_concept_file``'s rule)."""
never invention (``write_concept_file``'s rule).
Pinned to ``BundlePathNotFound`` by NAME since F3 added its sibling: both refusals quote the
caller's path, so a match on the path alone could no longer tell "this is nothing" from "this
is a document" - the substring two branches share."""
base = _write_tree(tmp_path, "korpus", dirs=2, per_dir=2)
with pytest.raises(ValueError, match="kategori-99"):
with pytest.raises(okf.BundlePathNotFound, match="kategori-99"):
_read_dir(base, "kategori-99")

224
tests/test_read_dir_wrong_rung_loadbearing.py Normal file
View file

@ -0,0 +1,224 @@
"""F3 - ``read_dir`` on a path that names a DOCUMENT answers with the rung that reads it.
**The measurement, and the premise it fells.** The live S7 acid test
(``docs/2026-09-07-syretest-s7-prepass-k2.md`` § 4) recorded arm C calling
``read_dir('del-ii-bilag-6-teknisk-oppsett')`` and then
``read_dir('del-ii-bilag-6-teknisk-oppsett.md')``; both were refused, and the two rounds came out
of a bounded round cap. The order reads that as "the path named a document that EXISTS (with or
without ``.md``)" and asks whether the refusal can name ``read_file`` and the real path, the way
``explore.DirectoryPathRefused`` already does in the other direction.
**Measured against the base that ran it, the premise is false.** That base holds 27 directories
named ``del-ii-bilag-N-...`` and 12 root documents named ``inbox-del-ii-bilag-N-....md``. The path
the model asked for matches NEITHER: it is the directory-naming convention applied to a document
whose real name carries an ``inbox-`` prefix. So the live two rounds were the UNKNOWN-path class,
not the wrong-RUNG class, and nothing in this file would have saved them - arm (g) is that fact,
gated, so no later reader can mistake this delivery for a fix of the measured cost.
**What IS real is the asymmetry.** ``read_file`` on a directory raises ``DirectoryPathRefused`` and
names ``read_dir``; ``read_dir`` on a document raised the generic "no such directory" and named
neither the rung that reads it nor the path it would take. The symmetry the order asked about is
absent in exactly one direction, and that is what this file closes.
The lookup is built from ``context_files``, NEVER ``files`` (arm (e)): a suggestion built from the
walk would name a ``type: verdict`` document by path - advertising, in a refusal, the one layer no
listing mentions and ``read_file`` refuses outright. It honours the ``dimension`` scope for the
same reason one rung up (arm (f)): naming a document the run would then refuse to open is the
"filter in name only" S2c measured, inverted.
Arms: (a) exact name * (b) the same name without ``.md`` * (c) an unknown path keeps its own
wording and never names ``read_file`` * (d) anti-vacuity: the named path is usable VERBATIM *
(e) the verdict layer is never named * (f) a foreign-dimension document is never named * (g) the
LIVE case is not this class * (h) the refusal is a ``ValueError``, not the crash channel.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any
import pytest
from portfolio_optimiser import okf
from portfolio_optimiser.explore import navigator_tools
#: ASCII-clean throughout: a tool result is serialised with ``\uXXXX`` escapes, and a probe with
#: Norwegian characters was red against a working implementation in session 99.
_ALPHA = "notat-alpha"
_GAMMA = "inbox-notat-gamma"
def _tools(bundle_dir: Path, *, dimension: str | None = None) -> dict[str, Any]:
return {t.name: t for t in navigator_tools((str(bundle_dir),), dimension=dimension)}
def _read_dir(bundle_dir: Path, path: str, *, dimension: str | None = None) -> dict[str, Any]:
tools = _tools(bundle_dir, dimension=dimension)
return tools["read_dir"].func(bundle_id=bundle_dir.name, path=path)
def _read_file(bundle_dir: Path, path: str) -> str:
return _tools(bundle_dir)["read_file"].func(bundle_id=bundle_dir.name, path=path)
def _base(root: Path) -> Path:
"""A base carrying every class the refusal must tell apart: a plain concept document at the
top, a real directory, a ``type: verdict`` document, a foreign-dimension document, and a
document whose real name carries the ``inbox-`` prefix the live model dropped."""
base = root / "korpus"
(base / "arkiv").mkdir(parents=True)
(base / f"{_ALPHA}.md").write_text(
"---\ntype: concept\ntitle: Notat alpha\n---\n\nalpha body.\n", encoding="utf-8"
)
(base / f"{_GAMMA}.md").write_text(
"---\ntype: concept\ntitle: Notat gamma\n---\n\ngamma body.\n", encoding="utf-8"
)
(base / "dom-beta.md").write_text(
"---\ntype: verdict\ntitle: Dom beta\n---\n\nverdict body.\n", encoding="utf-8"
)
(base / "energi-notat.md").write_text(
"---\ntype: concept\ntitle: Energi\ndimension: energi\n---\n\nenergi body.\n",
encoding="utf-8",
)
(base / "arkiv" / "dok-a.md").write_text(
"---\ntype: concept\ntitle: Dokument A\n---\n\narkiv body.\n", encoding="utf-8"
)
(base / "arkiv" / "index.md").write_text(
"---\ntype: index\n---\n\n- [Dokument A](dok-a.md)\n", encoding="utf-8"
)
(base / "index.md").write_text(
"---\ntype: index\n---\n\n"
f"- [Notat alpha]({_ALPHA}.md)\n"
f"- [Notat gamma]({_GAMMA}.md)\n"
"- [Dom beta](dom-beta.md)\n"
"- [Energi](energi-notat.md)\n"
"- [arkiv](arkiv/index.md)\n",
encoding="utf-8",
)
return base
# --- (a)/(b) the wrong rung, answered as such -----------------------------------------------------
def test_read_dir_on_a_documents_exact_name_names_read_file_and_the_path(tmp_path: Path) -> None:
"""(a) The symmetry ``explore.DirectoryPathRefused`` already has in the other direction. A
refusal that only says "no" leaves the caller with the same next move it just made."""
base = _base(tmp_path)
with pytest.raises(okf.DocumentPathRefused) as excinfo:
_read_dir(base, f"{_ALPHA}.md")
message = str(excinfo.value)
assert "read_file" in message, "the refusal does not name the rung that reads a document"
assert f"{_ALPHA}.md" in message, "the refusal does not name the path read_file would take"
def test_the_suffix_is_not_what_makes_it_a_document(tmp_path: Path) -> None:
"""(b) The live model asked BOTH ways in two consecutive rounds. A refusal that only knew the
``.md`` form would answer one of them and not the other."""
base = _base(tmp_path)
with pytest.raises(okf.DocumentPathRefused) as excinfo:
_read_dir(base, _ALPHA)
message = str(excinfo.value)
assert "read_file" in message
assert f"{_ALPHA}.md" in message, (
"the refusal echoed the caller's path instead of the document's real name; a path that "
"never existed is worse than no path"
)
# --- (c) the two branches must not share their wording --------------------------------------------
def test_an_unknown_path_keeps_its_own_wording(tmp_path: Path) -> None:
"""(c) Two refusals that share a substring cannot be told apart by a test OR by a model. The
unknown-path branch is unchanged and must stay unable to claim a document exists."""
base = _base(tmp_path)
with pytest.raises(okf.BundlePathNotFound) as excinfo:
_read_dir(base, "kategori-99")
message = str(excinfo.value)
assert "read_file" not in message, (
"the unknown-path refusal names read_file, so the two branches say the same thing about "
"two different facts"
)
assert not isinstance(excinfo.value, okf.DocumentPathRefused)
# --- (d) anti-vacuity: the named path must WORK ---------------------------------------------------
def test_the_named_path_is_usable_verbatim(tmp_path: Path) -> None:
"""(d) The ``_index_excerpt`` rule, one rung down: the caller is a model, so a suggested path
that does not resolve is worse than none. Proven by feeding it back."""
base = _base(tmp_path)
with pytest.raises(okf.DocumentPathRefused) as excinfo:
_read_dir(base, _ALPHA)
named = str(excinfo.value).split("'")[-2]
assert "alpha body." in _read_file(base, named)
# --- (e)/(f) the two gates the suggestion must not walk around ------------------------------------
def test_the_verdict_layer_is_never_named(tmp_path: Path) -> None:
"""(e) Built from ``context_files``, never ``files``. A suggestion built from the walk would
hand a navigator the path of a prior verdict - the one layer no listing mentions and
``read_file`` refuses outright (order 20260904T172353Z)."""
base = _base(tmp_path)
with pytest.raises(okf.BundlePathNotFound) as excinfo:
_read_dir(base, "dom-beta.md")
assert "read_file" not in str(excinfo.value)
assert not isinstance(excinfo.value, okf.DocumentPathRefused)
def test_a_foreign_dimension_document_is_never_named(tmp_path: Path) -> None:
"""(f) §4.1a, inverted: naming a document the run would then refuse to open is the "filter in
name only" S2c measured. ONE predicate (``in_dimension``) serves the listing and this."""
base = _base(tmp_path)
with pytest.raises(okf.BundlePathNotFound) as excinfo:
_read_dir(base, "energi-notat.md", dimension="tunnel")
assert "read_file" not in str(excinfo.value)
# The control: without a scope the SAME path is the wrong-rung class, so the arm above is the
# dimension deciding rather than the document being invisible.
with pytest.raises(okf.DocumentPathRefused):
_read_dir(base, "energi-notat.md")
# --- (g) the LIVE case is not this class ----------------------------------------------------------
def test_the_measured_live_path_is_still_the_unknown_class(tmp_path: Path) -> None:
"""(g) The felled premise, gated. The live base holds ``inbox-<name>.md`` documents beside
``<name>``-shaped directories, and the model asked for the un-prefixed form. That names no
document, so it is refused as unknown - and this delivery does NOT recover the two rounds the
order measured. Resolving it would mean guessing which document a caller meant, which is
invention rather than validation (``write_concept_file``'s rule)."""
base = _base(tmp_path)
assert (base / f"{_GAMMA}.md").exists(), "the fixture must hold the prefixed document"
with pytest.raises(okf.BundlePathNotFound) as excinfo:
_read_dir(base, _GAMMA.removeprefix("inbox-"))
assert "read_file" not in str(excinfo.value)
assert not isinstance(excinfo.value, okf.DocumentPathRefused)
# --- (h) the channel ------------------------------------------------------------------------------
def test_the_refusal_lands_on_the_refusal_tuple(tmp_path: Path) -> None:
"""(h) The ``BundlePathNotFound``/``DimensionScopeRefused`` precedent: the caller is a model
choosing a path, so this belongs on the CLI's refusal tuple and hosting's 400 arm rather than
the crash channel."""
assert issubclass(okf.DocumentPathRefused, ValueError)

240
tests/test_require_cost_baseline_loadbearing.py Normal file
View file

@ -0,0 +1,240 @@
"""F4 - a run that must be anchored can say so, and is refused before it costs anything.
**The measurement** (``docs/2026-09-07-syretest-s7-prepass-k2.md`` § 8, re-measured here against
the four artefacts the live run left). All three paid arms stamped
``cost_baseline_anchored: False``, so the validator's stage 0 - the ONE stage that tells a
fabricated cost line from a real one - was skipped, and every arm invented its codes:
``ENGRAVE_MARK``, ``RITB-HOURS``/``SYSINT-HOURS``, ``Material_Cost_Concrete``. Stages 2/4/5 judged
those numbers against each other and did their job; nothing tied a single line to K2.
**Option (c) is felled by measurement, not by preference.** The order offers "make
``--derive-cost-baseline`` reachable where it is not today". Measured against the base that ran:
``derive_cost_baseline`` refuses with *no concept file carries a markdown table whose header names
all three of code/quantity/unit_cost* - K2's delivered price schedule is a pandoc SIMPLE table with
ONE column header (``Prisskjema``) and every value collapsed into it. Reaching it would mean
inventing a rule for a form nobody has measured, which is precisely the honesty limit MAJOR-4
wrote down for itself.
**Option (b), and why not (a).** The visibility exists (``cost_baseline_notice``) and is honest;
what it cannot do is stop a machine-readable artefact that says ``validator_decision: validated``
over lines nothing anchored. So: an OPT-IN flag, never a default. Every bundle without a
``cost-baseline.json`` - every commons-owned golden - runs exactly as before, which is what (a)
was protecting; a caller who needs the guarantee asks for it by name.
The gate sits at the ONE place both branches have bound ``baseline`` and BEFORE the dry-run cut, so
it fires on the free trip too (arm (d)) and, on the paid one, before the first model call (arm (a)
asserts NULL calls, never merely the exception - at the exit code a refusal after the spend looks
identical to one before).
Arms: (a) refused, zero model calls * (b) control: without the flag the same base runs *
(c) an anchored base is untouched * (d) the dry run refuses too, with its own control *
(e) it composes with ``--derive-cost-baseline`` * (f)(g)(h) three CLI refusals BY NAME, each with
an rc-0 control on an argv that would otherwise be accepted * (i) the CLI wiring, measured on
calls * (j) the channel.
"""
from __future__ import annotations
import json
import shutil
from pathlib import Path
from typing import Any
import pytest
from portfolio_optimiser import run
from portfolio_optimiser.simulation import ScriptedChatClient
_FIXTURES = Path(__file__).parent / "fixtures"
_PRICED = str(_FIXTURES / "k2-prisskjema-SYNTETISK")
_EXAMPLES = Path(__file__).resolve().parents[1] / "shared" / "examples"
#: A base that SHIPS a hand-written ``cost-baseline.json`` - the anchored control.
_ANCHORED_SOURCE = _EXAMPLES / "tunnel-hauglia"
_IR_PROJECTION = {
"project_id": "K2",
"measure": "PLACEHOLDER - authored by this test to satisfy the bundle contract",
"affected_items": [{"code": "21.1", "quantity": 1250, "unit_cost": 850.0}],
"claimed_saving_nok": 1000.0,
}
def _runnable(source: str, tmp_path: Path, *, name: str = "runnable") -> str:
root = tmp_path / name
shutil.copytree(source, root)
(root / "validator-input.json").write_text(json.dumps(_IR_PROJECTION), encoding="utf-8")
return str(root)
def _scripted(sink: list[str] | None = None) -> Any:
return lambda role: ScriptedChatClient(sink=sink, role=role, default_reply="ok")
# --- (a)/(b)/(c) the library seam -----------------------------------------------------------------
async def test_an_unanchored_run_is_refused_before_it_costs_anything(tmp_path: Path) -> None:
"""(a) The refusal, and the assert that makes it worth having: NULL model calls. Measured on
calls rather than on the exception, because a refusal placed after the spend raises the same
exception (session 57's rule, session 82's arm)."""
bundle_dir = _runnable(_PRICED, tmp_path)
sink: list[str] = []
with pytest.raises(run.UnanchoredRunRefused):
await run.run_project(
"K2",
"local",
docs_dir=bundle_dir,
bundle_dir=bundle_dir,
require_cost_baseline=True,
client_factory=_scripted(sink),
)
assert sink == [], f"the run reached the model before it was refused ({len(sink)} calls)"
async def test_control_without_the_flag_the_same_base_runs(tmp_path: Path) -> None:
"""(b) The default is byte-for-byte the path every existing caller takes - which is what
option (a) of the order was protecting, kept rather than argued away."""
bundle_dir = _runnable(_PRICED, tmp_path)
report = await run.run_project(
"K2", "local", docs_dir=bundle_dir, bundle_dir=bundle_dir, live_dry_run=True
)
assert isinstance(report, run.DryRunReport)
assert report.cost_baseline_anchored is False
async def test_an_anchored_base_is_untouched_by_the_flag(tmp_path: Path) -> None:
"""(c) The gate reads the SAME ``baseline`` the validator is handed, so a base that ships the
file passes it. Without this arm the flag could be a refusal that always fires."""
bundle_dir = _runnable(str(_ANCHORED_SOURCE), tmp_path, name="anchored")
report = await run.run_project(
"K2",
"local",
docs_dir=bundle_dir,
bundle_dir=bundle_dir,
require_cost_baseline=True,
live_dry_run=True,
)
assert isinstance(report, run.DryRunReport)
assert report.cost_baseline_anchored is True
# --- (d) the free trip refuses too ----------------------------------------------------------------
async def test_the_dry_run_refuses_rather_than_reporting_an_unanchored_run(tmp_path: Path) -> None:
"""(d) The gate is ABOVE the dry-run cut on purpose: the dry run's whole job is to say what a
real run would do, and it is the cheapest place to learn that this one may not run at all."""
bundle_dir = _runnable(_PRICED, tmp_path)
with pytest.raises(run.UnanchoredRunRefused):
await run.run_project(
"K2",
"local",
docs_dir=bundle_dir,
bundle_dir=bundle_dir,
require_cost_baseline=True,
live_dry_run=True,
)
# --- (e) it composes with the derivation ----------------------------------------------------------
async def test_it_composes_with_the_derivation(tmp_path: Path) -> None:
"""(e) ``--derive-cost-baseline`` is the OTHER half of the answer: derive, and the requirement
is met. Two flags that could not be combined would leave the guarantee unreachable on exactly
the bases MAJOR-4 was built for."""
bundle_dir = _runnable(_PRICED, tmp_path)
report = await run.run_project(
"K2",
"local",
docs_dir=bundle_dir,
bundle_dir=bundle_dir,
derive_cost_baseline=True,
require_cost_baseline=True,
live_dry_run=True,
)
assert isinstance(report, run.DryRunReport)
assert report.cost_baseline_anchored is True
# --- (f)/(g)/(h) three CLI refusals, each with an rc-0 control -------------------------------------
def test_cli_requires_bundle_dir(capsys: pytest.CaptureFixture[str]) -> None:
"""(f) The road path is anchored by construction, so on it the flag could never fire. A flag
that cannot fire is a claim the surface makes about itself (the Fase-3 class)."""
rc = run.main(["P1", "--docs-dir", "docs", "--require-cost-baseline"])
assert rc == 1
assert "--require-cost-baseline" in capsys.readouterr().err
def test_cli_is_refused_in_portfolio_mode(capsys: pytest.CaptureFixture[str]) -> None:
"""(g) BY NAME, never by falling through to the ``--bundle-dir`` requirement: that message
would tell an operator who wrote ``--portfolio --require-cost-baseline`` to add the one flag
this mode also refuses (the ``--derive-cost-baseline`` precedent, and session 57's)."""
rc = run.main(["--portfolio", "--require-cost-baseline"])
assert rc == 1
err = capsys.readouterr().err
assert "--portfolio" in err, "the refusal never names the mode that refused it"
def test_cli_is_refused_in_report_mode(tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""(h) Report mode returns ABOVE every run dispatch, so an omission from this allowlist is a
SILENT DROP rather than a refusal - the gap F4 itself measured on ``--plan-review``."""
ledger = tmp_path / "ledger.json"
ledger.write_text("[]", encoding="utf-8")
assert run.main(["--report", "--ledger", str(ledger)]) == 0, "the control argv must be ACCEPTED"
capsys.readouterr()
rc = run.main(["--report", "--ledger", str(ledger), "--require-cost-baseline"])
assert rc == 1
# The allowlist's own wording, not the flag name: this partition refuses generically, and the
# discriminator is the rc-0 control above -- the same argv without the flag is ACCEPTED.
assert "mode-exclusive" in capsys.readouterr().err
# --- (i) the CLI wiring ---------------------------------------------------------------------------
def test_cli_wiring_refuses_the_unanchored_run_without_paying(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
"""(i) The flag must REACH ``run_project``. rc 1 alone would also be a parse error, so the
discriminator is that no model call happened - and the control proves the same argv without
the flag is accepted and DOES call."""
monkeypatch.delenv("PORTFOLIO_MODEL_MAP", raising=False)
bundle_dir = _runnable(_PRICED, tmp_path)
sink: list[str] = []
monkeypatch.setattr("portfolio_optimiser.run._default_factory", lambda profile: _scripted(sink))
argv = ["K2", "--docs-dir", bundle_dir, "--bundle-dir", bundle_dir]
assert run.main([*argv, "--live-dry-run"]) == 0, "the control argv must be ACCEPTED"
rc = run.main([*argv, "--require-cost-baseline"])
assert rc == 1
assert sink == [], f"the run was refused only after paying for it ({len(sink)} calls)"
assert "run refused" in capsys.readouterr().err
# --- (j) the channel ------------------------------------------------------------------------------
def test_the_refusal_lands_on_the_refusal_tuple() -> None:
"""(j) A ``ValueError``, the ``BundleIdMismatch``/``CostBaselineDerivationError`` precedent: an
argv that is wrong about what this base can offer belongs on the CLI's refusal tuple, never on
the crash channel."""
assert issubclass(run.UnanchoredRunRefused, ValueError)