portfolio-optimiser/tests/test_read_dir_wrong_rung_loadbearing.py
Kjell Tore Guttormsen eb4137415c feat(prepass,explore): padding dies at the prompt; a refusal the model can act on is a return value [skip-docs]
Order 20260908T195801Z. Findings 4 and 5 from the S7 acid test, then the two things
finding 99 measured and deliberately did not fix (D3, D2).

No user-facing surface changes: no new flag, no new command, no changed output
contract. Both seams are internal (the pre-pass rendering, and the shape a tool
answers a model with), so [skip-docs] rather than a README edit that would describe
nothing an operator can do differently.

FINDING 4 -- MEASURED, NOTHING BUILT. K2's price schedule IS readable without
guessing (8 column spans, 71 of 91 non-blank rows give >= 2 cells, the split stable
for K = 2..64). But 0 of 92 rows name all three of code/quantity/unit_cost -- also
under a looser substring match -- and 0 of 91 data rows carry code + quantity +
amount. The triple is not formatted away; it is not in the document. It is a price
SUMMARY plus nine rate cards whose unit-price columns are empty (pre-award). The
order's binding decision rule therefore falls against building:
--derive-cost-baseline keeps refusing, and MAJOR-4's own honesty limit holds.

FINDING 5 -- BUILT. Measured on the actual rendering path (concept_text, not the
raw file): the delivered excerpt is 104 lines / 67 245 chars, carrying 208 interior
whitespace runs, 117 of them >= 100 and the longest 887 -- 56 806 of 67 245
characters = 84.5 %, over 72 of 104 lines. collapse_padding, called from
_data_blocks (the one renderer both arms share, and therefore AFTER
verify_against_bundle -- collapsing in concept_text would break every payload's own
digest), gives -72.4 %: line count invariant, non-whitespace byte-identical, leading
indentation untouched, no number changed.

F99-D3 -- read_file / read_dir / read_bundle now RETURN their refusal. MAF turns a
tool raise into "Error: Function failed." (_tools.py:1410-1432, :1427) and counts it
against DEFAULT_MAX_CONSECUTIVE_ERRORS_PER_REQUEST = 3, so everything the refusing
arm knows is destroyed on the way out. The gates are unchanged; the property they
exist for -- the reason travels, the bytes never do -- is now asserted explicitly on
the returned value. The arm is keyed on named classes, never bare Exception, because
ExplorationError is itself a RuntimeError subclass.

F99-D2 -- the invariant row, plus one for finding 5 (a stated deviation from "one
row only": finding 5 is a separately built seam and the ledger's standing rule
requires its own row).

19 existing arms rewritten, never deleted and never weakened: where the class
carried a distinction, the refusal KIND carries it now.

13 mutations, all red against the whole suite (W1-W5, M1-M8), each restored from
scratchpad with shasum -c. Control 1543 passed / 5 skipped (from 1529/5, a strict
superset, 0 removed). Golden demo-transcript.stdout unchanged
(shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f).

Measurement: docs/2026-09-08-funn-4-5-og-read-nekt.md

Co-Authored-By: Claude <Opus 5>
2026-09-08 23:36:54 +02:00

221 lines
10 KiB
Python

"""F3 - ``read_dir`` on a path that names a DOCUMENT answers with the rung that reads it.
**The measurement, and the premise it fells.** The live S7 acid test
(``docs/2026-09-07-syretest-s7-prepass-k2.md`` § 4) recorded arm C calling
``read_dir('del-ii-bilag-6-teknisk-oppsett')`` and then
``read_dir('del-ii-bilag-6-teknisk-oppsett.md')``; both were refused, and the two rounds came out
of a bounded round cap. The order reads that as "the path named a document that EXISTS (with or
without ``.md``)" and asks whether the refusal can name ``read_file`` and the real path, the way
``explore.DirectoryPathRefused`` already does in the other direction.
**Measured against the base that ran it, the premise is false.** That base holds 27 directories
named ``del-ii-bilag-N-...`` and 12 root documents named ``inbox-del-ii-bilag-N-....md``. The path
the model asked for matches NEITHER: it is the directory-naming convention applied to a document
whose real name carries an ``inbox-`` prefix. So the live two rounds were the UNKNOWN-path class,
not the wrong-RUNG class, and nothing in this file would have saved them - arm (g) is that fact,
gated, so no later reader can mistake this delivery for a fix of the measured cost.
**What IS real is the asymmetry.** ``read_file`` on a directory raises ``DirectoryPathRefused`` and
names ``read_dir``; ``read_dir`` on a document raised the generic "no such directory" and named
neither the rung that reads it nor the path it would take. The symmetry the order asked about is
absent in exactly one direction, and that is what this file closes.
The lookup is built from ``context_files``, NEVER ``files`` (arm (e)): a suggestion built from the
walk would name a ``type: verdict`` document by path - advertising, in a refusal, the one layer no
listing mentions and ``read_file`` refuses outright. It honours the ``dimension`` scope for the
same reason one rung up (arm (f)): naming a document the run would then refuse to open is the
"filter in name only" S2c measured, inverted.
Arms: (a) exact name * (b) the same name without ``.md`` * (c) an unknown path keeps its own
wording and never names ``read_file`` * (d) anti-vacuity: the named path is usable VERBATIM *
(e) the verdict layer is never named * (f) a foreign-dimension document is never named * (g) the
LIVE case is not this class * (h) the refusal is a ``ValueError``, not the crash channel.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any
from portfolio_optimiser import okf
from portfolio_optimiser.explore import navigator_tools
#: ASCII-clean throughout: a tool result is serialised with ``\uXXXX`` escapes, and a probe with
#: Norwegian characters was red against a working implementation in session 99.
_ALPHA = "notat-alpha"
_GAMMA = "inbox-notat-gamma"
def _tools(bundle_dir: Path, *, dimension: str | None = None) -> dict[str, Any]:
return {t.name: t for t in navigator_tools((str(bundle_dir),), dimension=dimension)}
def _read_dir(bundle_dir: Path, path: str, *, dimension: str | None = None) -> dict[str, Any]:
tools = _tools(bundle_dir, dimension=dimension)
return tools["read_dir"].func(bundle_id=bundle_dir.name, path=path)
def _read_file(bundle_dir: Path, path: str) -> str:
return _tools(bundle_dir)["read_file"].func(bundle_id=bundle_dir.name, path=path)
def _base(root: Path) -> Path:
"""A base carrying every class the refusal must tell apart: a plain concept document at the
top, a real directory, a ``type: verdict`` document, a foreign-dimension document, and a
document whose real name carries the ``inbox-`` prefix the live model dropped."""
base = root / "korpus"
(base / "arkiv").mkdir(parents=True)
(base / f"{_ALPHA}.md").write_text(
"---\ntype: concept\ntitle: Notat alpha\n---\n\nalpha body.\n", encoding="utf-8"
)
(base / f"{_GAMMA}.md").write_text(
"---\ntype: concept\ntitle: Notat gamma\n---\n\ngamma body.\n", encoding="utf-8"
)
(base / "dom-beta.md").write_text(
"---\ntype: verdict\ntitle: Dom beta\n---\n\nverdict body.\n", encoding="utf-8"
)
(base / "energi-notat.md").write_text(
"---\ntype: concept\ntitle: Energi\ndimension: energi\n---\n\nenergi body.\n",
encoding="utf-8",
)
(base / "arkiv" / "dok-a.md").write_text(
"---\ntype: concept\ntitle: Dokument A\n---\n\narkiv body.\n", encoding="utf-8"
)
(base / "arkiv" / "index.md").write_text(
"---\ntype: index\n---\n\n- [Dokument A](dok-a.md)\n", encoding="utf-8"
)
(base / "index.md").write_text(
"---\ntype: index\n---\n\n"
f"- [Notat alpha]({_ALPHA}.md)\n"
f"- [Notat gamma]({_GAMMA}.md)\n"
"- [Dom beta](dom-beta.md)\n"
"- [Energi](energi-notat.md)\n"
"- [arkiv](arkiv/index.md)\n",
encoding="utf-8",
)
return base
# --- (a)/(b) the wrong rung, answered as such -----------------------------------------------------
def test_read_dir_on_a_documents_exact_name_names_read_file_and_the_path(tmp_path: Path) -> None:
"""(a) The symmetry ``explore.DirectoryPathRefused`` already has in the other direction. A
refusal that only says "no" leaves the caller with the same next move it just made."""
base = _base(tmp_path)
refusal = _read_dir(base, f"{_ALPHA}.md")
assert refusal["refusal"] == okf.DocumentPathRefused.__name__
message = refusal["refused"]
assert "read_file" in message, "the refusal does not name the rung that reads a document"
assert f"{_ALPHA}.md" in message, "the refusal does not name the path read_file would take"
def test_the_suffix_is_not_what_makes_it_a_document(tmp_path: Path) -> None:
"""(b) The live model asked BOTH ways in two consecutive rounds. A refusal that only knew the
``.md`` form would answer one of them and not the other."""
base = _base(tmp_path)
refusal = _read_dir(base, _ALPHA)
assert refusal["refusal"] == okf.DocumentPathRefused.__name__
message = refusal["refused"]
assert "read_file" in message
assert f"{_ALPHA}.md" in message, (
"the refusal echoed the caller's path instead of the document's real name; a path that "
"never existed is worse than no path"
)
# --- (c) the two branches must not share their wording --------------------------------------------
def test_an_unknown_path_keeps_its_own_wording(tmp_path: Path) -> None:
"""(c) Two refusals that share a substring cannot be told apart by a test OR by a model. The
unknown-path branch is unchanged and must stay unable to claim a document exists."""
base = _base(tmp_path)
refusal = _read_dir(base, "kategori-99")
message = refusal["refused"]
assert "read_file" not in message, (
"the unknown-path refusal names read_file, so the two branches say the same thing about "
"two different facts"
)
# F99-D3: the class distinction that used to be observable through ``isinstance`` is carried
# by the ``refusal`` KIND now that the refusal is returned rather than raised. Dropping it
# would have been a silent weakening of exactly this assertion.
assert refusal["refusal"] == okf.BundlePathNotFound.__name__
assert refusal["refusal"] != okf.DocumentPathRefused.__name__
# --- (d) anti-vacuity: the named path must WORK ---------------------------------------------------
def test_the_named_path_is_usable_verbatim(tmp_path: Path) -> None:
"""(d) The ``_index_excerpt`` rule, one rung down: the caller is a model, so a suggested path
that does not resolve is worse than none. Proven by feeding it back."""
base = _base(tmp_path)
named = _read_dir(base, _ALPHA)["refused"].split("'")[-2]
assert "alpha body." in _read_file(base, named)
# --- (e)/(f) the two gates the suggestion must not walk around ------------------------------------
def test_the_verdict_layer_is_never_named(tmp_path: Path) -> None:
"""(e) Built from ``context_files``, never ``files``. A suggestion built from the walk would
hand a navigator the path of a prior verdict - the one layer no listing mentions and
``read_file`` refuses outright (order 20260904T172353Z)."""
base = _base(tmp_path)
refusal = _read_dir(base, "dom-beta.md")
assert "read_file" not in refusal["refused"]
assert refusal["refusal"] == okf.BundlePathNotFound.__name__
def test_a_foreign_dimension_document_is_never_named(tmp_path: Path) -> None:
"""(f) §4.1a, inverted: naming a document the run would then refuse to open is the "filter in
name only" S2c measured. ONE predicate (``in_dimension``) serves the listing and this."""
base = _base(tmp_path)
scoped = _read_dir(base, "energi-notat.md", dimension="tunnel")
assert "read_file" not in scoped["refused"]
assert scoped["refusal"] == okf.BundlePathNotFound.__name__
# The control: without a scope the SAME path is the wrong-rung class, so the arm above is the
# dimension deciding rather than the document being invisible.
assert _read_dir(base, "energi-notat.md")["refusal"] == okf.DocumentPathRefused.__name__
# --- (g) the LIVE case is not this class ----------------------------------------------------------
def test_the_measured_live_path_is_still_the_unknown_class(tmp_path: Path) -> None:
"""(g) The felled premise, gated. The live base holds ``inbox-<name>.md`` documents beside
``<name>``-shaped directories, and the model asked for the un-prefixed form. That names no
document, so it is refused as unknown - and this delivery does NOT recover the two rounds the
order measured. Resolving it would mean guessing which document a caller meant, which is
invention rather than validation (``write_concept_file``'s rule)."""
base = _base(tmp_path)
assert (base / f"{_GAMMA}.md").exists(), "the fixture must hold the prefixed document"
refusal = _read_dir(base, _GAMMA.removeprefix("inbox-"))
assert "read_file" not in refusal["refused"]
assert refusal["refusal"] == okf.BundlePathNotFound.__name__
# --- (h) the channel ------------------------------------------------------------------------------
def test_the_refusal_lands_on_the_refusal_tuple(tmp_path: Path) -> None:
"""(h) The ``BundlePathNotFound``/``DimensionScopeRefused`` precedent: the caller is a model
choosing a path, so this belongs on the CLI's refusal tuple and hosting's 400 arm rather than
the crash channel."""
assert issubclass(okf.DocumentPathRefused, ValueError)