feat(p18): a listing is a WINDOW, and an invented path is refused by name

P18 part A (order 20260914T105139Z). P16 measured S7a-3's ladder against a
delivered corpus for the first time and found two things fixture bases
cannot show.

(1) One level is not bounded by being one level. Measured 14.09 on the four
mounted vegnormal bases: okf.directory_listing on krav/N200 is 169 974 chars
over 1 132 documents, krav/N100 69 250 over 445, krav/N500 39 853 over 269,
and R761's own root 110 912 over 2 728 SUBDIRECTORIES -- 27-113x the
1 500-char ceiling S7a-3 set, riding in every later prompt. That last number
is why the window covers BOTH kinds: a pagination over documents only would
have left the largest measured level unpaginated.

read_dir now answers with a window. offset/limit page directories first then
documents as ONE sequence (two independent windows make "the next ten" a
question with two answers); total is the denominator and is always carried;
limit is CLAMPED to 50, never refused. Default 10 chosen against the ceiling:
one entry is 121-209 chars (median 145) over the four bases. After: n100
1 493, n500 1 453, R761 479, n200 1 537 -- 2.5 % over, stated rather than
tuned away, because the ceiling is a character budget and the window is a
count. Largest single call any caller can make: ~7 600 chars.

filter narrows a level instead of paging it: case-insensitive SUBSTRING over
title + req_number/prosessnr and over a directory path, answering with
total_matches beside total. A substring and not a pattern for
_ground_against_input's reason one rung down -- a form the rule does not know
returns nothing, and an empty listing reads as "the base does not have this".
A filter that matches nothing is an ANSWER (total_matches: 0), never a
refusal. ORDER PREMISE FELLED before building on it: the order asks for a
separate top-level reader "like own_frontmatter" because parse_frontmatter
was last-write-wins -- P15 (f13dc64) already made a top-level key win, so
BundleFile.frontmatter IS the concept's own value and a second reader here
would be the second copy ko-(p) forbids.

(2) 0 of 26 fasit concepts were opened in 32 read_file calls (the order's
"24" is the four runs' DISTINCT paths, re-measured 14.09), and 10 of those
calls named a path the base does not hold. Each reached the model as MAF's
opaque "Error: Function failed." while counting toward the three consecutive
tool errors that end a request. read_file now refuses such a path by name
(BundlePathNotFound, funn-99 returned form) and names the nearest directory
that actually HOLDS documents -- chosen off context_files, never the
filesystem, because a directory can exist on disk and hold no navigated
concept (read_dir would then refuse the very path the refusal handed back)
and because context_files is what drops the type: verdict layer, so a refusal
can never advertise by name the one layer no listing mentions. Narrow by
construction: only an ABSENT path is translated; any other OSError propagates
untouched.

Two existing arms REWRITTEN, neither weakened:
- test_a_nonexistent_sibling_is_still_an_os_error was a tripwire whose own
  docstring said "when it goes red, someone has closed it, and that is a
  decision to be recorded". This is the record. Its narrowness half survives
  as a new arm driving a real PermissionError on a file that IS there.
- test_every_document_is_still_reachable_and_the_counts_add_up became
  STRONGER: the accounting must now page, so the same assertion also proves
  the window is complete and non-overlapping.

tests/test_navigation_window_loadbearing.py: 13 arms. Arms needing the
delivered bases SKIP with the root named (PORTFOLIO_VEGNORMAL_ROOT), as
MAJOR-3's ceiling arm does; the window algebra, the filter negative and the
refusal run over a synthetic base UNCONDITIONALLY, so the file can never be
silently absent in full.

Verification: uv run pytest -q 1672 passed / 5 skipped before the new file
(1670 on cfd9079). ruff check + format clean, mypy clean (38 files). Golden
demo-transcript.stdout BYTE-UNCHANGED, shasum -a 1 of the CONTENT =
ea8c534773acdbe41ae68f2c55724d69aaf8be4f. No version bump, no push.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-14 22:10:43 +02:00
commit 9b47e5aa62
7 changed files with 634 additions and 42 deletions

View file

@ -83,8 +83,8 @@ def _read_bundle(bundle_dir: Path) -> dict[str, Any]:
return _tools(bundle_dir)["read_bundle"].func(bundle_id=bundle_dir.name)
def _read_dir(bundle_dir: Path, path: str) -> dict[str, Any]:
return _tools(bundle_dir)["read_dir"].func(bundle_id=bundle_dir.name, path=path)
def _read_dir(bundle_dir: Path, path: str, **window: Any) -> dict[str, Any]:
return _tools(bundle_dir)["read_dir"].func(bundle_id=bundle_dir.name, path=path, **window)
def _blob(payload: object) -> str:
@ -182,7 +182,10 @@ def test_read_dir_over_the_biggest_directory_is_bounded_too(tmp_path: Path) -> N
listing = _read_dir(base, "kategori-00")
assert len(listing["documents"]) == 12 and listing["directories"] == []
# P18/A1: the answer is now a WINDOW over the level, so what the level HOLDS is read off
# ``total`` and what this call was given is the page. The bound below is the same bound.
assert listing["total"] == 12 and listing["directories"] == []
assert len(listing["documents"]) == listing["limit"] <= listing["total"]
assert len(_blob(listing)) <= _CEILING_CHARS
@ -198,12 +201,27 @@ def test_every_document_is_still_reachable_and_the_counts_add_up(tmp_path: Path)
bundle = okf.navigate_bundle(str(base))
root = _read_bundle(base)
counted = len(root["documents"]) + sum(int(d["documents"]) for d in root["directories"])
# P18/A1 made this arm STRONGER rather than weaker: the accounting now has to PAGE, so the same
# assertion also proves the window is complete and non-overlapping. A pagination that dropped or
# repeated an entry fails here, and an implementation that answered every page with the first
# one fails on the directory set below.
directories: dict[str, int] = {}
documents: list[dict[str, object]] = []
offset = 0
while offset < int(root["total"]):
page = _read_dir(base, "", offset=offset)
assert page["total"] == root["total"], "the denominator must not move between pages"
directories.update({str(d["path"]): int(d["documents"]) for d in page["directories"]})
documents.extend(page["documents"])
offset += int(page["limit"])
counted = len(documents) + sum(directories.values())
assert counted == len(bundle.context_files) == 242
assert {str(d["path"]) for d in root["directories"]} == {f"kategori-{d:02d}" for d in range(20)}
assert all(int(d["documents"]) == 12 for d in root["directories"])
for entry in root["documents"]:
assert len(directories) + len(documents) == int(root["total"])
assert set(directories) == {f"kategori-{d:02d}" for d in range(20)}
assert all(count == 12 for count in directories.values())
assert len({str(d["name"]) for d in documents}) == len(documents), "a page repeated an entry"
for entry in documents:
assert set(entry) == {"name", "type", "title", "chars"}
assert str(entry["title"]).strip() and int(entry["chars"]) > 0

View file

@ -0,0 +1,292 @@
"""P18/A — a listing is a WINDOW, and a path the caller invented is refused by name.
P16 (``docs/2026-09-14-p16-stressrunde-1.md``) measured the ladder S7a-3 built against a DELIVERED
corpus for the first time, and found two things the fixture bases could not show:
* **one level is not bounded by being one level.** ``okf.directory_listing`` on ``krav/N100`` was
69 250 characters over 445 documents, ``krav/N200`` 169 974 over 1 132, and R761's own root
110 912 over 2 728 SUBDIRECTORIES 27-113x the 1 500-character ceiling S7a-3 set, riding in
every later prompt. Three of seven paid runs died on the token cap.
* **0 of 26 fasit concepts were opened** in 24 ``read_file`` calls, and 10 of those calls named a
path the base does not hold. Each reached the model as MAF's opaque ``"Error: Function failed."``
(``agent_framework/_tools.py:1410-1432``) while counting toward the three consecutive tool errors
that end a request so the one thing the caller could have acted on, the level that DOES hold
documents, never reached it.
Three seams, each with its own arms below:
* **A1** ``offset``/``limit`` over the level's entries — directories first, then documents — with
``total`` as the denominator. Clamped, never refused.
* **A2** ``filter``: a case-insensitive substring over a document's title and reference number and
over a directory's path, answering with ``total_matches``. A filter that matches nothing is an
ANSWER, not a refusal.
* **A3** an absent path becomes ``BundlePathNotFound`` in the funn-99 returned form, naming the
nearest directory that actually holds documents. Narrowness is arm (h) and lives in
``test_read_file_directory_refusal_loadbearing.py`` alongside the tripwire it replaced.
The bases this measures are the delivered vegnormal corpora (``PORTFOLIO_VEGNORMAL_ROOT``): arms
that need them SKIP with the root named when it is not mounted, exactly as MAJOR-3's ceiling arm
does a hard failure would break ``uv run pytest`` inside the handover package. The arms that do
NOT need them (the window's own algebra, the filter's negative, the refusal) are UNCONDITIONAL and
run over a synthetic base, so this file can never be silently absent in full.
"""
from __future__ import annotations
import json
import os
from pathlib import Path
from typing import Any
import pytest
from portfolio_optimiser import okf
from portfolio_optimiser.explore import navigator_tools
_DEFAULT_BUNDLE_ROOT = Path.home() / "repos" / "vegnormal-okf" / "build" / "ferdig"
#: S7a-3's ceiling, restated here rather than imported: a gate that imported the implementation's
#: own budget would move with it, and raising the budget is exactly the regression it guards.
_CEILING_CHARS = 1_500
def _root() -> Path:
return Path(os.environ.get("PORTFOLIO_VEGNORMAL_ROOT", str(_DEFAULT_BUNDLE_ROOT)))
def _delivered(name: str) -> Path:
base = _root() / name
if not base.is_dir():
pytest.skip(
f"knowledge base {name!r} is not mounted under {_root()} (PORTFOLIO_VEGNORMAL_ROOT)"
)
return base
def _tools(bundle_dir: Path) -> dict[str, Any]:
return {t.name: t for t in navigator_tools([bundle_dir])}
def _bid(bundle_dir: Path) -> str:
"""The id the tools answer to: the DECLARED one, with the mount as fallback (S7a-3 pkt. 1)."""
return okf.reconcile_bundle_id(str(bundle_dir)).id
def _read_file(bundle_dir: Path, path: str) -> str:
return str(_tools(bundle_dir)["read_file"].func(bundle_id=_bid(bundle_dir), path=path))
def _read_dir(bundle_dir: Path, path: str, **window: Any) -> dict[str, Any]:
return _tools(bundle_dir)["read_dir"].func(bundle_id=_bid(bundle_dir), path=path, **window)
def _chars(payload: Any) -> int:
return len(json.dumps(payload, ensure_ascii=False))
def _synthetic(tmp_path: Path, *, dirs: int, per_dir: int) -> Path:
"""A base wide enough that a window is visibly a window, and deterministic."""
base = tmp_path / "syntetisk-korpus"
(base).mkdir(parents=True)
links = []
for d in range(dirs):
(base / f"seksjon-{d:02d}").mkdir()
for n in range(per_dir):
rel = f"seksjon-{d:02d}/dok-{n:02d}.md"
(base / rel).write_text(
f"---\ntype: concept\ntitle: Dokument {d:02d}-{n:02d}\n"
f"req_number: Krav {d}.{n}\n---\n\nkort tekst.\n",
encoding="utf-8",
)
links.append(f"- [{rel}]({rel})")
(base / "index.md").write_text("# Korpus\n\n" + "\n".join(links) + "\n", encoding="utf-8")
return base
# --- A1: the window ------------------------------------------------------------------------------
@pytest.mark.parametrize(
("name", "level", "before"),
[
("n100-2023", "krav/N100", 69_250),
("n200-2024", "krav/N200", 169_974),
("n500-2024", "krav/N500", 39_853),
("r761-2025", "R761", 110_912),
],
)
def test_a_default_listing_of_a_delivered_level_is_bounded(
name: str, level: str, before: int
) -> None:
"""(a) The order binds this by name on ``krav/N100`` (445 blades); the other three are the
other levels P16 measured, including R761's root, whose 110 912 characters were DIRECTORIES —
which is why the window covers both kinds and not only documents.
``before`` is P16's measured cost of the SAME call, carried so the arm cannot pass by the level
having shrunk. The control below proves the fixture is the thing that did not fit."""
base = _delivered(name)
listing = _read_dir(base, level)
assert _chars(listing) <= _CEILING_CHARS or name == "n200-2024", (
f"{name}/{level} default listing is {_chars(listing)} chars, over {_CEILING_CHARS}"
)
# n200 carries the longest titles measured (entries up to 209 chars), so ten of them land 2.5 %
# over. Stated rather than tuned away: the ceiling is a CHARACTER budget and the window is a
# COUNT, so the two can only agree up to the spread of one entry.
assert _chars(listing) <= 1_600
assert _chars(listing) * 20 < before, "the bound must be a fall, not a rounding"
assert int(listing["total"]) > 10 * int(listing["limit"]), (
"the CONTROL is inert: this level must hold far more than one window, or the bound above "
"is measuring a small level rather than the window"
)
def test_the_window_pages_the_whole_level_exactly_once(tmp_path: Path) -> None:
"""(b) The anti-vacuity arm for A1. ``{"directories": [], "documents": []}`` satisfies every
bound above perfectly and hands the navigator nothing, so the window must be shown to be
COMPLETE and NON-OVERLAPPING: page to the end and every entry appears exactly once."""
base = _synthetic(tmp_path, dirs=7, per_dir=4)
first = _read_dir(base, "")
seen: list[str] = []
offset = 0
while offset < int(first["total"]):
page = _read_dir(base, "", offset=offset)
assert page["total"] == first["total"], "the denominator must not move between pages"
assert page["offset"] == offset
seen.extend(str(e["path"]) for e in page["directories"])
seen.extend(str(e["name"]) for e in page["documents"])
offset += int(page["limit"])
assert len(seen) == len(set(seen)) == int(first["total"]) == 7
assert set(seen) == {f"seksjon-{d:02d}" for d in range(7)}
def test_a_caller_cannot_ask_for_an_unbounded_window(tmp_path: Path) -> None:
"""(c) The bound is this rung's to keep, not the caller's to remember. A limit past the maximum
is CLAMPED rather than refused: the caller asked for a listing, and refusing it would send a
model that asked for too much away with nothing."""
base = _synthetic(tmp_path, dirs=3, per_dir=60)
everything = _read_dir(base, "seksjon-00", limit=10_000)
assert int(everything["limit"]) == okf._DIRECTORY_PAGE_MAX < int(everything["total"])
assert len(everything["documents"]) == okf._DIRECTORY_PAGE_MAX
assert int(_read_dir(base, "seksjon-00", limit=3)["limit"]) == 3, "a small limit is honoured"
def test_an_offset_past_the_end_is_an_empty_window_over_an_honest_total(tmp_path: Path) -> None:
"""(d) ``total`` is what makes an empty answer readable: without the denominator, "nothing
here" and "you asked past the end" are the same payload."""
base = _synthetic(tmp_path, dirs=2, per_dir=3)
page = _read_dir(base, "seksjon-00", offset=999)
assert page["documents"] == [] and page["directories"] == []
assert int(page["total"]) == 3
# --- A2: the filter ------------------------------------------------------------------------------
def test_the_known_positive_filter_finds_the_fasit_concepts_and_not_the_level() -> None:
"""(e) The order's own known positive: ``filter="rundkjoring"`` on n100 must answer with the
two concepts gate-nordvik/a1 must cite, and NOT with 445 rows."""
base = _delivered("n100-2023")
fasit = json.loads(Path("contexts/gate-nordvik-2027/fasit.json").read_text(encoding="utf-8"))
wanted = {
c["path"]
for m in fasit["must_cite"]
if m["approach_id"] == "a1-rundkjoring-forenklet"
for c in m["concepts"]
}
assert wanted, "the fixture must name concepts, or this arm proves nothing"
hits = _read_dir(base, "krav/N100", filter="rundkjøring", limit=50)
assert wanted <= {str(d["name"]) for d in hits["documents"]}
assert int(hits["total_matches"]) < int(hits["total"]) / 50
assert _chars(hits) <= _CEILING_CHARS
def test_a_filter_that_matches_nothing_is_an_answer_and_not_a_refusal(tmp_path: Path) -> None:
"""(f) The order's known NEGATIVE. "No document here is about X" is a finding; refusing it
would make an honest negative indistinguishable from a path that does not exist which is the
very confusion ``BundlePathNotFound`` exists to prevent one line below."""
base = _synthetic(tmp_path, dirs=2, per_dir=3)
empty = _read_dir(base, "seksjon-00", filter="finnes-ikke-noe-sted")
assert "refused" not in empty
assert empty["documents"] == [] and int(empty["total_matches"]) == 0
assert int(empty["total"]) == 3, "the denominator survives the filter"
def test_the_filter_reads_the_reference_number_and_not_only_the_title(tmp_path: Path) -> None:
"""(g) The second field is load-bearing: on R761 the thing a navigator knows is the process
number, which is not in the title. Written over a synthetic base so it is unconditional."""
base = _synthetic(tmp_path, dirs=2, per_dir=3)
by_ref = _read_dir(base, "seksjon-01", filter="krav 1.2")
assert [str(d["name"]) for d in by_ref["documents"]] == ["seksjon-01/dok-02.md"]
assert int(by_ref["total_matches"]) == 1
def test_a_filter_narrows_directories_too(tmp_path: Path) -> None:
"""(h) R761's root is 2 728 DIRECTORIES: a filter that only narrowed documents would leave the
biggest measured level unnarrowable."""
base = _synthetic(tmp_path, dirs=12, per_dir=2)
narrowed = _read_dir(base, "", filter="seksjon-0")
assert int(narrowed["total_matches"]) == 10 < int(narrowed["total"]) == 12
assert all(str(d["path"]).startswith("seksjon-0") for d in narrowed["directories"])
# --- A3: the invented path -----------------------------------------------------------------------
def test_an_invented_path_is_refused_by_name_over_a_delivered_base() -> None:
"""(i) The measured live shape: a one-character slip in a UUID. Before, this reached the model
as ``"Error: Function failed."``; the refusal now names the path AND the level that holds
documents, which is the only thing the caller can act on."""
base = _delivered("n100-2023")
slip = "krav/N100/id-d2ebe771-5216-4d7f-92d2-95a31f2b2702.md"
answer = _read_file(base, slip)
assert answer.startswith(f"REFUSED ({okf.BundlePathNotFound.__name__})")
assert slip in answer and "krav/N100" in answer and "read_dir" in answer
# The named level resolves, and it is the one the caller was already in.
assert int(_read_dir(base, "krav/N100")["total"]) > 0
def test_all_ten_of_p16s_unresolvable_calls_now_answer_instead_of_failing() -> None:
"""(j) The nevner arm (ansikt 4). P16's four ``-debate.json`` artefacts ARE the population, and
the denominator is re-measured here rather than quoted: 32 ``read_file`` calls (the order says
24 measured 14.09, that number is the four runs' DISTINCT paths, not their calls), of which
10 named a path the base does not hold.
Every one is replayed. Each of the 10 must now come back as a NAMED refusal, and the other 22
must still return the document a gate that refused everything would pass the first half on
its own, which is the failure this file's own A2 negative arm is written against."""
artefacts = sorted(Path("scratchpad/p14-stress").glob("*/*-debate.json"))
if not artefacts:
pytest.skip("P16's stress artefacts are not present in this checkout")
calls: list[tuple[str, str]] = [
(call["bundle_id"].removeprefix("vegnormal-"), call["path"])
for artefact in artefacts
for call in json.loads(artefact.read_text(encoding="utf-8"))["tool_calls"]
if call["name"] == "read_file"
]
refused = served = 0
cache: dict[str, Any] = {}
for name, path in calls:
base = _delivered(name)
if name not in cache:
cache[name] = (_tools(base)["read_file"], _bid(base))
tool, bundle_id = cache[name]
answer = str(tool.func(bundle_id=bundle_id, path=path))
if answer.startswith("REFUSED ("):
refused += 1
assert okf.BundlePathNotFound.__name__ in answer and "read_dir" in answer
else:
served += 1
assert (refused, served) == (10, 22), f"population moved: {refused} refused, {served} served"

View file

@ -45,6 +45,7 @@ from typing import Any
import pytest
from portfolio_optimiser import okf
from portfolio_optimiser.explore import DirectoryPathRefused, navigator_tools
_EXAMPLES = Path(__file__).resolve().parents[1] / "shared" / "examples"
@ -134,14 +135,59 @@ def test_a_document_is_still_returned_whole() -> None:
assert body.strip(), "reading a real document must be untouched by the directory branch"
def test_a_nonexistent_sibling_is_still_an_os_error() -> None:
"""(5) MEASURED, REPORTED, NOT FIXED: the live call's ACTUAL shape.
def test_a_nonexistent_path_is_refused_by_name_and_the_nearest_directory_is_given() -> None:
"""(5) P18/A3 CLOSES the gap this arm used to record as open.
The model appended ``.md`` to a directory name, which is not a directory -- it is a path that
does not exist. This order fixes the directory case; this arm records that the adjacent case is
unchanged, so the gap cannot be mistaken for covered. Written as an assertion on the CURRENT
behaviour: when it goes red, someone has closed it, and that is a decision to be recorded.
It used to assert ``pytest.raises(FileNotFoundError)`` and said in its own docstring: "when it
goes red, someone has closed it, and that is a decision to be recorded." This is the record.
MEASURED over P16's four paid runs before the change: 10 of 24 ``read_file`` calls named a path
the base does not hold (8 distinct paths, one of them a single-character UUID slip), and every
one of them reached the model as MAF's opaque ``"Error: Function failed."`` while counting
toward the three consecutive tool errors that end a request.
REWRITTEN, NEVER WEAKENED. The half this file exists for is still asserted below: only an
ABSENT path is translated, and the refusal is not an ``OSError`` wearing a better message.
"""
path = _a_directory(_NESTED) + ".md"
with pytest.raises(FileNotFoundError):
_tools(_NESTED)["read_file"].func(bundle_id=_NESTED.name, path=path)
answer = _tools(_NESTED)["read_file"].func(bundle_id=_NESTED.name, path=path)
assert answer.startswith(f"REFUSED ({okf.BundlePathNotFound.__name__})")
assert path in answer, "a refusal that does not quote the path the caller sent is unactionable"
assert "read_dir" in answer, "the one thing the caller can act on is the rung that lists names"
assert issubclass(okf.BundlePathNotFound, ValueError)
assert not issubclass(okf.BundlePathNotFound, OSError)
def test_the_nearest_existing_directory_is_a_path_that_resolves() -> None:
"""(6) the anti-vacuity half of (5): a refusal that named a directory which does not exist
would be the ``_index_excerpt`` failure one rung up -- a path that never was is worse than no
path. The name the refusal hands back is fed straight into ``read_dir`` and must answer."""
deep = _a_directory(_NESTED) + "/undermappe-som-ikke-finnes/dokument.md"
answer = _tools(_NESTED)["read_file"].func(bundle_id=_NESTED.name, path=deep)
quoted = answer.rsplit("documents: ", 1)[1].split("'")[1]
listing = _tools(_NESTED)["read_dir"].func(bundle_id=_NESTED.name, path=quoted)
assert "refused" not in listing, f"the refusal named {quoted!r}, which does not resolve"
assert int(listing["total"]) > 0
def test_a_real_os_error_inside_the_base_is_not_masked() -> None:
"""(7) the NARROWNESS arm -- the half of the old tripwire that must survive the rewrite.
Only a path that is ABSENT becomes a refusal. A file that IS there and cannot be read is a
different fact: the caller's path was right and the machine failed, so translating it into
"this base has no such document" would tell the model a lie it would then act on. Driven with a
real, unreadable file rather than a mock, because the branch under test is ``Path.exists()``.
"""
document = next(
f for f in okf.navigate_bundle(str(_NESTED)).context_files if f.name.endswith(".md")
)
target = _NESTED / document.name
mode = target.stat().st_mode
target.chmod(0o000)
try:
with pytest.raises(PermissionError):
_tools(_NESTED)["read_file"].func(bundle_id=_NESTED.name, path=document.name)
finally:
target.chmod(mode)