Order 20260908T195801Z. Findings 4 and 5 from the S7 acid test, then the two things finding 99 measured and deliberately did not fix (D3, D2). No user-facing surface changes: no new flag, no new command, no changed output contract. Both seams are internal (the pre-pass rendering, and the shape a tool answers a model with), so [skip-docs] rather than a README edit that would describe nothing an operator can do differently. FINDING 4 -- MEASURED, NOTHING BUILT. K2's price schedule IS readable without guessing (8 column spans, 71 of 91 non-blank rows give >= 2 cells, the split stable for K = 2..64). But 0 of 92 rows name all three of code/quantity/unit_cost -- also under a looser substring match -- and 0 of 91 data rows carry code + quantity + amount. The triple is not formatted away; it is not in the document. It is a price SUMMARY plus nine rate cards whose unit-price columns are empty (pre-award). The order's binding decision rule therefore falls against building: --derive-cost-baseline keeps refusing, and MAJOR-4's own honesty limit holds. FINDING 5 -- BUILT. Measured on the actual rendering path (concept_text, not the raw file): the delivered excerpt is 104 lines / 67 245 chars, carrying 208 interior whitespace runs, 117 of them >= 100 and the longest 887 -- 56 806 of 67 245 characters = 84.5 %, over 72 of 104 lines. collapse_padding, called from _data_blocks (the one renderer both arms share, and therefore AFTER verify_against_bundle -- collapsing in concept_text would break every payload's own digest), gives -72.4 %: line count invariant, non-whitespace byte-identical, leading indentation untouched, no number changed. F99-D3 -- read_file / read_dir / read_bundle now RETURN their refusal. MAF turns a tool raise into "Error: Function failed." (_tools.py:1410-1432, :1427) and counts it against DEFAULT_MAX_CONSECUTIVE_ERRORS_PER_REQUEST = 3, so everything the refusing arm knows is destroyed on the way out. The gates are unchanged; the property they exist for -- the reason travels, the bytes never do -- is now asserted explicitly on the returned value. The arm is keyed on named classes, never bare Exception, because ExplorationError is itself a RuntimeError subclass. F99-D2 -- the invariant row, plus one for finding 5 (a stated deviation from "one row only": finding 5 is a separately built seam and the ledger's standing rule requires its own row). 19 existing arms rewritten, never deleted and never weakened: where the class carried a distinction, the refusal KIND carries it now. 13 mutations, all red against the whole suite (W1-W5, M1-M8), each restored from scratchpad with shasum -c. Control 1543 passed / 5 skipped (from 1529/5, a strict superset, 0 removed). Golden demo-transcript.stdout unchanged (shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f). Measurement: docs/2026-09-08-funn-4-5-og-read-nekt.md Co-Authored-By: Claude <Opus 5>
147 lines
7.8 KiB
Python
147 lines
7.8 KiB
Python
"""``read_file`` on a DIRECTORY is refused by name and pointed at ``read_dir``.
|
|
|
|
Finding (c) of the live K2 session (``docs/2026-09-06-major2-levende-k2.md`` § 3/§ 4): a live model
|
|
walking the ladder ``list_bundles -> read_bundle -> read_dir -> read_file`` reached a level holding
|
|
subdirectories (``30-1``, ``30-7``, ``521-001`` ...) and then called ``read_file`` on one of them.
|
|
Three such calls in one run.
|
|
|
|
**What was CHECKED FIRST, and what it found.** The order allows for two causes and asks which one
|
|
holds. It is the second. A listing DOES already separate the two kinds structurally: every
|
|
``directory_listing`` payload answers with ``directories`` (entries keyed ``path``, carrying a
|
|
subtree ``documents`` count) and ``documents`` (entries keyed ``name``, carrying ``type``, ``title``
|
|
and ``chars``) as two distinct keys, and no directory ever appears among the documents. Arm (4)
|
|
pins that, because it is a property this file now depends on rather than one it introduces. So no
|
|
trailing-``/`` marker is added: the shape already says which is which, and changing the payload
|
|
would move a listing three older gates measure byte for byte.
|
|
|
|
What was missing is the OTHER half. MEASURED before this work, on the shipped nested example base:
|
|
|
|
read_file(id, "a") -> builtins.IsADirectoryError: [Errno 21] Is a directory: '<abs path>'
|
|
|
|
An ``OSError``, so it lands on the crash channel rather than the CLI's refusal tuple and hosting's
|
|
400 arm, and it says nothing about which rung the caller should have used. The refusal now names
|
|
both -- that the path is a directory, and ``read_dir`` -- following ``VerdictLayerRefused`` and
|
|
``DimensionScopeRefused``: the rule lives in the TOOL, so it holds for both callers (the
|
|
exploration and, since S2c, the debate), and it is a ``ValueError`` because the caller is a model
|
|
choosing a path, not a program that is broken.
|
|
|
|
**MEASURED, REPORTED, NOT FIXED (outside this order).** The live call was
|
|
``read_file(".../30-7.md")`` -- the model appended ``.md`` to a directory NAME, which resolves to a
|
|
path that does not exist at all, not to the directory. Measured on the same base:
|
|
|
|
read_file(id, "a.md") -> builtins.FileNotFoundError: [Errno 2] No such file or directory
|
|
|
|
That is a second, adjacent defect with the same shape (an OSError on the crash channel where a
|
|
named refusal belongs), and the fix ordered here does not cover it. Arm (5) pins the measurement so
|
|
the gap is a fact in the suite rather than a sentence in a report.
|
|
|
|
Detach point: remove the directory branch from ``read_file`` -> RED on (1), (2) and (3).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
import pytest
|
|
|
|
from portfolio_optimiser.explore import DirectoryPathRefused, navigator_tools
|
|
|
|
_EXAMPLES = Path(__file__).resolve().parents[1] / "shared" / "examples"
|
|
#: The only NESTED base shipped -- the one that HAS a directory to ask for.
|
|
_NESTED = _EXAMPLES / "nav-golden-hierarchy" / "bundle"
|
|
|
|
|
|
def _tools(bundle_dir: Path) -> dict[str, Any]:
|
|
return {t.name: t for t in navigator_tools((str(bundle_dir),))}
|
|
|
|
|
|
async def _invoke(tool: Any, **arguments: Any) -> str:
|
|
"""A tool's answer as text (S2c's helper: ``invoke`` returns ``[Content]``, and a test that
|
|
stringified the list would compare object reprs and pass against anything)."""
|
|
return "".join(getattr(c, "text", "") or "" for c in await tool.invoke(arguments=arguments))
|
|
|
|
|
|
def _a_directory(bundle_dir: Path) -> str:
|
|
"""A directory path taken out of the listing itself -- the same way a model gets one."""
|
|
listing = _tools(bundle_dir)["read_bundle"].func(bundle_id=bundle_dir.name)
|
|
assert listing["directories"], "fixture has no subdirectory to ask for"
|
|
return str(listing["directories"][0]["path"])
|
|
|
|
|
|
def test_a_directory_path_is_refused_and_the_other_rung_is_named() -> None:
|
|
"""(1) the refusal says WHAT the path is and WHICH tool reads it."""
|
|
path = _a_directory(_NESTED)
|
|
message = _tools(_NESTED)["read_file"].func(bundle_id=_NESTED.name, path=path)
|
|
assert message.startswith(f"REFUSED ({DirectoryPathRefused.__name__})"), (
|
|
"the refusal is not distinguishable from a document's body"
|
|
)
|
|
assert path in message
|
|
assert "directory" in message
|
|
assert "read_dir" in message, "a refusal that does not name the other rung strands the caller"
|
|
|
|
|
|
def test_the_refusal_is_a_value_error_not_an_os_error() -> None:
|
|
"""(2) the channel, which is the half a message alone cannot carry.
|
|
|
|
``ValueError`` is the ``BundlePathNotFound``/``DimensionScopeRefused`` precedent -- the CLI's
|
|
refusal tuple and hosting's 400 arm. The ``OSError`` assert is what tells a REPLACEMENT from a
|
|
wrapper: re-raising ``IsADirectoryError`` with a better message would pass arm (1).
|
|
"""
|
|
path = _a_directory(_NESTED)
|
|
answer = _tools(_NESTED)["read_file"].func(bundle_id=_NESTED.name, path=path)
|
|
# F99-D3: the refusal is RETURNED now, so the branch is identified by the kind it names. The
|
|
# channel assertion moves onto the class itself and keeps both halves: it is on the CLI's
|
|
# refusal tuple, and it is NOT a re-raised ``IsADirectoryError`` wearing a better message --
|
|
# an ``OSError`` is outside ``_RETURNABLE_REFUSALS`` and would propagate rather than answer.
|
|
assert answer.startswith(f"REFUSED ({DirectoryPathRefused.__name__})")
|
|
assert issubclass(DirectoryPathRefused, ValueError)
|
|
assert not issubclass(DirectoryPathRefused, OSError)
|
|
|
|
|
|
async def test_the_refusal_holds_through_the_real_tool_surface() -> None:
|
|
"""(3) reached through ``invoke``, not only through ``.func``.
|
|
|
|
The seam a model actually calls is the tool, and a check that only ran on the plain function
|
|
would be absent from the one caller it was written for.
|
|
"""
|
|
path = _a_directory(_NESTED)
|
|
answer = await _invoke(_tools(_NESTED)["read_file"], bundle_id=_NESTED.name, path=path)
|
|
assert answer.startswith(f"REFUSED ({DirectoryPathRefused.__name__})")
|
|
assert "read_dir" in answer
|
|
|
|
|
|
def test_a_listing_already_separates_directories_from_documents() -> None:
|
|
"""(4) the "check first" arm: the payload's shape is what distinguishes the two kinds.
|
|
|
|
Also the anti-vacuity control for this file -- it proves the fixture really does hold both
|
|
kinds, so arm (1) is asking for a directory rather than for nothing.
|
|
"""
|
|
listing = _tools(_NESTED)["read_bundle"].func(bundle_id=_NESTED.name)
|
|
directories = {d["path"] for d in listing["directories"]}
|
|
documents = {d["name"] for d in listing["documents"]}
|
|
assert directories and documents, "fixture must hold both kinds for this to mean anything"
|
|
assert directories.isdisjoint(documents)
|
|
assert all(set(d) == {"path", "documents"} for d in listing["directories"])
|
|
assert all(set(d) == {"name", "type", "title", "chars"} for d in listing["documents"])
|
|
|
|
|
|
def test_a_document_is_still_returned_whole() -> None:
|
|
"""CONTROL: the branch is a gate, not a wall -- the rung it guards still works."""
|
|
listing = _tools(_NESTED)["read_bundle"].func(bundle_id=_NESTED.name)
|
|
name = listing["documents"][0]["name"]
|
|
body = _tools(_NESTED)["read_file"].func(bundle_id=_NESTED.name, path=name)
|
|
assert body.strip(), "reading a real document must be untouched by the directory branch"
|
|
|
|
|
|
def test_a_nonexistent_sibling_is_still_an_os_error() -> None:
|
|
"""(5) MEASURED, REPORTED, NOT FIXED: the live call's ACTUAL shape.
|
|
|
|
The model appended ``.md`` to a directory name, which is not a directory -- it is a path that
|
|
does not exist. This order fixes the directory case; this arm records that the adjacent case is
|
|
unchanged, so the gap cannot be mistaken for covered. Written as an assertion on the CURRENT
|
|
behaviour: when it goes red, someone has closed it, and that is a decision to be recorded.
|
|
"""
|
|
path = _a_directory(_NESTED) + ".md"
|
|
with pytest.raises(FileNotFoundError):
|
|
_tools(_NESTED)["read_file"].func(bundle_id=_NESTED.name, path=path)
|