feat(prepass,explore): padding dies at the prompt; a refusal the model can act on is a return value [skip-docs]

Order 20260908T195801Z. Findings 4 and 5 from the S7 acid test, then the two things
finding 99 measured and deliberately did not fix (D3, D2).

No user-facing surface changes: no new flag, no new command, no changed output
contract. Both seams are internal (the pre-pass rendering, and the shape a tool
answers a model with), so [skip-docs] rather than a README edit that would describe
nothing an operator can do differently.

FINDING 4 -- MEASURED, NOTHING BUILT. K2's price schedule IS readable without
guessing (8 column spans, 71 of 91 non-blank rows give >= 2 cells, the split stable
for K = 2..64). But 0 of 92 rows name all three of code/quantity/unit_cost -- also
under a looser substring match -- and 0 of 91 data rows carry code + quantity +
amount. The triple is not formatted away; it is not in the document. It is a price
SUMMARY plus nine rate cards whose unit-price columns are empty (pre-award). The
order's binding decision rule therefore falls against building:
--derive-cost-baseline keeps refusing, and MAJOR-4's own honesty limit holds.

FINDING 5 -- BUILT. Measured on the actual rendering path (concept_text, not the
raw file): the delivered excerpt is 104 lines / 67 245 chars, carrying 208 interior
whitespace runs, 117 of them >= 100 and the longest 887 -- 56 806 of 67 245
characters = 84.5 %, over 72 of 104 lines. collapse_padding, called from
_data_blocks (the one renderer both arms share, and therefore AFTER
verify_against_bundle -- collapsing in concept_text would break every payload's own
digest), gives -72.4 %: line count invariant, non-whitespace byte-identical, leading
indentation untouched, no number changed.

F99-D3 -- read_file / read_dir / read_bundle now RETURN their refusal. MAF turns a
tool raise into "Error: Function failed." (_tools.py:1410-1432, :1427) and counts it
against DEFAULT_MAX_CONSECUTIVE_ERRORS_PER_REQUEST = 3, so everything the refusing
arm knows is destroyed on the way out. The gates are unchanged; the property they
exist for -- the reason travels, the bytes never do -- is now asserted explicitly on
the returned value. The arm is keyed on named classes, never bare Exception, because
ExplorationError is itself a RuntimeError subclass.

F99-D2 -- the invariant row, plus one for finding 5 (a stated deviation from "one
row only": finding 5 is a separately built seam and the ledger's standing rule
requires its own row).

19 existing arms rewritten, never deleted and never weakened: where the class
carried a distinction, the refusal KIND carries it now.

13 mutations, all red against the whole suite (W1-W5, M1-M8), each restored from
scratchpad with shasum -c. Control 1543 passed / 5 skipped (from 1529/5, a strict
superset, 0 removed). Golden demo-transcript.stdout unchanged
(shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f).

Measurement: docs/2026-09-08-funn-4-5-og-read-nekt.md

Co-Authored-By: Claude <Opus 5>

src/portfolio_optimiser/explore.py

@@ -927,6 +927,58 @@ def _index_excerpt(body: str) -> tuple[str, bool]:
     return (body[:cut] if cut > 0 else body[:_CATALOGUE_EXCERPT_CHARS]), True
 
 
+#: The refusal classes the navigator's read tools turn into an ANSWER instead of an exception,
+#: named ONE BY ONE and never as a base class. ``ExplorationError`` is itself a ``RuntimeError``
+#: subclass, so an arm written as ``except RuntimeError`` would also swallow a failure nobody
+#: named -- turning an unknown fault into a confident-looking answer, which is worse than the
+#: opaque string this conversion removes. MEASURED: inside these three tool bodies the only
+#: reachable ``ExplorationError`` is ``_resolve_bundle``'s unknown-base refusal.
+_RETURNABLE_REFUSALS: Final = (
+    ExplorationError,
+    okf.BundleIdMismatch,
+    okf.BundlePathNotFound,
+    okf.DocumentPathRefused,
+    DirectoryPathRefused,
+    VerdictLayerRefused,
+    DimensionScopeRefused,
+)
+
+
+def _refusal_kind(exc: Exception) -> str:
+    """The refusal's KIND, from the exception's own class name -- one source, never a second table.
+
+    F3 measured that ``read_dir`` must tell "this is a document, use the other rung" apart from
+    "this does not exist", and before this change a caller could tell them apart by CLASS. A
+    returned refusal would lose that distinction unless it carries it, so it does.
+    """
+    return type(exc).__name__
+
+
+def _refused_mapping(exc: Exception) -> dict[str, Any]:
+    """A listing tool's refusal: a mapping with no key a successful listing has.
+
+    ``directory_listing`` answers with ``path`` / ``directories`` / ``documents``, so ``refused``
+    and ``refusal`` cannot be mistaken for a result -- the property that makes a returned refusal
+    worth anything at all.
+    """
+    return {"refused": str(exc), "refusal": _refusal_kind(exc)}
+
+
+def _refused_text(exc: Exception) -> str:
+    """``read_file``'s refusal, as text, because that tool answers with text.
+
+    **Why not a mapping here.** Making ``read_file`` return a dict would change the shape of every
+    SUCCESSFUL read, and therefore every prompt byte S2c measured when it gave the debate these
+    tools. The refusal is a string with a fixed leading sentinel instead.
+
+    **Honesty limit, stated.** A document whose first characters were exactly this sentinel would
+    be indistinguishable from a refusal. That is a real gap and it is recorded rather than papered
+    over; what IS guaranteed is the direction that matters -- a refusal never carries the bytes of
+    the document it refused, which is the property the verdict and dimension gates exist for.
+    """
+    return f"REFUSED ({_refusal_kind(exc)}): {exc}"
+
+
 def navigator_tools(
     bundle_dirs: Sequence[str], *, dimension: str | None = None
 ) -> list[FunctionTool]:
@@ -1022,6 +1074,12 @@ def navigator_tools(
         ),
     )
     def read_bundle(bundle_id: str) -> dict[str, Any]:
+        try:
+            return _read_bundle(bundle_id)
+        except _RETURNABLE_REFUSALS as exc:
+            return _refused_mapping(exc)
+
+    def _read_bundle(bundle_id: str) -> dict[str, Any]:
         bundle_dir = _resolve_bundle(index, bundle_id)
         bundle = okf.navigate_bundle(bundle_dir)
         # The base is OPENED here, so this is where it must be able to say what it IS (S7a-3
@@ -1044,16 +1102,25 @@ def navigator_tools(
         ),
     )
     def read_dir(bundle_id: str, path: str) -> dict[str, Any]:
-        bundle_dir = _resolve_bundle(index, bundle_id)
-        bundle = okf.navigate_bundle(bundle_dir)
-        okf.assert_declared_ids_agree(bundle)
-        return okf.directory_listing(bundle, path, dimension=dimension)
+        try:
+            bundle_dir = _resolve_bundle(index, bundle_id)
+            bundle = okf.navigate_bundle(bundle_dir)
+            okf.assert_declared_ids_agree(bundle)
+            return okf.directory_listing(bundle, path, dimension=dimension)
+        except _RETURNABLE_REFUSALS as exc:
+            return _refused_mapping(exc)
 
     @tool(
         name="read_file",
         description="Read ONE document inside a knowledge base, by base id and relative path.",
     )
     def read_file(bundle_id: str, path: str) -> str:
+        try:
+            return _read_file(bundle_id, path)
+        except _RETURNABLE_REFUSALS as exc:
+            return _refused_text(exc)
+
+    def _read_file(bundle_id: str, path: str) -> str:
         bundle_dir = _resolve_bundle(index, bundle_id)
         # safe_resolve is the ONE in-/out-of-bundle test in this repo, and it is fail-closed. A
         # model-chosen path is untrusted input by definition, so it goes through the same gate the

src/portfolio_optimiser/prepass.py

@@ -31,10 +31,12 @@ from __future__ import annotations
 
 import hashlib
 import json
+import re
 import unicodedata
 from collections.abc import Mapping
 from dataclasses import dataclass
 from pathlib import Path
+from typing import Final
 
 from pydantic import BaseModel, ConfigDict
 
@@ -574,14 +576,61 @@ def _excerpt_header(excerpt: PrepassExcerpt) -> str:
     return f"--- BEGIN DATA {excerpt.concept_id} ({', '.join(fields)}) ---"
 
 
+#: Two or more. There is deliberately no threshold to tune: "a run of horizontal whitespace inside
+#: a line is a column gap" is the whole rule, so no K can drift and no K has to be defended.
+_PADDING_RUN_RE: Final = re.compile(r"(?<=\S)[ \t]{2,}")
+
+
+def collapse_padding(text: str) -> str:
+    r"""Drop a spreadsheet render's column PADDING on the way into the prompt, and nothing else.
+
+    **Measured (finding 5, order 20260908T195801Z)** on K2's price schedule as
+    ``concept_text`` delivers it -- 104 lines / 67 245 characters: 208 interior whitespace runs of
+    two or more, **117 of them >= 100 characters and the longest 887**, together **56 806 of
+    67 245 characters = 84.5 %** of the excerpt, over 72 of 104 lines. So a label and its amount
+    reached the model hundreds of characters apart (``82`` / 593 spaces / a description / 250
+    spaces / ``5647500.0``), and most of what the run paid for was a column width pandoc chose.
+
+    **No judgement anywhere**, which is four separate promises and each is gated:
+
+    * interior runs of spaces and tabs collapse to ONE space;
+    * **LEADING whitespace is untouched** -- indentation is markdown structure (nested lists,
+      indented code), not padding, and flattening it would rewrite documents rather than unpad
+      them. That is what the ``(?<=\S)`` lookbehind buys;
+    * **no newline is touched**, so the line count is invariant and no row is merged with its
+      neighbour -- a naive ``re.sub(r"\s+", " ", text)`` turns a table into a paragraph;
+    * **no non-whitespace character is touched**, so no number can change. The gate asserts the
+      per-line sequence of non-whitespace characters is identical, not merely that the digits are
+      still somewhere.
+
+    **It runs in the RENDERING, never in :func:`concept_text`.** That function is the local
+    re-derivation ``verify_against_bundle`` binds a payload's bytes to; collapsing there would make
+    every payload ever written fail its own digest and would defeat the gate that stops a payload
+    delivering bytes the base does not hold. This is a display transform applied AFTER that gate.
+
+    **Honesty limit, stated.** Collapsing to a single space loses the CELL BOUNDARY: ``Post SUM``
+    becomes indistinguishable from prose containing those two words. Inventing a delimiter to carry
+    the boundary would be exactly the judgement this rule avoids, and no measurement says which
+    delimiter a model reads better -- so the loss is recorded rather than papered over. That a model
+    then USES the numbers is likewise not shown: it would take a paid run, which is a separate
+    order.
+    """
+    return "\n".join(_PADDING_RUN_RE.sub(" ", line) for line in text.split("\n"))
+
+
 def _data_blocks(payload: PrepassPayload) -> list[str]:
-    """The delimited DATA blocks, one per delivered excerpt (SS 9.3), shared by both arms."""
+    """The delimited DATA blocks, one per delivered excerpt (SS 9.3), shared by both arms.
+
+    :func:`collapse_padding` runs HERE, in the one renderer both arms share (kø-(p)): a fix in
+    ``render_context`` alone would leave the exploration reading padding, and two copies of a
+    display rule drift into two answers about one excerpt.
+    """
     lines: list[str] = []
     for excerpt in payload.excerpts:
         lines += [
             "",
             _excerpt_header(excerpt),
-            excerpt.text,
+            collapse_padding(excerpt.text),
             f"--- END DATA {excerpt.concept_id} ---",
         ]
     return lines