feat(p18): a listing is a WINDOW, and an invented path is refused by name
P18 part A (order 20260914T105139Z). P16 measured S7a-3's ladder against a delivered corpus for the first time and found two things fixture bases cannot show. (1) One level is not bounded by being one level. Measured 14.09 on the four mounted vegnormal bases: okf.directory_listing on krav/N200 is 169 974 chars over 1 132 documents, krav/N100 69 250 over 445, krav/N500 39 853 over 269, and R761's own root 110 912 over 2 728 SUBDIRECTORIES -- 27-113x the 1 500-char ceiling S7a-3 set, riding in every later prompt. That last number is why the window covers BOTH kinds: a pagination over documents only would have left the largest measured level unpaginated. read_dir now answers with a window. offset/limit page directories first then documents as ONE sequence (two independent windows make "the next ten" a question with two answers); total is the denominator and is always carried; limit is CLAMPED to 50, never refused. Default 10 chosen against the ceiling: one entry is 121-209 chars (median 145) over the four bases. After: n100 1 493, n500 1 453, R761 479, n200 1 537 -- 2.5 % over, stated rather than tuned away, because the ceiling is a character budget and the window is a count. Largest single call any caller can make: ~7 600 chars. filter narrows a level instead of paging it: case-insensitive SUBSTRING over title + req_number/prosessnr and over a directory path, answering with total_matches beside total. A substring and not a pattern for _ground_against_input's reason one rung down -- a form the rule does not know returns nothing, and an empty listing reads as "the base does not have this". A filter that matches nothing is an ANSWER (total_matches: 0), never a refusal. ORDER PREMISE FELLED before building on it: the order asks for a separate top-level reader "like own_frontmatter" because parse_frontmatter was last-write-wins -- P15 (f13dc64) already made a top-level key win, so BundleFile.frontmatter IS the concept's own value and a second reader here would be the second copy ko-(p) forbids. (2) 0 of 26 fasit concepts were opened in 32 read_file calls (the order's "24" is the four runs' DISTINCT paths, re-measured 14.09), and 10 of those calls named a path the base does not hold. Each reached the model as MAF's opaque "Error: Function failed." while counting toward the three consecutive tool errors that end a request. read_file now refuses such a path by name (BundlePathNotFound, funn-99 returned form) and names the nearest directory that actually HOLDS documents -- chosen off context_files, never the filesystem, because a directory can exist on disk and hold no navigated concept (read_dir would then refuse the very path the refusal handed back) and because context_files is what drops the type: verdict layer, so a refusal can never advertise by name the one layer no listing mentions. Narrow by construction: only an ABSENT path is translated; any other OSError propagates untouched. Two existing arms REWRITTEN, neither weakened: - test_a_nonexistent_sibling_is_still_an_os_error was a tripwire whose own docstring said "when it goes red, someone has closed it, and that is a decision to be recorded". This is the record. Its narrowness half survives as a new arm driving a real PermissionError on a file that IS there. - test_every_document_is_still_reachable_and_the_counts_add_up became STRONGER: the accounting must now page, so the same assertion also proves the window is complete and non-overlapping. tests/test_navigation_window_loadbearing.py: 13 arms. Arms needing the delivered bases SKIP with the root named (PORTFOLIO_VEGNORMAL_ROOT), as MAJOR-3's ceiling arm does; the window algebra, the filter negative and the refusal run over a synthetic base UNCONDITIONALLY, so the file can never be silently absent in full. Verification: uv run pytest -q 1672 passed / 5 skipped before the new file (1670 oncfd9079). ruff check + format clean, mypy clean (38 files). Golden demo-transcript.stdout BYTE-UNCHANGED, shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f. No version bump, no push. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
cfd9079a4a
commit
9b47e5aa62
7 changed files with 634 additions and 42 deletions
|
|
@ -197,8 +197,11 @@ _INSTRUCTIONS: Final = {
|
|||
"what exists, read_bundle to open ONE of them and see its top level, read_dir to open a "
|
||||
"directory that listing named, and read_file to read a document you picked. read_bundle "
|
||||
"and read_dir return LISTINGS, never the documents — a knowledge base can hold hundreds, "
|
||||
"so descend to the part that matters instead of asking for all of it. Quote only what "
|
||||
"read_file gave you, and never guess at content you have not read."
|
||||
"so descend to the part that matters instead of asking for all of it. A listing is a "
|
||||
"WINDOW: it reports 'total' for the level and gives you 'limit' of them from 'offset', so "
|
||||
"when 'total' is large do not page through it — pass read_dir a 'filter' word and read the "
|
||||
"'total_matches' it reports. Quote only what read_file gave you, and never guess at "
|
||||
"content you have not read: a path you invent is refused, it does not find a neighbour."
|
||||
),
|
||||
HYPOTHESISER_ROLE: (
|
||||
"You shape ONE candidate cost-saving direction at a time from what the navigator found. "
|
||||
|
|
@ -954,6 +957,28 @@ def _refusal_kind(exc: Exception) -> str:
|
|||
return type(exc).__name__
|
||||
|
||||
|
||||
def _nearest_listable(bundle_dir: str, path: str, dimension: str | None) -> str:
|
||||
"""The deepest ANCESTOR of ``path`` that ``read_dir`` will actually answer for.
|
||||
|
||||
Chosen off the NAVIGATED ``context_files`` rather than off the filesystem, for two reasons that
|
||||
are the same reason: a directory can exist on disk and hold no navigated concept (nothing links
|
||||
it), in which case ``read_dir`` refuses it and the refusal would have handed the caller a path
|
||||
that does not resolve — ``_index_excerpt``'s rule one rung up, a path that never was is worse
|
||||
than no path. And ``context_files`` is the property that drops the ``type: verdict`` layer, so a
|
||||
refusal can never advertise by name the one layer no listing mentions.
|
||||
|
||||
Falls back to ``""``, the base's own top level, which ``directory_listing`` always answers.
|
||||
"""
|
||||
bundle = okf.navigate_bundle(bundle_dir)
|
||||
reachable = [f for f in bundle.context_files if okf.in_dimension(f, dimension)]
|
||||
segments = path.strip("/").split("/")
|
||||
for depth in range(len(segments) - 1, 0, -1):
|
||||
candidate = "/".join(segments[:depth])
|
||||
if any(f.name.startswith(candidate + "/") for f in reachable):
|
||||
return candidate
|
||||
return ""
|
||||
|
||||
|
||||
def _refused_mapping(exc: Exception) -> dict[str, Any]:
|
||||
"""A listing tool's refusal: a mapping with no key a successful listing has.
|
||||
|
||||
|
|
@ -1098,15 +1123,34 @@ def navigator_tools(
|
|||
"Open ONE directory inside a knowledge base, by base id and the path a previous "
|
||||
"listing gave you. Answers in the same shape as read_bundle: the directories one level "
|
||||
"further down, and the concept documents that sit in this one. An unknown path is "
|
||||
"refused rather than answered as an empty directory."
|
||||
"refused rather than answered as an empty directory. The answer is a WINDOW: 'total' "
|
||||
"is how many entries the level holds, 'offset'/'limit' say which of them you were "
|
||||
"given (limit is capped, so ask for the next page instead of a bigger one). Use "
|
||||
"'filter' to ask for the entries whose title, requirement number or path contains a "
|
||||
"word -- e.g. read_dir(bundle_id, 'krav/N100', filter='rundkjoring') answers with the "
|
||||
"6 of 445 documents about roundabouts and reports total_matches: 6. A filter that "
|
||||
"matches nothing is an answer (total_matches: 0), not a refusal."
|
||||
),
|
||||
)
|
||||
def read_dir(bundle_id: str, path: str) -> dict[str, Any]:
|
||||
def read_dir(
|
||||
bundle_id: str,
|
||||
path: str,
|
||||
filter: str | None = None,
|
||||
offset: int = 0,
|
||||
limit: int | None = None,
|
||||
) -> dict[str, Any]:
|
||||
try:
|
||||
bundle_dir = _resolve_bundle(index, bundle_id)
|
||||
bundle = okf.navigate_bundle(bundle_dir)
|
||||
okf.assert_declared_ids_agree(bundle)
|
||||
return okf.directory_listing(bundle, path, dimension=dimension)
|
||||
return okf.directory_listing(
|
||||
bundle,
|
||||
path,
|
||||
dimension=dimension,
|
||||
filter=filter,
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
)
|
||||
except _RETURNABLE_REFUSALS as exc:
|
||||
return _refused_mapping(exc)
|
||||
|
||||
|
|
@ -1135,6 +1179,23 @@ def navigator_tools(
|
|||
f"{path!r} in knowledge base {bundle_id!r} is a directory, not a document; "
|
||||
"use read_dir to list what it holds, then read_file on one of the names it gives"
|
||||
)
|
||||
# P18/A3: a path that does not exist, answered as such. MEASURED over P16's four paid runs:
|
||||
# 10 of 24 ``read_file`` calls named a path the base does not hold (8 distinct -- one is a
|
||||
# single-character UUID slip, ``4d7f`` for the real ``4e7f``), and each one left the model
|
||||
# with MAF's opaque "Error: Function failed." while counting toward the three consecutive
|
||||
# errors that end a request. The nearest EXISTING directory is named because that is the
|
||||
# one thing the caller can act on: it is the argument for the rung that lists real names.
|
||||
#
|
||||
# Narrow BY CONSTRUCTION, and that is the half this replaces rather than weakens: only a
|
||||
# path that is absent is translated. Any other ``OSError`` -- an unreadable file, a broken
|
||||
# link -- still propagates untouched, because a refusal is a statement about the CALLER's
|
||||
# path and a failure to read something that IS there is not one.
|
||||
if not resolved.exists():
|
||||
raise okf.BundlePathNotFound(
|
||||
f"knowledge base {bundle_id!r} has no document {path!r}; nearest directory that "
|
||||
f"holds documents: {_nearest_listable(bundle_dir, path, dimension)!r} — list it "
|
||||
"with read_dir (it takes a filter) and read_file one of the names it gives"
|
||||
)
|
||||
# The verdict layer, refused HOWEVER the path was found (order 20260904T172353Z). No
|
||||
# listing names it — ``context_files`` drops it at every level, so ``read_bundle`` and
|
||||
# ``read_dir`` never mention one — but a GUESSED path reached it, and reaching it that way
|
||||
|
|
|
|||
|
|
@ -1192,8 +1192,64 @@ class DocumentPathRefused(ValueError):
|
|||
"""
|
||||
|
||||
|
||||
#: P18/A1 — the DEFAULT number of concept documents one ``read_dir`` answers with.
|
||||
#:
|
||||
#: CHOSEN BY MEASUREMENT, against the ceiling S7a-3 set (1 500 characters per listing). Measured
|
||||
#: 14.09 over the four delivered vegnormal bases: one document entry is 121-209 characters (median
|
||||
#: 145), and the worst level is ``krav/N200`` with 1 132 documents at 169 974 characters. Ten
|
||||
#: entries of the WORST measured size is 2 090 characters of entries; ten of the median size is
|
||||
#: 1 450 — so ten is the largest round window that keeps a default listing of the worst measured
|
||||
#: level in the neighbourhood of the ceiling instead of two orders of magnitude above it.
|
||||
#:
|
||||
#: A default page of ten out of 1 132 is not meant to be browsed to the end: ``total`` says how many
|
||||
#: there are and ``filter`` is the rung's answer to "which ones". That is the point — before this,
|
||||
#: the price of finding out WHAT is at a level was set by how much is at it.
|
||||
_DIRECTORY_PAGE_DEFAULT: Final = 10
|
||||
|
||||
#: The largest window a CALLER may ask for. A model-chosen limit is clamped to it rather than
|
||||
#: refused: the caller asked for a listing, and answering with more than this is the cost the
|
||||
#: pagination exists to bound.
|
||||
#:
|
||||
#: 50 is measured, not round: 50 entries of the worst measured size is ~10 400 characters and of the
|
||||
#: median ~7 250 — the same order as the 6 073-character level S7a-3 already measured and accepted
|
||||
#: as the honest price of a bundle-relative path, and 16x below the 169 974 a single unbounded call
|
||||
#: cost. No single call can therefore cost O(corpus): the worst it can cost is O(window).
|
||||
_DIRECTORY_PAGE_MAX: Final = 50
|
||||
|
||||
#: The frontmatter keys ``filter`` matches against, besides the title. TOP-LEVEL keys, read straight
|
||||
#: off ``BundleFile.frontmatter``: since P15 (f13dc64) a top-level key wins over an indented one of
|
||||
#: the same name, so this IS the concept's own value and a second "own frontmatter" reader here
|
||||
#: would be the second copy of one rule that kø-(p) forbids. MEASURED on the delivered bases:
|
||||
#: ``req_number`` on the N corpora ("Krav 4.1.2—1"), ``prosessnr`` on R761 ("'11.11'", quoted —
|
||||
#: hence ``unquote_scalar``, this repo's ONE de-quoting rule).
|
||||
_FILTER_FIELDS: Final = ("req_number", "prosessnr")
|
||||
|
||||
|
||||
def _matches_filter(file: BundleFile, needle: str) -> bool:
|
||||
"""Case-insensitive SUBSTRING over the document's title and its reference number.
|
||||
|
||||
A substring and not a pattern, for ``_ground_against_input``'s reason one rung down: the caller
|
||||
is a model choosing words, and a form the rule does not know would silently return nothing —
|
||||
an empty listing that reads like "the base does not have this". Substring fails toward showing
|
||||
MORE, which a navigator can narrow; a pattern fails toward showing nothing, which it cannot.
|
||||
|
||||
``description`` is deliberately NOT searched: measured, every concept in all four bases carries
|
||||
one, and matching prose would return most of a level for most words — the filter would look like
|
||||
it worked while bounding nothing.
|
||||
"""
|
||||
hay = [unquote_scalar(file.frontmatter.get("title", file.name))]
|
||||
hay.extend(unquote_scalar(file.frontmatter.get(k, "")) for k in _FILTER_FIELDS)
|
||||
return any(needle in value.casefold() for value in hay)
|
||||
|
||||
|
||||
def directory_listing(
|
||||
bundle: Bundle, path: str = "", *, dimension: str | None = None
|
||||
bundle: Bundle,
|
||||
path: str = "",
|
||||
*,
|
||||
dimension: str | None = None,
|
||||
filter: str | None = None,
|
||||
offset: int = 0,
|
||||
limit: int | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""One LEVEL of a navigated bundle: the subdirectories under ``path`` with what each holds, and
|
||||
the concept documents that sit directly in it.
|
||||
|
|
@ -1229,11 +1285,27 @@ def directory_listing(
|
|||
chooses against, and the tool description says which of the two it is rather than leaving the
|
||||
reader to guess.
|
||||
|
||||
**P18/A1 — the answer is a WINDOW, so one listing costs O(window) and never O(level).** S7a-3
|
||||
bound the cost to entries at ONE level rather than documents in the base; P16 then measured what
|
||||
one level costs on a delivered corpus: ``krav/N200`` 169 974 characters over 1 132 documents,
|
||||
``krav/N100`` 69 250 over 445, and R761's root 110 912 over 2 728 SUBDIRECTORIES — which is why
|
||||
the window covers both kinds and not only documents. ``offset``/``limit`` page through
|
||||
directories first, then documents; ``total`` is the denominator the window is taken from.
|
||||
After: 479-1 537 characters for a default listing of each of those four levels.
|
||||
|
||||
**P18/A2 — ``filter`` is how a navigator asks "which ones", instead of paging to find out.**
|
||||
Case-insensitive substring over a document's title and reference number (``req_number`` /
|
||||
``prosessnr``) and over a directory's path. A filter that matches nothing answers with an empty
|
||||
window and ``total_matches: 0`` — never a refusal: "no document here is about X" is an answer,
|
||||
and refusing it would make an honest negative indistinguishable from a path that does not exist.
|
||||
|
||||
:raises BundlePathNotFound: no navigated concept document lives under ``path``.
|
||||
"""
|
||||
prefix = "" if path in ("", ".") else path.strip("/") + "/"
|
||||
needle = None if filter is None else filter.casefold()
|
||||
directories: dict[str, int] = {}
|
||||
documents: list[dict[str, Any]] = []
|
||||
at_level: list[BundleFile] = []
|
||||
matched_documents: list[dict[str, Any]] = []
|
||||
for f in bundle.context_files:
|
||||
if not f.name.startswith(prefix) or not in_dimension(f, dimension):
|
||||
continue
|
||||
|
|
@ -1241,20 +1313,32 @@ def directory_listing(
|
|||
head, sep, _ = rest.partition("/")
|
||||
if sep:
|
||||
directories[prefix + head] = directories.get(prefix + head, 0) + 1
|
||||
else:
|
||||
documents.append(
|
||||
{
|
||||
"name": f.name,
|
||||
# ``or "document"`` mirrors ``bundle_context``'s own fallback for a file with no
|
||||
# declared type, so the two renderings cannot disagree about it.
|
||||
"type": f.type or "document",
|
||||
"title": unquote_scalar(f.frontmatter.get("title", f.name)),
|
||||
# What the next rung COSTS, in the unit the ceiling is measured in. A navigator
|
||||
# that cannot see the price cannot choose against a budget.
|
||||
"chars": len(f.body),
|
||||
}
|
||||
)
|
||||
if prefix and not directories and not documents:
|
||||
continue
|
||||
at_level.append(f)
|
||||
if needle is not None and not _matches_filter(f, needle):
|
||||
continue
|
||||
matched_documents.append(
|
||||
{
|
||||
"name": f.name,
|
||||
# ``or "document"`` mirrors ``bundle_context``'s own fallback for a file with no
|
||||
# declared type, so the two renderings cannot disagree about it.
|
||||
"type": f.type or "document",
|
||||
"title": unquote_scalar(f.frontmatter.get("title", f.name)),
|
||||
# What the next rung COSTS, in the unit the ceiling is measured in. A navigator
|
||||
# that cannot see the price cannot choose against a budget.
|
||||
"chars": len(f.body),
|
||||
}
|
||||
)
|
||||
# A directory has no frontmatter, so its PATH is all there is to match on — and it is what a
|
||||
# navigator filtering an R761 level for "65" means. MEASURED: without this, ``read_dir`` on
|
||||
# R761's root answered with 2 728 directory entries at 110 912 characters, a bigger unbounded
|
||||
# call than the 69 250-character document level this order was written for.
|
||||
matched_directories = [
|
||||
{"path": name, "documents": count}
|
||||
for name, count in sorted(directories.items())
|
||||
if needle is None or needle in name.casefold()
|
||||
]
|
||||
if prefix and not directories and not at_level:
|
||||
# The wrong RUNG, answered as such — the direction ``explore.DirectoryPathRefused`` already
|
||||
# covers, measured absent here (F3). Built from ``context_files`` and through the SAME
|
||||
# ``in_dimension`` predicate the listing above uses: a lookup over ``files`` would name a
|
||||
|
|
@ -1277,13 +1361,37 @@ def directory_listing(
|
|||
f"knowledge base {bundle.dir!r} has no directory {path!r}; it holds no concept "
|
||||
"document under that path"
|
||||
)
|
||||
return {
|
||||
# A1: the WINDOW. Clamped, never refused — a caller asking for more than the maximum asked for
|
||||
# a listing, and the bound is this rung's job to keep, not the caller's to remember. A negative
|
||||
# or absurd offset lands past the end and answers with an empty window over an honest ``total``,
|
||||
# which is what "there is nothing here" looks like when the denominator is stated.
|
||||
#
|
||||
# ONE window over BOTH kinds, directories first, because a level is one thing to page through:
|
||||
# two independent windows would make "show me the next ten" a question with two answers.
|
||||
window = (
|
||||
_DIRECTORY_PAGE_DEFAULT if limit is None else max(0, min(int(limit), _DIRECTORY_PAGE_MAX))
|
||||
)
|
||||
start = max(0, int(offset))
|
||||
page = (matched_directories + matched_documents)[start : start + window]
|
||||
listing: dict[str, Any] = {
|
||||
"path": path,
|
||||
"directories": [
|
||||
{"path": name, "documents": count} for name, count in sorted(directories.items())
|
||||
],
|
||||
"documents": documents,
|
||||
"directories": [e for e in page if "path" in e],
|
||||
"documents": [e for e in page if "name" in e],
|
||||
# The DENOMINATOR, always: how many entries — subdirectories plus concept documents — this
|
||||
# level holds within this run's scope. Carried whether or not a filter narrowed the answer,
|
||||
# because a window without a total is a measurement without a denominator, which is the one
|
||||
# thing every other count in this repo refuses to be (ansikt 4).
|
||||
"total": len(directories) + len(at_level),
|
||||
"offset": start,
|
||||
"limit": window,
|
||||
}
|
||||
if filter is not None:
|
||||
# A SECOND fact, not a second copy of the first: ``total`` says what is here, this says how
|
||||
# many of it the filter admitted. A navigator reading only one of the two cannot tell a
|
||||
# filter that was too narrow from a level that is nearly empty.
|
||||
listing["filter"] = filter
|
||||
listing["total_matches"] = len(matched_directories) + len(matched_documents)
|
||||
return listing
|
||||
|
||||
|
||||
def bundle_context(bundle: Bundle, *, dimension: str | None = None) -> str:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue