open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 4 Activity

feat(explore): stigen faar sitt manglende trinn - read_bundle gir ETT nivaa, read_dir det neste

Browse source 
S7a-3 pkt. 2. MAJOR-3 bygde read_bundle om fra HELE basen til en oppfoering per
konseptfil. Saa kom det foerste ekte korpuset: K2 navigerer til 629 konsepter bak
478 nestede indekser, og en listing av 629 koster 42 761 o200k-tokens som rir i
7 av 12 prompter = 89 % av alle prompt-tokens. Bindingen holdt asymptotisk og
priset likevel hele korpuset. De 478 indeksene ble bygget, konsumert og flatet ut
- agenten saa 629 soesken og fikk aldri vite at korpuset hadde en form.

MAALT (BEFORE og AFTER i samme oekt, samme kode, BEFORE som mutasjon):
  read_bundle-nyttelast  110 581 tegn / 42 761 tok  ->  3 954 tegn / 1 495 tok
  listing-tokens totalt         307 573 (89 %)      ->  12 595 (26 %)
  prompt-tokens i kjoeringen         343 826        ->  49 225   (-86 %)
BEFORE reproduserer S7a-2s publiserte tall til 0,03 % - kjent-positiv kontroll
paa instrumentet, som ogsaa maatte rettes (resultatet baerer name=None, saa en
sonde nøklet paa verktoeynavn rapporterer 0 kopier og leses som en ekte null).

- okf.directory_listing er ENESTE renderer; begge verktoey ER den paa hvert sitt
  nivaa. Kataloger utledes av STIER, aldri av index.md. Bygget av context_files,
  ALDRI files. Hver sti er bundle-relativ, brukbar ordrett i neste kall.
- Ukjent sti NEKTES ved navn (BundlePathNotFound) - en tom listing er umulig aa
  skille fra en katalog som finnes og er tom.
- Verktoeybeskrivelsene og navigatoerinstruksjonen flyttet i SAMME commit.

PREMISS FELT FOER BYGGING: context_files har aldri holdt hierarkiet tilbake -
navnene er fulle bundle-relative stier; det var RENDERINGEN som flatet det ut.
Derfor er bundle_context og begge nav-goldenene byte-identiske, gratis.

AVVIK fra ordren, uttalt: K2 kan ikke vaere testavhengighet (utenfor repoet), og
1 500 tegn er ikke oppnaaelig for en rot med 39 identifiserbare oppfoeringer
(maalt 3 954). Gaten binder 1 500 tegn per listing over basene den KAN se, pluss
egenskapen, med en FLAT kontroll over 5x taket.

Load-bearing MAALT: 9 mutasjoner alle roede mot HELE suiten, groenn kontroll
1252 passed / 5 skipped, golden byte-uendret. N1 4 / N2 7 / N3 12 / N4 5 / N5 1 /
N6 6 / N7 1 / N8 2 / N9 1.

Maaling: docs/2026-09-03-hierarkisk-navigasjon-k2.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-03 07:38:30 +02:00
commit baa6f450e8
8 changed files with 584 additions and 40 deletions
Download patch file Download diff file Expand all files Collapse all files

60
src/portfolio_optimiser/explore.py
View file

@ -193,10 +193,12 @@ HYPOTHESIS_MARKER: Final = "HYPOTHESIS:"
_INSTRUCTIONS: Final = {
NAVIGATOR_ROLE: (
"You read the project's knowledge bases. Use list_bundles to see what exists, "
"read_bundle to list the documents inside ONE of them, and read_file to read a document "
"you picked from that list. read_bundle returns a LISTING, never the documents — quote "
"only what read_file gave you, and never guess at content you have not read."
"You read the project's knowledge bases, one level at a time. Use list_bundles to see "
"what exists, read_bundle to open ONE of them and see its top level, read_dir to open a "
"directory that listing named, and read_file to read a document you picked. read_bundle "
"and read_dir return LISTINGS, never the documents — a knowledge base can hold hundreds, "
"so descend to the part that matters instead of asking for all of it. Quote only what "
"read_file gave you, and never guess at content you have not read."
),
HYPOTHESISER_ROLE: (
"You shape ONE candidate cost-saving direction at a time from what the navigator found. "
@ -893,12 +895,14 @@ def navigator_tools(bundle_dirs: Sequence[str]) -> list[FunctionTool]:
@tool(
name="read_bundle",
description=(
"Open ONE knowledge base by id and list what it holds: one entry per concept "
"document with its name, declared type, title and size in characters. This is a "
"LISTING, not the documents themselves — read_file(id, name) returns one whole."
"Open ONE knowledge base by id and list its TOP LEVEL: the directories it is divided "
"into (each with how many concept documents its whole subtree holds) and the documents "
"that sit directly at the top, each with name, declared type, title and size in "
"characters. Paths are usable as they are — read_dir(id, path) opens a directory, "
"read_file(id, name) returns one document whole."
),
)
def read_bundle(bundle_id: str) -> list[dict[str, Any]]:
def read_bundle(bundle_id: str) -> dict[str, Any]:
bundle_dir = _resolve_bundle(index, bundle_id)
bundle = okf.navigate_bundle(bundle_dir)
# The base is OPENED here, so this is where it must be able to say what it IS (S7a-3
@ -906,25 +910,25 @@ def navigator_tools(bundle_dirs: Sequence[str]) -> list[FunctionTool]:
# filesystem accident, recorded by the run rather than blocked here — but two concepts
# declaring two different corpora is a base no fallback can settle.
okf.assert_declared_ids_agree(bundle)
# ``context_files``, NEVER ``files``: it is the property that drops the ``type: verdict``
# layer AND nested ``index.md`` at every level. Building from ``files`` would put prior
# verdicts in front of the navigator around the gated ExpeL fold (målbilde §4), and the
# cost arms of the gate would all stay green while it happened.
return [
{
"name": f.name,
# ``or "document"`` mirrors ``okf.bundle_context``'s own fallback for a file with
# no declared type, so the two renderings of one bundle cannot disagree about it.
"type": f.type or "document",
# ``okf`` owns the unquoting rule; a hand-rolled strip here would be a second copy
# of it, and a second copy is the one that drifts (kø-(p)).
"title": okf.unquote_scalar(f.frontmatter.get("title", f.name)),
# What the next rung COSTS, in the unit the ceiling is measured in. A navigator
# that cannot see the price cannot choose against a budget.
"chars": len(f.body),
}
for f in bundle.context_files
]
# ONE renderer for both rungs (kø-(p)): this tool and ``read_dir`` differ only in WHICH
# level they ask for, and two copies of a listing rule would drift into two answers about
# one bundle. ``okf`` owns it, so the context seam stays framework-neutral.
return okf.directory_listing(bundle)
@tool(
name="read_dir",
description=(
"Open ONE directory inside a knowledge base, by base id and the path a previous "
"listing gave you. Answers in the same shape as read_bundle: the directories one level "
"further down, and the concept documents that sit in this one. An unknown path is "
"refused rather than answered as an empty directory."
),
)
def read_dir(bundle_id: str, path: str) -> dict[str, Any]:
bundle_dir = _resolve_bundle(index, bundle_id)
bundle = okf.navigate_bundle(bundle_dir)
okf.assert_declared_ids_agree(bundle)
return okf.directory_listing(bundle, path)
@tool(
name="read_file",
@ -937,7 +941,7 @@ def navigator_tools(bundle_dirs: Sequence[str]) -> list[FunctionTool]:
# navigation walk uses rather than a second, laxer check.
return Path(safe_resolve(bundle_dir, path)).read_text(encoding="utf-8")
return [list_bundles, read_bundle, read_file]
return [list_bundles, read_bundle, read_dir, read_file]
def quick_validate_tool(

78
src/portfolio_optimiser/okf.py
View file

@ -1015,6 +1015,84 @@ def assert_declared_ids_agree(bundle: Bundle) -> None:
)
class BundlePathNotFound(ValueError):
"""A listing was asked for a directory the navigated bundle does not have.
A ``ValueError``, the ``BundleIdMismatch`` precedent: it must land on the CLI's refusal tuple
and hosting's 400 arm rather than the crash channel. Refusing is the point — an unknown path
rendered as an empty listing is indistinguishable from a directory that exists and holds
nothing, and the caller is a model choosing a path out of a previous listing.
"""
def directory_listing(bundle: Bundle, path: str = "") -> dict[str, Any]:
"""One LEVEL of a navigated bundle: the subdirectories under ``path`` with what each holds, and
the concept documents that sit directly in it.
The rung between ``list_bundles`` (which bases exist) and ``read_file`` (what one document
says). Before it existed, ``read_bundle`` answered with EVERY concept document in the base —
measured on K2 at 42 761 o200k tokens riding in 7 of 12 prompts, 90 % of the run — because the
price of finding out what a base contains was set by how much it contains.
**The tree is derived from ``context_files`` NAMES, never from ``index.md`` files.** Every name
is already the full bundle-relative posix path, so the shape was never lost — only the
rendering flattened it, which is why ``bundle_context`` and both nav-goldens are byte-identical
after this was added. Deriving from names also means a directory whose index was never linked is
still visible, and that the two renderings of one bundle cannot disagree about what a concept is.
``context_files``, NEVER ``files``: it is the property that drops the ``type: verdict`` layer
AND nested ``index.md`` at every level. A listing built from ``files`` would put prior verdicts
in front of the navigator around the gated ExpeL fold (målbilde §4) — and a directory COUNT
built from ``files`` would advertise documents the navigator is not allowed to be shown.
**Every path is BUNDLE-RELATIVE, i.e. usable verbatim as the next call's argument.** A level-
relative name would have to be composed by the caller, and the caller is a model: a path that
never existed is worse than no path (``_index_excerpt``'s rule, one rung up).
``documents`` on a directory entry is the count of concept documents in its whole SUBTREE — what
the subtree holds, not what one ``read_dir`` on it returns. It is the price signal a navigator
chooses against, and the tool description says which of the two it is rather than leaving the
reader to guess.
:raises BundlePathNotFound: no navigated concept document lives under ``path``.
"""
prefix = "" if path in ("", ".") else path.strip("/") + "/"
directories: dict[str, int] = {}
documents: list[dict[str, Any]] = []
for f in bundle.context_files:
if not f.name.startswith(prefix):
continue
rest = f.name[len(prefix) :]
head, sep, _ = rest.partition("/")
if sep:
directories[prefix + head] = directories.get(prefix + head, 0) + 1
else:
documents.append(
{
"name": f.name,
# ``or "document"`` mirrors ``bundle_context``'s own fallback for a file with no
# declared type, so the two renderings cannot disagree about it.
"type": f.type or "document",
"title": unquote_scalar(f.frontmatter.get("title", f.name)),
# What the next rung COSTS, in the unit the ceiling is measured in. A navigator
# that cannot see the price cannot choose against a budget.
"chars": len(f.body),
}
)
if prefix and not directories and not documents:
raise BundlePathNotFound(
f"knowledge base {bundle.dir!r} has no directory {path!r}; it holds no concept "
"document under that path"
)
return {
"path": path,
"directories": [
{"path": name, "documents": count} for name, count in sorted(directories.items())
],
"documents": documents,
}
def bundle_context(bundle: Bundle, *, dimension: str | None = None) -> str:
"""Render a navigated bundle as agent read-context via progressive disclosure: the ``index.md``
summary, then each concept file as ``## {type}: {title}\\n{body}``. ``type: verdict`` files are