"""``read_bundle`` costs O(DOCUMENTS in one base), never O(bytes of that base). MAJOR-3 (``docs/2026-09-02-misjonsreview-v2.md`` § 7), measured before anything was changed in ``docs/2026-09-02-read-bundle-kontekstkostnad.md``: read_bundle payload bygg 3 861 / veglys 10 406 / tunnel 12 595 o200k_base tokens prompts one result rides in FIVE (navigator 1, manager 3, hypothesiser 1) share of every prompt-token 54 % / 59 % / 59 % of one CLI ``--explore`` run ``read_bundle`` returned ``okf.bundle_context`` — the WHOLE navigated base. One call turned the base into a ``function_result``, and because the exploration's participants share one conversation history that result rides in **every later prompt**, at full price, without anyone asking for it again. That is the same shape ``list_bundles`` was measured in and rebuilt out of in session 65 (``tests/test_catalogue_cost_loadbearing.py``), one rung down the ladder: the catalogue answers *which bases exist*, ``read_bundle`` answers *what is in this one*, and ``read_file`` answers *what does this document say*. Only the last of the three should cost what a document costs. So ``read_bundle`` now returns the CATALOGUE FORM: one entry per concept document — ``name``, ``type``, ``title``, ``chars`` — and ``read_file(id, name)`` is the next rung. This is a disclosure level, not data loss: every byte is still exactly one call away, and a navigator now pays for the documents it chose to open instead of for the ones it did not. **A premise felled before it was built on** (see the measurement doc § 2): "the index body is the base's own navigation prose, so it belongs here". The tunnel base's root index is **4 763 chars alone ≈ 1 400 tokens** — nearly the entire ceiling, for a field ``list_bundles`` already excerpts and ``read_file(id, "index.md")`` still returns whole. **The ceiling lives in THIS FILE, not in ``explore.py``** — the catalogue gate's rule, for the catalogue gate's reason: a test that imported the implementation's budget would move with it, and raising the budget is precisely the regression this file exists to catch. **DEVIATION from the order, stated:** the order words the gate as "< 1 500 o200k-tokens". This file bounds CHARACTERS instead. ``tiktoken`` is not a project dependency (and adding one for a gate would be a bigger decision than the gate), and a gate that skips when an optional package is missing is a gate that can be silently absent. The character ceiling is a proxy whose conversion was MEASURED rather than assumed: the new payload over the real tunnel base is **748 chars / 259 o200k tokens** (2.89 chars/token for this Norwegian markdown), so 1 500 characters is ≈ 520 tokens — comfortably inside the order's criterion, and twice the measured payload, so ordinary field growth does not force a rewrite. The order's own criterion is verified directly, once, by the instrument in the measurement doc. **AMENDED 2026-09-03 (S7a-3 pkt. 2), and the amendment is one level, not a rewrite.** The listing this file bounds is now the base's TOP LEVEL rather than every document in it: on K2 the flat form still cost 42 761 o200k tokens across 629 documents, because O(documents in the base) is only cheap while the base is small. ``read_dir`` is the rung that was missing. Every arm below still holds over the flat example bases -- a flat base has no subdirectories, so its document list is unchanged, byte for byte -- and the hierarchy's own gate is ``tests/test_hierarchical_navigation_loadbearing.py``. ``_documents()`` exists because ``len()`` of the payload is now the number of KEYS. What the arms pin, and what each one refuses: (a) the bound itself, over the REAL base the order names — refuses the unbounded form. A synthetic fixture here would measure the fixture writer, not the base; (b) cost tracks DOCUMENT COUNT, not document SIZE — ten times the prose, the same price. That is the property stated directly rather than inferred from (a); (c) the anti-vacuity arm: every concept document is still IDENTIFIED, with all four fields, and the entry count equals the navigated context files. Without this, "return an empty list" passes (a) perfectly and hands the navigator nothing to choose between — the repo's vacuous-gate class, which has now bitten twelve times; (d) the ``type: verdict`` layer stays EXCLUDED. ``read_bundle`` built from ``bundle.files`` instead of ``bundle.context_files`` would route prior verdicts into a hypothesis prompt around the gated ExpeL fold (målbilde §4), and nothing else in the suite would notice; (e) the ladder is intact — ``read_file`` still returns the COMPLETE document, so the bound is a disclosure level, not data loss; (f) the CONTROL the order names: ONE concept document alone blows the ceiling for the whole listing. A green (a) then means the bound fired, not that the base was small (Verifiseringsloven face 4: a gate that can only pass proves nothing). """ from __future__ import annotations import json from pathlib import Path from typing import Any from portfolio_optimiser import okf from portfolio_optimiser.explore import navigator_tools _EXAMPLES = Path(__file__).resolve().parents[1] / "shared" / "examples" #: The base the order names. Measured, not chosen: the largest of the three example bases. _TUNNEL = _EXAMPLES / "tunnel-hauglia" #: Characters one ``read_bundle`` listing may cost. Test-owned on purpose — see the docstring, and #: the measured chars/token conversion that ties it to the order's token criterion. _CEILING_CHARS = 1_500 def _tools(bundle_dir: Path) -> dict[str, Any]: return {t.name: t for t in navigator_tools((str(bundle_dir),))} def _listing(bundle_dir: Path) -> dict[str, Any]: """The WHOLE payload one ``read_bundle`` costs — since S7a-3 that is one LEVEL of the base (subdirectories plus the documents at this one), not every document in it.""" return _tools(bundle_dir)["read_bundle"].func(bundle_id=bundle_dir.name) def _documents(bundle_dir: Path) -> list[dict[str, Any]]: """The document half alone. A separate helper because ``len()`` of the payload is the number of KEYS — three, always — and an arm counting entries through it would be vacuously green.""" return list(_listing(bundle_dir)["documents"]) def _blob(payload: object) -> str: return json.dumps(payload, ensure_ascii=False) def _write_base( root: Path, name: str, *, body_chars: int, concepts: int = 3, verdicts: int = 0 ) -> Path: base = root / name base.mkdir(parents=True) lines = [f"# {name}", ""] for i in range(concepts): lines.append(f"- [konsept-{i}](konsept-{i}.md)") (base / f"konsept-{i}.md").write_text( f"---\ntype: concept\ntitle: Konsept {i}\n---\n\n" + ("innhold. " * (body_chars // 9)), encoding="utf-8", ) for i in range(verdicts): lines.append(f"- [dom-{i}](dom-{i}.md)") (base / f"dom-{i}.md").write_text( f"---\ntype: verdict\ntitle: Dom {i}\n---\n\nEksperten godkjente tiltaket.\n", encoding="utf-8", ) (base / "index.md").write_text( "---\ntype: index\n---\n\n" + "\n".join(lines) + "\n", encoding="utf-8" ) return base def test_read_bundle_over_the_real_tunnel_base_is_bounded() -> None: """(a) The headline, over the base the order names — not a fixture of my own making.""" blob = _blob(_listing(_TUNNEL)) assert len(blob) <= _CEILING_CHARS, ( f"read_bundle over {_TUNNEL.name} costs {len(blob)} chars, over the ceiling " f"{_CEILING_CHARS}; it used to be 39 583 (12 595 o200k tokens), riding in five prompts" ) def test_read_bundle_cost_does_not_track_document_size(tmp_path: Path) -> None: """(b) Ten times the prose, the same price. O(documents), not O(bytes).""" small = _write_base(tmp_path, "small", body_chars=500) large = _write_base(tmp_path, "large", body_chars=5_000) small_entries, large_entries = _documents(small), _documents(large) # Same document COUNT, same number of entries — and the only field that grew is the honest, # logarithmic ``chars`` digit, so the payloads differ by a handful of characters at most. assert len(small_entries) == len(large_entries) == 3 assert abs(len(_blob(large_entries)) - len(_blob(small_entries))) < 20, ( "ten times the body must not cost ten times the listing" ) assert len(_blob(large_entries)) <= _CEILING_CHARS def test_the_listing_still_identifies_every_document(tmp_path: Path) -> None: """(c) The anti-vacuity arm: bounded is not the same as empty. Bounded-and-useless passes (a) perfectly. What a navigator needs in order to choose a document is what it IS (``type``), what it is CALLED (``title``), what to ask for (``name``) and what it will cost (``chars``) — so all four are asserted, and the entry count is tied to the navigated context files rather than to a number written here. """ bundle = okf.navigate_bundle(str(_TUNNEL)) entries = _documents(_TUNNEL) assert len(entries) == len(bundle.context_files) > 0, ( "a listing that omits documents is a base the navigator cannot fully see" ) by_name = {str(e["name"]): e for e in entries} for f in bundle.context_files: entry = by_name[f.name] assert entry["type"] == (f.type or "document") assert entry["title"] == okf.unquote_scalar(f.frontmatter.get("title", f.name)) assert entry["chars"] == len(f.body) assert str(entry["title"]).strip(), "an untitled entry cannot be chosen between" def test_the_verdict_layer_is_still_excluded(tmp_path: Path) -> None: """(d) Prior verdicts reach a hypothesis ONLY through the gated ExpeL fold (målbilde §4). The old body returned ``okf.bundle_context``, which excludes ``type: verdict`` by construction. A listing built from ``bundle.files`` instead of ``bundle.context_files`` would put them back in front of the navigator — around the gate — and every other arm here would stay green. """ base = _write_base(tmp_path, "med-dommer", body_chars=200, concepts=2, verdicts=2) entries = _documents(base) assert len(entries) == 2, f"the verdict layer must not be listed as context: {entries!r}" assert not [e for e in entries if e["type"] == "verdict"] assert not [e for e in entries if str(e["name"]).startswith("dom-")] # The control that proves the fixture actually HAS verdicts to leak — without it this arm is # green against a bundle that simply carries none. assert len(okf.navigate_bundle(str(base)).verdicts) == 2 def test_the_whole_document_is_still_one_call_away() -> None: """(e) The bound is a disclosure LEVEL, not data loss.""" entries = _documents(_TUNNEL) biggest = max(entries, key=lambda e: int(e["chars"])) whole = _tools(_TUNNEL)["read_file"].func(bundle_id=_TUNNEL.name, path=str(biggest["name"])) assert len(whole) > _CEILING_CHARS, "read_file must still return the document, not a summary" assert str(biggest["chars"]) != "0" and int(biggest["chars"]) <= len(whole) def test_control_one_document_alone_would_blow_the_ceiling() -> None: """(f) The ceiling discriminates — proved, not assumed. The order names this control.""" bundle = okf.navigate_bundle(str(_TUNNEL)) biggest = max(len(f.body) for f in bundle.context_files) assert biggest > _CEILING_CHARS, ( f"the largest document in {_TUNNEL.name} is {biggest} chars; a ceiling it does not exceed " "would be a ceiling this base could pass while carrying everything" )