open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

feat(mcp): the bundle's map, and a working method that reads it first

Browse source 
C5. `bundlemap.build_map` lists a bundle in its own words: one line per
source document -- its name, then the titles of its concepts in document
order -- and documents whose names differ only in their numbers (a changelog
per release, a note per week) as ONE line: the name with every number as `#`,
the count, the first and last by natural order, and the titles across the
series that are words. `SERIES_MIN` = 5, at most `TITLES_PER_LINE` = 24 titles
a line, the lines capped at `MAP_MAX_BYTES` = 48 000 together with
`lines_truncated` counting the rest. Derived on every call, never stored.

The card (`okf card`, `okf_describe`) carries it as `map` and no longer
carries `source_files`: that list named every document a second time with no
series collapsed, a quarter of the reply on a large bundle, for names the map
already carries. Chose removal over keeping both because the describe reply
has to fit a client's tool-reply limit and the map says more.

The working method now reads: take the map first (`okf card`, or
`okf_describe`), write two to four sub-questions in its words, and send them
in ONE call (`--question` repeated, or `okf_ask` `questions`). Changed in the
skill template, the generated `skills/okf-consume`, and the server
instructions (held under the 2 KB a client keeps). The regeneration recipe for
`skills/okf-consume` gains `--for-bundle`: since v1.1 the generator writes the
generic skill by default, so the recipe as published produced the other file.

A test holds a four-sub-question `okf_ask` over concepts far over the passage
size under 50 000 bytes of reply text (25 000 tokens at a pessimistic two
bytes a token). The real-collection measurements are kept in local state.

Suite on a clean tree after `git add`: 2429 passed, 2 skipped, 4 xfailed.
ruff, ruff format, mypy --strict clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-21 08:37:12 +02:00
commit da6faf8776
9 changed files with 450 additions and 64 deletions
Download patch file Download diff file Expand all files Collapse all files

18
README.md
View file

@ -1331,15 +1331,15 @@ says why it exists:
| tool | what it answers |
|---|---|
| `okf_list` | which bundles are reachable right now, with each one's content identity and concept count (multi-bundle servers only) |
| `okf_describe` | what one bundle is: id, ref, concept count, source documents, and how many concepts carry each conditionally-written field. Omitting `bundle_id` on a multi-bundle server describes them all, as `okf_ask` does |
| `okf_ask` | one question, one bounded payload of excerpts, each with its bundle id, concept id, title and provenance locators. Omitting `bundle_id` on a multi-bundle server asks them all and splits the budget |
| `okf_describe` | what one bundle is: id, ref, concept count, how many concepts carry each conditionally-written field, and its `map` — one line per source document with its section titles, a series of like-named documents as one line. Omitting `bundle_id` on a multi-bundle server describes them all, as `okf_ask` does |
| `okf_ask` | one question, or two to four sub-questions in `questions`, and one bounded payload of excerpts, each with its bundle id, concept id, title and provenance locators. Omitting `bundle_id` on a multi-bundle server asks them all and splits the budget |
| `okf_fetch` | one named concept, verbatim, with its frontmatter and locators |
**The server carries the working method, because a subagent inherits MCP tools
and not skills.** Its `instructions` and the `okf_ask` description state the
short form — read the map, put the question into the bundle's own words, split
it into sub-questions, read what lay just outside the cut and ask again with
its words, then write one answer in the questioner's language. Claude Code
short form — read the map, write two to four sub-questions in the bundle's own
words and send them in ONE call, read what lay just outside the cut and ask
again with its words, then write one answer in the questioner's language. Claude Code
truncates both at 2 KB, so the long form stays in the skill, which has no such
cap; a test holds the short one under the limit with a control, because a
truncated method is worse than a missing one.
@ -1387,6 +1387,14 @@ never written into the bundle**, so there is no second artefact that can
disagree with the bytes beside it. It is therefore never stale, and one skill
serves every bundle a project holds.
**The card carries the bundle's map** (`map`): one line per source document —
its name, then the titles of its concepts in document order — and documents
whose names differ only in their numbers (a changelog per release) as one line
with the count and the span. It is the bundle's own words, to write
sub-questions in. The lines are capped at 48 000 bytes together
(`lines_truncated` counts what a larger bundle leaves out), and a line lists at
most 24 titles. The map replaced the card's flat `source_files` list.
`okf skill <bundle> --for-bundle` still writes the per-bundle form, with the
identity and the numbers measured into the text — which is exactly what makes
that file stale the moment the bundle is rebuilt. It refuses out loud when it