Commit graph

3 commits

Author SHA1 Message Date
9d1f4b14ed test(fixtures): replace sector-specific example material with generic, fictitious examples — green
Every fixture, test document, tool example and document now uses an invented
kitchen-and-baking handbook series, written in this repository. The package's
behaviour is unchanged; src/ changes are comments and help text only.

- Generated fixtures are regenerated from their generators. Their structural
  counts are identical before and after: elements, images, rows, cells,
  headings, bookmarks and the witness inventory's per-document totals. The
  image-inbox and accounting documents are renamed kapittel-84-*.
- tools/okf_accounting_gate.py: the two options that named one real corpus
  each are replaced by a generic, repeatable --corpus PATH with no default.
  Row 5 compares the PDF pair alone. Gate verdict unchanged: RED rows 2, 3, 6.
- tools/okf_witness.py: the STS JSON reader for one publisher's delivery is
  removed, along with its three twins and five tests. The mutation harness
  loses W09.
- docs/: 13 dated reports that documented runs on a retired reference corpus
  are removed, and 40 are neutralized. Dead links are removed, and no new
  dangling path is introduced.
- The synthetic MCP-gate corpus and the residual probe words are neutral.

Valgt: keep the `okf quality --fasit` bar value (the measured fraction, one corpus) and
rewrite only its provenance, because the verdict stays unchanged and the
number names nothing.

Term check with the local list: 0 of 411 tracked files, 0 file names, 0 of
27 binary fixtures. Suite after git add: 2457 passed, 1 skipped. The base
tree had 2460 passed and 2 skipped; five tests went with the JSON reader and
four were added by the term check. ruff, ruff format and mypy --strict src/
are clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:52:02 +02:00
df5a1183c9
feat(mcp): serve OKF bundles over MCP in two shapes, plus the generic skill
The eval was written RED at `5f1772e` with no server in the tree. This is the
capability it was written against.

`okf mcp --bundle <dir>` serves exactly one bundle, whose tools take no bundle
argument. `okf mcp --root <dir>` (repeatable) serves every bundle under the
roots and knows NONE of them by name. Four tools -- `okf_list`,
`okf_describe`, `okf_ask`, `okf_fetch` -- each carrying its reason in the
description a client actually reads.

Gate today: 1 (7/7) - 2 (83/181) - 3 (4/4) - 4 (9/9) - 5 (3/3) - 6 (6/6),
`GATE RED: rows 2`, exit 1.

THE PROTOCOL IS STDLIB, AND THAT IS THE PACKAGING INVARIANT KEPT RATHER THAN
A TASTE. An MCP SDK would be this package's second runtime dependency on the
DEFAULT install path, for four JSON-RPC methods and a newline framing, and
`test_the_only_runtime_dependency_is_the_security_boundary` pins that list
literally. Chosen hand-written because the surface needed is `initialize`,
`notifications/initialized`, `tools/list` and `tools/call`; `uv.lock` is
untouched.

NOTHING IS CACHED ACROSS CALLS, and row 3 is why. Every call re-walks the
roots and recomputes `bundle_ref`, so a bundle added, removed or rebuilt while
the process runs is seen by the next call with no restart, no configuration
edit and no code change -- 9 of 9 discovery checks over three bundles written
while the server was serving. The cost is paid per call and is published
rather than hidden: 0.75 s for the identity of a 2 756-concept bundle, 5.6 s
for one ask, 4 min 13 s for row 2's full run over four bundles.

CONTAINMENT IS TWO INDEPENDENT CHECKS: the bundle's own index must name the
concept, AND `connectors.safe_resolve` must place it inside the bundle. A
mutant removing either one alone still refuses -- with a DIFFERENT code, which
row 6 asserts by name -- and one removing both is killed. Row 6 declares a
code set per case because its first run had the 10 MB concept refused as
`concept_unknown`: the fixture had not named the file in the index, so the
size ceiling never ran and the row was green for a reason unrelated to the
attack.

`okf card <bundle>` and `okf skill --generic` are the one-to-many skill
candidate. The card is DERIVED on every run and never written into the bundle:
storing it would move the bytes of all six `examples/*/expected-bundle` trees
(23 files compared byte-for-byte) and of the pinned reference bundle, to keep
something recomputable in under a second, and a stored card is one more
artefact that can disagree with what is beside it. Measured here rather than
taken from the order: two per-bundle skills are identical on 280 of 312 and
310 lines; the 62 that differ are identity, concept count, the
conditional-field table, the whole-bundle cost and the breaking point. The
generic skill carries none of them, and `render_generic()` takes no argument,
so there is no bundle it could have read.

Row 2 decomposes into three numbers and the middle one is the finding: 99 of
181 (bundle, anchor) pairs are present in the bundles at all, 83 of those 99
were reached, and 0 of 83 were met by `okf_fetch` on the anchor as a concept
id. The set's anchors and this library's concept ids are different
vocabularies, so every pair met was met through the ranker -- 83 is a FLOOR on
the ceiling, never the ceiling.

13 mutants in a scratch copy, never in the working tree: 12 killed, 1 survived
with its mechanism printed, 0 errors, control green first. Suite 2323 passed,
2 skipped. The architecture choice between the two shapes is the OPERATOR's;
these rows are its input. Report: docs/2026-09-20-mcp-to-varianter.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-20 10:25:55 +02:00
5f1772e832
test(mcp-gate): the eval for an MCP surface over OKF bundles, written RED
No capability lands in this commit. `tools/okf_mcp_gate.py` measures six
rows against a server that does not exist, and the reason is the
interpreter's own words rather than this file's opinion: `python -c 'import
llm_ingestion_okf.mcp_server'` raises ModuleNotFoundError, printed in row 1.

Today: 1 (0/7) - 2 (0/0) - 3 (1/4) - 4 (0/9) - 5 (0/3) - 6 (0/6),
`GATE RED: rows 1, 2, 3, 4, 5, 6`, exit 1.

Row 3 is not 0, and that is a measurement rather than a slip. The order
predicted 0 of N on every row; the incumbent one-to-one skill already
refuses out loud when its bundle is rebuilt, because `okf check`'s
`bundle_mismatch` rule exists and compares the declared ref against the
payload's. One of the four artefact classes therefore passes the loudness
clause today -- at the cost the row also prints: one artefact to recreate
and one manual step per consuming project.

The server is a SUBPROCESS and never an import. Every row speaks
newline-delimited JSON-RPC over stdin/stdout starting at `initialize`, with
a client written narrowly here: a client built from the server's own framing
helpers would agree with the server by construction. Nothing in the gate
ranks, serves or writes a bundle.

Denominators are pinned in the gate and recounted a second time in the
tests. `REQUIRED_TOOLS` (7 across two variants), `DRILL_ARTEFACTS` (4),
`DISCOVERY_BUNDLES x DISCOVERY_CHECKS` (9), `CROSS_CHECKS` (3) and
`HOSTILE_CASES` (6) do not come from the run -- a row counting what the
server happened to offer goes green by offering less, which is the defect
rows 2 and 3 of the retrieval gate were caught on.

Row 2's denominator DOES come from the frozen graded question set, at run
time, and the set is never committed here: `--sett` plus `--frys`, with the
sha256 verified against the freeze file and a freeze version below the
ordered one refused as a usage error, not measured. Without them the row is
`0 of 0` with the reason stated. Its machinery carries a known-positive over
the synthetic corpus this file invents, and a green test proves those quotes
really are in the fixture on disk without asking any server -- so a 0 there
means the surface could not reach text that is demonstrably present.

Six tests are red, every one of them on an assert about behaviour: 0 of 7
tools answered, 1 of 4 artefacts loud, 0 of 9 discovery checks, 0 of 3
cross-bundle checks, 0 of 6 hostile cases, and `find_spec` returning None
rather than an ImportError. Eight instrument tests are green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-20 09:51:06 +02:00