open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

feat(mcp): serve OKF bundles over MCP in two shapes, plus the generic skill

Browse source 
The eval was written RED at `5f1772e` with no server in the tree. This is the
capability it was written against.

`okf mcp --bundle <dir>` serves exactly one bundle, whose tools take no bundle
argument. `okf mcp --root <dir>` (repeatable) serves every bundle under the
roots and knows NONE of them by name. Four tools -- `okf_list`,
`okf_describe`, `okf_ask`, `okf_fetch` -- each carrying its reason in the
description a client actually reads.

Gate today: 1 (7/7) - 2 (83/181) - 3 (4/4) - 4 (9/9) - 5 (3/3) - 6 (6/6),
`GATE RED: rows 2`, exit 1.

THE PROTOCOL IS STDLIB, AND THAT IS THE PACKAGING INVARIANT KEPT RATHER THAN
A TASTE. An MCP SDK would be this package's second runtime dependency on the
DEFAULT install path, for four JSON-RPC methods and a newline framing, and
`test_the_only_runtime_dependency_is_the_security_boundary` pins that list
literally. Chosen hand-written because the surface needed is `initialize`,
`notifications/initialized`, `tools/list` and `tools/call`; `uv.lock` is
untouched.

NOTHING IS CACHED ACROSS CALLS, and row 3 is why. Every call re-walks the
roots and recomputes `bundle_ref`, so a bundle added, removed or rebuilt while
the process runs is seen by the next call with no restart, no configuration
edit and no code change -- 9 of 9 discovery checks over three bundles written
while the server was serving. The cost is paid per call and is published
rather than hidden: 0.75 s for the identity of a 2 756-concept bundle, 5.6 s
for one ask, 4 min 13 s for row 2's full run over four bundles.

CONTAINMENT IS TWO INDEPENDENT CHECKS: the bundle's own index must name the
concept, AND `connectors.safe_resolve` must place it inside the bundle. A
mutant removing either one alone still refuses -- with a DIFFERENT code, which
row 6 asserts by name -- and one removing both is killed. Row 6 declares a
code set per case because its first run had the 10 MB concept refused as
`concept_unknown`: the fixture had not named the file in the index, so the
size ceiling never ran and the row was green for a reason unrelated to the
attack.

`okf card <bundle>` and `okf skill --generic` are the one-to-many skill
candidate. The card is DERIVED on every run and never written into the bundle:
storing it would move the bytes of all six `examples/*/expected-bundle` trees
(23 files compared byte-for-byte) and of the pinned reference bundle, to keep
something recomputable in under a second, and a stored card is one more
artefact that can disagree with what is beside it. Measured here rather than
taken from the order: two per-bundle skills are identical on 280 of 312 and
310 lines; the 62 that differ are identity, concept count, the
conditional-field table, the whole-bundle cost and the breaking point. The
generic skill carries none of them, and `render_generic()` takes no argument,
so there is no bundle it could have read.

Row 2 decomposes into three numbers and the middle one is the finding: 99 of
181 (bundle, anchor) pairs are present in the bundles at all, 83 of those 99
were reached, and 0 of 83 were met by `okf_fetch` on the anchor as a concept
id. The set's anchors and this library's concept ids are different
vocabularies, so every pair met was met through the ranker -- 83 is a FLOOR on
the ceiling, never the ceiling.

13 mutants in a scratch copy, never in the working tree: 12 killed, 1 survived
with its mechanism printed, 0 errors, control green first. Suite 2323 passed,
2 skipped. The architecture choice between the two shapes is the OPERATOR's;
these rows are its input. Report: docs/2026-09-20-mcp-to-varianter.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-20 10:25:55 +02:00
commit df5a1183c9
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
10 changed files with 1823 additions and 59 deletions
Download patch file Download diff file Expand all files Collapse all files

77
tests/test_mcp_gate.py
View file

@ -67,7 +67,7 @@ def test_every_pinned_denominator_is_recounted_here() -> None:
)
assert gate.DISCOVERY_BUNDLES * len(gate.DISCOVERY_CHECKS) == 3 * 3
assert len(gate.CROSS_CHECKS) == 3
assert gate.HOSTILE_CASES == (
assert tuple(gate.HOSTILE_CASES) == (
"traversal-in-bundle-id",
"traversal-in-concept-id",
"symlink-out-of-root",
@ -75,6 +75,9 @@ def test_every_pinned_denominator_is_recounted_here() -> None:
"oversized-concept",
"unknown-bundle-id",
)
# The 10 MB case has exactly one acceptable refusal. Anything else means
# the ceiling did not run -- the false green this row shipped with once.
assert gate.HOSTILE_CASES["oversized-concept"] == frozenset({"concept_too_large"})
assert gate.OVERSIZED_BYTES == 10 * 1024 * 1024
@ -128,7 +131,7 @@ def test_a_question_set_whose_bytes_moved_is_a_usage_error_not_a_red_row(
payload = {
"sporsmal": [
{
"bundle": "bridge-notes",
"bundles": ["bridge-notes"],
"atomer": [{"kilde_anker": "spennvidde", "kilde_sitat": "24 meter"}],
}
]
@ -171,6 +174,15 @@ def test_an_older_freeze_version_is_refused_rather_than_measured(scratch: Path)
gate.read_anchor_set(questions, freeze, want_version=4)
def test_a_refusal_code_is_read_from_the_message_a_client_sees() -> None:
"""Driven from both sides: a coded refusal yields its code, an uncoded one
yields the empty string rather than a plausible guess."""
assert gate.refusal_code(gate.RpcError(-32000, "refused (concept_too_large): 10 MB")) == (
"concept_too_large"
)
assert gate.refusal_code(gate.RpcError(-32000, "something went wrong")) == ""
def test_the_gate_reads_bundle_and_concept_ids_at_any_depth() -> None:
"""An `ask` answer carries one id per excerpt and a `list` answer one per
bundle. A rule reading the top level only would score the shape."""
@ -222,3 +234,64 @@ def test_one_documented_sequence_answers_from_two_bundles(scratch: Path) -> None
def test_hostile_input_is_refused_out_loud(scratch: Path) -> None:
row = _row(6, scratch)
assert row.k == row.m == 6, f"{row.k} of {row.m}: {row.details}"
def test_row_twos_denominator_is_the_sets_and_not_what_the_surface_reached(
scratch: Path,
) -> None:
"""The defect this repository has already met twice: a row that counts
against what the run produced closes by producing less.
Three pairs are declared and one of them names an anchor no bundle carries,
so a denominator taken from the run would read `2 of 2` and call the gate
satisfied. Driven from both sides -- the reachable pairs really are
reachable, so a `0 of 3` would be a different defect.
"""
import hashlib
bundles = gate.corpus(scratch / "base")
questions = scratch / "sporsmal.json"
questions.write_text(
json.dumps(
{
"sporsmal": [
{
"bundles": ["bridge-notes"],
"atomer": [
{"kilde_anker": "spennvidde", "kilde_sitat": "spennvidde 24 meter"},
{"kilde_anker": "rekkverk", "kilde_sitat": "1,2 meter hoeyt"},
{
"kilde_anker": "finnes-ikke",
"kilde_sitat": "dette staar ingen steder",
},
],
}
]
}
),
encoding="utf-8",
)
freeze = scratch / "frys.json"
freeze.write_text(
json.dumps(
{
"versjon": 4,
"sha256": {"sporsmal.json": hashlib.sha256(questions.read_bytes()).hexdigest()},
}
),
encoding="utf-8",
)
anchors = gate.read_anchor_set(questions, freeze, want_version=4)
assert len(anchors.pairs) == 3
rows = {
row.number: row
for row in gate.evaluate(
scratch / "run",
anchors=anchors,
real={"bridge-notes": bundles["bridge-notes"]},
)
}
row = rows[2]
assert row.m == 3, f"the denominator came from the run: {row.k} of {row.m}"
assert row.k == 2, f"{row.k} of {row.m}: {row.details}"
assert row.status == gate.RED