open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity
llm-ingestion-okf/src/llm_ingestion_okf/skill.py
Kjell Tore Guttormsen 30edd3f5d8 feat(skill): one generic skill by default, carrying a working method and an answer form 
The operator built a 2313-concept bundle from one project's own documentation,
asked it a question in his own words, and judged the result unusable. The
generated skill was an audit contract: all its discipline sat on the accounting
-- markings, denominators, budget lines, source pointers -- and none of it on
understanding the question, searching again, or writing one coherent answer.
Two sentences actively forbade the second of those.

**The two forbidding sentences are gone and their replacements are tested from
both sides.** "Do not go looking for context the pre-pass deliberately
withheld" read as "one run per question", and no wording of the operator's
question put the right document inside a single run's cut -- so a rule against
a second run was a rule against finding it at all. "Not something to retry with
a narrower question" generalised a budget-refusal case into the same ban.
SS 2.2 of the contract said the first of them, so the contract moved with the
skill rather than being left to disagree with it: a second pre-pass run with
other terms, and a fetch of a concept the payload NAMED, are reachable; SS 9's
two real boundaries -- directory enumeration, the verdict layer -- are not.

**Two new sections, and the checker requires them.** `## Working method`: read
the bundle's map, put the question into the bundle's own words, split a broad
question into 2-4 sub-questions, search per sub-question, read what lay just
outside the cut and search again with its words, same method across several
bundles, then assemble ONE answer ordered by sub-question, saying which source
holds and what is not covered. `## Answer form`: the questioner's language,
plain prose, no `below_k`, no digests, no budget lines, no denominators; short
textbook-style references (document + section, plus bundle where several were
read); and the audit trail written only when the questioner asks for it or
into a document that travels without the skill. `REQUIRED_SECTIONS` follows the
template and the contract's new SS 2.5 and SS 2.6 -- never the other way round.

**The generic skill becomes what `okf skill` and `okf project` write.** A
per-bundle skill's numbers go stale the moment its bundle is rebuilt, one copy
per consuming project, and a project with two bundles installs two
near-identical skills; the generic form carries no bundle's numbers and names
`okf card` for them. `--for-bundle` is the opt-in for the instantiated copy,
which still refuses out loud on a stale pairing -- safe to keep, not enough to
keep default. `rule_bundle_identity` learned to tell a generic skill from an
unfilled template by the frontmatter name the generator writes, so the template
still fails for the opposite reason: it declares no identity because it is
unfinished.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-20 23:53:34 +02:00

1006 lines
43 KiB
Python
Raw Blame History

"""Instantiate the consumption skill template for ONE named OKF bundle.
`skills/okf-consume-template/SKILL.md` is a template whose own rule is that a
copy leaving a `<PLACEHOLDER>` unfilled "is not configured, it is unfinished".
Filling it by hand is what first produced `skills/okf-consume/`, for one corpus.
This command does the same thing for any bundle, from values it measures rather
than values someone remembered; since 2026-09-11 that directory is this
command's own output for the golden bundle the repository ships.
**Why a generator rather than one generic skill.** The measurement this
paragraph used to rest on is CLOSED 2026-09-10. It read: the contract checker
passes the UNFILLED template against a real payload (exit 0, 15 rules, 0
findings), and passes a skill built for a different bundle against this one's
payload. `contract_check.rule_bundle_identity` now compares the identity a
skill declares with the identity its payload declares, so all three measured
pairs are refused at exit 1 with one `bundle_mismatch` finding over 16 rules
(17 since K3-21's `parent_unfollowable`):
a skill against another bundle's payload, the unfilled template against a real
payload, and -- the arm an id comparison would miss -- a payload sharing the
skill's `bundle_id` at a foreign `ref`. The right pair is untouched at exit 0
with 0 findings.
**The argument for a generator never rested on conformance, and still does
not.** It was made on what the skill has to state: SS 5's denominators, the
payload-cost section and SS 6.4's conditional-field list are all per-bundle
numbers, and a generic skill can either leave them as holes -- the template's
own definition of unfinished -- or carry another corpus's numbers, which is
worse.
**AND IT LOST 2026-09-20, TO A THIRD OPTION AND AN OPERATOR'S MEASUREMENT.**
The third option is `okf card`: the per-bundle numbers are DERIVED from the
bundle in under a second, so the generic skill neither invents them nor states
another bundle's -- it names the command that produces them. The measurement
is that the per-bundle form's cost is not hypothetical: it goes stale the
moment its bundle is rebuilt, one copy per consuming project, and a project
holding two bundles installs two near-identical skills (measured: identical on
281 of 313 and 311 lines). So `okf skill --out <dir>` writes the GENERIC form,
`okf project` installs it under one name that carries no bundle id, and
`--for-bundle` is the opt-in for the instantiated copy. That copy still
refuses out loud (`bundle_mismatch`) rather than answering from stale numbers,
which is what makes it safe to keep and not enough to keep it default.
**Zero model calls, zero network, no clock.** The same bundle bytes produce the
same skill bytes.
**It moved into the package on 2026-09-08 (O5), and so did the reason it could
not before.** The old objection was exact: a wheel-installed `okf skill` would
emit a command pointing at `tools/okf_consume.py`, which the wheel does not
contain. That objection was about what the generated skill NAMES, and the
answer was to change what it names. The emitted commands are now `okf consume`
and `okf check` -- names on PATH after an install, resolved by the shell and
not by this repository's layout. The pre-pass and the checker moved into the
package in the same step, so both names exist wherever the generated skill
does. Measured before the move: a skill generated from a checkout carried four
lines with an absolute path into that checkout, two of them the commands a
reader is told to run.
Exit codes are three, as elsewhere in this chain: 0 the skill was written, 1 the
run happened and refused, 2 the run did not happen.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from collections import Counter
from pathlib import Path
from . import consume as okf_consume
from .profiles import BundleProfile, block_scalar
#: The template, resolved to a copy that exists wherever this module does.
#:
#: A wheel carries `_data/okf-consume-template.md`, force-included from the
#: authored file below at build time; a source tree carries only the authored
#: file. Preferring the packaged copy and falling back keeps ONE authored
#: template -- a committed second copy is the drift this generator exists to
#: prevent, restated one directory up.
PACKAGED_TEMPLATE = Path(__file__).resolve().parent / "_data" / "okf-consume-template.md"
AUTHORED_TEMPLATE = (
Path(__file__).resolve().parents[2] / "skills" / "okf-consume-template" / "SKILL.md"
)
#: What the generated skill tells a reader to RUN. Names on PATH after an
#: install, never paths into a checkout: a generated skill is meant to be moved,
#: shared and run by someone who does not have this repository.
PRE_PASS_COMMAND = "okf consume"
CHECKER_COMMAND = "okf check"
#: The contract is prose in this repository and is NOT shipped in the wheel, so
#: the generated skill names it as a document rather than as a path a reader
#: could open. Naming an absolute path here is what made a generated skill
#: unmovable; naming a file the reader may not have is honest about which it is.
CONTRACT = "docs/consumption-contract.md in open/llm-ingestion-okf"
#: The profile the pre-pass reads a bundle under, spelled so the generated skill
#: can name it in § 9.2's sentence. The pre-pass's own default; a bundle built
#: under another profile needs a copy of this tool that says so.
PROFILE_NAME = "SEGMENTED_OKF_V0_2"
#: The conditional frontmatter keys the generated skill reports a denominator
#: for. Every one of them is written by SOME producer and not by others, which
#: is exactly what § 6.4 says a consumer must be told about rather than left to
#: infer from an absence.
CONDITIONAL_FIELDS = (
"adjudication",
"bundle_id",
"verified",
"req_number",
"sources",
"parent",
)
#: Tokens too short to carry a question. The same floor the pre-pass's own
#: matcher uses, so the derived example question cannot be shorter than what the
#: ranker can see.
MIN_QUESTION_TOKEN = 5
class SkillError(Exception):
"""The generator refused. Carries the code, like the rest of this chain."""
def __init__(self, message: str, *, code: str) -> None:
super().__init__(message)
self.code = code
def template_path() -> Path:
"""The template this generator instantiates, or a coded refusal.
Refuses rather than falling through to a default: a generator that
silently emitted a skill built from no template would produce a file
carrying this bundle's numbers and none of the contract's sections.
"""
for candidate in (PACKAGED_TEMPLATE, AUTHORED_TEMPLATE):
if candidate.is_file():
return candidate
raise SkillError(
f"the consumption skill template was not found at {PACKAGED_TEMPLATE} "
f"or {AUTHORED_TEMPLATE}",
code="template_missing",
)
# --- The blocks the template hands over verbatim ------------------------------
#: Every template block this generator rewrites WHOLE, by exact string. Held to
#: the template by a test: an edit that moves one of these would otherwise
#: produce a skill silently missing that rewrite, which is the drift the
#: instantiated copy exists to avoid.
TEMPLATE_HEADER = """**This file is a template.** Every `<PLACEHOLDER>` is a hole a per-corpus copy
fills; none of them has a default, and a copy that leaves one unfilled is not
configured, it is unfinished. The section headings are fixed:
`okf check` reads them, and a missing one makes the skill
non-conformant rather than merely thin."""
TEMPLATE_PRE_PASS = """```sh
<PRE_PASS_COMMAND> --bundle-root <BUNDLE_ROOT> --ref <REF> --out <PAYLOAD_PATH>
```"""
TEMPLATE_CHECK = """```sh
okf check --skill <SKILL_PATH> --payload <PAYLOAD_PATH>
```"""
TEMPLATE_CONTRACT_LINE = (
"The contract this skill is held to is `docs/consumption-contract.md`. Where this"
)
TEMPLATE_EXTENSIONS = """**Extensions, if this corpus needs any.** `<EXTENSION_MARKINGS: for each, the
literal, what it means here, and which of the five it would otherwise collapse
into. Write "none" if there are none.>`"""
TEMPLATE_CONDITIONAL = """**Conditionally-written fields in this corpus.** `<CONDITIONAL_FIELDS: each
field this profile writes only when a build-time condition held, and what its
absence does and does not mean. Absence is a measurement, not a fact.>`"""
TEMPLATE_SCALING = """**Scaling.** `<COST_SCALING: whether cost tracks the question or the corpus, what
the whole bundle at this ref costs by the same instrument, and the corpus size
at which this strategy stops fitting the budget.>`"""
TEMPLATE_DENOMINATORS = """The payload reports three counts — `considered`, `withheld`, `delivered` — and
`considered == withheld + delivered`. Carry them into your output."""
TEMPLATE_ENUMERATION = (
"- **No directory enumeration** unless `<PROFILE_NAME>` says the index is derived.\n"
" Searching again, with other words, is not enumeration: the pre-pass walks the\n"
" index and applies the same rules every time, and a second run is another\n"
" measurement, not a way around the first one."
)
TEMPLATE_OUTPUT = "Write to `<OUT>`. **The answer comes first and is written in the answer form**:"
#: Every per-corpus hole the template carries. A generic skill that left one
#: would be the unfilled template with better manners, so it is refused.
_PLACEHOLDER = re.compile(r"<[A-Z][A-Z_]*>")
REPLACED_BLOCKS = (
TEMPLATE_HEADER,
TEMPLATE_PRE_PASS,
TEMPLATE_CHECK,
TEMPLATE_CONTRACT_LINE,
TEMPLATE_EXTENSIONS,
TEMPLATE_CONDITIONAL,
TEMPLATE_SCALING,
TEMPLATE_DENOMINATORS,
TEMPLATE_ENUMERATION,
TEMPLATE_OUTPUT,
)
# --- What the generator measures ----------------------------------------------
def slug(value: str) -> str:
"""A Claude Code skill name from a bundle id: lowercase, hyphen-joined."""
reduced = re.sub(r"[^a-z0-9]+", "-", value.lower()).strip("-")
return reduced or "okf"
def example_question(titles: list[str]) -> str:
"""A question this bundle really answers, derived rather than invented.
The most frequent long token across the concepts' own titles, byte-sorted on
a tie. Derived because the shipped payload has to be one this bundle
produces: a question sharing no token with any concept is withheld under
`no_lexical_match` and the pre-pass refuses, so a hand-picked constant would
fail on the first bundle that does not happen to contain it.
"""
counts: Counter[str] = Counter()
for title in titles:
counts.update(
{token for token in okf_consume.normalise(title) if len(token) >= MIN_QUESTION_TOKEN}
)
if not counts:
# Every title is short or empty. Fall back to the longest title as it
# stands, which by construction matches at least its own concept.
longest = max(titles, key=lambda title: (len(title), title), default="")
if not longest:
raise SkillError(
"no concept in this bundle carries a title, so no example "
"question can be derived from it; pass --example-question",
code="no_example_question",
)
return longest
top = min(counts.items(), key=lambda item: (-item[1], item[0]))[0]
return f"Hva sier denne bundelen om {top}?"
def field_counts(concepts: list[okf_consume.Concept]) -> dict[str, int]:
"""How many concepts carry each conditional field. Set membership, never a
guess from two equal totals.
The `source_*` rows are DISCOVERED from the bundle rather than listed here,
for the same reason the excerpt carries them by prefix: a fixed list reports
a denominator for the producers someone thought of, and says nothing about
the locator this producer actually chose.
"""
discovered = sorted(
{
key
for concept in concepts
for key in concept.frontmatter
if key.startswith(okf_consume.SOURCE_KEY_PREFIX)
}
)
fields = (*CONDITIONAL_FIELDS, *discovered)
counts = {field: 0 for field in fields}
for concept in concepts:
for field in fields:
if concept.frontmatter.get(field, "").strip() or (
field == "sources" and concept.sources_present
):
counts[field] += 1
return counts
def whole_bundle_cost(concepts: list[okf_consume.Concept]) -> int:
"""What every concept in this bundle would cost by the gate's own
instrument, if a single answer delivered all of them."""
total = 0
for concept in concepts:
excerpt = okf_consume.excerpt_for(concept)
if excerpt is not None:
total += okf_consume.excerpt_weight(excerpt)
return total
def directory_control(bundle_root: Path, *, profile: BundleProfile) -> tuple[int, int]:
"""The index walk against the method § 9.2 forbids the CONSUMER from using.
Run HERE, once, at generation time -- never on the question path. § 9.2
binds a consumer reaching for context at query time; a build-time control is
what turns "the walk loses nothing" from an assumption into a number the
generated skill can quote.
"""
walked = len(okf_consume.enumerate_concepts(bundle_root, profile=profile))
suffix = profile.paths.concept_suffix
reserved = {profile.index.name, "log.md"}
on_disk = len([path for path in bundle_root.rglob(f"*{suffix}") if path.name not in reserved])
return walked, on_disk
# --- Where the reader runs the commands (O6) -----------------------------------
#: `okf project` writes the skill to `<root>/.claude/skills/<id>-consume` and
#: the bundle to `<root>/.okf/<id>`, then tells the reader to start `claude` in
#: `<root>`. So `<root>` is where the skill's commands are read from, and a path
#: written relative to it is the one a reader can paste.
SKILLS_LAYOUT = (".claude", "skills")
def project_root_of(out: Path) -> Path | None:
"""`<root>/.claude/skills/<name>` -> `<root>`; anything else -> None.
Read off the path rather than passed in, so `okf skill` and `okf project`
reach the same answer without a second parameter that could disagree with
the layout on disk.
"""
resolved = out.resolve()
if resolved.parts[-3:-1] == SKILLS_LAYOUT:
return resolved.parents[2]
return None
def as_written(path: Path, *, base: Path | None) -> str:
"""The path as the skill states it: relative to `base` when it is under it.
A path outside `base` stays absolute on purpose. `../../..` is not more
portable than `/Users/...`, it is only harder to read, and a skill that
states a path its reader cannot resolve is worse than one that states an
honest absolute.
"""
resolved = path.resolve()
if base is None:
return str(resolved)
try:
return resolved.relative_to(base).as_posix()
except ValueError:
return str(resolved)
# --- The instantiation --------------------------------------------------------
def render(
bundle_root: Path,
*,
out: Path,
profile: BundleProfile = okf_consume.DEFAULT_PROFILE,
question: str | None = None,
) -> tuple[str, dict[str, object]]:
"""The skill text and the example payload that proves it, for one bundle."""
bundle_root = bundle_root.resolve()
bundle_id = okf_consume.root_bundle_id_of(bundle_root, profile=profile)
ref = okf_consume.bundle_ref(bundle_root, profile=profile)
concept_ids = okf_consume.enumerate_concepts(bundle_root, profile=profile)
concepts = okf_consume.link_parents(
[
okf_consume.read_concept(
okf_consume.read_path_in_bundle(
bundle_root, f"{concept_id}{profile.paths.concept_suffix}"
),
bundle_root=bundle_root,
root_bundle_id=bundle_id,
)
for concept_id in concept_ids
]
)
if not concepts:
raise SkillError(
f"{bundle_root} has an index but no concept under it; a skill for an "
"empty bundle would state denominators of zero it never measured",
code="bundle_empty",
)
total = len(concepts)
asked = question or example_question([concept.title for concept in concepts])
payload = okf_consume.build_payload(bundle_root, question=asked, profile=profile)
counts = field_counts(concepts)
walked, on_disk = directory_control(bundle_root, profile=profile)
cost = whole_bundle_cost(concepts)
denominators = payload["denominators"]
assert isinstance(denominators, dict)
budget = payload["budget"]
assert isinstance(budget, dict)
withheld = payload["withheld"]
assert isinstance(withheld, dict)
bookkeeping = okf_consume.measure(json.dumps(withheld, ensure_ascii=False))
withheld_total = withheld["total"]
assert isinstance(withheld_total, int)
named = withheld["nearest"]
assert isinstance(named, list)
name = f"{slug(bundle_id)}-consume"
text = template_path().read_text(encoding="utf-8")
text = text.split("---\n", 2)[2]
base = project_root_of(out)
text = _rewrite(
text,
bundle_root=Path(as_written(bundle_root, base=base)),
skill_path=Path(as_written(out / "SKILL.md", base=base)),
bundle_id=bundle_id,
ref=ref,
name=name,
total=total,
counts=counts,
walked=walked,
on_disk=on_disk,
cost=cost,
asked=asked,
spent=int(budget["spent"]),
delivered=int(denominators["delivered"]),
bookkeeping=bookkeeping,
withheld_count=withheld_total,
withheld_named=len(named),
)
# Claude Code reads this header with a YAML reader, and `description`
# carries the root index's `bundle_id` raw -- a bundle this library did not
# build may call itself anything (K3-22).
description = block_scalar(_description(bundle_id, total, ref))
header = f"---\nname: {block_scalar(name)}\ndescription: {description}\n---\n"
return header + text, payload
def identity_line(bundle_id: str, ref: str) -> str:
"""The one sentence that says which bundle a generated skill belongs to.
Authored here because the generator writes it, and read back by
`contract_check.skill_identity`, whose `bundle_mismatch` rule is the reason
it has to be findable rather than merely present. Two copies of this
sentence would drift, and the copy nobody reads is the one that goes wrong,
so the coupling has its own test.
"""
return f"generated by `okf skill` for one bundle: `{bundle_id}` at ref\n`{ref}`"
def _description(bundle_id: str, total: int, ref: str) -> str:
return (
f"Answer one question about the OKF bundle `{bundle_id}` ({total} concepts, "
f"ref {ref}) from a bounded payload assembled by a deterministic pre-pass, "
"marking every claim with its source, its title and its provenance locator. "
"Use whenever a question is about what that bundle's documents require, say "
"or contain. Generated by `okf skill`; every value below is measured "
"against this bundle at this ref."
)
def _rewrite(
text: str,
*,
bundle_root: Path,
skill_path: Path,
bundle_id: str,
ref: str,
name: str,
total: int,
counts: dict[str, int],
walked: int,
on_disk: int,
cost: int,
asked: str,
spent: int,
delivered: int,
bookkeeping: int,
withheld_count: int,
withheld_named: int,
) -> str:
replacements: list[tuple[str, str]] = [
(
TEMPLATE_HEADER,
"**This file is an instantiated copy of "
"`skills/okf-consume-template/SKILL.md`,** "
f"{identity_line(bundle_id, ref)}. Every value below was measured "
"against those bytes. If the\n"
"bundle moves, the ref moves with it and this file is stale — regenerate\n"
"it rather than editing a number here. The section headings are fixed:\n"
"the contract checker reads them by name.",
),
(
TEMPLATE_PRE_PASS,
"```sh\n"
f"{PRE_PASS_COMMAND} \\\n"
f" {bundle_root} \\\n"
' --question "your question" \\\n'
f" --ref {ref} \\\n"
" --out /tmp/payload.json\n"
"```\n\n"
"`--ref` is an **assertion**, never an override: the identity is computed\n"
"from the bytes either way, and a mismatch refuses. Read the pre-pass's\n"
"own exit status, which carries three values: **0** a payload was written,\n"
"**1** the run happened and refused, **2** the run did not happen at all.",
),
(
TEMPLATE_CHECK,
"```sh\n"
f"{CHECKER_COMMAND} \\\n"
f" --skill {skill_path} \\\n"
" --payload /tmp/payload.json\n"
"```",
),
(
TEMPLATE_CONTRACT_LINE,
f"The contract this skill is held to is `{CONTRACT}`. Where this",
),
(
TEMPLATE_EXTENSIONS,
"**Extensions, if this corpus needs any: none.** This generated skill adds\n"
"no marking to the required five. § 4.3 makes the undeclared extension the\n"
"defect, so the absence is stated rather than left to be inferred — and a\n"
"corpus that does need a sixth needs a hand-edited copy that declares it.",
),
(
TEMPLATE_CONDITIONAL,
_conditional_table(total, counts),
),
(
TEMPLATE_SCALING,
_scaling(
total=total,
cost=cost,
asked=asked,
spent=spent,
delivered=delivered,
bookkeeping=bookkeeping,
withheld_count=withheld_count,
withheld_named=withheld_named,
),
),
(
TEMPLATE_DENOMINATORS,
_denominators(total, asked=asked, delivered=delivered),
),
(
TEMPLATE_ENUMERATION,
_enumeration(walked, on_disk),
),
(
TEMPLATE_OUTPUT,
"Write to the path the caller names, or to your answer if none was named.\n"
"**The answer comes first and is written in the answer form**:",
),
("`<CORPUS>` bundle", f"`{bundle_id}` bundle"),
("# <CORPUS> consumption", f"# {bundle_id} consumption"),
("<BUDGET_LIMIT>", str(okf_consume.DEFAULT_LIMIT)),
("<BUDGET_UNIT>", okf_consume.BUDGET_UNIT),
("<BUDGET_INSTRUMENT>", okf_consume.BUDGET_INSTRUMENT),
("<KNOWN_POSITIVE_CASE>", okf_consume.KNOWN_POSITIVE_CASE),
("<KNOWN_POSITIVE_EXPECTED>", str(okf_consume.KNOWN_POSITIVE_EXPECTED)),
# The working method's own command block. STRICT like the rest: a
# per-bundle skill telling its reader to search again against
# `<BUNDLE_ROOT>` would be the unfilled template's hole inside the one
# section that asks for a second run.
("<BUNDLE_ROOT>", str(bundle_root)),
]
for old, new in replacements:
if old not in text:
raise SkillError(
f"the template no longer carries the block this generator rewrites: {old[:70]!r}",
code="template_drift",
)
text = text.replace(old, new)
assert name # kept in the signature so a caller cannot forget to name the skill
return text
def _conditional_table(total: int, counts: dict[str, int]) -> str:
rows = "\n".join(
f"| `{field}` | **{count} of {total}** | "
f"{'the producer wrote none for that concept' if count else 'no concept in this bundle carries it'} | "
"that the source document lacks what the field asserts |"
for field, count in counts.items()
)
return (
"**Conditionally-written fields in this bundle, with what each absence does\n"
"and does not mean.** Every count is over the same denominator — "
f"**{total} concepts**, the set the index walk reaches. § 6.4: absence is a\n"
"measurement about the producer, never a fact about the source.\n\n"
"| Field | Present on | Absence means | Absence does NOT mean |\n"
"|---|---|---|---|\n"
f"{rows}\n\n"
"A field present on **0 of "
f"{total}** is a measured zero, not an unmeasured one: the count was taken\n"
"over every concept, and it is reported so a negative claim resting on it\n"
"carries its denominator."
)
def _scaling(
*,
total: int,
cost: int,
asked: str,
spent: int,
delivered: int,
bookkeeping: int,
withheld_count: int,
withheld_named: int,
) -> str:
share = (spent / cost * 100) if cost else 0.0
return (
"**Scaling. Cost tracks the question, not the corpus.** Measured on this\n"
f"bundle at generation time, with the question `{asked}`: the delivered set\n"
f"was **{delivered} excerpts** costing **{spent} {okf_consume.BUDGET_UNIT}**,\n"
f"against a whole bundle that would cost **{cost}** by the same instrument if\n"
f"one answer delivered all {total} concepts — so that answer was about\n"
f"**{share:.1f} %** of the corpus. One question is one measurement: a\n"
"different question moves `spent` and this figure with it.\n\n"
+ _breaking_point(
total=total,
bookkeeping=bookkeeping,
withheld_count=withheld_count,
withheld_named=withheld_named,
)
)
def _breaking_point(
*, total: int, bookkeeping: int, withheld_count: int, withheld_named: int
) -> str:
"""What the payload costs beyond its excerpts, and whether it grows.
Until `okf-consumption/2` this section extrapolated a concept count at
which the bookkeeping alone would fill the budget, because `withheld`
carried one entry per considered concept and grew linearly. Measured
2026-09-20 on a 2313-concept bundle, that growth had arrived: the list was
186 440 B = 65.5 % of the written file, none of it counted against
`spent`.
It does not grow that way any more, so this section no longer states a
concept count -- a number extrapolated from a slope the code no longer has
would be a measurement of the previous revision. What it states instead is
the two facts that are measured on THIS bundle: what the bookkeeping cost
here, and that it is bounded by the near-miss cap rather than by the
bundle's size.
"""
return (
"**What the payload costs beyond its excerpts.** The `withheld` block is\n"
"counts plus names, not one entry per concept: here it is\n"
f"**{bookkeeping} bytes** — {withheld_count} withheld of {total} concepts,\n"
f"of which **{withheld_named}** are named. Its size is bounded by that cap\n"
"and by the number of distinct withholding rules, not by the bundle's size,\n"
"so it does not overtake the excerpts as a bundle grows; the whole list is\n"
"still reachable with `--withheld-full`, and then it does. None of it counts\n"
"against `spent`. The pre-pass reads every concept body on every run, so\n"
"growth is a wall-clock cost with no precomputed index behind it."
)
def _denominators(total: int, *, asked: str, delivered: int) -> str:
return (
"The payload reports three counts — `considered`, `withheld`, `delivered` — and\n"
"`considered == withheld + delivered`. Carry them into your output.\n\n"
f"For this bundle `considered` is **{total}**, every concept the index walk\n"
"reaches, never the post-ranking shortlist. A concept dropped at the ranking\n"
"stage is `withheld` **with its rule**, not invisible, and the rules are a\n"
"closed set of seven: `verdict_layer_excluded` (a verdict-layer file, § 9.1),\n"
"`verified_unreadable` (a `verified` value this reader cannot decode, so no\n"
"tier can be derived), `no_lexical_match` (the concept shares no token with\n"
"the question), `over_budget_alone` (one excerpt exceeds the whole limit),\n"
"`source_quota_exceeded` (its source document already holds as many\n"
"delivered places as `--source-quota` allows, default 2 — the freed place\n"
"goes to the next candidate, so `k` is still delivered in full),\n"
"`below_k` (ranked outside the shortlist the cut considers) and\n"
"`over_budget_after_knapsack` (it ranked inside the shortlist and the pack\n"
"had no room). Naming the rule is what makes a drop visible.\n\n"
"**One limitation to carry into every negative claim.** `no_lexical_match` is\n"
'a per-concept relevance drop, not a whole-question "this bundle has no\n'
f'answer" gate: on the generation question `{asked}` it still returned\n'
f"{delivered} excerpts. **An empty `excerpts` list is evidence of absence; a\n"
"full one is not evidence of presence.** When the delivered excerpts do not\n"
"actually answer the question, say `[sourced-not-sufficient]` and report that\n"
"the cut found nothing responsive."
)
def _enumeration(walked: int, on_disk: int) -> str:
agreement = (
f"which costs nothing here: the walk reaches **{walked}** concepts and a\n"
f" directory walk finds **{on_disk}**"
if walked == on_disk
else f"and the two disagree — the walk reaches **{walked}** concepts where a\n"
f" directory walk finds **{on_disk}**, so some concept is unreachable through\n"
" the index and the bundle's producer should be told"
)
return (
"- **No directory enumeration.** This bundle is read under the\n"
f" `{PROFILE_NAME}` profile, whose index policy declares\n"
" `entries_match_directory = False`, so § 9.2's permission does not apply.\n"
f" The pre-pass walks the **index tree** instead, {agreement}\n"
" (controlled once at generation time, never on the question path). Do not\n"
" enumerate a directory yourself either."
)
def generate(
bundle_root: Path,
*,
out: Path,
profile: BundleProfile = okf_consume.DEFAULT_PROFILE,
question: str | None = None,
force: bool = False,
) -> Path:
"""Write `out/SKILL.md` and its reference payload. Returns the skill path."""
target = out / "SKILL.md"
if target.exists() and not force:
raise SkillError(
f"{target} already exists; pass --force to replace it. A silent "
"overwrite would destroy a hand-edited copy whose extra measurements "
"this generator cannot reproduce",
code="target_occupied",
)
text, payload = render(bundle_root, out=out, profile=profile, question=question)
(out / "references").mkdir(parents=True, exist_ok=True)
target.write_text(text, encoding="utf-8")
(out / "references" / "example-payload.json").write_text(
okf_consume.serialise(payload), encoding="utf-8"
)
return target
def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument(
"bundle",
type=Path,
nargs="?",
help="the OKF bundle to instantiate a skill for. Only read with --for-bundle",
)
parser.add_argument(
"--out", type=Path, required=True, help="the skill directory to write (SKILL.md inside)"
)
parser.add_argument(
"--example-question",
default=None,
help="the question the shipped reference payload answers. Derived from the "
"bundle's own titles when omitted",
)
parser.add_argument(
"--force", action="store_true", help="replace an existing SKILL.md at --out"
)
parser.add_argument(
"--generic",
action="store_true",
help="the default since 2026-09-20; accepted so existing call sites keep working",
)
parser.add_argument(
"--for-bundle",
action="store_true",
help=(
"write the per-bundle form instead: one skill carrying THIS bundle's "
"identity, concept count, conditional-field counts and cost. It goes "
"stale the moment the bundle is rebuilt and refuses out loud "
"(`bundle_mismatch`) when it was not regenerated, which is why it is "
"no longer the default. Requires `bundle`"
),
)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
try:
if args.for_bundle and args.bundle is None:
print("refused (bundle_missing): --for-bundle needs a bundle", file=sys.stderr)
return 2
written = (
generate(args.bundle, out=args.out, question=args.example_question, force=args.force)
if args.for_bundle
else generate_any(out=args.out, force=args.force)
)
except okf_consume.ConsumeError as exc:
print(f"refused ({exc.code}): {exc}")
return 1
except SkillError as exc:
print(f"refused ({exc.code}): {exc}")
return 1
except OSError as exc:
print(f"the run did not happen: {exc}")
return 2
print(f"wrote {written}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
# --- The one-to-many candidate ------------------------------------------------
#: The name the generic skill carries. Claude Code takes a project skill's
#: command from its DIRECTORY name and uses `name` only as a display label, so
#: this is the label and not the command.
GENERIC_NAME = "okf-consume-any"
#: The command that hands a reader the per-bundle numbers this skill does not
#: carry. It has to exist for the skill to be honest: a generic document that
#: told a reader to "check the denominators somewhere" would be the unfilled
#: template with better manners.
CARD_COMMAND = "okf card"
GENERIC_BUNDLE = "<the bundle you were pointed at>"
def render_generic() -> str:
"""One installable skill for ANY bundle, carrying no bundle's numbers.
The measured fact this answers: two skills generated for two different
bundles are identical on 280 of 312 and 310 lines (measured 2026-09-20 on
this machine, over `examples/ingest-golden-segmented-okf-v0-2` and
`tests/fixtures/consume-bundle`; the order's own 227 of 285 is a different
pair of bundles and neither number contradicts the other). The 30-odd lines
that differ are identity, concept count, the conditional-field table, the
whole-bundle cost and the breaking point -- all of them recomputable from
the bundle in under a second, and all of them what makes a generated skill
go stale the moment its bundle is rebuilt.
So this text carries NONE of them, and says where to read each one instead.
The property that makes that claim checkable is that this function takes no
argument: there is no bundle it could have read, and two calls return the
same bytes.
"""
text = template_path().read_text(encoding="utf-8")
text = text.split("---\n", 2)[2]
replacements: list[tuple[str, str]] = [
(
TEMPLATE_HEADER,
"**This file is generic: it carries no bundle's identity and no bundle's\n"
"numbers,** and it is therefore never stale. It serves whichever bundle you\n"
"are pointed at. Before answering, read that bundle's own card:\n\n"
"```sh\n"
f"{CARD_COMMAND} {GENERIC_BUNDLE}\n"
"```\n\n"
"The card is DERIVED from the bundle on every run, never stored in it, so\n"
"there is no second artefact that can disagree with the bytes. Its\n"
"`bundle_id` and `ref` are the identity to carry into your output; its\n"
"`concept_count`, `conditional_fields` and `whole_bundle_bytes` are the\n"
"denominators the sections below ask for. The section headings are fixed:\n"
"the contract checker reads them by name.",
),
(
TEMPLATE_PRE_PASS,
"```sh\n"
f"{PRE_PASS_COMMAND} \\\n"
f" {GENERIC_BUNDLE} \\\n"
' --question "your question" \\\n'
" --ref THE_REF \\\n"
" --out /tmp/payload.json\n"
"```\n\n"
"`--ref` is an **assertion**, never an override: the identity is computed\n"
"from the bytes either way, and a mismatch refuses. Read the pre-pass's\n"
"own exit status, which carries three values: **0** a payload was written,\n"
"**1** the run happened and refused, **2** the run did not happen at all.",
),
(
TEMPLATE_CHECK,
f"```sh\n{CHECKER_COMMAND} --skill <this file> --payload /tmp/payload.json\n```",
),
(
TEMPLATE_CONTRACT_LINE,
f"The contract this skill is held to is `{CONTRACT}`. Where this",
),
(
TEMPLATE_EXTENSIONS,
"**Extensions.** This skill declares none. A corpus needing one declares it\n"
"in its own documentation; the five markings below are never extended here,\n"
"because a marking invented for one bundle would travel to every other.",
),
(
TEMPLATE_CONDITIONAL,
"**Conditionally-written fields.** Read `conditional_fields` from the card:\n"
"it gives, per field, how many of the bundle's concepts carry it. A field\n"
"written on some concepts and not others means its ABSENCE on one concept\n"
"is a measurement about that concept, never a fact about the world — so\n"
"report the count beside any claim that rests on an absence. The fields\n"
f"this profile can write are: {', '.join(f'`{field}`' for field in CONDITIONAL_FIELDS)}.",
),
(
TEMPLATE_SCALING,
"**Scaling.** Cost tracks the QUESTION, not the corpus: the payload is cut\n"
f"to {okf_consume.DEFAULT_LIMIT} {okf_consume.BUDGET_UNIT} whatever the bundle's size. What\n"
"does track the corpus is the wall clock: the pre-pass reads every concept\n"
"body on every run, with no precomputed index behind it. The bookkeeping\n"
"does not — `withheld` is counts plus a capped sample of names, so it is\n"
"bounded by that cap rather than by the bundle. Read `whole_bundle_bytes` from\n"
"the card and compare it with the budget: a bundle costing less than the\n"
"budget could have been handed over whole, and the pre-pass is then a\n"
"convenience rather than a necessity.",
),
(
TEMPLATE_DENOMINATORS,
"The payload reports three counts — `considered`, `withheld`, `delivered` —\n"
"and `considered == withheld + delivered`. Carry them into your output, and\n"
"carry the card's `concept_count` beside them: `considered` is what the cut\n"
"looked at, and the card says how much of the bundle that was.",
),
(
TEMPLATE_ENUMERATION,
f"- **No directory enumeration** unless the profile (`{PROFILE_NAME}`) says the\n"
" index is derived. The payload's own `bundle.entries_match_directory` says\n"
" whether it does, for the bundle in front of you.",
),
(
TEMPLATE_OUTPUT,
"Write to the path the caller names, or to your answer if none was named.\n"
"**The answer comes first and is written in the answer form**:",
),
("`<CORPUS>` bundle", "bundle you were pointed at"),
("# <CORPUS> consumption", "# OKF bundle consumption"),
]
# The blocks are STRICT: a template that stopped carrying one has drifted,
# and rewriting the rest would ship a skill missing a whole section.
for old, new in replacements:
if old not in text:
raise SkillError(
f"the template no longer carries the block this generator rewrites: {old[:70]!r}",
code="template_drift",
)
text = text.replace(old, new)
# The tokens are LENIENT, and the sweep below is what makes that safe: a
# token may already have been consumed by the block that carried it, and a
# strict check here would only measure the order of this list.
for old, new in (
("<PROFILE_NAME>", PROFILE_NAME),
("<PRE_PASS_COMMAND>", PRE_PASS_COMMAND),
("<BUDGET_LIMIT>", str(okf_consume.DEFAULT_LIMIT)),
("<BUDGET_UNIT>", okf_consume.BUDGET_UNIT),
("<BUDGET_INSTRUMENT>", okf_consume.BUDGET_INSTRUMENT),
("<KNOWN_POSITIVE_CASE>", okf_consume.KNOWN_POSITIVE_CASE),
("<KNOWN_POSITIVE_EXPECTED>", str(okf_consume.KNOWN_POSITIVE_EXPECTED)),
("<BUNDLE_ROOT>", GENERIC_BUNDLE),
("<PAYLOAD_PATH>", "/tmp/payload.json"),
("<SKILL_PATH>", "this file"),
("<REF>", "the card's `ref`"),
("<OUT>", "the path the caller named"),
):
text = text.replace(old, new)
left = sorted(set(_PLACEHOLDER.findall(text)))
if left:
raise SkillError(
f"the generic skill still carries a per-corpus hole: {', '.join(left)}. A hole "
"left in a generic document is a number the reader is invited to invent",
code="placeholder_unfilled",
)
description = block_scalar(
"Answer one question about ANY OKF bundle from a bounded payload assembled "
"by a deterministic pre-pass, marking every claim with its source, its title "
"and its provenance locator. Carries no bundle's identity: read the bundle's "
f"own card with `{CARD_COMMAND}` first. Use when the user asks a question of, "
"or states a hypothesis about, a corpus held as an OKF bundle."
)
header = f"---\nname: {block_scalar(GENERIC_NAME)}\ndescription: {description}\n---\n"
return header + text
def generate_any(*, out: Path, force: bool = False) -> Path:
"""Write the generic skill -- what `okf skill` writes by default since
2026-09-20. Takes no bundle, by construction."""
target = out / "SKILL.md"
if target.exists() and not force:
raise SkillError(
f"{target} already exists; pass --force to replace it",
code="target_occupied",
)
out.mkdir(parents=True, exist_ok=True)
target.write_text(render_generic(), encoding="utf-8")
return target
#: The name this function carried until the generic form became the default.
#: Kept so a caller that named it does not break on a rename alone.
generate_generic = generate_any
def card_main(argv: list[str] | None = None) -> int:
"""`okf card <bundle>` -- the per-bundle half of a consumption skill, as JSON.
The generic skill above tells its reader to run this. It is DERIVED on every
run and never stored in the bundle: a stored card is one more artefact that
can disagree with the bytes beside it, which is the defect the generic skill
exists to remove.
"""
parser = argparse.ArgumentParser(
prog="okf card",
description=(
"Print one bundle's identity, concept count, conditional-field counts "
"and whole-bundle cost as JSON. Derived from the bundle on every run."
),
)
parser.add_argument("bundle", type=Path, help="the OKF bundle to describe")
args = parser.parse_args(argv)
from .mcp_server import card as build_card
try:
payload = build_card(args.bundle.resolve(), profile=okf_consume.DEFAULT_PROFILE)
except okf_consume.ConsumeError as exc:
print(f"refused ({exc.code}): {exc}", file=sys.stderr)
return 1
except OSError as exc:
print(f"the run did not happen: {exc}", file=sys.stderr)
return 2
print(json.dumps(payload, ensure_ascii=False, indent=2))
return 0
Reference in a new issue View git blame Copy permalink