open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

feat(skill): one generic skill by default, carrying a working method and an answer form

Browse source 
The operator built a 2313-concept bundle from one project's own documentation,
asked it a question in his own words, and judged the result unusable. The
generated skill was an audit contract: all its discipline sat on the accounting
-- markings, denominators, budget lines, source pointers -- and none of it on
understanding the question, searching again, or writing one coherent answer.
Two sentences actively forbade the second of those.

**The two forbidding sentences are gone and their replacements are tested from
both sides.** "Do not go looking for context the pre-pass deliberately
withheld" read as "one run per question", and no wording of the operator's
question put the right document inside a single run's cut -- so a rule against
a second run was a rule against finding it at all. "Not something to retry with
a narrower question" generalised a budget-refusal case into the same ban.
SS 2.2 of the contract said the first of them, so the contract moved with the
skill rather than being left to disagree with it: a second pre-pass run with
other terms, and a fetch of a concept the payload NAMED, are reachable; SS 9's
two real boundaries -- directory enumeration, the verdict layer -- are not.

**Two new sections, and the checker requires them.** `## Working method`: read
the bundle's map, put the question into the bundle's own words, split a broad
question into 2-4 sub-questions, search per sub-question, read what lay just
outside the cut and search again with its words, same method across several
bundles, then assemble ONE answer ordered by sub-question, saying which source
holds and what is not covered. `## Answer form`: the questioner's language,
plain prose, no `below_k`, no digests, no budget lines, no denominators; short
textbook-style references (document + section, plus bundle where several were
read); and the audit trail written only when the questioner asks for it or
into a document that travels without the skill. `REQUIRED_SECTIONS` follows the
template and the contract's new SS 2.5 and SS 2.6 -- never the other way round.

**The generic skill becomes what `okf skill` and `okf project` write.** A
per-bundle skill's numbers go stale the moment its bundle is rebuilt, one copy
per consuming project, and a project with two bundles installs two
near-identical skills; the generic form carries no bundle's numbers and names
`okf card` for them. `--for-bundle` is the opt-in for the instantiated copy,
which still refuses out loud on a stale pairing -- safe to keep, not enough to
keep default. `rule_bundle_identity` learned to tell a generic skill from an
unfilled template by the frontmatter name the generator writes, so the template
still fails for the opposite reason: it declares no identity because it is
unfinished.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-20 23:53:34 +02:00
commit 30edd3f5d8
12 changed files with 564 additions and 92 deletions
Download patch file Download diff file Expand all files Collapse all files

6
src/llm_ingestion_okf/consume.py
View file

@ -717,14 +717,14 @@ KNOWN_POSITIVE_CASE = "docs/consumption-contract.md, encoded as a JSON string"
#: `measure()`'s own answer for that file. Vacuous ALONE -- which is why the
#: delta below exists.
KNOWN_POSITIVE_EXPECTED = 16_738
KNOWN_POSITIVE_EXPECTED = 19_839
#: The second, independent route. `wc -c` reports 16 316 raw bytes for the same
#: The second, independent route. `wc -c` reports 19 360 raw bytes for the same
#: file; the difference is this file's JSON quoting and escaping overhead. A
#: reader can derive it without running `measure()` at all, and it moves the
#: moment `measure()` changes what it counts -- which is what stops
#: `expected == measured` from proving nothing.
KNOWN_POSITIVE_ENCODING_DELTA = 422
KNOWN_POSITIVE_ENCODING_DELTA = 479
#: The two places that file can be, resolved in this order.
#:

43
src/llm_ingestion_okf/contract_check.py
View file

@ -70,6 +70,26 @@ TRUST_TIERS = ("unverified", "machine-confirmed", "human-reviewed")
SKILL_IDENTITY = re.compile(r"for one bundle: `([^`<>]+)` at ref\s+`([^`<>]+)`")
#: The frontmatter `name` the GENERIC skill carries. A generic skill declares
#: no bundle identity BY CONSTRUCTION -- that is what makes it serve any bundle
#: and never go stale -- so `rule_bundle_identity` must be able to tell it from
#: the unfilled template, which declares none because it is unfinished. The
#: name is a structured declaration the generator writes, not a sentence of
#: prose that could be reached by paraphrase; `skill.GENERIC_NAME` is the one
#: authored copy and a test holds the two together.
GENERIC_SKILL_NAME = "okf-consume-any"
_FRONTMATTER_NAME = re.compile(r"^name:\s*(?:>-\s*\n\s+)?(\S+)\s*$", re.MULTILINE)
def skill_is_generic(skill_text: str) -> bool:
"""Whether this skill declares itself the one-to-many form."""
match = _FRONTMATTER_NAME.search(
skill_text.split("---\n", 2)[1] if "---\n" in skill_text else ""
)
return match is not None and match.group(1) == GENERIC_SKILL_NAME
def skill_identity(skill_text: str) -> tuple[str, str] | None:
"""The `(bundle_id, ref)` the skill declares, or `None` when it declares
none a reader could act on. `None` is a finding, never a silent pass: the
@ -86,6 +106,15 @@ def skill_identity(skill_text: str) -> tuple[str, str] | None:
REQUIRED_SECTIONS = (
"Pre-pass",
"Division of labour",
# Added 2026-09-20. Until then every heading here named a piece of
# BOOKKEEPING, and a skill could carry all seven while saying nothing
# about how to read a question, whether to search twice, or what the
# answer should look like -- which is the document the operator measured
# as unusable on a 2313-concept bundle. The rule follows the template, not
# the other way round: these two are required because the template now
# carries them, and a skill without them is thin in the way that mattered.
"Working method",
"Answer form",
"Markings",
"States",
"Budget",
@ -226,9 +255,19 @@ def rule_bundle_identity(ctx: Context) -> list[Finding]:
A payload that declares no identity at all is `rule_bundle_ref`'s defect,
not this one's: restating it would report one hole twice.
**The GENERIC skill declares no identity and that is not a hole.** It
carries no bundle's identity by construction -- which is precisely what
makes it serve any bundle and never go stale -- so the two clauses that
compare a skill against a payload do not apply to it, and it says which it
is in its frontmatter `name`. The unfilled template still fails, because it
declares none for the opposite reason: it is unfinished. The third clause,
an excerpt naming a bundle the payload does not, reads nothing from the
skill and runs either way.
"""
generic = skill_is_generic(ctx.skill)
declared = skill_identity(ctx.skill)
if declared is None:
if declared is None and not generic:
return [
Finding(
"bundle_mismatch",
@ -237,9 +276,9 @@ def rule_bundle_identity(ctx: Context) -> list[Finding]:
"is not an identity, and neither is its absence (SS 3.1, SS 3.3)",
)
]
skill_id, skill_ref = declared
bundle = _mapping(ctx.payload.get("bundle"))
payload_id, payload_ref = _text(bundle.get("bundle_id")), _text(bundle.get("ref"))
skill_id, skill_ref = declared if declared is not None else (payload_id, payload_ref)
disagreements = [
f"{key} (skill {mine!r}, payload {theirs!r})"
for key, mine, theirs in (

16
src/llm_ingestion_okf/project.py
View file

@ -56,6 +56,12 @@ CLI_ID = "okf project"
BUNDLE_DIR = ".okf"
SKILLS_DIR = Path(".claude") / "skills"
#: The skill directory, and it does NOT carry the bundle id. Claude Code takes
#: a project skill's command from its directory name, so one name is what lets
#: a second bundle in the same project reuse the skill instead of installing a
#: second one that says the same thing about a different bundle.
SKILL_NAME = "okf-consume-any"
#: What the bundle declares as its upstream version. A VALUE, and normally the
#: caller's (decision E1) -- but `okf project` has no catalog to ask, and a
#: required flag here would put the one-command form back behind a question
@ -208,8 +214,14 @@ def create(
f"{', '.join(report.unaccounted) or '(none named)'}",
code="conservation_failed",
)
skill_dir = out / SKILLS_DIR / f"{identity}-consume"
written = skill.generate(bundle, out=skill_dir, force=force)
# ONE skill, not one per bundle. A per-bundle skill carries the bundle's
# concept count, conditional-field counts and cost, so it goes stale the
# moment the bundle is rebuilt -- and refuses out loud when it was not
# regenerated. The generic one carries none of those numbers and tells its
# reader to run `okf card` for them, so a second project in the same
# directory, or a rebuild of this one, costs nothing.
skill_dir = out / SKILLS_DIR / SKILL_NAME
written = skill.generate_any(out=skill_dir, force=True)
concepts = len(consume.enumerate_concepts(bundle, profile=SEGMENTED_OKF_V0_2))
missing, whole = inventory(folder, bundle)
summary = summarise(folder, bundle, written, out, report, concepts, missing, whole)

86
src/llm_ingestion_okf/skill.py
View file

@ -21,14 +21,24 @@ skill's `bundle_id` at a foreign `ref`. The right pair is untouched at exit 0
with 0 findings.
**The argument for a generator never rested on conformance, and still does
not.** It was made on what the skill has to state:
§ 5's denominators, § 7.6's breaking point and § 6.4's conditional-field list
are all per-bundle numbers. A generic skill can either leave them as holes -- the
template's own definition of unfinished -- or carry another corpus's numbers,
which is worse, because a stated cost that is false for this bundle is a
measurement failure and not merely a gap. Instantiating is what makes them true.
And with several bundles connected at once, a generic skill has nothing to
select on: each generated skill carries the bundle's id in its own name.
not.** It was made on what the skill has to state: SS 5's denominators, the
payload-cost section and SS 6.4's conditional-field list are all per-bundle
numbers, and a generic skill can either leave them as holes -- the template's
own definition of unfinished -- or carry another corpus's numbers, which is
worse.
**AND IT LOST 2026-09-20, TO A THIRD OPTION AND AN OPERATOR'S MEASUREMENT.**
The third option is `okf card`: the per-bundle numbers are DERIVED from the
bundle in under a second, so the generic skill neither invents them nor states
another bundle's -- it names the command that produces them. The measurement
is that the per-bundle form's cost is not hypothetical: it goes stale the
moment its bundle is rebuilt, one copy per consuming project, and a project
holding two bundles installs two near-identical skills (measured: identical on
281 of 313 and 311 lines). So `okf skill --out <dir>` writes the GENERIC form,
`okf project` installs it under one name that carries no bundle id, and
`--for-bundle` is the opt-in for the instantiated copy. That copy still
refuses out loud (`bundle_mismatch`) rather than answering from stale numbers,
which is what makes it safe to keep and not enough to keep it default.
**Zero model calls, zero network, no clock.** The same bundle bytes produce the
same skill bytes.
@ -175,10 +185,13 @@ TEMPLATE_DENOMINATORS = """The payload reports three counts — `considered`, `w
`considered == withheld + delivered`. Carry them into your output."""
TEMPLATE_ENUMERATION = (
"- **No directory enumeration** unless `<PROFILE_NAME>` says the index is derived."
"- **No directory enumeration** unless `<PROFILE_NAME>` says the index is derived.\n"
" Searching again, with other words, is not enumeration: the pre-pass walks the\n"
" index and applies the same rules every time, and a second run is another\n"
" measurement, not a way around the first one."
)
TEMPLATE_OUTPUT = "Write to `<OUT>`. It must carry: the bundle ref; the findings, each with a"
TEMPLATE_OUTPUT = "Write to `<OUT>`. **The answer comes first and is written in the answer form**:"
#: Every per-corpus hole the template carries. A generic skill that left one
#: would be the unfilled template with better manners, so it is refused.
@ -526,7 +539,7 @@ def _rewrite(
(
TEMPLATE_OUTPUT,
"Write to the path the caller names, or to your answer if none was named.\n"
"It must carry: the bundle ref; the findings, each with a",
"**The answer comes first and is written in the answer form**:",
),
("`<CORPUS>` bundle", f"`{bundle_id}` bundle"),
("# <CORPUS> consumption", f"# {bundle_id} consumption"),
@ -535,6 +548,11 @@ def _rewrite(
("<BUDGET_INSTRUMENT>", okf_consume.BUDGET_INSTRUMENT),
("<KNOWN_POSITIVE_CASE>", okf_consume.KNOWN_POSITIVE_CASE),
("<KNOWN_POSITIVE_EXPECTED>", str(okf_consume.KNOWN_POSITIVE_EXPECTED)),
# The working method's own command block. STRICT like the rest: a
# per-bundle skill telling its reader to search again against
# `<BUNDLE_ROOT>` would be the unfilled template's hole inside the one
# section that asks for a second run.
("<BUNDLE_ROOT>", str(bundle_root)),
]
for old, new in replacements:
if old not in text:
@ -710,7 +728,7 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
"bundle",
type=Path,
nargs="?",
help="the OKF bundle to instantiate a skill for (unused with --generic)",
help="the OKF bundle to instantiate a skill for. Only read with --for-bundle",
)
parser.add_argument(
"--out", type=Path, required=True, help="the skill directory to write (SKILL.md inside)"
@ -727,10 +745,17 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser.add_argument(
"--generic",
action="store_true",
help="the default since 2026-09-20; accepted so existing call sites keep working",
)
parser.add_argument(
"--for-bundle",
action="store_true",
help=(
"write the one-to-many skill instead: one installable document for ANY "
"bundle, carrying no bundle's identity or numbers. `bundle` is then "
"unused, and the reader is told to run `okf card <bundle>` at run time"
"write the per-bundle form instead: one skill carrying THIS bundle's "
"identity, concept count, conditional-field counts and cost. It goes "
"stale the moment the bundle is rebuilt and refuses out loud "
"(`bundle_mismatch`) when it was not regenerated, which is why it is "
"no longer the default. Requires `bundle`"
),
)
return parser.parse_args(argv)
@ -739,15 +764,13 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
try:
if args.bundle is None and not args.generic:
print("refused (bundle_missing): name a bundle, or pass --generic", file=sys.stderr)
if args.for_bundle and args.bundle is None:
print("refused (bundle_missing): --for-bundle needs a bundle", file=sys.stderr)
return 2
written = (
generate_generic(out=args.out, force=args.force)
if args.generic
else generate(
args.bundle, out=args.out, question=args.example_question, force=args.force
)
generate(args.bundle, out=args.out, question=args.example_question, force=args.force)
if args.for_bundle
else generate_any(out=args.out, force=args.force)
)
except okf_consume.ConsumeError as exc:
print(f"refused ({exc.code}): {exc}")
@ -859,9 +882,10 @@ def render_generic() -> str:
TEMPLATE_SCALING,
"**Scaling.** Cost tracks the QUESTION, not the corpus: the payload is cut\n"
f"to {okf_consume.DEFAULT_LIMIT} {okf_consume.BUDGET_UNIT} whatever the bundle's size. What\n"
"does track the corpus is the bookkeeping — one `withheld` entry per\n"
"considered-and-not-delivered concept — so the point at which this strategy\n"
"stops fitting is a property of the bundle. Read `whole_bundle_bytes` from\n"
"does track the corpus is the wall clock: the pre-pass reads every concept\n"
"body on every run, with no precomputed index behind it. The bookkeeping\n"
"does not — `withheld` is counts plus a capped sample of names, so it is\n"
"bounded by that cap rather than by the bundle. Read `whole_bundle_bytes` from\n"
"the card and compare it with the budget: a bundle costing less than the\n"
"budget could have been handed over whole, and the pre-pass is then a\n"
"convenience rather than a necessity.",
@ -882,7 +906,7 @@ def render_generic() -> str:
(
TEMPLATE_OUTPUT,
"Write to the path the caller names, or to your answer if none was named.\n"
"It must carry: the bundle ref; the findings, each with a",
"**The answer comes first and is written in the answer form**:",
),
("`<CORPUS>` bundle", "bundle you were pointed at"),
("# <CORPUS> consumption", "# OKF bundle consumption"),
@ -932,8 +956,9 @@ def render_generic() -> str:
return header + text
def generate_generic(*, out: Path, force: bool = False) -> Path:
"""Write the generic skill. Takes no bundle, by construction."""
def generate_any(*, out: Path, force: bool = False) -> Path:
"""Write the generic skill -- what `okf skill` writes by default since
2026-09-20. Takes no bundle, by construction."""
target = out / "SKILL.md"
if target.exists() and not force:
raise SkillError(
@ -945,6 +970,11 @@ def generate_generic(*, out: Path, force: bool = False) -> Path:
return target
#: The name this function carried until the generic form became the default.
#: Kept so a caller that named it does not break on a rename alone.
generate_generic = generate_any
def card_main(argv: list[str] | None = None) -> int:
"""`okf card <bundle>` -- the per-bundle half of a consumption skill, as JSON.