feat(check): a skill and a payload naming different bundles is a finding
`okf check` had fifteen rules and none asked whether the skill and the payload
were talking about the same bundle. Reproduced on this HEAD before any code
moved: three pairs reported `conformant: 15 rules over 8 excerpts and 438
withheld entries, 0 findings` -- a skill generated from one corpus against
another corpus's payload, the unfilled template against that payload, and a
payload sharing the skill's `bundle_id` at a foreign `ref`. All three now exit 1
with one `bundle_mismatch` finding over 16 rules.
BOTH halves are compared and the `ref` half is load-bearing: three distinct
builds on this machine carry one `bundle_id`, so an id comparison would pass a
stale skill. SS 3.3: "a version is the producer's assertion; a ref is a fact
about bytes". An identity the rule cannot read is a finding, never a silent
pass -- that is what refuses the unfilled template.
No new field: the identity was already in the generated skill's prose, now
factored into `skill.identity_line` and read back by
`contract_check.skill_identity`. Generated skill bytes unchanged, measured on
both tracked bundles on one interpreter.
The rule's first real find is this repository's own hand-made
`skills/okf-consume/SKILL.md`, which predates `okf skill` and declares no
identity a reader can act on: 1 of 1. Nine tests that asserted the old, false
conformance now pair a skill with its own bundle's payload.
Measured, nothing else moved: `~/okf-test/dokumenter` `diff -r` empty old
source vs new on one interpreter (52 files, 26 concepts), `okf project` still
byte-equal to `okf build`, K2 pin unmodified and green (453 concepts, ranks
1,1,1,1,1,5), known-negative `{}` unchanged at 9 findings.
Report: docs/2026-09-10-k3-runde15-bundle-mismatch.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
b5df3355c5
commit
7cca9e079e
9 changed files with 468 additions and 56 deletions
|
|
@ -6,11 +6,20 @@ Filling it by hand is what produced `skills/okf-consume/` for one corpus. This
|
|||
command does the same thing for any bundle, from values it measures rather than
|
||||
values someone remembered.
|
||||
|
||||
**Why a generator rather than one generic skill.** Measured 2026-09-08: the
|
||||
contract checker passes the UNFILLED template against a real payload (exit 0, 15
|
||||
rules, 0 findings), and passes a skill built for a different bundle against this
|
||||
one's payload. So the checker cannot tell the two forms apart, and the choice
|
||||
could not be made on conformance. It was made on what the skill has to state:
|
||||
**Why a generator rather than one generic skill.** The measurement this
|
||||
paragraph used to rest on is CLOSED 2026-09-10. It read: the contract checker
|
||||
passes the UNFILLED template against a real payload (exit 0, 15 rules, 0
|
||||
findings), and passes a skill built for a different bundle against this one's
|
||||
payload. `contract_check.rule_bundle_identity` now compares the identity a
|
||||
skill declares with the identity its payload declares, so all three measured
|
||||
pairs are refused at exit 1 with one `bundle_mismatch` finding over 16 rules:
|
||||
a skill against another bundle's payload, the unfilled template against a real
|
||||
payload, and -- the arm an id comparison would miss -- a payload sharing the
|
||||
skill's `bundle_id` at a foreign `ref`. The right pair is untouched at exit 0
|
||||
with 0 findings.
|
||||
|
||||
**The argument for a generator never rested on conformance, and still does
|
||||
not.** It was made on what the skill has to state:
|
||||
§ 5's denominators, § 7.6's breaking point and § 6.4's conditional-field list
|
||||
are all per-bundle numbers. A generic skill can either leave them as holes -- the
|
||||
template's own definition of unfinished -- or carry another corpus's numbers,
|
||||
|
|
@ -383,6 +392,18 @@ def render(
|
|||
return header + text, payload
|
||||
|
||||
|
||||
def identity_line(bundle_id: str, ref: str) -> str:
|
||||
"""The one sentence that says which bundle a generated skill belongs to.
|
||||
|
||||
Authored here because the generator writes it, and read back by
|
||||
`contract_check.skill_identity`, whose `bundle_mismatch` rule is the reason
|
||||
it has to be findable rather than merely present. Two copies of this
|
||||
sentence would drift, and the copy nobody reads is the one that goes wrong,
|
||||
so the coupling has its own test.
|
||||
"""
|
||||
return f"generated by `okf skill` for one bundle: `{bundle_id}` at ref\n`{ref}`"
|
||||
|
||||
|
||||
def _description(bundle_id: str, total: int, ref: str) -> str:
|
||||
return (
|
||||
f"Answer one question about the OKF bundle `{bundle_id}` ({total} concepts, "
|
||||
|
|
@ -417,9 +438,9 @@ def _rewrite(
|
|||
(
|
||||
TEMPLATE_HEADER,
|
||||
"**This file is an instantiated copy of "
|
||||
"`skills/okf-consume-template/SKILL.md`,** generated by "
|
||||
f"`okf skill` for one bundle: `{bundle_id}` at ref\n"
|
||||
f"`{ref}`. Every value below was measured against those bytes. If the\n"
|
||||
"`skills/okf-consume-template/SKILL.md`,** "
|
||||
f"{identity_line(bundle_id, ref)}. Every value below was measured "
|
||||
"against those bytes. If the\n"
|
||||
"bundle moves, the ref moves with it and this file is stale — regenerate\n"
|
||||
"it rather than editing a number here. The section headings are fixed:\n"
|
||||
"the contract checker reads them by name.",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue