feat(check): a skill and a payload naming different bundles is a finding
`okf check` had fifteen rules and none asked whether the skill and the payload
were talking about the same bundle. Reproduced on this HEAD before any code
moved: three pairs reported `conformant: 15 rules over 8 excerpts and 438
withheld entries, 0 findings` -- a skill generated from one corpus against
another corpus's payload, the unfilled template against that payload, and a
payload sharing the skill's `bundle_id` at a foreign `ref`. All three now exit 1
with one `bundle_mismatch` finding over 16 rules.
BOTH halves are compared and the `ref` half is load-bearing: three distinct
builds on this machine carry one `bundle_id`, so an id comparison would pass a
stale skill. SS 3.3: "a version is the producer's assertion; a ref is a fact
about bytes". An identity the rule cannot read is a finding, never a silent
pass -- that is what refuses the unfilled template.
No new field: the identity was already in the generated skill's prose, now
factored into `skill.identity_line` and read back by
`contract_check.skill_identity`. Generated skill bytes unchanged, measured on
both tracked bundles on one interpreter.
The rule's first real find is this repository's own hand-made
`skills/okf-consume/SKILL.md`, which predates `okf skill` and declares no
identity a reader can act on: 1 of 1. Nine tests that asserted the old, false
conformance now pair a skill with its own bundle's payload.
Measured, nothing else moved: `~/okf-test/dokumenter` `diff -r` empty old
source vs new on one interpreter (52 files, 26 concepts), `okf project` still
byte-equal to `okf build`, K2 pin unmodified and green (453 concepts, ranks
1,1,1,1,1,5), known-negative `{}` unchanged at 9 findings.
Report: docs/2026-09-10-k3-runde15-bundle-mismatch.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
b5df3355c5
commit
7cca9e079e
9 changed files with 468 additions and 56 deletions
44
CLAUDE.md
44
CLAUDE.md
|
|
@ -687,13 +687,49 @@ and fixtures, never code.
|
|||
names on PATH. The template and `docs/consumption-contract.md` (the § 7.4
|
||||
known-positive) are force-included into the wheel from the file they are
|
||||
authored in — one authored copy, no committed duplicate.
|
||||
The form was chosen on a measurement: the contract checker passes the
|
||||
UNFILLED template and passes a skill built for another bundle, so it cannot
|
||||
The form was chosen on a measurement: the contract checker passed the
|
||||
UNFILLED template and passed a skill built for another bundle, so it could not
|
||||
tell the two apart — the choice rests on § 5/§ 6.4/§ 7.6 being per-bundle
|
||||
numbers a generic skill can only leave as holes or state falsely.
|
||||
numbers a generic skill can only leave as holes or state falsely, and that
|
||||
argument NEVER rested on conformance, so closing the measurement leaves it
|
||||
standing. **That half is CLOSED 2026-09-10 by `bundle_mismatch`, the checker's
|
||||
SIXTEENTH rule** (`RULES` is a tuple and `Report.rules_evaluated` is
|
||||
`len(RULES)`, so every published «15 rules» line is now «16 rules» — a
|
||||
contract change for anyone quoting it). It compares the identity a skill
|
||||
DECLARES against the identity its payload declares, **both `bundle_id` AND
|
||||
`ref`**: three distinct builds on this machine carry the one `bundle_id`
|
||||
`k2-trinn1-20260903` at three refs, so an id-only rule would pass a stale
|
||||
skill, which is the case the generated skill warns about in its own words.
|
||||
SS 3.3 is the ground — «a version is the producer's assertion; a ref is a fact
|
||||
about bytes» — and SS 3.1 for the same question one level down, an excerpt
|
||||
naming a bundle its payload does not. Reproduced on `113b3f8` before any code
|
||||
moved, three pairs at `conformant: 15 rules over 8 excerpts and 438 withheld
|
||||
entries, 0 findings` and rc 0; after, all three at rc 1 with **one** finding
|
||||
over 16 rules — a foreign corpus's payload, the unfilled template, and (the
|
||||
arm that separates a whole rule from half of one) a payload sharing the id at
|
||||
a foreign ref. **An identity the rule cannot READ is a finding, never a silent
|
||||
pass**, or the template passes again. The right pair is untouched at rc 0 / 0
|
||||
findings, and `{}` is unchanged at **9** findings because a payload declaring
|
||||
no identity stays `ref_missing`'s defect — no rule restates another. NO new
|
||||
field was needed: the identity already lived in the generated skill's prose,
|
||||
now one authored copy in `skill.identity_line` read back by
|
||||
`contract_check.skill_identity`, generated skill bytes byte-identical before
|
||||
and after on both tracked bundles. **The rule compares DECLARED against
|
||||
DECLARED and never opens the bundle**, so a payload lying about its own `ref`
|
||||
still passes — that is `okf consume --ref`'s job and needs a bundle path this
|
||||
command deliberately does not take; «closed» means the three measured forms
|
||||
now fell, not that no fourth exists. Report:
|
||||
`docs/2026-09-10-k3-runde15-bundle-mismatch.md`.
|
||||
The first instantiated consumption skill is `skills/okf-consume/`; the
|
||||
measurement behind it, including the control that FAILED, is
|
||||
`docs/2026-09-07-okf-konsumskill-maaling.md`. **The ranking is this
|
||||
`docs/2026-09-07-okf-konsumskill-maaling.md`. **That skill is the new rule's
|
||||
first REAL find**: hand-filled before `okf skill` existed, it declares no
|
||||
bundle identity a reader can act on, so `okf check` now refuses it against its
|
||||
own shipped example payload (rc 1, 1 finding) — **1 of 1** shipped hand-made
|
||||
instantiated skill. Behind it sits a second fact: its prose states K2 numbers
|
||||
while the payload beside it is regenerated from the tracked golden bundle,
|
||||
because K2 cannot ship. What that artefact should BE is an operator question,
|
||||
not a patch. **The ranking is this
|
||||
repository's own choice** — the contract binds a payload, not a retrieval
|
||||
algorithm (§ 10) — and it has FOUR optional widenings. Three are **off by
|
||||
default** and keep the default payload byte-identical; the fourth
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue