open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(check): a skill and a payload naming different bundles is a finding

Browse source 
`okf check` had fifteen rules and none asked whether the skill and the payload
were talking about the same bundle. Reproduced on this HEAD before any code
moved: three pairs reported `conformant: 15 rules over 8 excerpts and 438
withheld entries, 0 findings` -- a skill generated from one corpus against
another corpus's payload, the unfilled template against that payload, and a
payload sharing the skill's `bundle_id` at a foreign `ref`. All three now exit 1
with one `bundle_mismatch` finding over 16 rules.

BOTH halves are compared and the `ref` half is load-bearing: three distinct
builds on this machine carry one `bundle_id`, so an id comparison would pass a
stale skill. SS 3.3: "a version is the producer's assertion; a ref is a fact
about bytes". An identity the rule cannot read is a finding, never a silent
pass -- that is what refuses the unfilled template.

No new field: the identity was already in the generated skill's prose, now
factored into `skill.identity_line` and read back by
`contract_check.skill_identity`. Generated skill bytes unchanged, measured on
both tracked bundles on one interpreter.

The rule's first real find is this repository's own hand-made
`skills/okf-consume/SKILL.md`, which predates `okf skill` and declares no
identity a reader can act on: 1 of 1. Nine tests that asserted the old, false
conformance now pair a skill with its own bundle's payload.

Measured, nothing else moved: `~/okf-test/dokumenter` `diff -r` empty old
source vs new on one interpreter (52 files, 26 concepts), `okf project` still
byte-equal to `okf build`, K2 pin unmodified and green (453 concepts, ranks
1,1,1,1,1,5), known-negative `{}` unchanged at 9 findings.

Report: docs/2026-09-10-k3-runde15-bundle-mismatch.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-10 23:42:06 +02:00
commit 7cca9e079e
9 changed files with 468 additions and 56 deletions
Download patch file Download diff file Expand all files Collapse all files

42
CHANGELOG.md
View file

@ -7,6 +7,48 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **`okf check` refuses a skill and a payload that name different bundles
(`bundle_mismatch`).** The checker had published this hole about itself since
2026-09-08 and not closed it: reproduced on this repository's HEAD, it
reported `conformant: 15 rules over 8 excerpts and 438 withheld entries, 0
findings` for a skill generated from one corpus against a payload assembled
from another; the same line for the UNFILLED template against that payload;
and the same line again for a payload sharing the skill's `bundle_id` at a
foreign `ref`. All three now exit **1** with one finding. The right pair is
untouched at exit 0 with 0 findings, and a payload declaring no identity at
all stays `ref_missing`'s defect at 9 findings -- no rule restates another.
- **BOTH halves are compared, and the `ref` half is the load-bearing one.**
Three distinct builds on one machine were measured carrying the same
`bundle_id`, so an id comparison alone would pass a stale skill -- the case
the generated skill warns about in its own words ("if the bundle moves, the
ref moves with it and this file is stale"). SS 3.3: "a version is the
producer's assertion; a ref is a fact about bytes".
- **An identity the rule cannot read is a finding, never a silent pass.**
That is what refuses the unfilled template, whose `<CORPUS>` and `<REF>`
are not an identity. It also refuses the repository's own hand-made
`skills/okf-consume/SKILL.md`, which predates `okf skill` and declares no
bundle identity a reader can act on -- **1 of 1** shipped hand-made skill,
a real find and not a fixture.
- **The rule compares a DECLARED identity against a DECLARED identity and
never opens the bundle**, so a payload misreporting its own `ref` still
passes. Proving a ref against bytes is `okf consume --ref`'s job and needs
a bundle path this command deliberately does not take.
### Changed
- **The checker's rule count is 16, not 15**, and `Report.rules_evaluated` is
the denominator every report line quotes -- so `15 rules` becomes `16 rules`
in every published line. A consumer citing the old number is citing a number
that has changed. No payload bytes move: this is the checker, not the
pre-pass.
- `skill.identity_line` is now the single authored copy of the sentence a
generated skill declares its bundle in, read back by
`contract_check.skill_identity` and held to it by a test. Generated skill
bytes are unchanged -- measured, both tracked bundles byte-identical before
and after on the same interpreter.
## [0.8.1] — 2026-09-10
### Added