open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases Activity

docs(okf-v0.2): V-A8 executed green, and the two premises it falsified

Browse source 
V-A8 was scheduled to be WEAKENED to parse-and-render on the standing claim
that "there is no validator in okf/". Enumerating the repository tree instead
of that one subdirectory falsified it: OKFDocument.validate() sits at
okf/src/reference_agent/bundle/document.py:58 inside a working v0.2 reader.

So V-A8 ran as written, and passed 13/13 against
examples/ingest-golden-okf-v0-2/ at pinned 3fcbb9f. It needs nothing installed
-- document.py imports only yaml -- so .venv and the one-runtime-dependency
rule are untouched. The load-bearing assertion is not validate() (which checks
a single key, `type`); it is that a REAL yaml parser recovers our inline flow
forms as structures: `generated` as a mapping, `sources` as a list of mappings.
Our own parser is line-oriented and reads both as opaque strings, so no test of
ours could ever have answered this.

Second falsified premise, in the A-E6 rationale itself: "yaml.safe_load returns
"0.2" whether or not it was quoted". Measured against PyYAML 6.0.3 (the version
upstream requires), unquoted loads as float 0.2 and quoted as str '0.2'. The
BOM half of the same sentence is true. Correcting it surfaces what it hid --
unquoted `0.10` loads as `0.1`, indistinguishable from v0.1, and the type is
not stable across version shapes (`0.2` float, `0.2.1` str). Upstream's only
written instance, SPEC.md:773, is quoted.

That changes nothing today and D5 is NOT requoted: at 0.2 both forms are
unambiguous, and neither consumer reading the key parses YAML. It is recorded
because okf_version's value belongs to catalog (E1), so it is a constraint we
owe them, not a choice we may make for them.

Runbook: Step 2 now enumerates the REPOSITORY root, not okf/ -- the same
mistake this step already warned about, repeated one level up (toolbox/ and
samples/ sit outside okf/; 265 tracked files, 48 py + 43 ts, none of it on any
list). New Step 3a carries the V-A8 procedure per upstream release, including
that it can never be a pytest test and why.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012gwLPe5TY5aN3o3tejv9Nh
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 21:12:52 +02:00
commit 19b900b4f8
2 changed files with 152 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

89
docs/plan/okf-v0.2-alignment.md
View file

@ -949,9 +949,50 @@ output is the baseline. Materialize it twice with the *same* explicit
marker invisible to their gate *while still exiting 0* — a failure that
reports success.
Both are asserted on **raw bytes**, never on a parsed value: `yaml.safe_load`
returns `"0.2"` whether or not it was quoted and strips a BOM before any caller
sees it, so a parsed assertion masks precisely these two defects.
Both are asserted on **raw bytes**, never on a parsed value. **The reason given
here was half wrong, measured 2026-07-31 under V-A8, and the correction matters
more than the sentence did.**
*What it said:* "`yaml.safe_load` returns `"0.2"` whether or not it was quoted
and strips a BOM before any caller sees it, so a parsed assertion masks precisely
these two defects."
*Measured against PyYAML 6.0.3 — the version upstream's own `pyproject.toml`
requires:*
| Input | `safe_load` returns | Type |
|---|---|---|
| `okf_version: 0.2` | `0.2` | **float** |
| `okf_version: "0.2"` | `'0.2'` | str |
| BOM + `okf_version: 0.2` | `{'okf_version': 0.2}` | BOM stripped, key clean |
So the **BOM half is true** — a parsed assertion genuinely cannot see a BOM, and
that is the whole reason the BOM expectation is a byte assertion. The **quoting
half is false**: the two forms are not merely distinguishable after parsing, they
come back as *different types*. Raw-byte assertion remains the right call for
quoting too — the two consumers that actually read this key parse line-oriented,
so bytes are what they see — but it is right for that reason, not because parsing
cannot tell.
**The correction carries a finding the original claim was hiding**, and it is
about the value space rather than about this fixture:
- Unquoted, `okf_version: 0.10` loads as `0.1` — **indistinguishable from v0.1**
to any YAML-parsing consumer. Quoted, `'0.10' != '0.1'`.
- Unquoted, the *type* is not even stable across version shapes: `0.2` is a
float, `0.2.1` is a str.
- Upstream's only written instance of the key (`SPEC.md:773` @ `3fcbb9f`) is
**quoted**: `okf_version: "0.2"`. None of its four shipped bundles emit the key
at all, so the spec line is the entire prior art.
**This changes nothing today and is not a reason to requote D5.** At `0.2` both
forms are unambiguous, and neither consumer that reads the key parses YAML:
commons is line-oriented (`method-spec.md:90`) and catalog's gate unquotes before
a text-shape regex (`1ca27f6`). The exposure is latent and bounded — it opens at a
two-digit minor or a three-segment version. It is recorded here because
`okf_version`'s **value** belongs to catalog (decision E1), which makes this a
constraint we owe them before such a version exists rather than a choice we may
make for them.
**A green TEST B does not confirm the placement, and a red one does not
identify it.** The reader is placement-blind: green measures the value, never
@ -1344,11 +1385,48 @@ everything after. **Done 2026-07-26** (`1215f98`, `7bc366b`).
| V-A5 | No profile hard-codes an upstream version | Step 3 |
| V-A6 | Adding v0.2 support is behavior-neutral for v0.1 profiles | Golden suite byte-for-byte under `DEFAULT` **and** `STRICT_V1`; existing tests unmodified and green (C1 extended) |
| V-A7 | No profile can emit `timestamp` together with a malformed `generated` | Named construction-time test, same shape as C3's verdict reservation |
| V-A8 | A v0.2 bundle we emit is accepted by an independent v0.2 consumer | Validate the D5 fixture against upstream's reference implementation, not only against our own reader |
| V-A8 | A v0.2 bundle we emit is accepted by an independent v0.2 consumer | Validate the D5 fixture against upstream's reference implementation, not only against our own reader. **DONE 2026-07-31 @ `3fcbb9f`: 13/13.** Procedure is runbook Step 3a |
V-A8 is the one that keeps this honest. Every other test asks whether we agree
with ourselves.
**Executed 2026-07-31, and it was nearly weakened on a false premise.** The
standing note said V-A8 had to be re-scoped to parse-and-render because "there is
no validator in `okf/`". Enumerating the tree instead of the subdirectory showed
`OKFDocument.validate()` at `okf/src/reference_agent/bundle/document.py:58`, inside
a working v0.2 reader. The claim was a negative derived from a partial enumeration,
which is the failure mode runbook Step 2 now names explicitly.
The test ran unweakened. `document.py` imports only `yaml`, so upstream's reader
runs standalone against the pinned clone with nothing installed and `.venv`
untouched. **13/13 green** over `examples/ingest-golden-okf-v0-2/`:
- Both files parse under a real `yaml.safe_load`.
- `generated` arrives as a **mapping** — `{'by': 'process:okf-ingest', 'at':
datetime(2026, 7, 16, 12, 0, tzinfo=utc)}` — and `sources` as a **list of
mappings** carrying `id` and `resource`. This is the assertion no test of ours
could make: our parser is line-oriented and reads both as opaque strings.
- `validate()` passes, `trust_tier` → `unverified`, `normalize_verified` → `[]`,
`is_stale` → `False`, and `serialize()` → `parse()` preserves frontmatter
semantics and body bytes.
Two measured facts worth carrying, neither of them failures:
1. **`generated.at` type-coerces to `datetime`** under a real YAML parser while our
parser keeps the string. Same one-way asymmetry po-claude reported from the
other side on 2026-07-31 (their line-oriented parser sees the whole mapping as
one opaque string). Neither is wrong; ownership recognition
(`OwnershipPolicy.owns`) is a string prefix test and is unaffected by either.
2. **`serialize()` reflows our inline flow forms to block form.** An upstream
round-trip therefore yields bytes our own parser cannot read — the exact
pollution `test_a_block_list_pollutes_the_scalar_parsers_key_space`
characterizes. Expected, one-directional, and not a reason to emit block form.
**V-A8 cannot become a pytest test**, and that is not a gap: it needs PyYAML and
upstream's source, and this package has exactly one runtime dependency with a
packaging test enforcing it. It is a per-release runbook procedure (Step 3a), which
is also where an upstream reader change would be caught.
## Non-goals
- Implementing attestation execution — executors, attesters, receipts, verdicts
@ -1382,6 +1460,9 @@ with ourselves.
additions only).
4. V-A7 and V-A5 named tests present and failing-by-construction if removed.
5. V-A8: the v0.2 fixture validates under upstream's reference implementation.
**Done 2026-07-31 @ `3fcbb9f`, 13/13.** Re-run per upstream release via runbook
Step 3a — it is a procedure, not a pytest test, because it needs PyYAML and
upstream's source and this package has exactly one runtime dependency.
6. Boundary grep-gate still empty (`sanitize|quarantine|lexicon` absent outside
guard imports).
7. D6: release checklist contains the upstream-version re-check item, and it

71
docs/upstream-okf-upgrade-runbook.md
View file

@ -57,14 +57,35 @@ which felt like two confirmations and was one unstable reference read twice. Wor
"the current spec text" were different objects, and either alone would have been a
partial answer.
## Step 2 — Enumerate the whole `okf/` tree, not just `SPEC.md`
## Step 2 — Enumerate the whole REPOSITORY tree, not just `okf/`, not just `SPEC.md`
List every file and directory at the pinned commit before deciding what to read.
Start at the repository root:
*Prevents:* assuming the directory you know about is the one that matters. In the
v0.2 round `okf/` held `SPEC.md`, `README.md`, `pyproject.toml`, `src/`, `tests/`,
`samples/` **and** `bundles/`. The last one held the actual v0.2 example bundles and
was not on anyone's list until the tree was enumerated.
git ls-files | awk -F/ '{print $1}' | sort -u # top-level entries
git ls-files | sed 's/.*\.//' | sort | uniq -c # what kinds of file exist
*Prevents:* assuming the directory you know about is the one that matters. This
step has now failed twice at two different scopes, which is why its title names
the root rather than a subdirectory:
- **v0.2 round, inside `okf/`.** `okf/` held `SPEC.md`, `README.md`,
`pyproject.toml`, `src/`, `tests/`, `samples/` **and** `bundles/`. The last one
held the actual v0.2 example bundles and was not on anyone's list until the tree
was enumerated.
- **2026-07-31, one level up — the same mistake against `okf/` itself.** Scoping
enumeration to `okf/` (as this step previously instructed) hides that the
repository root also carries `toolbox/` and `samples/`. Measured at `3fcbb9f`:
265 tracked files, of which 48 are Python and 43 TypeScript, with `toolbox/`
holding two complete tools (`mdcode`, `enrichment`) plus a second copy of an OKF
bundle under `toolbox/mdcode/demo/okf/catalog/`. None of it was on any list.
The cost of getting this wrong is not only unread files: a **negative** claim
derived from a partial enumeration reads exactly like a measured one. "There is no
validator in `okf/`" was recorded as fact and used to plan a weakening of V-A8;
`okf/src/reference_agent/bundle/document.py:58` defines `validate()`, and the same
module is a working v0.2 reader. Enumerate before concluding that something is
absent — see Step 3a.
## Step 3 — Read the shipped examples, not only the normative text
@ -94,6 +115,46 @@ What reading the four v0.2 bundles produced, none of it derivable from `SPEC.md`
| Real frontmatter is multi-line block YAML: block lists of multi-key mappings, nested mappings, flow sequences, booleans, dates | Re-sized the emitter work. It had been scoped against a list of strings |
| §7's canonical tool actor is `<producer>/<version>`, and upstream uses it | Counter-evidence to a recommendation we had already sent to another repo |
## Step 3a — Run upstream's own reader against our fixture (V-A8)
Every other test in the suite asks whether we agree with ourselves. This one asks
an independent implementation, and it is cheap enough that there is no excuse for
skipping it.
**It is not a dependency.** `okf/src/reference_agent/bundle/document.py` imports
only `yaml`, so it runs standalone under system Python against the pinned clone —
nothing is installed, `.venv` is never touched (the one-runtime-dependency rule and
its packaging test both stay intact), and nothing is written.
import sys; sys.path.insert(0, "<clone>/okf/src")
from reference_agent.bundle.document import OKFDocument, trust_tier, is_stale
Assert, over each file of the current version's golden bundle:
1. `OKFDocument.parse()` accepts our bytes. **This is the load-bearing one.** Our
own parser is line-oriented and reads inline flow forms as opaque strings, so it
structurally cannot tell us whether a real YAML consumer recovers them as
structures. Only an outside parser can.
2. The values arrive as the *shapes* the profile intends — `generated` as a
mapping, `sources` as a list of mappings — not merely as something that parsed.
3. `validate()` passes. Note what this does and does not buy: at `3fcbb9f`
`REQUIRED_FRONTMATTER_KEYS = ("type",)`, so it checks one key. Treating a green
`validate()` as "upstream accepts our bundle" would overclaim badly; the parse in
(1) and the shape assertions in (2) are where the signal is.
4. The semantic readers upstream ships run over our frontmatter without raising —
at `3fcbb9f`: `trust_tier` (§5.3), `normalize_verified` (§5.2), `is_stale` (§5.5).
**Record the type each value arrives as, not just that it parsed.** A real YAML
parser coerces, and the coercions are load-bearing facts about the value space —
this is where the v0.2 round's `okf_version` float finding came from (see the
alignment plan's A-E6). Upstream's `serialize()` also reflows inline flow forms to
block form, so an upstream round-trip produces bytes our own line-oriented parser
cannot read. That is expected and one-directional; do not "fix" it by emitting
block form.
*Result, v0.2 round at `3fcbb9f`:* 13/13 green against
`examples/ingest-golden-okf-v0-2/`.
## Step 4 — Produce the diff, classified
From the spec's own "Changes from vN" section **plus** the example evidence,