docs(okf-v0.2): P1 — the read-only sweep, and the six findings it produced

17 bundles, 602 documents, five corpora: our goldens, portfolio-optimiser-claude,
catalog's green parity corpus, the wiki's bundle, and upstream's own four
reference bundles at the pinned commit. Read-only throughout; nothing was written
into a consumer tree and no consumer ran anything.

Every finding is on the JUDGE path, so none of them blocks v0.5.0a1, P2 or P3 —
both remaining pilot tests are emit-side. They are P4's work.

The two that matter most needed real data to surface:

- root_frontmatter conflates "permitted and ordered" with "required": 14 of 17
  bundles fail with exactly one violation each, including all four of upstream's
  own reference bundles, purely for exercising a MAY the emitter treats as
  optional. We emit a MAY correctly and grade it as a MUST.
- TypePolicy.allowed cannot judge a consumer that quotes its scalars: 0/526 wiki
  documents pass, 526/526 pass dequoted. The suite cannot see it because its
  tests hand-build the mapping and never parse a file.

Also confirmed against real third-party v0.2 data: V-A3 holds — no upstream
`generated` value reads as owned, so a real v0.2 bundle cannot talk the collision
gate into overwriting.

Test C: C-E1 differed (the quoting gap), C-E2/C-E3/C-E4 as expected. The wiki's
copy went out through coord per their condition, never as a file here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0186K8oLEnaFfXZ2dWGyHzrB
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 17:45:54 +02:00
commit a8ceea9667

View file

@ -448,6 +448,115 @@ deciding V1 — a participant, not a test site); `llm-ingestion-pipeline-securit
- **P4 — fix, then GA.** Address findings, then flip `OKF_LATEST` as its own
release.
### P1 results — measured 2026-07-31, before the tag
Read-only throughout: nothing was written into a consumer tree, and no consumer
ran anything. Corpora and the commits they were read at:
| Corpus | Bundles | Concept docs | Read at |
|---|---|---|---|
| ours (`examples/`) | 4 | 5 | `2504011` |
| `portfolio-optimiser-claude` | 4 | 11 | `8a14137` |
| `catalog` (`test/okf-parity-corpus/green-*`) | 4 | 6 | `3dfc513` |
| `claude-code-llm-wiki` (`bundle/`) | 1 | 526 | `c5141f8` |
| upstream reference bundles (`okf/bundles/`) | 4 | 53 + one `log.md` | `3fcbb9f` |
Upstream is **beyond the pilot set** and was added deliberately: it is the only
corpus authored by the version's own publisher, and it is what turned the first
finding from an opinion into a measurement. `node_modules` and `.obsidian` were
excluded from the listing, which is the caller's call to make — the library
enumerates nothing.
Every finding below is on the **judge** path. None of them touches what the
library emits, so none of them blocks `v0.5.0a1`, P2 or P3: both remaining pilot
tests are emit-side. They are recorded here against GA exit criterion 1 and are
P4's work, not this session's.
**P1-F1 — `root_frontmatter` conflates "permitted and ordered" with "required",
and the emitter and the judge disagree about the same key.** `IndexPolicy`
reports `index_root_key_missing` for every key in `root_frontmatter` absent from
the root index, so `OKF_V0_2` (which names `okf_version`) judges any bundle that
exercises upstream's MAY-by-omission as violating. Measured: **14 of the 17
bundles swept fail, each with exactly this one violation** — including all four
of upstream's own reference bundles and three of our four goldens. The three
that pass are the three carrying the key. Meanwhile D5's emitter treats it as
optional: omitting `root_frontmatter_values` writes no block. We emit a MAY
correctly and grade it as a MUST. `STRICT_V1` is unaffected in substance — the
wiki emits all three of its pinned keys (measured) — but it is the same field
carrying two meanings, so the fix has to separate them rather than relax them.
**P1-F2 — `TypePolicy.allowed` cannot be used against a consumer that quotes its
scalars.** All **526/526** wiki documents are rejected as `okf_type_not_allowed`
under `STRICT_V1`; **526/526** are clean when values are dequoted before judging.
The bundle quotes every value (5265 of 5267), and this library's line-oriented
parser has no scalar value model, so the quotes are part of the value. Blast
radius is exactly `TypePolicy.allowed`, because it is the only surface that
judges a value at all — `FrontmatterSchema` judges keys and value *shape*. The
suite cannot see this: its tests hand-build the mapping and never parse a file.
**P1-F3 — `FrontmatterSchema.nullable` is unreachable from a parsed document.**
Two wiki documents carry `summary: null`; the parser yields the string `"null"`,
which is a legal value, so `frontmatter_value_null` can never fire and a real
null is indistinguishable from the literal text. `STRICT_V1`'s
`nullable={"summary"}` is dead configuration on the read path. Same root cause
as F2, and worth stating separately because a fix for one does not imply the
other.
**P1-F4 — the line parser manufactures false top-level keys from real v0.2 block
YAML, and the count is large.** Over upstream's 54 documents it extracts 12
nesting-derived keys (`resource` 51, `by` 44, `at` 44, `id` 32, `author` 5,
`last_modified` 5, …) and 6 dash-prefixed artifacts (`- id` 22, `- title` 19,
`- resource` 16, `- { by` 8, `- { name` 2, `- term` 1), while the container keys
read as empty (`sources` 49/49, `generated` 44/53, `verified` 8/8). Consequence,
measured rather than predicted: `STRICT_V1`'s schema produces **444 spurious
findings** over those 54 documents (376 `frontmatter_key_not_allowed`, 68
`frontmatter_key_malformed`). This is U4 on the read path with a number attached,
and it is the case for D1b: **no closed-namespace profile should be pointed at a
real v0.2 bundle until the structured reader lands.**
**P1-F5 — the door-side type gate is not a read-path judgement, and its docstring
reads as though it were.** `TypePolicy.rejection` refuses the reserved `verdict`
layer and is called only from `manifest.py:202` and `inbox.py:105`/`:245` — write
paths both; Door C never calls it. Pointed at existing bundles it labels
legitimately promoted verdict files as violations, which is what happened on two
real po-claude bundles (one document each). Not a defect in the doors: a gap in
the read surface, which does not exist, plus a docstring that invites the
confusion by saying "which `okf_type` values a bundle admits".
**P1-F6 — v0.2 reserves two filenames and the profile models one.** §9/§12 make
`log.md` reserved and let it appear at any level; `IndexPolicy` names only
`index.md`, and no profile carries a reserved-file set. Measured on upstream's
`acme_retail`: it ships a root `log.md` (`type: Log`) and its root index does
**not** list it. So a caller building the listing an `entries_match_directory`
profile requires has to know to exclude reserved names, and this library tells
them only about `index.md`. Consistent with "the library enumerates nothing" —
but the caller is being asked to know something we hold and do not state.
**What the sweep confirmed, which matters as much as what it found:**
- **V-A3 holds against real third-party v0.2 data.** No `generated` value in
upstream's bundles reads as owned: 44 empty (block form), 9 someone else's
inline flow mapping, 1 absent. A real v0.2 bundle cannot talk the collision
gate into overwriting.
- **`OKF_V0_2.frontmatter` reports zero violations across all 602 documents in
all five corpora** — with F4's caveat that an open namespace is a weak test.
- **Test C — C-E1 differed, C-E2/C-E3/C-E4 as expected.** C-E1: 0/526 pass as
parsed, 526/526 pass dequoted, single cause, F2. C-E2: exactly two key-order
signatures across 526 documents (173× the eight required keys, 353× those plus
the three layer-specific ones), both prefixes of `_STRICT_V1_KEY_ORDER` in
order — no change required to their eight. C-E3: not one document carries
`generated`, `sources`, `verified`, `status` or `stale_after`, and `timestamp`
is on 526/526, so the §13.1 legacy path holds. C-E4: the root index carries
exactly its three pinned keys in the pinned order, nested indexes carry none,
and `entries_match_directory` is clean in both directions at `c5141f8`.
- **A count correction owed to the wiki.** Their pre-run ground truth was "529
`.md`, minus 4 `index.md`, minus 1 `README`/log = 524" at `b9b557b`. Measured
at that commit: **528** `.md` under `bundle/`, **3** files named exactly
`index.md`, 1 `README.md` — 524 is right, and both terms of the decomposition
are off by one in a way that cancels. The fourth "index" is a concept whose
filename ends in `-index.md`. At `c5141f8`, the same arithmetic gives **526**,
which is what the sweep judged.
### Pilot responses — all three accepted, and the asking paid for itself
All three replied on 2026-07-26, before the tag exists. That timing is the whole
@ -1130,6 +1239,10 @@ everything after. **Done 2026-07-26** (`1215f98`, `7bc366b`).
before any disk mutation.
7. **P1** — read-only sweep over real pilot bundles. Cheap, zero-risk, and the
first point where a finding can come from outside our own assumptions.
**Done 2026-07-31**: 17 bundles, 602 documents, five corpora (the pilot set
plus upstream's own reference bundles), six findings — all on the judge path,
none blocking the tag. Results and the Test C verdict are under "P1 results"
above; the wiki's copy went out through coord.
8. **Pre-release tag `v0.5.0a1`**, then **P2** (producer) and **P3** (gate).
Steps 18 are the whole pilot scope.
9. **P4** — address findings.