docs(okf-v0.2): P1 — the read-only sweep, and the six findings it produced
17 bundles, 602 documents, five corpora: our goldens, portfolio-optimiser-claude, catalog's green parity corpus, the wiki's bundle, and upstream's own four reference bundles at the pinned commit. Read-only throughout; nothing was written into a consumer tree and no consumer ran anything. Every finding is on the JUDGE path, so none of them blocks v0.5.0a1, P2 or P3 — both remaining pilot tests are emit-side. They are P4's work. The two that matter most needed real data to surface: - root_frontmatter conflates "permitted and ordered" with "required": 14 of 17 bundles fail with exactly one violation each, including all four of upstream's own reference bundles, purely for exercising a MAY the emitter treats as optional. We emit a MAY correctly and grade it as a MUST. - TypePolicy.allowed cannot judge a consumer that quotes its scalars: 0/526 wiki documents pass, 526/526 pass dequoted. The suite cannot see it because its tests hand-build the mapping and never parse a file. Also confirmed against real third-party v0.2 data: V-A3 holds — no upstream `generated` value reads as owned, so a real v0.2 bundle cannot talk the collision gate into overwriting. Test C: C-E1 differed (the quoting gap), C-E2/C-E3/C-E4 as expected. The wiki's copy went out through coord per their condition, never as a file here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0186K8oLEnaFfXZ2dWGyHzrB
This commit is contained in:
parent
2504011010
commit
a8ceea9667
1 changed files with 113 additions and 0 deletions
|
|
@ -448,6 +448,115 @@ deciding V1 — a participant, not a test site); `llm-ingestion-pipeline-securit
|
|||
- **P4 — fix, then GA.** Address findings, then flip `OKF_LATEST` as its own
|
||||
release.
|
||||
|
||||
### P1 results — measured 2026-07-31, before the tag
|
||||
|
||||
Read-only throughout: nothing was written into a consumer tree, and no consumer
|
||||
ran anything. Corpora and the commits they were read at:
|
||||
|
||||
| Corpus | Bundles | Concept docs | Read at |
|
||||
|---|---|---|---|
|
||||
| ours (`examples/`) | 4 | 5 | `2504011` |
|
||||
| `portfolio-optimiser-claude` | 4 | 11 | `8a14137` |
|
||||
| `catalog` (`test/okf-parity-corpus/green-*`) | 4 | 6 | `3dfc513` |
|
||||
| `claude-code-llm-wiki` (`bundle/`) | 1 | 526 | `c5141f8` |
|
||||
| upstream reference bundles (`okf/bundles/`) | 4 | 53 + one `log.md` | `3fcbb9f` |
|
||||
|
||||
Upstream is **beyond the pilot set** and was added deliberately: it is the only
|
||||
corpus authored by the version's own publisher, and it is what turned the first
|
||||
finding from an opinion into a measurement. `node_modules` and `.obsidian` were
|
||||
excluded from the listing, which is the caller's call to make — the library
|
||||
enumerates nothing.
|
||||
|
||||
Every finding below is on the **judge** path. None of them touches what the
|
||||
library emits, so none of them blocks `v0.5.0a1`, P2 or P3: both remaining pilot
|
||||
tests are emit-side. They are recorded here against GA exit criterion 1 and are
|
||||
P4's work, not this session's.
|
||||
|
||||
**P1-F1 — `root_frontmatter` conflates "permitted and ordered" with "required",
|
||||
and the emitter and the judge disagree about the same key.** `IndexPolicy`
|
||||
reports `index_root_key_missing` for every key in `root_frontmatter` absent from
|
||||
the root index, so `OKF_V0_2` (which names `okf_version`) judges any bundle that
|
||||
exercises upstream's MAY-by-omission as violating. Measured: **14 of the 17
|
||||
bundles swept fail, each with exactly this one violation** — including all four
|
||||
of upstream's own reference bundles and three of our four goldens. The three
|
||||
that pass are the three carrying the key. Meanwhile D5's emitter treats it as
|
||||
optional: omitting `root_frontmatter_values` writes no block. We emit a MAY
|
||||
correctly and grade it as a MUST. `STRICT_V1` is unaffected in substance — the
|
||||
wiki emits all three of its pinned keys (measured) — but it is the same field
|
||||
carrying two meanings, so the fix has to separate them rather than relax them.
|
||||
|
||||
**P1-F2 — `TypePolicy.allowed` cannot be used against a consumer that quotes its
|
||||
scalars.** All **526/526** wiki documents are rejected as `okf_type_not_allowed`
|
||||
under `STRICT_V1`; **526/526** are clean when values are dequoted before judging.
|
||||
The bundle quotes every value (5265 of 5267), and this library's line-oriented
|
||||
parser has no scalar value model, so the quotes are part of the value. Blast
|
||||
radius is exactly `TypePolicy.allowed`, because it is the only surface that
|
||||
judges a value at all — `FrontmatterSchema` judges keys and value *shape*. The
|
||||
suite cannot see this: its tests hand-build the mapping and never parse a file.
|
||||
|
||||
**P1-F3 — `FrontmatterSchema.nullable` is unreachable from a parsed document.**
|
||||
Two wiki documents carry `summary: null`; the parser yields the string `"null"`,
|
||||
which is a legal value, so `frontmatter_value_null` can never fire and a real
|
||||
null is indistinguishable from the literal text. `STRICT_V1`'s
|
||||
`nullable={"summary"}` is dead configuration on the read path. Same root cause
|
||||
as F2, and worth stating separately because a fix for one does not imply the
|
||||
other.
|
||||
|
||||
**P1-F4 — the line parser manufactures false top-level keys from real v0.2 block
|
||||
YAML, and the count is large.** Over upstream's 54 documents it extracts 12
|
||||
nesting-derived keys (`resource` 51, `by` 44, `at` 44, `id` 32, `author` 5,
|
||||
`last_modified` 5, …) and 6 dash-prefixed artifacts (`- id` 22, `- title` 19,
|
||||
`- resource` 16, `- { by` 8, `- { name` 2, `- term` 1), while the container keys
|
||||
read as empty (`sources` 49/49, `generated` 44/53, `verified` 8/8). Consequence,
|
||||
measured rather than predicted: `STRICT_V1`'s schema produces **444 spurious
|
||||
findings** over those 54 documents (376 `frontmatter_key_not_allowed`, 68
|
||||
`frontmatter_key_malformed`). This is U4 on the read path with a number attached,
|
||||
and it is the case for D1b: **no closed-namespace profile should be pointed at a
|
||||
real v0.2 bundle until the structured reader lands.**
|
||||
|
||||
**P1-F5 — the door-side type gate is not a read-path judgement, and its docstring
|
||||
reads as though it were.** `TypePolicy.rejection` refuses the reserved `verdict`
|
||||
layer and is called only from `manifest.py:202` and `inbox.py:105`/`:245` — write
|
||||
paths both; Door C never calls it. Pointed at existing bundles it labels
|
||||
legitimately promoted verdict files as violations, which is what happened on two
|
||||
real po-claude bundles (one document each). Not a defect in the doors: a gap in
|
||||
the read surface, which does not exist, plus a docstring that invites the
|
||||
confusion by saying "which `okf_type` values a bundle admits".
|
||||
|
||||
**P1-F6 — v0.2 reserves two filenames and the profile models one.** §9/§12 make
|
||||
`log.md` reserved and let it appear at any level; `IndexPolicy` names only
|
||||
`index.md`, and no profile carries a reserved-file set. Measured on upstream's
|
||||
`acme_retail`: it ships a root `log.md` (`type: Log`) and its root index does
|
||||
**not** list it. So a caller building the listing an `entries_match_directory`
|
||||
profile requires has to know to exclude reserved names, and this library tells
|
||||
them only about `index.md`. Consistent with "the library enumerates nothing" —
|
||||
but the caller is being asked to know something we hold and do not state.
|
||||
|
||||
**What the sweep confirmed, which matters as much as what it found:**
|
||||
|
||||
- **V-A3 holds against real third-party v0.2 data.** No `generated` value in
|
||||
upstream's bundles reads as owned: 44 empty (block form), 9 someone else's
|
||||
inline flow mapping, 1 absent. A real v0.2 bundle cannot talk the collision
|
||||
gate into overwriting.
|
||||
- **`OKF_V0_2.frontmatter` reports zero violations across all 602 documents in
|
||||
all five corpora** — with F4's caveat that an open namespace is a weak test.
|
||||
- **Test C — C-E1 differed, C-E2/C-E3/C-E4 as expected.** C-E1: 0/526 pass as
|
||||
parsed, 526/526 pass dequoted, single cause, F2. C-E2: exactly two key-order
|
||||
signatures across 526 documents (173× the eight required keys, 353× those plus
|
||||
the three layer-specific ones), both prefixes of `_STRICT_V1_KEY_ORDER` in
|
||||
order — no change required to their eight. C-E3: not one document carries
|
||||
`generated`, `sources`, `verified`, `status` or `stale_after`, and `timestamp`
|
||||
is on 526/526, so the §13.1 legacy path holds. C-E4: the root index carries
|
||||
exactly its three pinned keys in the pinned order, nested indexes carry none,
|
||||
and `entries_match_directory` is clean in both directions at `c5141f8`.
|
||||
- **A count correction owed to the wiki.** Their pre-run ground truth was "529
|
||||
`.md`, minus 4 `index.md`, minus 1 `README`/log = 524" at `b9b557b`. Measured
|
||||
at that commit: **528** `.md` under `bundle/`, **3** files named exactly
|
||||
`index.md`, 1 `README.md` — 524 is right, and both terms of the decomposition
|
||||
are off by one in a way that cancels. The fourth "index" is a concept whose
|
||||
filename ends in `-index.md`. At `c5141f8`, the same arithmetic gives **526**,
|
||||
which is what the sweep judged.
|
||||
|
||||
### Pilot responses — all three accepted, and the asking paid for itself
|
||||
|
||||
All three replied on 2026-07-26, before the tag exists. That timing is the whole
|
||||
|
|
@ -1130,6 +1239,10 @@ everything after. **Done 2026-07-26** (`1215f98`, `7bc366b`).
|
|||
before any disk mutation.
|
||||
7. **P1** — read-only sweep over real pilot bundles. Cheap, zero-risk, and the
|
||||
first point where a finding can come from outside our own assumptions.
|
||||
**Done 2026-07-31**: 17 bundles, 602 documents, five corpora (the pilot set
|
||||
plus upstream's own reference bundles), six findings — all on the judge path,
|
||||
none blocking the tag. Results and the Test C verdict are under "P1 results"
|
||||
above; the wiki's copy went out through coord.
|
||||
8. **Pre-release tag `v0.5.0a1`**, then **P2** (producer) and **P3** (gate).
|
||||
Steps 1–8 are the whole pilot scope.
|
||||
9. **P4** — address findings.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue