docs(okf-v0.2): record Test B's verdict where criterion 1 says it lives
Catalog ran Test B on 2026-07-31 and confirmed all three expectations, but the verdict existed only as a coord message. GA exit criterion 1 requires every P1-P3 finding to be fixed or explicitly accepted "with the reason recorded in this doc" -- a result readable only in a mailbox cannot be audited by the consumers this doc is written for, so criterion 1 could have been closed against a record nobody could check. Records the reader revision and source hashes (B-E3 is unreadable without them), the three confirmations, and all three findings with dispositions: the `timestamp` RECOMMENDED warning (accepted, catalog's own upstream drift, not ours to fix), `resource`/`description` (accepted as UNMEASURED, not as clean -- catalog refused to let them read as fine), and the two gates that were not run (their words: "not run" must not read as "as expected"). Also corrects a measured-false claim in the same file: Test B does not run on Test A's output. Its procedure says `examples/` at the tag, and that is what catalog archived. The two signals were never serialized, so Test A's absence never gated catalog. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TgTrQdexGdLYmt1ZMenPyC
This commit is contained in:
parent
6e0e8eb533
commit
5dc2090aba
1 changed files with 74 additions and 1 deletions
|
|
@ -715,7 +715,13 @@ structured reader.**
|
|||
- Test A **emits** v0.2. Both `generated` and — after requirement 1's re-sizing —
|
||||
`sources` are inline flow constructs, so both are formatted strings the
|
||||
existing emitter renders verbatim. Neither requires reading one back.
|
||||
- Test B runs catalog's gate on a fixture. That fixture is Test A's output.
|
||||
- Test B runs catalog's gate on a fixture from `examples/` at the tag — **not**
|
||||
on Test A's output. Corrected 2026-08-09 against the Test B procedure below and
|
||||
against what catalog actually ran (`git archive` of
|
||||
`examples/ingest-golden-okf-v0-2/expected-bundle` at `v0.5.0a2`). The
|
||||
consequence is not cosmetic: Test B is **not** downstream of Test A, so the two
|
||||
producer- and gate-side signals were never serialized, and Test A's absence
|
||||
never gated catalog.
|
||||
- Test C validates the wiki's **v0.1-shaped** documents (scalar frontmatter,
|
||||
`timestamp`, no `generated`) against a v0.2 *variant profile*. Their documents
|
||||
contain no v0.2 families, so block-list reading is never reached.
|
||||
|
|
@ -1076,6 +1082,73 @@ read the same file and see the same value. The distinction is kept because
|
|||
compare-vs-judge is precisely what B-E3 was written to settle, and recording it
|
||||
imprecisely would discard the finding.
|
||||
|
||||
**RESULT — run by catalog 2026-07-31, after the tag. All three expectations
|
||||
confirmed.** Recorded here 2026-08-09. Until then the verdict existed only as a
|
||||
coord message, which is not where GA exit criterion 1 says a finding lives: a
|
||||
result readable only in a mailbox cannot be audited by the consumers this doc is
|
||||
written for, and criterion 1 could have been closed against a record nobody could
|
||||
check.
|
||||
|
||||
What catalog ran, in their own identifiers — they gave the revision because we
|
||||
asked for it rather than for HEAD, and B-E3 is unreadable without it:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| reader | `okf-check.mjs` @ `1ca27f6`, blob `427af3b` |
|
||||
| | `okf-frontmatter.mjs` blob `02209e7` |
|
||||
| catalog HEAD | `9992235`, working tree clean for both files |
|
||||
| source | tag `v0.5.0a2` (`5095db5`), bundle tree `0dbc16b` |
|
||||
| node | v25.8.2 |
|
||||
|
||||
Read-only via `git archive` into their own scratchpad — nothing was checked out
|
||||
in this repo, no pin was written, no code changed. The control differs from the
|
||||
fixture by exactly one line (`okf_version: 0.2` → `0.1`).
|
||||
|
||||
- **B-E1 — confirmed.** Both pass, `exit 0`.
|
||||
- **B-E2 — confirmed.** No WARNING or ERROR mentions the version. The one
|
||||
version-related line is a neutral echo for human comparison.
|
||||
- **B-E3 — confirmed, and verified in the code rather than only observed.**
|
||||
Outputs are identical once the path line and version echo are stripped. Catalog
|
||||
then read the three mechanisms B-E3 names: the shape check is
|
||||
`/^\d+(\.\d+)*$/` — form, not value — and there is no equality comparison, no
|
||||
membership list, no switch on the value. So the answer is not "they matched this
|
||||
time" but "no code exists that *could* separate them", which is what B-E3 asked.
|
||||
Their stated ground: which upstream versions exist is Google's ownership (§12),
|
||||
so the gate refuses to hold an opinion on the value.
|
||||
|
||||
**Three findings, none of them blocking, each disposed here so criterion 1 is
|
||||
closed against a record rather than against a memory:**
|
||||
|
||||
1. **`timestamp` is in catalog's `RECOMMENDED` list** (`okf-check.mjs:35`), and a
|
||||
missing recommended field produces a warning, never a failure (`:114-115`). We
|
||||
had named a gate *requiring* `timestamp` as something that would surprise us;
|
||||
it warns instead, the bundle passes, and the list is version-unconditional —
|
||||
so it breaks neither B-E2 (never mentions the version) nor B-E3 (never
|
||||
separates the runs). **Accepted, not fixed, and it is not ours to fix:** our
|
||||
fixture already carries the superseding field (`generated: { by:, at: }`), and
|
||||
upstream `okf/SPEC.md:802` @ `3fcbb9f` states `timestamp` is superseded by
|
||||
`generated.at`. Catalog recorded this as their own upstream drift and did not
|
||||
ask us to respond. Whether they version-condition `RECOMMENDED` is their call.
|
||||
2. **`resource` and `description` also warn, and are recorded UNMEASURED.**
|
||||
Catalog measured only `timestamp`, because that is the one field we named, and
|
||||
they explicitly refused to let the other two read as "fine". `resource` exists
|
||||
in our fixture **nested under `sources:`**, and their `get()` reads top level
|
||||
only. **Accepted as open, not as clean:** we have not verified against upstream
|
||||
where `resource` and `description` belong, so we cannot say whether the warning
|
||||
is their reader's flat lookup or our placement. It touches no expectation and
|
||||
blocks nothing; it is carried as unmeasured rather than silently closed.
|
||||
3. **Scope — two of catalog's three gates were not run**, deliberately and at our
|
||||
own request's boundary: `check-okf-parity.mjs` runs over a fixed corpus and
|
||||
`check-nav-golden.mjs` over their own bundle directory, so neither can see our
|
||||
fixture without a corpus change we excluded. **Their verdict for those two is
|
||||
"not run", and this doc must not read it as "as expected"** — catalog said so
|
||||
in those words, and B-E3's scope line already committed us to it. Widening is a
|
||||
separate request with its own price, and it is the operator's call. Not taken.
|
||||
|
||||
**What this settles for GA.** Criterion 5 wants one consumer to have *consumed* a
|
||||
v0.2 bundle end-to-end and one to have *emitted* one. Catalog's run is the consume
|
||||
half, and it is done. Only the emit half — Test A — is outstanding.
|
||||
|
||||
#### Test C — expressiveness (`claude-code-llm-wiki`, run by us)
|
||||
|
||||
**Procedure (ours).** Construct a v0.2 variant of `STRICT_V1` here, read their
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue