open/ktg-plugin-marketplace
1
0
Fork 0
Code Issues Pull requests Releases Activity

test(okf): add running n-way per-file parity gate + adversarial corpus (STEG 3)

Browse source 
Make OKF §3 checker parity a RUNNING per-file red/green signal instead of a
stale one-shot claim. The catalog and okr checkers provably diverge on the same
input (catalog walks everything + no BOM/CRLF norm; okr skips innboks/dot-dirs +
normalizes); this gate turns that divergence into a monitored, tested fact.

- scripts/check-okf-parity.mjs: n-way runner, zero-dep. Normalizes each impl's
  existing checkBundle(root) return to {conceptCount, untyped[], okfVersion} and
  compares per file over default read-mode. Impl registry is availability-guarded
  so a catalog-only checkout degrades gracefully; a documented subprocess seam
  awaits llm-ingestion-okf's Python checker (fase 2) + Node port (fase 4).
- test/okf-parity-corpus/: committed byte-exact adversarial corpus + manifest.
  Red-proof fixtures (expected: diverge) the gate MUST go red on — byte axis
  (BOM+CRLF) and tree axis (innboks, dot-dir) each carry one; canon = both.
  .gitattributes -text pins the BOM/CRLF bytes. Green fixtures pin agreement
  incl. a SHARED gap (both descend into node_modules).
- check-okf-parity.test.mjs: 8/8. Corpus assertions + byte-tracking red-proof +
  runtime-materialized symlink/NFC-NFD meta-tests + graceful-skip seam.

Honest scope: only 2 impls expose a runnable per-file checker today, so the gate
runs 2-way and is architected n-way — no faked third impl, no cross-repo code.
spec §7 + log.md updated (no self-certified parity; fixes stay upstream-first).

Verified: parity 8/8, CLI exit 0; regression 33/33; check-versions 10 OK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeK9hkxrU9wFPBYGYnSV1V
This commit is contained in:
Kjell Tore Guttormsen 2026-07-24 01:23:46 +02:00
commit 09521f8f2d
31 changed files with 531 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

25
docs/okf-second-brain/log.md
View file

@ -161,6 +161,31 @@ protocol; 🟢 is reserved for the independent gate-verified step (operator veri
today, so okr's migration (move the value to `okf_layout`) is cheap when it happens, and it should
happen before that tightening. Additive + disambiguating: the §3 floor is unchanged and `okf_layout` is
optional, so **the convention version stays `0.1`** — no re-ratification triggered. _(catalog session.)_
- **2026-07-23** — **STEG 3: the running parity gate landed** (`scripts/check-okf-parity.mjs` +
`test/okf-parity-corpus/` + `check-okf-parity.test.mjs`). Parity is now a *running* per-file red/green
signal over a committed spec-adversarial corpus, not a stale one-shot claim (retracted above). **Verified
scope blocker (ground-truth this session, 2 independent agents + grep):** the round named ≥3 impls, but
**only two expose a runnable per-file checker** — catalog + okr. llm-ingestion-okf's Python lib is a
*producer* (no `check_bundle`, no CLI; checker is fase-2, its own repo); linkedin-studio's TS §3 logic
lives only in private test helpers (not a participant). **Decision (operator delegated on quality
grounds):** build the gate **n-way-capable, wire the 2 real impls now**, with a documented availability-
guarded seam for the Python checker (fase 2) and its Node port (fase 4) — no faked third impl, no
cross-repo code. Decisions the earlier record left open, now fixed: **(a) per-file contract** = each
impl's existing `checkBundle(root)` return normalized to `{ conceptCount, untyped[], okfVersion }`,
compared per file (concept-set + type verdict), over **default read-mode** (strictIngest off, not
files-scoped); **(b) adapter convention** = JS impls imported directly (both export `checkBundle`), zero
subprocess/zero dep; cross-repo/other-language impls get a subprocess `--json` adapter when they exist;
**(c) corpus form** = committed byte-exact fixtures (`.gitattributes -text` pins BOM/CRLF), the inspectable
"borne proof"; **(d) red-proof** = `expected: "diverge"` fixtures the gate MUST go red on (byte + tree
axes each carry one; canon = BOM+CRLF concept + `innboks/` raw file, catalog FAIL vs okr OK). Axis
decisions: IN = byte (BOM/CRLF), tree (innboks, dot-dir, node_modules[shared gap, both descend],
duplicate-name, hierarchical); **WITHDRAWN** = verbatim/trailing-newline (trinn E §1); **OUT of STEG 3** =
path-forms (rerouted to §1.2/STEG 5). The git/FS-hostile axes (live symlink, NFC/NFD filename) are
covered as **runtime-materialized meta-tests** (temp dir), not committed fixtures. The gate does **not**
claim the two checkers are identical (they provably are not) — it makes their divergence a monitored
fact. Verified: parity 8/8, `node check-okf-parity.mjs` → exit 0 ("parity holds"); regression 33/33;
`check-versions` 10 OK. Fixes stay **upstream-first** (patching BOM-norm into the catalog copy would just
re-diverge in the other direction). _(catalog session.)_
## Distilled architecture notes (OKF round, 2026-07)

25
docs/okf-second-brain/spec.md
View file

@ -93,16 +93,21 @@ for this convention's §3 minimal contract. Run it per bundle root:
node catalog/scripts/okf-check.mjs <bundle-root>
```
**Verdict parity with okr's *current* checker is not established — do not rely on it.** The catalog
copy was lifted from okr once (`c06e4d7`); okr began hardening its checker the next day (`3b45be7`,
2026-06-30: an `innboks/`+dot-dir walk ignore `okf-check.mjs:37-38`, and a scoped
`checkBundle(root, { strictIngest, files })` signature `:73`) and added BOM/CRLF normalization weeks
later (`482effb`, 2026-07-17; `lib/frontmatter.mjs:23`), while this copy stayed frozen at the lift — so
the two can already diverge on the same input. The one parity check ever run was at lift-time, against
okr's *own* fixtures, at bundle-verdict granularity — green then, but stale now and never per-file. A
CI-run parity gate over a spec-derived adversarial corpus (`check-okf-parity.mjs`, tracked, not yet
built) is what would *establish* parity; until then, treat the catalog checker as its **own**
implementation of the §3 floor, not a verified twin of okr's. A scaffolded linkedin-studio `brain/`
**The two checkers provably diverge on the same input — and that divergence is now a running, tested
signal rather than a latent risk.** The catalog copy was lifted from okr once (`c06e4d7`); okr began
hardening its checker the next day (`3b45be7`, 2026-06-30: an `innboks/`+dot-dir walk ignore
`okf-check.mjs:37-38`, and a scoped `checkBundle(root, { strictIngest, files })` signature `:73`) and
added BOM/CRLF normalization weeks later (`482effb`, 2026-07-17; `lib/frontmatter.mjs:23`), while this
copy stayed frozen at the lift. A **running parity gate** now measures exactly this: `scripts/check-okf-parity.mjs`
compares the implementations **per file** (concept-set membership + type verdict, over default
read-mode) across a committed spec-adversarial corpus (`test/okf-parity-corpus/`) that carries
**red-proof** fixtures — inputs the gate MUST go red on, proving it is not green-on-everything. Honest
scope: only two implementations expose a runnable per-file checker today (catalog + okr), so the gate
runs **2-way** now and is architected n-way — llm-ingestion-okf's Python checker (fase 2) and its future
Node port slot in behind an availability guard when they legitimately exist. The gate does **not** claim
the two checkers are identical (they are not); it makes their divergence a monitored red/green fact and
pins a shared latent gap (both still descend into `node_modules`). Treat the catalog checker as its
**own** implementation of the §3 floor whose drift against okr is now watched, not a verified twin. A scaffolded linkedin-studio `brain/`
validates clean under this gate ("OK: valid OKF bundle", exit 0). Each plugin may keep its own dev-loop
check (linkedin-studio's TypeScript impl under `scripts/brain/` stays for its own suite); the catalog
`.mjs` is the shared gate for this convention's §3 floor. The only Stage-3 remainder is *reconciling*