open/ktg-plugin-marketplace
1
0
Fork 0
Code Issues Releases Activity

test(okf): add running n-way per-file parity gate + adversarial corpus (STEG 3)

Browse source 
Make OKF §3 checker parity a RUNNING per-file red/green signal instead of a
stale one-shot claim. The catalog and okr checkers provably diverge on the same
input (catalog walks everything + no BOM/CRLF norm; okr skips innboks/dot-dirs +
normalizes); this gate turns that divergence into a monitored, tested fact.

- scripts/check-okf-parity.mjs: n-way runner, zero-dep. Normalizes each impl's
  existing checkBundle(root) return to {conceptCount, untyped[], okfVersion} and
  compares per file over default read-mode. Impl registry is availability-guarded
  so a catalog-only checkout degrades gracefully; a documented subprocess seam
  awaits llm-ingestion-okf's Python checker (fase 2) + Node port (fase 4).
- test/okf-parity-corpus/: committed byte-exact adversarial corpus + manifest.
  Red-proof fixtures (expected: diverge) the gate MUST go red on — byte axis
  (BOM+CRLF) and tree axis (innboks, dot-dir) each carry one; canon = both.
  .gitattributes -text pins the BOM/CRLF bytes. Green fixtures pin agreement
  incl. a SHARED gap (both descend into node_modules).
- check-okf-parity.test.mjs: 8/8. Corpus assertions + byte-tracking red-proof +
  runtime-materialized symlink/NFC-NFD meta-tests + graceful-skip seam.

Honest scope: only 2 impls expose a runnable per-file checker today, so the gate
runs 2-way and is architected n-way — no faked third impl, no cross-repo code.
spec §7 + log.md updated (no self-certified parity; fixes stay upstream-first).

Verified: parity 8/8, CLI exit 0; regression 33/33; check-versions 10 OK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeK9hkxrU9wFPBYGYnSV1V
This commit is contained in:
Kjell Tore Guttormsen 2026-07-24 01:23:46 +02:00
commit 09521f8f2d
31 changed files with 531 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

25
docs/okf-second-brain/spec.md
View file

@ -93,16 +93,21 @@ for this convention's §3 minimal contract. Run it per bundle root:
node catalog/scripts/okf-check.mjs <bundle-root>
```
**Verdict parity with okr's *current* checker is not established — do not rely on it.** The catalog
copy was lifted from okr once (`c06e4d7`); okr began hardening its checker the next day (`3b45be7`,
2026-06-30: an `innboks/`+dot-dir walk ignore `okf-check.mjs:37-38`, and a scoped
`checkBundle(root, { strictIngest, files })` signature `:73`) and added BOM/CRLF normalization weeks
later (`482effb`, 2026-07-17; `lib/frontmatter.mjs:23`), while this copy stayed frozen at the lift — so
the two can already diverge on the same input. The one parity check ever run was at lift-time, against
okr's *own* fixtures, at bundle-verdict granularity — green then, but stale now and never per-file. A
CI-run parity gate over a spec-derived adversarial corpus (`check-okf-parity.mjs`, tracked, not yet
built) is what would *establish* parity; until then, treat the catalog checker as its **own**
implementation of the §3 floor, not a verified twin of okr's. A scaffolded linkedin-studio `brain/`
**The two checkers provably diverge on the same input — and that divergence is now a running, tested
signal rather than a latent risk.** The catalog copy was lifted from okr once (`c06e4d7`); okr began
hardening its checker the next day (`3b45be7`, 2026-06-30: an `innboks/`+dot-dir walk ignore
`okf-check.mjs:37-38`, and a scoped `checkBundle(root, { strictIngest, files })` signature `:73`) and
added BOM/CRLF normalization weeks later (`482effb`, 2026-07-17; `lib/frontmatter.mjs:23`), while this
copy stayed frozen at the lift. A **running parity gate** now measures exactly this: `scripts/check-okf-parity.mjs`
compares the implementations **per file** (concept-set membership + type verdict, over default
read-mode) across a committed spec-adversarial corpus (`test/okf-parity-corpus/`) that carries
**red-proof** fixtures — inputs the gate MUST go red on, proving it is not green-on-everything. Honest
scope: only two implementations expose a runnable per-file checker today (catalog + okr), so the gate
runs **2-way** now and is architected n-way — llm-ingestion-okf's Python checker (fase 2) and its future
Node port slot in behind an availability guard when they legitimately exist. The gate does **not** claim
the two checkers are identical (they are not); it makes their divergence a monitored red/green fact and
pins a shared latent gap (both still descend into `node_modules`). Treat the catalog checker as its
**own** implementation of the §3 floor whose drift against okr is now watched, not a verified twin. A scaffolded linkedin-studio `brain/`
validates clean under this gate ("OK: valid OKF bundle", exit 0). Each plugin may keep its own dev-loop
check (linkedin-studio's TypeScript impl under `scripts/brain/` stays for its own suite); the catalog
`.mjs` is the shared gate for this convention's §3 floor. The only Stage-3 remainder is *reconciling*