ktg-plugin-marketplace/docs/okf-second-brain/spec.md
Kjell Tore Guttormsen 09521f8f2d test(okf): add running n-way per-file parity gate + adversarial corpus (STEG 3)
Make OKF §3 checker parity a RUNNING per-file red/green signal instead of a
stale one-shot claim. The catalog and okr checkers provably diverge on the same
input (catalog walks everything + no BOM/CRLF norm; okr skips innboks/dot-dirs +
normalizes); this gate turns that divergence into a monitored, tested fact.

- scripts/check-okf-parity.mjs: n-way runner, zero-dep. Normalizes each impl's
  existing checkBundle(root) return to {conceptCount, untyped[], okfVersion} and
  compares per file over default read-mode. Impl registry is availability-guarded
  so a catalog-only checkout degrades gracefully; a documented subprocess seam
  awaits llm-ingestion-okf's Python checker (fase 2) + Node port (fase 4).
- test/okf-parity-corpus/: committed byte-exact adversarial corpus + manifest.
  Red-proof fixtures (expected: diverge) the gate MUST go red on — byte axis
  (BOM+CRLF) and tree axis (innboks, dot-dir) each carry one; canon = both.
  .gitattributes -text pins the BOM/CRLF bytes. Green fixtures pin agreement
  incl. a SHARED gap (both descend into node_modules).
- check-okf-parity.test.mjs: 8/8. Corpus assertions + byte-tracking red-proof +
  runtime-materialized symlink/NFC-NFD meta-tests + graceful-skip seam.

Honest scope: only 2 impls expose a runnable per-file checker today, so the gate
runs 2-way and is architected n-way — no faked third impl, no cross-repo code.
spec §7 + log.md updated (no self-certified parity; fixes stay upstream-first).

Verified: parity 8/8, CLI exit 0; regression 33/33; check-versions 10 OK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NeK9hkxrU9wFPBYGYnSV1V
2026-07-24 01:23:46 +02:00

13 KiB

OKF-compatible second-brain form

A cross-plugin convention for how each plugin stores the user's own context — their personal/organizational "second brain" / LLM-wiki — as a portable, interoperable markdown bundle, compatible with Google's Open Knowledge Format (OKF) v0.1.

Version 0.1 · 2026-06-29 · Cross-cutting catalog artifact, owned by no single plugin. Reference design: linkedin-studio's brain/. Interop layer: Google OKF v0.1 (thin veneer). Change log + per-plugin rollout status + coordination protocol: log.md (same directory).

1. Purpose & scope

This convention exists for interop, not standard-adoption for its own sake. Three plugins in this marketplace independently grew a user-owned "second brain" — a wiki of the user's personal and organizational context the plugin retrieves from during chat and commands. This document defines the one shared on-disk form so a single reader can traverse all three, and so a future shared retrieval skill (if ever justified — §10) has one contract to build against.

  • In scope: the user's own context/data — the per-user second brain (e.g. ~/.claude/<plugin>/...).
  • Explicitly out of scope: each plugin's domain reference files (skill references/*). Those stay native Claude Code skill-references (Anthropic-recommended progressive disclosure). The decisive test, which all three plugins reached independently: not "is it an LLM-wiki" (both are) but "is there already a native, recommended mechanism?" — for skill-refs YES (skills + references + grep), for the second brain NO (it lived in ad-hoc org/*.md with no retrieval mechanism). OKF fills a real gap only for the second brain.

2. The three consumers

Plugin Second-brain maturity Role here
linkedin-studio Provenance-weighted learning system (episodic/semantic split, evidence-threshold promotion, temporal validity). Most mature. Reference design. Siblings rise toward it; it is not levelled down to bare OKF.
okr Built: writer + checker (okf-check.mjs, okf-index.mjs) + retrieval skill okr-second-brain-search. Reference checker (§7).
ms-ai-architect Designed, not built. Targets the fuller OKF package + a retrieval skill. Builds against this spec.

Live rollout status (🔵/🟡/🟢 + commit-refs) lives in log.md, not here.

3. Minimal contract (normative)

A conforming bundle is a directory tree of markdown files, one concept per file. Concept ID = file path minus .md.

  • MUST — every concept file (every .md except index.md) carries a type: frontmatter key (free string, e.g. Profile, Operations, JournalEntry).
  • MUST — every directory level has an index.md (directory enumeration, no frontmatter, carries progressive-disclosure prose).
  • MUST — the bundle-root index.md carries an okf_version marker (the upstream OKF version the bundle targets, currently 0.1).
  • MUST (consumers) — preserve unknown frontmatter keys, tolerate unknown type values, tolerate broken cross-links.

This is exactly okr's okf-check.mjs semantics, generalized (§7).

title, description, resource (canonical source URI), tags, timestamp. Supply where cheap.

  • Canonical name is resource (the OKF spec's name) — not source.
  • A field that would break a plugin's invariant may be omitted. Example: linkedin-studio omits timestamp (its serializer is pure/deterministic — a timestamp would break round-trip) and resource (an internal concept has no canonical URI), keeping type/title/description.

5. Extension keys — rich fields ride along

OKF's permissiveness is the whole point for us: conforming costs type + index.md, nothing more. A plugin's richer schema survives untouched as extension frontmatter keys that consumers MUST preserve. linkedin-studio's brain keeps provenance, first_seen, last_seen, evidence_count, status, and its episodic/semantic split — all as extension keys. Its model-collapse guard (provenance=published only) is unaffected. Plugins rise toward the richest design; they are not levelled down to bare OKF.

  • index.md — directory enumeration, no frontmatter, progressive-disclosure prose. The bundle-root one carries okf_version.
  • log.md — change log (optional per bundle; reserved name).
  • Cross-links — plain markdown (bundle-relative /... or relative); relation type conveyed by prose. Consumers MUST tolerate broken links.

7. Reference checker

okr's scripts/okf-check.mjs is the de-facto reference implementation of the minimal contract (§3): only type required (missing → fail + names the files), recommended fields → warnings, root index.md okf_version echoed for human comparison (no network — hooks are offline). ~91 lines, zero npm dependencies, only couples to a ~55-line frontmatter.mjs. The shared spec generalizes okr's semantics; it does not reinvent them. Reading okr's code is fine; writing okr is a separate go.

A shared checker now lives here: catalog/scripts/okf-check.mjs (+ vendored okf-frontmatter.mjs), originally lifted from okr's reference impl at c06e4d7 (2026-06-29), output in English, zero deps, self-contained. It is the canonical cross-plugin acceptance gate for this convention's §3 minimal contract. Run it per bundle root:

node catalog/scripts/okf-check.mjs <bundle-root>

The two checkers provably diverge on the same input — and that divergence is now a running, tested signal rather than a latent risk. The catalog copy was lifted from okr once (c06e4d7); okr began hardening its checker the next day (3b45be7, 2026-06-30: an innboks/+dot-dir walk ignore okf-check.mjs:37-38, and a scoped checkBundle(root, { strictIngest, files }) signature :73) and added BOM/CRLF normalization weeks later (482effb, 2026-07-17; lib/frontmatter.mjs:23), while this copy stayed frozen at the lift. A running parity gate now measures exactly this: scripts/check-okf-parity.mjs compares the implementations per file (concept-set membership + type verdict, over default read-mode) across a committed spec-adversarial corpus (test/okf-parity-corpus/) that carries red-proof fixtures — inputs the gate MUST go red on, proving it is not green-on-everything. Honest scope: only two implementations expose a runnable per-file checker today (catalog + okr), so the gate runs 2-way now and is architected n-way — llm-ingestion-okf's Python checker (fase 2) and its future Node port slot in behind an availability guard when they legitimately exist. The gate does not claim the two checkers are identical (they are not); it makes their divergence a monitored red/green fact and pins a shared latent gap (both still descend into node_modules). Treat the catalog checker as its own implementation of the §3 floor whose drift against okr is now watched, not a verified twin. A scaffolded linkedin-studio brain/ validates clean under this gate ("OK: valid OKF bundle", exit 0). Each plugin may keep its own dev-loop check (linkedin-studio's TypeScript impl under scripts/brain/ stays for its own suite); the catalog .mjs is the shared gate for this convention's §3 floor. The only Stage-3 remainder is reconciling the two language implementations into one — deferred until measured need, and not required for the gate to function.

8. Deliberately NOT mandated

  • Auto-classify / convert of arbitrary documents into the bundle — OKF provides nothing for it; a manual inbox/drop-zone seam suffices; build only on demonstrated need.
  • Retrieval mechanism — native Grep/Glob/Read (skill instruction "search the wiki first, open only what's relevant") vs. a dedicated fileskb MCP server. All three plugins lean native (Claude Code's Grep/Glob/Read already cover OKF's list/search/read). Per-plugin choice; a "build both, measure" candidate.
  • Degree of OKF formalism — full v0.1 conformance vs. this "OKF-compatible form." Plugins sit at different points (ms-ai-architect targets the fuller package; linkedin-studio emits the minimal form
    • extension keys; okr has writer + checker). The minimal contract (§3) is the floor all meet; going further is per-plugin and never required by this spec.

9. Verified premise corrections (dead-ends — do not plan against these)

Ground-truth-checked against the live GoogleCloudPlatform/knowledge-catalog repo (research agent, 2026-06-26, file+URL log retained). These overturn earlier framing in the per-plugin design notes:

  1. mdcode / kcmd is NOT an OKF tool. It is a Google Cloud Dataplex git-sync tool whose on-disk markdown carries a different frontmatter schema (id/resource.name/createTime/links) than OKF (type/title/description/tags/timestamp). Do not plan to emit or sync OKF bundles via mdcode. (Corrects the "metadata as code" pattern listed in ms-ai-architect's ecosystem digest.)
  2. No reusable OKF ingest code exists. The repo's reference_agent is a BigQuery+web → OKF producer, Gemini/GCP-bound; it reads a BQ dataset + seed URLs, not a document folder. The GCP-free reusable parts are the SPEC, the emit/serialize/validate core, and the index.md synthesispatterns, not a drop-in library. Classify/convert of arbitrary docs is 100% build-yourself.
  3. "OKF has no ingest" is true of the format, not the repo. And the per-plugin design notes never actually asked for auto-classification — all three frame the work as OKF as the storage form for a user-owned wiki + a retrieval skill + a maintenance mechanism, with ingest being light ("onboarding writes OKF-conformant").

10. Staged plan

  • Stage 1 — Shared form (this document). Cheap, delivers interop alone. Each plugin's user-data conforms; one reader traverses all three. This alone meets the interop goal.
  • Stage 2 — Measure divergence. Do the per-plugin retrieval paths diverge enough to hurt? Only a measured "yes" justifies Stage 3 (operator anti-pattern: ambitious initiatives where a config tweak suffices).
  • Stage 3 — Conditional shared skill. If justified: extract/generalize okr's working retrieval skill into one home (§11), with a discovery convention for where each plugin's brain lives. Do not build before Stage 2 says so.

11. Homes

  • This spec — catalog/marketplace level (here), owned by no single plugin.
  • A future shared skill (Stage 3 only) — a standalone marketplace plugin (own repo, release-tagged, catalog-pinned), installable alongside the others, serving the user's own context directly. Rejected alternatives: duplicate-per-plugin (drift risk); user-level ~/.claude/skills/ (unversioned, outside the catalog).

12. Versioning

Markers are kept deliberately separate. A single field must not carry two unrelated concerns — the upstream OKF version and a plugin's own layout revision — so those are two distinct markers:

  • okf_version — a marker line in each bundle-root index.md (§6); the upstream Google OKF version the bundle targets (currently 0.1). Required (§3). Its value set is owned by Google, so the field has exactly one meaning and stays single-valued — the bundle-root marker (§6) relies on that. When Google bumps OKF, each plugin re-checks conformance.
  • okf_layout — an optional marker line alongside it; a plugin's own layout-revision snapshot (e.g. kb-layout-2026-06). Its value set is owned by the emitting plugin — not by OKF, not by this convention. It is not a version marker, never triggers a cross-plugin re-check, and a consumer that doesn't recognize it ignores it. This is where a plugin stamps its own layout revision, keeping okf_version reserved for the upstream version alone.
  • This convention's version (top of this file) — bumped when the shared form (the §3 floor) changes. log.md records bumps of the two versions (okf_version and this one); each triggers a per-plugin re-check. Hooks are offline (no auto-poll); version drift is caught by human review + the okf_version echo in okf-check.

13. Success criterion

Measured against user value (does the plugin retrieve the right personal/org context in chat and commands?) + maintenance reliabilitynot against formal OKF conformance for its own sake. (Operator, inherited identically by all three tracks.)

14. References

  • OKF SPEC v0.1: github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md (12 June 2026, "a starting point, not a finished standard").
  • Per-plugin design notes: linkedin-studio docs/okf-convergence-brief.md; okr docs/okf-second-brain-note-2026-06.md; ms-ai-architect docs/okf-second-brain-brief-2026-06.md.
  • Shared checker (the cross-plugin gate): catalog/scripts/okf-check.mjs (+ okf-frontmatter.mjs, okf-check.test.mjs) — lifted from okr's reference impl okr/scripts/okf-check.mjs (+ okf-index.mjs, lib/frontmatter.mjs).
  • Reference design: linkedin-studio docs/second-brain/architecture.md; engine scripts/brain/.
  • Coordination + rollout status: log.md (this directory).