Operator, 2026-08-02: no MCP, no data lakes, no databases without a
stated need. We run on what we have -- defect fixes, improvements, and
features a consumer actually asked for.
That supersedes the 2026-07-27 line calling phase 5 committed, which
this file still carried. A roadmap that promises work nobody asked for
is a queue that generates its own deadlines, and the sql-is-sqlite-only
"gap" is the clearest case: closing it buys runtime dependency number
two for no asked-for use.
Phase 4 is not demoted the same way -- four named consumers have working
implementations to lift, so the need is real but untriggered. It starts
when one of them asks.
The upstream version policy is stated as the one exception and why: it
is a promise already made, so an upstream release is itself the need.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JxRjDS9U2szUGYDeuZB2M3
Measure first, widen after. The 19-fixture guard-surface suite was re-run
against v0.3.4 in a scratch venv before the range moved, and reproduced the
three deltas measured against v0.3.3 exactly, with none added. v0.3.4 is the
tag pinned rather than v0.3.3 because it shipped first and repairs a quadratic
regex (okf._MD_LINK_RE) that sits on Door C's own call path.
Door C now passes allow_reserved=False explicitly. The guard added the keyword
in the 0.3 line and defaults it True for received bundles, which would merge a
sender's index.md / log.md instead of rejecting them. The override keeps the
unconditional reserved-name refusal committed to before the keyword existed,
and the reason is structural rather than a second opinion on the guard's scan:
Door C generates the merged bundle's index.md from what it merged and writes
every merged concept verbatim, so a sender's index.md would be a second and
irreconcilable claim on one path.
This is not a behaviour change for anyone on the previous pin: under v0.2.0
the keyword did not exist and reserved names were refused by construction.
The floor is >=0.3 and not >=0.2 for a measured reason. allow_reserved is
absent in v0.2.0 and present from v0.3.0 onward, checked across all five tags:
a >=0.2 floor would admit a version that raises TypeError on every Door C
import. That measurement also corrects a recorded premise -- the plan said the
keyword "shipped in v0.3.3", which read the first version we ran the suite
against as the version it was introduced in. The conclusion held; the reason
did not, and the reason is what a future bump would have relied on.
test_door_c_pins_allow_reserved_false_against_the_guards_default locks both
halves: that the guard still defaults True, without which the override is a
no-op that would pass forever over nothing, and that Door C overrides it.
586 tests, mypy --strict clean, goldens byte-identical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V2v1hrDhrff2H3y2TNJHkF
The code already normalizes filenames and titles to Unicode NFC
(materialize.reduce_to_id_grammar, inbox.process_inbox) because macOS/APFS
hands filenames over decomposed, which would otherwise split one visual
title into two generated filenames. The invariant was implemented but not
documented anywhere in the repo.
Upstream §10 adds a concept type carrying a sanctioned way to compute a
value. This lands its FORMAT: the five contract fields for emission and
round-trip, and §10.2's one requirement. No execution — upstream defers
the receipt and verdict wire formats, so there is nothing to build a
runtime against.
Two additions, both additive:
- The five fields (`runtime`, `parameters`, `computation`, `executor`,
`attester`) join `OKF_V0_2`'s emission order as one block, internally
in §10.2's own listing order. Without it they still emit — in `emit`'s
sorted tail, where `attester` precedes `runtime`, alphabetical order
standing in for the contract's own. No bundle that carries none of the
keys changes by a byte, and the v0.1 profiles gain nothing.
- `FrontmatterSchema.required_by_type` expresses "`runtime` is REQUIRED
for this type and no other" — the first rule here keyed off a
frontmatter VALUE rather than a key. It cannot be `required`, which
would demand `runtime` of every document. A type the mapping does not
name carries no extra requirement, which is what keeps it inside §14:
a consumer must not reject on an unknown `type`, so a conditional
keyed on an unknown type stays silent rather than guesses.
Also pinned, measured today: the line-oriented parser cannot read §10's
canonical BLOCK form. `executor` and `attester` both carry a `resource`,
and with no indentation model the second overwrites the first —
`executor.resource` is lost silently, no error. Characterized rather
than fixed: reading that form needs the structured reader (D1b), and a
half-reader that drops half a contract is worse than one that never
claimed to read it. CLAUDE.md gains the invariant that falls out of it:
we emit flow form, never block, or we write bundles we cannot read back.
578 tests, mypy --strict clean, goldens byte-identical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KKKMwi7e7PVHoFW6dJK5XP
Placement settled by catalog's own reading of upstream at the pinned commit
3fcbb9f: SS8:509-510 and SS12:773-775 both put `okf_version` in a bundle-root
`index.md` frontmatter block, and SS12 calls it the only place frontmatter is
permitted in an index. Catalog's spec says the opposite about the same file;
that divergence is theirs against upstream, and we conform to upstream.
The value never touches a profile. `OKF_V0_2.index.root_frontmatter` names the
key; the caller supplies the value through a new keyword-only
`root_frontmatter_values` mapping. That keeps V4/V-A5 intact - `okf_version`'s
value tracks the upstream Google version and belongs to catalog (E1), so a
constant here would claim a decision we do not own and would have to be chased
on every upstream release. In the fixture the value is fixture DATA
(`okf-version.txt`), not a literal in our source.
Ordering comes from the policy, not the caller's mapping: a dict preserves
insertion order, so two callers passing the same keys would otherwise emit
different bytes. A key the policy does not name is refused fail-fast, before
any disk mutation. Omitting the argument emits no block at all - SS12 is a MAY
and none of upstream's four reference bundles declares the key.
The block is written only when the index is CREATED, so a re-run into an
existing bundle stays byte-identical (A-E5).
Raw-byte assertions rather than parsed ones, on the committed fixture as well
as on fresh runs: catalog measured that a quoted value fails their shape regex
with exit 1 and that a BOM hides the marker while still exiting 0.
`yaml.safe_load` returns "0.2" either way and strips a BOM first, so a parsed
assertion masks exactly those two defects. Asserting the frozen fixture catches
what a self-comparison cannot - regenerating from a broken emitter moves both
sides together.
A-E6 is now placement-explicit (promised catalog in 99cf987), and separates the
two byte properties: BOM-free is a property of the file, unquoted is a property
of CATALOG'S GATE and not of OKF v0.2 - upstream's own SS12 example is quoted,
so their gate rejects the spec's canonical form.
README gains the upstream-version section it was missing; CLAUDE.md gains the
mechanism behind "no profile hard-codes an upstream version": a profile names a
key, a caller owns its value.
550 -> 559 tests. test_profile_threading's `OKF_V0_2.index is DEFAULT.index`
assertion is replaced rather than deleted: object identity was a proxy for "the
shipped profiles differ in no NAME-bearing field", which is what makes the
synthetic test profile necessary, so the guard now asserts that directly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013dgkSPjkLpACjMayd9R5jx
Committed by the operator; the design is open and the scope is not. Recorded
in the roadmap with a scoping plan rather than in the v0.2 alignment doc, which
is about OKF v0.2 conformance and is the wrong home.
The plan settles nothing on purpose. It names the fork everything turns on —
whether we are the MCP server (an agent calls our doors, Door B's territory,
ours to design) or an MCP client (a manifest source type, which is normative in
commons' ingest-spec and therefore starts as a request to them, not as code
here). The two share a protocol and nothing else, including their owner.
Constraints written down because each has already refused something: the
one-runtime-dependency rule (an MCP SDK would be the second — stdlib JSON-RPC
or an optional extra), no model calls in the run path (the model may CALL us,
never the reverse), the guard persist gate (MCP is transport, not a bypass),
the network opt-in, and the required ingested_at.
The standing always-latest policy had no procedure, and v0.1 -> v0.2 showed what
that costs: the spec was read, the shipped example bundles were not, and five
findings a spec reading had settled wrongly surfaced only after someone asked
whether we had studied the examples. One of them re-sized the release itself.
docs/upstream-okf-upgrade-runbook.md is that procedure. Every step names the
concrete failure it prevents, and each is a failure that actually happened:
- Pin before reading. Two repos read main independently, which felt like two
confirmations and was one unstable reference read twice -- and SPEC.md was
edited after the migration commit, so neither alone was the whole answer.
- Enumerate the whole okf/ tree. bundles/ held the real examples and was on
nobody's list until the tree was listed.
- Read the shipped examples, not only the normative text. A directory named
samples/ turned out to be a runner config, not samples of the format. And
markdown-converting fetch tooling strips exactly the byte-level properties
(BOM, trailing newline, quoting) that break downstream gates.
- Name fixtures by repo AND path. Two repos had examples/ingest-golden-file/
holding different fixtures with zero content overlap, and a pilot baseline was
agreed by a name that pointed at two files.
The black-box commitment is now an invariant in CLAUDE.md rather than an
emergent property: a consumer's cost for an upstream release is a re-run.
Additive profiles, byte-stable existing profiles, keyword-only new parameters so
positional call sites survive, no churn in consumer fixtures. The boundary is
stated rather than glossed -- shape changes are absorbed, upstream changes to
consumer-authored content are not, and those get a measured exposure report per
consumer instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
All three pilots accepted. Their measurements land in the plan now rather than
as failed runs later, which is the entire return on asking before building.
Six corrections, each measured by the consumer rather than reasoned by us:
- A-E6 specified `okf_version: "0.2"` quoted. Catalog measured the quoted form
failing their shape regex with exit 1, and a UTF-8 BOM making the marker
invisible while still exiting 0. Both now stated as raw-byte expectations.
- A-E2 said the diff lies only *inside* the frontmatter block. No bundle whose
root index has no frontmatter block can satisfy that together with A-E6.
- B-E3 covers one of catalog's three gates; the other two cannot see our
fixture. Recorded so the report is not read as broader than it is.
- Our claim that no gate keys on the version value was wrong as formulated:
check-okf-parity.mjs puts the raw value in its signature. It compares without
judging -- verdict-invariant, not value-blind.
- C-E1's ground truth is 524 non-reserved documents, not 522; 522 was correct
at d2c12d2.
- `materialize_bundle` gains keyword-only `profile=DEFAULT` on po-claude's
request. The signature already has a `*`, so their three-positional call path
stays source-compatible and additivity becomes a property of the signature
instead of something a consumer measures.
Also recorded: po-claude closed their index-parser defect themselves (f41264f),
so do not design around it; the assumption under it -- "a generated index has no
frontmatter" -- is not peculiar to them. Dropping the superseded `timestamp`
draws one catalog warning per concept file, decided after the pilot, not before.
The estimate's open caveat is discharged by reading profiles.py:197: emit is one
formatting path, the validator already admits list values, so only the emitter
side is missing and the sizing stands.
CLAUDE.md gains a standing constraint: consumer content stays at form level in
public files. The wiki's bundle is private pending an Anthropic ToS assessment,
their report goes back through coord, and a push is not reversible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Operator directive 2026-07-26: consumers are better served by getting the
latest version early and reporting back than by us holding it until we judge
it finished.
This closes a real gap. Every test in the plan -- including V-A8 against
upstream's reference implementation -- asks whether output is conformant.
None asks whether it is usable: whether a bundle is awkward to construct,
whether a rejection message is actionable, whether the profile can express
what a consumer's actual data needs. Only real data surfaces that.
Pilot set is three repos, one axis each, chosen for signal:
portfolio-optimiser-claude for the producer path (one real manifest run),
catalog for gate acceptance (their gate on our fixture, measured rather than
inferred from the regex), and claude-code-llm-wiki for expressiveness across
522 real documents.
The load-bearing design detail is that most of the pilot is read-only on our
side: we run the v0.2 reader over real consumer bundles and send the report.
That needs no adoption, no writes into their trees (O2 holds), and no change
to a contract their operator ratified. Only the producer axis asks a
consumer to do anything, and it asks for one run.
Excluded with reasons rather than silently: okr (Node side not yet lifted),
linkedin-studio (v0.2's provenance families would put implicit pressure on
the ingest/published carve-out we agreed not to normalize), commons (they
are deciding V1 -- a participant, not a test site).
Shipping a provisional surface without owing stability rests on three rules,
not on saying "provisional": OKF_LATEST does not point at v0.2 until GA, so
flipping it is the GA event rather than a merge side effect; the docstring
and CHANGELOG name the pilot repos; and breaking changes during the pilot
get no deprecation cycle. Stating that last one up front is what buys the
freedom to act on feedback -- discovering it later is what turns a pilot
into a de-facto release.
Feedback is requested in five named parts, because unstructured reports are
not comparable across three repos. The one that matters most is "what was
awkward but worked": workarounds are the highest-value signal and never
appear as a failure.
GA exit criteria are testable, and carry one honest limit: a three-repo
pilot exercises only what those three use, so `sources` with usage_window,
multi-verifier `verified`, and Attested Computation will likely go
unexercised. Those stay marked provisional at GA instead of being silently
promoted -- claiming otherwise would be the same unearned-claim pattern that
"conform first, claim after" exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Operator directive 2026-07-26: the library always supports the current
latest version of Google OKF. v0.2 shipped 2026-07-25, so v0.2 support is
committed work rather than something a consumer has to request. This
overrides the previous default answer to open question V3 ("no until a
named consumer asks"), which is kept in the plan marked superseded so the
override reads as deliberate.
Support is additive: a new profile, never a migration of the existing two.
That single design choice is what makes an always-latest policy sustainable,
and it resolves the tension the directive would otherwise create with three
constraints that do not yield to it:
- DEFAULT states commons' ingest-spec section 5 layer, so its `generated`
shape is commons' call. Under the additive design this stops blocking us,
which takes commons off the critical path.
- STRICT_V1 mirrors the proving consumer's ratified contract; changing
another repo's contract from here would violate O2.
- v0.2 defers the attestation receipt and verdict wire formats upstream, so
the format is supportable and the unspecified runtime is not. It re-enters
scope when upstream specifies it.
This is also the first time the phase-3 profile abstraction is forced by
something outside this repo rather than by a second consumer, which is the
better test of whether the seam was cut in the right place.
Deliverables D1-D6 replace the earlier decision-round framing: a frontmatter
model that can carry block lists (`sources`, multi-verifier `verified`), an
OKF_V0_2 profile plus an OKF_LATEST alias whose moving-target tradeoff is
documented rather than hidden, Door C conformance against the consumer
tolerance rules, `Attested Computation` round-trip, v0.2 golden fixtures, and
a release-checklist re-check so the standing policy cannot decay silently.
Two new assumptions carry the weight. V-A7 forbids any profile from emitting
`timestamp` together with a malformed `generated`, since that combination
would have neither a valid `generated.at` nor an eligible section 13.1
fallback. V-A8 validates our own v0.2 fixture against upstream's reference
implementation, because every other test in the suite only asks whether we
agree with ourselves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Everything built so far targets OKF v0.1. Upstream published v0.2 on
2026-07-25, so the plan records how the library relates to it and, more
importantly, who owns each decision.
Read from the spec itself rather than secondhand, which corrected two
readings that a summary had gotten wrong:
- STRICT_V1's `timestamp` is NOT a defect. Section 13.1 grants consumers a
documented fallback to legacy `timestamp` precisely when `generated` is
absent, and STRICT_V1 emits no `generated`. Nothing is asked of the wiki.
- The one measured shape problem is DEFAULT's `generated: "true"`, because
v0.2 requires `generated.by` within `generated`. That key was not reserved
in v0.1, so it was legal when written; v0.2 claimed the name. DEFAULT
states commons' ingest-spec section 5 layer, so the fix is commons' call
and is raised there as open question V1 rather than patched locally.
Two findings shrink the work. The canonical form for `generated` and a
single `verified` is an inline flow mapping, which the existing scalar
parser already round-trips as an opaque string, so block-list support is
only needed for `sources` and multi-verifier `verified` -- and only if a
named consumer asks. And the collision degrades safely: `_is_ingest_owned`
returns False for a v0.2 mapping, so a foreign concept is refused rather
than overwritten.
The track sits between Phases 3 and 4 because Phase 4 freezes the
cross-runtime contract. Freezing a v0.1 shape into two runtimes would let
the shared fixture suite certify the drift instead of catching it.
Self-imposed rule, since the spec does not require it: conform first, claim
after. Declaring `okf_version: "0.2"` is a MAY with no conformance
checkpoint, so claiming it early would be permitted -- and would be the same
class of true-sounding misleading claim as reporting a 0.2.0 measurement
under a 0.3.1 heading.
Also moves the guard 0.3.1 measurement procedure out of session state and
into execution-order.md, where it belongs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
`IndexPolicy` grew five judging fields and `IndexPolicy.violations`, closing the
gap a test has been pinning since `848e395`. `STRICT_V1.index` is now the proving
consumer's shape rather than DEFAULT's.
The design was settled by a conflict rather than by a preference. The convention
owner reported that an index is an authored count of a directory's children and
that a validator enumerating the directory has implemented the wrong contract.
Checked against the proving consumer before adopting it, the two turn out to be
directly opposed: gate BUNDLE_INDEX_COMPLETE (validate.py:1081-1120, ERROR) builds
its expected set by enumerating the directory and demands an exact bidirectional
match, and bundle.py:498-567 writes every index from a tree walk. Neither is
incoherent inside its own spec, so authored-versus-derived is a policy field in
both directions and a library invariant in neither.
Consequences encoded rather than documented: nothing here enumerates a directory
— the caller supplies the listing, `violations` refuses one when the profile's
index is authored and refuses to run without one when it is derived, so code
written to the wrong contract fails at the call instead of passing every test one
would think to write. Root and nested indexes are asymmetric (confirmed in both
consumers, different key sets). A per-entry description is template-level, so
`render_link` refuses both a missing description and an unwanted one.
DEFAULT keeps upstream's root-only index and judges nothing, for the same measured
reason it carries no required frontmatter key set: upstream binds `index.md` to
the bundle root alone, so a judging default would condemn conforming bundles.
25 new tests, 493 green. C1 re-proven: `git diff --stat examples/` empty.
Adds llm_ingestion_guard>=0.2,<0.3 as this library's first and only runtime
dependency, and guard_adapter.py -- the one module that imports it. The flows
themselves are unchanged: they still take an injected gate, and importing the
package still does not import the guard, so a Door A consumer is unaffected by
the dependency's state.
Door B screens the exact bytes it persists. The guard's §6 bookends
(prepare_input -> model -> screen_output) assume a model call in between; this
library makes none, and prepare_input returns prompt-shaped text (sanitized
AND spotlight-fenced with a per-call nonce) that must never reach disk. So the
adapter calls screen_output alone, on the extracted text as it stands, and
hands that same text back -- the verdict is then a statement about the bytes
actually written. This supersedes the plan's "bookends" wording, recorded
there under "Settled during implementation (step 4)".
It follows that the gate refuses rather than repairs: a file carrying an
invisible carrier is rejected, not stripped and persisted. Sanitizing first
would write a document differing invisibly from the operator's file while
source_sha256 still points at the original bytes. Operator decision; the
policy is PRESET_USER_UPLOAD, so any finding at all is held back.
Door C hands the bundle over whole to okf.import_bundle, which resolves the
cross-link graph across concepts. Per-concept reasons are derived from the
scan findings (severity:label) because stamp_concept keeps the disposition and
drops the reason strings behind it.
Assumption B1 closes as a signature smoke test over what the adapters actually
call -- screen_output and okf.import_bundle signatures, the Disposition values
both doors compare by value, the Origin/Channel vocabularies Door C validates,
the result fields read, and the upload preset's shape. prepare_input is not
pinned: drift there cannot reach this library. Behaviour is pinned against the
real scanner too, including the persist-gate proof that a fail-secure fixture
leaves the bundle byte-identical.
B2 closes with it: git+https tag install over anonymously readable HTTPS, no
credential. A direct reference is an install-time channel, not the pin -- the
range stays in pyproject, is satisfied by the tag install today, and resolves
normally once the package index exists. A packaging test enforces that the
guard remains the only runtime dependency (verified by hand-mutation).
Reads an external OKF bundle as {bundle-relative path -> document text},
hands it WHOLE to an injected gate over the guard's okf.import_bundle (a
bundle-level call: it resolves the cross-link graph across concepts), and
merges only concepts clearing the non-blocking floor. Same injection pattern
as Door B, so the core stays dependency-free while the CI channel for the
real guard is settled.
Two constraints shaped the design and are pinned by tests:
- A merged concept is written VERBATIM. Stamping provenance into it would
require round-tripping its frontmatter through this library's line-oriented
parser, which cannot represent the block lists the guard's parser accepts --
silent data loss -- and would persist bytes the guard never screened.
- Ownership is therefore proven by content identity: identical bytes at the
target name are a no-op re-merge (re-import of an unchanged bundle is
idempotent), and anything else at the name is refused. Curated content and
an updated concept are refused alike; refusing is what never destroys.
The floor is fail-closed beyond the plan's "no error" wording: an error, an
unrecognised disposition, and a concept the gate returned no verdict for are
all refusals. quarantine_review stays its own bucket, as at Door B.
origin/channel are validated against the guard's pinned vocabulary -- it
derives trust from origin by enum identity, so an unrecognised string would be
silently downgraded rather than caught.
Three primitives promoted for reuse rather than duplicated:
reduce_to_id_grammar and check_filename_length to materialize.py, and
extract.decode_text. Door C slugs the WHOLE concept path, so tables/users.md
and views/users.md stay distinct. Concept discovery folds case explicitly
rather than globbing *.md, whose case-sensitivity follows the filesystem and
would import the same bundle differently on APFS and ext4.
README's "what is gated today" section corrected: it claimed nothing is gated,
which is no longer true, but the honest statement is narrower than "the doors
are gated" -- the library cannot verify that an injected adapter is a real
guard, and a permissive stub is believed.
405 tests green; ruff, ruff format and mypy --strict clean.
The Door A description said the §11 golden fixtures "are missing today".
They shipped in 9dd86b1 as ingest-golden-{file,sql,http}/. Correct the line
to state what is present.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
Operator decision: the initial v1 boundary (portfolio pair only) left
five of eight OKF surfaces uncovered. CLAUDE.md/README now define a
four-phase roadmap: door A, doors B/C, configurable bundle contract
(wiki strict-v1), and a zero-dep Node/ESM half for the second-brain
plugin world. Halves share contract and fixtures, never code. Security
stays with llm-ingestion-guard in both runtimes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QeqhJpYQyghASjiJo5EhGg
