The measured defect, as data: the 2026-08-26 bake-off had every arm retrieve
40/40, so quality could not separate them. The only axis that did was trap
exposure -- 18/20 for the OKF-index arm against 8/20 for a frontmatter
head-scan -- and both sides measured the reason independently: the flat index
carries title/date/status/supersedes 0 times while its own documents carry them
55/55/55/5. The metadata is in the bundle; the index throws it away.
FacetPolicy lets an index entry keep it. The grammar is thin on purpose (one
separator, then key: value joined by '; ') because index lines are read by
regex on both sides of this library, and a value carrying either delimiter is
REFUSED rather than escaped -- validation, not repair, as everywhere else here.
Additive by construction, not by caution. entry_pattern IS link_pattern when a
policy carries no facets, so DEFAULT and STRICT_V1 match the same lines and
emit the same bytes; the goldens are the proof. Facets arrive as STRUCTURED_V1,
a new profile, because DEFAULT states commons' ingest-spec index layer and
changing its bytes from here would be this repo editing a contract it does not
own.
17 new tests; suite 660 -> 677.
Commons ratified V1 2026-08-02 and executed it at `54e0ec7`; verified
against their tree rather than taken on report. ingest-spec.md:217 now
defines `generated` as `{ by: process:okf-ingest, at: <ingested_at> }`,
unquoted, `at` repeating `ingested_at` verbatim. `generated: true` no
longer appears in the spec.
`DEFAULT` states commons' §5 layer, so its stamp is theirs to decide.
`DEFAULT.ownership` gains the actor; the four goldens this repo's plan
named in advance were regenerated by RUNNING the materializer, each on
its own case's `ingested-at.txt`. The v0.2 golden was untouched, as
predicted -- it has carried the O2 form since D5.
Not a migration onto OKF v0.2: `DEFAULT` stays v0.1 on every axis
upstream owns and still emits no `sources`. Commons' spec and the Google
version are independent axes, and comments that narrated them as one
were rewritten rather than left to mislead. README and CLAUDE.md said
the additive rule without that boundary, which would have told a
consumer their DEFAULT bytes can never move; both now state it.
V-A3 is amended, not dropped. `DEFAULT` must OWN the mapping it now
writes -- a profile refusing its own output fires the collision gate on
files its own previous run wrote -- while a mapping naming a foreign
actor, or §7's `human:` actor on curated content, stays unowned. That
half is what carried the safety and it is asserted directly.
§11's stamp-integrity condition moved with the value: the forgeable
stamp was `true` and is now the mapping naming the ingest actor. The
defence was never the value -- the §3 scan globs `ingest-*.md`, so a
Door C import is unreachable however well it forges. Second spoof test
added; both were hand-mutated (glob widened to `*.md`) to confirm they
can fail.
The characterization test derived its foreign-stamp fixture from the
literal `generated: true`, which V1 leaves without a referent -- a
silent no-op waiting to happen. It now derives the needle from the
profile and asserts the substitution occurred.
Door B is deliberately untouched: not the ingest-spec's, marker is
`generated` + `source_file`, disjoint from Door A's `ingest_manifest`,
and the divergence predates V1.
Nothing released or notified. The pilot set pins `v0.5.0a2`, not `main`,
so this is invisible to portfolio-optimiser's freeze and demo; the
consumer exposure report is owed at the release that carries this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VwcjUXbKySLbEG5WqTNkta
Upstream §10 adds a concept type carrying a sanctioned way to compute a
value. This lands its FORMAT: the five contract fields for emission and
round-trip, and §10.2's one requirement. No execution — upstream defers
the receipt and verdict wire formats, so there is nothing to build a
runtime against.
Two additions, both additive:
- The five fields (`runtime`, `parameters`, `computation`, `executor`,
`attester`) join `OKF_V0_2`'s emission order as one block, internally
in §10.2's own listing order. Without it they still emit — in `emit`'s
sorted tail, where `attester` precedes `runtime`, alphabetical order
standing in for the contract's own. No bundle that carries none of the
keys changes by a byte, and the v0.1 profiles gain nothing.
- `FrontmatterSchema.required_by_type` expresses "`runtime` is REQUIRED
for this type and no other" — the first rule here keyed off a
frontmatter VALUE rather than a key. It cannot be `required`, which
would demand `runtime` of every document. A type the mapping does not
name carries no extra requirement, which is what keeps it inside §14:
a consumer must not reject on an unknown `type`, so a conditional
keyed on an unknown type stays silent rather than guesses.
Also pinned, measured today: the line-oriented parser cannot read §10's
canonical BLOCK form. `executor` and `attester` both carry a `resource`,
and with no indentation model the second overwrites the first —
`executor.resource` is lost silently, no error. Characterized rather
than fixed: reading that form needs the structured reader (D1b), and a
half-reader that drops half a contract is worse than one that never
claimed to read it. CLAUDE.md gains the invariant that falls out of it:
we emit flow form, never block, or we write bundles we cannot read back.
578 tests, mypy --strict clean, goldens byte-identical.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KKKMwi7e7PVHoFW6dJK5XP
v0.5.0a1 shipped the one actor value the spec owner had already excluded.
Caught before any pilot was notified, so it costs a tag, not a migration.
Commons decided <fast id> = process:okf-ingest on 2026-07-31, on this repo's
own proposal, superseding option (d) (process:llm-ingestion-okf) chosen here on
07-27. The exclusion is ingest-spec.md:7-8, frozen on the spec being
framework-neutral: normalising OUR repo name into the normative id would force
every other conformant implementation to write it into its own output. Verified
against three independent sources before touching anything — commons' coord
message 20260731T154140Z, their plan :215-216/:244, and their STATE :37.
Why this had to land before the pilot notifications rather than after:
actor is both the stamp written and the value owned back (OwnershipPolicy),
and recognition is one-way. A pilot that had run Test A against the excluded
id would hold bundles this library stops recognising the moment the id is
corrected — collision_unstamped on their OWN files. That is the A-E5 failure
mode, and we would have inflicted it.
Worse, it would not have shown up as a failure: the plan's A-E3 expectation
(:854) named the same excluded value as the code, so Test A would have PASSED
and confirmed the error. Expectation and implementation agreeing is not
evidence when both predate the decision.
Nothing in the wild carried the old value: OKF_V0_2 did not exist at v0.4.0, so
the profile has never been released. v0.5.0a1 is abandoned, not moved — a tag
already on a public remote does not get force-pushed, and the history should
say plainly that a1 was wrong.
A-E3 now records both corrections with dates. The V1 paragraph at :1169 is
superseded in place rather than rewritten: its reasoning still holds, only its
outcome moved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVTup4v7tW9QiVyBENk2LV
`IndexPolicy.root_frontmatter` carried two claims at once: which keys the
root index may carry and in what order, and which it must carry. The judge
read it as the second, so `OKF_V0_2` — which names `okf_version` only to fix
its position, upstream granting it as a MAY (§8:509-510, §12:773-775) —
reported every bundle exercising that MAY as violating. Measured in P1 over
17 bundles: 14 failed with exactly this one violation, upstream's own four
reference bundles among them, while D5's emitter treated the same key as
optional. We emitted a MAY correctly and graded it a MUST.
`root_frontmatter` now permits and orders; `root_frontmatter_required`
demands. A required key outside the ordered set fails at construction, the
same contradiction `FrontmatterSchema` already refuses when `required`
strays outside `allowed`.
- `OKF_V0_2` requires none — upstream's MAY, stated on the judge side too.
- `STRICT_V1` requires all three: the proving consumer's root index carries
exactly those keys in that order on every bundle measured (`c5141f8`), so
separating the meanings costs them nothing.
- `DEFAULT` names no root keys and is untouched.
Re-measured over the eight bundle-root indexes reachable locally (our four
goldens, upstream's four): 7 of 8 failing under the old semantics, 0 of 8
after. Emit path byte-identical — the golden suite fails otherwise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JqCmfJ2ukpFXeFjfab8wvy
Placement settled by catalog's own reading of upstream at the pinned commit
3fcbb9f: SS8:509-510 and SS12:773-775 both put `okf_version` in a bundle-root
`index.md` frontmatter block, and SS12 calls it the only place frontmatter is
permitted in an index. Catalog's spec says the opposite about the same file;
that divergence is theirs against upstream, and we conform to upstream.
The value never touches a profile. `OKF_V0_2.index.root_frontmatter` names the
key; the caller supplies the value through a new keyword-only
`root_frontmatter_values` mapping. That keeps V4/V-A5 intact - `okf_version`'s
value tracks the upstream Google version and belongs to catalog (E1), so a
constant here would claim a decision we do not own and would have to be chased
on every upstream release. In the fixture the value is fixture DATA
(`okf-version.txt`), not a literal in our source.
Ordering comes from the policy, not the caller's mapping: a dict preserves
insertion order, so two callers passing the same keys would otherwise emit
different bytes. A key the policy does not name is refused fail-fast, before
any disk mutation. Omitting the argument emits no block at all - SS12 is a MAY
and none of upstream's four reference bundles declares the key.
The block is written only when the index is CREATED, so a re-run into an
existing bundle stays byte-identical (A-E5).
Raw-byte assertions rather than parsed ones, on the committed fixture as well
as on fresh runs: catalog measured that a quoted value fails their shape regex
with exit 1 and that a BOM hides the marker while still exiting 0.
`yaml.safe_load` returns "0.2" either way and strips a BOM first, so a parsed
assertion masks exactly those two defects. Asserting the frozen fixture catches
what a self-comparison cannot - regenerating from a broken emitter moves both
sides together.
A-E6 is now placement-explicit (promised catalog in 99cf987), and separates the
two byte properties: BOM-free is a property of the file, unquoted is a property
of CATALOG'S GATE and not of OKF v0.2 - upstream's own SS12 example is quoted,
so their gate rejects the spec's canonical form.
README gains the upstream-version section it was missing; CLAUDE.md gains the
mechanism behind "no profile hard-codes an upstream version": a profile names a
key, a caller owns its value.
550 -> 559 tests. test_profile_threading's `OKF_V0_2.index is DEFAULT.index`
assertion is replaced rather than deleted: object identity was a proxy for "the
shipped profiles differ in no NAME-bearing field", which is what makes the
synthetic test profile necessary, so the guard now asserts that directly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013dgkSPjkLpACjMayd9R5jx
The v0.2 profile lands additively: DEFAULT keeps stating commons' §5 layer
byte for byte (the golden suite is the proof, not the claim), and OKF_V0_2
adds the families v0.2 introduces on top of DEFAULT's key order.
Two questions the plan left open were the operator's, and both were decided
before code rather than discovered during it:
- `generated.by` is `process:llm-ingestion-okf` — plan V1's option (d), the
§7 process actor form. The value carries no version, which is what keeps a
byte-compared fixture stable across releases and leaves a shared
cross-implementation fixture possible. A-E3 was written against the
`<producer>/<version>` form and is now STALE in the pilot specification; the
correction is owed to portfolio-optimiser-claude before they run.
- `sources[].resource` is the manifest source's locator verbatim: the file
root, the sql connection_ref (an env-var NAME, never its value), or the http
base_url. `credential_ref` is not a locator and is never emitted.
Ownership becomes a policy on the profile rather than a literal in the gate.
The emitter and `_is_ingest_owned` are coupled through the stamp value, so
OwnershipPolicy is where they meet and can only change together. Recognition
is one-way by decision: OKF_V0_2 owns the v0.1 stamp too, so a DEFAULT-written
bundle re-runs IN PLACE (operator, 2026-07-27), while DEFAULT still refuses a
v0.2 file rather than replacing it — V-A3's fail-safe is preserved.
An unquotable locator is refused rather than emitted. Measured with PyYAML
rather than reasoned: `[{ id: a, resource: data, backup }]` raises nothing and
parses to a mapping with a `backup` key nobody wrote, so the failure mode is a
silently wrong provenance record. Validation, not repair.
Deliberately NOT here: the `okf_version` declaration. §12 makes it a MAY and
none of upstream's four reference bundles exercises it; WHERE it goes is open
between upstream's root-index frontmatter and catalog's body-line convention,
and catalog verifies against upstream first. It is declared once, at D5.
V-A5 is extended to the new profile — the one whose NAME is the place a
version literal would look natural, and it carries none.
542 tests, mypy --strict clean.
The gate refuses a schema that names both `timestamp` and `generated`,
because OKF §13.1 grants the legacy fallback only while `generated` is
ABSENT. `__post_init__` built the named set from `order | required |
allowed` and left `nullable` out, so
FrontmatterSchema(order=("timestamp", "title"),
nullable=frozenset({"generated"}))
constructed, and `violations()` on a document carrying both returned ().
The namespace is open by default, so a key named only by `nullable` is
admitted exactly as surely as an emitted one — the omission was a hole in
the gate, not a narrower gate.
Found by the proving consumer the same day the gate was written; their
construction is the red test case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Plan step 3's remaining half, and the one place steps 1-3 add behavior rather
than only characterizing it. Operator decision this session: build the gate NOW
rather than at D2, so the multi-file D2 work is developed with the invariant
already in place.
OKF section 13.1 reads `timestamp` as a legacy stand-in for `generated.at`
only while `generated` is ABSENT. A schema able to name both can therefore
describe a document with neither a valid `generated.at` nor an eligible
fallback. Refused at construction, the same shape as the reserved `verdict`
layer -- and one level below the sketch that was approved, on
`FrontmatterSchema` rather than `BundleProfile`: the schema is the policy that
names frontmatter keys, it is the exact C3 analog, and a profile must build its
schema first, so the stricter placement subsumes the other.
Stated as key names rather than as a judgement on values, deliberately: every
`generated` a schema can express today is a scalar (`_is_legal_value`) and so
malformed as a v0.2 mapping, which makes naming both exactly the hazard. It
narrows when a value model can express a well-formed `generated` -- recorded at
the constant rather than left for a reader to reconstruct.
Behavior-neutral for everything shipped, and not by assertion: DEFAULT emits
`generated` and no `timestamp`, STRICT_V1 the reverse (which is what puts the
wiki on the legacy path rather than in a defect), and the byte-exact golden
suite is green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
`IndexPolicy` grew five judging fields and `IndexPolicy.violations`, closing the
gap a test has been pinning since `848e395`. `STRICT_V1.index` is now the proving
consumer's shape rather than DEFAULT's.
The design was settled by a conflict rather than by a preference. The convention
owner reported that an index is an authored count of a directory's children and
that a validator enumerating the directory has implemented the wrong contract.
Checked against the proving consumer before adopting it, the two turn out to be
directly opposed: gate BUNDLE_INDEX_COMPLETE (validate.py:1081-1120, ERROR) builds
its expected set by enumerating the directory and demands an exact bidirectional
match, and bundle.py:498-567 writes every index from a tree walk. Neither is
incoherent inside its own spec, so authored-versus-derived is a policy field in
both directions and a library invariant in neither.
Consequences encoded rather than documented: nothing here enumerates a directory
— the caller supplies the listing, `violations` refuses one when the profile's
index is authored and refuses to run without one when it is derived, so code
written to the wrong contract fails at the call instead of passing every test one
would think to write. Root and nested indexes are asymmetric (confirmed in both
consumers, different key sets). A per-entry description is template-level, so
`render_link` refuses both a missing description and an unwanted one.
DEFAULT keeps upstream's root-only index and judges nothing, for the same measured
reason it carries no required frontmatter key set: upstream binds `index.md` to
the bundle root alone, so a judging default would condemn conforming bundles.
25 new tests, 493 green. C1 re-proven: `git diff --stat examples/` empty.
Phase 3 step 3, frontmatter half. FrontmatterSchema grows the four judging
fields the proving consumer's contract needs — required, allowed, nullable,
key_pattern — and `violations` is what reads them, so no field lands as an
untested claim.
STRICT_V1 requires the eight keys their operator ratified on 2026-07-25 inside
the eleven-key allowlist that doubles as emission order, both derived from one
tuple so they cannot drift; `summary` is the single nullable key; a value is a
string scalar or a NON-EMPTY LIST OF STRINGS, because their Release pages carry
block lists and a scalars-only schema would reject pages they ship today.
Measured, not assumed: DEFAULT must NOT grow a required set. Door A emits seven
keys, Door B six, and Door C writes an imported concept verbatim with whatever
the sender wrote, so any required set on DEFAULT would declare invalid the
bundles this library itself produces. The plan's other cross-profile direction —
a wiki bundle rejected under DEFAULT — therefore has to come from index and path
shape, not from frontmatter. Recorded here because the plan implied otherwise.
Violations are reported, not raised, like TypeRejection, and sorted by
(key, code): a report that depended on mapping order would not be reproducible.
An unsatisfiable schema — a required or nullable key outside its own allowlist —
fails at construction.
Two boundaries kept explicit. `violations` judges an already-parsed mapping;
parsing strict frontmatter needs a parser this library does not have, which is
the same reason Door C writes verbatim. And STRICT_V1's index policy is still
DEFAULT's, pinned by a test as a known gap: per-directory indexes and an entry
description have no field on IndexPolicy yet, and nothing consumes a profile's
index policy today.
C1 re-proven: 443 existing tests unmodified and green (468 total), and
`git diff --stat examples/` empty.
Phase 3 step 1. What Phases 1 and 2 hard-coded about a valid bundle now
lives on one frozen `BundleProfile`, and `DEFAULT` states exactly the
ingest-spec v1 + Phase 2 contract. Nothing observable changes: the 425
existing tests are unmodified and green, and `git diff --stat examples/`
is empty, which is assumption C1's whole proof. Both oracles are real
rather than nominal — the golden suite compares `read_bytes()`, and Door
B pins its frontmatter block as an exact string.
Moved onto the profile, each one previously a constant with a reader:
the index name and its managed-link shape (template plus pattern, kept
honest by a round-trip test), the three filename namespaces
(`ingest-`/`inbox-`/`import-` and `.md`), the reserved `verdict` layer
(duplicated in `manifest` and `inbox` before this), the frontmatter key
order, and the `source_query` whitespace collapse.
Two details are worth naming. The DEFAULT key order spans both doors:
Door A's seven keys and Door B's six are subsequences of one nine-key
order, so a single canonical order reproduces both doors byte-for-byte.
And emission follows commons decision D1 — ordered prefix, then any
remaining keys sorted — which is the mechanism the proving consumer's
hash registry needs; under DEFAULT the tail is always empty.
`TypePolicy` refuses the reserved layer at CONSTRUCTION (assumption C3),
so a profile admitting `verdict` cannot be built, let alone passed to a
door. It reports refusals rather than raising them, because Door A
refuses with `ManifestError` and Door B with `MaterializationError` for
the same type — the wording and the stable code come from the policy,
the exception class stays each door's own.
Deliberately NOT moved, each for a stated reason: the required and
allowlisted key sets the proving consumer needs (no code reads them
until the STRICT_V1 validator exists, and a field nothing reads is a
claim nothing tests), the id grammar (a pattern the slugger derives a
separator class from, not a flat name), `NAME_MAX_BYTES` (a filesystem
fact, not a contract choice), and every disposition/origin/channel
vocabulary (guard territory, always). The profile is also not exported
from the package root yet — the optional `profile` argument on the
flows is step 3's plumbing change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
