Zero occurrences in the v0.2 SPEC.md, zero in the v0.1 SPEC.md at the parent of
the migration commit, no reference in upstream's reference_agent source, and one
file in the whole bundle set. An undocumented hand-authored key, legal under
§14's unknown-keys tolerance — a demonstration of the open namespace rather than
a field to support.
The same sweep settles '# Cited by': body prose in two policy files, absent from
the spec, and independently DERIVED by the visualizer (viz.html:56) from sources.
Both fit the upstream pattern already visible in credibility scores and trust
tiers — OKF stores signals, consumers derive views.
Answering this needed the v0.1 spec text, upstream's source, and a whole-tree
grep at once, which is what the local pinned clone made possible. That is U8's
lesson applied rather than restated.
Per-claim attribution moves to a markdown footnote keyed to a sources[].id
(SPEC §4.2 :224-225, §5.1 :347-360), and the shipped acme_retail bundle carries
it inside a schema table. Our supersession row stated the destination and not
the mechanism, so a consumer following it would migrate the list and drop the
claim->source binding.
Measurable at the pinned commit the whole time; surfaced by Google's launch post
2026-07-27. U1-U7 measured frontmatter in all four reference bundles and walked
past the bodies. V-A1's discharge is narrowed rather than reworded: the pin
holds, the reading's depth did not.
Consequence outside the doc: the v0.2 exposure report sent to the wiki states
the incomplete half under the heading that exists to size their cost. Correction
owed, same class as A-E3.
The operator chose V1(d) for OKF_V0_2, so the pilot expectation A-E3 was
written against — a producer-and-version actor — is stale. Corrected in place,
with the reason, before po-claude receives the tag rather than after they run.
V1's own section records what the choice does and does not settle: DEFAULT's
generated is commons' §5 layer and stays in their queue as item 8. TDD step 5
is marked done at c90171d.
The v0.2 profile lands additively: DEFAULT keeps stating commons' §5 layer
byte for byte (the golden suite is the proof, not the claim), and OKF_V0_2
adds the families v0.2 introduces on top of DEFAULT's key order.
Two questions the plan left open were the operator's, and both were decided
before code rather than discovered during it:
- `generated.by` is `process:llm-ingestion-okf` — plan V1's option (d), the
§7 process actor form. The value carries no version, which is what keeps a
byte-compared fixture stable across releases and leaves a shared
cross-implementation fixture possible. A-E3 was written against the
`<producer>/<version>` form and is now STALE in the pilot specification; the
correction is owed to portfolio-optimiser-claude before they run.
- `sources[].resource` is the manifest source's locator verbatim: the file
root, the sql connection_ref (an env-var NAME, never its value), or the http
base_url. `credential_ref` is not a locator and is never emitted.
Ownership becomes a policy on the profile rather than a literal in the gate.
The emitter and `_is_ingest_owned` are coupled through the stamp value, so
OwnershipPolicy is where they meet and can only change together. Recognition
is one-way by decision: OKF_V0_2 owns the v0.1 stamp too, so a DEFAULT-written
bundle re-runs IN PLACE (operator, 2026-07-27), while DEFAULT still refuses a
v0.2 file rather than replacing it — V-A3's fail-safe is preserved.
An unquotable locator is refused rather than emitted. Measured with PyYAML
rather than reasoned: `[{ id: a, resource: data, backup }]` raises nothing and
parses to a mapping with a `backup` key nobody wrote, so the failure mode is a
silently wrong provenance record. Validation, not repair.
Deliberately NOT here: the `okf_version` declaration. §12 makes it a MAY and
none of upstream's four reference bundles exercises it; WHERE it goes is open
between upstream's root-index frontmatter and catalog's body-line convention,
and catalog verifies against upstream first. It is declared once, at D5.
V-A5 is extended to the new profile — the one whose NAME is the place a
version literal would look natural, and it carries none.
542 tests, mypy --strict clean.
Upstream §12 puts okf_version in the root index.md's frontmatter;
catalog's own spec says an index.md has no frontmatter and carries the
marker as a body line. A bundle conforming to one is non-conformant
against the other, so the placement A-E6 leaves unstated is no longer
cosmetic. Their gate cannot settle it: the shape regex is unanchored and
matches in both places, so a green TEST B measures the value and never
the placement. D5 stays unfrozen until catalog has verified against
upstream; A-E6 becomes placement-explicit before the pilot receives it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Two corrections, both measured against guard 0.2.0 today rather than
reasoned.
Requirement 1: the T2 parser refuses every route to a mapping — the flow
form on the disallowed-indicator set, the block form on the nested-mapping
check, dotted keys on the key pattern. A v0.2 `generated` mapping has no
expressible form through Door C at all, so the chosen flow form is not
worse than the alternatives; there are none. The decision stands on its
original grounds. What changes is a claim: no round-trip through a
guard-gated import may be asserted. Door B never parses frontmatter, so
the grammar binds import, not emission.
A-E5: po-claude located a hole in A-E1. The emitter's `generated: true`
and the ownership predicate are coupled through that literal, and
OKF_V0_2 changes exactly it — one run compared byte for byte cannot see
the predicate stall, since that needs two runs into the same directory.
A-E5 now names the same-directory second run and expects it to succeed.
The separate in-place DEFAULT-to-v0.2 question is marked undecided rather
than quietly turned into an expectation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
The gate refuses a schema that names both `timestamp` and `generated`,
because OKF §13.1 grants the legacy fallback only while `generated` is
ABSENT. `__post_init__` built the named set from `order | required |
allowed` and left `nullable` out, so
FrontmatterSchema(order=("timestamp", "title"),
nullable=frozenset({"generated"}))
constructed, and `violations()` on a document carrying both returned ().
The namespace is open by default, so a key named only by `nullable` is
admitted exactly as surely as an emitted one — the omission was a hole in
the gate, not a narrower gate.
Found by the proving consumer the same day the gate was written; their
construction is the red test case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Caught while verifying facts for a consumer notice: the runbook stated in the
past tense that `profile` landed as a keyword-only argument with a DEFAULT
default. Measured -- no door takes the argument at all. The decision is real and
stays; the claim that it is in the code was about to be repeated to a consumer
as a property they could build on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
The plan's own line said steps 1-3 change no behavior. V-A7's construction gate
made that untrue, and a stale premise in a plan is the class of thing this repo
has already paid for twice. Recorded where it was wrong, with the operator
decision that put the gate before D2 rather than at it, and with Door C's six
tolerance cases measured rather than expected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Plan step 3's remaining half, and the one place steps 1-3 add behavior rather
than only characterizing it. Operator decision this session: build the gate NOW
rather than at D2, so the multi-file D2 work is developed with the invariant
already in place.
OKF section 13.1 reads `timestamp` as a legacy stand-in for `generated.at`
only while `generated` is ABSENT. A schema able to name both can therefore
describe a document with neither a valid `generated.at` nor an eligible
fallback. Refused at construction, the same shape as the reserved `verdict`
layer -- and one level below the sketch that was approved, on
`FrontmatterSchema` rather than `BundleProfile`: the schema is the policy that
names frontmatter keys, it is the exact C3 analog, and a profile must build its
schema first, so the stricter placement subsumes the other.
Stated as key names rather than as a judgement on values, deliberately: every
`generated` a schema can express today is a scalar (`_is_legal_value`) and so
malformed as a v0.2 mapping, which makes naming both exactly the hazard. It
narrows when a value model can express a well-formed `generated` -- recorded at
the constant rather than left for a reader to reconstruct.
Behavior-neutral for everything shipped, and not by assertion: DEFAULT emits
`generated` and no `timestamp`, STRICT_V1 the reverse (which is what puts the
wiki on the legacy path rather than in a defect), and the byte-exact golden
suite is green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Plan steps 1-3: characterization only, no production code, so the one part of
the v0.2 work we called risky -- `_is_ingest_owned`, the pre-mutation collision
gate -- is pinned before it moves.
Step 1. An inline flow mapping round-trips through the scalar
`parse_frontmatter` verbatim (V-A2); a block list does not, and the measured
key-space pollution (`- id`, `resource` arriving as frontmatter keys) is what
requirement 1 chose the flow form over. A v0.2 `generated` mapping never makes
a file ingest-owned (V-A3), with the v0.1 `generated: true` control alongside
so the refusal is attributable to `generated` and not to an unreadable
manifest reference -- and end to end, such a file is refused with
`collision_unstamped` rather than overwritten. Today's fail-safe becomes a
documented guarantee.
Step 2. Door C against the five OKF section 14 consumer MUST NOTs plus the
section 5.2 bare-`verified` mapping. Its tolerance is structural rather than
lenient: the door writes the guard's bytes verbatim and never parses the
sender's frontmatter, so a D1b reader that starts judging shape at this door
breaks these cases -- which is when we want to hear about it.
Step 3. No profile hard-codes an upstream version (V4/V-A5), walked
recursively through dataclasses and collections and falsified against a
planted literal; the key name is what a profile pins, and the root-frontmatter
policy accepts any value. The `okf_version` value is catalog's (E1), so a
constant here would be both a decision we do not own and the thing that would
have to be chased on every upstream release.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
U4 escalated tag requirement 1: real v0.2 frontmatter is block YAML, and "emit
accepts a block-list value" had been scoped against a list of strings when
upstream's canonical sources is a list of five-key mappings. Re-sized against
the code rather than against its tests, and the answer runs the other way.
Measured: emit (profiles.py:197) takes Mapping[str, str] and renders one
"key: value" line per key through _render (:211), verbatim but for
source_query's whitespace collapse; _is_legal_value (:97) admits a string or a
non-empty list of strings, never mappings; the only production call sites are
materialize.py:166 and inbox.py:129, both passing dict[str, str].
The earlier "discharged" note read the validator correctly and drew too much
from it. But the gap it left is not U4's either: A-E4 owes at least a resource,
not upstream's five keys, and a manifest source has no author, no last_modified,
and no bundle-internal resource in upstream's sense. A field with no reader is
not written. So the question was never how large a YAML emitter we need -- it
was which form sources takes.
Chosen: sources: [{ id: <id>, resource: <ref> }], the same inline flow form A-E3
already chose for generated. It keeps parse_frontmatter line-oriented, which
matters because a block list pollutes that parser with false keys (- id,
resource) -- the failure mode documented for po-claude -- and _is_ingest_owned,
the one piece of this tag we called risky, reads THROUGH that parser. The block
form would force the collision gate and the parser to be hardened in the same
session. It also satisfies commons' section 5 single-line MUST, so the DEFAULT
seam U4 opened stays closed, and section 11 asks for parseable YAML, not block
YAML. The deviation from upstream's emitted shape is real and is recorded as a
decision, like U1, not as compliance we may claim.
Consequences: requirement 1 is smaller than both earlier sizings -- zero emitter
work -- and D1a drops off the critical path to v0.5.0a1. A-E4 is corrected from
"block list" to the flow form before the pilot receives it; a test spec that
outlives its decision is the A-E6 defect class.
Swept the same defect class while here: the TDD order and three other passages
wrote okf_version: "0.2" QUOTED, contradicting A-E6, which is precisely the form
catalog measured as exit 1. All five now unquoted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Measured, after asserting the opposite to the security repo an hour earlier: no
door in this library can create, import, or persist an executable file. Door C
skips every suffix that is not .md (importer.py:226-231); Door B has no extractor
for .py in either the core or the optional set (extract.py:113,131-142); Door A
writes only ingest-*.md. We told the guard a third-party attester .py could reach
a consumer's tree through our door. It cannot. Reasoned, not measured, corrected
the same hour.
The real exposure is the inverse. Door C writes a merged concept verbatim,
frontmatter included, and under v0.2 that frontmatter can carry
executor.resource and attester.resource. So we import the reference to executable
code while never importing the code. Either the pointer dangles -- noisy, safe --
or it resolves to something else already in, or later added to, the destination
tree, and the imported concept references code it never shipped. That second one
is a substituted reference, not a broken one, and it looks valid.
Also recorded: the guard is a PERSIST gate, and executable code carries its risk
at EXECUTION. A .py entirely safe to store may be dangerous to run, so no persist
gate covers the class section 10 introduces, and the execution boundary has no
owner in this stack -- upstream deferred the attester ABI and sandboxing, and we
declared runtime out of scope on that same deferral. Written down because "the
guard covers it" is a reasonable conclusion to draw and a hard one to catch later.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Operator directive: when the v0.2 implementation is ready, every OKF-consuming
repo is told through the coord mailbox what to do or test. That was a step title
in the runbook and is now a table with content -- nine repos, each with the
action asked of them and, more importantly, what happens if they do nothing.
For most the honest answer is "nothing changes", and saying that plainly is more
useful than an open invitation to review. That is the black-box commitment made
checkable rather than asserted.
Two new GA exit criteria: every repo in the table has been sent its message and
the table was re-derived rather than reused; and the security repo's section 10
boundary questions are either answered or their being open is recorded as an
accepted, stated limit. The second matters because Door C is the path an
external bundle's executable attester would take into a consumer's tree.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
The verification criterion said the release checklist carries an
upstream-version re-check. It now names the procedure that check runs, which is
the part that makes it more than a reminder.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
The standing always-latest policy had no procedure, and v0.1 -> v0.2 showed what
that costs: the spec was read, the shipped example bundles were not, and five
findings a spec reading had settled wrongly surfaced only after someone asked
whether we had studied the examples. One of them re-sized the release itself.
docs/upstream-okf-upgrade-runbook.md is that procedure. Every step names the
concrete failure it prevents, and each is a failure that actually happened:
- Pin before reading. Two repos read main independently, which felt like two
confirmations and was one unstable reference read twice -- and SPEC.md was
edited after the migration commit, so neither alone was the whole answer.
- Enumerate the whole okf/ tree. bundles/ held the real examples and was on
nobody's list until the tree was listed.
- Read the shipped examples, not only the normative text. A directory named
samples/ turned out to be a runner config, not samples of the format. And
markdown-converting fetch tooling strips exactly the byte-level properties
(BOM, trailing newline, quoting) that break downstream gates.
- Name fixtures by repo AND path. Two repos had examples/ingest-golden-file/
holding different fixtures with zero content overlap, and a pilot baseline was
agreed by a name that pointed at two files.
The black-box commitment is now an invariant in CLAUDE.md rather than an
emergent property: a consumer's cost for an upstream release is a re-run.
Additive profiles, byte-stable existing profiles, keyword-only new parameters so
positional call sites survive, no churn in consumer fixtures. The boundary is
stated rather than glossed -- shape changes are absorbed, upstream changes to
consumer-authored content are not, and those get a measured exposure report per
consumer instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Two corrections to claims this repo made, both measured rather than argued.
1. `examples/ingest-golden-file/` names TWO different fixtures. Ours holds
ingest-orders.md + ingest-products.md; po-claude's holds ingest-costs.md +
ingest-edge.md; the manifests differ at line 3 char 41; content-file overlap
is zero and the only shared filename, index.md, is byte-different. We agreed
a pilot baseline by name, and the name pointed at two files. A-E1 is the stop
condition, so a mismatched baseline would have surfaced as a profile defect.
The pilot runs THEIR fixture -- A-E1 asserts byte-identity with what their
pinned version produces, which is a claim about their baseline, not ours --
and the report must state repo and path.
2. "The shared fixture, ratified by two independent implementations" was false.
We took it from an incoming message and passed it on the same day without
measuring, while holding a standing rule that incoming messages are premises,
not facts, and applying that rule only to spec readings. Commons caught it.
The :29 argument in V1 survives in commons' stronger form: the clause requires
reproducing "the shared golden extractions", the definite article presumes one
set where there are two, so it has no referent today. (b) therefore does not
break a shared fixture we have -- it cements that one can never arise, since a
producer-bearing field inside the byte-compared prefix makes convergence
impossible by construction. The :280 half stays observable and fires on our next
release regardless.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
V-A1 is discharged: everything is now read at pinned commit 3fcbb9f, not off
main. v0.2 itself landed in 780fe9d; SPEC.md was edited after it, so "the v0.2
commit" and "the current spec text" are different objects.
Session 19 read the spec. It never read the four bundles the tooling migrated
to v0.2, and reading them corrected things a spec reading had settled wrongly.
okf/samples/ga4_merch_store is not a bundle -- it is a runner config that
generates one, so it says nothing about output shape.
- U1 No reference bundle declares okf_version. All four root index.md files
have no frontmatter at all. A-E6 is therefore a stricter posture than
upstream's own implementation -- defensible, but a decision rather than an
assumption. po-claude's "deviation" #2 was conformance. catalog's gate reports
MISSING on every official Google bundle.
- U2 Root index.md is the ONLY index permitted to carry frontmatter (12).
Constrains the fixture and the index policy.
- U3 `generated` is used for HUMAN-authored content upstream, so it cannot be a
machine-generated or ownership predicate. Empirical confirmation of commons'
axis finding, stronger than their spec-text argument.
- U4 Real v0.2 frontmatter is multi-line block YAML: flow sequences, block lists
of flow mappings, nested mappings, block lists of multi-key mappings, booleans,
dates. Requirement 1 was scoped against a list of strings; the canonical shape
is a list of five-key mappings. The one-session sizing must be redone.
- U5 7's actor convention gives `<producer>/<version>` for tools, and upstream
uses it. That is real evidence for (b), against the (d) we recommended to
commons today. Sent to them as a correction.
- U6 Conformance and the consumer-tolerance MUSTs quoted verbatim, including the
bare-`verified`-as-one-element-list requirement on D1's reader.
- U7 An undocumented `not:` family observed in a bundle, unresolved, recorded so
it is not mistaken for checked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
We carried (b) -- generated.by naming this library and its version. Commons
found it collides with ingest-spec's conformance clause: :29 requires a
conformant implementation to reproduce the shared goldens byte for byte, and
`generated` sits inside the mandatory ordered prefix. A producer's name and
version inside a shared fixture means no sibling implementation can reproduce
it, and Golden regression fires on every version bump with no contract change.
They measured it as normative-only for them, since examples/ingest-golden-*
does not exist in commons. Measured here, it is observable today: four fixtures
carry `generated:` on line 8 inside the byte-compared prefix, and
ingest-golden-file is both the shared fixture against the sibling
implementation and the baseline po-claude runs the pilot against.
So we back (d) -- `process:<fixed id>` per v0.2 section 7. Second reason,
visible only from this side: it keeps _is_ingest_owned an equality check.
Under (b) the collision gate must either prefix-match and ignore the version --
parsing a value we had just made structured -- or stop recognizing files an
earlier version wrote, which fails safe but breaks re-running a manifest across
a version bump.
The cost of (d) is recorded rather than minimized: producer identity leaves the
bundle, and a log does not travel through Door C. The decision stays commons'
to ratify.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
All three pilots accepted. Their measurements land in the plan now rather than
as failed runs later, which is the entire return on asking before building.
Six corrections, each measured by the consumer rather than reasoned by us:
- A-E6 specified `okf_version: "0.2"` quoted. Catalog measured the quoted form
failing their shape regex with exit 1, and a UTF-8 BOM making the marker
invisible while still exiting 0. Both now stated as raw-byte expectations.
- A-E2 said the diff lies only *inside* the frontmatter block. No bundle whose
root index has no frontmatter block can satisfy that together with A-E6.
- B-E3 covers one of catalog's three gates; the other two cannot see our
fixture. Recorded so the report is not read as broader than it is.
- Our claim that no gate keys on the version value was wrong as formulated:
check-okf-parity.mjs puts the raw value in its signature. It compares without
judging -- verdict-invariant, not value-blind.
- C-E1's ground truth is 524 non-reserved documents, not 522; 522 was correct
at d2c12d2.
- `materialize_bundle` gains keyword-only `profile=DEFAULT` on po-claude's
request. The signature already has a `*`, so their three-positional call path
stays source-compatible and additivity becomes a property of the signature
instead of something a consumer measures.
Also recorded: po-claude closed their index-parser defect themselves (f41264f),
so do not design around it; the assumption under it -- "a generated index has no
frontmatter" -- is not peculiar to them. Dropping the superseded `timestamp`
draws one catalog warning per concept file, decided after the pilot, not before.
The estimate's open caveat is discharged by reading profiles.py:197: emit is one
formatting path, the validator already admits list values, so only the emitter
side is missing and the sizing stands.
CLAUDE.md gains a standing constraint: consumer content stays at form level in
public files. The wiki's bundle is private pending an Anthropic ToS assessment,
their report goes back through coord, and a push is not reversible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Walking each pilot test back to what it actually exercises produces a result
the deliverable ordering had obscured: none of the three needs D1's
structured reader.
Test A emits v0.2 -- `generated` is an inline flow mapping, a formatted
string, and `sources` needs the emitter to accept a block-list value. Neither
requires reading one back. Test B runs catalog's gate on Test A's output.
Test C validates the wiki's v0.1-shaped documents against a v0.2 variant
profile; their documents carry no v0.2 families, so block-list reading is
never reached.
The structured reader exists to consume third-party v0.2 bundles -- D3, Door
C -- and no pilot repo sends us one. So D1 splits: the emitter half is in the
tag, and the reader half moves behind the pilot where the feedback can inform
it. Two consequences worth stating plainly.
V2 is off the critical path. It chooses the reader's design, and the reader
now lands after the pilot; the tag is not waiting on a decision.
The tag is roughly a session of work, not a phase: an emitter change, a
profile constant following the existing STRICT_V1 pattern, a profile-aware
ownership predicate, `sources` derivation, one generated fixture.
Risk concentrates rather than spreads. `_is_ingest_owned` is the pre-mutation
collision gate, so a defect there is expensive and quiet -- its
characterization test is written before it is touched. The emitter change
runs through DEFAULT's path, so V-A6 is proven by the golden suite rather
than assumed.
The sizing itself carries a caveat in the doc: it treats
FrontmatterSchema.emit as a single formatting function, inferred from its
tests rather than read. That is a premise like any other and gets verified
before the estimate is relied on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
The pilot invitations asked for feedback without specifying what to run or
what we expect to see. That returns a description, not a measurement: only a
stated expectation lets someone else's run falsify our model instead of
merely confirming it.
Each of the three tests now carries a numbered procedure, numbered expected
outcomes, and an explicit list of what would surprise us. Naming the
surprises is the load-bearing part -- a consumer who sees something odd but
passing otherwise has no reason to mention it.
Test A (producer, portfolio-optimiser-claude) runs one already-working
manifest twice with the same explicit ingested_at, once under DEFAULT as
baseline and once under OKF_V0_2, and diffs. A-E1 is the stop condition: if
the DEFAULT run is not byte-identical to their current pinned output, we have
broken a v0.1 consumer and the pilot ends there. The named surprises include
`at` differing from the ingested_at they passed, which would mean a
wall-clock crept in, and a collision refusal on a file that IS theirs, which
is the inverse of the fail-safe we predicted -- we expect foreign files to be
refused, so a false refusal of their own is the defect.
Test B (gate, catalog) runs their unmodified gate on the v0.2 fixture and on
a 0.1 control. B-E3 -- that no gate other than the version gate behaves
differently between the two -- is what converts our reading of their form
regex into a measurement. A membership list or equality comparison anywhere
in their chain is exactly what this is paid for.
Test C (expressiveness, wiki) is run by us, read-only at a recorded commit,
and the report goes to them. What we ask them to check is the part we cannot
measure from outside: whether a field we called an optional addition is in
fact load-bearing in their pipeline, and whether "no change required" holds
operationally rather than only formally.
Feedback gains a per-expectation verdict line so three independent runs are
comparable and a disagreement is located rather than merely reported. The
request also says outright that a wrong expectation is a better result than a
clean run, since a clean run only confirms what we already believed.
Inputs come from this repo at the pre-release tag; nothing is transported
through the mailbox except the specification.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Operator directive 2026-07-26: consumers are better served by getting the
latest version early and reporting back than by us holding it until we judge
it finished.
This closes a real gap. Every test in the plan -- including V-A8 against
upstream's reference implementation -- asks whether output is conformant.
None asks whether it is usable: whether a bundle is awkward to construct,
whether a rejection message is actionable, whether the profile can express
what a consumer's actual data needs. Only real data surfaces that.
Pilot set is three repos, one axis each, chosen for signal:
portfolio-optimiser-claude for the producer path (one real manifest run),
catalog for gate acceptance (their gate on our fixture, measured rather than
inferred from the regex), and claude-code-llm-wiki for expressiveness across
522 real documents.
The load-bearing design detail is that most of the pilot is read-only on our
side: we run the v0.2 reader over real consumer bundles and send the report.
That needs no adoption, no writes into their trees (O2 holds), and no change
to a contract their operator ratified. Only the producer axis asks a
consumer to do anything, and it asks for one run.
Excluded with reasons rather than silently: okr (Node side not yet lifted),
linkedin-studio (v0.2's provenance families would put implicit pressure on
the ingest/published carve-out we agreed not to normalize), commons (they
are deciding V1 -- a participant, not a test site).
Shipping a provisional surface without owing stability rests on three rules,
not on saying "provisional": OKF_LATEST does not point at v0.2 until GA, so
flipping it is the GA event rather than a merge side effect; the docstring
and CHANGELOG name the pilot repos; and breaking changes during the pilot
get no deprecation cycle. Stating that last one up front is what buys the
freedom to act on feedback -- discovering it later is what turns a pilot
into a de-facto release.
Feedback is requested in five named parts, because unstructured reports are
not comparable across three repos. The one that matters most is "what was
awkward but worked": workarounds are the highest-value signal and never
appear as a failure.
GA exit criteria are testable, and carry one honest limit: a three-repo
pilot exercises only what those three use, so `sources` with usage_window,
multi-verifier `verified`, and Attested Computation will likely go
unexercised. Those stay marked provisional at GA instead of being silently
promoted -- claiming otherwise would be the same unearned-claim pattern that
"conform first, claim after" exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Operator directive 2026-07-26: the library always supports the current
latest version of Google OKF. v0.2 shipped 2026-07-25, so v0.2 support is
committed work rather than something a consumer has to request. This
overrides the previous default answer to open question V3 ("no until a
named consumer asks"), which is kept in the plan marked superseded so the
override reads as deliberate.
Support is additive: a new profile, never a migration of the existing two.
That single design choice is what makes an always-latest policy sustainable,
and it resolves the tension the directive would otherwise create with three
constraints that do not yield to it:
- DEFAULT states commons' ingest-spec section 5 layer, so its `generated`
shape is commons' call. Under the additive design this stops blocking us,
which takes commons off the critical path.
- STRICT_V1 mirrors the proving consumer's ratified contract; changing
another repo's contract from here would violate O2.
- v0.2 defers the attestation receipt and verdict wire formats upstream, so
the format is supportable and the unspecified runtime is not. It re-enters
scope when upstream specifies it.
This is also the first time the phase-3 profile abstraction is forced by
something outside this repo rather than by a second consumer, which is the
better test of whether the seam was cut in the right place.
Deliverables D1-D6 replace the earlier decision-round framing: a frontmatter
model that can carry block lists (`sources`, multi-verifier `verified`), an
OKF_V0_2 profile plus an OKF_LATEST alias whose moving-target tradeoff is
documented rather than hidden, Door C conformance against the consumer
tolerance rules, `Attested Computation` round-trip, v0.2 golden fixtures, and
a release-checklist re-check so the standing policy cannot decay silently.
Two new assumptions carry the weight. V-A7 forbids any profile from emitting
`timestamp` together with a malformed `generated`, since that combination
would have neither a valid `generated.at` nor an eligible section 13.1
fallback. V-A8 validates our own v0.2 fixture against upstream's reference
implementation, because every other test in the suite only asks whether we
agree with ourselves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Everything built so far targets OKF v0.1. Upstream published v0.2 on
2026-07-25, so the plan records how the library relates to it and, more
importantly, who owns each decision.
Read from the spec itself rather than secondhand, which corrected two
readings that a summary had gotten wrong:
- STRICT_V1's `timestamp` is NOT a defect. Section 13.1 grants consumers a
documented fallback to legacy `timestamp` precisely when `generated` is
absent, and STRICT_V1 emits no `generated`. Nothing is asked of the wiki.
- The one measured shape problem is DEFAULT's `generated: "true"`, because
v0.2 requires `generated.by` within `generated`. That key was not reserved
in v0.1, so it was legal when written; v0.2 claimed the name. DEFAULT
states commons' ingest-spec section 5 layer, so the fix is commons' call
and is raised there as open question V1 rather than patched locally.
Two findings shrink the work. The canonical form for `generated` and a
single `verified` is an inline flow mapping, which the existing scalar
parser already round-trips as an opaque string, so block-list support is
only needed for `sources` and multi-verifier `verified` -- and only if a
named consumer asks. And the collision degrades safely: `_is_ingest_owned`
returns False for a v0.2 mapping, so a foreign concept is refused rather
than overwritten.
The track sits between Phases 3 and 4 because Phase 4 freezes the
cross-runtime contract. Freezing a v0.1 shape into two runtimes would let
the shared fixture suite certify the drift instead of catching it.
Self-imposed rule, since the spec does not require it: conform first, claim
after. Declaring `okf_version: "0.2"` is a MAY with no conformance
checkpoint, so claiming it early would be permitted -- and would be the same
class of true-sounding misleading claim as reporting a 0.2.0 measurement
under a 0.3.1 heading.
Also moves the guard 0.3.1 measurement procedure out of session state and
into execution-order.md, where it belongs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
They named both on a live channel today, having withdrawn the half of their
original question that turned out not to bind them. Recorded here rather than in
a reply so the scope freeze has to answer them explicitly.
`write_index` ownership was genuinely ambiguous in this plan — they read
deliverable 4's "write" and asked which line owns it. It is deliverable 3, and
the plan now says so.
Their vocabulary requirement is already the Python half's shape:
`TypePolicy(allowed=None)` is the default and rewrites nothing, so there is no
DEFAULT_TYPE to snap to. The reference checker's collapse is a known deviation to
resolve with the convention owner under step 1, not a behaviour to lift.
`IndexPolicy` grew five judging fields and `IndexPolicy.violations`, closing the
gap a test has been pinning since `848e395`. `STRICT_V1.index` is now the proving
consumer's shape rather than DEFAULT's.
The design was settled by a conflict rather than by a preference. The convention
owner reported that an index is an authored count of a directory's children and
that a validator enumerating the directory has implemented the wrong contract.
Checked against the proving consumer before adopting it, the two turn out to be
directly opposed: gate BUNDLE_INDEX_COMPLETE (validate.py:1081-1120, ERROR) builds
its expected set by enumerating the directory and demands an exact bidirectional
match, and bundle.py:498-567 writes every index from a tree walk. Neither is
incoherent inside its own spec, so authored-versus-derived is a policy field in
both directions and a library invariant in neither.
Consequences encoded rather than documented: nothing here enumerates a directory
— the caller supplies the listing, `violations` refuses one when the profile's
index is authored and refuses to run without one when it is derived, so code
written to the wrong contract fails at the call instead of passing every test one
would think to write. Root and nested indexes are asymmetric (confirmed in both
consumers, different key sets). A per-entry description is template-level, so
`render_link` refuses both a missing description and an unwanted one.
DEFAULT keeps upstream's root-only index and judges nothing, for the same measured
reason it carries no required frontmatter key set: upstream binds `index.md` to
the bundle root alone, so a judging default would condemn conforming bundles.
25 new tests, 493 green. C1 re-proven: `git diff --stat examples/` empty.
Phase 3 step 3, frontmatter half. FrontmatterSchema grows the four judging
fields the proving consumer's contract needs — required, allowed, nullable,
key_pattern — and `violations` is what reads them, so no field lands as an
untested claim.
STRICT_V1 requires the eight keys their operator ratified on 2026-07-25 inside
the eleven-key allowlist that doubles as emission order, both derived from one
tuple so they cannot drift; `summary` is the single nullable key; a value is a
string scalar or a NON-EMPTY LIST OF STRINGS, because their Release pages carry
block lists and a scalars-only schema would reject pages they ship today.
Measured, not assumed: DEFAULT must NOT grow a required set. Door A emits seven
keys, Door B six, and Door C writes an imported concept verbatim with whatever
the sender wrote, so any required set on DEFAULT would declare invalid the
bundles this library itself produces. The plan's other cross-profile direction —
a wiki bundle rejected under DEFAULT — therefore has to come from index and path
shape, not from frontmatter. Recorded here because the plan implied otherwise.
Violations are reported, not raised, like TypeRejection, and sorted by
(key, code): a report that depended on mapping order would not be reproducible.
An unsatisfiable schema — a required or nullable key outside its own allowlist —
fails at construction.
Two boundaries kept explicit. `violations` judges an already-parsed mapping;
parsing strict frontmatter needs a parser this library does not have, which is
the same reason Door C writes verbatim. And STRICT_V1's index policy is still
DEFAULT's, pinned by a test as a known gap: per-directory indexes and an entry
description have no field on IndexPolicy yet, and nothing consumes a profile's
index policy today.
C1 re-proven: 443 existing tests unmodified and green (468 total), and
`git diff --stat examples/` empty.
The proving consumer answered Q1-Q4 on 2026-07-25, each measured or read out
of their code that day. No row in the mapping moved. Three of the four answers
carry a sharper reason than this table drafted, and one reverses a correction
this table made a day earlier.
Q2 is the substantive change. This table had corrected the plan's eight-key
required set down to four, against the consumer's REQUIRED_KEYS constant.
Their operator ratified eight: the constant is their emit-path minimum, not
their contract, and all eight keys are present on 522/522 of their documents.
STRICT_V1 is therefore deliberately stricter than the consumer's own
validator; if they ever ship a class lacking source_sha they bump the profile
rather than bend the bundle, and they say so before shipping it. The eleven-key
allowlist is unchanged and still a different thing from the required set:
version/date are layer-specific and summary is the one nullable key, so all
three stay optional-when-absent.
Q1, Q3 and Q4 keep their Consumer/Guard assignments, now with the reasons that
justify them: gate 5 escalates on run state (disposition == verbatim) rather
than document class, so a profile structurally cannot see it; gate 4's
changelog rule is a vendor slug list that is not scoped to concepts/ at all;
gates 19 and 22 need external state, one of them by construction never
blocking. Assumption C2 is confirmed at 25 gates and the index shape is
confirmed non-negotiable.
Both reviews - the consumer's and the operator's - are now in, so the port may
begin. The Guard column is unaffected by all of it.
Phase 3 step 1. What Phases 1 and 2 hard-coded about a valid bundle now
lives on one frozen `BundleProfile`, and `DEFAULT` states exactly the
ingest-spec v1 + Phase 2 contract. Nothing observable changes: the 425
existing tests are unmodified and green, and `git diff --stat examples/`
is empty, which is assumption C1's whole proof. Both oracles are real
rather than nominal — the golden suite compares `read_bytes()`, and Door
B pins its frontmatter block as an exact string.
Moved onto the profile, each one previously a constant with a reader:
the index name and its managed-link shape (template plus pattern, kept
honest by a round-trip test), the three filename namespaces
(`ingest-`/`inbox-`/`import-` and `.md`), the reserved `verdict` layer
(duplicated in `manifest` and `inbox` before this), the frontmatter key
order, and the `source_query` whitespace collapse.
Two details are worth naming. The DEFAULT key order spans both doors:
Door A's seven keys and Door B's six are subsequences of one nine-key
order, so a single canonical order reproduces both doors byte-for-byte.
And emission follows commons decision D1 — ordered prefix, then any
remaining keys sorted — which is the mechanism the proving consumer's
hash registry needs; under DEFAULT the tail is always empty.
`TypePolicy` refuses the reserved layer at CONSTRUCTION (assumption C3),
so a profile admitting `verdict` cannot be built, let alone passed to a
door. It reports refusals rather than raising them, because Door A
refuses with `ManifestError` and Door B with `MaterializationError` for
the same type — the wording and the stable code come from the policy,
the exception class stays each door's own.
Deliberately NOT moved, each for a stated reason: the required and
allowlisted key sets the proving consumer needs (no code reads them
until the STRICT_V1 validator exists, and a field nothing reads is a
claim nothing tests), the id grammar (a pattern the slugger derives a
separator class from, not a flat name), `NAME_MAX_BYTES` (a filesystem
fact, not a contract choice), and every disposition/origin/channel
vocabulary (guard territory, always). The profile is also not exported
from the package root yet — the optional `profile` argument on the
flows is step 3's plumbing change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Their answer crossed with the draft rather than responding to it, so the
two readings of the same validator are independent. They agree on every
substantive point -- four required keys, three reserved basenames,
summary as the only nullable key, the index structure, the closed sets --
and on the column assignments. That is corroboration, not confirmation
bias, and it is worth more than either reading alone.
Four refinements folded in:
- Emission order is an ordered prefix followed by remaining keys SORTED,
not just an ordered list. This is the commons D1 outcome and it is what
keeps regeneration byte-identical; normalising key order any other way
breaks their BUNDLE_HASH_REGISTRY gate, which is why D1 mattered.
- Values may be a non-empty LIST of strings, not only scalars. Same block
-list shape Door C already refuses to round-trip, which is why imported
concepts are written verbatim -- a scalars-only parser would destroy
their Release pages, and a scalars-only profile would reject them.
- Non-required allowlisted keys are validated against closed sets WHEN
PRESENT, not required to be present.
- Internal-link resolution generalises only because path_policy owns the
layer layout; noted on the gate row as their caveat, accepted.
Gate 5's row now distinguishes the Guard COLUMN (functionality that is
the guard's and is never reimplemented) from the call SITE (theirs, in
their acquisition stage) -- the one place our wording and theirs differed.
Both arrived within minutes of the v0.4.0 announcement and both change
what we would otherwise have built or claimed.
CATALOG -- a profile field we must NOT add. Their steg 5 measured bundle
placement across four consumers: exactly one installs a bundle anywhere,
the other three take the bundle directory as a per-run argument with no
default, no discovery and no $HOME expansion. Placement is unmandated, so
BundleProfile gets no root-location or install-path field, "no
installation root" is a conforming consumer shape, and write policy must
not be coupled to placement. Recorded in the split table, which also now
spells the proving consumer's layer prefixes bundle-relative
(concepts/, releases/) rather than repeating the bundle/ prefix their own
repository layout carries -- that prefix was placement leaking into a
contract.
COMMONS -- our "Fixed" framing outruns their frozen text. Removing every
ingest-stamped file is verbatim what ingest-spec v1 section 5 mandates:
v1 assumes one manifest per bundle and defers the multi-manifest case as
a named extension point. So v0.4.0 implements that extension point rather
than fixing a defect against the spec. The mechanic survives ratification
either way -- stem matching follows from their own {stem}@{h} stamp
definition, since {h} changes on every content edit -- but the prose is
queued for amendment and is not ratified, so the changelog now says so
instead of claiming conformance it does not have.
Phase 3 starts with a written mapping of the proving consumer's gates to
exactly one of Profile / Guard / Consumer, reviewed before any porting.
Drafted here against claude-code-llm-wiki's validate.py and bundle.py at
their 9ee5a8e, which is their single gate entrypoint, so the enumeration
is exhaustive against that commit and no other.
Reading the validator corrected two things this plan asserted before it
had been read, both of which would have produced a wrong STRICT_V1:
- Required frontmatter keys are FOUR (type, title, description,
timestamp), not the eight the plan listed. Those eight were a
truncation of FRONTMATTER_KEY_ORDER, which is an eleven-key allowlist
doubling as canonical emission order. frontmatter_schema therefore
needs three fields -- required, allowlisted, ordered -- not one
required/optional pair.
- Reserved files are THREE basenames, not two: README.md is reserved
like index.md and log.md but in a third way, present and exempt from
the document gates while still content-scanned. A two-valued
required/required-absent policy cannot express it.
Two of the 25 gates bundle checks from different columns
(LLM_OUTPUT_LINT, BUNDLE_URL_ALLOWLIST). They are decomposed per check
rather than assigned wholesale to a dominant column, which would have
moved the residue to the wrong side of the boundary.
Also recorded: index policy is the widest gap between the two profiles
(per-bundle vs per-directory, two link formats, append-if-absent vs
exact match, root index with or without frontmatter), and C4 is
confirmed with the operator -- profiles in code, no config-file loading
in v1.
Four open questions are addressed to the proving consumer in the
artifact; no porting until they and the table come back reviewed.
The v0.4.0 release commit claimed the [tool.uv.sources] entry "never
reaches consumers". Measured against the pushed tag in clean
environments, that is wrong in one direction and understated in the
other:
- uv, installing this package from git, DOES pick the guard up from the
tag automatically. uv reads pyproject.toml when it builds from the
source tree, so the source applies to the consumer too, not just to
development here.
- pip does not read it, and fails outright: "No matching distribution
found for llm-ingestion-guard<0.3,>=0.2". The guard must be installed
from its own tag FIRST. Verified that the guard-first order then
installs cleanly and that the shipped adapter runs against the real
guard in that environment.
What did hold is the part that matters for the pin: the built wheel
carries `Requires-Dist: llm-ingestion-guard<0.3,>=0.2`, so the range is
still the declared dependency and the git URL is still a channel.
README now states the pip order as the default instruction, with the uv
shortcut beside it, because a consumer hitting the resolution error would
otherwise read it as a broken release.
Phase 2 ships: Doors B and C, the extraction registry, the guard adapter,
and with them the first and only runtime dependency this library takes.
The minor bump is the news consumers are waiting on -- installing the
package now installs llm-ingestion-guard>=0.2,<0.3, though importing it
still does not import the guard.
Two behaviour changes carried in the same release are called out in the
changelog rather than buried: an extraction title containing `[` or `]`
now fails at manifest load (ingest-spec section 4), and the section 3
ownership scan no longer deletes files another manifest stamped into the
same bundle.
uv.lock was stale -- it predated the guard dependency and could not be
regenerated, because the guard is not on a package index. A
[tool.uv.sources] entry pins the same tag the README documents, so
`uv lock`/`uv sync` resolve. It is uv-specific dev metadata: the built
wheel still carries `Requires-Dist: llm-ingestion-guard<0.3,>=0.2`,
verified against the wheel, so the range remains the pin and the git URL
remains a channel.
Binary extraction (pdf/docx/xlsx behind [extract]) is deliberately not in
this release and is stated as outstanding in both README and changelog.
425 tests green; ruff, ruff format and mypy --strict clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
Both were wrong, and the convention owner corrected them: this library is not
a marketplace plugin, so there is no catalog ref to re-pin -- the integration
is a registry entry in the catalog's parity gate. And the catalog's vendored
okf-check.mjs is not byte-identical to okr's (~141 differing lines of code, an
intentional layer-1 floor), so it cannot serve as a conformance oracle. Parity
is behavioural over the spec section 3 floor in default read-mode, per file,
measured by that gate over an adversarial corpus.
Adds llm_ingestion_guard>=0.2,<0.3 as this library's first and only runtime
dependency, and guard_adapter.py -- the one module that imports it. The flows
themselves are unchanged: they still take an injected gate, and importing the
package still does not import the guard, so a Door A consumer is unaffected by
the dependency's state.
Door B screens the exact bytes it persists. The guard's §6 bookends
(prepare_input -> model -> screen_output) assume a model call in between; this
library makes none, and prepare_input returns prompt-shaped text (sanitized
AND spotlight-fenced with a per-call nonce) that must never reach disk. So the
adapter calls screen_output alone, on the extracted text as it stands, and
hands that same text back -- the verdict is then a statement about the bytes
actually written. This supersedes the plan's "bookends" wording, recorded
there under "Settled during implementation (step 4)".
It follows that the gate refuses rather than repairs: a file carrying an
invisible carrier is rejected, not stripped and persisted. Sanitizing first
would write a document differing invisibly from the operator's file while
source_sha256 still points at the original bytes. Operator decision; the
policy is PRESET_USER_UPLOAD, so any finding at all is held back.
Door C hands the bundle over whole to okf.import_bundle, which resolves the
cross-link graph across concepts. Per-concept reasons are derived from the
scan findings (severity:label) because stamp_concept keeps the disposition and
drops the reason strings behind it.
Assumption B1 closes as a signature smoke test over what the adapters actually
call -- screen_output and okf.import_bundle signatures, the Disposition values
both doors compare by value, the Origin/Channel vocabularies Door C validates,
the result fields read, and the upload preset's shape. prepare_input is not
pinned: drift there cannot reach this library. Behaviour is pinned against the
real scanner too, including the persist-gate proof that a fail-secure fixture
leaves the bundle byte-identical.
B2 closes with it: git+https tag install over anonymously readable HTTPS, no
credential. A direct reference is an install-time channel, not the pin -- the
range stays in pyproject, is satisfied by the tag install today, and resolves
normally once the package index exists. A packaging test enforces that the
guard remains the only runtime dependency (verified by hand-mutation).
Reads an external OKF bundle as {bundle-relative path -> document text},
hands it WHOLE to an injected gate over the guard's okf.import_bundle (a
bundle-level call: it resolves the cross-link graph across concepts), and
merges only concepts clearing the non-blocking floor. Same injection pattern
as Door B, so the core stays dependency-free while the CI channel for the
real guard is settled.
Two constraints shaped the design and are pinned by tests:
- A merged concept is written VERBATIM. Stamping provenance into it would
require round-tripping its frontmatter through this library's line-oriented
parser, which cannot represent the block lists the guard's parser accepts --
silent data loss -- and would persist bytes the guard never screened.
- Ownership is therefore proven by content identity: identical bytes at the
target name are a no-op re-merge (re-import of an unchanged bundle is
idempotent), and anything else at the name is refused. Curated content and
an updated concept are refused alike; refusing is what never destroys.
The floor is fail-closed beyond the plan's "no error" wording: an error, an
unrecognised disposition, and a concept the gate returned no verdict for are
all refusals. quarantine_review stays its own bucket, as at Door B.
origin/channel are validated against the guard's pinned vocabulary -- it
derives trust from origin by enum identity, so an unrecognised string would be
silently downgraded rather than caught.
Three primitives promoted for reuse rather than duplicated:
reduce_to_id_grammar and check_filename_length to materialize.py, and
extract.decode_text. Door C slugs the WHOLE concept path, so tables/users.md
and views/users.md stay distinct. Concept discovery folds case explicitly
rather than globbing *.md, whose case-sensitivity follows the filesystem and
would import the same bundle differently on APFS and ext4.
README's "what is gated today" section corrected: it claimed nothing is gated,
which is no longer true, but the honest statement is narrower than "the doors
are gated" -- the library cannot verify that an injected adapter is a real
guard, and a permissive stub is believed.
405 tests green; ruff, ruff format and mypy --strict clean.
`process_inbox(inbox_dir, bundle_dir, ingested_at, *, okf_type, gate)`:
per dropped file, bytes -> extract_text -> guard gate -> render -> collision
gate -> write -> index link. Returns an InboxResult whose four buckets
(persisted / quarantined / rejected / failed) are disjoint and complete, so a
file that vanished shows up as a missing entry rather than as nothing.
The guard is INJECTED rather than imported. The library calls no guard
function and makes no security decision: the Gate adapter returns a
GateDecision carrying the guard's own disposition value and the sanitized
text, and the flow only branches on it. That keeps the core dependency-free
while B2 (the guard's CI channel) is still open, and lets the persist and
refuse branches run deterministically against a test double.
Decisions taken with the operator this session:
- What is persisted is the gate's SANITIZED text, not the extracted text.
Screening one string and writing another would make the verdict a statement
about bytes nobody kept. `sanitize` is exported by the guard precisely for
callers composing the checklist themselves, so this is sanctioned API, not
a reimplementation. Door B has no model call, so the fenced text the
bookends produce for a transform is never persisted.
- Quarantine is reported apart from rejection. QUARANTINE_REVIEW means "hold
for human review" — an operator queue — where FAIL_SECURE is a decision.
No quarantine directory in v1; that stays an extension point.
Fails closed by construction: only the guard's non-blocking floor (`warn`)
persists, so a renamed member, a future disposition or an adapter typo lands
in `rejected` rather than being guessed safe.
One bad file never aborts the run. Only three conditions fail the whole run,
and each is wrong for every file at once: an invalid `ingested_at`, a reserved
`okf_type`, and a missing inbox directory.
New code `inbox_slug_collision`: two dropped files reducing to one generated
name are BOTH refused. Persisting one would let iteration order decide the
winner, and overwriting would lose the other's content.
Phase 1 primitives are reused, never duplicated, so four helpers become
package-internal names (write_bytes, link_in_index, parse_frontmatter,
INDEX_NAME) and link_in_index learns to append to an empty index — Door A
always seeds its index with bundle_summary, but Door B has no summary to
invent and must not open with a blank line.
The slug inherited the whole filename stem with no length bound, so a long
dropped name produced a filename the filesystem cannot hold — surfacing as an
untyped OSError at the write, with an errno that differs per platform (63 on
macOS, 36 on Linux). The inbox contract promises a typed per-file outcome, so
the gate belongs in the pure function that forms the name.
Refusal, not truncation: truncating is lossy AND collision-prone — two long
names sharing a prefix would reduce to one filename and the second write would
silently claim the first file. Same posture as `inbox_slug_empty`: the library
never invents a filename the operator did not give it. The message carries the
actual size and the limit, since the fix is to rename the dropped file.
Limit verified empirically on APFS 2026-07-25: a 255-byte component writes, a
258-byte one raises errno 63. ext4 and NTFS cap at the same 255.
New MaterializationError code `inbox_slug_too_long`, registered in the errors
docstring and in the error-code conformance suite.
Second guard-independent step of Phase 2: the pure functions that turn a
dropped file's name and extracted text into an OKF concept file, with the
§7-analogous honesty marker. No runtime dependency, no guard call, no model
call — the persist gate is still steps 4–5.
- `inbox_slug(filename)` reduces a name to the Phase 1 id grammar
(`[a-z0-9][a-z0-9-]*`): drop the final extension, lowercase, collapse every
run of non-grammar characters to one `-`, strip the ends. A name that
reduces to nothing fails fast rather than inventing an `untitled` fallback
that would silently collide across unrelated files.
- `inbox_filename(slug)` namespaces it `inbox-{slug}.md`, disjoint from
`index.md`, Door A's `ingest-*`, and `promoted-verdict-*` for every slug the
grammar admits (asserted over a hostile-name table, not one example).
- `render_inbox_concept(...)` emits the six-key layer in fixed order — `type`,
`title`, `source_file`, `source_sha256`, `ingested_at`, `generated: true` —
with the digest taken over the ORIGINAL dropped bytes, never the extracted
text, so provenance stays re-verifiable against the operator's file. Body is
normalised to LF-only with exactly one trailing newline (dropped files
legitimately arrive with CRLF; Door A can validate instead because it renders
its own bodies).
Slugging normalises to NFC first. macOS (APFS) hands filenames over
decomposed, so `é` arrives as `e` + combining acute: without normalising, the
combining mark alone becomes a separator and the base letter survives
(`cafe`), where the composed form is one non-grammar character (`caf`) — one
visual filename, two slugs, depending on where the string came from. The test
pins both forms with explicit escapes so it cannot depend on the test file's
own encoding. No transliteration, deliberately: mapping non-ASCII to ASCII is
a semantic claim the slugger cannot make (Norwegian `møte`/meeting would
become `mote`/fashion). The readable name survives verbatim in `title`.
Fail-fast gates, all typed: reserved `verdict` layer refused at this door too
(the promotion gate stays the only path in); a title that is multi-line or
contains `[`/`]` refused, matching Door A's manifest rule, because it renders
verbatim into `- [title](target)`; a multi-line `source_file` refused because
line-oriented frontmatter would take the injected lines. Four new stable
MaterializationError codes, each with its entry in the test_error_codes.py
registry: `inbox_slug_empty`, `inbox_title_invalid`,
`inbox_source_file_invalid`, `okf_type_reserved`.
`ingested_at` validation moves to a shared `validate_ingested_at` in
materialize.py and is called by both doors — one rule in one place is what
keeps the determinism contract from drifting apart per door.
TDD: tests/test_inbox.py precedes the implementation. 325 tests green;
mypy --strict, ruff check/format, and the sanitize|quarantine|lexicon
boundary grep-gate clean; runtime dependencies still exactly none; the Phase 1
golden suite still passes byte-for-byte.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
First, guard-independent step of Phase 2: a stdlib-only registry mapping a
dropped file's extension to its text extractor, with the fail-fast gates that
keep binary parsing out of core. `extract_text(filename, data)` dispatches
(case-insensitively) to:
- `md`/`txt` — utf-8-sig passthrough (BOM never leaks, baseline parity with
Door A's read_csv);
- `csv` — the Phase 1 `render_table` (renderer reused, not duplicated);
- `json` — verbatim inside `render_fenced_block`;
- `html`/`htm` — text via `html.parser`, `script`/`style` stripped, tags as
word boundaries (spec B3: adequate for v1, richer is out of scope).
`pdf`/`docx`/`xlsx` are `[extract]`-gated; until that extra ships a parser they
fail fast with a typed error naming the extra — never a silent skip, never a
bundled parser in core. New `ExtractionError(IngestError)` carries four stable
codes (`extractor_unknown`, `extractor_extra_missing`, `extractor_decode_error`,
`extractor_empty_csv`); a non-UTF-8 file is a typed corrupt-input failure, never
a leaked UnicodeDecodeError. `extract_text` returns text content only — LF
framing and concept frontmatter are the materializer's job (step 2).
No runtime dependency and no guard call yet (the guard pin and 0.4.0 land with
the persist gate in steps 4–5). TDD: test_extract.py + the four codes in the
test_error_codes.py registry precede the implementation; mypy --strict, ruff,
and the `sanitize|quarantine|lexicon` boundary grep-gate all clean; the Phase 1
golden suite still passes byte-for-byte.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
Stage 0 guard-readiness check (this session) verified the real
`llm-ingestion-guard` v0.2.0 surface against ground truth: the Door C plan
assumed `okf.import_bundle(bundle, origin=…, channel=…, allow_reserved=False)`,
but the shipped signature is `import_bundle(bundle, *, origin, channel)` — no
`allow_reserved` parameter. Reserved-name rejection of `index.md`/`log.md` is
unconditional in the guard (OKFPathError, no toggle), so the intended default
semantics are unchanged; only the documented call was wrong and would have
raised TypeError if written verbatim. Correct the plan line before Phase 2
code is written against it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
Sequences the remaining roadmap (Stage 0 guard gate -> Phase 2 -> Phase 3
-> Phase 4) with the quality rationale for the order: guard readiness
verified before any Phase 2 code, Phase 2 split guard-independent-first,
Phase 3's configurability extracted only after two concrete consumers
exist, Phase 4 code last on a frozen contract with its coordination
started early. Connective tissue over the four per-phase plans; owns the
order and reasons, not the phase detail.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
Ingest-spec §4 (ratified D1, commons bfa5a9b) requires a title to reject
`[` and `]` fail-fast at manifest load. The title is rendered verbatim into
the index link label and frontmatter (§5/§6), where a bracket would break
index-link and navigation parsing (method-spec §3 Step 1). The rule was
specced but unenforced: validation only checked single-line.
Satisfies the commons "Title link-safety" load-bearing §11 seam. Same
`manifest_schema` error code as the sibling single-line/max_rows rules;
parametrized cases added to test_manifest and test_error_codes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016oMpAhQcJZVGBW182stSPn
The six `raise TypedError(...) from exc` sites (manifest.py x3, materialize.py,
connectors.py x2) turn leaked OSError / ValueError / sqlite3.Error / URLError
into typed IngestErrors while keeping the original under __cause__ — but no
test asserted the chain, so a stray bare `raise` could drop the cause
unnoticed. Add a conformance suite, one test per site, each asserting the
chained cause type. The module docstring documents the invariant. Companion
to test_error_codes.py, which pins the `.code` on the same sites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
The §3 ownership scan classified every ingest-stamped file as replaceable,
so materializing one manifest into a bundle deleted the ingest files another
manifest had stamped there — and removed their index links too.
Narrow ownership to files whose stamp names the running manifest by stem.
The stamp is `{stem}@{sha256[:16]}`; matching on the stem, not the whole
stamp, lets an edited manifest still reclaim the files a prior run of the
same manifest wrote (the stem is stable across content edits), while a
different manifest sharing the bundle keeps its own files. Implements the
trinn-e §10.2 decision. This does not close the operator-copy restriction
(a generated file copied into curated content), which stays documented in
ingest-spec.md:69-72, not enforced.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
The Door A description said the §11 golden fixtures "are missing today".
They shipped in 9dd86b1 as ingest-golden-{file,sql,http}/. Correct the line
to state what is present.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
a7e1bc8 unified title rendering at both sites and routed the open question
to commons, the spec owner: does §5's whitespace collapse apply to `title`,
or only to `source_query`? The ruling is source_query only. §5 mandates the
collapse for that one field — where a multi-line SQL SELECT must render on
one line — while every other value is validated single-line at manifest load
and emitted as-is: validation, not repair.
_single_line is renamed _collapse_whitespace and now runs for source_query
alone. Frontmatter values, the index link label, and the labels_by_target
map all pass through verbatim. A title carrying an internal whitespace run
(or any other operator-supplied bytes) now survives byte-for-byte at both
the frontmatter and the index-label site, instead of being silently altered.
No golden bytes change: no title in the shipped fixtures or either consumer
carries a collapsible whitespace run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
