llm-ingestion-okf/docs/plan/okf-v0.2-alignment.md
Kjell Tore Guttormsen cb23de4700 docs(okf-v0.2): supporting the latest OKF version becomes standing policy
Operator directive 2026-07-26: the library always supports the current
latest version of Google OKF. v0.2 shipped 2026-07-25, so v0.2 support is
committed work rather than something a consumer has to request. This
overrides the previous default answer to open question V3 ("no until a
named consumer asks"), which is kept in the plan marked superseded so the
override reads as deliberate.

Support is additive: a new profile, never a migration of the existing two.
That single design choice is what makes an always-latest policy sustainable,
and it resolves the tension the directive would otherwise create with three
constraints that do not yield to it:

- DEFAULT states commons' ingest-spec section 5 layer, so its `generated`
  shape is commons' call. Under the additive design this stops blocking us,
  which takes commons off the critical path.
- STRICT_V1 mirrors the proving consumer's ratified contract; changing
  another repo's contract from here would violate O2.
- v0.2 defers the attestation receipt and verdict wire formats upstream, so
  the format is supportable and the unspecified runtime is not. It re-enters
  scope when upstream specifies it.

This is also the first time the phase-3 profile abstraction is forced by
something outside this repo rather than by a second consumer, which is the
better test of whether the seam was cut in the right place.

Deliverables D1-D6 replace the earlier decision-round framing: a frontmatter
model that can carry block lists (`sources`, multi-verifier `verified`), an
OKF_V0_2 profile plus an OKF_LATEST alias whose moving-target tradeoff is
documented rather than hidden, Door C conformance against the consumer
tolerance rules, `Attested Computation` round-trip, v0.2 golden fixtures, and
a release-checklist re-check so the standing policy cannot decay silently.

Two new assumptions carry the weight. V-A7 forbids any profile from emitting
`timestamp` together with a malformed `generated`, since that combination
would have neither a valid `generated.at` nor an eligible section 13.1
fallback. V-A8 validates our own v0.2 fixture against upstream's reference
implementation, because every other test in the suite only asks whether we
agree with ourselves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
2026-07-26 13:41:30 +02:00

16 KiB
Raw Blame History

OKF v0.2 support — plan

Status: committed implementation track. Standing policy (operator, 2026-07-26): the library always supports the current latest version of Google OKF. v0.2 is out, so v0.2 support is committed work — not conditional on a consumer requesting it.

Read with: docs/plan/execution-order.md (sequencing), docs/plan/phase-3-configurable-contract.md (the profile object that carries this), docs/phase-3-split-table.md (the profile/guard/consumer column discipline).

What "support the latest version" means here

It means additive: the library can read, validate, materialize, and index v0.2-shaped bundles. It does not mean migrating existing profiles onto v0.2. That distinction is the whole design, and it is what makes a standing always-latest policy implementable rather than a recurring crisis:

  • DEFAULT states commons' ingest-spec §5 layer. Its shape is commons' authorship, and editing it locally is a standing non-goal.
  • STRICT_V1 mirrors the proving consumer's contract, ratified by their operator on 2026-07-25. Changing another repo's contract from here violates O2.
  • So v0.2 support arrives as a new profile, alongside the existing two. The phase-3 profile object is exactly the seam that makes this possible; this is the first time that abstraction pays for itself against an external forcing function rather than a second consumer.

The upper bound is set upstream, not by ambition: v0.2 defers the receipt and verdict wire formats and the full attestation runtime protocol to a future revision. The format is fully supportable; an unspecified runtime is not. See D4.

Provenance of the facts below

Read from GoogleCloudPlatform/knowledge-catalog, okf/SPEC.md at main on 2026-07-26 — a branch, not a tag. First implementation step is to re-read the spec at a pinned commit and record it, exactly as assumption C2 pins the proving consumer's validator to 9ee5a8e. An enumeration read off a moving branch is a premise, not a fact.

What v0.2 actually changes

Two supersessions — the only backward-breaking changes, both with a documented consumer fallback (§13.1):

v0.1 v0.2 Fallback the spec grants
timestamp generated.at Consumers MAY fall back to legacy timestamp when generated is absent
body # Citations list sources frontmatter Consumers SHOULD read sources, MAY still parse legacy # Citations

Additive, all optional upstream (§5) — and all in scope for us under the standing policy: sources (per-source credibility signals — author, usage_count, last_modified, framed by usage_window), generated { by, at }, verified [ { by, at } ], status (draft/stable/deprecated, absent means stable), stale_after (absolute YYYY-MM-DD).

Also new:

  • Actor convention (§7). <producer>/<version> for an agent or tool, human:<id>, process:<id>. Consumers that classify trust key off the human: prefix, so producers MUST use it for hand-authored or human-confirmed content. Trust tiers (unverified / machine-confirmed / human-reviewed) are derived advisory signals, never a stored field.
  • Attested Computation concept type, with runtime, parameters, computation, executor, attester.
  • okf_version stays root-index.md-only and is a MAY: a declaration of the version a bundle targets, with no conformance checkpoint tying the declaration to compliance.
  • Consumer tolerance is unchanged and broad. A consumer MUST NOT reject a bundle for missing optional fields, unknown type values, unknown frontmatter keys, broken cross-links, or missing index.md. One additional MUST: a bare verified mapping must be read as a one-element list.

One YAML detail that sizes the parser work: the canonical form for generated and a single verified is an inline flow mapping on one line (generated: { by: …, at: … }), which the existing scalar parser already round-trips as an opaque string. Only a multi-verifier verified and sources are block lists, and those are what force a richer frontmatter model.

Exposure audit — measured against the tree, not reasoned

Surface Status under v0.2 Decision owner
DEFAULT emits generated: "true" (materialize.py:164, inbox.py:127) Malformed for a v0.2 field: generated.by is REQUIRED within generated. generated was not reserved in v0.1, so this was legal when written — v0.2 claimed the name commons (ingest-spec §5)
_is_ingest_owned requires generated == "true" (materialize.py:143-149) Fail-safe. A v0.2 mapping is not the string "true", so the predicate returns False → target not ours → collision → refuse to overwrite. No safety hole this repo
STRICT_V1 emits timestamp, and no generated Conformant via the §13.1 legacy fallback, which applies precisely because generated is absent. No action is forced on the wiki wiki
parse_frontmatter is scalar-only (partition(":")dict[str, str]) Inline flow mappings round-trip as opaque strings. Block lists are unreachable, so sources and multi-verifier verified need a richer model — now in scope (D1) this repo
Fixtures and tests pin okf_version: 0.1 Accurate for the v0.1 profiles. The new v0.2 profile declares "0.2" catalog (E1: okf_version tracks the upstream Google version)
catalog's gate checks okf_version on form (/^\d+(\.\d+)*$/), not membership "0.2" passes unchanged — an upstream bump breaks no gate catalog

Two results that cut against the first reading of the change:

  1. STRICT_V1's timestamp is not broken. It sits on the documented legacy path. An earlier framing of it as a required-field defect was wrong.
  2. DEFAULT is the only surface with a real shape problem, and it is the one surface this repo may not unilaterally change. Under the additive design this stops being on our critical path (D2, V1).

The interaction that keeps (1) true and must never be violated: the timestamp fallback is granted only when generated is absent. A profile emitting both a timestamp and a malformed generated would have neither a valid generated.at nor an eligible fallback. No current profile does this — DEFAULT has no timestamp, STRICT_V1 has no generated — and no profile may be allowed to reach that combination. This becomes a profile-construction invariant (V-A7).

The sequencing rule we impose on ourselves

Conform first, claim after. No bundle declares an okf_version its shape has not earned.

The spec does not require this — §12 is a MAY with no conformance checkpoint, so declaring "0.2" early would be permitted. We adopt the rule anyway, for the same reason as measure first, widen after on the guard pin: a version claim that is literally permitted but not earned is the true-sounding misleading claim, and that class of mistake has already cost this repo a session. Concretely, the v0.2 profile declares "0.2" only once D1D3 and their fixtures are green.

Deliverables

D1 — A frontmatter model that carries block lists. The single largest piece, and the one that must be designed rather than improvised: parse_frontmatter returns dict[str, str] today, and sources / multi-verifier verified cannot be expressed in it. Three directions:

  • (a) Widen the value type in place (str | Mapping | list). Cheapest to write, worst to consume: every existing caller must newly narrow, and mypy --strict makes that cost visible everywhere at once.
  • (b) A typed frontmatter model for all profiles. Cleanest end state, but it rewrites the v0.1 read path and puts C1 (default-profile behavior neutrality) at risk for no v0.1 benefit.
  • (c) Recommended. Keep the scalar reader as the v0.1 path, untouched, and add a structured reader selected by the profile. C1 stays trivially true (DEFAULT and STRICT_V1 do not change code paths at all), the new complexity is reachable only from the new profile, and the two readers share the same byte-level guarantees.

D2 — An OKF_V0_2 profile. Frontmatter schema covering the §5 families; generated: { by: "llm-ingestion-okf/<version>", at: <ingested_at> } using the §7 actor convention; sources emission from the manifest's source; status and stale_after expressible; root-index okf_version: "0.2".

Also an OKF_LATEST alias, which is how the standing policy becomes visible in code. Document the tradeoff rather than hiding it: an alias that moves means a consumer bound to OKF_LATEST inherits upstream's breaking changes on a library upgrade. The versioned constants are the stable binding and are what consumers should pin; OKF_LATEST is for callers who have explicitly opted into tracking.

at binds to the already-validated required ingested_at argument, so no wall-clock default is introduced and determinism holds.

D3 — Door C accepts any v0.2 bundle. This is where "we support v0.2" is most user-visible: the §14 consumer tolerance MUST NOTs, plus the bare-verified coercion. Measured with one test per rule — Door C is a consumer in the OKF sense and either honours these or does not.

D4 — Attested Computation: format yes, runtime no. Support the type and its fields (runtime, parameters, computation, executor, attester) for parse, validate, round-trip, and index. Do not implement execution or attestation: the receipt and verdict wire formats are explicitly deferred upstream, so there is no contract to build against, and "did this run produce the value the sanctioned way" sits against the guard's boundary rather than this library's plumbing. When upstream specifies the protocol, the standing policy brings it back into scope — and the boundary question goes to the operator first.

D5 — v0.2 golden fixtures. A byte-exact v0.2 bundle in examples/. This is what converts "we support v0.2" from a claim into a test, and in phase 4 it becomes the cross-runtime conformance oracle for the Node half too.

D6 — Keep the policy from decaying. "Always latest" silently rots unless something checks. Two mechanisms, both cheap: the spec commit is pinned and recorded in this doc (V-A1), and the release checklist gains one item — re-check upstream for a newer OKF version and record the result, even when unchanged. A standing policy with no detection mechanism is an intention, not a policy.

Open questions

V1 — generated's shape in DEFAULT (commons'). No longer blocks our v0.2 support, because D2 puts v0.2 in a new profile. Still raised with commons, because DEFAULT keeps emitting a v0.2-reserved key with a v1-era value. Recommendation to carry: generated: { by: "llm-ingestion-okf/<v>", at: <ingested_at> }, deriving the machine-generated predicate from generated.by instead of a bespoke boolean — spec-native, removes a hand-rolled key, costs a _is_ingest_owned change and a fixture regeneration. If commons declines, the deviation is documented, not patched here.

V2 — Which D1 direction? (c) recommended above; needs a decision before code because it determines whether mypy --strict churn lands across the existing call sites or stays confined to the new path.

V3 — Do we add v0.2 families at all? Answered: yes, by the standing policy. Superseded — kept here to record that the earlier default answer ("no until a named consumer asks") was overridden deliberately, not forgotten.

V4 — okf_version values. Policy, not constant. E1 gives the value to catalog. Our obligation is narrower and testable: no profile hard-codes a version, and the root-frontmatter policy expresses any value.

V5 — Attestation scope. Answered by D4: format in, runtime out, on upstream's own deferral rather than our preference.

TDD order

Steps 13 add tests and change no behavior, so they are safe first and de-risk everything after.

  1. Characterize what already holds. parse_frontmatter round-trips an inline flow mapping opaquely; _is_ingest_owned returns False for a v0.2 generated mapping. The second converts today's fail-safe from an accident into a documented guarantee.
  2. Audit Door C against the tolerance rules (D3): one test per MUST NOT plus the bare-verified MUST. Failures here are real defects, not plan revisions.
  3. Assert no hard-coded version anywhere a profile should decide (V4), and the no-timestamp-plus-malformed-generated construction invariant (V-A7).
  4. D1 — the structured reader, behind the profile seam, with the v0.1 scalar path untouched and the full existing suite as the net.
  5. D2 — the OKF_V0_2 profile and OKF_LATEST alias.
  6. D5 — v0.2 golden fixtures; okf_version: "0.2" is declared only here, once 15 are green (conform first, claim after).
  7. D4Attested Computation as a type with round-trip fidelity.
  8. V1 to commons in parallel throughout; D6 lands with the release.

Key assumptions, each with its test

# Assumption Test
V-A1 The spec enumeration here is complete Re-read okf/SPEC.md at a pinned commit, record the commit, diff against this doc before step 4
V-A2 Inline flow mappings survive the scalar parser unmodified Step 1 characterization; byte-exact round-trip
V-A3 v0.2 input cannot cause an unowned overwrite Step 1: _is_ingest_owned False on a generated mapping
V-A4 Door C meets the v0.2 consumer tolerance rules Step 2, one test per MUST NOT
V-A5 No profile hard-codes an upstream version Step 3
V-A6 Adding v0.2 support is behavior-neutral for v0.1 profiles Golden suite byte-for-byte under DEFAULT and STRICT_V1; existing tests unmodified and green (C1 extended)
V-A7 No profile can emit timestamp together with a malformed generated Named construction-time test, same shape as C3's verdict reservation
V-A8 A v0.2 bundle we emit is accepted by an independent v0.2 consumer Validate the D5 fixture against upstream's reference implementation, not only against our own reader

V-A8 is the one that keeps this honest. Every other test asks whether we agree with ourselves.

Non-goals

  • Implementing attestation execution — executors, attesters, receipts, verdicts (D4, on upstream's deferral).
  • Migrating DEFAULT or STRICT_V1 onto v0.2. Support is additive; those two contracts belong to commons and the wiki.
  • Editing commons' ingest-spec locally. V1 is raised there.
  • Declaring okf_version: "0.2" on any bundle whose shape has not earned it.
  • Config-file/DSL profile loading — still a phase-3 extension point.

Coordination — who needs to know what

  • commons — V1. generated's shape in ingest-spec §5. No longer blocking us, still theirs to decide.
  • catalog — form owner. Upstream moved to v0.2; their form-not-membership gate absorbs the bump with no change; V4 is theirs; and a new profile now declares "0.2", which is the first bundle shape in this repo to do so.
  • wiki — informational. Their timestamp is legacy-but-readable under §13.1; nothing is required of them, and the earlier reading that called it a defect was wrong.
  • okr — before the reference-implementation lift, so the Node half ports a contract that already knows about v0.2 rather than a v0.1 shape.

Verification

  1. Spec re-read at a pinned commit; hash recorded in this doc.
  2. pytest green; mypy --strict src/; ruff check . and ruff format --check . clean.
  3. V-A6: golden suite byte-for-byte under both existing profiles; zero fixture changes outside the new v0.2 fixtures (git diff --stat examples/ shows additions only).
  4. V-A7 and V-A5 named tests present and failing-by-construction if removed.
  5. V-A8: the v0.2 fixture validates under upstream's reference implementation.
  6. Boundary grep-gate still empty (sanitize|quarantine|lexicon absent outside guard imports).
  7. D6: release checklist contains the upstream-version re-check item.