llm-ingestion-okf/docs/upstream-okf-upgrade-runbook.md

Upstream OKF upgrade — runbook

Standing policy (operator, 2026-07-26): this library always supports the current latest version of Google OKF. This document is the procedure that policy runs on. It exists because a standing policy without a procedure is an intention, and because the v0.1 → v0.2 round produced five findings that a spec reading alone had settled wrongly — each step below names the concrete failure it prevents.

Written after the v0.2 round. Every prevented-failure note is something that actually happened, not a hypothetical.

The goal that shapes the procedure: this repo is a black box

An upstream release is our problem, not our consumers'. The target cost to a repo that consumes OKF through this library is a re-run, and nothing else.

That is a design commitment with teeth, not an aspiration:

• Support is additive — a new profile, never a migration. Existing profiles stay byte-stable. A consumer who does not opt in sees no change at all.
• New public parameters are keyword-only with defaults. A consumer's existing positional call sites stay source-compatible across an upgrade. This is why profile is specified as *, profile: BundleProfile = DEFAULT — decided, not yet shipped: measured 2026-07-26, no door takes the argument at all. It turns additivity from something a consumer has to measure into a property of the signature.
• Consumer golden fixtures must not churn. If an upgrade rewrites bytes in a consumer's frozen fixtures, the upgrade is wrong, not the fixture.

The honest boundary — state it every time, never let the black box be oversold. The library absorbs shape changes. It cannot absorb upstream changes to content the consumer authored. v0.2 superseded timestamp with generated.at and the body # Citations list with sources; no library change makes a consumer's existing timestamp field become something else. For that class the deliverable is not absorption but a measured exposure report, per consumer, before they ask — which is what step 5 produces.

Trigger

"Always latest" decays silently: nothing fails when upstream ships and we do not notice. So the re-check is an item on the release checklist — run it at every release of this library, and record the result even when unchanged, because an unrecorded check is indistinguishable from a skipped one.

Check GoogleCloudPlatform/knowledge-catalog, path okf/. A version bump appears as a commit against okf/SPEC.md §12 and, in the v0.2 round, as an explicit migration commit (okf: migrate format and tooling to Open Knowledge Format v0.2).

Step 1 — Pin before reading anything

Record the exact commit SHA and read everything at it.

Prevents: reading a moving branch and treating the result as fact. In the v0.2 round both this repo and portfolio-optimiser-commons independently read main, which felt like two confirmations and was one unstable reference read twice. Worse, SPEC.md was edited after the v0.2 migration commit — so "the v0.2 commit" and "the current spec text" were different objects, and either alone would have been a partial answer.

Step 2 — Enumerate the whole REPOSITORY tree, not just okf/, not just SPEC.md

List every file and directory at the pinned commit before deciding what to read. Start at the repository root:

git ls-files | awk -F/ '{print $1}' | sort -u      # top-level entries
git ls-files | sed 's/.*\.//' | sort | uniq -c     # what kinds of file exist

Prevents: assuming the directory you know about is the one that matters. This step has now failed twice at two different scopes, which is why its title names the root rather than a subdirectory:

• v0.2 round, inside okf/. okf/ held SPEC.md, README.md, pyproject.toml, src/, tests/, samples/ and bundles/. The last one held the actual v0.2 example bundles and was not on anyone's list until the tree was enumerated.
• 2026-07-31, one level up — the same mistake against okf/ itself. Scoping enumeration to okf/ (as this step previously instructed) hides that the repository root also carries toolbox/ and samples/. Measured at 3fcbb9f: 265 tracked files, of which 48 are Python and 43 TypeScript, with toolbox/ holding two complete tools (mdcode, enrichment) plus a second copy of an OKF bundle under toolbox/mdcode/demo/okf/catalog/. None of it was on any list.

The cost of getting this wrong is not only unread files: a negative claim derived from a partial enumeration reads exactly like a measured one. "There is no validator in okf/" was recorded as fact and used to plan a weakening of V-A8; okf/src/reference_agent/bundle/document.py:58 defines validate(), and the same module is a working v0.2 reader. Enumerate before concluding that something is absent — see Step 3a.

Step 3 — Read the shipped examples, not only the normative text

This is the step the v0.2 round skipped, and it is the reason this document exists.

A spec says what is permitted. An example shows what upstream actually emits. Those differ, and when we are about to freeze a fixture and hand it to consumers, the second one is what we are being measured against.

Two traps, both hit in the v0.2 round:

• A directory called samples/ was not samples of the format. okf/samples/ga4_merch_store/ is a runner config (README.md + seeds.txt) that drives the reference agent to generate a bundle. It says nothing about output shape. The real examples were in okf/bundles/ — four of them.
• Fetch tooling that converts to markdown will strip or reformat frontmatter. Byte-level questions need byte-level fetches. Check for a BOM, check the trailing newline, check whether a value is quoted. Those three are exactly the properties that break downstream gates and exactly the ones a summarizing fetch destroys.

What reading the four v0.2 bundles produced, none of it derivable from SPEC.md:

Finding	Consequence
No reference bundle declares okf_version at all; root index.md has no frontmatter	§12 is a MAY and upstream declines it. Our stricter posture became a decision instead of an assumption
generated is written for human-authored content (by: human:…)	It can never be an ownership or machine-generated predicate. Governs the collision gate
Real frontmatter is multi-line block YAML: block lists of multi-key mappings, nested mappings, flow sequences, booleans, dates	Re-sized the emitter work. It had been scoped against a list of strings
§7's canonical tool actor is <producer>/<version>, and upstream uses it	Counter-evidence to a recommendation we had already sent to another repo

Step 3a — Run upstream's own reader against our fixture (V-A8)

Every other test in the suite asks whether we agree with ourselves. This one asks an independent implementation, and it is cheap enough that there is no excuse for skipping it.

It is not a dependency. okf/src/reference_agent/bundle/document.py imports only yaml, so it runs standalone under system Python against the pinned clone — nothing is installed, .venv is never touched (the one-runtime-dependency rule and its packaging test both stay intact), and nothing is written.

import sys; sys.path.insert(0, "<clone>/okf/src")
from reference_agent.bundle.document import OKFDocument, trust_tier, is_stale

Assert, over each file of the current version's golden bundle:

1. OKFDocument.parse() accepts our bytes. This is the load-bearing one. Our own parser is line-oriented and reads inline flow forms as opaque strings, so it structurally cannot tell us whether a real YAML consumer recovers them as structures. Only an outside parser can.
2. The values arrive as the shapes the profile intends — generated as a mapping, sources as a list of mappings — not merely as something that parsed.
3. validate() passes. Note what this does and does not buy: at 3fcbb9f REQUIRED_FRONTMATTER_KEYS = ("type",), so it checks one key. Treating a green validate() as "upstream accepts our bundle" would overclaim badly; the parse in (1) and the shape assertions in (2) are where the signal is.
4. The semantic readers upstream ships run over our frontmatter without raising — at 3fcbb9f: trust_tier (§5.3), normalize_verified (§5.2), is_stale (§5.5).

Record the type each value arrives as, not just that it parsed. A real YAML parser coerces, and the coercions are load-bearing facts about the value space — this is where the v0.2 round's okf_version float finding came from (see the alignment plan's A-E6). Upstream's serialize() also reflows inline flow forms to block form, so an upstream round-trip produces bytes our own line-oriented parser cannot read. That is expected and one-directional; do not "fix" it by emitting block form.

Result, v0.2 round at 3fcbb9f: 13/13 green against examples/ingest-golden-okf-v0-2/.

Step 4 — Produce the diff, classified

From the spec's own "Changes from vN" section plus the example evidence, classify every change as:

• Breaking — name it, and measure consumer exposure in step 5.
• Additive — new optional fields, new conventional headings.
• Deferred upstream — a format specified but its runtime protocol left to a future revision. The format is supportable; an unspecified runtime is not something to build against. Record it as out of scope on upstream's deferral, not on our preference.
• Observed but undocumented — present in an official bundle, not found in the spec sections read. Record as unresolved. Do not silently promote it to either category. (v0.2 round: a not: family in metrics/gross-margin.md.)

Step 5 — Measure our exposure and each consumer's, in that order

Ours: grep the profiles, the fixtures, and the emitter for every field the diff touches. Cite file:line. Never reason about a key set — measure it.

Theirs: for each breaking change, state what would have to be true for it to reach them, then ask them to measure it against their real data rather than telling them whether they are affected. In the v0.2 round both portfolio-optimiser-claude and linkedin-studio came back with measured zero exposure and found the framing useful precisely because it was falsifiable.

Step 6 — Plan additively, then pilot before general availability

The plan is a new profile, sequenced deliverables, and stated invariants. Rollout is pilot-first: a pre-release tag to a small pilot set chosen for signal, revised on their feedback, then GA. Flipping the OKF_LATEST alias is the GA event, not a merge side effect.

State each pilot test's expected results before the run, numbered, plus what would surprise us. An expectation we get wrong is a better result than a clean run; only a stated expectation can be falsified by someone else's run.

Name fixtures by repo and path, never by path alone. In the v0.2 round both this repo and portfolio-optimiser-claude had a directory named examples/ingest-golden-file/ holding different fixtures with zero content overlap. A pilot baseline was agreed by name, and the name pointed at two files.

Step 7 — Inform every OKF-consuming repo

All of them, not only the pilots, and not only when something breaks.

Send a bounded loop of directed messages, not a broadcast — a broadcast is inherited by every future repo, which is the wrong lifetime for a version-specific notice.

Each message carries:

1. The version, the pinned commit, and where to read it.
2. The breaking changes by name, with the shape of the exposure — so they can measure themselves rather than take our word.
3. The black-box promise and its boundary: their cost is a re-run; existing profiles are byte-stable; new parameters are keyword-only. And the part we cannot absorb, said plainly.
4. What we want back, if anything, and what happens if they say nothing.

Consumers to cover (2026-07): portfolio-optimiser-commons (spec authorship), portfolio-optimiser, portfolio-optimiser-claude, claude-code-llm-wiki, catalog, okr, linkedin-studio, ms-ai-architect. Re-derive this list each round rather than trusting it — it is a premise like any other.

Step 8 — Close the loop in the repo

The plan doc records the pinned commit, the classified diff, the measured exposure, and every correction a consumer sent back. Corrections stay visible: a premise that survived verification is only known to have survived if the check is recorded, and a claim we withdrew is only safely withdrawn if the withdrawal is written where the claim was.

Invariants this procedure protects

• No profile hard-codes an upstream version.
• No bundle declares a version its shape has not earned. The spec may permit declaring conformance without a checkpoint; we decline. Conform first, claim after.
• DEFAULT states commons' ingest-spec §5 layer — raised there, never patched here.
• STRICT_V1 mirrors a consumer's ratified contract — never changed from here.
• The value of okf_version belongs to the catalog.
• Security stays the guard's domain in every version.