llm-ingestion-okf/docs/upstream-okf-upgrade-runbook.md

# Upstream OKF upgrade — runbook

Standing policy (operator, 2026-07-26): **this library always supports the current
latest version of Google OKF.** This document is the procedure that policy runs on.
It exists because a standing policy without a procedure is an intention, and because
the v0.1 → v0.2 round produced five findings that a spec reading alone had settled
wrongly — each step below names the concrete failure it prevents.

Written after the v0.2 round. Every prevented-failure note is something that
actually happened, not a hypothetical.

## The goal that shapes the procedure: this repo is a black box

An upstream release is our problem, not our consumers'. The target cost to a repo
that consumes OKF through this library is **a re-run, and nothing else.**

That is a design commitment with teeth, not an aspiration:

- **Support is additive — a new profile, never a migration.** Existing profiles stay
  byte-stable. A consumer who does not opt in sees no change at all.
- **New public parameters are keyword-only with defaults.** A consumer's existing
  positional call sites stay source-compatible across an upgrade. This is why
  `profile` is specified as `*, profile: BundleProfile = DEFAULT` — **decided, not
  yet shipped**: measured 2026-07-26, no door takes the argument at all. It turns
  additivity
  from something a consumer has to measure into a property of the signature.
- **Consumer golden fixtures must not churn.** If an upgrade rewrites bytes in a
  consumer's frozen fixtures, the upgrade is wrong, not the fixture.

**The honest boundary — state it every time, never let the black box be oversold.**
The library absorbs *shape* changes. It cannot absorb upstream changes to content the
consumer authored. v0.2 superseded `timestamp` with `generated.at` and the body
`# Citations` list with `sources`; no library change makes a consumer's existing
`timestamp` field become something else. For that class the deliverable is not
absorption but a **measured exposure report, per consumer, before they ask** — which
is what step 5 produces.

## Trigger

"Always latest" decays silently: nothing fails when upstream ships and we do not
notice. So the re-check is an item on the release checklist — run it at every
release of this library, and record the result **even when unchanged**, because an
unrecorded check is indistinguishable from a skipped one.

**Since 2026-08-23 the trigger also fires without a release**, weekly, from
`tools/okf_watch.py`. See § The weekly watch below. The watch decides *whether*
this procedure runs; it never substitutes for it.

Check `GoogleCloudPlatform/open-knowledge-format`. **That is the canonical home of
the spec, the reference agent and the sample bundles as of 2026-08-21.** A version
bump appears as a commit against `SPEC.md` §12 and, in the v0.2 round, as an
explicit migration commit (`okf: migrate format and tooling to Open Knowledge
Format v0.2`).

**Do not check `GoogleCloudPlatform/knowledge-catalog`, path `okf/`.** That copy is
a frozen snapshot by upstream's own notice (`6265173`, "anything built against it
will drift out of date"), and this repo was pinned to it until the 2026-08-23 round.
Two consequences, both measured that round and neither hypothetical:

- **The two trees have already diverged**, and not only in the direction you would
  expect: the frozen repository's *head* carries a fix (`38c713f`, eight `tags:`
  values written as sequences rather than as one plain scalar) that the canonical
  repo does not. The canonical tree is authoritative for the *spec*; it is not
  automatically a superset. Note the pin-level precision, measured 2026-08-23:
  `38c713f` is **not** an ancestor of the old pin `3fcbb9f` either, so moving the
  pin lost nothing — canonical simply ships a form its own frozen predecessor has
  already repaired. Enumerated in full in
  `docs/plan/okf-2026-08-timestamp-tightening.md` § Known divergence.
- **A round run against the frozen tree reports "no change" truthfully and
  uselessly** — the exact shape of a negative result that is not a measurement.

*Prevents:* tracking a tree that has stopped being the thing you promised to track.
"Always latest" is a promise about an object with an identity, and the identity can
move without the version number moving.

## Step 1 — Pin before reading anything

Record the exact commit SHA and read everything at it.

*Prevents:* reading a moving branch and treating the result as fact. In the v0.2
round both this repo and `portfolio-optimiser-commons` independently read `main`,
which felt like two confirmations and was one unstable reference read twice. Worse,
`SPEC.md` was edited **after** the v0.2 migration commit — so "the v0.2 commit" and
"the current spec text" were different objects, and either alone would have been a
partial answer.

## Step 2 — Enumerate the whole REPOSITORY tree, not just `okf/`, not just `SPEC.md`

List every file and directory at the pinned commit before deciding what to read.
Start at the repository root:

    git ls-files | awk -F/ '{print $1}' | sort -u      # top-level entries
    git ls-files | sed 's/.*\.//' | sort | uniq -c     # what kinds of file exist

*Prevents:* assuming the directory you know about is the one that matters. This
step has now failed twice at two different scopes, which is why its title names
the root rather than a subdirectory:

- **v0.2 round, inside `okf/`.** `okf/` held `SPEC.md`, `README.md`,
  `pyproject.toml`, `src/`, `tests/`, `samples/` **and** `bundles/`. The last one
  held the actual v0.2 example bundles and was not on anyone's list until the tree
  was enumerated.
- **2026-07-31, one level up — the same mistake against `okf/` itself.** Scoping
  enumeration to `okf/` (as this step previously instructed) hides that the
  repository root also carries `toolbox/` and `samples/`. Measured at `3fcbb9f`:
  265 tracked files, of which 48 are Python and 43 TypeScript, with `toolbox/`
  holding two complete tools (`mdcode`, `enrichment`) plus a second copy of an OKF
  bundle under `toolbox/mdcode/demo/okf/catalog/`. None of it was on any list.

The cost of getting this wrong is not only unread files: a **negative** claim
derived from a partial enumeration reads exactly like a measured one. "There is no
validator in `okf/`" was recorded as fact and used to plan a weakening of V-A8;
`okf/src/reference_agent/bundle/document.py:58` defines `validate()`, and the same
module is a working v0.2 reader. Enumerate before concluding that something is
absent — see Step 3a.

## Step 3 — Read the shipped examples, not only the normative text

This is the step the v0.2 round skipped, and it is the reason this document exists.

**A spec says what is permitted. An example shows what upstream actually emits.**
Those differ, and when we are about to freeze a fixture and hand it to consumers,
the second one is what we are being measured against.

Two traps, both hit in the v0.2 round:

- **A directory called `samples/` was not samples of the format.**
  `okf/samples/ga4_merch_store/` is a runner config (`README.md` + `seeds.txt`) that
  drives the reference agent to *generate* a bundle. It says nothing about output
  shape. The real examples were in `okf/bundles/` — four of them.
- **Fetch tooling that converts to markdown will strip or reformat frontmatter.**
  Byte-level questions need byte-level fetches. Check for a BOM, check the trailing
  newline, check whether a value is quoted. Those three are exactly the properties
  that break downstream gates and exactly the ones a summarizing fetch destroys.

What reading the four v0.2 bundles produced, none of it derivable from `SPEC.md`:

| Finding | Consequence |
|---|---|
| No reference bundle declares `okf_version` at all; root `index.md` has no frontmatter | §12 is a MAY and upstream declines it. Our stricter posture became a *decision* instead of an assumption |
| `generated` is written for **human-authored** content (`by: human:…`) | It can never be an ownership or machine-generated predicate. Governs the collision gate |
| Real frontmatter is multi-line block YAML: block lists of multi-key mappings, nested mappings, flow sequences, booleans, dates | Re-sized the emitter work. It had been scoped against a list of strings |
| §7's canonical tool actor is `<producer>/<version>`, and upstream uses it | Counter-evidence to a recommendation we had already sent to another repo |

## Step 3a — Run upstream's own reader against our fixture (V-A8)

Every other test in the suite asks whether we agree with ourselves. This one asks
an independent implementation, and it is cheap enough that there is no excuse for
skipping it.

**It is not a dependency.** `okf/src/reference_agent/bundle/document.py` imports
only `yaml`, so it runs standalone under system Python against the pinned clone —
nothing is installed, `.venv` is never touched (the one-runtime-dependency rule and
its packaging test both stay intact), and nothing is written.

    import sys; sys.path.insert(0, "<clone>/okf/src")
    from reference_agent.bundle.document import OKFDocument, trust_tier, is_stale

Assert, over each file of the current version's golden bundle:

1. `OKFDocument.parse()` accepts our bytes. **This is the load-bearing one.** Our
   own parser is line-oriented and reads inline flow forms as opaque strings, so it
   structurally cannot tell us whether a real YAML consumer recovers them as
   structures. Only an outside parser can.
2. The values arrive as the *shapes* the profile intends — `generated` as a
   mapping, `sources` as a list of mappings — not merely as something that parsed.
3. `validate()` passes. Note what this does and does not buy:
   `REQUIRED_FRONTMATTER_KEYS = ("type",)`, so it checks one key — measured
   unchanged at `3fcbb9f` and at `ad30107`. Treating a green `validate()` as
   "upstream accepts our bundle" would overclaim badly; the parse in (1) and the
   shape assertions in (2) are where the signal is.
4. The semantic readers upstream ships run over our frontmatter without raising —
   at `3fcbb9f` and at `ad30107`: `trust_tier` (§5.3), `normalize_verified` (§5.2),
   `is_stale` (§5.5). **They take the frontmatter mapping, not the document.**
   Passing the `OKFDocument` raises `AttributeError: 'OKFDocument' object has no
   attribute 'get'` — a harness bug that reads exactly like an upstream
   incompatibility if the traceback is not read. Cost this round: one re-run.
   `OKFDocument.parse()` likewise takes the text only; there is no `path=` keyword.

**Record the type each value arrives as, not just that it parsed.** A real YAML
parser coerces, and the coercions are load-bearing facts about the value space —
this is where the v0.2 round's `okf_version` float finding came from (see the
alignment plan's A-E6). Upstream's `serialize()` also reflows inline flow forms to
block form, so an upstream round-trip produces bytes our own line-oriented parser
cannot read. That is expected and one-directional; do not "fix" it by emitting
block form.

*Result, v0.2 round at `3fcbb9f`:* 13/13 green against
`examples/ingest-golden-okf-v0-2/`.

*Result, 2026-08 round at `ad30107`:* 9/9 of our bundle documents parse, across all
four goldens, with the three semantic readers running over each without raising;
78/78 of upstream's own bundle documents parse. **Re-run 2026-08-23 when the pin
moved to `ad30107`: 9/9 parsed, 0 failures, 27/27 semantic-reader calls clean.** Do not compare the two counts —
the second harness asserted `validate()` on root `index.md` files too, which
correctly fails (an index carries no `type`), so it counts differently by
construction. **A clean sweep is worth nothing until the harness is shown able to
fail:** feed it a deliberate `type: [unclosed` and confirm it raises before you
report the zeroes.

## Step 4 — Produce the diff, classified

From the spec's own "Changes from vN" section **plus** the example evidence,
classify every change as:

- **Breaking** — name it, and measure consumer exposure in step 5.
- **Additive** — new optional fields, new conventional headings.
- **Deferred upstream** — a format specified but its runtime protocol left to a
  future revision. The format is supportable; an unspecified runtime is not
  something to build against. Record it as out of scope *on upstream's deferral*,
  not on our preference.
- **Observed but undocumented** — present in an official bundle, not found in the
  spec sections read. Record as unresolved. Do not silently promote it to either
  category. (v0.2 round: a `not:` family in `metrics/gross-margin.md`.)
- **Tightened in place, no version bump** — the normative text narrows what a key
  may hold, under the *same* version number. A value that conformed last round does
  not conform now, and nothing in the version string says so. (2026-08 round: every
  timestamp-valued key became an ISO 8601 datetime with an explicit offset, and
  `SPEC.md` still declares Version 0.2.) Two things follow. First, do not reach for
  the additive-profile machinery on reflex — there is no new version to express, and
  minting one would declare a version upstream has not. Second, **"conformant with
  v0.2" stops identifying one document**; the spec commit has to be named alongside
  the version, which is what `okf_spec_commit` is for.

## Step 5 — Measure our exposure and each consumer's, in that order

Ours: grep the profiles, the fixtures, and the emitter for every field the diff
touches. Cite file:line. Never reason about a key set — measure it.

Theirs: for each breaking change, state what would have to be true for it to reach
them, then **ask them to measure it against their real data** rather than telling
them whether they are affected. In the v0.2 round both `portfolio-optimiser-claude`
and `linkedin-studio` came back with measured zero exposure and found the framing
useful precisely because it was falsifiable.

## Step 6 — Plan additively, then pilot before general availability

The plan is a new profile, sequenced deliverables, and stated invariants. Rollout is
pilot-first: a pre-release tag to a small pilot set chosen for signal, revised on
their feedback, then GA. Flipping the `OKF_LATEST` alias **is** the GA event, not a
merge side effect.

State each pilot test's expected results **before** the run, numbered, plus what
would surprise us. An expectation we get wrong is a better result than a clean run;
only a stated expectation can be falsified by someone else's run.

**Name fixtures by repo and path, never by path alone.** In the v0.2 round both this
repo and `portfolio-optimiser-claude` had a directory named
`examples/ingest-golden-file/` holding *different* fixtures with zero content
overlap. A pilot baseline was agreed by name, and the name pointed at two files.

## Step 7 — Inform every OKF-consuming repo

All of them, not only the pilots, and not only when something breaks.

Send a **bounded loop of directed messages, not a broadcast** — a broadcast is
inherited by every future repo, which is the wrong lifetime for a
version-specific notice.

Each message carries:

1. The version, the pinned commit, and where to read it.
2. The breaking changes by name, with the shape of the exposure — so they can
   measure themselves rather than take our word.
3. **The black-box promise and its boundary**: their cost is a re-run; existing
   profiles are byte-stable; new parameters are keyword-only. And the part we
   cannot absorb, said plainly.
4. What we want back, if anything, and what happens if they say nothing.

Consumers to cover (2026-07): `portfolio-optimiser-commons` (spec authorship),
`portfolio-optimiser`, `portfolio-optimiser-claude`, `claude-code-llm-wiki`,
`catalog`, `okr`, `linkedin-studio`, `ms-ai-architect`. Re-derive this list each
round rather than trusting it — it is a premise like any other.

## Step 8 — Close the loop in the repo

The plan doc records the pinned commit, the classified diff, the measured exposure,
and every correction a consumer sent back. Corrections stay visible: a premise that
survived verification is only known to have survived if the check is recorded, and a
claim we withdrew is only safely withdrawn if the withdrawal is written where the
claim was.

## The weekly watch — `tools/okf_watch.py`

Answers one question on a schedule: *has canonical moved past our pin, and does
the move touch anything that bears the contract?* On a hit it sends a coord
message to this repo and, as FYI, to `.claude`. On a miss it prints one line and
exits 0.

It is deliberately **not** part of the package: it lives in `tools/`, outside
`src/`, so it never enters a wheel and a consumer's install surface is unchanged.
`tests/test_packaging.py` holds that as a promise rather than an accident.

**Run it:**

    python3 tools/okf_watch.py                 # the real weekly run
    python3 tools/okf_watch.py --dry-run       # print the messages, send nothing
    python3 tools/okf_watch.py --pin <sha>     # demonstrate the hit path (implies --dry-run)

**Cadence: weekly is the floor.** It costs one `git fetch` against a
`blob:none` mirror, so running it daily is not meaningfully more expensive.

### Three properties, and why each is load-bearing

1. **A failed call is never an empty result.** Every `git` invocation raises on a
   non-zero exit and carries stderr. The failure mode this closes is specific and
   has been met before: `grep … | head; echo $?` reports the exit status of
   `head`, and a query that failed then reads as a query that found nothing.
2. **It proves it can find, on every run.** Before believing any zero, the watch
   re-runs its full detect-and-classify path over `ad30107^1..ad30107` — a range
   known to have changed `SPEC.md`. If that comes back empty the query is broken,
   and the run aborts loudly instead of reporting a clean sweep. This is
   Verification-law face 4 made executable rather than remembered.
3. **It reports on change, not on state.** A JSON state file records which
   commits have already been announced, keyed on the pin. Moving the pin resets
   it, because a pin move means everything behind it was absorbed.

### Quiet is the enumerated list; signal is not

`QUIET_PREFIXES` names the paths measured *not* to bear the contract
(`.github/`, `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `LICENSE.md`, the HTML
viewer, generated `viz.html`). **Everything else reports.**

The inverse design — enumerate what counts as normative — can only match what
upstream has already invented, so anything new falls outside the list and the
watch goes quiet about it. That fails in the direction nobody notices.
Over-firing is visible and fixable by widening the quiet list; under-firing is
neither. **If the watch becomes noisy, widen `QUIET_PREFIXES`. Do not narrow the
signal.**

`README.md` is deliberately not quiet: upstream announced the repository move in
a README commit, and that move is the change with the longest reach this library
has seen.

### What the watch cannot do — state this when reporting it

It sees commits. It cannot see meaning.

When upstream tightened v0.2 in place on 2026-08-21, the watch would have fired
correctly, because `SPEC.md` changed. But **no commit list says "a value that
conformed last month does not conform now"**, and none says `is_stale` has
reversed for date-only inputs. Those were found by reading the diff and running
both readers against the same input. So the watch's output is always *run the
runbook*, never *here is your exposure* — and the message it sends says so in as
many words.

Two further blind spots, named rather than left to be discovered:

- **A silent relocation.** The last move was caught only because upstream
  committed a notice to `README.md`. A move announced anywhere other than this
  git history is invisible here.
- **A tightening with no commit at all** — a spec whose meaning is changed by an
  external document, an errata page, a changed reference implementation shipped
  under a different repository. Nothing local can see that. The release-checklist
  trigger, which reads rather than diffs, is the only cover.

The watch narrows the window between an upstream change and our noticing it. It
does not close it, and a session that treats a quiet watch as proof that upstream
is unchanged has made exactly the mistake the watch was built to prevent.

## Invariants this procedure protects

- No profile hard-codes an upstream version.
- No bundle declares a version its shape has not earned. The spec may permit
  declaring conformance without a checkpoint; we decline. *Conform first, claim
  after.*
- `DEFAULT` states commons' ingest-spec §5 layer — raised there, never patched here.
- `STRICT_V1` mirrors a consumer's ratified contract — never changed from here.
- The value of `okf_version` belongs to the catalog.
- Security stays the guard's domain in every version.