okf-wiki/docs/upstream-runbook.md
Kjell Tore Guttormsen 0fd86a23a3
okf-wiki 1.1.1
First public release, published as one snapshot. What changed between
versions is in CHANGELOG.md.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
2026-11-14 21:03:20 +01:00

7 KiB

Upstream OKF upgrade runbook

Rule R26 (docs/rule-decisions.md): this repository supports the latest published Google Open Knowledge Format, and upstream is checked at least weekly. Rule R32: every upstream release is handled with this runbook. CLAUDE.md § The standard: SPEC.md at the pinned commit is the source of truth.

A standing policy without a procedure is an intention. Each step below names the failure it prevents; each of those failures happened in an earlier round of the same work, not in theory.

Trigger

The morning job (tools/upstream_watch.py, daily at 07:15) sends a coord message when canonical has moved past our pin in a path that bears the contract, and when issues or pull requests are new or changed. The message is the trigger to run this runbook, never its result: the job sees commits, numbers and times, not meaning.

Run the runbook also without a message when a session reads upstream for another reason and finds a change. Record the result in PLAN.md § Logg even when nothing changed: an unrecorded check cannot be told from a skipped one.

The canonical home is GoogleCloudPlatform/open-knowledge-format. Our pin is PIN in tools/upstream_watch.py.

Step 1: pin before reading anything

Record the exact commit SHA you are about to read, and read everything at it.

Prevents: reading a moving branch and treating the result as fact. Two readers of main are one unstable reference read twice, and a spec edited after a migration commit makes "the migration commit" and "the current spec" two different objects.

Step 2: enumerate the whole repository tree

List every file at the pinned commit before deciding what to read, starting at the root:

git ls-tree -r --name-only <sha> | awk -F/ '{print $1}' | sort -u     # top-level entries
git ls-tree -r --name-only <sha> | sed 's/.*\.//' | sort | uniq -c    # kinds of file

Prevents: assuming the folder you know is the one that matters. A negative claim from a partial enumeration ("there is no validator upstream") reads exactly like a measured one, and was wrong when it was last made.

Step 3: read the shipped example bundles, not only the normative text

A spec says what is permitted; an example shows what upstream emits. Read the bundles upstream ships under the pinned commit at byte level: check for a BOM, the final newline, and whether a value is quoted. A fetch that converts to markdown or summarises destroys exactly those three properties.

Prevents: freezing a page shape that conforms to the text but not to what upstream's own tools write. Earlier rounds learned from the examples, not the spec, that the root index.md carries no frontmatter, that generated is written for human-authored content too, and that real frontmatter is block YAML with nested mappings.

Step 3a: run upstream's own reader against our pages

The reference reader (src/reference_agent/bundle/document.py at the last pin) imports only yaml. Run it from the pinned clone, without installing anything, over a built collection and over the mutant gate's known-positive bundle (gates-local/mutants/), and record per page:

  1. OKFDocument.parse() accepts the bytes (it takes the text only).
  2. The values arrive as the shapes we intend: generated a mapping, sources a list of mappings, and our x- fields as the types docs/x-fields.md gives them. Record the type each value arrives as, not only that it parsed.
  3. validate() passes. It checks one key (type), so a green validate() alone is not "upstream accepts the page".
  4. Upstream's semantic readers (trust_tier, normalize_verified, is_stale) run over the frontmatter mapping, not the document, without raising.

A clean sweep is worth nothing until the harness is shown able to fail: feed it a page with type: [unclosed and confirm it raises before you report the zeroes. Google's viewer is the second outside reader: criterion 5 requires it to read every collection without an error.

Step 4: classify the diff

From the spec's own "Changes from vN" section plus the evidence of steps 3 and 3a, put every change in one class:

  • Breaking: name it, and measure our exposure in step 5.
  • Additive: new optional fields, new conventional headings.
  • Deferred upstream: a format specified but its runtime left to a later revision. Out of scope on upstream's deferral, not on our preference.
  • Observed but undocumented: present in an official bundle, absent from the spec sections read. Recorded as open; never promoted silently to another class.
  • Tightened in place, no version bump: the text narrows what a key may hold under the same version number. A value that conformed last round does not conform now, so "conformant with v0.2" must name the spec commit too.

Step 5: measure our exposure

For every change the diff touches, search the code that writes and checks pages and cite file:line: the auditor (okfwiki/audit.py), the assembly and link code (okfwiki/kassemble.py, okfwiki/klink.py), the skill (skills/okf-wiki/SKILL.md), the x- field list (docs/x-fields.md), the gates under gates/ and their cases under gates-local/. Never reason about a key set; measure it.

Where upstream now has a standard field for something we carry in an x- field, the x- field moves to it (CLAUDE.md § The standard). That is a format change: it is proposed to the operator with the measurement, and it is his decision.

There are no consumers of this repository yet (R33 is dropped). When something depends on okf-wiki, add a step here that tells each consumer what changed and lets them measure their own exposure.

Step 6: move the pin

Only after steps 1 to 5 are written down:

  1. Change PIN in tools/upstream_watch.py and the test that holds it (tests/test_upstream_watch.py), test first.
  2. Run the three test suites, mypy --strict, the mutant gate (gates/README.md) and the check against Google's viewer, and record the numbers.
  3. Write the pinned commit, the classified diff, the measured exposure and every correction in PLAN.md § Logg. A claim withdrawn is only safely withdrawn where the claim was written.

Moving the pin resets what the morning job counts as already announced, because everything behind the new pin has been absorbed.

What the morning job cannot see

  • Meaning. When upstream tightened v0.2 in place, a watch on SPEC.md fires correctly, but no commit list says "a value that conformed last month does not conform now". The message therefore always says "run the runbook", never "here is your exposure".
  • A silent relocation. The last move of the repository was announced in a README.md commit, which is why README.md is not on the quiet list. A move announced anywhere else is invisible to the job.
  • A tightening with no commit: a changed reference implementation shipped elsewhere, or errata outside the repository. Reading upstream, as this runbook does, is the only cover.

The job narrows the window between an upstream change and our noticing it. It does not close it: a quiet morning is not proof that upstream is unchanged.