okf-wiki 1.1.1

First public release, published as one snapshot. What changed between
versions is in CHANGELOG.md.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-11-14 21:03:20 +01:00
commit 0fd86a23a3
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
227 changed files with 96077 additions and 0 deletions

117
docs/upstream-runbook.md Normal file
View file

@ -0,0 +1,117 @@
# Upstream OKF upgrade runbook
Rule R26 (`docs/rule-decisions.md`): this repository supports the latest published Google Open Knowledge Format, and
upstream is checked at least weekly. Rule R32: every upstream release is handled with this runbook. `CLAUDE.md`
§ The standard: `SPEC.md` at the pinned commit is the source of truth.
A standing policy without a procedure is an intention. Each step below names the failure it prevents; each of those
failures happened in an earlier round of the same work, not in theory.
## Trigger
The morning job (`tools/upstream_watch.py`, daily at 07:15) sends a coord message when canonical has moved past our
pin in a path that bears the contract, and when issues or pull requests are new or changed. The message is the trigger
to run this runbook, never its result: the job sees commits, numbers and times, not meaning.
Run the runbook also without a message when a session reads upstream for another reason and finds a change. Record the
result in `PLAN.md § Logg` **even when nothing changed**: an unrecorded check cannot be told from a skipped one.
The canonical home is `GoogleCloudPlatform/open-knowledge-format`. Our pin is `PIN` in `tools/upstream_watch.py`.
## Step 1: pin before reading anything
Record the exact commit SHA you are about to read, and read everything at it.
*Prevents:* reading a moving branch and treating the result as fact. Two readers of `main` are one unstable reference
read twice, and a spec edited after a migration commit makes "the migration commit" and "the current spec" two
different objects.
## Step 2: enumerate the whole repository tree
List every file at the pinned commit before deciding what to read, starting at the root:
git ls-tree -r --name-only <sha> | awk -F/ '{print $1}' | sort -u # top-level entries
git ls-tree -r --name-only <sha> | sed 's/.*\.//' | sort | uniq -c # kinds of file
*Prevents:* assuming the folder you know is the one that matters. A negative claim from a partial enumeration ("there
is no validator upstream") reads exactly like a measured one, and was wrong when it was last made.
## Step 3: read the shipped example bundles, not only the normative text
A spec says what is permitted; an example shows what upstream emits. Read the bundles upstream ships under the pinned
commit at byte level: check for a BOM, the final newline, and whether a value is quoted. A fetch that converts to
markdown or summarises destroys exactly those three properties.
*Prevents:* freezing a page shape that conforms to the text but not to what upstream's own tools write. Earlier rounds
learned from the examples, not the spec, that the root `index.md` carries no frontmatter, that `generated` is written
for human-authored content too, and that real frontmatter is block YAML with nested mappings.
## Step 3a: run upstream's own reader against our pages
The reference reader (`src/reference_agent/bundle/document.py` at the last pin) imports only `yaml`. Run it from the
pinned clone, without installing anything, over a built collection and over the mutant gate's known-positive bundle
(`gates-local/mutants/`), and record per page:
1. `OKFDocument.parse()` accepts the bytes (it takes the text only).
2. The values arrive as the shapes we intend: `generated` a mapping, `sources` a list of mappings, and our `x-` fields
as the types `docs/x-fields.md` gives them. Record the type each value arrives as, not only that it parsed.
3. `validate()` passes. It checks one key (`type`), so a green `validate()` alone is not "upstream accepts the page".
4. Upstream's semantic readers (`trust_tier`, `normalize_verified`, `is_stale`) run over the frontmatter mapping, not
the document, without raising.
**A clean sweep is worth nothing until the harness is shown able to fail:** feed it a page with `type: [unclosed` and
confirm it raises before you report the zeroes. Google's viewer is the second outside reader: criterion 5 requires it
to read every collection without an error.
## Step 4: classify the diff
From the spec's own "Changes from vN" section plus the evidence of steps 3 and 3a, put every change in one class:
- **Breaking:** name it, and measure our exposure in step 5.
- **Additive:** new optional fields, new conventional headings.
- **Deferred upstream:** a format specified but its runtime left to a later revision. Out of scope on upstream's
deferral, not on our preference.
- **Observed but undocumented:** present in an official bundle, absent from the spec sections read. Recorded as open;
never promoted silently to another class.
- **Tightened in place, no version bump:** the text narrows what a key may hold under the same version number. A value
that conformed last round does not conform now, so "conformant with v0.2" must name the spec commit too.
## Step 5: measure our exposure
For every change the diff touches, search the code that writes and checks pages and cite `file:line`: the auditor
(`okfwiki/audit.py`), the assembly and link code (`okfwiki/kassemble.py`, `okfwiki/klink.py`), the skill
(`skills/okf-wiki/SKILL.md`), the `x-` field list (`docs/x-fields.md`), the gates under `gates/` and their cases under
`gates-local/`. Never reason about a key set; measure it.
Where upstream now has a standard field for something we carry in an `x-` field, the `x-` field moves to it
(`CLAUDE.md` § The standard). That is a format change: it is proposed to the operator with the measurement, and it is
his decision.
There are no consumers of this repository yet (R33 is dropped). When something depends on `okf-wiki`, add a step
here that tells each consumer what changed and lets them measure their own exposure.
## Step 6: move the pin
Only after steps 1 to 5 are written down:
1. Change `PIN` in `tools/upstream_watch.py` and the test that holds it (`tests/test_upstream_watch.py`), test first.
2. Run the three test suites, `mypy --strict`, the mutant gate (`gates/README.md`) and the check against Google's
viewer, and record the numbers.
3. Write the pinned commit, the classified diff, the measured exposure and every correction in `PLAN.md § Logg`. A
claim withdrawn is only safely withdrawn where the claim was written.
Moving the pin resets what the morning job counts as already announced, because everything behind the new pin has
been absorbed.
## What the morning job cannot see
- **Meaning.** When upstream tightened v0.2 in place, a watch on `SPEC.md` fires correctly, but no commit list says "a
value that conformed last month does not conform now". The message therefore always says "run the runbook", never
"here is your exposure".
- **A silent relocation.** The last move of the repository was announced in a `README.md` commit, which is why
`README.md` is not on the quiet list. A move announced anywhere else is invisible to the job.
- **A tightening with no commit:** a changed reference implementation shipped elsewhere, or errata outside the
repository. Reading upstream, as this runbook does, is the only cover.
The job narrows the window between an upstream change and our noticing it. It does not close it: a quiet morning is not
proof that upstream is unchanged.