open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

docs(upstream): run the runbook against OKF at its new canonical repo

Browse source 
Operator GO 2026-08-23. Two findings outrank the change that triggered
the round.

OKF moved to GoogleCloudPlatform/open-knowledge-format (6265173). The
tree this repo pinned, knowledge-catalog path okf/, is now a frozen
snapshot by upstream's own notice, and the runbook's Trigger step named
it. The two trees have already diverged: the frozen copy carries a fix
(38c713f, eight tags: values as sequences rather than one plain scalar)
that the canonical repo does not.

The spec tightened without a version bump. SPEC.md still declares
Version 0.2, but every timestamp-valued key is now an ISO 8601 datetime
with an explicit offset, so a value that conformed in July does not
conform now under the same version number. Step 4 gained a fifth diff
class for it.

Our exposure is zero, measured: 10 of 10 timestamp values across the
four goldens already carry a Z offset, gated by _INGESTED_AT_RE. The
three tightened keys (stale_after, last_modified, usage_window) are
named but never written by any door. V-A8 at the new pin: 9/9 of our
bundle documents parse, 78/78 of upstream's do.

Two reference-reader behaviour changes recorded. A date-only
stale_after now reads as never stale rather than as stale -- silent,
fails open, and the class the black box cannot absorb. Timestamps now
survive a parse as written, so our ingested_at arrives as str where it
used to arrive as datetime.

No new profile: there is no new version to express. The pin move and
consumer notification are named as deliverables and deliberately not
taken here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013j8ZEcjiZQpz22fecNsyLZ
This commit is contained in:
Kjell Tore Guttormsen 2026-08-23 18:54:15 +02:00
commit b1c2f5ecee
2 changed files with 390 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

351
docs/plan/okf-2026-08-timestamp-tightening.md Normal file
View file

@ -0,0 +1,351 @@
# Upstream OKF round, 2026-08-23 — the timestamp tightening, and the repository move
Runbook run of `docs/upstream-okf-upgrade-runbook.md`, executed 2026-08-23 on an
operator GO. Previous round: `docs/plan/okf-v0.2-alignment.md` (v0.1 → v0.2, pinned
`3fcbb9f`).
This round found two things the runbook was not shaped for, and both matter more
than the change that triggered it:
1. **OKF moved to its own repository.** The tree we had pinned is now a declared
frozen snapshot. The runbook's Trigger step pointed at it by name.
2. **The spec changed without a version bump.** `SPEC.md` still says Version 0.2.
Step 4's classification vocabulary had no bucket for that.
The triggering change itself — every timestamp becomes an ISO 8601 datetime with an
explicit offset — costs this library nothing. That conclusion is measured, not
assumed; the denominators are below.
---
## Step 1 — Pins
Everything below was read at a fixed commit, never at a branch.
| What | Commit | Date | Role |
|---|---|---|---|
| `GoogleCloudPlatform/open-knowledge-format` | `ad30107c31c06aec8a7d5636e0d1058118604e6f` | 2026-08-21 | **New canonical source.** Read at this pin |
| `GoogleCloudPlatform/knowledge-catalog` | `395a0c311979858dcecfed767810658baf3ca3f3` | 2026-08-22 | Frozen `okf/` snapshot, read for divergence |
| `GoogleCloudPlatform/knowledge-catalog` | `3fcbb9f828c2f23d109c855ee403c3a4c81f3a96` | 2026-07-24 | **Our current pin**, the v0.2-round baseline |
The order that authorized this run named `62432a09` as the target. That commit is
real (`okf: make every timestamp an ISO 8601 datetime with an explicit offset
(#323)`, 2026-08-20) but it is a commit in the *frozen* tree, and it was not the
head of anything by the time this ran. Verified before use rather than adopted:
`knowledge-catalog` had advanced 53 commits past our pin, and the canonical tree is
a different repository.
Local read-only mirrors: `~/repos/_okf-upstream` (knowledge-catalog, pre-existing)
and `~/repos/_okf-canonical` (new, cloned this round, push URL disabled).
## Step 2 — Enumeration, with denominators
**Canonical repo at `ad30107`: 132 tracked files.**
| Top level | Files |
|---|---|
| `bundles/` | 83 (acme_retail 19, crypto_bitcoin 16, ga4 15, stackoverflow 33) |
| `src/` | 27 |
| `tests/` | 8 |
| `samples/` | 6 |
| `connectors/` | 1 — **new**, not present in the tree we had pinned |
| root files | 7 (`SPEC.md`, `README.md`, `pyproject.toml`, `LICENSE.md`, `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `.gitignore`) |
By extension: 89 `md`, 31 `py`, 5 `html`, 3 `txt`, 1 each of `toml`, `js`, `css`,
`gitignore`.
Of the 53 `knowledge-catalog` commits between our pin and its current head, **3
touched `okf/`**. The other 50 are `toolbox/` work (`mdcode`, `enrichment`) and are
out of scope for this library — stated with the denominator, not as an impression.
## Step 3 — The move, read from the primary source
`6265173 okf: point readers at the dedicated OKF repository (#324)` adds a notice to
`okf/README.md`:
> **OKF now lives in its own repository:
> GoogleCloudPlatform/open-knowledge-format.** … **Stop using the copy under `okf/`
> in this repository.** It is a frozen snapshot, no longer maintained, and anything
> built against it will drift out of date.
This is the finding with the longest reach. "Always latest" is a promise about a
tree, and the tree changed identity. The runbook's Trigger step — *Check
`GoogleCloudPlatform/knowledge-catalog`, path `okf/`* — would have kept sending
future rounds at a snapshot that upstream says will drift, and each such round would
have reported "no change" truthfully and uselessly.
### The two trees have already diverged
The canonical repo does **not** carry `38c713f okf: write stackoverflow bundle tags
as YAML lists (#293)` (2026-08-14), which is present in the snapshot it declares
obsolete. Measured across the 8 affected files, each differing by exactly the `tags:`
line and nothing else:
```
canonical: tags: stackoverflow, posts, questions # one plain scalar
frozen kc: tags: [stackoverflow, posts, questions] # a sequence
```
Upstream's own commit message for that fix records what the scalar form cost them: a
JavaScript string is iterable, so each one expanded into one label *per character*.
The canonical repository currently ships the form upstream had already diagnosed as
corrupting.
We are not exposed: this library neither writes nor reads OKF `tags` (measured — one
occurrence of the substring across 11 modules, in `extract.py:86`, about HTML tags).
Door C is exposed only in the sense that it merges external concepts **verbatim** by
design, so a consumer importing the canonical `stackoverflow` bundle would carry the
scalar form through unchanged. That is the verbatim invariant working, not a defect.
## Step 4 — The diff, classified
Between our pin `3fcbb9f:okf/` (129 files) and canonical `ad30107` (132 files), every
`.py` file differs by a 14-line Apache licence header added at the move. Confirmed as
an artifact rather than assumed: stripping those 14 lines makes the files byte-
identical. Excluding it, the substantive set is small.
**Normative (`SPEC.md`), 22 changed lines, all one change.** Every timestamp-valued
key becomes an ISO 8601 datetime with an explicit UTC offset. New sentence in §5:
> Every timestamp-valued key in OKF is an ISO 8601 datetime with an explicit UTC
> offset, for example `2026-06-30T14:00:00Z`.
Three keys change form; the rest of the diff is the examples and prose following
them:
| Key | Was | Is |
|---|---|---|
| `stale_after` (§5.5) | `YYYY-MM-DD` | datetime with offset |
| `sources[].last_modified` (§5.1) | `YYYY-MM-DD` | datetime with offset |
| `usage_window.{from,to}` (§5.1) | date range | datetime range |
`generated.at` and `verified.at` were already datetimes with `Z` and are untouched.
**Not a version bump.** `SPEC.md` still declares Version 0.2, and §12's minor/major
rules are unchanged. This is a *tightening of the normative text in place*: a value
that conformed on 2026-07-24 does not conform now, under the same version number. The
runbook's Step 4 vocabulary (Breaking / Additive / Deferred upstream / Observed but
undocumented) has no bucket for it, which is why it is named here as a fifth class —
**tightened in place, no version bump** — and why declaring conformance to "v0.2"
without also naming a commit is now ambiguous. Our `okf_spec_commit` key already
anticipated this; this round is the first time it earns its keep.
**Reference reader (`src/reference_agent/bundle/document.py`), 34 substantive
lines, two behaviour changes.**
*Change 1 — a date-only `stale_after` is now silently ignored.* `is_stale` returns
`False` for any value without a `T`, on the stated reasoning that a date-only value
"names a different instant in every timezone, so it is ignored rather than guessed
at". Measured on identical input at both pins, with `now` set 17 months past the
date:
| `stale_after` | reader @ `3fcbb9f` | reader @ `ad30107` |
|---|---|---|
| `2026-12-31` | `is_stale=True` | **`is_stale=False`** |
| `2026-12-31T00:00:00Z` | `True` | `True` |
| `2026-12-31T00:00:00+00:00` | — | `True` |
This is the class the black-box promise explicitly cannot absorb. No library change
makes an already-authored date-only `stale_after` mean what its author meant; it now
reads as *never stale* rather than as *stale*, which fails open. Nothing in this
library emits `stale_after`, so our first-order exposure is zero — but any consumer
who authored one has content whose meaning reversed without their file changing.
*Change 2 — timestamps now survive a parse as the text the author wrote.* Upstream
subclasses `SafeLoader` and drops the `tag:yaml.org,2002:timestamp` implicit
resolver, because PyYAML implements YAML 1.1 and was rewriting frontmatter on a
round-trip. Measured against our own golden, same bytes, both pins:
| Key | type @ `3fcbb9f` | type @ `ad30107` |
|---|---|---|
| `ingested_at` | `datetime` | **`str`** (`'2026-07-16T12:00:00Z'`) |
| `generated.at` | `datetime` | **`str`** |
| `okf_version` | `float` (`0.2`) | `float` (`0.2`) — unchanged |
This is a change in our favour: the value an upstream consumer recovers from our
bundle is now the literal we wrote. It also narrows the A-E6 finding — the float
coercion on `okf_version` survives, so that one is still real and still upstream's.
**Additive:** `connectors/gcp-knowledge-catalog.md` (new file), `CONTRIBUTING.md` and
`CODE_OF_CONDUCT.md` (repository furniture that came with the move).
**Deferred upstream (unchanged):** §12 still defers the receipt and verdict wire
formats, the attester ABI, attestation caching, and semantic-layer templates. Our
`Attested Computation` support stays format-only on upstream's deferral, not on our
preference.
**Observed but undocumented:** none this round.
## Step 3a — V-A8, upstream's reader against our fixtures
Run with `/usr/local/bin/python3` (3.14.0, PyYAML 6.0.3) against the pinned clone.
The `.venv` was not touched and nothing was installed; the one-runtime-dependency
rule and its packaging test are intact.
| Bundle | `.md` files | parsed | parse failures |
|---|---|---|---|
| our `ingest-golden-file` | 3 | 3 | 0 |
| our `ingest-golden-http` | 2 | 2 | 0 |
| our `ingest-golden-okf-v0-2` | 2 | 2 | 0 |
| our `ingest-golden-sql` | 2 | 2 | 0 |
| upstream `acme_retail` | 17 | 17 | 0 |
| upstream `crypto_bitcoin` | 15 | 15 | 0 |
| upstream `ga4` | 14 | 14 | 0 |
| upstream `stackoverflow` | 32 | 32 | 0 |
**9 of 9 of our own bundle documents parse under the new reader**, and
`trust_tier`, `normalize_verified` and `is_stale` run over each without raising.
A clean sweep is a weak result unless the harness is shown to be able to fail, so it
was put to a known-positive: a document with `type: [unclosed` raises
`OKFDocumentError` as expected, while a well-formed control parses. The zeroes above
are measurements, not the absence of one.
Two notes on what V-A8 does *not* buy, both carried forward from the v0.2 round:
- `validate()` checks `REQUIRED_FRONTMATTER_KEYS = ("type",)` — one key. It fails on
our root `index.md`, correctly: an index is not a concept and carries no `type` by
§12. The runbook's recorded "13/13" counted concept documents; this run's harness
asserted `validate()` on index files too and so reports one expected failure per
bundle root. Not a regression — a stricter harness. Do not read the two counts as
comparable.
- Upstream's `serialize()` still reflows inline flow forms to block form. Unchanged,
one-directional, and still not a reason to emit block form (D4).
## Step 5 — Exposure
### Ours: measured zero
Every timestamp-valued byte this library emits, across all four golden bundles —
denominator 10 of 10, every one already conformant:
```
examples/ingest-golden-file/expected-bundle/ingest-orders.md:5 ingested_at: 2026-07-16T12:00:00Z
examples/ingest-golden-file/expected-bundle/ingest-orders.md:7 generated: { by: process:okf-ingest, at: 2026-07-16T12:00:00Z }
... 4 further bundles, same two keys, same form
```
The gate is `materialize.py:37`,
`_INGESTED_AT_RE = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$")`, enforced by
`validate_ingested_at` (`materialize.py:49`) and called on every Door A run
(`materialize.py:386`). `generated.at` derives from the same validated value via
`profile.ownership.stamp(ingested_at)` (`materialize.py:235`), so both keys are
covered by one check. The spec's own example of an explicit UTC offset is `Z`, so
this form conforms as written.
The three keys the tightening actually changes are not written by any door:
- `stale_after` is **named but not written** — `profiles.py:757` places it in
`_OKF_V0_2_KEY_ORDER` so a caller can emit it in canonical order and the schema can
judge a document carrying one. The comment at `profiles.py:736` says so directly:
*"Naming a family is not writing it … Door A writes none of them."*
- `last_modified`, `usage_window`, `usage_count` are not emitted at all.
`materialize.py:188` records the decision: a manifest source has no `author` and no
`last_modified`, *"and inventing them would be writing fields with no reader."*
Nothing in this library validates a timestamp value other than `ingested_at`
(measured across 11 modules; the query was validated against a known-positive control
before its empty result was believed).
One test fixture carries a now-non-conformant literal:
`tests/test_okf_v0_2_characterization.py:188` uses `stale_after: 2027-01-01` inside a
§14 *consumer-tolerance* case — a document a conformant consumer MUST NOT reject.
That is still exactly what it tests, and the date-only form arguably tests it better
now. **No change proposed.** Flagged so a later reader does not "fix" it.
### Consumers: framed for them to measure, not decided for them
The list was re-derived against disk rather than trusted from the runbook, and all
eight are present (two were initially recorded as absent from a guessed path — a
wrong-path negative, corrected before use):
`portfolio-optimiser-commons`, `portfolio-optimiser`, `portfolio-optimiser-claude`,
`claude-code-llm-wiki`, `catalog`, `okr`, `linkedin-studio`, `ms-ai-architect`.
What has to be true for the tightening to reach a consumer — one falsifiable question
each, for them to run against their real data:
1. **Do you author `stale_after`, `last_modified`, or `usage_window` anywhere?** If
no, your exposure is zero and this round costs you nothing. If yes, and the value
is date-only, an OKF v0.2 reader at the current spec now reads it as *never stale*
rather than as *stale*. It fails open, and it is silent.
2. **Do you declare conformance as "v0.2" without naming a commit?** That string no
longer identifies one document. Naming the commit alongside it is the fix.
3. **Do you read OKF from `GoogleCloudPlatform/knowledge-catalog`, path `okf/`?**
That copy is frozen by upstream's own notice, and has already diverged.
Nothing here asserts which consumers are affected. The v0.2 round showed the
falsifiable framing is what made the answers useful.
## Step 6 — The plan
**No new profile.** Support is additive by rule, but there is nothing additive to
express: the spec did not bump, the three tightened keys are ones we do not write,
and what we do write already conforms. Inventing an `OKF_V0_2_1` profile would
declare a version upstream has not minted. *Conform first, claim after.*
**Deliverables, in order:**
1. **Correct the runbook's Trigger step** to name
`GoogleCloudPlatform/open-knowledge-format` as canonical, record that
`knowledge-catalog/okf/` is frozen and already divergent, and add the fifth
diff class (*tightened in place, no version bump*) to Step 4. Done this round —
leaving a known-false instruction in the procedure would send the next run at a
dead tree.
2. **Move the pin** `3fcbb9f` → `ad30107`, and with it the repository. Cost, measured:
**zero bytes in our fixtures.** `okf_spec_commit` is a key the `STRICT_V1` profile
requires (`profiles.py:721,726`); its *value* is the caller's under D5, and no
shipped fixture carries one. The cost lands where D5 says it should — on the
caller that stamps it, as a re-run. This is a separate decision and is **not taken
here**, per the order.
3. **Re-record V-A8 at the new pin** in `docs/plan/okf-v0.2-alignment.md`'s
verification table, which currently reads *"DONE 2026-07-31 @ `3fcbb9f`: 13/13"*.
Pending the pin decision, so the two move together.
4. **Notify consumers** (Step 7) with the three questions above. Not sent this
round — the order scoped this to analysis and a plan document, and notification
is a sent action.
**No pilot, no GA event.** Pilot-first governs a *new upstream version* reaching a
new profile. There is no new version and no new profile, so `OKF_LATEST` does not
move and there is nothing to flip. Saying this explicitly because the standing
policy's machinery is easy to run out of habit.
**Guard seam:** nothing in this round touches it. The tightening is a value-format
question inside frontmatter we already gate; the persist gates are unchanged, and the
standing posture that everything entering a bundle passes the guard is unaffected.
Noted, not built.
## Step 8 — Verification log
Every claim above and the command class that produced it. Corrections stay visible.
| Claim | How verified |
|---|---|
| OKF moved; `okf/` is frozen | `git show 6265173` — primary source, upstream's own notice text |
| Canonical pin `ad30107`, 132 files | `git log -1`, `git ls-files \| wc -l` at the pin |
| 3 of 53 kc commits touched `okf/` | `git log --oneline 3fcbb9f..origin/main -- okf` |
| `.py` diffs are a licence header | Stripped 14 lines, files byte-identical (`diff` empty) |
| Spec change is timestamps only | `diff -u` of the two `SPEC.md` files, whole diff read |
| Still Version 0.2, no bump | `SPEC.md:3`, §12 read in full at the new pin |
| Canonical lacks `38c713f` | `diff -rq` frozen-vs-canonical; 8 files, each 2 lines, all `tags:` |
| We never write `tags` | `grep` across 11 modules — 1 unrelated hit (`extract.py:86`) |
| `is_stale` reversal | Same input run against both pinned readers, `now=2027-06-01` |
| `ingested_at` type change | V-A8 type dump, same golden bytes, both pins |
| Our exposure is 10/10 conformant | Frontmatter scan over all 10 `examples/**/*.md` |
| No other timestamp validation | `grep` over 11 modules, **validated against a known-positive control first** |
| `okf_spec_commit` has no fixture value | `grep examples/` exit 1, after `git grep` confirmed the key exists in 3 tracked places |
| V-A8 harness can detect failure | Deliberate `type: [unclosed` → `OKFDocumentError`; control parses |
| All 8 consumers present | `find ~/repos -maxdepth 3 -type d -name <n>`, with a known-positive control |
| Suite green | `.venv/bin/python -m pytest -q` → 596 passed |
**Corrections made during this run, kept visible:**
- The order named `62432a09` as the target. It is a commit in the frozen tree, not
the canonical head. Verified before it was used as a premise.
- `linkedin-studio` and `ms-ai-architect` were first recorded as absent, from a
guessed path (`~/repos/<name>`). A wider search found both under
`~/repos/ktg-plugin-marketplace/`. A negative from a guessed path is not a
measurement.
- An early `grep … | head; echo $?` reported the exit status of `head`, not of
`grep` — a negative that measured nothing. Re-run with the status of the actual
query before its empty result was used.

42
docs/upstream-okf-upgrade-runbook.md
View file

@ -42,9 +42,27 @@ notice. So the re-check is an item on the release checklist — run it at every
release of this library, and record the result **even when unchanged**, because an
unrecorded check is indistinguishable from a skipped one.
Check `GoogleCloudPlatform/knowledge-catalog`, path `okf/`. A version bump appears
as a commit against `okf/SPEC.md` §12 and, in the v0.2 round, as an explicit
migration commit (`okf: migrate format and tooling to Open Knowledge Format v0.2`).
Check `GoogleCloudPlatform/open-knowledge-format`. **That is the canonical home of
the spec, the reference agent and the sample bundles as of 2026-08-21.** A version
bump appears as a commit against `SPEC.md` §12 and, in the v0.2 round, as an
explicit migration commit (`okf: migrate format and tooling to Open Knowledge
Format v0.2`).
**Do not check `GoogleCloudPlatform/knowledge-catalog`, path `okf/`.** That copy is
a frozen snapshot by upstream's own notice (`6265173`, "anything built against it
will drift out of date"), and this repo was pinned to it until the 2026-08-23 round.
Two consequences, both measured that round and neither hypothetical:
- **The two trees have already diverged**, and not only in the direction you would
expect: the frozen copy carries a fix (`38c713f`, eight `tags:` values written as
sequences rather than as one plain scalar) that the canonical repo does not. The
canonical tree is authoritative for the *spec*; it is not automatically a superset.
- **A round run against the frozen tree reports "no change" truthfully and
uselessly** — the exact shape of a negative result that is not a measurement.
*Prevents:* tracking a tree that has stopped being the thing you promised to track.
"Always latest" is a promise about an object with an identity, and the identity can
move without the version number moving.
## Step 1 — Pin before reading anything
@ -155,6 +173,15 @@ block form.
*Result, v0.2 round at `3fcbb9f`:* 13/13 green against
`examples/ingest-golden-okf-v0-2/`.
*Result, 2026-08 round at `ad30107`:* 9/9 of our bundle documents parse, across all
four goldens, with the three semantic readers running over each without raising;
78/78 of upstream's own bundle documents parse. Do not compare the two counts —
the second harness asserted `validate()` on root `index.md` files too, which
correctly fails (an index carries no `type`), so it counts differently by
construction. **A clean sweep is worth nothing until the harness is shown able to
fail:** feed it a deliberate `type: [unclosed` and confirm it raises before you
report the zeroes.
## Step 4 — Produce the diff, classified
From the spec's own "Changes from vN" section **plus** the example evidence,
@ -169,6 +196,15 @@ classify every change as:
- **Observed but undocumented** — present in an official bundle, not found in the
spec sections read. Record as unresolved. Do not silently promote it to either
category. (v0.2 round: a `not:` family in `metrics/gross-margin.md`.)
- **Tightened in place, no version bump** — the normative text narrows what a key
may hold, under the *same* version number. A value that conformed last round does
not conform now, and nothing in the version string says so. (2026-08 round: every
timestamp-valued key became an ISO 8601 datetime with an explicit offset, and
`SPEC.md` still declares Version 0.2.) Two things follow. First, do not reach for
the additive-profile machinery on reflex — there is no new version to express, and
minting one would declare a version upstream has not. Second, **"conformant with
v0.2" stops identifying one document**; the spec commit has to be named alongside
the version, which is what `okf_spec_commit` is for.
## Step 5 — Measure our exposure and each consumer's, in that order