llm-ingestion-okf/src/llm_ingestion_okf/profiles.py
Kjell Tore Guttormsen 8318605e34 feat(profiles): DEFAULT stamps commons' O2 generated, V1 executed
Commons ratified V1 2026-08-02 and executed it at `54e0ec7`; verified
against their tree rather than taken on report. ingest-spec.md:217 now
defines `generated` as `{ by: process:okf-ingest, at: <ingested_at> }`,
unquoted, `at` repeating `ingested_at` verbatim. `generated: true` no
longer appears in the spec.

`DEFAULT` states commons' §5 layer, so its stamp is theirs to decide.
`DEFAULT.ownership` gains the actor; the four goldens this repo's plan
named in advance were regenerated by RUNNING the materializer, each on
its own case's `ingested-at.txt`. The v0.2 golden was untouched, as
predicted -- it has carried the O2 form since D5.

Not a migration onto OKF v0.2: `DEFAULT` stays v0.1 on every axis
upstream owns and still emits no `sources`. Commons' spec and the Google
version are independent axes, and comments that narrated them as one
were rewritten rather than left to mislead. README and CLAUDE.md said
the additive rule without that boundary, which would have told a
consumer their DEFAULT bytes can never move; both now state it.

V-A3 is amended, not dropped. `DEFAULT` must OWN the mapping it now
writes -- a profile refusing its own output fires the collision gate on
files its own previous run wrote -- while a mapping naming a foreign
actor, or §7's `human:` actor on curated content, stays unowned. That
half is what carried the safety and it is asserted directly.

§11's stamp-integrity condition moved with the value: the forgeable
stamp was `true` and is now the mapping naming the ingest actor. The
defence was never the value -- the §3 scan globs `ingest-*.md`, so a
Door C import is unreachable however well it forges. Second spoof test
added; both were hand-mutated (glob widened to `*.md`) to confirm they
can fail.

The characterization test derived its foreign-stamp fixture from the
literal `generated: true`, which V1 leaves without a referent -- a
silent no-op waiting to happen. It now derives the needle from the
profile and asserts the substitution occurred.

Door B is deliberately untouched: not the ingest-spec's, marker is
`generated` + `source_file`, disjoint from Door A's `ingest_manifest`,
and the divergence predates V1.

Nothing released or notified. The pilot set pins `v0.5.0a2`, not `main`,
so this is invisible to portfolio-optimiser's freeze and demo; the
consumer exposure report is owed at the release that carries this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VwcjUXbKySLbEG5WqTNkta
2026-08-09 12:29:05 +02:00

854 lines
39 KiB
Python

"""The bundle contract as configuration (Phase 3).
What a valid bundle looks like — which concept types exist, which frontmatter
keys are emitted and in which order, which filename namespaces the doors own,
and what an index line looks like — is a profile, not a set of constants
scattered across the doors. `DEFAULT` is exactly the ingest-spec v1 + Phase 2
contract, so nothing observable changes for a caller that never mentions a
profile; the golden fixtures are the byte-level proof.
Two things deliberately do NOT live here. Security is the guard's, always: no
disposition, origin or channel vocabulary belongs on a profile. And the
reserved `verdict` layer is a spec invariant rather than profile config — it is
refused at construction, so a profile admitting it cannot be built, let alone
passed to a door. The `timestamp`/`generated` pair is refused the same way, and
for the same reason: both are things a profile must not be able to express.
Profiles are constructed in code. Config-file loading and inheritance chains
are extension points, not v1 (settled with the operator at phase start).
"""
from __future__ import annotations
import re
from collections.abc import Collection, Mapping, Sequence
from dataclasses import dataclass, field, replace
# The one layer no profile may admit (ingest-spec §3): the promotion gate is
# the only path into it. Compared case-insensitively, as both doors already do.
RESERVED_OKF_TYPE = "verdict"
# The one key PAIR no profile may name (OKF §13.1): `timestamp` is readable as
# a legacy stand-in for `generated.at` only while `generated` is ABSENT, so a
# schema able to name both can describe a document that has neither. Stated as
# key names rather than as a judgement on values because every `generated` a
# schema can express today is a scalar (`_is_legal_value`) and therefore
# malformed as a v0.2 mapping — naming both IS the hazard here. When a value
# model can express a well-formed `generated`, this narrows with it.
# "Naming" spans EVERY field that puts a key in the schema's namespace — `order`,
# `required`, `allowed` and `nullable` alike. A field left out of that union is a
# hole in the gate, not a narrower gate: the namespace is open by default, so a
# key named only by `nullable` is admitted just as surely as an emitted one.
_TIMESTAMP_FALLBACK_PAIR = frozenset({"timestamp", "generated"})
@dataclass(frozen=True)
class TypeRejection:
"""Why a profile refuses an `okf_type`, for the door to frame and raise.
The policy does not raise: Door A refuses with `ManifestError` and Door B
with `MaterializationError`, so the refusal has to be reported rather than
thrown. `reason` completes the sentence "<label>okf_type ..." and `code` is
the stable `IngestError.code` the caller asserts on.
"""
reason: str
code: str
@dataclass(frozen=True)
class TypePolicy:
"""Which `okf_type` values a bundle admits.
`allowed` is `None` for an open set — Phase 1/2 accept any type the
operator names — or a closed enum. Either way the reserved layer is
excluded, and a closed set that names it fails at construction.
"""
allowed: frozenset[str] | None = None
def __post_init__(self) -> None:
if self.allowed is None:
return
named = sorted(value for value in self.allowed if value.lower() == RESERVED_OKF_TYPE)
if named:
raise ValueError(
f"a profile must not admit the reserved {RESERVED_OKF_TYPE!r} layer "
f"(got {', '.join(repr(value) for value in named)}) — the promotion "
"gate is the only path into it (ingest-spec §3)"
)
def rejection(self, okf_type: str) -> TypeRejection | None:
"""The refusal for `okf_type`, or `None` when the profile admits it."""
if okf_type.lower() == RESERVED_OKF_TYPE:
return TypeRejection(
reason=f"must not be {RESERVED_OKF_TYPE!r} (reserved layer)",
code="okf_type_reserved",
)
if self.allowed is not None and okf_type not in self.allowed:
return TypeRejection(
reason=f"must be one of {', '.join(sorted(self.allowed))}, got {okf_type!r}",
code="okf_type_not_allowed",
)
return None
@dataclass(frozen=True)
class SchemaViolation:
"""One way a document's frontmatter departs from the schema.
Reported rather than raised, like `TypeRejection`: a document can depart in
several ways at once, and the caller decides whether that is a refusal, a
report line, or a migration to-do. `code` is the stable machine-readable
part; `reason` completes the sentence "<key> ...".
"""
key: str
reason: str
code: str
def _is_legal_value(value: object) -> bool:
"""A string scalar, or a non-empty list of strings — nothing else.
The list shape is real, not hypothetical: the proving consumer's `Release`
pages carry block lists. A schema that assumed scalars-only would reject
pages they ship today.
"""
if isinstance(value, str):
return True
if isinstance(value, list):
return bool(value) and all(isinstance(item, str) for item in value)
return False
@dataclass(frozen=True)
class FrontmatterSchema:
"""The frontmatter key namespace, what it admits, and how it is emitted.
`order` is the canonical emission order and `collapsed_keys` names the keys
whose whitespace runs collapse to single spaces on the way out. The DEFAULT
order spans both doors' key sets: Door A emits seven of these keys and Door
B six, and each door's subset comes out in exactly the order it wrote by
hand in Phases 1 and 2.
The judging fields — `required`, `allowed`, `nullable`, `key_pattern` — are
read by `violations`. `allowed` is `None` for an open namespace, which is
what DEFAULT keeps: Door A emits seven keys, Door B six, and Door C writes
an imported concept verbatim with whatever the sender wrote, so a closed
namespace or a required set on DEFAULT would declare invalid the very
bundles this library produces.
`required_by_type` is the one judging field that keys off a frontmatter
*value* rather than a key: `{<type value>: {<keys that type must carry>}}`.
OKF v0.2 §10.2 introduces exactly one such rule — `runtime` is REQUIRED for
`Attested Computation` and for no other type — and it cannot be expressed
through `required`, which would demand the key of every document. A type
the mapping does not name carries no extra requirement, which is what keeps
the field inside §14: a consumer must not reject on an unknown `type`, so a
conditional keyed on a type we do not know stays silent rather than guesses.
"""
order: tuple[str, ...]
collapsed_keys: frozenset[str] = field(default_factory=frozenset)
required: frozenset[str] = field(default_factory=frozenset)
allowed: frozenset[str] | None = None
nullable: frozenset[str] = field(default_factory=frozenset)
key_pattern: re.Pattern[str] | None = None
required_by_type: Mapping[str, frozenset[str]] = field(default_factory=dict)
def __post_init__(self) -> None:
conditional: frozenset[str] = frozenset().union(*self.required_by_type.values())
named = set(self.order) | set(self.required) | set(self.nullable) | conditional
if self.allowed is not None:
named |= set(self.allowed)
if _TIMESTAMP_FALLBACK_PAIR <= named:
raise ValueError(
"a profile must not name both 'timestamp' and 'generated' (OKF "
"§13.1 grants the timestamp fallback only while `generated` is "
"ABSENT, so a schema naming both can describe a document with "
"neither a valid `generated.at` nor an eligible fallback)"
)
if self.allowed is None:
return
for label, keys in (
("required", self.required),
("nullable", self.nullable),
("type-conditional required", conditional),
):
stray = sorted(keys - self.allowed)
if stray:
raise ValueError(
f"{label} keys must be inside the allowlist "
f"(got {', '.join(repr(key) for key in stray)}) — a schema that "
"demands a key it also forbids can never be satisfied"
)
def violations(self, values: Mapping[str, object]) -> tuple[SchemaViolation, ...]:
"""Every way `values` departs from this schema, deterministically ordered.
Sorted by `(key, code)` rather than reported in mapping order: two runs
over the same document must produce the same report, and mapping order
is an accident of how the document was parsed.
Takes an already-parsed mapping. Parsing strict frontmatter is a
separate concern — this library's line-oriented parser cannot represent
the block lists and nulls a strict schema admits, which is why Door C
writes imported concepts verbatim rather than round-tripping them.
"""
found: list[SchemaViolation] = []
for key in self.required - set(values):
found.append(SchemaViolation(key, "is required and absent", "frontmatter_key_missing"))
# The type-conditional rules (§10.2 today). Keyed off the VALUE, so it
# fires only on a type this schema names: an absent, non-scalar or
# unknown `type` carries no extra requirement. A separate code from the
# unconditional case because the two are different claims — one says the
# document is malformed, the other that it is malformed *for what it
# says it is* — and a caller may well treat them differently.
declared = values.get("type")
if isinstance(declared, str):
for key in self.required_by_type.get(declared, frozenset()) - set(values):
found.append(
SchemaViolation(
key,
f"is required for type {declared!r} and absent",
"frontmatter_key_missing_for_type",
)
)
for key, value in values.items():
if self.key_pattern is not None and not self.key_pattern.fullmatch(key):
found.append(
SchemaViolation(
key,
f"is not a legal key name (must match {self.key_pattern.pattern})",
"frontmatter_key_malformed",
)
)
continue
if self.allowed is not None and key not in self.allowed:
found.append(
SchemaViolation(
key, "is not on the key allowlist", "frontmatter_key_not_allowed"
)
)
continue
if value is None:
if key not in self.nullable:
found.append(SchemaViolation(key, "must not be null", "frontmatter_value_null"))
continue
if not _is_legal_value(value):
found.append(
SchemaViolation(
key,
"must be a string or a non-empty list of strings",
"frontmatter_value_shape",
)
)
return tuple(sorted(found, key=lambda violation: (violation.key, violation.code)))
def emit(self, values: Mapping[str, str]) -> str:
"""Render `values` as line-oriented `key: value`, one line per key.
Commons decision D1: the keys named in `order` come first, in that
order, followed by any remaining keys SORTED. Ordering the tail rather
than trusting insertion order is what makes a regeneration over the
same data byte-identical.
Returns the lines only — the caller owns the `---` fences.
"""
named = [key for key in self.order if key in values]
tail = sorted(key for key in values if key not in self.order)
return "\n".join(f"{key}: {self._render(key, values[key])}" for key in [*named, *tail])
def _render(self, key: str, value: str) -> str:
# §5 mandates whitespace-run collapse for `source_query` only
# (ingest-spec.md:140-141), where a legitimately multi-line SQL SELECT
# must render on one line. Every other value is validated single-line
# at load and emitted verbatim — validation, not repair — so
# operator-supplied bytes (e.g. a title's internal double space)
# survive.
return " ".join(value.split()) if key in self.collapsed_keys else value
# The v0.1 ingest stamp. A literal rather than a configurable value: it is what
# every bundle this library has already written carries, and recognising it is
# what keeps those bundles re-runnable under a later profile.
_V0_1_STAMP = "true"
@dataclass(frozen=True)
class OwnershipPolicy:
"""The `generated` value this profile writes, and the values it owns back.
Ownership is the §3 collision gate's question — may this run replace the
file already sitting at a generated name? — and the answer is a profile's,
because the stamp differs per profile. The literal `true` is the older
form; the O2 form is `generated: { by: <actor>, at: <ingested_at> }` (§5),
where the actor takes §7's `process:<id>` form.
Which form a profile writes is NOT read off an upstream version. `DEFAULT`
states commons' ingest-spec layer and writes the O2 form because commons
ratified it (V1, `54e0ec7`), while remaining v0.1 on every axis upstream
owns; `STRICT_V1` names no `generated` at all. The stamp tracks whichever
contract the profile states, and those contracts move independently.
`actor` is `None` for the literal stamp. Where it is set it carries no version,
deliberately: the value sits inside a byte-compared golden, so a producer
version there would fire golden regression on every release without any
contract having changed, and would make a shared cross-implementation
fixture impossible by construction (plan V1(d), operator 2026-07-27).
Recognition is ONE-WAY, and both directions are decisions rather than
accidents. A profile with an actor owns the literal stamp as well, so a
bundle written before V1 re-runs IN PLACE — the black-box promise is that a
spec release costs a consumer a re-run and nothing more. The reverse is
refused: a profile without an actor fails the run rather than replacing a
file whose shape it does not read (V-A3).
"Owns the O2 form" is never "owns any mapping". The prefix binds the
profile's OWN actor, so a mapping naming a different one — another
implementation's, or §7's `human:` actor on curated content — stays
unowned. That is what keeps the key's mere presence from proving authorship,
which upstream v0.2 makes load-bearing by writing `generated` on
hand-authored files too.
The actor test is a PREFIX rather than an equality, because the value carries
`ingested_at` and therefore differs on every run by design. It works because
`parse_frontmatter` returns the whole flow mapping as one opaque string
(V-A2) — no structure this library cannot yet read is parsed here.
"""
actor: str | None = None
def stamp(self, ingested_at: str) -> str:
"""The `generated` value a run at `ingested_at` writes."""
if self.actor is None:
return _V0_1_STAMP
return f"{{ by: {self.actor}, at: {ingested_at} }}"
def owns(self, value: str) -> bool:
"""Whether a `generated` value read back marks this library's output."""
if value == _V0_1_STAMP:
return True
if self.actor is None:
return False
return value.startswith(f"{{ by: {self.actor},")
@dataclass(frozen=True)
class PathPolicy:
"""The filename namespaces the three doors write into.
Each prefix keeps its door's generated names disjoint from `index.md`, from
the other doors, and from `promoted-verdict-*`, for every id the grammar
admits. The id grammar itself stays in `materialize`: it is a pattern the
slugger derives a separator class from, not a name to configure.
"""
concept_suffix: str
ingest_prefix: str
inbox_prefix: str
import_prefix: str
def _split_frontmatter(text: str) -> tuple[dict[str, str], list[str]]:
"""The leading `---` block as ordered keys, and the body lines after it.
Line-oriented, the same shape `materialize.parse_frontmatter` reads —
duplicated rather than imported because `materialize` imports this module,
and because that one takes a path where an index reader has only text.
"""
lines = text.splitlines()
if not lines or lines[0].strip() != "---":
return {}, lines
head: dict[str, str] = {}
for offset, line in enumerate(lines[1:], start=1):
if line.strip() == "---":
return head, lines[offset + 1 :]
key, sep, value = line.partition(":")
if sep:
head[key.strip()] = value.strip()
return head, []
@dataclass(frozen=True)
class IndexViolation:
"""One way an index file departs from the policy.
Reported rather than raised, like `SchemaViolation`. `subject` is whatever
the finding is about — a line, a link target, or a frontmatter key — and
`code` is the stable machine-readable part.
"""
subject: str
reason: str
code: str
@dataclass(frozen=True)
class IndexPolicy:
"""The index file, the shape of its entries, and where indexes must exist.
`link_template` renders a managed line and `link_pattern` recognises one.
Both are carried because §6 does both — append on write, rewrite on
maintenance — and a round-trip test is what keeps the pair honest. The
pattern is anchored to the whole line by construction: removal keys on this
exact shape, never a bare substring, so curated prose that mentions a
target inline survives verbatim.
The judging fields are read by `violations` and `required_indexes`, and
every one of them is off under DEFAULT. They exist because two consumers
disagree about this file in ways no single shape can hold:
- `per_directory` — an index at every level, or only at the bundle root.
Upstream OKF binds `index.md` to the root ALONE, so a default demanding
one per level would declare upstream-conforming bundles invalid. Policy,
never an OKF rule.
- `heading_required` / `allows_prose` — the wiki's index is a generated
heading plus entries and nothing else; the catalog's *requires*
progressive-disclosure prose. Opposite requirements, both expressible.
- `entries_match_directory` — whether the index must match the directory
exactly, in both directions. The wiki enforces this at ERROR
(`BUNDLE_INDEX_COMPLETE`); the catalog holds that an index is an
AUTHORED count of a directory's children and that a validator
enumerating the directory has implemented the wrong contract. Neither
posture is baked in, and this library never enumerates anything: the
caller supplies the listing, and a policy that does not judge it refuses
to be handed one at all — so code written to the wrong contract fails at
the call rather than passing every test one would think to write.
- `root_frontmatter` — the ordered keys the ROOT index may carry, where
nested indexes carry none. Confirmed independently in both consumers, so
the asymmetry is shape rather than one repo's preference. Naming a key
here PERMITS it and fixes its position; it does not demand it.
- `root_frontmatter_required` — the subset that must actually be present.
Separate from the tuple above because the two are different claims, and
collapsing them was P1-F1: `OKF_V0_2` names `okf_version` to fix its
position, but upstream §8/§12 grant it as a MAY, and judging the pair as
one field failed 14 of 17 real bundles — upstream's own four included —
each with exactly this one violation, while our emitter treated the same
key as optional. A profile whose consumer really does demand the keys
says so here, which is what `STRICT_V1` does.
"""
name: str
link_template: str
link_pattern: re.Pattern[str]
per_directory: bool = False
heading_required: bool = False
allows_prose: bool = True
entries_match_directory: bool = False
root_frontmatter: tuple[str, ...] = ()
root_frontmatter_required: frozenset[str] = field(default_factory=frozenset)
def __post_init__(self) -> None:
stray = sorted(self.root_frontmatter_required - set(self.root_frontmatter))
if stray:
raise ValueError(
f"required root keys must be named in the ordered set "
f"(got {', '.join(repr(key) for key in stray)}) — a key demanded "
"but never named could not be judged for position"
)
if self.requires_description and "description" not in self.link_pattern.groupindex:
raise ValueError(
"link_template names {description} but link_pattern has no "
"'description' group — the pair would round-trip in one "
"direction only"
)
@property
def requires_description(self) -> bool:
"""Whether an entry carries a description alongside label and target."""
return "{description}" in self.link_template
def render_link(self, label: str, target: str, description: str | None = None) -> str:
if self.requires_description and description is None:
raise ValueError(
"this index policy's entries carry a description; rendering "
"without one emits a half-written entry that parses as nothing"
)
if not self.requires_description and description is not None:
raise ValueError(
"this index policy's entries carry no description; the value "
"offered would be dropped silently"
)
return self.link_template.format(label=label, target=target, description=description)
def required_indexes(self, directories: Sequence[str]) -> tuple[str, ...]:
"""The index paths this policy requires, given the caller's directories.
Bundle-relative, with `""` for the root. The directories are an
argument precisely so that nothing here reaches the filesystem.
"""
if not self.per_directory:
return (self.name,)
return tuple(sorted(f"{d}/{self.name}" if d else self.name for d in directories))
def violations(
self,
text: str,
*,
is_root: bool = False,
expected_targets: Collection[str] | None = None,
) -> tuple[IndexViolation, ...]:
"""Every way `text` departs from this policy, deterministically ordered.
`expected_targets` is the directory's contents as the CALLER sees them,
required exactly when `entries_match_directory` is set and refused
otherwise. Sorted by `(subject, code)` so two runs over the same file
agree.
"""
if self.entries_match_directory and expected_targets is None:
raise ValueError(
"this index policy judges the index against the directory, so "
"a listing is required — skipping the check silently would "
"pass a gate whose input never arrived"
)
if not self.entries_match_directory and expected_targets is not None:
raise ValueError(
"this index policy's index is authored rather than derived "
"from the directory, so a listing has nothing to judge"
)
found: list[IndexViolation] = []
head, body = _split_frontmatter(text)
found.extend(self._frontmatter_violations(head, is_root=is_root))
headings: list[str] = []
listed: set[str] = set()
for line in body:
stripped = line.strip()
if not stripped:
continue
if line.startswith("# "):
headings.append(line)
continue
match = self.link_pattern.match(line)
if match is not None:
listed.add(match.group("target"))
continue
if not self.allows_prose:
found.append(
IndexViolation(
stripped,
"is neither the heading nor an entry",
"index_prose_not_allowed",
)
)
if self.heading_required:
if not headings:
found.append(
IndexViolation("", "the index carries no `# ` heading", "index_heading_missing")
)
found.extend(
IndexViolation(extra, "is a second `# ` heading", "index_heading_extra")
for extra in headings[1:]
)
if expected_targets is not None:
expected = set(expected_targets)
found.extend(
IndexViolation(
target, "is in the directory but not in the index", "index_entry_missing"
)
for target in expected - listed
)
found.extend(
IndexViolation(
target, "is in the index but not in the directory", "index_entry_unexpected"
)
for target in listed - expected
)
return tuple(sorted(found, key=lambda violation: (violation.subject, violation.code)))
def _frontmatter_violations(
self, head: Mapping[str, str], *, is_root: bool
) -> list[IndexViolation]:
if not self.root_frontmatter:
return []
if not is_root:
# One finding about the block, not one per key: a nested index
# carrying frontmatter is a single structural fact.
first = next(iter(head), None)
if first is None:
return []
return [
IndexViolation(
first,
"is frontmatter on a nested index, which carries none",
"index_root_frontmatter_unexpected",
)
]
found = [
IndexViolation(key, "is pinned on the root index and absent", "index_root_key_missing")
for key in self.root_frontmatter
if key in self.root_frontmatter_required and key not in head
]
present = [key for key in head if key in self.root_frontmatter]
declared = [key for key in self.root_frontmatter if key in head]
if present != declared:
found.append(
IndexViolation(
present[0],
f"breaks the pinned key order {', '.join(self.root_frontmatter)}",
"index_root_key_order",
)
)
return found
@dataclass(frozen=True)
class BundleProfile:
"""One bundle contract: types, frontmatter, filenames, index."""
types: TypePolicy
frontmatter: FrontmatterSchema
paths: PathPolicy
index: IndexPolicy
ownership: OwnershipPolicy = field(default_factory=OwnershipPolicy)
# The ingest-spec + Phase 2 contract. Every value here was a constant in
# `manifest`, `materialize`, `inbox` or `importer` before this module existed;
# the golden suite is what proves that move changed no bytes.
#
# This profile tracks COMMONS' spec, not an upstream Google version — the two
# axes are independent, and `ownership` is where they visibly part company. Its
# `generated` is the O2 mapping because commons ratified that shape for their
# §5 layer, while the profile remains v0.1 on every axis upstream owns.
DEFAULT = BundleProfile(
types=TypePolicy(allowed=None),
frontmatter=FrontmatterSchema(
# Door A's seven keys and Door B's six, merged into one order that
# contains both as subsequences — neither door's output moves.
order=(
"type",
"title",
"source_system",
"source_query",
"source_file",
"source_sha256",
"ingested_at",
"ingest_manifest",
"generated",
),
collapsed_keys=frozenset({"source_query"}),
),
paths=PathPolicy(
concept_suffix=".md",
ingest_prefix="ingest-",
inbox_prefix="inbox-",
import_prefix="import-",
),
index=IndexPolicy(
name="index.md",
link_template="- [{label}]({target})",
link_pattern=re.compile(r"^- \[(?P<label>[^\]]*)\]\((?P<target>[^)]+)\)$"),
),
# V1, ratified 2026-08-02 and executed by commons 2026-08-09 (`54e0ec7`).
# The actor is the same constant `OKF_V0_2` carries, and that is commons'
# doing rather than a merge of the two profiles: ingest-spec §7 names
# `process:okf-ingest` as THE ingest actor, so any profile stating that
# spec's layer writes it. The profiles still differ everywhere else.
ownership=OwnershipPolicy(actor="process:okf-ingest"),
)
# `FRONTMATTER_KEY_ORDER` in the proving consumer's `bundle.py`: an eleven-key
# allowlist that doubles as canonical emission order. The first eight are the
# required set their operator ratified on 2026-07-25 (measured present on
# 522/522 documents); the last three are layer-specific and stay optional.
# Deriving both sets from this one tuple is what keeps them from drifting apart
# — their BUNDLE_HASH_REGISTRY gate depends on the order.
_STRICT_V1_KEY_ORDER = (
"type",
"title",
"description",
"timestamp",
"layer",
"source_tier",
"source_url",
"source_sha",
"version",
"date",
"summary",
)
# The `claude-code-llm-wiki` contract, from `docs/phase-3-split-table.md`.
#
# Deliberately stricter than the consumer's own validator: their REQUIRED_KEYS
# constant is the first four keys, which is their emit-path minimum rather than
# their contract. If they ever ship a document class that legitimately lacks
# `source_sha`, the agreed resolution is to bump this profile — they send the
# requirement before shipping the class, not after.
#
# `paths` is DEFAULT's and that is the whole truth for it: the filename
# namespaces are an ingest-spec invariant about what the doors write, not a
# consumer preference.
#
# The index shape is read out of their code rather than recalled:
# `bundle.py:498-524` renders `* [Title](target) - description` with
# subdirectories linking to their own index; `bundle.py:527-567` walks every
# level, writing frontmatter at the bundle root only and a bare `# ` heading
# below it; `validate.py:1081-1120` (gate BUNDLE_INDEX_COMPLETE, ERROR) demands
# the index and the directory match exactly in both directions.
STRICT_V1 = BundleProfile(
types=TypePolicy(allowed=frozenset({"Concept", "Guide", "Reference", "Release"})),
frontmatter=FrontmatterSchema(
order=_STRICT_V1_KEY_ORDER,
required=frozenset(_STRICT_V1_KEY_ORDER[:8]),
allowed=frozenset(_STRICT_V1_KEY_ORDER),
nullable=frozenset({"summary"}),
key_pattern=re.compile(r"^[a-z_]+$"),
),
paths=DEFAULT.paths,
index=IndexPolicy(
name="index.md",
link_template="* [{label}]({target}) - {description}",
link_pattern=re.compile(
r"^\* \[(?P<label>[^\]]*)\]\((?P<target>[^)\s]+)\) - (?P<description>.+)$"
),
per_directory=True,
heading_required=True,
allows_prose=False,
entries_match_directory=True,
root_frontmatter=("okf_version", "bundle_profile", "okf_spec_commit"),
# The proving consumer demands all three, not merely permits them: their
# root index carries exactly these keys in exactly this order on every
# bundle measured (`c5141f8`). Stated explicitly so that separating
# "permitted" from "required" costs them nothing.
root_frontmatter_required=frozenset({"okf_version", "bundle_profile", "okf_spec_commit"}),
),
)
# OKF v0.2, as an ADDITIVE profile: `DEFAULT` states commons' ingest-spec §5
# layer and keeps stating it, so nothing here migrates anything. The key order
# is DEFAULT's followed by the §5 families v0.2 adds, which is also the order
# upstream's own reference bundles emit them in (`generated` before `sources`).
#
# Naming a family is not writing it. `verified`, `status` and `stale_after` are
# expressible so a caller can emit them in canonical order and so the schema can
# judge a document that carries them; Door A writes none of them, because a
# field with no reader is not written. Their structured v0.2 values — block
# lists of mappings — are beyond this library's value model until D1's reader
# lands; that is a reader gap, not an ordering one.
# §10 `Attested Computation`: the type's contract fields, in §10.2's own listing
# order. FORMAT only — this library supports writing, judging and round-tripping
# the contract, and implements no execution: upstream defers the receipt and
# verdict wire formats, so there is nothing to build a runtime against, and
# "did this run produce the value the sanctioned way" sits on the far side of
# this library's boundary in any case.
ATTESTED_COMPUTATION_TYPE = "Attested Computation"
_ATTESTED_COMPUTATION_FIELDS = ("runtime", "parameters", "computation", "executor", "attester")
_OKF_V0_2_KEY_ORDER = (
*DEFAULT.frontmatter.order,
"sources",
"verified",
"status",
"stale_after",
# §10.2's contract fields, appended as one block and internally in the order
# that section enumerates them. Appended rather than interleaved because the
# keys before them are what the doors actually emit, and because `emit`
# writes only the keys present — so naming these costs no byte in any bundle
# that carries none of them. Without the block they would still be emitted,
# in `emit`'s sorted tail, where `attester` precedes `runtime`: alphabetical
# order standing in for the contract's own.
*_ATTESTED_COMPUTATION_FIELDS,
)
# PROVISIONAL. Shipped first as a pre-release (`v0.5.0a1`) to a named pilot set
# — `portfolio-optimiser-claude`, the plugin marketplace catalog, and
# `claude-code-llm-wiki` — and this surface may change on their feedback without
# a deprecation cycle. Saying so is what buys the freedom to act on the
# feedback; discovering it later is what would make the pilot a de-facto
# release. The versioned constants are the stable binding.
#
# Two things this profile deliberately does NOT do:
#
# - **It closes nothing.** §14 forbids a conformant consumer to reject on an
# unknown `type` value or on unknown additional keys, so an allowlist or a key
# pattern here would put the profile in violation of the version it is named
# for. `type` is required and is the only one (§4, §11).
# - **It NAMES `okf_version` but never carries its value.** The value tracks the
# upstream Google version and belongs to catalog (decision E1), so a constant
# here would be this repo claiming a decision it does not own — and the one
# that would have to be chased on every upstream release. The caller supplies
# it (`materialize_bundle(..., root_frontmatter_values=...)`); this policy
# fixes only the key and its position.
#
# WHERE it goes was open until 2026-07-31 between upstream's root-index
# frontmatter block and catalog's body-line convention. Catalog verified
# upstream themselves at the pinned commit `3fcbb9f` and reported §8:509-510
# ("Index files contain no frontmatter, with one exception: a bundle-root
# `index.md` MAY carry an `okf_version` key") and §12:773-775 ("in a
# bundle-root `index.md` frontmatter block (the only place frontmatter is
# permitted in an `index.md`)"). Frontmatter it is; their own spec diverges
# from upstream here, and that divergence is theirs to resolve.
#
# Declaring it stays a MAY: none of upstream's four reference bundles carries
# the key at all (catalog grepped `okf/bundles` and `okf/samples`: zero hits),
# so omitting `root_frontmatter_values` emits no block. The JUDGE says the same
# thing since P1-F1 — the key is named here and left out of
# `root_frontmatter_required` — because for one release it did not, and a
# bundle exercising the MAY was reported as violating by the very profile that
# had emitted it correctly.
#
# **Measured limitation (guard 0.2.0, 2026-07-26):** a bundle emitted under this
# profile cannot be read back through a guard-gated import. The guard's T2
# frontmatter grammar admits scalars and flat lists of strings, and refuses every
# route to a mapping — flow on the disallowed-indicator set, block on the
# nested-mapping check, dotted keys on the key pattern. So `generated` as the
# mapping v0.2 specifies has no expressible form through that gate at all. This
# binds what can be IMPORTED (Door C), never what we emit: Door B's
# `screen_output` does not run that parser.
OKF_V0_2 = BundleProfile(
types=TypePolicy(allowed=None),
frontmatter=FrontmatterSchema(
order=_OKF_V0_2_KEY_ORDER,
collapsed_keys=DEFAULT.frontmatter.collapsed_keys,
required=frozenset({"type"}),
# §10.2's one type-conditional rule, and the whole of it: `runtime` is
# REQUIRED for this type because it is what says how to run the
# computation and therefore what `parameters` mean. The other four
# contract fields stay optional — `computation` absent means the body
# fence IS the computation (§10.3), which is a valid concept.
required_by_type={ATTESTED_COMPUTATION_TYPE: frozenset({"runtime"})},
),
paths=DEFAULT.paths,
# DEFAULT's index in every respect but one: the root MAY carry `okf_version`
# (§8, §12). Built with `replace` rather than restated so a later change to
# the shared shape cannot drift between the two.
index=replace(DEFAULT.index, root_frontmatter=("okf_version",)),
# Byte-identical to `DEFAULT.ownership` since V1, and deliberately NOT
# written as a reference to it. The two agree by coincidence of commons
# adopting §7's actor, not by dependency: this profile states UPSTREAM's
# v0.2, so if commons ever moves their actor again, this one must not
# follow. Restating it is what keeps that independence expressible.
ownership=OwnershipPolicy(actor="process:okf-ingest"),
)
# "The latest version supported as STABLE", not the latest present in this
# module. It therefore keeps v0.1 UPSTREAM semantics for as long as v0.2 is
# provisional, and flipping it is the GA event — one auditable action rather
# than a side effect of a merge.
#
# "v0.1 semantics" is about upstream and has never covered commons' layer. V1
# moved `DEFAULT`'s stamp, so this alias's bytes moved with it, before GA and
# without the flip. That is not a leak in the alias: the two contracts are
# independent axes, and a consumer bound here tracks both by construction.
#
# The tradeoff is stated rather than hidden: an alias that moves means a consumer
# bound to it inherits upstream's breaking changes on a library upgrade. The
# versioned constants are the stable binding and are what a consumer should pin;
# this is for callers who have explicitly opted into tracking.
OKF_LATEST = DEFAULT