open/llm-ingestion-okf
1
0
Fork 0
Code Issues Pull requests Releases Activity

feat(okf-v0.2): D2 — the profile, the ownership policy, and sources

Browse source 
The v0.2 profile lands additively: DEFAULT keeps stating commons' §5 layer
byte for byte (the golden suite is the proof, not the claim), and OKF_V0_2
adds the families v0.2 introduces on top of DEFAULT's key order.

Two questions the plan left open were the operator's, and both were decided
before code rather than discovered during it:

- `generated.by` is `process:llm-ingestion-okf` — plan V1's option (d), the
  §7 process actor form. The value carries no version, which is what keeps a
  byte-compared fixture stable across releases and leaves a shared
  cross-implementation fixture possible. A-E3 was written against the
  `<producer>/<version>` form and is now STALE in the pilot specification; the
  correction is owed to portfolio-optimiser-claude before they run.
- `sources[].resource` is the manifest source's locator verbatim: the file
  root, the sql connection_ref (an env-var NAME, never its value), or the http
  base_url. `credential_ref` is not a locator and is never emitted.

Ownership becomes a policy on the profile rather than a literal in the gate.
The emitter and `_is_ingest_owned` are coupled through the stamp value, so
OwnershipPolicy is where they meet and can only change together. Recognition
is one-way by decision: OKF_V0_2 owns the v0.1 stamp too, so a DEFAULT-written
bundle re-runs IN PLACE (operator, 2026-07-27), while DEFAULT still refuses a
v0.2 file rather than replacing it — V-A3's fail-safe is preserved.

An unquotable locator is refused rather than emitted. Measured with PyYAML
rather than reasoned: `[{ id: a, resource: data, backup }]` raises nothing and
parses to a mapping with a `backup` key nobody wrote, so the failure mode is a
silently wrong provenance record. Validation, not repair.

Deliberately NOT here: the `okf_version` declaration. §12 makes it a MAY and
none of upstream's four reference bundles exercises it; WHERE it goes is open
between upstream's root-index frontmatter and catalog's body-line convention,
and catalog verifies against upstream first. It is declared once, at D5.

V-A5 is extended to the new profile — the one whose NAME is the place a
version literal would look natural, and it carries none.

542 tests, mypy --strict clean.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-27 09:11:12 +02:00
commit c90171dad0
5 changed files with 562 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

4
src/llm_ingestion_okf/errors.py
View file

@ -96,6 +96,10 @@ class MaterializationError(IngestError):
- `ingested_at_invalid` — ingested_at is not ISO-8601 UTC with a Z suffix
- `collision_unstamped` — the §3 collision gate: a generated name is
occupied by a file without the ingest stamp
- `source_reference_unquotable` — a manifest source's id or locator
contains a character that would restructure the `sources` flow mapping
(Door A, v0.2 profiles); refused rather than emitted, because the
resulting document parses cleanly into a record no one wrote
- `inbox_slug_empty` — a dropped file's name reduces to an empty slug
under the id grammar (Door B; never an invented fallback name)
- `inbox_slug_too_long` — the generated inbox filename would exceed the

93
src/llm_ingestion_okf/materialize.py
View file

@ -23,11 +23,12 @@ from .manifest import (
FileSource,
HttpSource,
Manifest,
Source,
SqlSource,
generated_filename,
load_manifest_bytes,
)
from .profiles import DEFAULT
from .profiles import DEFAULT, BundleProfile
from .render import render_fenced_block, render_table
_LOGGER = logging.getLogger(__name__)
@ -128,10 +129,18 @@ def parse_frontmatter(path: Path) -> dict[str, str]:
return frontmatter
def _is_ingest_owned(path: Path, manifest_stem: str) -> bool:
# §3/§5 ownership: the ingest stamp is `generated: true` AND an
# `ingest_manifest` reference. Promoted verdict files carry neither key,
# so they can never classify as ingest-owned.
def _is_ingest_owned(path: Path, manifest_stem: str, *, profile: BundleProfile = DEFAULT) -> bool:
# §3/§5 ownership: the profile's ingest stamp AND an `ingest_manifest`
# reference. Promoted verdict files carry neither key, so they can never
# classify as ingest-owned.
#
# The stamp is the PROFILE's because it differs per profile (v0.1 writes
# `generated: true`, v0.2 a `{ by: ..., at: ... }` mapping) — and because the
# emitter and this predicate are coupled through that value. Changing the
# emitted form without the predicate is what makes the library stop
# recognising its own output, firing the collision gate on the files its own
# previous run wrote. `OwnershipPolicy` is where the two meet, so they can
# only be changed together.
#
# §10.2 per-manifest ownership: a file is THIS manifest's to replace only
# when the reference names it by stem. The stamp is `{stem}@{sha256[:16]}`;
@ -141,7 +150,8 @@ def _is_ingest_owned(path: Path, manifest_stem: str) -> bool:
# keeps its own. rsplit strips the trailing `@{sha}`, so a stem that itself
# contains `@` still compares correctly.
frontmatter = parse_frontmatter(path)
if frontmatter.get("generated") != "true":
generated = frontmatter.get("generated")
if generated is None or not profile.ownership.owns(generated):
return False
reference = frontmatter.get("ingest_manifest")
if reference is None:
@ -149,8 +159,67 @@ def _is_ingest_owned(path: Path, manifest_stem: str) -> bool:
return reference.rsplit("@", 1)[0] == manifest_stem
# The characters that terminate or restructure a YAML flow mapping. `:\s`
# catches a colon that would open a nested key; a colon inside `https://host`
# does not, and stays a plain scalar.
_FLOW_UNSAFE_RE = re.compile(r"[,\[\]{}]|:\s")
def _source_locator(source: Source) -> str:
"""Where a manifest source points, per source type.
A filesystem root, the NAME of the environment variable holding the DSN, or
the base URL. `credential_ref` is not a locator and is never returned here:
a credential reference has no reader in a bundle.
"""
if isinstance(source, FileSource):
return source.root
if isinstance(source, SqlSource):
return source.connection_ref
return source.base_url
def _render_sources(source: Source) -> str:
"""§5 `sources` as an inline flow sequence of one flow mapping.
Two keys, not upstream's five: a manifest source has no `author`, no
`last_modified`, and no bundle-internal `resource` in upstream's sense, and
inventing them would be writing fields with no reader.
The flow form rather than upstream's block list, measured and chosen: a
block list read back through this library's line-oriented parser turns each
item line into a KEY nobody wrote — and `_is_ingest_owned` reads through
that same parser. The flow form also satisfies commons' §5 "all values MUST
be single-line", and §11 requires parseable YAML rather than block YAML.
Refusing an unquotable locator is the point of the check rather than a
nicety: `[{ id: x, resource: data, backup }]` is not a parse ERROR, it is a
mapping with a `backup` key nobody wrote. A silently wrong provenance record
is worse than a refused run, and repairing the value by quoting it would
change bytes the operator supplied. Validation, not repair — the same
posture as the filename-length gate.
"""
locator = _source_locator(source)
for label, value in (("id", source.id), ("resource", locator)):
if _FLOW_UNSAFE_RE.search(value):
raise MaterializationError(
f"the source {label} {value!r} contains a character that would "
"restructure the `sources` flow mapping (one of `,[]{}` or a "
"colon followed by whitespace) — refusing to emit a provenance "
"record that parses cleanly into something no one wrote",
code="source_reference_unquotable",
)
return f"[{{ id: {source.id}, resource: {locator} }}]"
def _render_concept_file(
manifest: Manifest, extraction: Extraction, body: str, *, ingested_at: str, stamp: str
manifest: Manifest,
extraction: Extraction,
body: str,
*,
ingested_at: str,
stamp: str,
profile: BundleProfile = DEFAULT,
) -> str:
# §5 frontmatter: exactly these keys. The order and the `source_query`
# whitespace collapse are the profile's — DEFAULT states the §5 layer.
@ -161,9 +230,15 @@ def _render_concept_file(
"source_query": extraction.query,
"ingested_at": ingested_at,
"ingest_manifest": stamp,
"generated": "true",
"generated": profile.ownership.stamp(ingested_at),
}
return f"---\n{DEFAULT.frontmatter.emit(frontmatter)}\n---\n\n{body}"
# Written only by a profile that NAMES it. `emit` sorts an unnamed key into
# the tail rather than dropping it, so building one mapping for both
# profiles would append `sources` to every v0.1 bundle — additivity is a
# property of what is constructed here, not of the emitter.
if "sources" in profile.frontmatter.order:
frontmatter["sources"] = _render_sources(manifest.source)
return f"---\n{profile.frontmatter.emit(frontmatter)}\n---\n\n{body}"
def write_bytes(bundle_dir: Path, name: str, content: str) -> Path:

126
src/llm_ingestion_okf/profiles.py
View file

@ -242,6 +242,58 @@ class FrontmatterSchema:
return " ".join(value.split()) if key in self.collapsed_keys else value
# The v0.1 ingest stamp. A literal rather than a configurable value: it is what
# every bundle this library has already written carries, and recognising it is
# what keeps those bundles re-runnable under a later profile.
_V0_1_STAMP = "true"
@dataclass(frozen=True)
class OwnershipPolicy:
"""The `generated` value this profile writes, and the values it owns back.
Ownership is the §3 collision gate's question — may this run replace the
file already sitting at a generated name? — and the answer is a profile's,
because the stamp differs per profile. v0.1 writes the literal `true`; v0.2
writes `generated: { by: <actor>, at: <ingested_at> }` (§5), where the actor
takes §7's `process:<id>` form.
`actor` is `None` for the v0.1 stamp. Where it is set it carries no version,
deliberately: the value sits inside a byte-compared golden, so a producer
version there would fire golden regression on every release without any
contract having changed, and would make a shared cross-implementation
fixture impossible by construction (plan V1(d), operator 2026-07-27).
Recognition is ONE-WAY, and both directions are decisions rather than
accidents. A v0.2 profile owns the v0.1 stamp as well, so a bundle written
under `DEFAULT` re-runs IN PLACE — the black-box promise is that an upstream
release costs a consumer a re-run and nothing more. The reverse is refused:
`DEFAULT` meeting a v0.2 file fails the run rather than replacing a file
whose shape it does not read (V-A3).
The v0.2 test is a PREFIX rather than an equality, because the value carries
`ingested_at` and therefore differs on every run by design. It works because
`parse_frontmatter` returns the whole flow mapping as one opaque string
(V-A2) — no structure this library cannot yet read is parsed here.
"""
actor: str | None = None
def stamp(self, ingested_at: str) -> str:
"""The `generated` value a run at `ingested_at` writes."""
if self.actor is None:
return _V0_1_STAMP
return f"{{ by: {self.actor}, at: {ingested_at} }}"
def owns(self, value: str) -> bool:
"""Whether a `generated` value read back marks this library's output."""
if value == _V0_1_STAMP:
return True
if self.actor is None:
return False
return value.startswith(f"{{ by: {self.actor},")
@dataclass(frozen=True)
class PathPolicy:
"""The filename namespaces the three doors write into.
@ -497,6 +549,7 @@ class BundleProfile:
frontmatter: FrontmatterSchema
paths: PathPolicy
index: IndexPolicy
ownership: OwnershipPolicy = field(default_factory=OwnershipPolicy)
# The ingest-spec v1 + Phase 2 contract, unchanged. Every value here was a
@ -595,3 +648,76 @@ STRICT_V1 = BundleProfile(
root_frontmatter=("okf_version", "bundle_profile", "okf_spec_commit"),
),
)
# OKF v0.2, as an ADDITIVE profile: `DEFAULT` states commons' ingest-spec §5
# layer and keeps stating it, so nothing here migrates anything. The key order
# is DEFAULT's followed by the §5 families v0.2 adds, which is also the order
# upstream's own reference bundles emit them in (`generated` before `sources`).
#
# Naming a family is not writing it. `verified`, `status` and `stale_after` are
# expressible so a caller can emit them in canonical order and so the schema can
# judge a document that carries them; Door A writes none of them, because a
# field with no reader is not written. Their structured v0.2 values — block
# lists of mappings — are beyond this library's value model until D1's reader
# lands; that is a reader gap, not an ordering one.
_OKF_V0_2_KEY_ORDER = (
*DEFAULT.frontmatter.order,
"sources",
"verified",
"status",
"stale_after",
)
# PROVISIONAL. Shipped first as a pre-release (`v0.5.0a1`) to a named pilot set
# — `portfolio-optimiser-claude`, the plugin marketplace catalog, and
# `claude-code-llm-wiki` — and this surface may change on their feedback without
# a deprecation cycle. Saying so is what buys the freedom to act on the
# feedback; discovering it later is what would make the pilot a de-facto
# release. The versioned constants are the stable binding.
#
# Two things this profile deliberately does NOT do:
#
# - **It closes nothing.** §14 forbids a conformant consumer to reject on an
# unknown `type` value or on unknown additional keys, so an allowlist or a key
# pattern here would put the profile in violation of the version it is named
# for. `type` is required and is the only one (§4, §11).
# - **It does not declare `okf_version`.** The index policy is DEFAULT's, which
# binds `index.md` to the bundle root alone — upstream's shape (§8), and none
# of upstream's four reference bundles declares the version at all (§12 makes
# it a MAY). Declaring it is D5's, once, in the fixture: the value belongs to
# catalog (E1), and WHERE it goes is open between upstream's root-index
# frontmatter block and catalog's body-line convention. A profile that pinned
# one of those today would be pinning the wrong one half the time.
#
# **Measured limitation (guard 0.2.0, 2026-07-26):** a bundle emitted under this
# profile cannot be read back through a guard-gated import. The guard's T2
# frontmatter grammar admits scalars and flat lists of strings, and refuses every
# route to a mapping — flow on the disallowed-indicator set, block on the
# nested-mapping check, dotted keys on the key pattern. So `generated` as the
# mapping v0.2 specifies has no expressible form through that gate at all. This
# binds what can be IMPORTED (Door C), never what we emit: Door B's
# `screen_output` does not run that parser.
OKF_V0_2 = BundleProfile(
types=TypePolicy(allowed=None),
frontmatter=FrontmatterSchema(
order=_OKF_V0_2_KEY_ORDER,
collapsed_keys=DEFAULT.frontmatter.collapsed_keys,
required=frozenset({"type"}),
),
paths=DEFAULT.paths,
index=DEFAULT.index,
ownership=OwnershipPolicy(actor="process:llm-ingestion-okf"),
)
# "The latest version supported as STABLE", not the latest present in this
# module. It therefore keeps v0.1 semantics for as long as v0.2 is provisional,
# and flipping it is the GA event — one auditable action rather than a side
# effect of a merge.
#
# The tradeoff is stated rather than hidden: an alias that moves means a consumer
# bound to it inherits upstream's breaking changes on a library upgrade. The
# versioned constants are the stable binding and are what a consumer should pin;
# this is for callers who have explicitly opted into tracking.
OKF_LATEST = DEFAULT

339
tests/test_okf_v0_2_profile.py Normal file
View file

@ -0,0 +1,339 @@
"""D2 — the `OKF_V0_2` profile, profile-aware ownership, `sources` derivation.
Three things, in the order the risk runs:
1. **The profile constant.** Names the §5 families, requires only `type` (§4:
"the only always-required key"), and closes nothing — §14 forbids a consumer
to reject on unknown types or unknown keys, so an allowlist here would put
the profile in violation of the spec it is named after.
2. **Ownership.** `_is_ingest_owned` is the pre-mutation collision gate, so it
is the one piece of this work whose defects are expensive and quiet. The v0.2
stamp is a flow mapping carrying `ingested_at`, so equality is impossible by
construction and the predicate is a prefix test. The operator decided the
direction: `OKF_V0_2` accepts BOTH stamps, so a bundle written under `DEFAULT`
re-runs IN PLACE; `DEFAULT` does not accept the v0.2 form, so the recognition
is one-way and the fail-safe from V-A3 is preserved.
3. **`sources`.** Derived from the manifest's source as an inline flow sequence
(requirement 1's measured form), with the locator taken verbatim per source
type: `root`, `connection_ref`, `base_url`. `credential_ref` is never a
locator and never reaches frontmatter.
`generated.by` is `process:llm-ingestion-okf` — option (d) from the plan's V1,
chosen by the operator 2026-07-27 over the `<producer>/<version>` form A-E3 was
written against. The value carries no version, which is what keeps a byte-
compared fixture stable across releases. **A-E3 is now stale in the pilot
specification and owes po-claude a correction.**
"""
from __future__ import annotations
import re
from pathlib import Path
import pytest
from llm_ingestion_okf.errors import MaterializationError
from llm_ingestion_okf.manifest import (
Extraction,
FileSource,
HttpSource,
Manifest,
Source,
SqlSource,
)
from llm_ingestion_okf.materialize import _is_ingest_owned, _render_concept_file
from llm_ingestion_okf.profiles import DEFAULT, OKF_LATEST, OKF_V0_2, STRICT_V1, BundleProfile
INGESTED_AT = "2026-07-25T12:00:00Z"
STAMP = "manifest@0123456789abcdef"
V0_2_STAMP = f"{{ by: process:llm-ingestion-okf, at: {INGESTED_AT} }}"
def _manifest(source: Source) -> Manifest:
return Manifest(
manifest_version=1,
source=source,
bundle_summary="A test bundle.",
extractions=(
Extraction(
id="orders", title="Orders", query="orders.csv", okf_type="dataset", max_rows=10
),
),
)
def _concept(source: Source, *, profile: BundleProfile = OKF_V0_2) -> str:
manifest = _manifest(source)
return _render_concept_file(
manifest,
manifest.extractions[0],
"Body.\n",
ingested_at=INGESTED_AT,
stamp=STAMP,
profile=profile,
)
def _stamped_concept(generated: str) -> bytes:
return (
"---\ntype: dataset\ntitle: Orders\n"
f"ingest_manifest: {STAMP}\n"
f"generated: {generated}\n---\n\nBody.\n"
).encode()
# --- the profile constant -------------------------------------------------
def test_the_v0_2_profile_names_the_five_families_it_must_express() -> None:
"""C-E3's list, as emission order rather than as prose. Naming a key is what
lets a caller emit it in canonical order; judging a structured value is D1's
reader and deliberately absent here.
"""
order = OKF_V0_2.frontmatter.order
for key in ("generated", "sources", "verified", "status", "stale_after"):
assert key in order
assert "timestamp" not in order # §13.1: the fallback is for v0.1 documents
assert order.index("generated") < order.index("sources")
def test_the_v0_2_profile_requires_only_type() -> None:
"""§4/§11: `type` is the only always-required key, and a concept carrying
just `type` is fully conformant."""
assert OKF_V0_2.frontmatter.required == frozenset({"type"})
def test_the_v0_2_profile_closes_neither_the_type_set_nor_the_key_namespace() -> None:
"""§14's MUST NOTs, expressed as configuration: a consumer must not reject
on an unknown `type` value or on unknown additional keys. A profile that
closed either would be non-conformant against the version it is named for.
"""
assert OKF_V0_2.types.allowed is None
assert OKF_V0_2.frontmatter.allowed is None
assert OKF_V0_2.frontmatter.key_pattern is None
def test_the_v0_2_profile_keeps_the_source_query_collapse_and_the_default_paths() -> None:
"""The parts that are commons' §5 rather than upstream's: the doors write
the same filenames whatever the OKF version, and a multi-line SELECT still
renders on one line."""
assert OKF_V0_2.frontmatter.collapsed_keys == frozenset({"source_query"})
assert OKF_V0_2.paths == DEFAULT.paths
def test_okf_latest_still_means_v0_1_until_ga() -> None:
"""`OKF_LATEST` is "the latest version supported as STABLE", not the latest
present in the tree. Flipping it is the GA event — a single auditable
action rather than a side effect of this merge — so during the pilot a
consumer bound to the alias keeps v0.1 semantics.
"""
assert OKF_LATEST is DEFAULT
# --- ownership ------------------------------------------------------------
def test_the_default_profile_still_stamps_the_v0_1_literal() -> None:
assert DEFAULT.ownership.stamp(INGESTED_AT) == "true"
def test_the_v0_2_stamp_is_a_flow_mapping_whose_at_is_the_argument_exactly() -> None:
"""A-E3, corrected: the actor is the §7 `process:<id>` form and carries no
version. `at` binds to the already-validated `ingested_at` argument, so no
wall-clock is introduced and two runs with the same argument agree.
"""
assert OKF_V0_2.ownership.stamp(INGESTED_AT) == V0_2_STAMP
assert "llm-ingestion-okf/" not in OKF_V0_2.ownership.stamp(INGESTED_AT)
def test_the_v0_2_predicate_accepts_the_stamp_its_own_profile_writes(tmp_path: Path) -> None:
"""A-E5's second run: a v0.2 bundle re-runs into the directory it wrote."""
path = tmp_path / "ingest-orders.md"
path.write_bytes(_stamped_concept(V0_2_STAMP))
assert _is_ingest_owned(path, "manifest", profile=OKF_V0_2) is True
def test_the_v0_2_predicate_also_accepts_the_v0_1_stamp(tmp_path: Path) -> None:
"""The operator's decision (2026-07-27): a `DEFAULT`-written bundle is
re-runnable IN PLACE under `OKF_V0_2`. Without this the profile switch would
fire `collision_unstamped` on the library's own previous output, and the
black-box promise — an upstream release costs a consumer a re-run, nothing
more — would not hold across the profile change.
"""
path = tmp_path / "ingest-orders.md"
path.write_bytes(_stamped_concept("true"))
assert _is_ingest_owned(path, "manifest", profile=OKF_V0_2) is True
def test_the_default_predicate_still_refuses_the_v0_2_stamp(tmp_path: Path) -> None:
"""Recognition is one-way, which is what preserves V-A3. `DEFAULT` meeting a
v0.2 file refuses the run rather than replacing the file."""
path = tmp_path / "ingest-orders.md"
path.write_bytes(_stamped_concept(V0_2_STAMP))
assert _is_ingest_owned(path, "manifest") is False
assert _is_ingest_owned(path, "manifest", profile=DEFAULT) is False
def test_a_foreign_v0_2_actor_is_never_ingest_owned(tmp_path: Path) -> None:
"""U3 as a test: `generated` is attribution, and upstream writes it for
HAND-AUTHORED files. Presence proves nothing — the actor has to be ours, or
the collision gate would claim a human's file that merely carries the key.
"""
path = tmp_path / "ingest-orders.md"
path.write_bytes(_stamped_concept(f"{{ by: human:jsmith@acme, at: {INGESTED_AT} }}"))
assert _is_ingest_owned(path, "manifest", profile=OKF_V0_2) is False
def test_the_v0_2_stamp_still_has_to_name_this_manifest(tmp_path: Path) -> None:
"""§10.2 per-manifest ownership is unchanged by the profile: a sibling
manifest sharing the bundle keeps its own files."""
path = tmp_path / "ingest-orders.md"
path.write_bytes(_stamped_concept(V0_2_STAMP))
assert _is_ingest_owned(path, "other-manifest", profile=OKF_V0_2) is False
# --- `sources` derivation -------------------------------------------------
SOURCE_CASES = [
pytest.param(FileSource(id="golden-catalogue", root="fixture"), "fixture", id="file"),
pytest.param(
SqlSource(id="golden-db", connection_ref="OKF_GOLDEN_SQL_DB"),
"OKF_GOLDEN_SQL_DB",
id="sql",
),
pytest.param(
HttpSource(id="golden-api", base_url="https://golden.example.test"),
"https://golden.example.test",
id="http",
),
]
@pytest.mark.parametrize("source,resource", SOURCE_CASES)
def test_sources_is_an_inline_flow_sequence_carrying_the_locator(
source: Source, resource: str
) -> None:
"""A-E4. One entry, `id` from the manifest source and `resource` its locator
verbatim — the filesystem root, the env-var NAME holding the DSN, or the base
URL. Upstream's `resource` is a bundle-internal path; ours is not, and
inventing the `author`/`last_modified` upstream also carries would be writing
fields with no reader.
"""
line = f"sources: [{{ id: {source.id}, resource: {resource} }}]"
assert line in _concept(source).splitlines()
def test_sources_follows_generated_in_the_emitted_block() -> None:
"""The order the schema declares is the order that reaches disk."""
lines = _concept(FileSource(id="golden-catalogue", root="fixture")).splitlines()
assert [line.split(":")[0] for line in lines[1:9]] == [
"type",
"title",
"source_system",
"source_query",
"ingested_at",
"ingest_manifest",
"generated",
"sources",
]
def test_the_default_profile_emits_neither_sources_nor_a_v0_2_generated() -> None:
"""V-A6 at the unit level: the seven-key v0.1 block, byte for byte. The
golden suite proves the same thing end to end; this one localizes a failure
to the renderer rather than to a fixture diff.
"""
rendered = _concept(FileSource(id="golden-catalogue", root="fixture"), profile=DEFAULT)
assert rendered == (
"---\n"
"type: dataset\n"
"title: Orders\n"
"source_system: golden-catalogue\n"
"source_query: orders.csv\n"
f"ingested_at: {INGESTED_AT}\n"
f"ingest_manifest: {STAMP}\n"
"generated: true\n"
"---\n"
"\n"
"Body.\n"
)
def test_the_http_credential_reference_never_reaches_the_frontmatter() -> None:
"""A credential reference is not a locator. The convention forbids the
credential itself in frontmatter; the env-var NAME holding it has no reader
in a bundle either, so it is not written.
"""
source = HttpSource(
id="golden-api",
base_url="https://golden.example.test",
credential_ref="OKF_GOLDEN_API_TOKEN",
)
assert "OKF_GOLDEN_API_TOKEN" not in _concept(source)
@pytest.mark.parametrize(
"root",
["data, backup", "data{1}", "data[1]", "key: value"],
ids=["comma", "braces", "brackets", "colon-space"],
)
def test_a_locator_that_would_restructure_the_flow_mapping_is_refused(root: str) -> None:
"""Fail-fast rather than emit a silently WRONG provenance record.
These characters terminate or restructure a YAML flow mapping, and the
failure is quiet rather than loud. Measured with PyYAML 2026-07-27 rather
than argued: `[{ id: a, resource: data, backup }]` raises nothing and yields
`[{'id': 'a', 'resource': 'data', 'backup': None}]` — a clean parse into a
record no one wrote. Refusing is the same posture as the filename-length
gate: validation, not repair.
The same run confirms what IS emitted parses as intended — `generated` to a
two-key mapping, `sources` to a one-element list of one — including a base
URL, whose colons stay inside a plain scalar in flow context.
"""
with pytest.raises(MaterializationError) as excinfo:
_concept(FileSource(id="golden-catalogue", root=root))
assert excinfo.value.code == "source_reference_unquotable"
def test_the_default_profile_is_unaffected_by_an_unquotable_locator() -> None:
"""The refusal is a property of the v0.2 emission, not a new gate on v0.1:
`DEFAULT` never writes the locator, so it has nothing to refuse. A shared
refusal would have broken additivity — a manifest that ran yesterday must
still run."""
rendered = _concept(FileSource(id="golden-catalogue", root="data, backup"), profile=DEFAULT)
assert "data, backup" not in rendered
# --- V-A5 extended to the new profile -------------------------------------
_VERSION_LITERAL_RE = re.compile(r"^\d+(\.\d+)+$")
@pytest.mark.parametrize(
"profile", [DEFAULT, STRICT_V1, OKF_V0_2], ids=["DEFAULT", "STRICT_V1", "OKF_V0_2"]
)
def test_no_profile_hard_codes_an_upstream_version(profile: BundleProfile) -> None:
"""V-A5, restated over the profile the name of which is the one place a
version literal would look natural. `OKF_V0_2` is a NAME; the value belongs
to catalog (E1) and is declared at D5, in the fixture, once.
Kept alongside the characterization suite's copy rather than replacing it:
that one pins the two shipped profiles at the point they were measured.
"""
from test_okf_v0_2_characterization import _strings_in
assert sorted({s for s in _strings_in(profile) if _VERSION_LITERAL_RE.match(s)}) == []

11
tests/test_profile.py
View file

@ -207,11 +207,18 @@ def test_source_query_is_the_only_collapsed_key() -> None:
def test_a_profile_is_assembled_from_its_policies() -> None:
"""The profile is the four policies and nothing else — no behavior branches
outside what the object expresses."""
"""The profile is its policies and nothing else — no behavior branches
outside what the object expresses.
`ownership` joined the original four at D2, and it is the same rule rather
than an exception to it: the ingest stamp differs per profile, so the
alternative was a version branch inside the collision gate. A policy on the
object is what keeps the emitter and the predicate changeable only together.
"""
assert {field.name for field in dataclasses.fields(BundleProfile)} == {
"types",
"frontmatter",
"paths",
"index",
"ownership",
}