fix(okf): close the inert ingest-stamp guard, then land okf 0.8.5 -- and read the block sources form all four bases actually write

P13 measured this lift and REFUSED it, because okf >=0.8.5 emits the ownership
stamp as the V1 flow mapping `generated: { by: process:okf-ingest, at: ... }`
where 0.3.2 emitted `true`, and `_carries_complete_ingest_stamp` read the new
form as NOT a stamp -- write_concept_file's forgery refusal would have shipped
DISARMED with the whole fail-closed suite green. That blocker is closed first,
red-first, and then the pin moves.

ROW 1, THE SECURITY HALF. `_claims_ingest_ownership` widens the predicate from
"reads as boolean True" to "claims ingest ownership", of which the boolean is
the pre-V1 spelling. The recogniser for the new half is `decode_flow_value` --
the module's ONE flow decoder, the same argument write_concept_file already
makes for `verified`: the writer refuses exactly what the reader can read. A
value the decoder REFUSES is therefore not an ownership claim and writes
through, which is what keeps this from collapsing into "any non-empty
generated". Two arms red before the fix; no YAML library introduced.

THE PIN. okf v0.3.2 -> v0.8.5, guard v0.3.4 -> v1.4.0 spelled `tag =`, not
`rev =`, and not the declared floor 1.2.0 -- both P13 premises hold and the
reason now lives next to the pin in pyproject.toml. The ":40" comment is
corrected: okf has ONE runtime dependency, the guard, and that is what binds
the two lines together. 27/27 imported names resolve across five modules.

THE GOLDENS, REGENERATED AS A DECISION. Seven concept files across four
examples/ingest-golden-* bundles, one line each. Two were regenerated by the
REAL materializer; the other five are derived (http/sql/mcp cannot materialize
outside the tests' stubs) and then MEASURED -- all four golden suites compare
byte for byte against what the stubs produce, and all four are green. The four
`generated == "true"` asserts now read ONE source, conftest.
expected_generated_stamp: four literals for one emitter fact are four places a
later release can leave half-corrected, which is exactly how the pre-V1 form
survived until P13 measured it. tests/test_okf.py keeps its literal on purpose
-- that one round-trips a CURATED half-stamp through our own writer.

THE BLOCK READER. Measured with the full denominator: all four delivered
knowledge bases write `sources` as a BLOCK sequence and none in flow form
(n100 446/446, n200 1133/1133, n500 270/270, r761 2756/2756 = 4605/4605), and
`evidence_for` reported `unreadable` on 4605 of 4605 -- the falsification layer
had no address for any document in any base. `okf.decode_block_mappings` is the
second CARRIER of one grammar, never a second grammar: colon-SPACE separator,
unquote_scalar, duplicate keys refused, SPEC 5.2's actor rule applied. okf's
consume.read_sources was READ for the form and not called; po calls no okf
reader, which is measured and deliberate. After: 4605 present / 4605 entries.
Reading is not a licence to WRITE -- the emitter is untouched and both writers
still refuse what decode_flow_value refuses.

THREE FINDINGS. (1) The first block reader INVENTED data on `- { k: v }` items
-- SPEC-canonical, and the shape tests/golden/block-form-provenance writes for
`verified` -- decoding it as `{'{ id': '...'}`. No arm caught it: the 5.2 actor
rule shielded the fixture by accident. Closed with a flow-decoder branch and
four new arms. (2) One of my own arms was VACUOUS, found by my own mutation M5:
it claimed to prove the colon-SPACE rule and stayed green under first-colon,
because the two rules agree on every delivered value. Renamed, labelled, and
the claim moved to the arm that actually witnesses it. (3) OPEN, and it needs
the operator: the commons-owned worked example declares its second concept
`unreadable`/`block-sequence`, which is now false for po. `shared/` is
pull-only, so closing it needs a commons amendment; the test asserts the
divergence instead of skipping it, keeping the discriminating half (the example
says two entries were seen and the reader returns exactly two).

NINE EXISTING ARMS REWRITTEN, NONE WEAKENED. All nine pinned "the block form is
unreadable" -- the behaviour this order changes. Each keeps its claim on a
specimen that is still unreadable for a reason of its own (5.2: an entry naming
no actor), or pins the REVERSED direction where the old arm stood so the change
cannot be silent. Two got STRONGER: multi-verified.md was authored for "a reader
keeping the last entry reports machine-confirmed for a concept a human signed",
and that could not be tested while the form was unreadable. Three node ids were
renamed; nothing was removed in substance.

Suite 1582 -> 1606 passed / 5 skipped. Both demo goldens byte-unchanged
(ea8c534... / ede3e2f..., shasum -a 1 of the CONTENT, never the git blob id).
ruff check / ruff format / mypy green. shared/ untouched.

Six mutations, all red against the WHOLE suite, each with its own signature:
row 1 detached (2) / block reader detached (17) / flow-item branch detached (7)
/ a stray indented line folds into an INVENTED entry (4) / separator becomes the
first colon (1 -- and that is finding 2) / the stamp expectation reverts to
"true" (4).

Order: 20260912T195112Z-995611104-from-.claude
Record: docs/2026-09-12-p13b-okf-bump.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-12 23:02:38 +02:00
commit fed69790ac
25 changed files with 964 additions and 89 deletions

View file

@ -26,7 +26,8 @@ have forgotten, as a fourth family:
never a manifest field, and it refuses BEFORE any transport call);
- the §8 ``max_rows`` cap, enforced as an ERROR rather than a silent truncation;
- §5 verbatim fenced-body rendering, including the code-fence-marker refusal;
- the §7 ``generated: true`` + ``ingest_manifest`` provenance stamp;
- the §7 ``generated`` + ``ingest_manifest`` provenance stamp (``generated`` is the V1 mapping
``{ by: <actor>, at: <timestamp> }`` as of okf 0.8.5; it was the literal ``true`` under 0.3.2);
- the §4 rule that credentials resolve at run time from the environment, never from the manifest.
**MAF-free (D7-portable).** MCP is an open protocol; ``agent_framework`` is one framework's runtime.

View file

@ -35,6 +35,7 @@ import posixpath
import re
from dataclasses import dataclass
from pathlib import Path
from collections.abc import Sequence
from typing import Any, Final, Literal
from portfolio_optimiser.ir import CostBaseline, CostBaselineLine
@ -271,6 +272,80 @@ def _decode_flow_mapping(item: str, raw: str, key: str | None) -> dict[str, str]
return entry
def decode_block_mappings(
continuation: Sequence[str], *, key: str | None = None
) -> tuple[dict[str, str], ...] | None:
"""Decode a BLOCK sequence of mappings into entries, or ``None`` when it cannot be read.
``None`` means "this reader cannot decode it", NEVER "there is nothing here": an indented line
before any ``- `` opens no entry, and folding it into one would invent an entry the document
does not carry. The caller turns ``None`` into the ``UnreadableProvenance`` it already returned.
**The second carrier of the same shape, never a second grammar.** The pair separator is
colon-SPACE via ``_find_pair_separator``, names and values go through ``unquote_scalar``,
duplicate keys are refused, and the one key-specific rule SPEC §5.2 imposes (a ``verified``
entry must name an actor) applies here too all of it the SAME rule ``_decode_flow_mapping``
applies to the flow carrier. Two grammars would be two answers to one question, and a delivered
base would read differently depending on which spelling its producer chose. The colon-SPACE part
is load-bearing rather than stylistic: every delivered ``resource`` is a URL, so a reader
splitting on the FIRST colon would truncate all 4605 of them at ``https``.
Measured 2026-09-12: all four delivered knowledge bases write ``sources`` in this form and none
in flow form (n100 446/446, n200 1133/1133, n500 270/270, r761 2756/2756). Reading it is not a
licence to WRITE it ``write_concept_file``/``verified_field`` still refuse exactly what
``decode_flow_value`` refuses, so the emission rule and the round-trip gate are untouched.
Gated by ``tests/test_block_sources_reader_loadbearing.py``."""
entries: list[dict[str, str]] = []
from_flow = False
for line in continuation:
if not line.strip():
continue
item = line.strip()
opened = item.startswith("- ")
if opened:
entries.append({})
item = item[2:].strip()
from_flow = False
elif not entries:
return None
if item.startswith("{"):
# A FLOW mapping as the item. Decoded by ``decode_flow_value`` — the ONE decoder —
# rather than by the pair loop below, which would read ``{ id`` as a key and hand back
# an entry the document does not carry. Measured 2026-09-12: this is the shape
# ``tests/golden/block-form-provenance`` writes for ``verified``, and the pair loop
# produced ``{'{ id': 'a, resource: ... }'}`` for it.
if not opened:
return None
try:
decoded = decode_flow_value(item, key=key)
except FlowDecodeError:
return None
if len(decoded) != 1:
return None
entries[-1] = dict(decoded[0])
from_flow = True
continue
if from_flow:
# A bare pair continuing an entry that was opened as a flow mapping. Mixing the two
# spellings within one entry is refused rather than merged: merging would decide
# silently which carrier wins for a document that used both.
return None
at = _find_pair_separator(item)
if at < 0:
return None
name = unquote_scalar(item[:at])
value = unquote_scalar(item[at + len(_FLOW_PAIR_SEPARATOR) :])
if name in entries[-1]:
return None
entries[-1][name] = value
if not entries:
return None
if key == "verified" and any(not entry.get("by") for entry in entries):
return None
return tuple(entries)
def decode_flow_value(raw: str, *, key: str | None = None) -> tuple[dict[str, str], ...]:
"""Decode the accepted single-line flow subset into a tuple of entries.
@ -707,6 +782,14 @@ def read_provenance(
# inside a value is text, not an item, and counting it would inflate the number the caller
# acts on.
items = sum(1 for ln in continuation if ln.strip().startswith("- "))
# P13b: the block sequence of mappings is SPEC §5.1's own example and the form all four
# delivered bases actually write (4605/4605, measured). It is decoded by the second carrier
# of the one grammar; a block MAPPING (no item opened) and anything the grammar refuses
# still come back unreadable, which is what keeps "we could not read it" from quietly
# becoming "there was nothing to read".
block = decode_block_mappings(continuation, key=key)
if block is not None:
return block
return UnreadableProvenance(
file=str(path),
key=key,
@ -1236,10 +1319,37 @@ Widening past what a YAML reader actually resolves would over-block curated cont
pipeline ever produces, on a form nothing downstream would honour as the stamp either."""
def _claims_ingest_ownership(value: str) -> bool:
"""Whether a ``generated`` value claims ingest ownership, in EITHER spelling the pinned writer
has used.
Two forms, one claim. The pre-V1 spelling is a YAML boolean (``_YAML_TRUE_LITERALS``). The V1
spelling is the provenance mapping ``{ by: <actor>, at: <timestamp> }`` measured 2026-09-12 as
what ``llm-ingestion-okf`` >=0.8.5 writes into every generated concept, where 0.3.2 wrote
``true``. A detector that knew only the boolean read the V1 form as NOT a stamp, so the forgery
refusal in ``write_concept_file`` went inert on exactly the output of the writer it guards
against, with the whole fail-closed suite green (P13, ``docs/2026-09-12-p13-okf-pin-r761.md``
§ 2c). Widening the predicate from "reads as boolean True" to "claims ownership" is what makes
the guard survive the emitter, rather than the emitter's current spelling.
The recogniser for the V1 half is ``decode_flow_value`` the module's ONE flow decoder, never a
second copy of the rule (the -(p) precedent, and the same argument ``write_concept_file``
already makes for ``verified``: the writer refuses exactly what the reader can read). A value the
decoder REFUSES is therefore not an ownership claim and writes through, which is what keeps this
from collapsing into "any non-empty ``generated``"."""
literal = value.strip().strip('"').lower()
if literal in _YAML_TRUE_LITERALS:
return True
try:
return bool(decode_flow_value(value.strip(), key="generated"))
except FlowDecodeError:
return False
def _carries_complete_ingest_stamp(frontmatter: dict[str, str]) -> bool:
"""Whether ``frontmatter`` carries BOTH halves of the ingest ownership stamp: a ``generated``
value a YAML reader would read as boolean ``True`` (``_YAML_TRUE_LITERALS``) together with a
non-empty ``ingest_manifest`` reference (ingest-spec §7).
value claiming ingest ownership (``_claims_ingest_ownership``) together with a non-empty
``ingest_manifest`` reference (ingest-spec §7).
FAIL-CLOSED on the value literal: the field previously matched only the exact string ``"true"``,
so a pinned ingest writer emitting any other YAML-1.1 truthy form (``yes``, ``on``) would have
@ -1249,9 +1359,8 @@ def _carries_complete_ingest_stamp(frontmatter: dict[str, str]) -> bool:
field, and a verbatim round-trip of one half must keep working. Values are compared the way
``parse_frontmatter`` yields them (line-oriented strings, quotes retained), so surrounding quotes
and case are normalised away here."""
generated = str(frontmatter.get("generated", "")).strip().strip('"').lower()
manifest = str(frontmatter.get("ingest_manifest", "")).strip().strip('"')
return generated in _YAML_TRUE_LITERALS and bool(manifest)
return _claims_ingest_ownership(str(frontmatter.get("generated", ""))) and bool(manifest)
def write_concept_file(bundle_dir: str, name: str, frontmatter: dict[str, str], body: str) -> Path:
@ -1275,7 +1384,8 @@ def write_concept_file(bundle_dir: str, name: str, frontmatter: dict[str, str],
if _carries_complete_ingest_stamp(frontmatter):
raise IngestStampError(
"refusing to write a curated concept file carrying the COMPLETE ingest ownership stamp "
"(generated: true + ingest_manifest); only the ingest materializer may claim it "
"(a `generated` ownership claim — `true` or the V1 `{ by: ..., at: ... }` mapping — "
"together with ingest_manifest); only the ingest materializer may claim it "
"(ingest-spec §3) — either field alone is permitted"
)
resolved = Path(safe_resolve(bundle_dir, name))