Order 20260902T150716Z from .claude -- a K5 blocker in the OKF programme.
`parse_frontmatter` rejected `sources` in every form the spec and its
producers actually use. Measured 02.09 by two consumers independently:
`sources: [{ id: a, resource: x }]` raised on the `[` indicator (one entry
as well as two), and the block sequence of block mappings -- SPEC.md 5.1's
OWN example -- raised "nested mappings are not supported". `resource` is
REQUIRED within a `sources` entry, so the whole provenance family was
unwritable and a bundle written the way the spec documents it was refused.
Measured against the spec before coding, not reasoned: 5.1's example block
is the canonical carrier for a REQUIRED field and 11.1 defines conformance
as parseable frontmatter, so refusing it refuses a conformant bundle. Both
carriers now parse to the same list of dicts.
The load-bearing change is not the carrier, it is WHO admits `resource`.
1.2.0 left it off the allowlist arguing the parser could not tell
`sources[].resource` (5.1, a citation) from `executor.resource` /
`attester.resource` (10, a pointer to code to be run -- the door-C route
closed in 1.1.0). That premise was false: the owning key is in scope at
every call site and was simply never threaded through. It is threaded now,
so the discrimination is structural, and door C stays shut through EVERY
carrier including the two this adds -- pinned by a new test that drives
`executor`/`attester` through all four.
Refusal stays the default elsewhere. A flow sequence of plain scalars
(`tags: [a, b]`) still raises: the sequence carrier is opened for the flow
mapping element and nothing else. A `sources` entry admits scalar leaves
only, so 5.1's optional PER-ENTRY `usage_window` is refused -- no nesting
past depth 1 is a security property and it was not spent here; registered
as a conformance gap rather than left as an oversight. A block list may not
mix scalars and mappings, because a consumer reading `entry.get("id")` over
one gets an AttributeError off the first str.
New residual registered: `sources[].resource` is scanned as text (T1) but
never URL-validated. T3's https allowlist cannot reach it without
over-blocking conformant bundles -- 5.1 permits bundle-relative paths and
scope descriptors, and the producers' own golden emits `resource: fixture`.
A consumer that dereferences it must call `validate_resource_url` itself.
Suite 834 -> 859 green. 25 new rows; four pre-existing rows changed because
this release changed the behaviour they pinned, two of them renamed since
their names asserted the old invariant (`exactly_one_route_to_a_mapping`,
`two_keys_per_item_is_where_the_block_list_hard_rejects`). Not "unchanged".
130/130 classes, 6/6 gaps hold, 44 -> 45 limitations, ReDoS 0/152 (the
sweep adds no evidence here -- this change adds no regex and the splitting
is linear). Six version surfaces bumped by hand, no sed. Re-measured alone
after the bump.
No exported surface changed; no detector behaviour and no calibration
changed.
OKF v0.2 writes its whole trust and provenance layer as mappings, and T2 gave
the mapping class no expressible form. A consumer measured 0 of 53 upstream
concepts through the gate on 0.3.4, 1.0.0 and 1.1.0. That was a contract
collision, not a calibration setting: SPEC.md @ 62432a09 uses flow mappings in
its own 5.1/5.2 examples, and 11 carries a hard MUST for consumers ("MUST treat
a bare `verified` mapping as a one-element list") that presupposes they parse.
Admitted: a flow mapping, as a value or as a block-list item, whose every key is
on a nine-name allowlist and whose every leaf is a plain scalar run through the
unchanged dangerous-value and mapping-construct predicates. The form is safe
because the allowlist inspects every key -- the blanket refusal was the
enforcement, not the point.
Refused, each on its own rule and ground-truthed against PyYAML 6.0.3: a key off
the allowlist, a nested collection, a quoted leaf, a duplicate key, an empty or
unclosed mapping, trailing junk, and {a:b} (which PyYAML reads as the KEY a:b).
A refused mapping still raises rather than degrading into a string, so the 1.1.0
type-confusion defect is not reopened, and the block, dotted and inline-colon
routes still raise.
`resource` is deliberately off the allowlist though SPEC.md 5.1 names it inside
a `sources` entry: it is a pointer rather than a label and the only key T3
exists for, so admitting it would let `executor: { resource: skills/run.md }`
carry an executable-code pointer through in typed clothes -- the door-C route
closed in 1.1.0. It costs nothing today, because the conformant carrier for
sources[].resource is a block sequence of block mappings, which stays refused.
Mapping leaves are scanned like every other frontmatter value (T1). Coverage
matrix 130/130 (new row: the off-allowlist key). No exported surface, detector
behaviour or calibration changed.
T2 gives the mapping class no expressible form by design. Two routes escaped
that by parsing "successfully" into the wrong TYPE instead of raising:
sources:\n - uri: https://e.com/a -> the string 'uri: https://e.com/a'
attester: resource: attesters/x.py -> the string 'resource: attesters/x.py'
Only the first was documented (LIMITATIONS.md:43). The inline second colon was
found by measurement while closing it -- a real YAML parser refuses that line
outright, ours accepted it. Shipping the list half alone would have left a
LIMITATIONS rewrite that overclaims.
Same consequence either way: a pointer parked in a degraded mapping rides
through in a key the `resource` allowlist never inspects, and mode-b
import_bundle wrote the merged concept verbatim (WARN). Both now FAIL_SECURE at
T2, before the allowlist is reached.
The boundary is where YAML puts it, ground-truthed against PyYAML 6.0.3 rather
than reasoned: ": " and a trailing ":" are exactly the two shapes where a plain
scalar becomes a mapping. A colon carrying neither a space nor a line end opens
no mapping -- domain:security and https://e.com:8443/a still parse -- and a
quoted scalar is still a scalar. Over-blocking a conformant bundle is itself a
failure mode, so the seven admitted shapes get rows of their own.
Iron Law: the four rejected rows and both import_bundle rows were written
first and seen red (7 failures, each DID NOT RAISE) before okf.py was touched.
Suite 792 -> 802. LIMITATIONS stays at 35: the bullet is reworded, not
retired -- the restricted grammar is still a limitation, the silent misparse
is no longer part of it.
`scan_active_content` called directly and `okf.link_graph` were the two surfaces
still reading attacker-supplied text with no cap — the first reached by an
adapter that wants the active-content classes alone, the second running a
`findall` over every body in a bundle. Both are detection-shaped, so they
truncate and flag rather than raise the way the transform surfaces do: what a
detector shortens is its own coverage, not the caller's content.
Truncation is only honest if it is visible, so neither goes quiet: the scanner
emits `active:oversize-input` (LLM10), and `link_graph` records
`(from_id, body_length)` in `LinkGraphResult.truncated` — the field that lets a
caller tell "no links past here" from "no links read past here".
Reached through `scan_output`, the text is already under that surface's cap and
`max_scan_chars` is now passed down, so the flag is raised once, there.
The generalised sweep found what 0.3.2's hand-written rows missed. All three are
the documented class -- a run in front of a required literal that never arrives,
so every start position rescans the tail -- and all three are worse than the
0.3.3 findings, because `sanitize`, `neutralize`, `scan_active_content` and the
okf link graph apply NO input cap. `scan_lexicon`/`scan_output` are the only
entry points that do, so there is no ceiling to extrapolate to.
sanitize._HTML_COMMENT_RE `<!--`*100_000 20.1s, exponent 1.96-2.14
active_content.URL_IN_TEXT_RE `<a `+`A`*100_000 12.99s / 14.9s, exponent ~2.0
okf._MD_LINK_RE `[`*100_000 7.1s, exponent 1.99-2.05
Each fix is the one the pattern's own shape allows, not a copied choice:
- The comment stripper drops the regex for `str.find`. Excluding `<` would lose
every comment containing markup; bounding the run would be a carrier bypass
of the exact construct the stripper exists to remove.
- `URL_IN_TEXT_RE` bounds its scheme run to an RFC 3986 scheme (`{0,63}`).
Bounding is safe *here* only because it is a defanger inside a tag already
flagged `active:raw-html`. A lookbehind was measured too and rejected: it
drops `-http://evil.com`, a one-character evasion. Bounded: 0.185s at 1M.
- `_MD_LINK_RE` excludes `[`, matching `active_content.MD_LINK_RE` exactly,
including the nested-label trade already documented there.
`sanitize` claimed "no catastrophic backtracking" in a comment; that claim was
wrong in the same way `output`'s was before 0.3.2, and is corrected in place.
676 tests (+10), coverage 128/128 + 6/6 gaps, sweep clean across 150 patterns.
The okf destination run gets no row: `[^)\s]+` cannot fail, so a row for it
could never go red.
A received OKF bundle MAY legitimately carry index.md (directory listing, read
first under progressive disclosure) and log.md (update history) at any level
(spec §3.1/§6/§7). import_bundle previously hard-rejected those basenames in the
T4 path gate, so a conformant third-party bundle was over-blocked in full
(FAIL_SECURE) — and because the reject fired before scan_concept, index.md's
body (the highest-priority injection surface) was never scanned.
import_bundle now defaults allow_reserved=True: reserved basenames are scanned
as structural files (path-safety checks — traversal / absolute / backslash / .md
— still apply). The shadow-reject (an *upload* masquerading as index.md) is
preserved: the front-end passes allow_reserved=False so a materialized upload
landing on a reserved basename is still refused. That front-end opt-in was
required to keep the shadow-reject once the default flipped (not in the plan's
Filer set; traced from the code).
- okf.py: validate_concept_path/_validate_concept/import_bundle gain the
keyword; validate_concept_path default stays False (strict standalone).
- tests: +3 (legit index/log admit; injection in index.md body caught;
okf_version frontmatter admits). Per-concept-iteration test switched to a
traversal vector; mode-b showcase's index.md surface reframed from
reserved-name-reject to index.md-body-scan.
- README honest-limits + CLAUDE.md context note the mode-b/upload distinction.
Suite: 341 -> 344 passed. Core invariant intact (dependencies=[]).
