open/llm-ingestion-pipeline-security
1
0
Fork 0
Code Issues Releases 4 Activity

feat(okf): sources becomes expressible, and the parent key is what admits resource

Browse source 
Order 20260902T150716Z from .claude -- a K5 blocker in the OKF programme.
`parse_frontmatter` rejected `sources` in every form the spec and its
producers actually use. Measured 02.09 by two consumers independently:
`sources: [{ id: a, resource: x }]` raised on the `[` indicator (one entry
as well as two), and the block sequence of block mappings -- SPEC.md 5.1's
OWN example -- raised "nested mappings are not supported". `resource` is
REQUIRED within a `sources` entry, so the whole provenance family was
unwritable and a bundle written the way the spec documents it was refused.

Measured against the spec before coding, not reasoned: 5.1's example block
is the canonical carrier for a REQUIRED field and 11.1 defines conformance
as parseable frontmatter, so refusing it refuses a conformant bundle. Both
carriers now parse to the same list of dicts.

The load-bearing change is not the carrier, it is WHO admits `resource`.
1.2.0 left it off the allowlist arguing the parser could not tell
`sources[].resource` (5.1, a citation) from `executor.resource` /
`attester.resource` (10, a pointer to code to be run -- the door-C route
closed in 1.1.0). That premise was false: the owning key is in scope at
every call site and was simply never threaded through. It is threaded now,
so the discrimination is structural, and door C stays shut through EVERY
carrier including the two this adds -- pinned by a new test that drives
`executor`/`attester` through all four.

Refusal stays the default elsewhere. A flow sequence of plain scalars
(`tags: [a, b]`) still raises: the sequence carrier is opened for the flow
mapping element and nothing else. A `sources` entry admits scalar leaves
only, so 5.1's optional PER-ENTRY `usage_window` is refused -- no nesting
past depth 1 is a security property and it was not spent here; registered
as a conformance gap rather than left as an oversight. A block list may not
mix scalars and mappings, because a consumer reading `entry.get("id")` over
one gets an AttributeError off the first str.

New residual registered: `sources[].resource` is scanned as text (T1) but
never URL-validated. T3's https allowlist cannot reach it without
over-blocking conformant bundles -- 5.1 permits bundle-relative paths and
scope descriptors, and the producers' own golden emits `resource: fixture`.
A consumer that dereferences it must call `validate_resource_url` itself.

Suite 834 -> 859 green. 25 new rows; four pre-existing rows changed because
this release changed the behaviour they pinned, two of them renamed since
their names asserted the old invariant (`exactly_one_route_to_a_mapping`,
`two_keys_per_item_is_where_the_block_list_hard_rejects`). Not "unchanged".
130/130 classes, 6/6 gaps hold, 44 -> 45 limitations, ReDoS 0/152 (the
sweep adds no evidence here -- this change adds no regex and the splitting
is linear). Six version surfaces bumped by hand, no sed. Re-measured alone
after the bump.

No exported surface changed; no detector behaviour and no calibration
changed.
This commit is contained in:
Kjell Tore Guttormsen 2026-09-02 17:21:26 +02:00
commit a965e8ac5b
10 changed files with 555 additions and 105 deletions
Download patch file Download diff file Expand all files Collapse all files

52
CHANGELOG.md
View file

@ -5,6 +5,58 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.3.0] — 2026-09-02
### Added — the `sources` provenance family becomes expressible in both spec carriers
`okf.parse_frontmatter` rejected `sources` in every form the OKF spec and its
producers actually use. Measured 2026-09-02 by two consumers independently:
`sources: [{ id: a, resource: x }]` raised on the `[` indicator (with one entry
as with two), and the block sequence of block mappings — SPEC.md §5.1's *own*
example — raised `"nested mappings are not supported"`. `resource` is REQUIRED
within a `sources` entry (§5.1), so the whole provenance family was unwritable,
and a bundle written the way the spec documents it was refused.
Admitted now, both parsing to the same value (a list of dicts):
```yaml
sources: [{ id: a, resource: https://e.com/a }]
sources:
- id: a
resource: https://e.com/a
```
**`resource` is allowlisted inside a `sources` entry and nowhere else.** 1.2.0
left it off the allowlist on the argument that the parser could not tell
`sources[].resource` (§5.1, a citation) from `executor.resource` /
`attester.resource` (§10, a pointer to code to be run — the door-C route closed
in 1.1.0). That premise was false: the owning key is in scope at every call
site and was simply never threaded through. It is threaded now, so the
discrimination is structural rather than a judgement about the value, and
`executor: [{ resource: skills/run.md }]` and `attester:\n - resource: …` are
refused on the allowlist through *every* carrier, including the two this adds.
Refusal stays the default everywhere else. A flow sequence of plain scalars
(`tags: [a, b]`) still raises — the sequence carrier is opened for the flow
mapping element and nothing else. A `sources` entry still admits scalar leaves
only, so SPEC §5.1's optional *per-entry* `usage_window` (a mapping inside a
mapping) is refused: no nesting past depth 1 is a security property, and it was
not spent here. Registered as a conformance gap in `docs/LIMITATIONS.md`. A
block list may not mix scalar items and mappings. Off-allowlist keys, anchors,
aliases, tags, duplicate keys and unclosed collections raise as before, and a
refused mapping still raises rather than degrading into a string.
**New residual, registered:** `sources[].resource` is scanned as text (T1) but
never validated as a URL. T3's https allowlist cannot be extended to it without
over-blocking conformant bundles — §5.1 permits a bundle-relative path or a
scope descriptor a consumer cannot follow at all. A consumer that dereferences
it must call `okf.validate_resource_url` itself.
No exported surface changed and no detector behaviour or calibration changed.
Suite 834 → 859 (25 new rows, plus four pre-existing rows updated where this
release changed the behaviour they pinned); 130/130 classes, 6/6 documented
gaps hold, 45 limitations, ReDoS sweep 0/152 candidates flagged.
## [1.2.0] — 2026-08-23
### Added — OKF frontmatter can express one mapping form: typed and allowlisted