open/llm-ingestion-pipeline-security
1
0
Fork 0
Code Issues Releases 4 Activity

feat(okf): sources becomes expressible, and the parent key is what admits resource

Browse source 
Order 20260902T150716Z from .claude -- a K5 blocker in the OKF programme.
`parse_frontmatter` rejected `sources` in every form the spec and its
producers actually use. Measured 02.09 by two consumers independently:
`sources: [{ id: a, resource: x }]` raised on the `[` indicator (one entry
as well as two), and the block sequence of block mappings -- SPEC.md 5.1's
OWN example -- raised "nested mappings are not supported". `resource` is
REQUIRED within a `sources` entry, so the whole provenance family was
unwritable and a bundle written the way the spec documents it was refused.

Measured against the spec before coding, not reasoned: 5.1's example block
is the canonical carrier for a REQUIRED field and 11.1 defines conformance
as parseable frontmatter, so refusing it refuses a conformant bundle. Both
carriers now parse to the same list of dicts.

The load-bearing change is not the carrier, it is WHO admits `resource`.
1.2.0 left it off the allowlist arguing the parser could not tell
`sources[].resource` (5.1, a citation) from `executor.resource` /
`attester.resource` (10, a pointer to code to be run -- the door-C route
closed in 1.1.0). That premise was false: the owning key is in scope at
every call site and was simply never threaded through. It is threaded now,
so the discrimination is structural, and door C stays shut through EVERY
carrier including the two this adds -- pinned by a new test that drives
`executor`/`attester` through all four.

Refusal stays the default elsewhere. A flow sequence of plain scalars
(`tags: [a, b]`) still raises: the sequence carrier is opened for the flow
mapping element and nothing else. A `sources` entry admits scalar leaves
only, so 5.1's optional PER-ENTRY `usage_window` is refused -- no nesting
past depth 1 is a security property and it was not spent here; registered
as a conformance gap rather than left as an oversight. A block list may not
mix scalars and mappings, because a consumer reading `entry.get("id")` over
one gets an AttributeError off the first str.

New residual registered: `sources[].resource` is scanned as text (T1) but
never URL-validated. T3's https allowlist cannot reach it without
over-blocking conformant bundles -- 5.1 permits bundle-relative paths and
scope descriptors, and the producers' own golden emits `resource: fixture`.
A consumer that dereferences it must call `validate_resource_url` itself.

Suite 834 -> 859 green. 25 new rows; four pre-existing rows changed because
this release changed the behaviour they pinned, two of them renamed since
their names asserted the old invariant (`exactly_one_route_to_a_mapping`,
`two_keys_per_item_is_where_the_block_list_hard_rejects`). Not "unchanged".
130/130 classes, 6/6 gaps hold, 44 -> 45 limitations, ReDoS 0/152 (the
sweep adds no evidence here -- this change adds no regex and the splitting
is linear). Six version surfaces bumped by hand, no sed. Re-measured alone
after the bump.

No exported surface changed; no detector behaviour and no calibration
changed.
This commit is contained in:
Kjell Tore Guttormsen 2026-09-02 17:21:26 +02:00
commit a965e8ac5b
10 changed files with 555 additions and 105 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/ADOPTION-BRIEF.md
View file

@ -4,7 +4,7 @@
especially one converging on Google's Open Knowledge Format (OKF v0.1) — and needs
to decide **when** and **where** to add a write-time ingestion guard.
**Status of the guard:** `v1.2.0`. Stdlib-only core, framework-agnostic. The
**Status of the guard:** `v1.3.0`. Stdlib-only core, framework-agnostic. The
exported Python surface is frozen under semver — nothing exported is removed,
renamed or given a different meaning without a `2.0.0`. Detection behaviour is
*not* frozen: severities, thresholds and lexicon entries are calibration and move
@ -142,9 +142,9 @@ live payload:
python -m llm_ingestion_guard.coverage # exit 0 = all as documented
```
As of `v1.2.0`: **130 / 130 defended classes demonstrated (recall 100%)** and **6 /
As of `v1.3.0`: **130 / 130 defended classes demonstrated (recall 100%)** and **6 /
6 documented gaps still hold** (a *closed* gap fails the test, forcing a doc
update). The matrix is the single source of truth for the test suite (**834
update). The matrix is the single source of truth for the test suite (**859
passing**), which also asserts total recall, that every lexicon pattern has a
case (so the matrix cannot fall behind the lexicon), the full LLM02 secret-egress
set, and the container-layer front-end (CSV formula-injection, zip-slip/bomb,