open/llm-ingestion-pipeline-security
1
0
Fork 0
Code Issues Releases 3 Activity

fix(okf): a mapping construct no longer degrades into a string, and there were two routes

Browse source 
T2 gives the mapping class no expressible form by design. Two routes escaped
that by parsing "successfully" into the wrong TYPE instead of raising:

  sources:\n  - uri: https://e.com/a   -> the string 'uri: https://e.com/a'
  attester: resource: attesters/x.py   -> the string 'resource: attesters/x.py'

Only the first was documented (LIMITATIONS.md:43). The inline second colon was
found by measurement while closing it -- a real YAML parser refuses that line
outright, ours accepted it. Shipping the list half alone would have left a
LIMITATIONS rewrite that overclaims.

Same consequence either way: a pointer parked in a degraded mapping rides
through in a key the `resource` allowlist never inspects, and mode-b
import_bundle wrote the merged concept verbatim (WARN). Both now FAIL_SECURE at
T2, before the allowlist is reached.

The boundary is where YAML puts it, ground-truthed against PyYAML 6.0.3 rather
than reasoned: ": " and a trailing ":" are exactly the two shapes where a plain
scalar becomes a mapping. A colon carrying neither a space nor a line end opens
no mapping -- domain:security and https://e.com:8443/a still parse -- and a
quoted scalar is still a scalar. Over-blocking a conformant bundle is itself a
failure mode, so the seven admitted shapes get rows of their own.

Iron Law: the four rejected rows and both import_bundle rows were written
first and seen red (7 failures, each DID NOT RAISE) before okf.py was touched.

Suite 792 -> 802. LIMITATIONS stays at 35: the bullet is reworded, not
retired -- the restricted grammar is still a limitation, the silent misparse
is no longer part of it.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-13 22:58:27 +02:00
commit da30211bc7
3 changed files with 122 additions and 39 deletions
Download patch file Download diff file Expand all files Collapse all files

43
docs/LIMITATIONS.md
View file

@ -40,26 +40,29 @@ items; this is the full list, each with the mechanism.
(an injection in a directory listing is caught) rather than path-rejecting the
conformant bundle. A front-end materialising individual uploads keeps the opposite
rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused.
- **OKF frontmatter is a restricted grammar, and a one-key block-sequence item is
silently misparsed.** Gate T2 accepts a line-oriented subset deliberately — full
YAML is a larger parse-attack surface than a write-time gate needs. Nested mappings
and flow collections (`[a, b]`, `{k: v}`) are *rejected outright*, which fails
secure. **All three routes to a mapping fail, each on a different rule** — flow
(`{k: v}`) on the disallowed value-start indicator, block (`k:\n sub: v`) on the
nested-mapping check, and dotted keys (`k.sub: v`) on the key pattern — so the
mapping *class* has no expressible form, rather than one form being preferable to
another. What survives is scalars and flat lists of strings. The defect is between
those two outcomes: a block sequence whose items carry
exactly **one** key parses "successfully" into the wrong type —
`sources:\n - uri: https://e.com/a` yields the **string** `'uri: https://e.com/a'`,
not a mapping, while the same list with two keys per item hard-rejects. A pointer
can therefore ride through in a key the `resource` allowlist never inspects
(`attester:\n - resource: attesters/sql_equality.py` → WARN), whereas a top-level
`resource:` with a relative path correctly fails secure. The shape is not conformant
OKF, so a well-formed bundle will not produce it; a malformed or hostile one can, and
mode-b `import_bundle` writes the merged concept verbatim. Note the three block-list
shapes are *not* one case: flat scalars parse correctly, one key per item misparses
silently, two keys per item hard-rejects.
- **OKF frontmatter is a restricted grammar: the mapping class has no expressible
form.** Gate T2 accepts a line-oriented subset deliberately — full YAML is a larger
parse-attack surface than a write-time gate needs. Nested mappings and flow
collections (`[a, b]`, `{k: v}`) are *rejected outright*, which fails secure.
**All four routes to a mapping fail, each on a different rule** — flow (`{k: v}`)
on the disallowed value-start indicator, block (`k:\n sub: v`) on the
nested-mapping check, dotted keys (`k.sub: v`) on the key pattern, and the inline
second colon (`k: sub: v`) on the mapping-construct check — so the mapping *class*
has no expressible form, rather than one form being preferable to another. What
survives is scalars and flat lists of strings. **Two of those routes used to
degrade into a string instead of failing, and that defect is closed in `1.1.0`**:
a block-sequence item carrying exactly one key (`sources:\n - uri: https://e.com/a`
yielded the *string* `'uri: https://e.com/a'`) and the inline second colon
(`attester: resource: attesters/sql_equality.py`, which a real YAML parser refuses
outright). Both parsed "successfully" into the wrong *type*, and a pointer parked
in one rode through in a key the `resource` allowlist never inspects — mode-b
`import_bundle` returned WARN and wrote the merged concept verbatim. Both now
FAIL_SECURE at T2, before the allowlist is reached. **The boundary is where YAML
puts it**, ground-truthed against PyYAML 6.0.3: `": "` and a trailing `":"` open a
mapping and are refused; a colon carrying neither a space nor a line end
(`domain:security`, `https://e.com:8443/a`) does not and still parses, as does a
quoted scalar (`- "uri: x"`). Quotes are retained rather than stripped — a
divergence from YAML that remains, pinned in `tests/test_okf.py`.
- **T2 constrains import, not emission.** The frontmatter grammar runs on
`okf.import_bundle` (door C) only — `parse_frontmatter` is referenced nowhere in the
door A/B persist path, so frontmatter that fails secure on import passes