feat(okf): the mapping class gets one expressible form, typed and allowlisted
OKF v0.2 writes its whole trust and provenance layer as mappings, and T2 gave
the mapping class no expressible form. A consumer measured 0 of 53 upstream
concepts through the gate on 0.3.4, 1.0.0 and 1.1.0. That was a contract
collision, not a calibration setting: SPEC.md @ 62432a09 uses flow mappings in
its own 5.1/5.2 examples, and 11 carries a hard MUST for consumers ("MUST treat
a bare `verified` mapping as a one-element list") that presupposes they parse.
Admitted: a flow mapping, as a value or as a block-list item, whose every key is
on a nine-name allowlist and whose every leaf is a plain scalar run through the
unchanged dangerous-value and mapping-construct predicates. The form is safe
because the allowlist inspects every key -- the blanket refusal was the
enforcement, not the point.
Refused, each on its own rule and ground-truthed against PyYAML 6.0.3: a key off
the allowlist, a nested collection, a quoted leaf, a duplicate key, an empty or
unclosed mapping, trailing junk, and {a:b} (which PyYAML reads as the KEY a:b).
A refused mapping still raises rather than degrading into a string, so the 1.1.0
type-confusion defect is not reopened, and the block, dotted and inline-colon
routes still raise.
`resource` is deliberately off the allowlist though SPEC.md 5.1 names it inside
a `sources` entry: it is a pointer rather than a label and the only key T3
exists for, so admitting it would let `executor: { resource: skills/run.md }`
carry an executable-code pointer through in typed clothes -- the door-C route
closed in 1.1.0. It costs nothing today, because the conformant carrier for
sources[].resource is a block sequence of block mappings, which stays refused.
Mapping leaves are scanned like every other frontmatter value (T1). Coverage
matrix 130/130 (new row: the off-allowlist key). No exported surface, detector
behaviour or calibration changed.
This commit is contained in:
parent
7cb4553301
commit
58704834b6
7 changed files with 432 additions and 43 deletions
39
CHANGELOG.md
39
CHANGELOG.md
|
|
@ -7,6 +7,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Added — OKF frontmatter can express one mapping form: typed and allowlisted
|
||||||
|
|
||||||
|
`okf.parse_frontmatter` gave the mapping *class* no expressible form at all. OKF
|
||||||
|
v0.2 writes its whole trust and provenance layer as mappings — SPEC.md @
|
||||||
|
`62432a09` uses flow form in its own §5.1/§5.2 examples, and §11 carries a hard
|
||||||
|
MUST for consumers ("MUST treat a bare `verified` mapping as a one-element
|
||||||
|
list") that presupposes they parse. A consumer measured **0 of 53** upstream
|
||||||
|
concepts through the gate on 0.3.4, 1.0.0 and 1.1.0. That was a contract
|
||||||
|
collision, not a calibration setting: no threshold would have moved it.
|
||||||
|
|
||||||
|
Admitted now: a flow mapping (`generated: { by: x, at: y }`), as a value or as a
|
||||||
|
block-list item, whose every key is on a nine-name allowlist (`by`, `at`, `from`,
|
||||||
|
`to`, `id`, `title`, `author`, `usage_count`, `last_modified`) and whose every
|
||||||
|
leaf is a plain scalar run through the *unchanged* dangerous-value and
|
||||||
|
mapping-construct predicates.
|
||||||
|
|
||||||
|
The form is additive and refusal stays the default. A key off the allowlist, a
|
||||||
|
nested collection, a quoted leaf, a duplicate key, an empty or unclosed mapping,
|
||||||
|
and `{a:b}` (which PyYAML 6.0.3 reads as the *key* `a:b`) all raise, and a
|
||||||
|
refused mapping still raises rather than degrading into a string — the 1.1.0
|
||||||
|
defect is not reopened. Nested-block (`k:\n sub: v`), dotted (`k.sub: v`) and
|
||||||
|
inline-second-colon (`k: sub: v`) routes to a mapping still raise, each on its
|
||||||
|
own rule.
|
||||||
|
|
||||||
|
**`resource` is deliberately off the allowlist**, though SPEC.md §5.1 names it
|
||||||
|
inside a `sources` entry. It is a pointer rather than a label and the only key
|
||||||
|
T3 exists for: admitting it would let `executor: { resource: skills/run.md }`
|
||||||
|
carry an executable-code pointer through in typed clothes, which is the door-C
|
||||||
|
route closed in 1.1.0. It costs nothing today — the conformant carrier for
|
||||||
|
`sources[].resource` is a block sequence of block mappings, which this form does
|
||||||
|
not admit either way.
|
||||||
|
|
||||||
|
Mapping leaves are scanned like every other frontmatter value (T1), so an
|
||||||
|
injection parked in `generated: { by: ... }` reaches `scan_output`. Coverage
|
||||||
|
matrix: 130 classes, up from 129 (the new row is the off-allowlist key).
|
||||||
|
|
||||||
|
No exported surface changed; no detector behaviour and no calibration changed.
|
||||||
|
|
||||||
|
|
||||||
### Changed — the ReDoS sweep now measures on the same clock as the bounds it justifies
|
### Changed — the ReDoS sweep now measures on the same clock as the bounds it justifies
|
||||||
|
|
||||||
`docs/redos-sweep.py` timed on `time.monotonic()` while every ReDoS bound in the
|
`docs/redos-sweep.py` timed on `time.monotonic()` while every ReDoS bound in the
|
||||||
|
|
|
||||||
|
|
@ -15,7 +15,14 @@ Repoet er på **v1.1.0** — den eksporterte Python-surfacen er frosset under se
|
||||||
(deteksjonsatferd er det IKKE; kalibrering flytter seg i 1.x). Stdlib-kjernen er
|
(deteksjonsatferd er det IKKE; kalibrering flytter seg i 1.x). Stdlib-kjernen er
|
||||||
bygget og testet (15 moduler +
|
bygget og testet (15 moduler +
|
||||||
topp-nivå wiring, showcase + korpus), inkl. OKF-adapter og aktivt-innhold-
|
topp-nivå wiring, showcase + korpus), inkl. OKF-adapter og aktivt-innhold-
|
||||||
detektor (EchoLeak-klassen) i output-gaten. Mode-b `import_bundle` skanner
|
detektor (EchoLeak-klassen) i output-gaten. OKF-frontmatterens mapping-klasse
|
||||||
|
har **én** uttrykkbar form (G3, 21.08): en flow-mapping (`generated: { by: x, at: y }`) — som verdi eller som blokkliste-
|
||||||
|
element — der HVER nøkkel står på en ni-navns allowlist og hvert blad er en ren
|
||||||
|
skalar. Formen er trygg fordi allowlisten inspiserer hver nøkkel; det blanke
|
||||||
|
avslaget var håndhevelsen, ikke poenget. `resource` er bevisst UTE av
|
||||||
|
allowlisten (peker, ikke etikett — den ene nøkkelen T3 finnes for). Blokk-,
|
||||||
|
dotted- og inline-kolon-rutene raiser fortsatt, og en avvist mapping raiser —
|
||||||
|
den degraderer aldri til en streng (1.1.0-defekten). Mode-b `import_bundle` skanner
|
||||||
reserverte strukturfiler (`index.md`/`log.md`) i mottatte bundles i stedet for å
|
reserverte strukturfiler (`index.md`/`log.md`) i mottatte bundles i stedet for å
|
||||||
path-avvise dem; upload-front-end beholder shadow-reject (`allow_reserved=False`).
|
path-avvise dem; upload-front-end beholder shadow-reject (`allow_reserved=False`).
|
||||||
Output-gatens decode-and-rescan mater dekodet base64-klartekst gjennom BÅDE lexicon
|
Output-gatens decode-and-rescan mater dekodet base64-klartekst gjennom BÅDE lexicon
|
||||||
|
|
|
||||||
12
README.md
12
README.md
|
|
@ -165,7 +165,15 @@ Per-concept gates: **path / reserved-name** (rejects `..` traversal and reserved
|
||||||
reject-by-default loader that refuses anchors, aliases, and explicit tags *by
|
reject-by-default loader that refuses anchors, aliases, and explicit tags *by
|
||||||
construction*, so a billion-laughs alias expansion or a `!!python/object` coercion
|
construction*, so a billion-laughs alias expansion or a `!!python/object` coercion
|
||||||
cannot occur (it is deliberately **not** a general YAML engine, whose own features
|
cannot occur (it is deliberately **not** a general YAML engine, whose own features
|
||||||
are the attack surface); **`resource` https-allowlist** (hard-rejects
|
are the attack surface). The one mapping form it accepts is OKF v0.2's flow
|
||||||
|
mapping — `generated: { by: x, at: y }`, `verified: { … }` bare or listed,
|
||||||
|
`usage_window: { from: …, to: … }` — admitted key-by-key against a nine-name
|
||||||
|
allowlist (`by`, `at`, `from`, `to`, `id`, `title`, `author`, `usage_count`,
|
||||||
|
`last_modified`) with plain-scalar leaves only. A key off that list, a nested
|
||||||
|
collection or a duplicate key is refused, and `resource` is deliberately not on
|
||||||
|
it; the block, dotted and inline-colon routes to a mapping still raise. See
|
||||||
|
[LIMITATIONS](docs/LIMITATIONS.md) for what that admits and what it still walls
|
||||||
|
off (a `sources` block list of mappings is still refused); **`resource` https-allowlist** (hard-rejects
|
||||||
`data:`/`javascript:`/`file:` before commit — a reject-gate, not defang);
|
`data:`/`javascript:`/`file:` before commit — a reject-gate, not defang);
|
||||||
**whole-concept scan** (frontmatter *values* + body through `scan_output`);
|
**whole-concept scan** (frontmatter *values* + body through `scan_output`);
|
||||||
**cross-link graph** (surfaces dangling targets, the dormant-injection signal, and
|
**cross-link graph** (surfaces dangling targets, the dormant-injection signal, and
|
||||||
|
|
@ -179,7 +187,7 @@ driven by a **live payload** in the coverage matrix — run it to watch all 134
|
||||||
in your own environment:
|
in your own environment:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python -m llm_ingestion_guard.coverage # 129/129 classes; exit 0 = all as documented
|
python -m llm_ingestion_guard.coverage # 130/130 classes; exit 0 = all as documented
|
||||||
```
|
```
|
||||||
|
|
||||||
| Anchor | Attack classes it stops (representative) |
|
| Anchor | Attack classes it stops (representative) |
|
||||||
|
|
|
||||||
|
|
@ -40,16 +40,27 @@ items; this is the full list, each with the mechanism.
|
||||||
(an injection in a directory listing is caught) rather than path-rejecting the
|
(an injection in a directory listing is caught) rather than path-rejecting the
|
||||||
conformant bundle. A front-end materialising individual uploads keeps the opposite
|
conformant bundle. A front-end materialising individual uploads keeps the opposite
|
||||||
rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused.
|
rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused.
|
||||||
- **OKF frontmatter is a restricted grammar: the mapping class has no expressible
|
- **OKF frontmatter is a restricted grammar: the mapping class has exactly one
|
||||||
form.** Gate T2 accepts a line-oriented subset deliberately — full YAML is a larger
|
expressible form.** Gate T2 accepts a line-oriented subset deliberately — full YAML
|
||||||
parse-attack surface than a write-time gate needs. Nested mappings and flow
|
is a larger parse-attack surface than a write-time gate needs. Flow sequences
|
||||||
collections (`[a, b]`, `{k: v}`) are *rejected outright*, which fails secure.
|
(`[a, b]`) and nested mappings are *rejected outright*, which fails secure.
|
||||||
**All four routes to a mapping fail, each on a different rule** — flow (`{k: v}`)
|
**Three of the four routes to a mapping fail, each on a different rule** — block
|
||||||
on the disallowed value-start indicator, block (`k:\n sub: v`) on the
|
(`k:\n sub: v`) on the nested-mapping check, dotted keys (`k.sub: v`) on the key
|
||||||
nested-mapping check, dotted keys (`k.sub: v`) on the key pattern, and the inline
|
pattern, and the inline second colon (`k: sub: v`) on the mapping-construct check.
|
||||||
second colon (`k: sub: v`) on the mapping-construct check — so the mapping *class*
|
**The fourth, the flow form, is admitted only when every key is on an allowlist**
|
||||||
has no expressible form, rather than one form being preferable to another. What
|
(`by`, `at`, `from`, `to`, `id`, `title`, `author`, `usage_count`,
|
||||||
survives is scalars and flat lists of strings. **Two of those routes used to
|
`last_modified` — the keys SPEC.md @ `62432a09` §5.1/§5.2 names inside a mapping)
|
||||||
|
and every leaf is a plain scalar, itself run through the same value predicates as a
|
||||||
|
top-level scalar. Nested collections, quoted leaves, duplicate keys, an empty or
|
||||||
|
unclosed mapping, and `{a:b}` (which PyYAML 6.0.3 reads as the *key* `a:b`, not as
|
||||||
|
a scalar) all raise. The form is expressible, never trusted: the allowlist
|
||||||
|
inspects every key, which is the property that carried the security when the
|
||||||
|
blanket refusal was doing the enforcing. **`resource` is deliberately off the
|
||||||
|
allowlist** although §5.1 names it inside a `sources` entry — it is a pointer
|
||||||
|
rather than a label and the only key T3 exists for, so admitting it would let
|
||||||
|
`executor: { resource: skills/run.md }` carry an executable-code pointer through a
|
||||||
|
key the https allowlist never inspects. What else survives is scalars and flat
|
||||||
|
lists of strings. **Two routes used to
|
||||||
degrade into a string instead of failing, and that defect is closed in `1.1.0`**:
|
degrade into a string instead of failing, and that defect is closed in `1.1.0`**:
|
||||||
a block-sequence item carrying exactly one key (`sources:\n - uri: https://e.com/a`
|
a block-sequence item carrying exactly one key (`sources:\n - uri: https://e.com/a`
|
||||||
yielded the *string* `'uri: https://e.com/a'`) and the inline second colon
|
yielded the *string* `'uri: https://e.com/a'`) and the inline second colon
|
||||||
|
|
@ -74,13 +85,19 @@ items; this is the full list, each with the mechanism.
|
||||||
door A/B persist path, so frontmatter that fails secure on import passes
|
door A/B persist path, so frontmatter that fails secure on import passes
|
||||||
`screen_output` unremarked. The grammar therefore bounds what a consumer can *receive*,
|
`screen_output` unremarked. The grammar therefore bounds what a consumer can *receive*,
|
||||||
never what a producer can *emit*. Verified identical on 0.2.0 and 0.3.1.
|
never what a producer can *emit*. Verified identical on 0.2.0 and 0.3.1.
|
||||||
- **Consequence: an OKF v0.2 concept cannot traverse the external-import path.** Both
|
- **An OKF v0.2 concept traverses the external-import path only if its `sources` are
|
||||||
of v0.2's backward-breaking migration targets are nested — `timestamp` → `generated.at`,
|
flat.** The wall used to be total: both of v0.2's backward-breaking migration targets
|
||||||
and body `# Citations` → a `sources` block list of mappings — so a conformant v0.2
|
are mappings — `timestamp` → `generated.at`, and body `# Citations` → a `sources`
|
||||||
concept fails secure at the frontmatter gate. This is the correct direction but it is
|
block list of mappings — and a consumer measured **0 of 53** upstream concepts
|
||||||
a compatibility wall, not a policy: v0.2 support requires a deliberate parse-safety
|
through the gate. The trust and provenance layer now passes in its spec form
|
||||||
decision about widening the grammar, and the dangling-or-substituted `executor`/
|
(`generated`, `verified` bare or listed, `usage_window`), so `generated.at` is no
|
||||||
`attester` pointer question only becomes live once that decision is made.
|
longer a wall. **`sources` still is**: SPEC.md writes each entry as a block mapping
|
||||||
|
under a block sequence (`- id: …\n resource: …`), and that carrier stays refused —
|
||||||
|
it is the shape whose one-key degradation smuggled a pointer before `1.1.0`, and
|
||||||
|
reopening it is a separate parse-safety decision, not a corollary of the flow form.
|
||||||
|
A concept whose `sources` are flat strings, or absent, imports. The
|
||||||
|
dangling-or-substituted `executor`/`attester` pointer question stays out of reach
|
||||||
|
for the same reason: both are mappings whose payload key is `resource`.
|
||||||
- **A persist gate cannot cover execution risk.** OKF v0.2 introduces concepts whose
|
- **A persist gate cannot cover execution risk.** OKF v0.2 introduces concepts whose
|
||||||
purpose is to *name code to be run* (`runtime`, `executor.resource`,
|
purpose is to *name code to be run* (`runtime`, `executor.resource`,
|
||||||
`attester.resource`). This library answers "is this safe to **store**"; executable
|
`attester.resource`). This library answers "is this safe to **store**"; executable
|
||||||
|
|
|
||||||
|
|
@ -542,8 +542,11 @@ def _build_cases() -> list[Case]:
|
||||||
lambda: okf.parse_frontmatter("---\nkey:\n nested: x\n---\nbody\n"), owasp="LLM10"),
|
lambda: okf.parse_frontmatter("---\nkey:\n nested: x\n---\nbody\n"), owasp="LLM10"),
|
||||||
_raise_case("okf", "T2 frontmatter block scalar", "OKFFrontmatterError",
|
_raise_case("okf", "T2 frontmatter block scalar", "OKFFrontmatterError",
|
||||||
lambda: okf.parse_frontmatter("---\ndesc: |\n block\n---\nbody\n"), owasp="LLM10"),
|
lambda: okf.parse_frontmatter("---\ndesc: |\n block\n---\nbody\n"), owasp="LLM10"),
|
||||||
_raise_case("okf", "T2 frontmatter flow collection", "OKFFrontmatterError",
|
_raise_case("okf", "T2 frontmatter flow sequence", "OKFFrontmatterError",
|
||||||
lambda: okf.parse_frontmatter("---\ntags: [a, b]\n---\nbody\n"), owasp="LLM10"),
|
lambda: okf.parse_frontmatter("---\ntags: [a, b]\n---\nbody\n"), owasp="LLM10"),
|
||||||
|
_raise_case("okf", "T2 mapping key off the allowlist", "OKFFrontmatterError",
|
||||||
|
lambda: okf.parse_frontmatter(
|
||||||
|
"---\ngenerated: { by: a, tool: shell }\n---\nbody\n"), owasp="LLM10"),
|
||||||
_raise_case("okf", "T3 resource non-https (http)", "OKFResourceError",
|
_raise_case("okf", "T3 resource non-https (http)", "OKFResourceError",
|
||||||
lambda: okf.validate_resource_url("http://insecure.test/x"), owasp="LLM05"),
|
lambda: okf.validate_resource_url("http://insecure.test/x"), owasp="LLM05"),
|
||||||
_raise_case("okf", "T3 resource data: scheme", "OKFResourceError",
|
_raise_case("okf", "T3 resource data: scheme", "OKFResourceError",
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@ and feeds scannable text regions into the existing ``sanitize`` / ``scan_output`
|
||||||
|
|
||||||
T2 — frontmatter parse-safety gate. ``parse_frontmatter`` is a *strict,
|
T2 — frontmatter parse-safety gate. ``parse_frontmatter`` is a *strict,
|
||||||
reject-by-default* loader for the minimal OKF frontmatter subset: flat
|
reject-by-default* loader for the minimal OKF frontmatter subset: flat
|
||||||
``key: value`` scalars plus block ``- item`` lists. Every construct the
|
``key: value`` scalars, block ``- item`` lists, and one typed, allowlisted
|
||||||
|
mapping form (``{ by: x, at: y }`` — see :func:`_parse_flow_mapping`). Every construct the
|
||||||
"block anchor/alias DoS + dangerous type coercion" requirement names is refused
|
"block anchor/alias DoS + dangerous type coercion" requirement names is refused
|
||||||
*by construction* — you cannot suffer a billion-laughs alias expansion or a
|
*by construction* — you cannot suffer a billion-laughs alias expansion or a
|
||||||
``!!python/object`` coercion if anchors, aliases and explicit tags are rejected
|
``!!python/object`` coercion if anchors, aliases and explicit tags are rejected
|
||||||
|
|
@ -16,7 +17,9 @@ philosophy, the frontmatter analogue of the ``resource`` reject-gate (T3).
|
||||||
|
|
||||||
Deliberately NOT a general YAML parser. A security tool whose thesis is
|
Deliberately NOT a general YAML parser. A security tool whose thesis is
|
||||||
minimal-dependency should not pull in a full YAML engine whose own features
|
minimal-dependency should not pull in a full YAML engine whose own features
|
||||||
(anchors, tags, merges) are the attack surface being defended against. Quoted
|
(anchors, tags, merges) are the attack surface being defended against. The one
|
||||||
|
mapping form it does admit is admitted key-by-key against an allowlist, not
|
||||||
|
parsed generally: the mapping class is expressible, never trusted. Quoted
|
||||||
scalars are kept verbatim (quotes included) rather than unquoted — the value is
|
scalars are kept verbatim (quotes included) rather than unquoted — the value is
|
||||||
still scanned as text downstream, so an injection inside a quoted value is not
|
still scanned as text downstream, so an injection inside a quoted value is not
|
||||||
lost; richer scalar forms are a future refinement, not a silent parse.
|
lost; richer scalar forms are a future refinement, not a silent parse.
|
||||||
|
|
@ -62,13 +65,32 @@ _KEY_RE = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9_-]*$")
|
||||||
# A plain OKF scalar cannot *begin* with a YAML structural indicator. Any value
|
# A plain OKF scalar cannot *begin* with a YAML structural indicator. Any value
|
||||||
# starting with one signals an anchor (&), alias (*), explicit tag (!), block
|
# starting with one signals an anchor (&), alias (*), explicit tag (!), block
|
||||||
# scalar (|, >), flow collection ([ ] { }), directive (%) or reserved char
|
# scalar (|, >), flow collection ([ ] { }), directive (%) or reserved char
|
||||||
# (@ `) — all outside the supported subset and all rejected.
|
# (@ `) — all outside the supported subset and all rejected. `{` is tried as the
|
||||||
|
# allowlisted mapping form FIRST (G3); it reaches this predicate only as a leaf
|
||||||
|
# inside one, where a nested collection is refused before it can be read.
|
||||||
_DANGEROUS_VALUE_STARTS = frozenset("&*!|>[]{}%@`")
|
_DANGEROUS_VALUE_STARTS = frozenset("&*!|>[]{}%@`")
|
||||||
# A quoted scalar is a scalar in YAML however many colons it carries, so the
|
# A quoted scalar is a scalar in YAML however many colons it carries, so the
|
||||||
# mapping check steps aside for one. The quotes are retained rather than
|
# mapping check steps aside for one. The quotes are retained rather than
|
||||||
# stripped — a pre-existing divergence, pinned in tests/test_okf.py.
|
# stripped — a pre-existing divergence, pinned in tests/test_okf.py.
|
||||||
_QUOTE_STARTS = frozenset("\"'")
|
_QUOTE_STARTS = frozenset("\"'")
|
||||||
|
|
||||||
|
# G3 — the one mapping form T2 can express (operator decision, 2026-08-21).
|
||||||
|
# Every key inside a mapping must be on this allowlist: the form is safe because
|
||||||
|
# the allowlist inspects each key, not because mappings became trusted. The keys
|
||||||
|
# are the ones OKF v0.2 names inside a mapping - `by`/`at` (SPEC.md @ 62432a09
|
||||||
|
# §5.2 `generated`/`verified`), `from`/`to` (§5.1 `usage_window`) and the
|
||||||
|
# `sources`-entry fields (§5.1). `resource` is the one §5.1 key deliberately
|
||||||
|
# LEFT OFF: it is a pointer rather than a label, it is the only key T3 exists
|
||||||
|
# for, and admitting it inside a mapping would re-open the door-C route closed
|
||||||
|
# in 1.1.0 (`executor: {resource: skills/run.md}` puts an executable-code
|
||||||
|
# pointer in a key the https allowlist never inspects). It costs nothing today,
|
||||||
|
# because the conformant carrier for `sources[].resource` is the block-sequence
|
||||||
|
# of block-mappings, which this form does not admit either way.
|
||||||
|
_MAPPING_KEY_ALLOWLIST = frozenset({
|
||||||
|
"by", "at", "from", "to", "id", "title", "author", "usage_count",
|
||||||
|
"last_modified",
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
class OKFError(Exception):
|
class OKFError(Exception):
|
||||||
"""Base class for OKF adapter rejections."""
|
"""Base class for OKF adapter rejections."""
|
||||||
|
|
@ -106,7 +128,10 @@ def parse_frontmatter(document):
|
||||||
|
|
||||||
Raises ``OKFFrontmatterError`` on an unterminated fence or any construct
|
Raises ``OKFFrontmatterError`` on an unterminated fence or any construct
|
||||||
outside the minimal flat subset (anchors, aliases, explicit tags, merge
|
outside the minimal flat subset (anchors, aliases, explicit tags, merge
|
||||||
keys, block scalars, flow collections, nested mappings).
|
keys, block scalars, flow sequences, nested mappings). The single exception
|
||||||
|
is the typed, allowlisted flow mapping (:func:`_parse_flow_mapping`), which
|
||||||
|
parses into a ``dict`` of allowlisted keys with plain-scalar leaves — every
|
||||||
|
other route to a mapping still raises.
|
||||||
"""
|
"""
|
||||||
lines = document.split("\n")
|
lines = document.split("\n")
|
||||||
if not lines or lines[0].strip() != _FENCE:
|
if not lines or lines[0].strip() != _FENCE:
|
||||||
|
|
@ -149,13 +174,29 @@ def _scannable_regions(frontmatter, body):
|
||||||
"""The text regions of a concept that carry attacker-controlled content."""
|
"""The text regions of a concept that carry attacker-controlled content."""
|
||||||
regions = [body]
|
regions = [body]
|
||||||
for value in frontmatter.values():
|
for value in frontmatter.values():
|
||||||
if isinstance(value, list):
|
regions.extend(_value_regions(value))
|
||||||
regions.extend(value)
|
|
||||||
elif value:
|
|
||||||
regions.append(value)
|
|
||||||
return regions
|
return regions
|
||||||
|
|
||||||
|
|
||||||
|
def _value_regions(value):
|
||||||
|
"""Every scannable leaf of one frontmatter value.
|
||||||
|
|
||||||
|
A mapping value (G3) is a new *shape* on this surface, not a new exemption:
|
||||||
|
its leaves are scanned exactly like a scalar or a list item, so an injection
|
||||||
|
parked in ``generated: { by: ... }`` reaches ``scan_output`` like any other
|
||||||
|
frontmatter text. Mapping *keys* are not scanned because they cannot carry
|
||||||
|
attacker text - the allowlist admits nine fixed names and nothing else.
|
||||||
|
"""
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return [leaf for leaf in value.values() if leaf]
|
||||||
|
if isinstance(value, list):
|
||||||
|
regions = []
|
||||||
|
for item in value:
|
||||||
|
regions.extend(_value_regions(item))
|
||||||
|
return regions
|
||||||
|
return [value] if value else []
|
||||||
|
|
||||||
|
|
||||||
def validate_concept_path(path, *, allow_reserved=False):
|
def validate_concept_path(path, *, allow_reserved=False):
|
||||||
"""Validate a bundle-relative concept path and return its concept-ID.
|
"""Validate a bundle-relative concept path and return its concept-ID.
|
||||||
|
|
||||||
|
|
@ -579,6 +620,12 @@ def _parse_flat(fm_lines):
|
||||||
result[key] = items if items is not None else ""
|
result[key] = items if items is not None else ""
|
||||||
continue
|
continue
|
||||||
|
|
||||||
|
mapping = _parse_flow_mapping(value)
|
||||||
|
if mapping is not None:
|
||||||
|
result[key] = mapping
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
|
||||||
_reject_dangerous_value(value)
|
_reject_dangerous_value(value)
|
||||||
_reject_mapping_construct(value)
|
_reject_mapping_construct(value)
|
||||||
result[key] = value
|
result[key] = value
|
||||||
|
|
@ -605,6 +652,11 @@ def _consume_block_list(fm_lines, start):
|
||||||
continue
|
continue
|
||||||
if raw[:1] in (" ", "\t") and stripped.startswith("- "):
|
if raw[:1] in (" ", "\t") and stripped.startswith("- "):
|
||||||
item = stripped[2:].strip()
|
item = stripped[2:].strip()
|
||||||
|
mapping = _parse_flow_mapping(item)
|
||||||
|
if mapping is not None:
|
||||||
|
items.append(mapping)
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
_reject_dangerous_value(item)
|
_reject_dangerous_value(item)
|
||||||
_reject_mapping_construct(item)
|
_reject_mapping_construct(item)
|
||||||
items.append(item)
|
items.append(item)
|
||||||
|
|
@ -627,8 +679,10 @@ def _reject_dangerous_value(value):
|
||||||
def _reject_mapping_construct(value):
|
def _reject_mapping_construct(value):
|
||||||
"""Reject a scalar that YAML reads as a mapping rather than as a string.
|
"""Reject a scalar that YAML reads as a mapping rather than as a string.
|
||||||
|
|
||||||
T2 gives the mapping *class* no expressible form — flow, nested-block and
|
T2 gives the mapping *class* exactly one expressible form, the typed
|
||||||
dotted-key routes all raise. Two routes used to escape that by degrading
|
allowlisted flow mapping (G3); the nested-block and dotted-key routes still
|
||||||
|
raise, and this predicate is what keeps them raising — both at the top level
|
||||||
|
and on a leaf *inside* an admitted mapping. Two routes used to escape by degrading
|
||||||
into a string instead: a block-sequence item carrying exactly one key
|
into a string instead: a block-sequence item carrying exactly one key
|
||||||
(``- uri: x``), and an inline second colon (``attester: resource: x``).
|
(``- uri: x``), and an inline second colon (``attester: resource: x``).
|
||||||
Both parsed "successfully" into the wrong *type*, and a pointer parked in
|
Both parsed "successfully" into the wrong *type*, and a pointer parked in
|
||||||
|
|
@ -648,3 +702,99 @@ def _reject_mapping_construct(value):
|
||||||
raise OKFFrontmatterError(
|
raise OKFFrontmatterError(
|
||||||
"a mapping is not expressible in OKF frontmatter: %r" % (value,)
|
"a mapping is not expressible in OKF frontmatter: %r" % (value,)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_flow_mapping(value):
|
||||||
|
"""Parse ``{ key: value, ... }`` into a typed dict, or refuse it (G3).
|
||||||
|
|
||||||
|
Returns ``None`` when ``value`` does not open a flow mapping, so the caller
|
||||||
|
falls through to the unchanged scalar rules. Otherwise the value either
|
||||||
|
parses into a ``dict`` of allowlisted keys with plain-scalar leaves, or
|
||||||
|
raises - it never degrades into a string, which is the defect closed in
|
||||||
|
1.1.0 and not reopened here.
|
||||||
|
|
||||||
|
Why the mapping class needed *a* form at all: OKF v0.2 writes its whole
|
||||||
|
trust and provenance layer as mappings, and SPEC.md @ ``62432a09`` uses flow
|
||||||
|
form in its own examples (§5.1 ``usage_window``, §5.2 ``generated`` /
|
||||||
|
``verified``). §11 goes further than "should": a consumer *MUST* treat a
|
||||||
|
bare ``verified`` mapping as a one-element list - a rule that presupposes
|
||||||
|
the mapping parses. With no form, 0 of 53 upstream concepts reached the
|
||||||
|
gate, and no threshold would have changed that.
|
||||||
|
|
||||||
|
Why this form is safe: the allowlist inspects **every key**, which is the
|
||||||
|
property that actually carried the security in T2 - the blanket refusal was
|
||||||
|
the enforcement, not the point. Admitted, ground-truthed against PyYAML
|
||||||
|
6.0.3:
|
||||||
|
|
||||||
|
- one flow mapping per value, closed on the same line (``{ a: b }``);
|
||||||
|
- keys on :data:`_MAPPING_KEY_ALLOWLIST` and matching ``_KEY_RE``, no
|
||||||
|
duplicates - PyYAML resolves a duplicate last-wins, which is a way to
|
||||||
|
show one claim and mean another;
|
||||||
|
- plain-scalar leaves only, each run through the *unchanged*
|
||||||
|
``_reject_dangerous_value`` / ``_reject_mapping_construct`` predicates, so
|
||||||
|
a leaf can no more open an anchor, a tag or a nested mapping than a
|
||||||
|
top-level scalar can.
|
||||||
|
|
||||||
|
Refused, each on its own rule: nested collections (``{ a: { b: c } }``,
|
||||||
|
``{ a: [1] }``), quoted leaves, an empty mapping, an unclosed or
|
||||||
|
trailing-junk value (``{ a: b } x``, which PyYAML also refuses), a key
|
||||||
|
outside the allowlist, and ``{a:b}`` - which PyYAML reads as the *key*
|
||||||
|
``a:b``, not as a scalar, and which the required ``": "`` separator catches.
|
||||||
|
|
||||||
|
Two deliberate divergences from PyYAML, both toward refusal: a quoted leaf
|
||||||
|
(``{ title: 'a, b' }``) and a trailing comment (``{ a: b } # note``) are
|
||||||
|
conformant YAML that this rejects. Splitting quoted commas correctly needs a
|
||||||
|
quote state machine whose failure mode is *accepting* something YAML would
|
||||||
|
refuse; refusing is the cheaper side to be wrong on, and the keys that
|
||||||
|
plausibly need a comma (``title``, ``author``) only occur inside ``sources``
|
||||||
|
entries, whose block-sequence carrier is refused anyway.
|
||||||
|
"""
|
||||||
|
if not value or value[0] != "{":
|
||||||
|
return None
|
||||||
|
if not value.endswith("}"):
|
||||||
|
raise OKFFrontmatterError(
|
||||||
|
"a flow mapping must be closed by '}' on the same line: %r" % (value,)
|
||||||
|
)
|
||||||
|
|
||||||
|
inner = value[1:-1].strip()
|
||||||
|
if inner.endswith(","): # a trailing comma is legal YAML; one, and only one
|
||||||
|
inner = inner[:-1].strip()
|
||||||
|
if not inner:
|
||||||
|
raise OKFFrontmatterError("an empty flow mapping carries nothing: %r" % (value,))
|
||||||
|
for char in "{}[]":
|
||||||
|
if char in inner:
|
||||||
|
raise OKFFrontmatterError(
|
||||||
|
"a flow mapping admits scalar leaves only, not %r: %r" % (char, value)
|
||||||
|
)
|
||||||
|
for quote in _QUOTE_STARTS:
|
||||||
|
if quote in inner:
|
||||||
|
raise OKFFrontmatterError(
|
||||||
|
"a quoted scalar inside a flow mapping is not a supported form: %r"
|
||||||
|
% (value,)
|
||||||
|
)
|
||||||
|
|
||||||
|
mapping = {}
|
||||||
|
for entry in inner.split(","):
|
||||||
|
entry = entry.strip()
|
||||||
|
key, sep, leaf = entry.partition(": ")
|
||||||
|
if not sep:
|
||||||
|
raise OKFFrontmatterError(
|
||||||
|
"a flow-mapping entry must be 'key: value': %r" % (entry,)
|
||||||
|
)
|
||||||
|
key = key.strip()
|
||||||
|
leaf = leaf.strip()
|
||||||
|
if not _KEY_RE.match(key):
|
||||||
|
raise OKFFrontmatterError("invalid flow-mapping key: %r" % (key,))
|
||||||
|
if key not in _MAPPING_KEY_ALLOWLIST:
|
||||||
|
raise OKFFrontmatterError(
|
||||||
|
"flow-mapping key %r is not on the OKF mapping allowlist: %r"
|
||||||
|
% (key, value)
|
||||||
|
)
|
||||||
|
if key in mapping:
|
||||||
|
raise OKFFrontmatterError(
|
||||||
|
"duplicate flow-mapping key %r: %r" % (key, value)
|
||||||
|
)
|
||||||
|
_reject_dangerous_value(leaf)
|
||||||
|
_reject_mapping_construct(leaf)
|
||||||
|
mapping[key] = leaf
|
||||||
|
return mapping
|
||||||
|
|
|
||||||
|
|
@ -596,9 +596,11 @@ _DEGRADED_TO_STRING = [
|
||||||
@pytest.mark.parametrize("cid,fm,yaml_reads_as", _DEGRADED_TO_STRING,
|
@pytest.mark.parametrize("cid,fm,yaml_reads_as", _DEGRADED_TO_STRING,
|
||||||
ids=[c[0] for c in _DEGRADED_TO_STRING])
|
ids=[c[0] for c in _DEGRADED_TO_STRING])
|
||||||
def test_a_mapping_construct_never_degrades_into_a_string(cid, fm, yaml_reads_as):
|
def test_a_mapping_construct_never_degrades_into_a_string(cid, fm, yaml_reads_as):
|
||||||
# The mapping *class* has no expressible form through T2 — so a mapping
|
# None of these shapes is the one form T2 admits (G3, the allowlisted flow
|
||||||
# construct must RAISE, never parse "successfully" into the wrong type. A
|
# mapping) — so each must RAISE, never parse "successfully" into the wrong
|
||||||
# consumer reading frontmatter["sources"][0].get("uri") must not be handed a str.
|
# type. A consumer reading frontmatter["sources"][0].get("uri") must not be
|
||||||
|
# handed a str, and that holds whether the mapping class has no expressible
|
||||||
|
# form or one.
|
||||||
with pytest.raises(OKFFrontmatterError):
|
with pytest.raises(OKFFrontmatterError):
|
||||||
parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")
|
parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")
|
||||||
|
|
||||||
|
|
@ -651,13 +653,17 @@ def test_pointer_in_a_degraded_mapping_no_longer_reaches_the_consumer_tree(cid,
|
||||||
assert result.disposition is Disposition.FAIL_SECURE, "hole reopened — see LIMITATIONS.md"
|
assert result.disposition is Disposition.FAIL_SECURE, "hole reopened — see LIMITATIONS.md"
|
||||||
|
|
||||||
|
|
||||||
def test_every_route_to_a_mapping_fails_on_a_different_rule():
|
def test_exactly_one_route_to_a_mapping_is_expressible():
|
||||||
# The v0.2 wall is not a choice between two forms where one is better: ALL three
|
# Was: ALL FOUR routes failed, each on its own rule, so the mapping *class* had
|
||||||
# ways to express a mapping fail, each on its own rule, so the mapping *class* has
|
# no expressible form (and v0.2's `generated` could not be written at all). G3
|
||||||
# no expressible form through T2. v0.2's `generated` IS a mapping (`by` required
|
# opens exactly ONE of them - the allowlisted flow form - and the other three
|
||||||
# when present), so it cannot be expressed at all.
|
# still fail, each on its own rule. That the openable route is the one whose
|
||||||
|
# every key the allowlist inspects is the whole design: block, dotted and inline
|
||||||
|
# give the allowlist nothing to inspect, so they stay shut.
|
||||||
|
assert parse_frontmatter("---\nid: x\ngenerated: { by: x, at: y }\n---\n\nbody\n")[0][
|
||||||
|
"generated"] == {"by": "x", "at": "y"}
|
||||||
|
|
||||||
routes = {
|
routes = {
|
||||||
"flow": "generated: { by: x, at: y }\n",
|
|
||||||
"block": "generated:\n by: x\n",
|
"block": "generated:\n by: x\n",
|
||||||
"dotted": "generated.by: x\n",
|
"dotted": "generated.by: x\n",
|
||||||
"inline": "generated: by: x\n",
|
"inline": "generated: by: x\n",
|
||||||
|
|
@ -667,11 +673,10 @@ def test_every_route_to_a_mapping_fails_on_a_different_rule():
|
||||||
with pytest.raises(OKFFrontmatterError) as exc:
|
with pytest.raises(OKFFrontmatterError) as exc:
|
||||||
parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")
|
parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")
|
||||||
errors[name] = str(exc.value)
|
errors[name] = str(exc.value)
|
||||||
assert "indicator" in errors["flow"]
|
|
||||||
assert "nested mappings" in errors["block"]
|
assert "nested mappings" in errors["block"]
|
||||||
assert "key" in errors["dotted"]
|
assert "key" in errors["dotted"]
|
||||||
assert "mapping" in errors["inline"]
|
assert "mapping" in errors["inline"]
|
||||||
assert len(set(errors.values())) == 4, "routes must fail distinctly, not collapse"
|
assert len(set(errors.values())) == 3, "routes must fail distinctly, not collapse"
|
||||||
|
|
||||||
|
|
||||||
_BLOCK_LIST_ITEM_SHAPES = [
|
_BLOCK_LIST_ITEM_SHAPES = [
|
||||||
|
|
@ -698,7 +703,9 @@ def test_two_keys_per_item_is_where_the_block_list_hard_rejects():
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("fm", [
|
@pytest.mark.parametrize("fm", [
|
||||||
"generated: { by: x, at: y }\n", "sources: [{ id: a }]\n", "tags: [a, b]\n",
|
# The flow row carries a key OFF the G3 allowlist: the shape is admitted, the
|
||||||
|
# key is not, so this stays a T2 rejection and the door A/B half still holds.
|
||||||
|
"generated: { by: x, tool: y }\n", "sources: [{ id: a }]\n", "tags: [a, b]\n",
|
||||||
"generated:\n by: x\n", "generated.by: x\n",
|
"generated:\n by: x\n", "generated.by: x\n",
|
||||||
"sources:\n - id: a\n resource: file://x\n",
|
"sources:\n - id: a\n resource: file://x\n",
|
||||||
])
|
])
|
||||||
|
|
@ -738,3 +745,161 @@ def test_link_extraction_survives_the_redos_fix():
|
||||||
assert extract_link_targets("see [x](./a.md) and [y](/b.md)") == ["./a.md", "/b.md"]
|
assert extract_link_targets("see [x](./a.md) and [y](/b.md)") == ["./a.md", "/b.md"]
|
||||||
assert extract_link_targets("[a b](./c.md)") == ["./c.md"]
|
assert extract_link_targets("[a b](./c.md)") == ["./c.md"]
|
||||||
assert extract_link_targets("text [](./t.md)") == ["./i.png"]
|
assert extract_link_targets("text [](./t.md)") == ["./i.png"]
|
||||||
|
|
||||||
|
|
||||||
|
# --- G3: the typed, allowlisted mapping form (2026-08-21) --------------------
|
||||||
|
# Door 1 of three (operator decision, 2026-08-21). The mapping *class* had no
|
||||||
|
# expressible form, and OKF v0.2 writes its whole trust and provenance layer as
|
||||||
|
# mappings — SPEC.md @ 62432a09 §5.2 uses flow form in its own examples, and §11
|
||||||
|
# carries a hard MUST that presupposes they parse ("consumers MUST treat a bare
|
||||||
|
# `verified` mapping as a one-element list"). A consumer measured 0 of 53
|
||||||
|
# upstream concepts through the gate. This admits ONE shape: a flow mapping whose
|
||||||
|
# every key is on the allowlist and whose every leaf is a plain scalar.
|
||||||
|
|
||||||
|
def test_spec_flow_mapping_parses_into_a_typed_mapping():
|
||||||
|
# SPEC.md §5.2, verbatim. This is the red test: it must fail before the form
|
||||||
|
# exists and pass after, with a real dict — never a degraded string.
|
||||||
|
doc = (
|
||||||
|
"---\ntype: table\n"
|
||||||
|
"generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z }\n"
|
||||||
|
"---\nbody\n"
|
||||||
|
)
|
||||||
|
assert parse_frontmatter(doc)[0]["generated"] == {
|
||||||
|
"by": "reference_agent/gemini-2.5-pro",
|
||||||
|
"at": "2026-06-20T22:53:05Z",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_spec_bare_verified_mapping_parses():
|
||||||
|
# SPEC.md §5.2's bare form, which §11 turns into a hard MUST for consumers
|
||||||
|
# ("MUST treat a bare `verified` mapping as a one-element list") - a rule that
|
||||||
|
# cannot be obeyed by a consumer that cannot parse the mapping.
|
||||||
|
doc = "---\ntype: table\nverified: { by: human:ahormati, at: 2026-06-25T09:00:00Z }\n---\nb\n"
|
||||||
|
assert parse_frontmatter(doc)[0]["verified"] == {
|
||||||
|
"by": "human:ahormati", "at": "2026-06-25T09:00:00Z"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_spec_verified_list_of_flow_mappings_parses():
|
||||||
|
# §5.2's list form. This is the SAME typed form in list position, not the
|
||||||
|
# block-sequence-with-one-key route (`- uri: x`), which stays shut below.
|
||||||
|
doc = (
|
||||||
|
"---\ntype: table\nverified:\n"
|
||||||
|
" - { by: human:ahormati, at: 2026-06-25T09:00:00Z }\n"
|
||||||
|
" - { by: process:finance-nightly, at: 2026-06-26T02:00:00Z }\n"
|
||||||
|
"---\nbody\n"
|
||||||
|
)
|
||||||
|
assert parse_frontmatter(doc)[0]["verified"] == [
|
||||||
|
{"by": "human:ahormati", "at": "2026-06-25T09:00:00Z"},
|
||||||
|
{"by": "process:finance-nightly", "at": "2026-06-26T02:00:00Z"},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_spec_usage_window_parses():
|
||||||
|
doc = "---\ntype: table\nusage_window: { from: 2026-06-01T00:00:00Z, to: 2026-06-30T00:00:00Z }\n---\nb\n"
|
||||||
|
assert parse_frontmatter(doc)[0]["usage_window"] == {
|
||||||
|
"from": "2026-06-01T00:00:00Z", "to": "2026-06-30T00:00:00Z"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_an_unknown_key_inside_a_mapping_is_still_rejected():
|
||||||
|
# The rejection side of the allowlist. Without this test the allowlist could
|
||||||
|
# silently grow to "anything" - or be emptied - and nothing would fail.
|
||||||
|
with pytest.raises(OKFFrontmatterError) as exc:
|
||||||
|
parse_frontmatter("---\nid: x\ngenerated: { by: a, tool: shell }\n---\n\nbody\n")
|
||||||
|
assert "allowlist" in str(exc.value)
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_allowlist_is_not_empty_and_admits_only_the_spec_keys():
|
||||||
|
# Both directions of the same guard: a shrunk allowlist breaks the first
|
||||||
|
# assertion, a widened one the second.
|
||||||
|
for key in ("by", "at", "from", "to", "id", "title", "author", "usage_count",
|
||||||
|
"last_modified"):
|
||||||
|
assert parse_frontmatter(f"---\nid: x\nk: {{ {key}: v }}\n---\n\nb\n")[0]["k"] == {key: "v"}
|
||||||
|
for key in ("resource", "executor", "attester", "runtime", "command", "uri"):
|
||||||
|
with pytest.raises(OKFFrontmatterError):
|
||||||
|
parse_frontmatter(f"---\nid: x\nk: {{ {key}: v }}\n---\n\nb\n")
|
||||||
|
|
||||||
|
|
||||||
|
_FLOW_REJECTED = [
|
||||||
|
# (id, value, what PyYAML 6.0.3 makes of it)
|
||||||
|
("nested mapping", "{ by: { at: x } }", "a nested mapping"),
|
||||||
|
("nested sequence", "{ by: [a, b] }", "a sequence leaf"),
|
||||||
|
("anchor leaf", "{ by: &a x }", "an anchor definition, silently"),
|
||||||
|
("tag leaf", "{ by: !!python/object:os.system x }", "refused outright"),
|
||||||
|
("block scalar leaf", "{ by: | }", "a scanner error"),
|
||||||
|
("nested colon leaf", "{ by: sub: v }", "refused outright"),
|
||||||
|
("no space after colon", "{by:x}", "the KEY 'by:x', not a scalar"),
|
||||||
|
("quoted leaf", "{ title: 'a, b' }", "a scalar - we refuse, deliberately"),
|
||||||
|
("empty mapping", "{}", "an empty mapping"),
|
||||||
|
("empty leaf", "{ by: }", "None"),
|
||||||
|
("unclosed", "{ by: x", "a parse error"),
|
||||||
|
("trailing junk", "{ by: x } more", "a parse error"),
|
||||||
|
("duplicate key", "{ by: a, by: b }", "last-wins, silently"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("cid,value,yaml_reads_as", _FLOW_REJECTED,
|
||||||
|
ids=[c[0] for c in _FLOW_REJECTED])
|
||||||
|
def test_the_mapping_form_admits_scalar_leaves_on_allowlisted_keys_only(cid, value, yaml_reads_as):
|
||||||
|
with pytest.raises(OKFFrontmatterError):
|
||||||
|
parse_frontmatter(f"---\nid: x\ngenerated: {value}\n---\n\nbody\n")
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_rejected_mapping_never_degrades_into_a_string():
|
||||||
|
# The 1.1.0 defect, re-asserted against the NEW form: a refused mapping must
|
||||||
|
# raise, not arrive as a str a consumer will .get() a key out of.
|
||||||
|
for value in ("{ by: { at: x } }", "{ tool: shell }", "{ by: x"):
|
||||||
|
with pytest.raises(OKFFrontmatterError):
|
||||||
|
parse_frontmatter(f"---\nid: x\ngenerated: {value}\n---\n\nbody\n")
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_admitted_mapping_is_a_dict_not_a_string():
|
||||||
|
value = parse_frontmatter("---\nid: x\ngenerated: { by: a, at: b }\n---\n\nb\n")[0]["generated"]
|
||||||
|
assert isinstance(value, dict), "a typed form that arrives as a str is the 1.1.0 defect"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("cid,fm", [
|
||||||
|
("block sequence, one key", "attester:\n - resource: attesters/sql_equality.py\n"),
|
||||||
|
("inline second colon", "attester: resource: attesters/sql_equality.py\n"),
|
||||||
|
("block mapping", "attester:\n resource: attesters/sql_equality.py\n"),
|
||||||
|
("flow mapping, pointer key", "attester: { resource: attesters/sql_equality.py }\n"),
|
||||||
|
])
|
||||||
|
def test_the_pointer_routes_stay_shut(cid, fm):
|
||||||
|
# G3 is additive: none of the routes that put an executable-code pointer in a
|
||||||
|
# key the https allowlist never inspects is reopened. The fourth row is why
|
||||||
|
# `resource` is off the allowlist - the form would otherwise have carried the
|
||||||
|
# door-C pointer through in typed clothes instead of degraded ones.
|
||||||
|
doc = f"---\nid: x\ntype: Attested Computation\n{fm}---\n\nbody\n"
|
||||||
|
with pytest.raises(OKFFrontmatterError):
|
||||||
|
parse_frontmatter(doc)
|
||||||
|
assert import_bundle({"computations/x.md": doc}).disposition is Disposition.FAIL_SECURE
|
||||||
|
|
||||||
|
|
||||||
|
def test_injection_in_a_mapping_leaf_is_caught_by_the_scan():
|
||||||
|
# T1 is not weakened by the new shape: a mapping leaf is scanned exactly like a
|
||||||
|
# scalar value or a list item. A typed form that parses but is not scanned would
|
||||||
|
# be a hole, not a fix.
|
||||||
|
doc = f"---\ntype: table\ngenerated: {{ by: {_INJECTION} }}\n---\nclean body\n"
|
||||||
|
assert scan_concept(doc).found is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_injection_in_a_listed_mapping_leaf_is_caught_by_the_scan():
|
||||||
|
doc = f"---\ntype: table\nverified:\n - {{ by: {_INJECTION} }}\n---\nclean body\n"
|
||||||
|
assert scan_concept(doc).found is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_conformant_v02_trust_layer_now_reaches_the_gate():
|
||||||
|
# The measured consequence: a consumer reported 0 of 53 upstream concepts through
|
||||||
|
# the gate, because every one of them carries §5.2 trust frontmatter.
|
||||||
|
doc = (
|
||||||
|
"---\n"
|
||||||
|
"type: table\n"
|
||||||
|
"title: Users\n"
|
||||||
|
"resource: https://example.com/users\n"
|
||||||
|
"generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z }\n"
|
||||||
|
"verified: { by: human:ahormati, at: 2026-06-25T09:00:00Z }\n"
|
||||||
|
"usage_window: { from: 2026-06-01T00:00:00Z, to: 2026-06-30T00:00:00Z }\n"
|
||||||
|
"---\nThe users table.\n"
|
||||||
|
)
|
||||||
|
result = import_bundle({"tables/users.md": doc})
|
||||||
|
assert result.disposition is Disposition.WARN
|
||||||
|
assert result.concepts[0].error is None
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue