Compare commits
No commits in common. "main" and "v1.1.0" have entirely different histories.
15 changed files with 94 additions and 1281 deletions
126
CHANGELOG.md
126
CHANGELOG.md
|
|
@ -5,119 +5,9 @@ All notable changes to this project will be documented in this file.
|
||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
## [1.3.0] — 2026-09-02
|
## [Unreleased]
|
||||||
|
|
||||||
### Added — the `sources` provenance family becomes expressible in both spec carriers
|
Nothing yet.
|
||||||
|
|
||||||
`okf.parse_frontmatter` rejected `sources` in every form the OKF spec and its
|
|
||||||
producers actually use. Measured 2026-09-02 by two consumers independently:
|
|
||||||
`sources: [{ id: a, resource: x }]` raised on the `[` indicator (with one entry
|
|
||||||
as with two), and the block sequence of block mappings — SPEC.md §5.1's *own*
|
|
||||||
example — raised `"nested mappings are not supported"`. `resource` is REQUIRED
|
|
||||||
within a `sources` entry (§5.1), so the whole provenance family was unwritable,
|
|
||||||
and a bundle written the way the spec documents it was refused.
|
|
||||||
|
|
||||||
Admitted now, both parsing to the same value (a list of dicts):
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
sources: [{ id: a, resource: https://e.com/a }]
|
|
||||||
sources:
|
|
||||||
- id: a
|
|
||||||
resource: https://e.com/a
|
|
||||||
```
|
|
||||||
|
|
||||||
**`resource` is allowlisted inside a `sources` entry and nowhere else.** 1.2.0
|
|
||||||
left it off the allowlist on the argument that the parser could not tell
|
|
||||||
`sources[].resource` (§5.1, a citation) from `executor.resource` /
|
|
||||||
`attester.resource` (§10, a pointer to code to be run — the door-C route closed
|
|
||||||
in 1.1.0). That premise was false: the owning key is in scope at every call
|
|
||||||
site and was simply never threaded through. It is threaded now, so the
|
|
||||||
discrimination is structural rather than a judgement about the value, and
|
|
||||||
`executor: [{ resource: skills/run.md }]` and `attester:\n - resource: …` are
|
|
||||||
refused on the allowlist through *every* carrier, including the two this adds.
|
|
||||||
|
|
||||||
Refusal stays the default everywhere else. A flow sequence of plain scalars
|
|
||||||
(`tags: [a, b]`) still raises — the sequence carrier is opened for the flow
|
|
||||||
mapping element and nothing else. A `sources` entry still admits scalar leaves
|
|
||||||
only, so SPEC §5.1's optional *per-entry* `usage_window` (a mapping inside a
|
|
||||||
mapping) is refused: no nesting past depth 1 is a security property, and it was
|
|
||||||
not spent here. Registered as a conformance gap in `docs/LIMITATIONS.md`. A
|
|
||||||
block list may not mix scalar items and mappings. Off-allowlist keys, anchors,
|
|
||||||
aliases, tags, duplicate keys and unclosed collections raise as before, and a
|
|
||||||
refused mapping still raises rather than degrading into a string.
|
|
||||||
|
|
||||||
**New residual, registered:** `sources[].resource` is scanned as text (T1) but
|
|
||||||
never validated as a URL. T3's https allowlist cannot be extended to it without
|
|
||||||
over-blocking conformant bundles — §5.1 permits a bundle-relative path or a
|
|
||||||
scope descriptor a consumer cannot follow at all. A consumer that dereferences
|
|
||||||
it must call `okf.validate_resource_url` itself.
|
|
||||||
|
|
||||||
No exported surface changed and no detector behaviour or calibration changed.
|
|
||||||
Suite 834 → 868 (34 new rows, plus four pre-existing rows updated where this
|
|
||||||
release changed the behaviour they pinned); 130/130 classes, 6/6 documented
|
|
||||||
gaps hold, 45 limitations, ReDoS sweep 0/152 candidates flagged.
|
|
||||||
|
|
||||||
## [1.2.0] — 2026-08-23
|
|
||||||
|
|
||||||
### Added — OKF frontmatter can express one mapping form: typed and allowlisted
|
|
||||||
|
|
||||||
`okf.parse_frontmatter` gave the mapping *class* no expressible form at all. OKF
|
|
||||||
v0.2 writes its whole trust and provenance layer as mappings — SPEC.md @
|
|
||||||
`62432a09` uses flow form in its own §5.1/§5.2 examples, and §11 carries a hard
|
|
||||||
MUST for consumers ("MUST treat a bare `verified` mapping as a one-element
|
|
||||||
list") that presupposes they parse. A consumer measured **0 of 53** upstream
|
|
||||||
concepts through the gate on 0.3.4, 1.0.0 and 1.1.0. That was a contract
|
|
||||||
collision, not a calibration setting: no threshold would have moved it.
|
|
||||||
|
|
||||||
Admitted now: a flow mapping (`generated: { by: x, at: y }`), as a value or as a
|
|
||||||
block-list item, whose every key is on a nine-name allowlist (`by`, `at`, `from`,
|
|
||||||
`to`, `id`, `title`, `author`, `usage_count`, `last_modified`) and whose every
|
|
||||||
leaf is a plain scalar run through the *unchanged* dangerous-value and
|
|
||||||
mapping-construct predicates.
|
|
||||||
|
|
||||||
The form is additive and refusal stays the default. A key off the allowlist, a
|
|
||||||
nested collection, a quoted leaf, a duplicate key, an empty or unclosed mapping,
|
|
||||||
and `{a:b}` (which PyYAML 6.0.3 reads as the *key* `a:b`) all raise, and a
|
|
||||||
refused mapping still raises rather than degrading into a string — the 1.1.0
|
|
||||||
defect is not reopened. Nested-block (`k:\n sub: v`), dotted (`k.sub: v`) and
|
|
||||||
inline-second-colon (`k: sub: v`) routes to a mapping still raise, each on its
|
|
||||||
own rule.
|
|
||||||
|
|
||||||
**`resource` is deliberately off the allowlist**, though SPEC.md §5.1 names it
|
|
||||||
inside a `sources` entry. It is a pointer rather than a label and the only key
|
|
||||||
T3 exists for: admitting it would let `executor: { resource: skills/run.md }`
|
|
||||||
carry an executable-code pointer through in typed clothes, which is the door-C
|
|
||||||
route closed in 1.1.0. It costs nothing today — the conformant carrier for
|
|
||||||
`sources[].resource` is a block sequence of block mappings, which this form does
|
|
||||||
not admit either way.
|
|
||||||
|
|
||||||
Mapping leaves are scanned like every other frontmatter value (T1), so an
|
|
||||||
injection parked in `generated: { by: ... }` reaches `scan_output`. Coverage
|
|
||||||
matrix: 130 classes, up from 129 (the new row is the off-allowlist key).
|
|
||||||
|
|
||||||
No exported surface changed; no detector behaviour and no calibration changed.
|
|
||||||
|
|
||||||
|
|
||||||
### Changed — the ReDoS sweep now measures on the same clock as the bounds it justifies
|
|
||||||
|
|
||||||
`docs/redos-sweep.py` timed on `time.monotonic()` while every ReDoS bound in the
|
|
||||||
suite moved to process CPU time (`tests/redos_clock.py`), so the 1.5 ms
|
|
||||||
sensitivity floor and the "~23 s at the cap" figure published in
|
|
||||||
`docs/LIMITATIONS.md` came from a different instrument than the bounds they
|
|
||||||
support. The script now imports `scan_seconds` rather than timing itself.
|
|
||||||
|
|
||||||
The floor was re-derived on that instrument and **stayed at 1.5 ms**: over twelve
|
|
||||||
full runs of all 2585 arms the median ratio is 1.95-2.03 in every size bucket
|
|
||||||
above 50 µs, but two-point excursions past the 2.6 flag threshold persist at every
|
|
||||||
magnitude (p99 ratio 2.9-3.3 even above 1 ms) — 6.9 flagged arms per run at a
|
|
||||||
0.5 ms floor, 1.1 at 1.0 ms, 0.33 at 1.5 ms. Descheduling was never what made this
|
|
||||||
sweep noisy; a ratio computed from two points is. Four arms flagged across those
|
|
||||||
twelve runs, each in exactly one of them, and six arms that have ever flagged
|
|
||||||
re-measure at exponent 0.97-1.09 over six doublings — at most 1.2 s at the
|
|
||||||
1 000 000-char cap. The pattern count the script prints is 152, not the 150 of the
|
|
||||||
0.3.4 entry below; `docs/LIMITATIONS.md` now carries the current number.
|
|
||||||
|
|
||||||
No exported surface, no detector behaviour and no calibration changed.
|
|
||||||
|
|
||||||
|
|
||||||
## [1.1.0] — 2026-08-13
|
## [1.1.0] — 2026-08-13
|
||||||
|
|
@ -143,12 +33,6 @@ while closing it**, and is the reason this release names two routes rather than
|
||||||
Neither shape is conformant OKF — a well-formed bundle does not produce them; a
|
Neither shape is conformant OKF — a well-formed bundle does not produce them; a
|
||||||
malformed or hostile one can.
|
malformed or hostile one can.
|
||||||
|
|
||||||
**What closed is the type confusion, not pointer-smuggling as a class.** T3 still
|
|
||||||
inspects `resource` and nothing else, so an honest string under another key rides
|
|
||||||
through exactly as before: `attester: attesters/sql_equality.py` is WARN, while
|
|
||||||
the same path under `resource:` FAIL_SECUREs. The string is still scanned like any
|
|
||||||
other frontmatter value under T1. Nothing about that changed here.
|
|
||||||
|
|
||||||
**The boundary is where YAML puts it**, ground-truthed against PyYAML 6.0.3 rather
|
**The boundary is where YAML puts it**, ground-truthed against PyYAML 6.0.3 rather
|
||||||
than reasoned: `": "` and a trailing `":"` are exactly the two shapes where a plain
|
than reasoned: `": "` and a trailing `":"` are exactly the two shapes where a plain
|
||||||
scalar becomes a mapping, and they are refused. A colon carrying neither a space nor
|
scalar becomes a mapping, and they are refused. A colon carrying neither a space nor
|
||||||
|
|
@ -162,10 +46,8 @@ here — the `1.0.0` entry says exactly this is a fix, not a break. A consumer w
|
||||||
bundles carry an unquoted `": "` in a frontmatter value will now see those concepts
|
bundles carry an unquoted `": "` in a frontmatter value will now see those concepts
|
||||||
refused at import; quote the value, and it parses.
|
refused at import; quote the value, and it parses.
|
||||||
|
|
||||||
Suite 792 → **802**: 13 rows added (4 rejected shapes, 7 admitted ones, 2 through
|
Suite 792 → **802** (11 rows added, 1 retired). 129/129 classes, 6/6 documented
|
||||||
`import_bundle`), 3 retired (the two that pinned the defect, and the one-key row
|
gaps, 35 limitations — all unchanged.
|
||||||
in the block-list table). 129/129 classes, 6/6 documented gaps, 35 limitations —
|
|
||||||
all unchanged.
|
|
||||||
|
|
||||||
|
|
||||||
## [1.0.0] — 2026-08-13
|
## [1.0.0] — 2026-08-13
|
||||||
|
|
|
||||||
20
CLAUDE.md
20
CLAUDE.md
|
|
@ -11,27 +11,11 @@ framework-agnostisk kode.
|
||||||
Referanse-implementasjon: `claude-code-llm-wiki` Stage B (`tools/wiki_ingest/`).
|
Referanse-implementasjon: `claude-code-llm-wiki` Stage B (`tools/wiki_ingest/`).
|
||||||
Lexikon-seed: `injection-patterns.mjs` fra `llm-security`-pluginen.
|
Lexikon-seed: `injection-patterns.mjs` fra `llm-security`-pluginen.
|
||||||
|
|
||||||
Repoet er på **v1.3.0** — den eksporterte Python-surfacen er frosset under semver
|
Repoet er på **v1.1.0** — den eksporterte Python-surfacen er frosset under semver
|
||||||
(deteksjonsatferd er det IKKE; kalibrering flytter seg i 1.x). Stdlib-kjernen er
|
(deteksjonsatferd er det IKKE; kalibrering flytter seg i 1.x). Stdlib-kjernen er
|
||||||
bygget og testet (15 moduler +
|
bygget og testet (15 moduler +
|
||||||
topp-nivå wiring, showcase + korpus), inkl. OKF-adapter og aktivt-innhold-
|
topp-nivå wiring, showcase + korpus), inkl. OKF-adapter og aktivt-innhold-
|
||||||
detektor (EchoLeak-klassen) i output-gaten. OKF-frontmatterens mapping-klasse
|
detektor (EchoLeak-klassen) i output-gaten. Mode-b `import_bundle` skanner
|
||||||
har **fire** uttrykkbare bærere (G3 21.08, G30 02.09): flow-mapping som verdi
|
|
||||||
og som blokkliste-element, flow-sekvens av flow-mappinger, og blokk-sekvens av
|
|
||||||
blokk-mappinger (SPEC §5.1s egen form). HVER nøkkel i alle fire står på
|
|
||||||
allowlisten og hvert blad er en ren skalar. Formen er trygg fordi allowlisten
|
|
||||||
inspiserer hver nøkkel; det blanke avslaget var håndhevelsen, ikke poenget.
|
|
||||||
**`resource` er allowlistet KUN inne i en `sources`-oppføring** — foreldre-
|
|
||||||
nøkkelen avgjør, så `executor`/`attester` sin `resource` (§10, dør C) avvises
|
|
||||||
gjennom hver eneste bærer. 1.2.0s begrunnelse for å utelate den (parseren
|
|
||||||
manglet foreldre-kontekst) var målt feil: konteksten var der, den var bare
|
|
||||||
aldri tredd gjennom. Topp-nivå blokk-mapping, dotted- og inline-kolon-rutene
|
|
||||||
raiser fortsatt, flow-sekvens av rene skalarer (`tags: [a, b]`) raiser, en
|
|
||||||
blokkliste kan ikke blande skalarer og mappinger, og en avvist mapping raiser —
|
|
||||||
den degraderer aldri til en streng (1.1.0-defekten). `sources[].resource`
|
|
||||||
URL-valideres ALDRI (T3 ser kun topp-nivå `resource`) — §5.1 tillater
|
|
||||||
bundle-relative stier og scope-beskrivelser, så en https-gate ville over-blokkert
|
|
||||||
konforme bundles; konsumenten må selv kalle `validate_resource_url`. Mode-b `import_bundle` skanner
|
|
||||||
reserverte strukturfiler (`index.md`/`log.md`) i mottatte bundles i stedet for å
|
reserverte strukturfiler (`index.md`/`log.md`) i mottatte bundles i stedet for å
|
||||||
path-avvise dem; upload-front-end beholder shadow-reject (`allow_reserved=False`).
|
path-avvise dem; upload-front-end beholder shadow-reject (`allow_reserved=False`).
|
||||||
Output-gatens decode-and-rescan mater dekodet base64-klartekst gjennom BÅDE lexicon
|
Output-gatens decode-and-rescan mater dekodet base64-klartekst gjennom BÅDE lexicon
|
||||||
|
|
|
||||||
32
README.md
32
README.md
|
|
@ -2,7 +2,7 @@
|
||||||
|
|
||||||
Write-time defensive layer for Python pipelines that persist LLM output: sanitize, fence, tool-less quarantined transform, capability isolation, scan before persist, fail-secure.
|
Write-time defensive layer for Python pipelines that persist LLM output: sanitize, fence, tool-less quarantined transform, capability isolation, scan before persist, fail-secure.
|
||||||
|
|
||||||

|

|
||||||

|

|
||||||

|

|
||||||

|

|
||||||
|
|
@ -33,7 +33,7 @@ at write time, never assumed from the format. Any pipeline ingesting external da
|
||||||
into an agent-read store has this shape; an OKF wiki is its canonical form — which
|
into an agent-read store has this shape; an OKF wiki is its canonical form — which
|
||||||
is why the guard ships a first-class OKF adapter (below).
|
is why the guard ships a first-class OKF adapter (below).
|
||||||
|
|
||||||
**Status:** `v1.3.0`. The stdlib-only core — its detector, contract, and
|
**Status:** `v1.1.0`. The stdlib-only core — its detector, contract, and
|
||||||
OKF-adapter modules plus the top-level wiring — is built and tested, exercised by
|
OKF-adapter modules plus the top-level wiring — is built and tested, exercised by
|
||||||
an end-to-end showcase and adversarial + false-positive corpora. The exported
|
an end-to-end showcase and adversarial + false-positive corpora. The exported
|
||||||
Python surface is now frozen under semver: nothing exported is removed, renamed or
|
Python surface is now frozen under semver: nothing exported is removed, renamed or
|
||||||
|
|
@ -41,24 +41,12 @@ given a different meaning without a `2.0.0`. **Detection behaviour is not frozen
|
||||||
severities, thresholds and lexicon entries are calibration and move in `1.x`. There
|
severities, thresholds and lexicon entries are calibration and move in `1.x`. There
|
||||||
are real limitations, stated plainly below; read them.
|
are real limitations, stated plainly below; read them.
|
||||||
|
|
||||||
## Table of Contents
|
|
||||||
|
|
||||||
- [Install](#install)
|
|
||||||
- [Quickstart — the two bookends](#quickstart--the-two-bookends)
|
|
||||||
- [OKF / LLM-wiki support (shipped)](#okf--llm-wiki-support-shipped)
|
|
||||||
- [What it protects against](#what-it-protects-against)
|
|
||||||
- [The reusable contract (adopt-this checklist)](#the-reusable-contract-adopt-this-checklist)
|
|
||||||
- [Known limitations](#known-limitations)
|
|
||||||
- [Non-goals](#non-goals)
|
|
||||||
- [Design & threat model](#design--threat-model)
|
|
||||||
- [License](#license)
|
|
||||||
|
|
||||||
## Install
|
## Install
|
||||||
|
|
||||||
Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
|
Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v1.3.0"
|
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v1.1.0"
|
||||||
```
|
```
|
||||||
|
|
||||||
The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or
|
The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or
|
||||||
|
|
@ -165,15 +153,7 @@ Per-concept gates: **path / reserved-name** (rejects `..` traversal and reserved
|
||||||
reject-by-default loader that refuses anchors, aliases, and explicit tags *by
|
reject-by-default loader that refuses anchors, aliases, and explicit tags *by
|
||||||
construction*, so a billion-laughs alias expansion or a `!!python/object` coercion
|
construction*, so a billion-laughs alias expansion or a `!!python/object` coercion
|
||||||
cannot occur (it is deliberately **not** a general YAML engine, whose own features
|
cannot occur (it is deliberately **not** a general YAML engine, whose own features
|
||||||
are the attack surface). The one mapping form it accepts is OKF v0.2's flow
|
are the attack surface); **`resource` https-allowlist** (hard-rejects
|
||||||
mapping — `generated: { by: x, at: y }`, `verified: { … }` bare or listed,
|
|
||||||
`usage_window: { from: …, to: … }` — admitted key-by-key against a nine-name
|
|
||||||
allowlist (`by`, `at`, `from`, `to`, `id`, `title`, `author`, `usage_count`,
|
|
||||||
`last_modified`) with plain-scalar leaves only. A key off that list, a nested
|
|
||||||
collection or a duplicate key is refused, and `resource` is deliberately not on
|
|
||||||
it; the block, dotted and inline-colon routes to a mapping still raise. See
|
|
||||||
[LIMITATIONS](docs/LIMITATIONS.md) for what that admits and what it still walls
|
|
||||||
off (a `sources` block list of mappings is still refused); **`resource` https-allowlist** (hard-rejects
|
|
||||||
`data:`/`javascript:`/`file:` before commit — a reject-gate, not defang);
|
`data:`/`javascript:`/`file:` before commit — a reject-gate, not defang);
|
||||||
**whole-concept scan** (frontmatter *values* + body through `scan_output`);
|
**whole-concept scan** (frontmatter *values* + body through `scan_output`);
|
||||||
**cross-link graph** (surfaces dangling targets, the dormant-injection signal, and
|
**cross-link graph** (surfaces dangling targets, the dormant-injection signal, and
|
||||||
|
|
@ -187,7 +167,7 @@ driven by a **live payload** in the coverage matrix — run it to watch all 134
|
||||||
in your own environment:
|
in your own environment:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python -m llm_ingestion_guard.coverage # 130/130 classes; exit 0 = all as documented
|
python -m llm_ingestion_guard.coverage # 129/129 classes; exit 0 = all as documented
|
||||||
```
|
```
|
||||||
|
|
||||||
| Anchor | Attack classes it stops (representative) |
|
| Anchor | Attack classes it stops (representative) |
|
||||||
|
|
@ -269,7 +249,7 @@ a green scan means safe content. The highest-impact items:
|
||||||
two of the three corpora are living, so the cells are not rewritten in place.
|
two of the three corpora are living, so the cells are not rewritten in place.
|
||||||
Method and before/after: [`docs/rawhtml-census.py`](docs/rawhtml-census.py).
|
Method and before/after: [`docs/rawhtml-census.py`](docs/rawhtml-census.py).
|
||||||
|
|
||||||
**Full list — 45 items, each with the mechanism, plus the out-of-scope boundary:**
|
**Full list — 35 items, each with the mechanism, plus the out-of-scope boundary:**
|
||||||
[`docs/LIMITATIONS.md`](docs/LIMITATIONS.md). Several carry field measurements from
|
[`docs/LIMITATIONS.md`](docs/LIMITATIONS.md). Several carry field measurements from
|
||||||
consumer corpora, including the false positives the URL-shape rule actually produces.
|
consumer corpora, including the false positives the URL-shape rule actually produces.
|
||||||
|
|
||||||
|
|
|
||||||
10
SECURITY.md
10
SECURITY.md
|
|
@ -25,12 +25,12 @@ you happened to observe.
|
||||||
**Do not open a public issue for a vulnerability.** Public disclosure before a fix
|
**Do not open a public issue for a vulnerability.** Public disclosure before a fix
|
||||||
gives an attacker a window against every downstream consumer.
|
gives an attacker a window against every downstream consumer.
|
||||||
|
|
||||||
Instead, report it privately to <security@fromaitochitta.com> — mark the subject
|
Instead, report it **privately** to the maintainer via the canonical repository on
|
||||||
`SECURITY`.
|
Forgejo:
|
||||||
|
|
||||||
- Canonical repository: https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security
|
- Repository: `git.fromaitochitta.com/open/llm-ingestion-pipeline-security`
|
||||||
- Alternatively, contact the maintainer directly through that Forgejo instance
|
- Contact the maintainer directly through that Forgejo instance (private message /
|
||||||
(private message / maintainer contact) and mark the subject `SECURITY`.
|
maintainer contact) and mark the subject `SECURITY`.
|
||||||
|
|
||||||
Please include:
|
Please include:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@
|
||||||
especially one converging on Google's Open Knowledge Format (OKF v0.1) — and needs
|
especially one converging on Google's Open Knowledge Format (OKF v0.1) — and needs
|
||||||
to decide **when** and **where** to add a write-time ingestion guard.
|
to decide **when** and **where** to add a write-time ingestion guard.
|
||||||
|
|
||||||
**Status of the guard:** `v1.3.0`. Stdlib-only core, framework-agnostic. The
|
**Status of the guard:** `v1.1.0`. Stdlib-only core, framework-agnostic. The
|
||||||
exported Python surface is frozen under semver — nothing exported is removed,
|
exported Python surface is frozen under semver — nothing exported is removed,
|
||||||
renamed or given a different meaning without a `2.0.0`. Detection behaviour is
|
renamed or given a different meaning without a `2.0.0`. Detection behaviour is
|
||||||
*not* frozen: severities, thresholds and lexicon entries are calibration and move
|
*not* frozen: severities, thresholds and lexicon entries are calibration and move
|
||||||
|
|
@ -142,9 +142,9 @@ live payload:
|
||||||
python -m llm_ingestion_guard.coverage # exit 0 = all as documented
|
python -m llm_ingestion_guard.coverage # exit 0 = all as documented
|
||||||
```
|
```
|
||||||
|
|
||||||
As of `v1.3.0`: **130 / 130 defended classes demonstrated (recall 100%)** and **6 /
|
As of `v1.1.0`: **129 / 129 defended classes demonstrated (recall 100%)** and **6 /
|
||||||
6 documented gaps still hold** (a *closed* gap fails the test, forcing a doc
|
6 documented gaps still hold** (a *closed* gap fails the test, forcing a doc
|
||||||
update). The matrix is the single source of truth for the test suite (**868
|
update). The matrix is the single source of truth for the test suite (**802
|
||||||
passing**), which also asserts total recall, that every lexicon pattern has a
|
passing**), which also asserts total recall, that every lexicon pattern has a
|
||||||
case (so the matrix cannot fall behind the lexicon), the full LLM02 secret-egress
|
case (so the matrix cannot fall behind the lexicon), the full LLM02 secret-egress
|
||||||
set, and the container-layer front-end (CSV formula-injection, zip-slip/bomb,
|
set, and the container-layer front-end (CSV formula-injection, zip-slip/bomb,
|
||||||
|
|
|
||||||
|
|
@ -3,7 +3,7 @@
|
||||||
**A reusable, minimal, dependency-light defensive layer for LLM *ingestion*
|
**A reusable, minimal, dependency-light defensive layer for LLM *ingestion*
|
||||||
pipelines — the write-time siblings of query-time chatbot guardrails.**
|
pipelines — the write-time siblings of query-time chatbot guardrails.**
|
||||||
|
|
||||||
Status: implemented — v1.3.0, exported surface frozen under semver. This document defines what the repo contains
|
Status: implemented — v1.1.0, exported surface frozen under semver. This document defines what the repo contains
|
||||||
and why; the stdlib-only core is built and tested (see `README.md` for usage and
|
and why; the stdlib-only core is built and tested (see `README.md` for usage and
|
||||||
`docs/PLAN.md` for the build order).
|
`docs/PLAN.md` for the build order).
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -40,31 +40,16 @@ items; this is the full list, each with the mechanism.
|
||||||
(an injection in a directory listing is caught) rather than path-rejecting the
|
(an injection in a directory listing is caught) rather than path-rejecting the
|
||||||
conformant bundle. A front-end materialising individual uploads keeps the opposite
|
conformant bundle. A front-end materialising individual uploads keeps the opposite
|
||||||
rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused.
|
rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused.
|
||||||
- **OKF frontmatter is a restricted grammar: a mapping is expressible through
|
- **OKF frontmatter is a restricted grammar: the mapping class has no expressible
|
||||||
four carriers, all of them key-allowlisted.** Gate T2 accepts a line-oriented
|
form.** Gate T2 accepts a line-oriented subset deliberately — full YAML is a larger
|
||||||
subset deliberately — full YAML is a larger parse-attack surface than a
|
parse-attack surface than a write-time gate needs. Nested mappings and flow
|
||||||
write-time gate needs. The admitted carriers are the flow mapping as a value
|
collections (`[a, b]`, `{k: v}`) are *rejected outright*, which fails secure.
|
||||||
(`generated: { by: x, at: y }`), the flow mapping as a block-list item, the
|
**All four routes to a mapping fail, each on a different rule** — flow (`{k: v}`)
|
||||||
flow *sequence* of flow mappings (`sources: [{ id: a, resource: x }]`) and the
|
on the disallowed value-start indicator, block (`k:\n sub: v`) on the
|
||||||
block sequence of block mappings (SPEC.md §5.1's own form). **The routes that
|
nested-mapping check, dotted keys (`k.sub: v`) on the key pattern, and the inline
|
||||||
still fail, each on a different rule:** a top-level block *mapping*
|
second colon (`k: sub: v`) on the mapping-construct check — so the mapping *class*
|
||||||
(`k:\n sub: v`) on the nested-mapping check, dotted keys (`k.sub: v`) on the
|
has no expressible form, rather than one form being preferable to another. What
|
||||||
key pattern, the inline second colon (`k: sub: v`) on the mapping-construct
|
survives is scalars and flat lists of strings. **Two of those routes used to
|
||||||
check, and a flow sequence of plain *scalars* (`tags: [a, b]`) on the `[`
|
|
||||||
indicator — the sequence carrier is opened for the mapping element and nothing
|
|
||||||
else. **Every carrier is admitted only when every key is on an allowlist**
|
|
||||||
(`by`, `at`, `from`, `to`, `id`, `title`, `author`, `usage_count`,
|
|
||||||
`last_modified` — the keys SPEC.md @ `62432a09` §5.1/§5.2 names inside a mapping,
|
|
||||||
plus `resource` and `usage_window` *under `sources` only*, see below)
|
|
||||||
and every leaf is a plain scalar, itself run through the same value predicates as a
|
|
||||||
top-level scalar. Nested collections, quoted leaves, duplicate keys, an empty or
|
|
||||||
unclosed mapping, and `{a:b}` (which PyYAML 6.0.3 reads as the *key* `a:b`, not as
|
|
||||||
a scalar) all raise. The form is expressible, never trusted: the allowlist
|
|
||||||
inspects every key, which is the property that carried the security when the
|
|
||||||
blanket refusal was doing the enforcing. **A block list may not mix scalar items
|
|
||||||
and mappings** — YAML permits it, but a consumer iterating `sources` and reading
|
|
||||||
`entry.get("id")` gets an `AttributeError` off the first `str`. What else
|
|
||||||
survives is scalars and flat lists of strings. **Two routes used to
|
|
||||||
degrade into a string instead of failing, and that defect is closed in `1.1.0`**:
|
degrade into a string instead of failing, and that defect is closed in `1.1.0`**:
|
||||||
a block-sequence item carrying exactly one key (`sources:\n - uri: https://e.com/a`
|
a block-sequence item carrying exactly one key (`sources:\n - uri: https://e.com/a`
|
||||||
yielded the *string* `'uri: https://e.com/a'`) and the inline second colon
|
yielded the *string* `'uri: https://e.com/a'`) and the inline second colon
|
||||||
|
|
@ -72,13 +57,7 @@ items; this is the full list, each with the mechanism.
|
||||||
outright). Both parsed "successfully" into the wrong *type*, and a pointer parked
|
outright). Both parsed "successfully" into the wrong *type*, and a pointer parked
|
||||||
in one rode through in a key the `resource` allowlist never inspects — mode-b
|
in one rode through in a key the `resource` allowlist never inspects — mode-b
|
||||||
`import_bundle` returned WARN and wrote the merged concept verbatim. Both now
|
`import_bundle` returned WARN and wrote the merged concept verbatim. Both now
|
||||||
FAIL_SECURE at T2, before the allowlist is reached. **What closed is the type
|
FAIL_SECURE at T2, before the allowlist is reached. **The boundary is where YAML
|
||||||
confusion, not pointer-smuggling as a class:** T3 still inspects `resource` and
|
|
||||||
nothing else, so an honest *string* under another key rides through exactly as
|
|
||||||
before — `attester: attesters/sql_equality.py` is WARN, while the same path
|
|
||||||
under `resource:` FAIL_SECUREs. That is by design (the string is scanned like
|
|
||||||
any other frontmatter value under T1) and it is not what `1.1.0` changed.
|
|
||||||
**The boundary is where YAML
|
|
||||||
puts it**, ground-truthed against PyYAML 6.0.3: `": "` and a trailing `":"` open a
|
puts it**, ground-truthed against PyYAML 6.0.3: `": "` and a trailing `":"` open a
|
||||||
mapping and are refused; a colon carrying neither a space nor a line end
|
mapping and are refused; a colon carrying neither a space nor a line end
|
||||||
(`domain:security`, `https://e.com:8443/a`) does not and still parses, as does a
|
(`domain:security`, `https://e.com:8443/a`) does not and still parses, as does a
|
||||||
|
|
@ -89,58 +68,13 @@ items; this is the full list, each with the mechanism.
|
||||||
door A/B persist path, so frontmatter that fails secure on import passes
|
door A/B persist path, so frontmatter that fails secure on import passes
|
||||||
`screen_output` unremarked. The grammar therefore bounds what a consumer can *receive*,
|
`screen_output` unremarked. The grammar therefore bounds what a consumer can *receive*,
|
||||||
never what a producer can *emit*. Verified identical on 0.2.0 and 0.3.1.
|
never what a producer can *emit*. Verified identical on 0.2.0 and 0.3.1.
|
||||||
- **`sources` passes in both of its spec carriers; the per-entry `usage_window`
|
- **Consequence: an OKF v0.2 concept cannot traverse the external-import path.** Both
|
||||||
does not.** The wall used to be total: both of v0.2's backward-breaking migration
|
of v0.2's backward-breaking migration targets are nested — `timestamp` → `generated.at`,
|
||||||
targets are mappings — `timestamp` → `generated.at`, and body `# Citations` → a
|
and body `# Citations` → a `sources` block list of mappings — so a conformant v0.2
|
||||||
`sources` block list of mappings — and a consumer measured **0 of 53** upstream
|
concept fails secure at the frontmatter gate. This is the correct direction but it is
|
||||||
concepts through the gate. `generated.at` stopped being a wall in `1.2.0`;
|
a compatibility wall, not a policy: v0.2 support requires a deliberate parse-safety
|
||||||
`sources` stopped being one in `1.3.0`, which admits both the block sequence of
|
decision about widening the grammar, and the dangling-or-substituted `executor`/
|
||||||
block mappings (SPEC.md §5.1's own example) and the flow sequence of flow
|
`attester` pointer question only becomes live once that decision is made.
|
||||||
mappings (the form the OKF producers emit, measured 2026-09-02 by two consumers
|
|
||||||
independently). **`resource` is allowlisted inside a `sources` entry and nowhere
|
|
||||||
else.** `1.2.0` left it off on the argument that the parser could not tell
|
|
||||||
`sources[].resource` (§5.1, a citation) from `executor.resource` / `attester.resource`
|
|
||||||
(§10, a pointer to code to be run — door C). That premise was measured false: the
|
|
||||||
owning key is in scope at every call site and was simply never threaded through.
|
|
||||||
It is threaded now, so `executor: [{ resource: skills/run.md }]` and
|
|
||||||
`attester:\n - resource: …` are refused on the allowlist through *every* carrier,
|
|
||||||
including the two this opened. **What stays refused: a `usage_window` inside a
|
|
||||||
`sources` entry.** SPEC §5.1 permits it per entry ("A single entry MAY carry its
|
|
||||||
own `usage_window`"), and it is a mapping inside a mapping — depth 2, which this
|
|
||||||
parser admits at no key. A bundle using the per-entry override is refused; the
|
|
||||||
shared sibling `usage_window` (the §5.1 example's own form) passes. This is a
|
|
||||||
registered conformance gap, not an oversight: no-nesting-past-depth-1 is a
|
|
||||||
security property, and spending it was not what the fix was for.
|
|
||||||
- **`sources[].resource` is scanned as text but never validated as a URL.** T3's
|
|
||||||
https allowlist inspects the *top-level* `resource` and nothing else. It cannot
|
|
||||||
be extended to `sources` entries without over-blocking conformant bundles: SPEC
|
|
||||||
§5.1 explicitly permits a bundle-relative path, a path into `references/`, or a
|
|
||||||
scope descriptor a consumer cannot follow at all (the OKF producers' own golden
|
|
||||||
bundle emits `resource: fixture`). So a `sources` entry may carry
|
|
||||||
`file://`, `javascript:` or any other string; it goes through T1's scan like any
|
|
||||||
other frontmatter value, and nothing else. **A consumer that dereferences
|
|
||||||
`sources[].resource` must validate it itself** — `okf.validate_resource_url` is
|
|
||||||
exported for exactly that. The dangling-or-substituted `executor`/`attester`
|
|
||||||
pointer question stays out of reach separately: both are top-level block
|
|
||||||
mappings, a carrier that is still refused.
|
|
||||||
- **`tags` and `description` block the OKF import corpus universally, before the
|
|
||||||
trust layer is even reached.** The line-flat frontmatter parser has no
|
|
||||||
sequence-value type at all: `tags` is present in 53/53 upstream concept
|
|
||||||
documents — 9/53 as a flow sequence (`[a, b, c]`, rejected on the `[`
|
|
||||||
indicator) and 44/53 as a block sequence (`- a` / `- b`, rejected as
|
|
||||||
`"malformed frontmatter line"`) — 100% rejection regardless of form.
|
|
||||||
`description` is present in 53/53; 29/53 is a folded plain scalar continuing
|
|
||||||
on an indented second line, which the parser has no continuation-line model
|
|
||||||
for and misreads as `"nested mappings are not supported"` (the remaining
|
|
||||||
24/53 are single-line and parse fine). Measured directly on the upstream
|
|
||||||
reference bundles (`_okf-upstream/okf` @ `3fcbb9f`): removing `tags` alone
|
|
||||||
lets 4/53 documents pass; removing both `tags` and `description` together
|
|
||||||
(trust layer untouched) lets the same 4/53 pass, and all four then parse
|
|
||||||
`generated` correctly as a mapping. **Independent of the mapping-form work
|
|
||||||
above:** neither `1.2.0`'s flow mapping nor `1.3.0`'s `sources` carriers move
|
|
||||||
anything on this corpus, because `tags`/`description` reject before `sources`
|
|
||||||
is ever read. No sequence-value type or continuation-line model exists in the
|
|
||||||
stdlib-only parser to close this with.
|
|
||||||
- **A persist gate cannot cover execution risk.** OKF v0.2 introduces concepts whose
|
- **A persist gate cannot cover execution risk.** OKF v0.2 introduces concepts whose
|
||||||
purpose is to *name code to be run* (`runtime`, `executor.resource`,
|
purpose is to *name code to be run* (`runtime`, `executor.resource`,
|
||||||
`attester.resource`). This library answers "is this safe to **store**"; executable
|
`attester.resource`). This library answers "is this safe to **store**"; executable
|
||||||
|
|
@ -364,21 +298,6 @@ items; this is the full list, each with the mechanism.
|
||||||
`test_split_tightens_the_trusted_tier_when_both_carriers_are_present` constructs
|
`test_split_tightens_the_trusted_tier_when_both_carriers_are_present` constructs
|
||||||
and pins. Read the zero as "not observed in any of the three populations, each
|
and pins. Read the zero as "not observed in any of the three populations, each
|
||||||
counted against its own denominator", never as "cannot happen".
|
counted against its own denominator", never as "cannot happen".
|
||||||
- **The split also LOOSENS the upload door for a lone anchor — the direction it was
|
|
||||||
built for, and the one with a residual worth naming.** Measured as shipped:
|
|
||||||
`<a href="https://ext.example/p">t</a>` on its own emits one
|
|
||||||
`active:raw-html-link` at MEDIUM and disposes `quarantine_review` under
|
|
||||||
`PRESET_USER_UPLOAD` (`warn` under `PRESET_TRUSTED_SOURCE`); through 0.6.1 the
|
|
||||||
same document graded HIGH and `fail_secure`d. An `<img src>` to the same host is
|
|
||||||
untouched — `active:raw-html`, HIGH, `fail_secure`. So an anchor pointing at an
|
|
||||||
attacker-controlled host, arriving on an untrusted upload, is now a human decision
|
|
||||||
rather than a halt. The trade is deliberate and it removes an asymmetry that came
|
|
||||||
from syntax rather than affordance: following an anchor needs a click, exactly like
|
|
||||||
the markdown inline link that has graded MEDIUM since 0.3.1, so the same URL no
|
|
||||||
longer grades two different ways depending on which syntax carries it. It is
|
|
||||||
recorded here so 0.7.0's "frees N documents" is not read as free — what was freed
|
|
||||||
is the click-required class, and MEDIUM is a real grade drop on the door where
|
|
||||||
every finding is trust-escalated.
|
|
||||||
- **Raw-HTML findings no longer count end tags, and that moved a published field.**
|
- **Raw-HTML findings no longer count end tags, and that moved a published field.**
|
||||||
Through 0.6.1 `</a>` was active by name on its own, so `count` ran roughly 1.6×
|
Through 0.6.1 `</a>` was active by name on its own, so `count` ran roughly 1.6×
|
||||||
the opening-tag total (measured on one corpus) and a start/end pair counted 2.
|
the opening-tag total (measured on one corpus) and a start/end pair counted 2.
|
||||||
|
|
@ -387,44 +306,6 @@ items; this is the full list, each with the mechanism.
|
||||||
every document carrying `</a>`, on a field whose meaning did not change. The
|
every document carrying `</a>`, on a field whose meaning did not change. The
|
||||||
finding count is unaffected: the class still collapses to one finding per class
|
finding count is unaffected: the class still collapses to one finding per class
|
||||||
per document, and `count` was never a document count.
|
per document, and `count` was never a document count.
|
||||||
- **Which tags the no-URL narrowing may render inert is a judgement about affordance,
|
|
||||||
and no test in this repo can derive it.** `_URL_AFFORDANCE_TAGS` holds the nine
|
|
||||||
names whose entire active affordance *is* the URL they name — `a`, `area`, `img`,
|
|
||||||
`video`, `audio`, `source`, `track`, `frame`, `frameset` — so carrying no URL
|
|
||||||
attribute they name no target and grade inert. Every other name in the active set
|
|
||||||
stays active with no attributes at all, because it does something a URL cannot
|
|
||||||
describe: `<script>` executes its body, `<style>` restyles, `<form>` submits. That
|
|
||||||
boundary is asserted, not measured. A name placed in the set whose affordance does
|
|
||||||
*not* reduce to its URL would go silently invisible, and no corpus can catch it,
|
|
||||||
because what it produces is an absence — the census counts findings, and a tag that
|
|
||||||
stopped firing contributes nothing to count. The fail-secure choice one branch
|
|
||||||
further in holds the other way and is worth reading beside it: a URL attribute whose
|
|
||||||
*value* this module cannot resolve keeps the tag active
|
|
||||||
(`active_content.py:320-324`), a branch the three corpora exercise **0** times.
|
|
||||||
That zero is empirical, so the predicate is written not to depend on it.
|
|
||||||
- **"Clean" means *graded, no finding raised* — never *cleaned bytes* — and at one
|
|
||||||
measured consumer's door, `warn` is the floor a document must clear to be
|
|
||||||
persisted rather than rejected.** The word is the library's own: a WARN
|
|
||||||
disposition with nothing to report carries the reason string `"clean: no
|
|
||||||
findings"` (`disposition.py:264`), and this project has repeated that word in the
|
|
||||||
tables it sends consumers. `screen_output` is a judgement API — its
|
|
||||||
`DispositionResult` carries `assessment` / `disposition` / `max_severity` /
|
|
||||||
`reasons`, with no sanitized-text field to read off it. Defanging lives in a
|
|
||||||
separate, deliberate call — `neutralize` — that a caller must invoke itself;
|
|
||||||
nothing upstream of that call transforms a byte. Measured against
|
|
||||||
`llm-ingestion-okf`'s `0.7.0` pin (2026-08-13): `inbox.py:139` sets its persist
|
|
||||||
floor to `warn`, and `inbox.py:323` persists anything carrying that disposition
|
|
||||||
into the bundle; its adapter (`guard_adapter.py:70`) forwards the original
|
|
||||||
extracted text, because nothing upstream ever handed it a transformed one. Four
|
|
||||||
raw-HTML carrier forms the 0.7.0 no-URL narrowing grades inert — an
|
|
||||||
`<a aria-label>` with no `href`, a bare `</a>`, `<Frame>`, `<video />` — verified
|
|
||||||
here (`screen_output(..., PRESET_USER_UPLOAD)`) to dispose `warn, clean: no
|
|
||||||
findings`; at that consumer's door the same four land written into the bundle,
|
|
||||||
carrier present verbatim. Neither library is wrong: `screen_output` never
|
|
||||||
promised transformed bytes, and the consumer never called `neutralize` for them.
|
|
||||||
The gap is in reading "clean" as "sanitized" rather than "no finding raised" — a
|
|
||||||
reading this project's own reports invite, and one that will mislead any caller
|
|
||||||
that persists on `warn` without calling `neutralize` itself.
|
|
||||||
- **Measured, document by document: a large minority of *benign* documents do not
|
- **Measured, document by document: a large minority of *benign* documents do not
|
||||||
persist unattended at the upload door.** The bullets above bound single rules on
|
persist unattended at the upload door.** The bullets above bound single rules on
|
||||||
single URLs. This one bounds the thing a consumer actually feels — how often an
|
single URLs. This one bounds the thing a consumer actually feels — how often an
|
||||||
|
|
@ -551,23 +432,6 @@ items; this is the full list, each with the mechanism.
|
||||||
HIGH under a low-trust preset is `fail_secure`. Report-only means the text is
|
HIGH under a low-trust preset is `fail_secure`. Report-only means the text is
|
||||||
never mutated — it does not mean the finding cannot block.
|
never mutated — it does not mean the finding cannot block.
|
||||||
|
|
||||||
- **The two labels a `<script>` tag raises come from patterns that do not match the
|
|
||||||
same strings.** `hybrid-xss:script-tag` is `<script\b[^><]*>`; `active:raw-html`
|
|
||||||
reads the same tag through `HTML_TAG_RE`, which consumes quoted attribute runs
|
|
||||||
atomically and so tolerates both `<` and `>` inside a quoted value. The lexicon's
|
|
||||||
`<` exclusion is not a modelling choice — it is the 0.3.3 ReDoS fix, and widening
|
|
||||||
it back to `[^>]` restores a quadratic arm (the row below carries the numbers).
|
|
||||||
Measured through both scanners: `<script src="a<b">` raises `active:raw-html` and
|
|
||||||
**no** XSS label, while `<script data-t="a>b">` raises both, the lexicon's match
|
|
||||||
simply ending at the quoted `>`. The disposition never moves — the raw-HTML branch
|
|
||||||
grades `<script>` HIGH with no attributes at all, so every shape here still
|
|
||||||
`fail_secure`s under `PRESET_USER_UPLOAD` — so what the divergence costs is the
|
|
||||||
*label*: a consumer filtering findings on the XSS id sees a subset of the script
|
|
||||||
tags the gate actually caught, and must not read that id as the gate's script-tag
|
|
||||||
census. The residual is practically dead in prose (a `<` inside a script tag's
|
|
||||||
quoted attribute region is not an ordinary shape) and is recorded because the
|
|
||||||
asymmetry is invisible from either scanner alone.
|
|
||||||
|
|
||||||
- **A connection-string password longer than 256 chars is not matched.** The
|
- **A connection-string password longer than 256 chars is not matched.** The
|
||||||
password run in the `*-connstr` egress patterns is bounded by
|
password run in the `*-connstr` egress patterns is bounded by
|
||||||
`MAX_CONNSTR_VALUE`; unbounded, it sits in front of a mandatory `@` and makes
|
`MAX_CONNSTR_VALUE`; unbounded, it sits in front of a mandatory `@` and makes
|
||||||
|
|
@ -584,96 +448,24 @@ items; this is the full list, each with the mechanism.
|
||||||
missed; on one preset it is held for review instead of halted.
|
missed; on one preset it is held for review instead of halted.
|
||||||
|
|
||||||
- **The ReDoS sweep has a measured sensitivity floor, not a clean bill of
|
- **The ReDoS sweep has a measured sensitivity floor, not a clean bill of
|
||||||
health.** All 152 compiled patterns across all eleven regex-bearing modules are
|
health.** All 150 compiled patterns across all eleven regex-bearing modules are
|
||||||
swept arm by arm — payloads synthesised per run from each pattern's own
|
swept arm by arm — payloads synthesised per run from each pattern's own
|
||||||
skeleton, so `[`, `[system]` and `[system](` are each probed separately rather
|
skeleton, so `[`, `[system]` and `[system](` are each probed separately rather
|
||||||
than relying on generic units, and each pattern is timed in the call mode the
|
than relying on generic units, and each pattern is timed in the call mode the
|
||||||
production code uses (`.sub()`/`.finditer()` visit every start position where
|
production code uses (`.sub()`/`.finditer()` visit every start position where
|
||||||
`.match()` cannot). Five patterns were quadratic across 0.3.3 and 0.3.4; all
|
`.match()` cannot). Five patterns were quadratic across 0.3.3 and 0.3.4; all
|
||||||
are fixed. But the sweep flags on *timing*, and it ignores measurements below a
|
are fixed. But the sweep flags on *timing*, and it ignores measurements below a
|
||||||
1.5 ms noise floor at N=8000 — process CPU time, re-derived on that instrument
|
1.5 ms noise floor at N=8000. A quadratic arm sitting just under that floor
|
||||||
(see the clock bullet below) rather than inherited from the wall clock the
|
would still cost **up to ~23 s** at the 1 000 000-char cap. So the claim this
|
||||||
script used through 1.1.0. A quadratic arm sitting just under that floor would
|
sweep supports is "no arm worse than ~23 s at the cap", not "no quadratic arm
|
||||||
still cost **up to ~23 s** at the 1 000 000-char cap — that figure is arithmetic
|
remains". The method's blind spot is real and has now been demonstrated twice:
|
||||||
and not a measurement: a quadratic arm costs the square of the length ratio, and
|
a generic-payload pass found only one of 0.3.3's two patterns, and 0.3.2's
|
||||||
1.5 ms × 125 × 125 is 23.4 s. So the claim this sweep supports is "no arm worse
|
|
||||||
than ~23 s at the cap", not "no quadratic arm remains". The method's blind spot
|
|
||||||
is real and has now been demonstrated twice: a generic-payload pass found only
|
|
||||||
one of 0.3.3's two patterns, and 0.3.2's
|
|
||||||
hand-written rows missed all three of 0.3.4's — including one on `sanitize`,
|
hand-written rows missed all three of 0.3.4's — including one on `sanitize`,
|
||||||
the first thing every ingested document touches. **Two arm shapes the unit-
|
the first thing every ingested document touches. **Two arm shapes the unit-
|
||||||
repetition payloads cannot express** are pinned by hand as a result: a tag that
|
repetition payloads cannot express** are pinned by hand as a result: a tag that
|
||||||
*closes* around a long body, and a run of plain characters carrying no anchor
|
*closes* around a long body, and a run of plain characters carrying no anchor
|
||||||
at all.
|
at all.
|
||||||
|
|
||||||
- **A green ReDoS row is evidence only if it has been seen red, and three rows in
|
|
||||||
this suite had never been.** The class is not a bad bound but a payload that cannot
|
|
||||||
reach the defect, and it leaves the row passing under the vulnerable form too. The
|
|
||||||
sub-agent row is the clearest case: the seed's unbounded lazy run costs per *prefix
|
|
||||||
match*, not per character — each start position where `spawn an agent that ` matches
|
|
||||||
drives its own O(N) scan to end-of-string looking for a capability keyword the
|
|
||||||
payload never supplies, so K prefix matches cost K×O(N), and the bounded
|
|
||||||
`(?:\S+\s+){0,12}?` port caps each scan at 12 tokens for K×O(1). A payload that
|
|
||||||
matches the prefix **once** and then pads pays a single lazy run and is linear
|
|
||||||
however long the pad is — two earlier shapes did exactly that, and the row sat
|
|
||||||
measured-dead at 1.2× until the payload was rebuilt as
|
|
||||||
`"spawn an agent that " * 3000`. (The nesting an older comment blamed is a red
|
|
||||||
herring: the inner `.*?` sits in an optional group, never a repeated one.) Measured
|
|
||||||
through `scan_lexicon` with the seed form patched back in — exponent **1.92** against
|
|
||||||
the shipped **1.01**, and **4.091s vs 0.190s** at 12 000 words, the seed breaking the
|
|
||||||
2.0s bound outright. Two siblings were dead for different reasons.
|
|
||||||
`lexicon-script-tag` had to be given its own N=200 000: at the shared N=100 000 the
|
|
||||||
vulnerable `[^>]` form measured only ~1.2–1.4s — under the 2.0s assert, so the row
|
|
||||||
was green under both forms and proved nothing. And
|
|
||||||
`test_gate_is_bounded_on_the_long_attribute_arm` was killed by **this repo's own
|
|
||||||
narrowing**: 0.7.0 put `<a>` in `_URL_AFFORDANCE_TAGS`, so its `<a ` + 100k + `>`
|
|
||||||
payload became inert and returned *before* the body ever reached the arm the row
|
|
||||||
exists to guard — separation 1.0×, 0.028s and no findings, against 12.475s for the
|
|
||||||
same payload carried by `<script `. The carrier was moved to `<script `, which is
|
|
||||||
active by name with no attributes, so no future URL-shaped narrowing can hollow it
|
|
||||||
out the same way. The general rule the three share: a payload must **deny** the
|
|
||||||
literal the vulnerable run sits in front of — a unit that supplies it matches
|
|
||||||
immediately and never exercises the run. **Nothing but hand measurement finds this
|
|
||||||
class.** The row is green either way, so
|
|
||||||
the suite cannot report its own blind spot, and every bound in it should be read as
|
|
||||||
"verified red under the vulnerable form" only where a comment says it was.
|
|
||||||
|
|
||||||
- **Every ReDoS bound in the suite is measured on process CPU time, and so is the
|
|
||||||
sweep that sets the published sensitivity floor.** `tests/redos_clock.py` is the one
|
|
||||||
clock all six test files import — `time.process_time()` — because a blowup is spent
|
|
||||||
cycles while a loaded machine steals wall clock without adding any. On
|
|
||||||
`time.monotonic()` two 0.7.0 rows failed at **2.24s / 3.66s** against a 2.0s bound
|
|
||||||
while two census processes held the CPU, and passed 3/3 on an idle machine: they had
|
|
||||||
been descheduled, not slowed. It lives in one module rather than five copies because
|
|
||||||
`test_output.py::test_the_redos_clock_ignores_time_this_process_did_not_spend` pins
|
|
||||||
one implementation, and four unpinned copies would be free to drift back to a wall
|
|
||||||
clock with nothing going red. **What the CPU clock gives up, stated: a scan that
|
|
||||||
BLOCKS forever burns no CPU, so it would hang the suite instead of failing it.**
|
|
||||||
That is acceptable only because every scanner it measures is pure regex over an
|
|
||||||
in-memory string, with no I/O and no locks — the last wall-clock holdout was retired
|
|
||||||
by auditing its path for anything that could block, not by assumption, and a wall
|
|
||||||
clock guarding a mode that cannot occur still charges the false-red premium
|
|
||||||
(measured there at 21.6s against a 10.0s bound, on a scan that spent 7.6s). **The sweep
|
|
||||||
now runs on the same clock, and closing that divergence bought no sensitivity.**
|
|
||||||
`docs/redos-sweep.py` imports `scan_seconds` instead of timing on
|
|
||||||
`time.monotonic()`, so the 1.5 ms floor at N=8000 and the "~23 s at the cap"
|
|
||||||
figure above are finally in the same currency as the bounds they justify. What
|
|
||||||
the move did *not* do is quiet the sweep, and the floor came back unchanged.
|
|
||||||
Measured over **twelve full runs of all 2585 arms**: the median ratio sits at
|
|
||||||
**1.95–2.03** in every size bucket above 50 µs — the whole surface measures
|
|
||||||
linear — while two-point excursions past the 2.6 flag threshold survive at every
|
|
||||||
magnitude, p99 ratio **2.9–3.3 even above 1 ms**. Flagged arms per run by floor:
|
|
||||||
**6.9 at 0.5 ms, 1.1 at 1.0 ms, 0.33 at 1.5 ms** (0–2 per run), so 1.5 ms is
|
|
||||||
still the knee. Descheduling was never what made this sweep noisy — a ratio
|
|
||||||
computed from two points is. Four distinct arms flagged at the shipped floor
|
|
||||||
across those twelve runs, **each in exactly one of them**, and nine of the twelve
|
|
||||||
runs were clean; eight consecutive runs of the shipped script immediately after a
|
|
||||||
full test run flagged 0–3 arms each, so machine load still moves the count even
|
|
||||||
on a CPU clock. Six flagged arms re-measured over six doublings give exponent
|
|
||||||
**0.97–1.09** and at most 1.2 s at the 1 000 000-char cap. **A single clean run
|
|
||||||
of this sweep is therefore not evidence either** — and neither is a single
|
|
||||||
flagged one.
|
|
||||||
|
|
||||||
- **Every surface now bounds its input, but not all of them the same way.**
|
- **Every surface now bounds its input, but not all of them the same way.**
|
||||||
`sanitize`, `fence` and `neutralize` raise `OversizeInputError` above
|
`sanitize`, `fence` and `neutralize` raise `OversizeInputError` above
|
||||||
`MAX_INPUT_CHARS` (1 000 000) rather than returning a partially transformed
|
`MAX_INPUT_CHARS` (1 000 000) rather than returning a partially transformed
|
||||||
|
|
@ -736,23 +528,6 @@ fails the test, forcing this doc to be updated:
|
||||||
names the outward fetch whatever severity it carries — and must not infer
|
names the outward fetch whatever severity it carries — and must not infer
|
||||||
"nothing was seen" from a low `Severity`.
|
"nothing was seen" from a low `Severity`.
|
||||||
|
|
||||||
- **Under the default action map the assessment axis carries exactly one judgement
|
|
||||||
the disposition does not.** `DEFAULT_ACTION_MAP` sends `NONE` and `LOW` to `WARN`,
|
|
||||||
`ELEVATED` to `QUARANTINE_REVIEW` and `SEVERE` to `FAIL_SECURE` — the last two 1:1.
|
|
||||||
So for any document that carries a finding at all, `assessment` is a relabelling of
|
|
||||||
`disposition` and nothing more; the only thing it adds is *clean* versus *findings
|
|
||||||
present, none dispositive in this context*, which 0.4.0 rendered identically. That
|
|
||||||
collapse is the point (the map is what keeps the separation additive, so a caller
|
|
||||||
ignoring the new axis sees no change), and it is also the limitation: reading
|
|
||||||
`assessment` buys a consumer nothing until it supplies its own `action_map` or needs
|
|
||||||
the clean/low distinction. **The second consequence is on this document.** The
|
|
||||||
published false-positive rates are counts of documents *disposed non-WARN*, and they
|
|
||||||
are a statement about assessed risk only while `NONE` + `LOW` are exactly the WARN
|
|
||||||
pre-image. `tests/test_corpus.py::test_the_published_fp_metric_is_a_risk_statement`
|
|
||||||
pins that equivalence — but it pins it for `DEFAULT_ACTION_MAP`. A caller running its
|
|
||||||
own map makes "disposed non-WARN" a different claim from the one measured here, with
|
|
||||||
nothing in either repo failing to say so.
|
|
||||||
|
|
||||||
- **A ZWJ hidden between two emoji is exempt, and ZWNJ's own false-positive
|
- **A ZWJ hidden between two emoji is exempt, and ZWNJ's own false-positive
|
||||||
class is untouched.** U+200D composes emoji (👩💻 is WOMAN + ZWJ + PERSONAL
|
class is untouched.** U+200D composes emoji (👩💻 is WOMAN + ZWJ + PERSONAL
|
||||||
COMPUTER), so testing it on codepoint membership alone flagged *and stripped*
|
COMPUTER), so testing it on codepoint membership alone flagged *and stripped*
|
||||||
|
|
@ -771,22 +546,6 @@ fails the test, forcing this doc to be updated:
|
||||||
not pictographic) and no corpus is available here to verify it against, so it
|
not pictographic) and no corpus is available here to verify it against, so it
|
||||||
is parked as a known false-positive class rather than guessed at.
|
is parked as a known false-positive class rather than guessed at.
|
||||||
|
|
||||||
- **That context test is one predicate on two surfaces, and the symbol carrying it is
|
|
||||||
private.** `sanitize` owns `_is_joiner_in_emoji_sequence`; `output` imports it
|
|
||||||
(`output.py:74`) instead of restating it, because the same defect had to be fixed on
|
|
||||||
both surfaces and a split would let the input side stop flagging while the output
|
|
||||||
side kept hard-blocking — or the reverse, which is how a carrier reaches a persisted
|
|
||||||
artifact after passing the input gate. The agreement is pinned by
|
|
||||||
`tests/test_output.py::test_output_zwj_narrowing_matches_the_sanitize_side`, which
|
|
||||||
asserts `stripped == flagged` across six shapes — half-context on either side, a
|
|
||||||
leading and a trailing joiner, one genuine in-sequence joiner, and a word split.
|
|
||||||
Two things that pin does not give. The six shapes are hand-written rather than
|
|
||||||
drawn from a corpus, so everywhere outside them
|
|
||||||
the surfaces agree by *shared implementation*, not by test — which is the stronger
|
|
||||||
guarantee only for as long as the import survives. And the leading underscore means
|
|
||||||
the predicate is **not** part of the surface frozen under semver: a consumer that
|
|
||||||
imports it is pinning a private name 1.x makes no promise about.
|
|
||||||
|
|
||||||
## Out-of-scope (documented boundary)
|
## Out-of-scope (documented boundary)
|
||||||
|
|
||||||
Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
|
Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
|
||||||
|
|
|
||||||
|
|
@ -37,30 +37,17 @@ import importlib
|
||||||
import json
|
import json
|
||||||
import re
|
import re
|
||||||
import sys
|
import sys
|
||||||
|
import time
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parent.parent
|
SRC = Path(__file__).resolve().parent.parent / "src" / "llm_ingestion_guard"
|
||||||
SRC = ROOT / "src" / "llm_ingestion_guard"
|
|
||||||
sys.path.insert(0, str(SRC.parent))
|
sys.path.insert(0, str(SRC.parent))
|
||||||
sys.path.insert(0, str(ROOT / "tests"))
|
|
||||||
|
|
||||||
from llm_ingestion_guard.lexicon import load_lexicon # noqa: E402
|
from llm_ingestion_guard.lexicon import load_lexicon # noqa: E402
|
||||||
from redos_clock import scan_seconds # noqa: E402
|
|
||||||
|
|
||||||
N1, N2 = 4_000, 8_000
|
N1, N2 = 4_000, 8_000
|
||||||
RATIO_FLAG = 2.6
|
RATIO_FLAG = 2.6
|
||||||
# RE-DERIVED on the CPU clock, not inherited from the wall clock this script used
|
|
||||||
# through 1.1.0. The clock move fixed false REDS in the suite's bounds; it bought
|
|
||||||
# this sweep no sensitivity. Over twelve full runs (2585 arms each) the median
|
|
||||||
# ratio is 1.95-2.03 in every size bucket above 50 us -- the whole surface
|
|
||||||
# measures linear -- yet two-point excursions past RATIO_FLAG survive at every
|
|
||||||
# magnitude (p99 ratio 2.9-3.3 even above 1 ms). Flagged arms per run by floor:
|
|
||||||
# 6.9 at 0.5 ms, 1.1 at 1.0 ms, 0.33 at 1.5 ms. The knee is here. Four arms
|
|
||||||
# flagged across those twelve runs, each in exactly ONE of them, and six arms
|
|
||||||
# that have ever flagged re-measure at exponent 0.97-1.09 over six doublings.
|
|
||||||
# Descheduling was never what made this sweep noisy -- a two-point ratio is, so
|
|
||||||
# read a clean run and a flagged run with the same suspicion.
|
|
||||||
NOISE_FLOOR = 0.0015
|
NOISE_FLOOR = 0.0015
|
||||||
HARD_CAP = 20.0
|
HARD_CAP = 20.0
|
||||||
|
|
||||||
|
|
@ -147,35 +134,21 @@ def build(unit: str, n: int) -> str:
|
||||||
|
|
||||||
|
|
||||||
def t(rx: re.Pattern[str], text: str, mode: str = "search") -> float:
|
def t(rx: re.Pattern[str], text: str, mode: str = "search") -> float:
|
||||||
"""Time one scan of ``text`` in the mode the production code actually uses.
|
"""Time one scan of ``text`` in the mode the production code actually uses."""
|
||||||
|
|
||||||
On the SAME clock every ReDoS bound in the suite is measured against --
|
|
||||||
``tests/redos_clock.py``, process CPU time -- imported rather than restated
|
|
||||||
here, for the reason that module gives: a blowup is spent cycles, and a
|
|
||||||
loaded machine steals wall clock without adding any. Until 1.1.0 this timed
|
|
||||||
on ``time.monotonic()``, which made the floor below and the cap figure
|
|
||||||
derived from it numbers from a different instrument than the bounds they
|
|
||||||
justify. The closure is built BEFORE the clock starts, so only the scan is
|
|
||||||
charged.
|
|
||||||
"""
|
|
||||||
mode = mode.rstrip("*")
|
mode = mode.rstrip("*")
|
||||||
|
start = time.monotonic()
|
||||||
if mode == "finditer":
|
if mode == "finditer":
|
||||||
def scan(s: str) -> None:
|
for _ in rx.finditer(text):
|
||||||
for _ in rx.finditer(s):
|
|
||||||
pass
|
pass
|
||||||
elif mode == "sub":
|
elif mode == "sub":
|
||||||
def scan(s: str) -> None:
|
rx.sub("", text)
|
||||||
rx.sub("", s)
|
|
||||||
elif mode == "match":
|
elif mode == "match":
|
||||||
def scan(s: str) -> None:
|
rx.match(text)
|
||||||
rx.match(s)
|
|
||||||
elif mode == "fullmatch":
|
elif mode == "fullmatch":
|
||||||
def scan(s: str) -> None:
|
rx.fullmatch(text)
|
||||||
rx.fullmatch(s)
|
|
||||||
else:
|
else:
|
||||||
def scan(s: str) -> None:
|
rx.search(text)
|
||||||
rx.search(s)
|
return time.monotonic() - start
|
||||||
return scan_seconds(scan, text)
|
|
||||||
|
|
||||||
|
|
||||||
# --- targets ----------------------------------------------------------------
|
# --- targets ----------------------------------------------------------------
|
||||||
|
|
|
||||||
7
llms.txt
7
llms.txt
|
|
@ -1,7 +0,0 @@
|
||||||
# llm-ingestion-guard
|
|
||||||
|
|
||||||
> Write-time defensive layer for Python pipelines that persist LLM output: sanitize, fence, tool-less quarantined transform, capability isolation, scan before persist, fail-secure.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v1.1.0"
|
|
||||||
```
|
|
||||||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "llm-ingestion-guard"
|
name = "llm-ingestion-guard"
|
||||||
version = "1.3.0"
|
version = "1.1.0"
|
||||||
description = "Write-time defensive layer for Python pipelines that persist LLM output: sanitize, fence, tool-less quarantined transform, capability isolation, scan before persist, fail-secure."
|
description = "Write-time defensive layer for Python pipelines that persist LLM output: sanitize, fence, tool-less quarantined transform, capability isolation, scan before persist, fail-secure."
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.10"
|
requires-python = ">=3.10"
|
||||||
|
|
|
||||||
|
|
@ -63,7 +63,7 @@ from .grounding import (
|
||||||
)
|
)
|
||||||
from . import okf
|
from . import okf
|
||||||
|
|
||||||
__version__ = "1.3.0"
|
__version__ = "1.1.0"
|
||||||
|
|
||||||
|
|
||||||
# --- §6 bookends: the two library-side halves around the transform ---------
|
# --- §6 bookends: the two library-side halves around the transform ---------
|
||||||
|
|
|
||||||
|
|
@ -542,11 +542,8 @@ def _build_cases() -> list[Case]:
|
||||||
lambda: okf.parse_frontmatter("---\nkey:\n nested: x\n---\nbody\n"), owasp="LLM10"),
|
lambda: okf.parse_frontmatter("---\nkey:\n nested: x\n---\nbody\n"), owasp="LLM10"),
|
||||||
_raise_case("okf", "T2 frontmatter block scalar", "OKFFrontmatterError",
|
_raise_case("okf", "T2 frontmatter block scalar", "OKFFrontmatterError",
|
||||||
lambda: okf.parse_frontmatter("---\ndesc: |\n block\n---\nbody\n"), owasp="LLM10"),
|
lambda: okf.parse_frontmatter("---\ndesc: |\n block\n---\nbody\n"), owasp="LLM10"),
|
||||||
_raise_case("okf", "T2 frontmatter flow sequence", "OKFFrontmatterError",
|
_raise_case("okf", "T2 frontmatter flow collection", "OKFFrontmatterError",
|
||||||
lambda: okf.parse_frontmatter("---\ntags: [a, b]\n---\nbody\n"), owasp="LLM10"),
|
lambda: okf.parse_frontmatter("---\ntags: [a, b]\n---\nbody\n"), owasp="LLM10"),
|
||||||
_raise_case("okf", "T2 mapping key off the allowlist", "OKFFrontmatterError",
|
|
||||||
lambda: okf.parse_frontmatter(
|
|
||||||
"---\ngenerated: { by: a, tool: shell }\n---\nbody\n"), owasp="LLM10"),
|
|
||||||
_raise_case("okf", "T3 resource non-https (http)", "OKFResourceError",
|
_raise_case("okf", "T3 resource non-https (http)", "OKFResourceError",
|
||||||
lambda: okf.validate_resource_url("http://insecure.test/x"), owasp="LLM05"),
|
lambda: okf.validate_resource_url("http://insecure.test/x"), owasp="LLM05"),
|
||||||
_raise_case("okf", "T3 resource data: scheme", "OKFResourceError",
|
_raise_case("okf", "T3 resource data: scheme", "OKFResourceError",
|
||||||
|
|
|
||||||
|
|
@ -7,8 +7,7 @@ and feeds scannable text regions into the existing ``sanitize`` / ``scan_output`
|
||||||
|
|
||||||
T2 — frontmatter parse-safety gate. ``parse_frontmatter`` is a *strict,
|
T2 — frontmatter parse-safety gate. ``parse_frontmatter`` is a *strict,
|
||||||
reject-by-default* loader for the minimal OKF frontmatter subset: flat
|
reject-by-default* loader for the minimal OKF frontmatter subset: flat
|
||||||
``key: value`` scalars, block ``- item`` lists, and one typed, allowlisted
|
``key: value`` scalars plus block ``- item`` lists. Every construct the
|
||||||
mapping form (``{ by: x, at: y }`` — see :func:`_parse_flow_mapping`). Every construct the
|
|
||||||
"block anchor/alias DoS + dangerous type coercion" requirement names is refused
|
"block anchor/alias DoS + dangerous type coercion" requirement names is refused
|
||||||
*by construction* — you cannot suffer a billion-laughs alias expansion or a
|
*by construction* — you cannot suffer a billion-laughs alias expansion or a
|
||||||
``!!python/object`` coercion if anchors, aliases and explicit tags are rejected
|
``!!python/object`` coercion if anchors, aliases and explicit tags are rejected
|
||||||
|
|
@ -17,9 +16,7 @@ philosophy, the frontmatter analogue of the ``resource`` reject-gate (T3).
|
||||||
|
|
||||||
Deliberately NOT a general YAML parser. A security tool whose thesis is
|
Deliberately NOT a general YAML parser. A security tool whose thesis is
|
||||||
minimal-dependency should not pull in a full YAML engine whose own features
|
minimal-dependency should not pull in a full YAML engine whose own features
|
||||||
(anchors, tags, merges) are the attack surface being defended against. The one
|
(anchors, tags, merges) are the attack surface being defended against. Quoted
|
||||||
mapping form it does admit is admitted key-by-key against an allowlist, not
|
|
||||||
parsed generally: the mapping class is expressible, never trusted. Quoted
|
|
||||||
scalars are kept verbatim (quotes included) rather than unquoted — the value is
|
scalars are kept verbatim (quotes included) rather than unquoted — the value is
|
||||||
still scanned as text downstream, so an injection inside a quoted value is not
|
still scanned as text downstream, so an injection inside a quoted value is not
|
||||||
lost; richer scalar forms are a future refinement, not a silent parse.
|
lost; richer scalar forms are a future refinement, not a silent parse.
|
||||||
|
|
@ -65,49 +62,13 @@ _KEY_RE = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9_-]*$")
|
||||||
# A plain OKF scalar cannot *begin* with a YAML structural indicator. Any value
|
# A plain OKF scalar cannot *begin* with a YAML structural indicator. Any value
|
||||||
# starting with one signals an anchor (&), alias (*), explicit tag (!), block
|
# starting with one signals an anchor (&), alias (*), explicit tag (!), block
|
||||||
# scalar (|, >), flow collection ([ ] { }), directive (%) or reserved char
|
# scalar (|, >), flow collection ([ ] { }), directive (%) or reserved char
|
||||||
# (@ `) — all outside the supported subset and all rejected. `{` and `[` are
|
# (@ `) — all outside the supported subset and all rejected.
|
||||||
# tried as the allowlisted mapping form (G3) and the flow sequence of them (G30)
|
|
||||||
# FIRST; they reach this predicate only as a leaf inside one, where a nested
|
|
||||||
# collection is refused before it can be read.
|
|
||||||
_DANGEROUS_VALUE_STARTS = frozenset("&*!|>[]{}%@`")
|
_DANGEROUS_VALUE_STARTS = frozenset("&*!|>[]{}%@`")
|
||||||
# A quoted scalar is a scalar in YAML however many colons it carries, so the
|
# A quoted scalar is a scalar in YAML however many colons it carries, so the
|
||||||
# mapping check steps aside for one. The quotes are retained rather than
|
# mapping check steps aside for one. The quotes are retained rather than
|
||||||
# stripped — a pre-existing divergence, pinned in tests/test_okf.py.
|
# stripped — a pre-existing divergence, pinned in tests/test_okf.py.
|
||||||
_QUOTE_STARTS = frozenset("\"'")
|
_QUOTE_STARTS = frozenset("\"'")
|
||||||
|
|
||||||
# G3 - the one mapping form T2 can express (operator decision, 2026-08-21).
|
|
||||||
# Every key inside a mapping must be on this allowlist: the form is safe because
|
|
||||||
# the allowlist inspects each key, not because mappings became trusted. The keys
|
|
||||||
# are the ones OKF v0.2 names inside a mapping - `by`/`at` (SPEC.md @ 62432a09
|
|
||||||
# §5.2 `generated`/`verified`) and `from`/`to` (§5.1 `usage_window`), plus the
|
|
||||||
# §5.1 `sources`-entry labels.
|
|
||||||
_MAPPING_KEY_ALLOWLIST = frozenset({
|
|
||||||
"by", "at", "from", "to", "id", "title", "author", "usage_count",
|
|
||||||
"last_modified",
|
|
||||||
})
|
|
||||||
|
|
||||||
# G30 - the two §5.1 keys admitted inside a `sources` entry and NOWHERE else
|
|
||||||
# (operator decision, 2026-09-02). `resource` is REQUIRED within a `sources`
|
|
||||||
# entry, so leaving it off left the whole provenance family unwritable; but the
|
|
||||||
# same field name in §10 (`executor.resource`, `attester.resource`) names run
|
|
||||||
# instructions and code - the door-C route closed in 1.1.0. 1.2.0 argued the
|
|
||||||
# parser could not tell the two apart without parent-key context it did not
|
|
||||||
# have. That premise was false: the owning key is in scope at every call site
|
|
||||||
# below, it was simply never threaded through. It is threaded now, so the
|
|
||||||
# discrimination is structural rather than a judgement about the value.
|
|
||||||
# `usage_window` is allowlisted here for accuracy of refusal - §5.1 permits it
|
|
||||||
# per entry, and it is then refused on the depth rule (a mapping inside a
|
|
||||||
# mapping, which this parser admits at no key) rather than refused as if the
|
|
||||||
# key were unknown.
|
|
||||||
_SOURCES_ENTRY_KEYS = frozenset({"resource", "usage_window"})
|
|
||||||
|
|
||||||
|
|
||||||
def _allowed_mapping_keys(parent_key):
|
|
||||||
"""The mapping-key allowlist for a mapping owned by ``parent_key``."""
|
|
||||||
if parent_key == "sources":
|
|
||||||
return _MAPPING_KEY_ALLOWLIST | _SOURCES_ENTRY_KEYS
|
|
||||||
return _MAPPING_KEY_ALLOWLIST
|
|
||||||
|
|
||||||
|
|
||||||
class OKFError(Exception):
|
class OKFError(Exception):
|
||||||
"""Base class for OKF adapter rejections."""
|
"""Base class for OKF adapter rejections."""
|
||||||
|
|
@ -145,10 +106,7 @@ def parse_frontmatter(document):
|
||||||
|
|
||||||
Raises ``OKFFrontmatterError`` on an unterminated fence or any construct
|
Raises ``OKFFrontmatterError`` on an unterminated fence or any construct
|
||||||
outside the minimal flat subset (anchors, aliases, explicit tags, merge
|
outside the minimal flat subset (anchors, aliases, explicit tags, merge
|
||||||
keys, block scalars, flow sequences, nested mappings). The single exception
|
keys, block scalars, flow collections, nested mappings).
|
||||||
is the typed, allowlisted flow mapping (:func:`_parse_flow_mapping`), which
|
|
||||||
parses into a ``dict`` of allowlisted keys with plain-scalar leaves — every
|
|
||||||
other route to a mapping still raises.
|
|
||||||
"""
|
"""
|
||||||
lines = document.split("\n")
|
lines = document.split("\n")
|
||||||
if not lines or lines[0].strip() != _FENCE:
|
if not lines or lines[0].strip() != _FENCE:
|
||||||
|
|
@ -191,29 +149,11 @@ def _scannable_regions(frontmatter, body):
|
||||||
"""The text regions of a concept that carry attacker-controlled content."""
|
"""The text regions of a concept that carry attacker-controlled content."""
|
||||||
regions = [body]
|
regions = [body]
|
||||||
for value in frontmatter.values():
|
for value in frontmatter.values():
|
||||||
regions.extend(_value_regions(value))
|
|
||||||
return regions
|
|
||||||
|
|
||||||
|
|
||||||
def _value_regions(value):
|
|
||||||
"""Every scannable leaf of one frontmatter value.
|
|
||||||
|
|
||||||
A mapping value (G3) is a new *shape* on this surface, not a new exemption:
|
|
||||||
its leaves are scanned exactly like a scalar or a list item, so an injection
|
|
||||||
parked in ``generated: { by: ... }`` reaches ``scan_output`` like any other
|
|
||||||
frontmatter text. The same holds for a *list* of mappings (G30, ``sources``),
|
|
||||||
which this function already flattens through its list branch. Mapping *keys*
|
|
||||||
are not scanned because they cannot carry attacker text - the allowlist
|
|
||||||
admits a fixed, per-parent name set and nothing else.
|
|
||||||
"""
|
|
||||||
if isinstance(value, dict):
|
|
||||||
return [leaf for leaf in value.values() if leaf]
|
|
||||||
if isinstance(value, list):
|
if isinstance(value, list):
|
||||||
regions = []
|
regions.extend(value)
|
||||||
for item in value:
|
elif value:
|
||||||
regions.extend(_value_regions(item))
|
regions.append(value)
|
||||||
return regions
|
return regions
|
||||||
return [value] if value else []
|
|
||||||
|
|
||||||
|
|
||||||
def validate_concept_path(path, *, allow_reserved=False):
|
def validate_concept_path(path, *, allow_reserved=False):
|
||||||
|
|
@ -635,22 +575,10 @@ def _parse_flat(fm_lines):
|
||||||
raise OKFFrontmatterError("invalid frontmatter key: %r" % key)
|
raise OKFFrontmatterError("invalid frontmatter key: %r" % key)
|
||||||
|
|
||||||
if value == "":
|
if value == "":
|
||||||
items, i = _consume_block_list(fm_lines, i + 1, key)
|
items, i = _consume_block_list(fm_lines, i + 1)
|
||||||
result[key] = items if items is not None else ""
|
result[key] = items if items is not None else ""
|
||||||
continue
|
continue
|
||||||
|
|
||||||
mapping = _parse_flow_mapping(value, key)
|
|
||||||
if mapping is not None:
|
|
||||||
result[key] = mapping
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
|
|
||||||
sequence = _parse_flow_sequence(value, key)
|
|
||||||
if sequence is not None:
|
|
||||||
result[key] = sequence
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
|
|
||||||
_reject_dangerous_value(value)
|
_reject_dangerous_value(value)
|
||||||
_reject_mapping_construct(value)
|
_reject_mapping_construct(value)
|
||||||
result[key] = value
|
result[key] = value
|
||||||
|
|
@ -659,26 +587,14 @@ def _parse_flat(fm_lines):
|
||||||
return result
|
return result
|
||||||
|
|
||||||
|
|
||||||
def _consume_block_list(fm_lines, start, parent_key=None):
|
def _consume_block_list(fm_lines, start):
|
||||||
"""Consume `` - item`` lines following a bare ``key:``.
|
"""Consume `` - item`` lines following a bare ``key:``.
|
||||||
|
|
||||||
Returns ``(items, next_index)`` - ``items`` is ``None`` (and ``next_index``
|
Returns ``(items, next_index)`` — ``items`` is ``None`` (and ``next_index``
|
||||||
unchanged) when no list item follows, so the caller can treat the key as an
|
unchanged) when no list item follows, so the caller can treat the key as an
|
||||||
empty scalar and let the next line trip the nested-structure guard.
|
empty scalar and let the next line trip the nested-structure guard.
|
||||||
|
|
||||||
An item is one of three shapes, decided by the item text alone: a flow
|
|
||||||
mapping (G3), a block mapping (G30 - an unquoted ``key: value`` opening a
|
|
||||||
run of more-indented sibling entries, which is SPEC.md §5.1's own carrier
|
|
||||||
for ``sources``), or a plain scalar. ``parent_key`` is the key that owns the
|
|
||||||
list; it decides the mapping-key allowlist, which is how ``sources[].resource``
|
|
||||||
is admitted while ``executor``/``attester`` ``resource`` stays refused.
|
|
||||||
|
|
||||||
A list may not mix scalars and mappings. YAML permits it, but a consumer
|
|
||||||
iterating ``sources`` and reading ``entry.get("id")`` gets an
|
|
||||||
``AttributeError`` off the first ``str`` - refusing is the cheaper failure.
|
|
||||||
"""
|
"""
|
||||||
items = []
|
items = []
|
||||||
kinds = set()
|
|
||||||
i = start
|
i = start
|
||||||
n = len(fm_lines)
|
n = len(fm_lines)
|
||||||
while i < n:
|
while i < n:
|
||||||
|
|
@ -687,113 +603,19 @@ def _consume_block_list(fm_lines, start, parent_key=None):
|
||||||
if stripped == "" or stripped.startswith("#"):
|
if stripped == "" or stripped.startswith("#"):
|
||||||
i += 1
|
i += 1
|
||||||
continue
|
continue
|
||||||
if not (raw[:1] in (" ", "\t") and stripped.startswith("- ")):
|
if raw[:1] in (" ", "\t") and stripped.startswith("- "):
|
||||||
break
|
|
||||||
|
|
||||||
item = stripped[2:].strip()
|
item = stripped[2:].strip()
|
||||||
|
|
||||||
mapping = _parse_flow_mapping(item, parent_key)
|
|
||||||
if mapping is not None:
|
|
||||||
items.append(mapping)
|
|
||||||
kinds.add("mapping")
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
|
|
||||||
entry = _block_mapping_entry(item)
|
|
||||||
if entry is not None:
|
|
||||||
mapping, i = _consume_block_mapping(fm_lines, i + 1, entry, parent_key)
|
|
||||||
items.append(mapping)
|
|
||||||
kinds.add("mapping")
|
|
||||||
continue
|
|
||||||
|
|
||||||
_reject_dangerous_value(item)
|
_reject_dangerous_value(item)
|
||||||
_reject_mapping_construct(item)
|
_reject_mapping_construct(item)
|
||||||
items.append(item)
|
items.append(item)
|
||||||
kinds.add("scalar")
|
|
||||||
i += 1
|
i += 1
|
||||||
|
continue
|
||||||
if len(kinds) > 1:
|
break
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"a block list may not mix scalar items and mappings: %r" % (parent_key,)
|
|
||||||
)
|
|
||||||
if not items:
|
if not items:
|
||||||
return None, start
|
return None, start
|
||||||
return items, i
|
return items, i
|
||||||
|
|
||||||
|
|
||||||
def _block_mapping_entry(text):
|
|
||||||
"""Read ``text`` as one ``key: value`` block-mapping entry, or return ``None``.
|
|
||||||
|
|
||||||
The trigger is deliberately the same shape ``_reject_mapping_construct``
|
|
||||||
uses to *refuse* a scalar: an unquoted ``": "``. What changes in 1.3.0 is
|
|
||||||
only what happens next - the entry is admitted key-by-key against the
|
|
||||||
allowlist instead of refused wholesale. Every shape that is a scalar to
|
|
||||||
PyYAML stays one here: a quoted item, a colon with no space
|
|
||||||
(``domain:security``, ``https://e.com:8443/a``) and a trailing colon all
|
|
||||||
return ``None`` and fall through to the unchanged scalar rules.
|
|
||||||
"""
|
|
||||||
if not text or text[0] in _QUOTE_STARTS:
|
|
||||||
return None
|
|
||||||
key, sep, leaf = text.partition(": ")
|
|
||||||
if not sep:
|
|
||||||
return None
|
|
||||||
key = key.strip()
|
|
||||||
if not _KEY_RE.match(key):
|
|
||||||
return None
|
|
||||||
return key, leaf.strip()
|
|
||||||
|
|
||||||
|
|
||||||
def _consume_block_mapping(fm_lines, start, first_entry, parent_key):
|
|
||||||
"""Consume the sibling entries of a block mapping opened by a ``- `` item.
|
|
||||||
|
|
||||||
Returns ``(mapping, next_index)``. A sibling is an indented line that does
|
|
||||||
not open a new list item; the run ends at a blank line, a comment, a new
|
|
||||||
``- `` item, or a line at column zero. Depth is capped at one by giving the
|
|
||||||
leaves the *unchanged* scalar predicates: a nested collection opens with
|
|
||||||
``{`` or ``[`` and is refused by ``_reject_dangerous_value``, and a further
|
|
||||||
block level is refused by ``_reject_mapping_construct``.
|
|
||||||
"""
|
|
||||||
allowed = _allowed_mapping_keys(parent_key)
|
|
||||||
mapping = {}
|
|
||||||
_admit_mapping_entry(mapping, first_entry[0], first_entry[1], allowed, parent_key)
|
|
||||||
|
|
||||||
i = start
|
|
||||||
n = len(fm_lines)
|
|
||||||
while i < n:
|
|
||||||
raw = fm_lines[i]
|
|
||||||
stripped = raw.strip()
|
|
||||||
if stripped == "" or stripped.startswith("#"):
|
|
||||||
break
|
|
||||||
if raw[:1] not in (" ", "\t") or stripped.startswith("- "):
|
|
||||||
break
|
|
||||||
entry = _block_mapping_entry(stripped)
|
|
||||||
if entry is None:
|
|
||||||
_reject_dangerous_value(stripped)
|
|
||||||
_reject_mapping_construct(stripped)
|
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"a block-mapping entry must be 'key: value': %r" % (raw,)
|
|
||||||
)
|
|
||||||
_admit_mapping_entry(mapping, entry[0], entry[1], allowed, parent_key)
|
|
||||||
i += 1
|
|
||||||
return mapping, i
|
|
||||||
|
|
||||||
|
|
||||||
def _admit_mapping_entry(mapping, key, leaf, allowed, parent_key):
|
|
||||||
"""Admit one mapping entry, or raise. The single gate both carriers pass."""
|
|
||||||
if not _KEY_RE.match(key):
|
|
||||||
raise OKFFrontmatterError("invalid mapping key: %r" % (key,))
|
|
||||||
if key not in allowed:
|
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"mapping key %r is not on the OKF mapping allowlist under %r"
|
|
||||||
% (key, parent_key)
|
|
||||||
)
|
|
||||||
if key in mapping:
|
|
||||||
raise OKFFrontmatterError("duplicate mapping key %r" % (key,))
|
|
||||||
_reject_dangerous_value(leaf)
|
|
||||||
_reject_mapping_construct(leaf)
|
|
||||||
mapping[key] = leaf
|
|
||||||
|
|
||||||
|
|
||||||
def _reject_dangerous_value(value):
|
def _reject_dangerous_value(value):
|
||||||
if value and value[0] in _DANGEROUS_VALUE_STARTS:
|
if value and value[0] in _DANGEROUS_VALUE_STARTS:
|
||||||
raise OKFFrontmatterError(
|
raise OKFFrontmatterError(
|
||||||
|
|
@ -805,10 +627,8 @@ def _reject_dangerous_value(value):
|
||||||
def _reject_mapping_construct(value):
|
def _reject_mapping_construct(value):
|
||||||
"""Reject a scalar that YAML reads as a mapping rather than as a string.
|
"""Reject a scalar that YAML reads as a mapping rather than as a string.
|
||||||
|
|
||||||
T2 gives the mapping *class* exactly one expressible form, the typed
|
T2 gives the mapping *class* no expressible form — flow, nested-block and
|
||||||
allowlisted flow mapping (G3); the nested-block and dotted-key routes still
|
dotted-key routes all raise. Two routes used to escape that by degrading
|
||||||
raise, and this predicate is what keeps them raising — both at the top level
|
|
||||||
and on a leaf *inside* an admitted mapping. Two routes used to escape by degrading
|
|
||||||
into a string instead: a block-sequence item carrying exactly one key
|
into a string instead: a block-sequence item carrying exactly one key
|
||||||
(``- uri: x``), and an inline second colon (``attester: resource: x``).
|
(``- uri: x``), and an inline second colon (``attester: resource: x``).
|
||||||
Both parsed "successfully" into the wrong *type*, and a pointer parked in
|
Both parsed "successfully" into the wrong *type*, and a pointer parked in
|
||||||
|
|
@ -828,148 +648,3 @@ def _reject_mapping_construct(value):
|
||||||
raise OKFFrontmatterError(
|
raise OKFFrontmatterError(
|
||||||
"a mapping is not expressible in OKF frontmatter: %r" % (value,)
|
"a mapping is not expressible in OKF frontmatter: %r" % (value,)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _parse_flow_mapping(value, parent_key=None):
|
|
||||||
"""Parse ``{ key: value, ... }`` into a typed dict, or refuse it (G3).
|
|
||||||
|
|
||||||
Returns ``None`` when ``value`` does not open a flow mapping, so the caller
|
|
||||||
falls through to the unchanged scalar rules. Otherwise the value either
|
|
||||||
parses into a ``dict`` of allowlisted keys with plain-scalar leaves, or
|
|
||||||
raises - it never degrades into a string, which is the defect closed in
|
|
||||||
1.1.0 and not reopened here.
|
|
||||||
|
|
||||||
Why the mapping class needed *a* form at all: OKF v0.2 writes its whole
|
|
||||||
trust and provenance layer as mappings, and SPEC.md @ ``62432a09`` uses flow
|
|
||||||
form in its own examples (§5.1 ``usage_window``, §5.2 ``generated`` /
|
|
||||||
``verified``). §11 goes further than "should": a consumer *MUST* treat a
|
|
||||||
bare ``verified`` mapping as a one-element list - a rule that presupposes
|
|
||||||
the mapping parses. With no form, 0 of 53 upstream concepts reached the
|
|
||||||
gate, and no threshold would have changed that.
|
|
||||||
|
|
||||||
Why this form is safe: the allowlist inspects **every key**, which is the
|
|
||||||
property that actually carried the security in T2 - the blanket refusal was
|
|
||||||
the enforcement, not the point. Admitted, ground-truthed against PyYAML
|
|
||||||
6.0.3:
|
|
||||||
|
|
||||||
- one flow mapping per value, closed on the same line (``{ a: b }``);
|
|
||||||
- keys on :data:`_MAPPING_KEY_ALLOWLIST` and matching ``_KEY_RE``, no
|
|
||||||
duplicates - PyYAML resolves a duplicate last-wins, which is a way to
|
|
||||||
show one claim and mean another;
|
|
||||||
- plain-scalar leaves only, each run through the *unchanged*
|
|
||||||
``_reject_dangerous_value`` / ``_reject_mapping_construct`` predicates, so
|
|
||||||
a leaf can no more open an anchor, a tag or a nested mapping than a
|
|
||||||
top-level scalar can.
|
|
||||||
|
|
||||||
Refused, each on its own rule: nested collections (``{ a: { b: c } }``,
|
|
||||||
``{ a: [1] }``), quoted leaves, an empty mapping, an unclosed or
|
|
||||||
trailing-junk value (``{ a: b } x``, which PyYAML also refuses), a key
|
|
||||||
outside the allowlist, and ``{a:b}`` - which PyYAML reads as the *key*
|
|
||||||
``a:b``, not as a scalar, and which the required ``": "`` separator catches.
|
|
||||||
|
|
||||||
Two deliberate divergences from PyYAML, both toward refusal: a quoted leaf
|
|
||||||
(``{ title: 'a, b' }``) and a trailing comment (``{ a: b } # note``) are
|
|
||||||
conformant YAML that this rejects. Splitting quoted commas correctly needs a
|
|
||||||
quote state machine whose failure mode is *accepting* something YAML would
|
|
||||||
refuse; refusing is the cheaper side to be wrong on, and the keys that
|
|
||||||
plausibly need a comma (``title``, ``author``) only occur inside ``sources``
|
|
||||||
entries, whose block-sequence carrier is refused anyway.
|
|
||||||
"""
|
|
||||||
if not value or value[0] != "{":
|
|
||||||
return None
|
|
||||||
if not value.endswith("}"):
|
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"a flow mapping must be closed by '}' on the same line: %r" % (value,)
|
|
||||||
)
|
|
||||||
|
|
||||||
inner = value[1:-1].strip()
|
|
||||||
if inner.endswith(","): # a trailing comma is legal YAML; one, and only one
|
|
||||||
inner = inner[:-1].strip()
|
|
||||||
if not inner:
|
|
||||||
raise OKFFrontmatterError("an empty flow mapping carries nothing: %r" % (value,))
|
|
||||||
for char in "{}[]":
|
|
||||||
if char in inner:
|
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"a flow mapping admits scalar leaves only, not %r: %r" % (char, value)
|
|
||||||
)
|
|
||||||
for quote in _QUOTE_STARTS:
|
|
||||||
if quote in inner:
|
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"a quoted scalar inside a flow mapping is not a supported form: %r"
|
|
||||||
% (value,)
|
|
||||||
)
|
|
||||||
|
|
||||||
allowed = _allowed_mapping_keys(parent_key)
|
|
||||||
mapping = {}
|
|
||||||
for entry in inner.split(","):
|
|
||||||
entry = entry.strip()
|
|
||||||
key, sep, leaf = entry.partition(": ")
|
|
||||||
if not sep:
|
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"a flow-mapping entry must be 'key: value': %r" % (entry,)
|
|
||||||
)
|
|
||||||
_admit_mapping_entry(mapping, key.strip(), leaf.strip(), allowed, parent_key)
|
|
||||||
return mapping
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_flow_sequence(value, parent_key=None):
|
|
||||||
"""Parse ``[{ ... }, { ... }]`` into a list of typed dicts, or refuse it (G30).
|
|
||||||
|
|
||||||
Returns ``None`` when ``value`` does not open a flow sequence, so the caller
|
|
||||||
falls through to the unchanged scalar rules - where ``[`` is still a
|
|
||||||
disallowed indicator. This carrier is opened for the flow-mapping element
|
|
||||||
and nothing else: it is the form the OKF producers emit for ``sources``
|
|
||||||
(measured 02.09 against llm-ingestion-okf's golden bundle, where a
|
|
||||||
one-element sequence raised on the ``[`` just as a two-element one did).
|
|
||||||
|
|
||||||
A flow sequence of plain *scalars* (``tags: [a, b, c]``) stays refused. It
|
|
||||||
is a different shape with its own quoting and comma-splitting problem, whose
|
|
||||||
failure mode would be accepting something YAML reads differently - and the
|
|
||||||
block-sequence carrier already covers it for every consumer measured so far.
|
|
||||||
|
|
||||||
Elements are split on ``}`` rather than on commas, which is sound precisely
|
|
||||||
because ``_parse_flow_mapping`` admits no nested collection: a ``}`` inside
|
|
||||||
an element cannot occur, so the first ``}`` after ``{`` always closes it.
|
|
||||||
Anything between elements that is not a separating comma is refused, which
|
|
||||||
is what makes trailing junk and a mixed sequence fail rather than parse.
|
|
||||||
"""
|
|
||||||
if not value or value[0] != "[":
|
|
||||||
return None
|
|
||||||
if not value.endswith("]"):
|
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"a flow sequence must be closed by ']' on the same line: %r" % (value,)
|
|
||||||
)
|
|
||||||
|
|
||||||
inner = value[1:-1].strip()
|
|
||||||
if not inner:
|
|
||||||
raise OKFFrontmatterError("an empty flow sequence carries nothing: %r" % (value,))
|
|
||||||
|
|
||||||
items = []
|
|
||||||
i = 0
|
|
||||||
n = len(inner)
|
|
||||||
while True:
|
|
||||||
while i < n and inner[i] in " \t":
|
|
||||||
i += 1
|
|
||||||
if i >= n:
|
|
||||||
break
|
|
||||||
if inner[i] != "{":
|
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"a flow sequence admits flow mappings only: %r" % (value,)
|
|
||||||
)
|
|
||||||
close = inner.find("}", i)
|
|
||||||
if close == -1:
|
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"an unclosed flow mapping inside a flow sequence: %r" % (value,)
|
|
||||||
)
|
|
||||||
items.append(_parse_flow_mapping(inner[i:close + 1], parent_key))
|
|
||||||
i = close + 1
|
|
||||||
while i < n and inner[i] in " \t":
|
|
||||||
i += 1
|
|
||||||
if i >= n:
|
|
||||||
break
|
|
||||||
if inner[i] != ",":
|
|
||||||
raise OKFFrontmatterError(
|
|
||||||
"trailing junk after a flow-sequence element: %r" % (value,)
|
|
||||||
)
|
|
||||||
i += 1
|
|
||||||
return items
|
|
||||||
|
|
|
||||||
|
|
@ -28,12 +28,10 @@ from pathlib import Path
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
import redos_clock
|
|
||||||
from llm_ingestion_guard import Disposition, PRESET_USER_UPLOAD, Risk, screen_output
|
from llm_ingestion_guard import Disposition, PRESET_USER_UPLOAD, Risk, screen_output
|
||||||
from llm_ingestion_guard import active_content as ac
|
from llm_ingestion_guard import active_content as ac
|
||||||
|
|
||||||
_DOCS = Path(__file__).resolve().parent.parent / "docs"
|
_DOCS = Path(__file__).resolve().parent.parent / "docs"
|
||||||
_LIMITATIONS = _DOCS / "LIMITATIONS.md"
|
|
||||||
|
|
||||||
|
|
||||||
def _load(filename: str):
|
def _load(filename: str):
|
||||||
|
|
@ -43,7 +41,6 @@ def _load(filename: str):
|
||||||
spec = importlib.util.spec_from_file_location(name, path)
|
spec = importlib.util.spec_from_file_location(name, path)
|
||||||
assert spec and spec.loader, f"cannot load {path}"
|
assert spec and spec.loader, f"cannot load {path}"
|
||||||
module = importlib.util.module_from_spec(spec)
|
module = importlib.util.module_from_spec(spec)
|
||||||
sys.modules[name] = module # @dataclass resolves its own module via sys.modules
|
|
||||||
spec.loader.exec_module(module)
|
spec.loader.exec_module(module)
|
||||||
return module
|
return module
|
||||||
|
|
||||||
|
|
@ -52,7 +49,6 @@ def _load(filename: str):
|
||||||
# script's import list fails the whole file loudly instead of one quiet test.
|
# script's import list fails the whole file loudly instead of one quiet test.
|
||||||
fp_sweep = _load("fp-sweep.py")
|
fp_sweep = _load("fp-sweep.py")
|
||||||
census = _load("rawhtml-census.py")
|
census = _load("rawhtml-census.py")
|
||||||
redos_sweep = _load("redos-sweep.py")
|
|
||||||
|
|
||||||
|
|
||||||
# --- docs/fp-sweep.py --------------------------------------------------------
|
# --- docs/fp-sweep.py --------------------------------------------------------
|
||||||
|
|
@ -249,38 +245,6 @@ def test_census_production_row_equals_its_shipped_candidate(name, attrs):
|
||||||
assert candidate(name, attrs) == ac.active_tag_class(name, attrs)
|
assert candidate(name, attrs) == ac.active_tag_class(name, attrs)
|
||||||
|
|
||||||
|
|
||||||
# --- docs/redos-sweep.py ------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_redos_sweep_times_on_the_suite_clock_not_a_reimplementation():
|
|
||||||
# Until 1.1.0 this script timed on `time.monotonic()`, a different instrument
|
|
||||||
# than every ReDoS bound in the suite. `t()` must call the shared
|
|
||||||
# `scan_seconds` — imported, not restated — and the module must not import
|
|
||||||
# `time` itself, else a drift back to a wall clock would go unnoticed here.
|
|
||||||
assert redos_sweep.scan_seconds is redos_clock.scan_seconds
|
|
||||||
assert not hasattr(redos_sweep, "time"), "module must not import time itself"
|
|
||||||
|
|
||||||
|
|
||||||
def test_redos_sweep_floor_and_flag_match_the_published_numbers():
|
|
||||||
# docs/LIMITATIONS.md publishes the 1.5 ms floor and the 2.6 flag ratio this
|
|
||||||
# script derives from twelve full runs. Pin both sides: the constants, and
|
|
||||||
# that the doc still states the same numbers — either drifting alone is a bug.
|
|
||||||
assert redos_sweep.NOISE_FLOOR == 0.0015
|
|
||||||
assert redos_sweep.RATIO_FLAG == 2.6
|
|
||||||
text = _LIMITATIONS.read_text(encoding="utf-8")
|
|
||||||
assert "1.5 ms noise floor" in text
|
|
||||||
assert "2.6 flag threshold" in text
|
|
||||||
|
|
||||||
|
|
||||||
def test_redos_sweep_collector_covers_152_patterns_across_11_tables():
|
|
||||||
# The count docs/LIMITATIONS.md carries as "all 152 compiled patterns across
|
|
||||||
# all eleven regex-bearing modules". A pattern added or removed in `src/`
|
|
||||||
# without re-measuring would drift the doc's claim silently otherwise.
|
|
||||||
assert len(redos_sweep.TABLES) == 11
|
|
||||||
total = sum(len(collect()) for collect in redos_sweep.TABLES.values())
|
|
||||||
assert total == 152
|
|
||||||
|
|
||||||
|
|
||||||
# --- both scripts: the argument-less contract --------------------------------
|
# --- both scripts: the argument-less contract --------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -548,10 +548,7 @@ _V02_REJECTED = [
|
||||||
("generated (nested)", "generated:\n at: 2026-07-26T10:00:00Z\n"),
|
("generated (nested)", "generated:\n at: 2026-07-26T10:00:00Z\n"),
|
||||||
("executor (nested)", "executor:\n resource: skills/run-on-bq.md\n"),
|
("executor (nested)", "executor:\n resource: skills/run-on-bq.md\n"),
|
||||||
("attester (nested)", "attester:\n resource: attesters/sql_equality.py\n"),
|
("attester (nested)", "attester:\n resource: attesters/sql_equality.py\n"),
|
||||||
# The carrier is admitted as of 1.3.0 (G30); this row now measures the KEY
|
("sources (block list of mappings)",
|
||||||
# SET - `uri`/`kind` are producer-invented, not SPEC §5.1 - and stays red for
|
|
||||||
# that reason. SPEC's own §5.1 keys parse; see the G30 block at the foot.
|
|
||||||
("sources (block list, off-allowlist keys)",
|
|
||||||
"sources:\n - uri: https://e.com/a\n kind: doc\n"),
|
"sources:\n - uri: https://e.com/a\n kind: doc\n"),
|
||||||
("flow sequence", "tags: [a, b, c]\n"),
|
("flow sequence", "tags: [a, b, c]\n"),
|
||||||
("flow mapping", "executor: {resource: skills/run.md}\n"),
|
("flow mapping", "executor: {resource: skills/run.md}\n"),
|
||||||
|
|
@ -589,8 +586,6 @@ def test_v02_flat_frontmatter_still_parses(cid, fm):
|
||||||
|
|
||||||
_DEGRADED_TO_STRING = [
|
_DEGRADED_TO_STRING = [
|
||||||
# (id, frontmatter, what PyYAML 6.0.3 makes of it)
|
# (id, frontmatter, what PyYAML 6.0.3 makes of it)
|
||||||
# Still red after G30 opened the block-mapping carrier: `uri` is not a §5.1
|
|
||||||
# key, so the item is refused by the allowlist instead of by the grammar.
|
|
||||||
("one key per item", "sources:\n - uri: https://e.com/a\n", "[{'uri': ...}]"),
|
("one key per item", "sources:\n - uri: https://e.com/a\n", "[{'uri': ...}]"),
|
||||||
("item, trailing colon", "sources:\n - uri:\n", "[{'uri': None}]"),
|
("item, trailing colon", "sources:\n - uri:\n", "[{'uri': None}]"),
|
||||||
("inline double colon", "attester: resource: attesters/sql_equality.py\n", "parse error"),
|
("inline double colon", "attester: resource: attesters/sql_equality.py\n", "parse error"),
|
||||||
|
|
@ -601,11 +596,9 @@ _DEGRADED_TO_STRING = [
|
||||||
@pytest.mark.parametrize("cid,fm,yaml_reads_as", _DEGRADED_TO_STRING,
|
@pytest.mark.parametrize("cid,fm,yaml_reads_as", _DEGRADED_TO_STRING,
|
||||||
ids=[c[0] for c in _DEGRADED_TO_STRING])
|
ids=[c[0] for c in _DEGRADED_TO_STRING])
|
||||||
def test_a_mapping_construct_never_degrades_into_a_string(cid, fm, yaml_reads_as):
|
def test_a_mapping_construct_never_degrades_into_a_string(cid, fm, yaml_reads_as):
|
||||||
# None of these shapes is the one form T2 admits (G3, the allowlisted flow
|
# The mapping *class* has no expressible form through T2 — so a mapping
|
||||||
# mapping) — so each must RAISE, never parse "successfully" into the wrong
|
# construct must RAISE, never parse "successfully" into the wrong type. A
|
||||||
# type. A consumer reading frontmatter["sources"][0].get("uri") must not be
|
# consumer reading frontmatter["sources"][0].get("uri") must not be handed a str.
|
||||||
# handed a str, and that holds whether the mapping class has no expressible
|
|
||||||
# form or one.
|
|
||||||
with pytest.raises(OKFFrontmatterError):
|
with pytest.raises(OKFFrontmatterError):
|
||||||
parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")
|
parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")
|
||||||
|
|
||||||
|
|
@ -658,20 +651,13 @@ def test_pointer_in_a_degraded_mapping_no_longer_reaches_the_consumer_tree(cid,
|
||||||
assert result.disposition is Disposition.FAIL_SECURE, "hole reopened — see LIMITATIONS.md"
|
assert result.disposition is Disposition.FAIL_SECURE, "hole reopened — see LIMITATIONS.md"
|
||||||
|
|
||||||
|
|
||||||
def test_the_expressible_mapping_routes_are_the_ones_the_allowlist_inspects():
|
def test_every_route_to_a_mapping_fails_on_a_different_rule():
|
||||||
# Was `test_exactly_one_route_to_a_mapping_is_expressible` (1.2.0), and before
|
# The v0.2 wall is not a choice between two forms where one is better: ALL three
|
||||||
# that ALL FOUR routes failed so the mapping *class* had no expressible form.
|
# ways to express a mapping fail, each on its own rule, so the mapping *class* has
|
||||||
# There are four expressible carriers as of 1.3.0 - flow mapping as a value,
|
# no expressible form through T2. v0.2's `generated` IS a mapping (`by` required
|
||||||
# flow mapping as a list item, flow sequence of flow mappings, block sequence
|
# when present), so it cannot be expressed at all.
|
||||||
# of block mappings - and the criterion that admits them is unchanged: each
|
|
||||||
# hands the allowlist every key. The routes below stay shut for the same
|
|
||||||
# reason, each on its own rule: a top-level block MAPPING (not a sequence),
|
|
||||||
# a dotted key, and an inline second colon give the allowlist nothing to
|
|
||||||
# inspect.
|
|
||||||
assert parse_frontmatter("---\nid: x\ngenerated: { by: x, at: y }\n---\n\nbody\n")[0][
|
|
||||||
"generated"] == {"by": "x", "at": "y"}
|
|
||||||
|
|
||||||
routes = {
|
routes = {
|
||||||
|
"flow": "generated: { by: x, at: y }\n",
|
||||||
"block": "generated:\n by: x\n",
|
"block": "generated:\n by: x\n",
|
||||||
"dotted": "generated.by: x\n",
|
"dotted": "generated.by: x\n",
|
||||||
"inline": "generated: by: x\n",
|
"inline": "generated: by: x\n",
|
||||||
|
|
@ -681,10 +667,11 @@ def test_the_expressible_mapping_routes_are_the_ones_the_allowlist_inspects():
|
||||||
with pytest.raises(OKFFrontmatterError) as exc:
|
with pytest.raises(OKFFrontmatterError) as exc:
|
||||||
parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")
|
parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")
|
||||||
errors[name] = str(exc.value)
|
errors[name] = str(exc.value)
|
||||||
|
assert "indicator" in errors["flow"]
|
||||||
assert "nested mappings" in errors["block"]
|
assert "nested mappings" in errors["block"]
|
||||||
assert "key" in errors["dotted"]
|
assert "key" in errors["dotted"]
|
||||||
assert "mapping" in errors["inline"]
|
assert "mapping" in errors["inline"]
|
||||||
assert len(set(errors.values())) == 3, "routes must fail distinctly, not collapse"
|
assert len(set(errors.values())) == 4, "routes must fail distinctly, not collapse"
|
||||||
|
|
||||||
|
|
||||||
_BLOCK_LIST_ITEM_SHAPES = [
|
_BLOCK_LIST_ITEM_SHAPES = [
|
||||||
|
|
@ -703,33 +690,17 @@ def test_block_lists_admitted_by_item_shape(cid, fm, expected):
|
||||||
assert parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")[0][key] == expected
|
assert parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")[0][key] == expected
|
||||||
|
|
||||||
|
|
||||||
def test_the_block_list_rejects_on_the_key_set_not_on_arity():
|
def test_two_keys_per_item_is_where_the_block_list_hard_rejects():
|
||||||
# Was `test_two_keys_per_item_is_where_the_block_list_hard_rejects`: a
|
|
||||||
# two-key item was refused because the block mapping had no expressible form
|
|
||||||
# at all. G30 gives it one (SPEC.md §5.1's own carrier), so arity is no
|
|
||||||
# longer the boundary - the key set is. The same two-key item parses when its
|
|
||||||
# keys are §5.1's, and still raises when one of them is not.
|
|
||||||
parsed = parse_frontmatter(
|
|
||||||
"---\nid: x\nsources:\n - id: a\n resource: file://x\n---\n\nbody\n"
|
|
||||||
)[0]
|
|
||||||
assert parsed["sources"] == [{"id": "a", "resource": "file://x"}]
|
|
||||||
with pytest.raises(OKFFrontmatterError):
|
with pytest.raises(OKFFrontmatterError):
|
||||||
parse_frontmatter(
|
parse_frontmatter(
|
||||||
"---\nid: x\nsources:\n - id: a\n uri: file://x\n---\n\nbody\n"
|
"---\nid: x\nsources:\n - id: a\n resource: file://x\n---\n\nbody\n"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("fm", [
|
@pytest.mark.parametrize("fm", [
|
||||||
# The flow row carries a key OFF the G3 allowlist: the shape is admitted, the
|
"generated: { by: x, at: y }\n", "sources: [{ id: a }]\n", "tags: [a, b]\n",
|
||||||
# key is not, so this stays a T2 rejection and the door A/B half still holds.
|
|
||||||
# The two `sources` rows that lived here until 1.3.0 now PARSE - that is the
|
|
||||||
# G30 fix, not a weakening of this property. Their replacements are the same
|
|
||||||
# carriers with a key off the allowlist, so the row still measures what it
|
|
||||||
# says: the shape is admitted, the key set is not.
|
|
||||||
"generated: { by: x, tool: y }\n", "sources: [{ id: a, uri: u }]\n", "tags: [a, b]\n",
|
|
||||||
"generated:\n by: x\n", "generated.by: x\n",
|
"generated:\n by: x\n", "generated.by: x\n",
|
||||||
"sources:\n - id: a\n uri: file://x\n",
|
"sources:\n - id: a\n resource: file://x\n",
|
||||||
"executor: [{ resource: skills/run.md }]\n",
|
|
||||||
])
|
])
|
||||||
def test_t2_constrains_import_not_emission(fm):
|
def test_t2_constrains_import_not_emission(fm):
|
||||||
# T2 runs on door C only. The same frontmatter that FAIL_SECUREs through
|
# T2 runs on door C only. The same frontmatter that FAIL_SECUREs through
|
||||||
|
|
@ -767,368 +738,3 @@ def test_link_extraction_survives_the_redos_fix():
|
||||||
assert extract_link_targets("see [x](./a.md) and [y](/b.md)") == ["./a.md", "/b.md"]
|
assert extract_link_targets("see [x](./a.md) and [y](/b.md)") == ["./a.md", "/b.md"]
|
||||||
assert extract_link_targets("[a b](./c.md)") == ["./c.md"]
|
assert extract_link_targets("[a b](./c.md)") == ["./c.md"]
|
||||||
assert extract_link_targets("text [](./t.md)") == ["./i.png"]
|
assert extract_link_targets("text [](./t.md)") == ["./i.png"]
|
||||||
|
|
||||||
|
|
||||||
# --- G3: the typed, allowlisted mapping form (2026-08-21) --------------------
|
|
||||||
# Door 1 of three (operator decision, 2026-08-21). The mapping *class* had no
|
|
||||||
# expressible form, and OKF v0.2 writes its whole trust and provenance layer as
|
|
||||||
# mappings — SPEC.md @ 62432a09 §5.2 uses flow form in its own examples, and §11
|
|
||||||
# carries a hard MUST that presupposes they parse ("consumers MUST treat a bare
|
|
||||||
# `verified` mapping as a one-element list"). A consumer measured 0 of 53
|
|
||||||
# upstream concepts through the gate. This admits ONE shape: a flow mapping whose
|
|
||||||
# every key is on the allowlist and whose every leaf is a plain scalar.
|
|
||||||
|
|
||||||
def test_spec_flow_mapping_parses_into_a_typed_mapping():
|
|
||||||
# SPEC.md §5.2, verbatim. This is the red test: it must fail before the form
|
|
||||||
# exists and pass after, with a real dict — never a degraded string.
|
|
||||||
doc = (
|
|
||||||
"---\ntype: table\n"
|
|
||||||
"generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z }\n"
|
|
||||||
"---\nbody\n"
|
|
||||||
)
|
|
||||||
assert parse_frontmatter(doc)[0]["generated"] == {
|
|
||||||
"by": "reference_agent/gemini-2.5-pro",
|
|
||||||
"at": "2026-06-20T22:53:05Z",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def test_spec_bare_verified_mapping_parses():
|
|
||||||
# SPEC.md §5.2's bare form, which §11 turns into a hard MUST for consumers
|
|
||||||
# ("MUST treat a bare `verified` mapping as a one-element list") - a rule that
|
|
||||||
# cannot be obeyed by a consumer that cannot parse the mapping.
|
|
||||||
doc = "---\ntype: table\nverified: { by: human:ahormati, at: 2026-06-25T09:00:00Z }\n---\nb\n"
|
|
||||||
assert parse_frontmatter(doc)[0]["verified"] == {
|
|
||||||
"by": "human:ahormati", "at": "2026-06-25T09:00:00Z"}
|
|
||||||
|
|
||||||
|
|
||||||
def test_spec_verified_list_of_flow_mappings_parses():
|
|
||||||
# §5.2's list form. This is the SAME typed form in list position, not the
|
|
||||||
# block-sequence-with-one-key route (`- uri: x`), which stays shut below.
|
|
||||||
doc = (
|
|
||||||
"---\ntype: table\nverified:\n"
|
|
||||||
" - { by: human:ahormati, at: 2026-06-25T09:00:00Z }\n"
|
|
||||||
" - { by: process:finance-nightly, at: 2026-06-26T02:00:00Z }\n"
|
|
||||||
"---\nbody\n"
|
|
||||||
)
|
|
||||||
assert parse_frontmatter(doc)[0]["verified"] == [
|
|
||||||
{"by": "human:ahormati", "at": "2026-06-25T09:00:00Z"},
|
|
||||||
{"by": "process:finance-nightly", "at": "2026-06-26T02:00:00Z"},
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def test_spec_usage_window_parses():
|
|
||||||
doc = "---\ntype: table\nusage_window: { from: 2026-06-01T00:00:00Z, to: 2026-06-30T00:00:00Z }\n---\nb\n"
|
|
||||||
assert parse_frontmatter(doc)[0]["usage_window"] == {
|
|
||||||
"from": "2026-06-01T00:00:00Z", "to": "2026-06-30T00:00:00Z"}
|
|
||||||
|
|
||||||
|
|
||||||
def test_an_unknown_key_inside_a_mapping_is_still_rejected():
|
|
||||||
# The rejection side of the allowlist. Without this test the allowlist could
|
|
||||||
# silently grow to "anything" - or be emptied - and nothing would fail.
|
|
||||||
with pytest.raises(OKFFrontmatterError) as exc:
|
|
||||||
parse_frontmatter("---\nid: x\ngenerated: { by: a, tool: shell }\n---\n\nbody\n")
|
|
||||||
assert "allowlist" in str(exc.value)
|
|
||||||
|
|
||||||
|
|
||||||
def test_the_allowlist_is_not_empty_and_admits_only_the_spec_keys():
|
|
||||||
# Both directions of the same guard: a shrunk allowlist breaks the first
|
|
||||||
# assertion, a widened one the second.
|
|
||||||
for key in ("by", "at", "from", "to", "id", "title", "author", "usage_count",
|
|
||||||
"last_modified"):
|
|
||||||
assert parse_frontmatter(f"---\nid: x\nk: {{ {key}: v }}\n---\n\nb\n")[0]["k"] == {key: "v"}
|
|
||||||
for key in ("resource", "executor", "attester", "runtime", "command", "uri"):
|
|
||||||
with pytest.raises(OKFFrontmatterError):
|
|
||||||
parse_frontmatter(f"---\nid: x\nk: {{ {key}: v }}\n---\n\nb\n")
|
|
||||||
|
|
||||||
|
|
||||||
_FLOW_REJECTED = [
|
|
||||||
# (id, value, what PyYAML 6.0.3 makes of it)
|
|
||||||
("nested mapping", "{ by: { at: x } }", "a nested mapping"),
|
|
||||||
("nested sequence", "{ by: [a, b] }", "a sequence leaf"),
|
|
||||||
("anchor leaf", "{ by: &a x }", "an anchor definition, silently"),
|
|
||||||
("tag leaf", "{ by: !!python/object:os.system x }", "refused outright"),
|
|
||||||
("block scalar leaf", "{ by: | }", "a scanner error"),
|
|
||||||
("nested colon leaf", "{ by: sub: v }", "refused outright"),
|
|
||||||
("no space after colon", "{by:x}", "the KEY 'by:x', not a scalar"),
|
|
||||||
("quoted leaf", "{ title: 'a, b' }", "a scalar - we refuse, deliberately"),
|
|
||||||
("empty mapping", "{}", "an empty mapping"),
|
|
||||||
("empty leaf", "{ by: }", "None"),
|
|
||||||
("unclosed", "{ by: x", "a parse error"),
|
|
||||||
("trailing junk", "{ by: x } more", "a parse error"),
|
|
||||||
("duplicate key", "{ by: a, by: b }", "last-wins, silently"),
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("cid,value,yaml_reads_as", _FLOW_REJECTED,
|
|
||||||
ids=[c[0] for c in _FLOW_REJECTED])
|
|
||||||
def test_the_mapping_form_admits_scalar_leaves_on_allowlisted_keys_only(cid, value, yaml_reads_as):
|
|
||||||
with pytest.raises(OKFFrontmatterError):
|
|
||||||
parse_frontmatter(f"---\nid: x\ngenerated: {value}\n---\n\nbody\n")
|
|
||||||
|
|
||||||
|
|
||||||
def test_a_rejected_mapping_never_degrades_into_a_string():
|
|
||||||
# The 1.1.0 defect, re-asserted against the NEW form: a refused mapping must
|
|
||||||
# raise, not arrive as a str a consumer will .get() a key out of.
|
|
||||||
for value in ("{ by: { at: x } }", "{ tool: shell }", "{ by: x"):
|
|
||||||
with pytest.raises(OKFFrontmatterError):
|
|
||||||
parse_frontmatter(f"---\nid: x\ngenerated: {value}\n---\n\nbody\n")
|
|
||||||
|
|
||||||
|
|
||||||
def test_the_admitted_mapping_is_a_dict_not_a_string():
|
|
||||||
value = parse_frontmatter("---\nid: x\ngenerated: { by: a, at: b }\n---\n\nb\n")[0]["generated"]
|
|
||||||
assert isinstance(value, dict), "a typed form that arrives as a str is the 1.1.0 defect"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("cid,fm", [
|
|
||||||
("block sequence, one key", "attester:\n - resource: attesters/sql_equality.py\n"),
|
|
||||||
("inline second colon", "attester: resource: attesters/sql_equality.py\n"),
|
|
||||||
("block mapping", "attester:\n resource: attesters/sql_equality.py\n"),
|
|
||||||
("flow mapping, pointer key", "attester: { resource: attesters/sql_equality.py }\n"),
|
|
||||||
])
|
|
||||||
def test_the_pointer_routes_stay_shut(cid, fm):
|
|
||||||
# G3 is additive: none of the routes that put an executable-code pointer in a
|
|
||||||
# key the https allowlist never inspects is reopened. The fourth row is why
|
|
||||||
# `resource` is off the allowlist - the form would otherwise have carried the
|
|
||||||
# door-C pointer through in typed clothes instead of degraded ones.
|
|
||||||
doc = f"---\nid: x\ntype: Attested Computation\n{fm}---\n\nbody\n"
|
|
||||||
with pytest.raises(OKFFrontmatterError):
|
|
||||||
parse_frontmatter(doc)
|
|
||||||
assert import_bundle({"computations/x.md": doc}).disposition is Disposition.FAIL_SECURE
|
|
||||||
|
|
||||||
|
|
||||||
def test_injection_in_a_mapping_leaf_is_caught_by_the_scan():
|
|
||||||
# T1 is not weakened by the new shape: a mapping leaf is scanned exactly like a
|
|
||||||
# scalar value or a list item. A typed form that parses but is not scanned would
|
|
||||||
# be a hole, not a fix.
|
|
||||||
doc = f"---\ntype: table\ngenerated: {{ by: {_INJECTION} }}\n---\nclean body\n"
|
|
||||||
assert scan_concept(doc).found is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_injection_in_a_listed_mapping_leaf_is_caught_by_the_scan():
|
|
||||||
doc = f"---\ntype: table\nverified:\n - {{ by: {_INJECTION} }}\n---\nclean body\n"
|
|
||||||
assert scan_concept(doc).found is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_a_conformant_v02_trust_layer_now_reaches_the_gate():
|
|
||||||
# The measured consequence: a consumer reported 0 of 53 upstream concepts through
|
|
||||||
# the gate, because every one of them carries §5.2 trust frontmatter.
|
|
||||||
doc = (
|
|
||||||
"---\n"
|
|
||||||
"type: table\n"
|
|
||||||
"title: Users\n"
|
|
||||||
"resource: https://example.com/users\n"
|
|
||||||
"generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z }\n"
|
|
||||||
"verified: { by: human:ahormati, at: 2026-06-25T09:00:00Z }\n"
|
|
||||||
"usage_window: { from: 2026-06-01T00:00:00Z, to: 2026-06-30T00:00:00Z }\n"
|
|
||||||
"---\nThe users table.\n"
|
|
||||||
)
|
|
||||||
result = import_bundle({"tables/users.md": doc})
|
|
||||||
assert result.disposition is Disposition.WARN
|
|
||||||
assert result.concepts[0].error is None
|
|
||||||
|
|
||||||
|
|
||||||
# --- G30: the `sources` provenance layer becomes reachable (2026-09-02) ------
|
|
||||||
# Door 2. G3 gave the mapping *class* one expressible form but left `sources`
|
|
||||||
# unreachable: SPEC.md §5.1 writes an entry as a MAPPING carrying a REQUIRED
|
|
||||||
# `resource`, so neither of the two carriers the spec and the producers actually
|
|
||||||
# use could parse. Measured 02.09 by two consumers independently -- a flow
|
|
||||||
# sequence of flow mappings raised on the `[` indicator, a block sequence of
|
|
||||||
# block mappings raised "nested mappings are not supported".
|
|
||||||
#
|
|
||||||
# Why `resource` is admissible now when 1.2.0 argued it was not: the old
|
|
||||||
# argument was that the parser could not tell `sources[].resource` (§5.1, a
|
|
||||||
# citation) from `executor.resource` (§10, a code pointer). That premise was
|
|
||||||
# false -- the owning key is in scope at every call site, it was simply never
|
|
||||||
# threaded through. `resource` is allowlisted for `sources` entries ONLY, so
|
|
||||||
# the door-C routes above stay shut on the same input.
|
|
||||||
|
|
||||||
_SPEC_51_BLOCK = (
|
|
||||||
"sources:\n"
|
|
||||||
" - id: ga4-schema\n"
|
|
||||||
" resource: https://developers.google.com/analytics/bigquery/export-schema\n"
|
|
||||||
" title: GA4 BigQuery Export schema\n"
|
|
||||||
" author: team:ga4-docs\n"
|
|
||||||
" usage_count: 5000\n"
|
|
||||||
" last_modified: 2026-05-30T00:00:00Z\n"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_spec_sources_block_sequence_of_block_mappings_parses():
|
|
||||||
# SPEC.md §5.1's own example block, verbatim. It is the spec's canonical
|
|
||||||
# carrier for a REQUIRED field, so §11.1 ("parseable YAML frontmatter") makes
|
|
||||||
# a bundle written this way conformant -- refusing it refuses a conformant
|
|
||||||
# bundle, which is the failure mode G3 was opened to end.
|
|
||||||
fm, _ = parse_frontmatter(f"---\ntype: table\n{_SPEC_51_BLOCK}---\n\nbody\n")
|
|
||||||
assert fm["sources"] == [{
|
|
||||||
"id": "ga4-schema",
|
|
||||||
"resource": "https://developers.google.com/analytics/bigquery/export-schema",
|
|
||||||
"title": "GA4 BigQuery Export schema",
|
|
||||||
"author": "team:ga4-docs",
|
|
||||||
"usage_count": "5000",
|
|
||||||
"last_modified": "2026-05-30T00:00:00Z",
|
|
||||||
}]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("cid,fm,expected", [
|
|
||||||
("one entry",
|
|
||||||
"sources: [{ id: golden-v0-2-sales, resource: fixture }]\n",
|
|
||||||
[{"id": "golden-v0-2-sales", "resource": "fixture"}]),
|
|
||||||
("two entries",
|
|
||||||
"sources: [{ id: a, resource: https://e.com/a }, { id: b, resource: https://e.com/b }]\n",
|
|
||||||
[{"id": "a", "resource": "https://e.com/a"},
|
|
||||||
{"id": "b", "resource": "https://e.com/b"}]),
|
|
||||||
])
|
|
||||||
def test_sources_flow_sequence_of_flow_mappings_parses(cid, fm, expected):
|
|
||||||
# The form the producer emits today (llm-ingestion-okf's golden
|
|
||||||
# expected-bundle/ingest-sales.md, measured 02.09). One entry raised too, so
|
|
||||||
# this is not an arity bug: the `[` indicator refused the carrier outright.
|
|
||||||
assert parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")[0]["sources"] == expected
|
|
||||||
|
|
||||||
|
|
||||||
def test_the_two_sources_carriers_parse_to_the_same_value():
|
|
||||||
block = parse_frontmatter(
|
|
||||||
"---\ntype: t\nsources:\n - id: a\n resource: https://e.com/a\n---\n\nb\n")[0]
|
|
||||||
flow = parse_frontmatter(
|
|
||||||
"---\ntype: t\nsources: [{ id: a, resource: https://e.com/a }]\n---\n\nb\n")[0]
|
|
||||||
assert block["sources"] == flow["sources"] == [{"id": "a", "resource": "https://e.com/a"}]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("cid,fm", [
|
|
||||||
("unknown key, flow", "sources: [{ id: a, uri: https://e.com/a }]\n"),
|
|
||||||
("unknown key, block", "sources:\n - id: a\n uri: https://e.com/a\n"),
|
|
||||||
("unknown key, flow value", "sources: { id: a, kind: doc }\n"),
|
|
||||||
])
|
|
||||||
def test_an_unknown_key_in_a_sources_entry_is_still_rejected(cid, fm):
|
|
||||||
# The negative control. The carrier is admitted; the key set is not. A
|
|
||||||
# producer-invented key gets no free ride on the new shape.
|
|
||||||
with pytest.raises(OKFFrontmatterError) as exc:
|
|
||||||
parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")
|
|
||||||
assert "allowlist" in str(exc.value)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("cid,fm", [
|
|
||||||
("executor, block sequence", "executor:\n - resource: skills/run-on-bq.md\n"),
|
|
||||||
("executor, flow sequence", "executor: [{ resource: skills/run-on-bq.md }]\n"),
|
|
||||||
("attester, flow sequence", "attester: [{ resource: attesters/sql_equality.py }]\n"),
|
|
||||||
("attester, block sequence", "attester:\n - resource: attesters/sql_equality.py\n"),
|
|
||||||
])
|
|
||||||
def test_resource_is_allowlisted_for_sources_entries_only(cid, fm):
|
|
||||||
# The whole reason `resource` can be admitted at all: the owning key decides.
|
|
||||||
# §10's `executor.resource` / `attester.resource` name run instructions and
|
|
||||||
# code -- door C -- and stay refused through EVERY carrier, including the two
|
|
||||||
# this change opens for `sources`.
|
|
||||||
doc = f"---\nid: x\ntype: Attested Computation\n{fm}---\n\nbody\n"
|
|
||||||
with pytest.raises(OKFFrontmatterError) as exc:
|
|
||||||
parse_frontmatter(doc)
|
|
||||||
assert "allowlist" in str(exc.value)
|
|
||||||
assert import_bundle({"computations/x.md": doc}).disposition is Disposition.FAIL_SECURE
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("cid,fm", [
|
|
||||||
# SPEC §5.1: "A single entry MAY carry its own `usage_window`". That is a
|
|
||||||
# mapping inside a mapping -- depth 2 -- and stays refused: no nesting deeper
|
|
||||||
# than one is a security property this change does not spend. Registered as a
|
|
||||||
# conformance gap in docs/LIMITATIONS.md, not as an oversight.
|
|
||||||
("per-entry usage_window, flow",
|
|
||||||
"sources: [{ id: a, usage_window: { from: x, to: y } }]\n"),
|
|
||||||
("per-entry usage_window, block",
|
|
||||||
"sources:\n - id: a\n usage_window: { from: x, to: y }\n"),
|
|
||||||
# A sequence inside an entry is the same depth violation.
|
|
||||||
("nested sequence", "sources: [{ id: a, title: [x, y] }]\n"),
|
|
||||||
])
|
|
||||||
def test_a_sources_entry_admits_scalar_leaves_only(cid, fm):
|
|
||||||
with pytest.raises(OKFFrontmatterError):
|
|
||||||
parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("cid,fm", [
|
|
||||||
("scalar then mapping", "sources:\n - https://e.com/a\n - id: b\n title: B\n"),
|
|
||||||
("mapping then scalar", "sources:\n - id: a\n title: A\n - https://e.com/b\n"),
|
|
||||||
])
|
|
||||||
def test_a_block_list_may_not_mix_scalars_and_mappings(cid, fm):
|
|
||||||
# A consumer that reads `entry.get("id")` over the list crashes on the str.
|
|
||||||
# One list, one item type -- refuse rather than hand back a mixed tree.
|
|
||||||
with pytest.raises(OKFFrontmatterError) as exc:
|
|
||||||
parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")
|
|
||||||
assert "mix" in str(exc.value)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("cid,fm", [
|
|
||||||
("flow sequence of scalars", "tags: [a, b, c]\n"),
|
|
||||||
("empty flow sequence", "sources: []\n"),
|
|
||||||
("flow sequence, unclosed", "sources: [{ id: a }\n"),
|
|
||||||
("flow sequence, trailing junk", "sources: [{ id: a }] x\n"),
|
|
||||||
("flow sequence, mixed", "sources: [{ id: a }, plain]\n"),
|
|
||||||
("flow sequence, nested sequence", "sources: [[ id ]]\n"),
|
|
||||||
])
|
|
||||||
def test_the_flow_sequence_admits_flow_mappings_only(cid, fm):
|
|
||||||
# The sequence carrier is opened for the mapping form and nothing else. A
|
|
||||||
# flow sequence of plain scalars stays refused -- it is a separate shape with
|
|
||||||
# its own quoting and splitting problems, and no measured consumer needs it.
|
|
||||||
with pytest.raises(OKFFrontmatterError):
|
|
||||||
parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")
|
|
||||||
|
|
||||||
|
|
||||||
def test_injection_in_a_sources_entry_leaf_is_caught_by_the_scan():
|
|
||||||
# T1 over the new shape: every leaf of every entry reaches scan_output.
|
|
||||||
doc = f"---\ntype: table\nsources: [{{ id: a, title: {_INJECTION} }}]\n---\nclean\n"
|
|
||||||
assert scan_concept(doc).found is True
|
|
||||||
doc_block = f"---\ntype: table\nsources:\n - id: a\n title: {_INJECTION}\n---\nclean\n"
|
|
||||||
assert scan_concept(doc_block).found is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_the_producer_golden_now_passes_the_gate():
|
|
||||||
# llm-ingestion-okf's expected-bundle/ingest-sales.md, the K5 blocker.
|
|
||||||
doc = (
|
|
||||||
"---\n"
|
|
||||||
"type: dataset\n"
|
|
||||||
"title: Regional Sales\n"
|
|
||||||
"source_system: golden-v0-2-sales\n"
|
|
||||||
"ingested_at: 2026-07-16T12:00:00Z\n"
|
|
||||||
"generated: { by: process:okf-ingest, at: 2026-07-16T12:00:00Z }\n"
|
|
||||||
"sources: [{ id: golden-v0-2-sales, resource: fixture }]\n"
|
|
||||||
"---\n\n| region | units |\n| --- | --- |\n| nord | 412 |\n"
|
|
||||||
)
|
|
||||||
result = import_bundle({"datasets/sales.md": doc})
|
|
||||||
assert result.disposition is Disposition.WARN
|
|
||||||
assert result.concepts[0].error is None
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("cid,fm", [
|
|
||||||
# A `- ` item whose text begins with a YAML indicator AND carries a `": "`
|
|
||||||
# must fall through to the scalar predicates, not into the block-mapping
|
|
||||||
# route with a half-validated key. Each of these fails `_KEY_RE` on the key
|
|
||||||
# side, so `_reject_dangerous_value` gets the item intact.
|
|
||||||
("anchor", "sources:\n - &anchor id: a\n"),
|
|
||||||
("alias", "sources:\n - *anchor id: a\n"),
|
|
||||||
("tag", "sources:\n - !!str id: a\n"),
|
|
||||||
("directive", "sources:\n - %YAML id: a\n"),
|
|
||||||
("reserved", "sources:\n - `x id: a\n"),
|
|
||||||
("merge key", "sources:\n - id: a\n <<: *base\n"),
|
|
||||||
])
|
|
||||||
def test_an_indicator_in_a_block_item_is_refused_before_the_mapping_route(cid, fm):
|
|
||||||
with pytest.raises(OKFFrontmatterError):
|
|
||||||
parse_frontmatter(f"---\ntype: t\n{fm}---\n\nbody\n")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("cid,fm", [
|
|
||||||
# The block mapping ends at a blank line and at a line in column zero. Both
|
|
||||||
# hand control back with an index that must not skip or re-read a line: a
|
|
||||||
# dangling `resource:` line left over from a mapping that closed early must
|
|
||||||
# RAISE, never be silently dropped -- a pointer that vanishes rather than
|
|
||||||
# failing is exactly this repo's failure class.
|
|
||||||
("blank line inside the mapping", "sources:\n - id: a\n\n resource: b\n"),
|
|
||||||
("top-level key interleaved", "sources:\n - id: a\ntags: x\n resource: b\n"),
|
|
||||||
])
|
|
||||||
def test_a_line_orphaned_by_the_mapping_boundary_raises_rather_than_vanishing(cid, fm):
|
|
||||||
with pytest.raises(OKFFrontmatterError):
|
|
||||||
parse_frontmatter(f"---\ntype: t\n{fm}---\n\nbody\n")
|
|
||||||
|
|
||||||
|
|
||||||
def test_the_block_list_hands_back_an_index_that_resumes_at_the_next_key():
|
|
||||||
# The return-index contract: a top-level key following a multi-entry block
|
|
||||||
# list is neither swallowed by the list nor re-read as a list item.
|
|
||||||
fm, _ = parse_frontmatter(
|
|
||||||
"---\ntype: t\nsources:\n - id: a\n resource: b\n - id: c\ntitle: T\n---\n\nbody\n")
|
|
||||||
assert fm == {"type": "t", "sources": [{"id": "a", "resource": "b"}, {"id": "c"}],
|
|
||||||
"title": "T"}
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue