Compare commits
19 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| d19de8cb38 | |||
| 3e324a1f86 | |||
| 6e7c8d2b98 | |||
| 44e2b31afd | |||
| a965e8ac5b | |||
| 0184df9ed9 | |||
| 9aeceb02c0 | |||
| 58704834b6 | |||
| 7cb4553301 | |||
| 01f6f382c4 | |||
| 74656123f9 | |||
| 4472f209a4 | |||
| 2b03b8d643 | |||
| 246e1fc1d3 | |||
| 45c2b06315 | |||
| 869f9058f7 | |||
| ca4f97c8c9 | |||
| da30211bc7 | |||
| 6cd4694613 |
21 changed files with 2313 additions and 119 deletions
3
.gitignore
vendored
3
.gitignore
vendored
|
|
@ -24,3 +24,6 @@ coverage/
|
|||
*.local.md
|
||||
*.local.sh
|
||||
.DS_Store
|
||||
|
||||
# --- scratchpad: measurement scripts are tracked, the extracted corpus copy is not ---
|
||||
/scratchpad/corpus/
|
||||
|
|
|
|||
209
CHANGELOG.md
209
CHANGELOG.md
|
|
@ -5,9 +5,214 @@ All notable changes to this project will be documented in this file.
|
|||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
## [1.4.0] — 2026-09-08
|
||||
|
||||
Nothing yet.
|
||||
### Added — `tags: [a, b, c]`, the one flow-sequence form SPEC §4.1 writes out
|
||||
|
||||
`okf.parse_frontmatter` refused a flow sequence of plain scalars. That is the
|
||||
form SPEC.md §4.1's own frontmatter skeleton uses for `tags`, and `tags` is
|
||||
present in 53/53 upstream reference concepts, 9/53 of them in exactly this form.
|
||||
Admitted now, parsing to the same value its block-sequence sibling already did:
|
||||
|
||||
```yaml
|
||||
tags: [finance, revenue, headline-metric]
|
||||
```
|
||||
|
||||
An element is a plain scalar only if it is non-empty and carries none of
|
||||
`{ } [ ] : , " ' #`, and it then passes the unchanged scalar-indicator rule.
|
||||
The refusals that make that a rule rather than a preference: a quoted element
|
||||
(this parser retains quotes rather than stripping them, so reading one would
|
||||
hand back a value YAML does not), an element carrying a colon or a comma, a
|
||||
sequence inside a sequence, an empty element, an anchor or an alias. A sequence
|
||||
may not **mix** scalars and mappings — the same rule the block list already
|
||||
carries, for the same reason: a consumer iterating the value and reading
|
||||
`entry.get("id")` crashes on the first `str`. The `sources` flow-mapping
|
||||
carrier added in 1.3.0 is unchanged and pinned against regression.
|
||||
|
||||
**Measured, with the denominator: this takes the pinned OKF corpus from 0/53 to
|
||||
6/53** (`_okf-upstream/okf` @ `3fcbb9f`, 53 non-reserved documents; the six are
|
||||
all in `acme_retail`). It does not close the corpus. **44/53 still stop on
|
||||
`generated` written as a top-level block mapping** — a form refused on the
|
||||
no-nesting-past-depth-1 rule, which is a security decision and out of scope
|
||||
here. Two neighbouring predicates were measured and deliberately not built: a
|
||||
flush-left block sequence and a folded plain scalar release **0/53** each on
|
||||
their own and leave the corpus at 6/53 when stacked on this one. Detection
|
||||
behaviour is not frozen under semver; the exported surface is unchanged.
|
||||
|
||||
### Changed
|
||||
|
||||
- `docs/LIMITATIONS.md`'s `tags`/`description` entry is rewritten against the
|
||||
measurement. Three of its claims were wrong: the parser *does* have a
|
||||
sequence-value type (since 1.3.0 — what it lacks is the indentation the
|
||||
corpus omits), the figure is 6/53 and not the 1.2.0-era 4/53, and
|
||||
`tags`/`description` are *not* the residual that blocks the corpus.
|
||||
- Self-safety: the new predicate is character-level with no regex, so
|
||||
`docs/redos-sweep.py` cannot see it (it collects compiled patterns). It is
|
||||
measured instead — linear in both element length (exponent 0.86-0.99) and
|
||||
element count (0.97-1.05) over four doublings to 800_000, and pinned by two
|
||||
CPU-clock bounds in `tests/test_okf.py`.
|
||||
|
||||
## [1.3.0] — 2026-09-02
|
||||
|
||||
### Added — the `sources` provenance family becomes expressible in both spec carriers
|
||||
|
||||
`okf.parse_frontmatter` rejected `sources` in every form the OKF spec and its
|
||||
producers actually use. Measured 2026-09-02 by two consumers independently:
|
||||
`sources: [{ id: a, resource: x }]` raised on the `[` indicator (with one entry
|
||||
as with two), and the block sequence of block mappings — SPEC.md §5.1's *own*
|
||||
example — raised `"nested mappings are not supported"`. `resource` is REQUIRED
|
||||
within a `sources` entry (§5.1), so the whole provenance family was unwritable,
|
||||
and a bundle written the way the spec documents it was refused.
|
||||
|
||||
Admitted now, both parsing to the same value (a list of dicts):
|
||||
|
||||
```yaml
|
||||
sources: [{ id: a, resource: https://e.com/a }]
|
||||
sources:
|
||||
- id: a
|
||||
resource: https://e.com/a
|
||||
```
|
||||
|
||||
**`resource` is allowlisted inside a `sources` entry and nowhere else.** 1.2.0
|
||||
left it off the allowlist on the argument that the parser could not tell
|
||||
`sources[].resource` (§5.1, a citation) from `executor.resource` /
|
||||
`attester.resource` (§10, a pointer to code to be run — the door-C route closed
|
||||
in 1.1.0). That premise was false: the owning key is in scope at every call
|
||||
site and was simply never threaded through. It is threaded now, so the
|
||||
discrimination is structural rather than a judgement about the value, and
|
||||
`executor: [{ resource: skills/run.md }]` and `attester:\n - resource: …` are
|
||||
refused on the allowlist through *every* carrier, including the two this adds.
|
||||
|
||||
Refusal stays the default everywhere else. A flow sequence of plain scalars
|
||||
(`tags: [a, b]`) still raises — the sequence carrier is opened for the flow
|
||||
mapping element and nothing else. A `sources` entry still admits scalar leaves
|
||||
only, so SPEC §5.1's optional *per-entry* `usage_window` (a mapping inside a
|
||||
mapping) is refused: no nesting past depth 1 is a security property, and it was
|
||||
not spent here. Registered as a conformance gap in `docs/LIMITATIONS.md`. A
|
||||
block list may not mix scalar items and mappings. Off-allowlist keys, anchors,
|
||||
aliases, tags, duplicate keys and unclosed collections raise as before, and a
|
||||
refused mapping still raises rather than degrading into a string.
|
||||
|
||||
**New residual, registered:** `sources[].resource` is scanned as text (T1) but
|
||||
never validated as a URL. T3's https allowlist cannot be extended to it without
|
||||
over-blocking conformant bundles — §5.1 permits a bundle-relative path or a
|
||||
scope descriptor a consumer cannot follow at all. A consumer that dereferences
|
||||
it must call `okf.validate_resource_url` itself.
|
||||
|
||||
No exported surface changed and no detector behaviour or calibration changed.
|
||||
Suite 834 → 868 (34 new rows, plus four pre-existing rows updated where this
|
||||
release changed the behaviour they pinned); 130/130 classes, 6/6 documented
|
||||
gaps hold, 45 limitations, ReDoS sweep 0/152 candidates flagged.
|
||||
|
||||
## [1.2.0] — 2026-08-23
|
||||
|
||||
### Added — OKF frontmatter can express one mapping form: typed and allowlisted
|
||||
|
||||
`okf.parse_frontmatter` gave the mapping *class* no expressible form at all. OKF
|
||||
v0.2 writes its whole trust and provenance layer as mappings — SPEC.md @
|
||||
`62432a09` uses flow form in its own §5.1/§5.2 examples, and §11 carries a hard
|
||||
MUST for consumers ("MUST treat a bare `verified` mapping as a one-element
|
||||
list") that presupposes they parse. A consumer measured **0 of 53** upstream
|
||||
concepts through the gate on 0.3.4, 1.0.0 and 1.1.0. That was a contract
|
||||
collision, not a calibration setting: no threshold would have moved it.
|
||||
|
||||
Admitted now: a flow mapping (`generated: { by: x, at: y }`), as a value or as a
|
||||
block-list item, whose every key is on a nine-name allowlist (`by`, `at`, `from`,
|
||||
`to`, `id`, `title`, `author`, `usage_count`, `last_modified`) and whose every
|
||||
leaf is a plain scalar run through the *unchanged* dangerous-value and
|
||||
mapping-construct predicates.
|
||||
|
||||
The form is additive and refusal stays the default. A key off the allowlist, a
|
||||
nested collection, a quoted leaf, a duplicate key, an empty or unclosed mapping,
|
||||
and `{a:b}` (which PyYAML 6.0.3 reads as the *key* `a:b`) all raise, and a
|
||||
refused mapping still raises rather than degrading into a string — the 1.1.0
|
||||
defect is not reopened. Nested-block (`k:\n sub: v`), dotted (`k.sub: v`) and
|
||||
inline-second-colon (`k: sub: v`) routes to a mapping still raise, each on its
|
||||
own rule.
|
||||
|
||||
**`resource` is deliberately off the allowlist**, though SPEC.md §5.1 names it
|
||||
inside a `sources` entry. It is a pointer rather than a label and the only key
|
||||
T3 exists for: admitting it would let `executor: { resource: skills/run.md }`
|
||||
carry an executable-code pointer through in typed clothes, which is the door-C
|
||||
route closed in 1.1.0. It costs nothing today — the conformant carrier for
|
||||
`sources[].resource` is a block sequence of block mappings, which this form does
|
||||
not admit either way.
|
||||
|
||||
Mapping leaves are scanned like every other frontmatter value (T1), so an
|
||||
injection parked in `generated: { by: ... }` reaches `scan_output`. Coverage
|
||||
matrix: 130 classes, up from 129 (the new row is the off-allowlist key).
|
||||
|
||||
No exported surface changed; no detector behaviour and no calibration changed.
|
||||
|
||||
|
||||
### Changed — the ReDoS sweep now measures on the same clock as the bounds it justifies
|
||||
|
||||
`docs/redos-sweep.py` timed on `time.monotonic()` while every ReDoS bound in the
|
||||
suite moved to process CPU time (`tests/redos_clock.py`), so the 1.5 ms
|
||||
sensitivity floor and the "~23 s at the cap" figure published in
|
||||
`docs/LIMITATIONS.md` came from a different instrument than the bounds they
|
||||
support. The script now imports `scan_seconds` rather than timing itself.
|
||||
|
||||
The floor was re-derived on that instrument and **stayed at 1.5 ms**: over twelve
|
||||
full runs of all 2585 arms the median ratio is 1.95-2.03 in every size bucket
|
||||
above 50 µs, but two-point excursions past the 2.6 flag threshold persist at every
|
||||
magnitude (p99 ratio 2.9-3.3 even above 1 ms) — 6.9 flagged arms per run at a
|
||||
0.5 ms floor, 1.1 at 1.0 ms, 0.33 at 1.5 ms. Descheduling was never what made this
|
||||
sweep noisy; a ratio computed from two points is. Four arms flagged across those
|
||||
twelve runs, each in exactly one of them, and six arms that have ever flagged
|
||||
re-measure at exponent 0.97-1.09 over six doublings — at most 1.2 s at the
|
||||
1 000 000-char cap. The pattern count the script prints is 152, not the 150 of the
|
||||
0.3.4 entry below; `docs/LIMITATIONS.md` now carries the current number.
|
||||
|
||||
No exported surface, no detector behaviour and no calibration changed.
|
||||
|
||||
|
||||
## [1.1.0] — 2026-08-13
|
||||
|
||||
### Fixed — a mapping construct in OKF frontmatter no longer degrades into a string
|
||||
|
||||
`okf.parse_frontmatter` gives the mapping *class* no expressible form by design
|
||||
(T2). Two routes escaped that: they parsed "successfully" into the wrong **type**
|
||||
instead of raising. Both are closed, and both now `FAIL_SECURE` through
|
||||
`okf.import_bundle` (door C).
|
||||
|
||||
| route | was | now |
|
||||
|---|---|---|
|
||||
| `sources:`<br>` - uri: https://e.com/a` | string `'uri: https://e.com/a'` — WARN | `OKFFrontmatterError` — FAIL_SECURE |
|
||||
| `sources:`<br>` - uri:` | string `'uri:'` — WARN | `OKFFrontmatterError` — FAIL_SECURE |
|
||||
| `attester: resource: attesters/x.py` | string `'resource: attesters/x.py'` — WARN | `OKFFrontmatterError` — FAIL_SECURE |
|
||||
|
||||
The security consequence was the same in each: a pointer parked in a degraded
|
||||
mapping rides through in a key the `resource` allowlist never inspects, and mode-b
|
||||
`import_bundle` writes the merged concept verbatim. The first route was documented
|
||||
at `docs/LIMITATIONS.md:43`; the inline second colon was **found by measurement
|
||||
while closing it**, and is the reason this release names two routes rather than one.
|
||||
Neither shape is conformant OKF — a well-formed bundle does not produce them; a
|
||||
malformed or hostile one can.
|
||||
|
||||
**What closed is the type confusion, not pointer-smuggling as a class.** T3 still
|
||||
inspects `resource` and nothing else, so an honest string under another key rides
|
||||
through exactly as before: `attester: attesters/sql_equality.py` is WARN, while
|
||||
the same path under `resource:` FAIL_SECUREs. The string is still scanned like any
|
||||
other frontmatter value under T1. Nothing about that changed here.
|
||||
|
||||
**The boundary is where YAML puts it**, ground-truthed against PyYAML 6.0.3 rather
|
||||
than reasoned: `": "` and a trailing `":"` are exactly the two shapes where a plain
|
||||
scalar becomes a mapping, and they are refused. A colon carrying neither a space nor
|
||||
a line end opens no mapping — `domain:security` and `https://e.com:8443/a` still
|
||||
parse — and a quoted scalar (`- "uri: x"`) is still a scalar. Quotes are retained
|
||||
rather than stripped; that divergence from YAML is unchanged and now pinned.
|
||||
|
||||
**This is a behaviour change inside the freeze, not a break of it.** No exported
|
||||
name moved. A document that disposed `WARN` on `1.0.0` may dispose `FAIL_SECURE`
|
||||
here — the `1.0.0` entry says exactly this is a fix, not a break. A consumer whose
|
||||
bundles carry an unquoted `": "` in a frontmatter value will now see those concepts
|
||||
refused at import; quote the value, and it parses.
|
||||
|
||||
Suite 792 → **802**: 13 rows added (4 rejected shapes, 7 admitted ones, 2 through
|
||||
`import_bundle`), 3 retired (the two that pinned the defect, and the one-key row
|
||||
in the block-list table). 129/129 classes, 6/6 documented gaps, 35 limitations —
|
||||
all unchanged.
|
||||
|
||||
|
||||
## [1.0.0] — 2026-08-13
|
||||
|
|
|
|||
33
CLAUDE.md
33
CLAUDE.md
|
|
@ -11,11 +11,40 @@ framework-agnostisk kode.
|
|||
Referanse-implementasjon: `claude-code-llm-wiki` Stage B (`tools/wiki_ingest/`).
|
||||
Lexikon-seed: `injection-patterns.mjs` fra `llm-security`-pluginen.
|
||||
|
||||
Repoet er på **v1.0.0** — den eksporterte Python-surfacen er frosset under semver
|
||||
Repoet er på **v1.4.0 i koden, UUTGITT** (`pyproject.toml` + `__init__.py` er
|
||||
bumpet; README-badge, install-pinnen, ADOPTION-BRIEF og BRIEF står med vilje
|
||||
igjen på `1.3.0`, som er den siste taggen som FINNES — en install-pin må peke på
|
||||
en ekte tag). Release-commiten (CHANGELOG-overskrift datert, de fire
|
||||
dokumentflatene bumpet, tag) er ikke tatt. Den eksporterte Python-surfacen er frosset under semver
|
||||
(deteksjonsatferd er det IKKE; kalibrering flytter seg i 1.x). Stdlib-kjernen er
|
||||
bygget og testet (15 moduler +
|
||||
topp-nivå wiring, showcase + korpus), inkl. OKF-adapter og aktivt-innhold-
|
||||
detektor (EchoLeak-klassen) i output-gaten. Mode-b `import_bundle` skanner
|
||||
detektor (EchoLeak-klassen) i output-gaten. OKF-frontmatterens mapping-klasse
|
||||
har **fire** uttrykkbare bærere (G3 21.08, G30 02.09): flow-mapping som verdi
|
||||
og som blokkliste-element, flow-sekvens av flow-mappinger, og blokk-sekvens av
|
||||
blokk-mappinger (SPEC §5.1s egen form). HVER nøkkel i alle fire står på
|
||||
allowlisten og hvert blad er en ren skalar. Formen er trygg fordi allowlisten
|
||||
inspiserer hver nøkkel; det blanke avslaget var håndhevelsen, ikke poenget.
|
||||
**`resource` er allowlistet KUN inne i en `sources`-oppføring** — foreldre-
|
||||
nøkkelen avgjør, så `executor`/`attester` sin `resource` (§10, dør C) avvises
|
||||
gjennom hver eneste bærer. 1.2.0s begrunnelse for å utelate den (parseren
|
||||
manglet foreldre-kontekst) var målt feil: konteksten var der, den var bare
|
||||
aldri tredd gjennom. Topp-nivå blokk-mapping, dotted- og inline-kolon-rutene
|
||||
raiser fortsatt, en blokkliste kan ikke blande skalarer og mappinger, og en
|
||||
avvist mapping raiser — den degraderer aldri til en streng (1.1.0-defekten).
|
||||
**Flow-sekvens av rene skalarer (`tags: [a, b]`) PARSER fra 1.4.0** (P1,
|
||||
operatørbeslutning 08.09) — SPEC §4.1s eget skjelett. Et element er en ren
|
||||
skalar kun hvis det er ikke-tomt og uten `{ } [ ] : , " ' #`, og så gjelder den
|
||||
uendrete indikator-regelen; sitert element, kolon/komma i elementet, sekvens i
|
||||
sekvens, tomt element, anker og alias raiser fortsatt, og en flow-sekvens kan
|
||||
ikke blande skalarer og mappinger. **Målt med nevner: 0/53 → 6/53** på pinnet
|
||||
OKF-korpus (`3fcbb9f`). Den bindende skranken er IKKE tags/description, men
|
||||
`generated` som topp-nivå blokk-mapping (44/53) — den bruker opp dybde-1 og er
|
||||
en sikkerhetsbeslutning. P2 (blokksekvens uten innrykk) og P3 (foldet plain
|
||||
scalar) er MÅLT til 0/53 hver og bevisst IKKE bygget. `sources[].resource`
|
||||
URL-valideres ALDRI (T3 ser kun topp-nivå `resource`) — §5.1 tillater
|
||||
bundle-relative stier og scope-beskrivelser, så en https-gate ville over-blokkert
|
||||
konforme bundles; konsumenten må selv kalle `validate_resource_url`. Mode-b `import_bundle` skanner
|
||||
reserverte strukturfiler (`index.md`/`log.md`) i mottatte bundles i stedet for å
|
||||
path-avvise dem; upload-front-end beholder shadow-reject (`allow_reserved=False`).
|
||||
Output-gatens decode-and-rescan mater dekodet base64-klartekst gjennom BÅDE lexicon
|
||||
|
|
|
|||
36
README.md
36
README.md
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Write-time defensive layer for Python pipelines that persist LLM output: sanitize, fence, tool-less quarantined transform, capability isolation, scan before persist, fail-secure.
|
||||
|
||||

|
||||

|
||||

|
||||

|
||||

|
||||
|
|
@ -33,7 +33,7 @@ at write time, never assumed from the format. Any pipeline ingesting external da
|
|||
into an agent-read store has this shape; an OKF wiki is its canonical form — which
|
||||
is why the guard ships a first-class OKF adapter (below).
|
||||
|
||||
**Status:** `v1.0.0`. The stdlib-only core — its detector, contract, and
|
||||
**Status:** `v1.3.0`. The stdlib-only core — its detector, contract, and
|
||||
OKF-adapter modules plus the top-level wiring — is built and tested, exercised by
|
||||
an end-to-end showcase and adversarial + false-positive corpora. The exported
|
||||
Python surface is now frozen under semver: nothing exported is removed, renamed or
|
||||
|
|
@ -41,12 +41,24 @@ given a different meaning without a `2.0.0`. **Detection behaviour is not frozen
|
|||
severities, thresholds and lexicon entries are calibration and move in `1.x`. There
|
||||
are real limitations, stated plainly below; read them.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Install](#install)
|
||||
- [Quickstart — the two bookends](#quickstart--the-two-bookends)
|
||||
- [OKF / LLM-wiki support (shipped)](#okf--llm-wiki-support-shipped)
|
||||
- [What it protects against](#what-it-protects-against)
|
||||
- [The reusable contract (adopt-this checklist)](#the-reusable-contract-adopt-this-checklist)
|
||||
- [Known limitations](#known-limitations)
|
||||
- [Non-goals](#non-goals)
|
||||
- [Design & threat model](#design--threat-model)
|
||||
- [License](#license)
|
||||
|
||||
## Install
|
||||
|
||||
Not on PyPI. The guard is distributed from its Forgejo origin — pin a release tag:
|
||||
|
||||
```bash
|
||||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v1.0.0"
|
||||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v1.3.0"
|
||||
```
|
||||
|
||||
The `open/` mirror is anonymously readable, so CI needs no deploy key, token, or
|
||||
|
|
@ -153,7 +165,19 @@ Per-concept gates: **path / reserved-name** (rejects `..` traversal and reserved
|
|||
reject-by-default loader that refuses anchors, aliases, and explicit tags *by
|
||||
construction*, so a billion-laughs alias expansion or a `!!python/object` coercion
|
||||
cannot occur (it is deliberately **not** a general YAML engine, whose own features
|
||||
are the attack surface); **`resource` https-allowlist** (hard-rejects
|
||||
are the attack surface). The one mapping form it accepts is OKF v0.2's flow
|
||||
mapping — `generated: { by: x, at: y }`, `verified: { … }` bare or listed,
|
||||
`usage_window: { from: …, to: … }` — admitted key-by-key against a nine-name
|
||||
allowlist (`by`, `at`, `from`, `to`, `id`, `title`, `author`, `usage_count`,
|
||||
`last_modified`) with plain-scalar leaves only. A key off that list, a nested
|
||||
collection or a duplicate key is refused, and `resource` is deliberately not on
|
||||
it; the block, dotted and inline-colon routes to a mapping still raise. A
|
||||
*sequence* value has two carriers — the block list, and (as of `1.4.0`) the flow
|
||||
sequence `tags: [a, b, c]`, which is SPEC §4.1's own skeleton — whose elements
|
||||
are either all plain scalars or all flow mappings, never a mix. A scalar element
|
||||
carrying any of `{ } [ ] : , " ' #` is refused rather than guessed at. See
|
||||
[LIMITATIONS](docs/LIMITATIONS.md) for what that admits and what it still walls
|
||||
off (a `sources` block list of mappings is still refused); **`resource` https-allowlist** (hard-rejects
|
||||
`data:`/`javascript:`/`file:` before commit — a reject-gate, not defang);
|
||||
**whole-concept scan** (frontmatter *values* + body through `scan_output`);
|
||||
**cross-link graph** (surfaces dangling targets, the dormant-injection signal, and
|
||||
|
|
@ -167,7 +191,7 @@ driven by a **live payload** in the coverage matrix — run it to watch all 134
|
|||
in your own environment:
|
||||
|
||||
```bash
|
||||
python -m llm_ingestion_guard.coverage # 129/129 classes; exit 0 = all as documented
|
||||
python -m llm_ingestion_guard.coverage # 130/130 classes; exit 0 = all as documented
|
||||
```
|
||||
|
||||
| Anchor | Attack classes it stops (representative) |
|
||||
|
|
@ -249,7 +273,7 @@ a green scan means safe content. The highest-impact items:
|
|||
two of the three corpora are living, so the cells are not rewritten in place.
|
||||
Method and before/after: [`docs/rawhtml-census.py`](docs/rawhtml-census.py).
|
||||
|
||||
**Full list — 35 items, each with the mechanism, plus the out-of-scope boundary:**
|
||||
**Full list — 45 items, each with the mechanism, plus the out-of-scope boundary:**
|
||||
[`docs/LIMITATIONS.md`](docs/LIMITATIONS.md). Several carry field measurements from
|
||||
consumer corpora, including the false positives the URL-shape rule actually produces.
|
||||
|
||||
|
|
|
|||
10
SECURITY.md
10
SECURITY.md
|
|
@ -25,12 +25,12 @@ you happened to observe.
|
|||
**Do not open a public issue for a vulnerability.** Public disclosure before a fix
|
||||
gives an attacker a window against every downstream consumer.
|
||||
|
||||
Instead, report it **privately** to the maintainer via the canonical repository on
|
||||
Forgejo:
|
||||
Instead, report it privately to <security@fromaitochitta.com> — mark the subject
|
||||
`SECURITY`.
|
||||
|
||||
- Repository: `git.fromaitochitta.com/open/llm-ingestion-pipeline-security`
|
||||
- Contact the maintainer directly through that Forgejo instance (private message /
|
||||
maintainer contact) and mark the subject `SECURITY`.
|
||||
- Canonical repository: https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security
|
||||
- Alternatively, contact the maintainer directly through that Forgejo instance
|
||||
(private message / maintainer contact) and mark the subject `SECURITY`.
|
||||
|
||||
Please include:
|
||||
|
||||
|
|
|
|||
317
docs/2026-09-07-limitations-44-maaling.md
Normal file
317
docs/2026-09-07-limitations-44-maaling.md
Normal file
|
|
@ -0,0 +1,317 @@
|
|||
# Beslutningsgrunnlag — LIMITATIONS «punkt 44» (tags/description-gapet)
|
||||
|
||||
**Målt:** 2026-09-07, mot `HEAD` = `44e2b31` (v1.3.0, urørt) og pinnet SPEC
|
||||
`_okf-canonical` @ `ad30107`. **Korpus:** `_okf-upstream` @ `3fcbb9f`.
|
||||
**Hva dette er:** underlaget for én operatørbeslutning — hvilket predikat, om
|
||||
noe, skal slippe inn den formen `tags` faktisk har i korpuset.
|
||||
**Hva dette ikke er:** beslutningen. Ingen fil i `src/` er endret, ingen
|
||||
versjon er bumpet, ingenting er pushet (0 upushede commits ved øktstart).
|
||||
|
||||
---
|
||||
|
||||
## 1. Referenten: «punkt 44» er ikke grepbart
|
||||
|
||||
Ordren og STATE peker på «LIMITATIONS punkt 44». Målt:
|
||||
|
||||
```
|
||||
$ grep -c '^- \*\*' docs/LIMITATIONS.md
|
||||
45
|
||||
$ grep -n '^- \*\*' docs/LIMITATIONS.md | awk -F: 'NR==44 {print $1}'
|
||||
756 # -> ZWJ-oppføringen, ikke tags/description
|
||||
```
|
||||
|
||||
Tallet stammer fra commit `0184df9` (25.08), som skrev «43 -> 44 items» i
|
||||
commit-meldingen: **44 var antallet oppføringer etter innsettingen, ikke
|
||||
oppføringens posisjon.** Oppføringen ble satt inn midt i dokumentet og er i dag
|
||||
den **12. av 45**, på `docs/LIMITATIONS.md:126`. Etiketten «punkt 44» løser seg
|
||||
altså ikke opp verken ved posisjon eller ved `grep`, og den vil peke feil igjen
|
||||
neste gang listen vokser. Referenten er entydig i prosa («tags/description»),
|
||||
men nummeret bør ikke brukes videre.
|
||||
|
||||
## 2. Hva punkt 44 sier i dag, ordrett
|
||||
|
||||
`docs/LIMITATIONS.md:126-144`:
|
||||
|
||||
> **`tags` and `description` block the OKF import corpus universally, before the
|
||||
> trust layer is even reached.** The line-flat frontmatter parser has no
|
||||
> sequence-value type at all: `tags` is present in 53/53 upstream concept
|
||||
> documents — 9/53 as a flow sequence (`[a, b, c]`, rejected on the `[`
|
||||
> indicator) and 44/53 as a block sequence (`- a` / `- b`, rejected as
|
||||
> `"malformed frontmatter line"`) — 100% rejection regardless of form.
|
||||
> `description` is present in 53/53; 29/53 is a folded plain scalar continuing
|
||||
> on an indented second line, which the parser has no continuation-line model
|
||||
> for and misreads as `"nested mappings are not supported"` (the remaining
|
||||
> 24/53 are single-line and parse fine). Measured directly on the upstream
|
||||
> reference bundles (`_okf-upstream/okf` @ `3fcbb9f`): removing `tags` alone
|
||||
> lets 4/53 documents pass; removing both `tags` and `description` together
|
||||
> (trust layer untouched) lets the same 4/53 pass, and all four then parse
|
||||
> `generated` correctly as a mapping. **Independent of the mapping-form work
|
||||
> above:** neither `1.2.0`'s flow mapping nor `1.3.0`'s `sources` carriers move
|
||||
> anything on this corpus, because `tags`/`description` reject before `sources`
|
||||
> is ever read. No sequence-value type or continuation-line model exists in the
|
||||
> stdlib-only parser to close this with.
|
||||
|
||||
## 3. SPEC, ordrett fra pinnet kopi (`ad30107`)
|
||||
|
||||
**§2 Terminology** (`SPEC.md:81`):
|
||||
|
||||
> - **Frontmatter**: A YAML metadata block delimited by `---` at the top of
|
||||
> a markdown file.
|
||||
|
||||
**§4 Concept documents** (`SPEC.md:157-159`):
|
||||
|
||||
> 1. A **YAML frontmatter block**, delimited by `---` on its own line at the
|
||||
> start of the file and a closing `---` on its own line.
|
||||
|
||||
**§4.1 Frontmatter**, skjelettet (`SPEC.md:163-173`):
|
||||
|
||||
> ```yaml
|
||||
> ---
|
||||
> type: <Type name> # REQUIRED
|
||||
> title: <Optional display name>
|
||||
> description: <Optional one-line summary>
|
||||
> resource: <Optional canonical URI for the underlying asset>
|
||||
> tags: [<tag>, <tag>, ...] # Optional
|
||||
> ```
|
||||
|
||||
**§4.1, de to bærende definisjonene** (`SPEC.md:194-199`):
|
||||
|
||||
> - `description`: A single sentence summarizing the concept. Used by
|
||||
> `index.md` generators, search snippets, and previews.
|
||||
> - `tags`: A YAML list of short strings for cross-cutting categorization.
|
||||
|
||||
**§11 Conformance** (`SPEC.md:738-742`):
|
||||
|
||||
> A bundle is **conformant** with OKF v0.2 if:
|
||||
>
|
||||
> 1. Every non-reserved `.md` file in the tree contains a parseable YAML
|
||||
> frontmatter block.
|
||||
|
||||
### 3.1 Det spec-en ikke sier
|
||||
|
||||
`grep -n -i 'nest\|depth\|indent' SPEC.md` gir **null** treff som uttrykker en
|
||||
dybde- eller innrykksregel (13 treff, alle i andre betydninger — «flat list of»
|
||||
i §9 og §13, «indent» ingen). **SPEC-en har ingen dybderegel.** «Ingen nesting
|
||||
forbi dybde 1» er utelukkende vår egen sikkerhetsegenskap. Konformans-gulvet er
|
||||
«a parseable YAML frontmatter block» — altså *hele* YAML.
|
||||
|
||||
To presiseringer som følger av ordlyden:
|
||||
|
||||
- `tags` er definert som «A YAML list» — **ikke** som flow-formen. Skjelettets
|
||||
`[<tag>, ...]` er ett eksempel, ikke formkravet. Blokkformen er like konform.
|
||||
- `description` er «A single sentence». En setning brutt over to linjer som
|
||||
foldet plain scalar er samme setning; spec-en stiller ingen linjekrav.
|
||||
|
||||
## 4. Måleoppsett
|
||||
|
||||
Korpuset er hentet ut ved pinnen, ikke fra arbeidstreet — `_okf-upstream` står
|
||||
i dag på `9a15b13`, og `3fcbb9f` er en ekte forgjenger (`git merge-base
|
||||
--is-ancestor` → 0), med 23 filer endret i `okf/` mellom dem.
|
||||
|
||||
```
|
||||
$ git -C _okf-upstream archive 3fcbb9f okf/bundles | tar -x -C scratchpad/corpus
|
||||
$ find scratchpad/corpus -name '*.md' ! -name index.md ! -name log.md | wc -l
|
||||
53
|
||||
```
|
||||
|
||||
**Nevner = 53.** Definisjonen er «hver `.md` under `okf/bundles/` som ikke er et
|
||||
reservert strukturnavn» — som er nøyaktig §11.1s «every non-reserved `.md` file».
|
||||
|
||||
**Positiv kontroll.** Et null-resultat må sjekkes mot et kjent-positivt tilfelle
|
||||
før det konsumeres. Følgende dokument parser i dag, mot samme `HEAD`:
|
||||
|
||||
```yaml
|
||||
type: Metric
|
||||
description: Recognized revenue for a period.
|
||||
tags:
|
||||
- finance # INNRYKKET blokksekvens
|
||||
- revenue
|
||||
generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-30T14:00:00Z }
|
||||
sources:
|
||||
- id: revenue-policy
|
||||
resource: policies/revenue-recognition.md
|
||||
```
|
||||
→ `tags` blir `['finance', 'revenue']`, `sources` blir en liste av dicter.
|
||||
Instrumentet avviser altså ikke alt; `tags` som blokksekvens **fungerer allerede
|
||||
i dag**, forutsatt innrykk.
|
||||
|
||||
## 5. Målingen
|
||||
|
||||
### 5.1 Formsensus over de 53 (`scratchpad/shapes.py`, exit 0)
|
||||
|
||||
| nøkkel | form | antall |
|
||||
|---|---|---|
|
||||
| `tags` | blokksekvens **uten innrykk** (`- a` i kolonne 0) | **36/53** |
|
||||
| `tags` | flow-sekvens `[a, b, c]` | 9/53 |
|
||||
| `tags` | enkeltlinje-skalar | 8/53 |
|
||||
| `description` | skalar + innrykket fortsettelseslinje | 29/53 |
|
||||
| `description` | enkeltlinje-skalar | 24/53 |
|
||||
| `generated` | **topp-nivå blokk-mapping** (` by:` på neste linje) | **44/53** |
|
||||
| `generated` | flow-mapping `{ ... }` | 9/53 |
|
||||
| `sources` | blokksekvens uten innrykk | 44/53 |
|
||||
| `sources` | blokksekvens med innrykk | 5/53 |
|
||||
|
||||
### 5.2 Baseline og kandidater (`scratchpad/candidates.py`, exit 0)
|
||||
|
||||
Kandidatene er målt ved å **normalisere overflateformen** inn i en form
|
||||
parseren allerede godtar, og så importere `parse_frontmatter`. Predikatet er
|
||||
aldri re-implementert; bare stavemåten på inputen er skrevet om.
|
||||
|
||||
| variant | passerer | dominerende residual |
|
||||
|---|---|---|
|
||||
| baseline (v1.3.0 som utgitt) | **0/53** | 32× nested mappings, 12× malformed line, 9× flow sequence |
|
||||
| **P1** skalar-flow-sekvens | **6/53** | 32× nested mappings, 12× malformed line |
|
||||
| **P2** blokksekvens uten innrykk | 0/53 | 44× nested mappings |
|
||||
| **P3** foldet plain scalar | 0/53 | 36× malformed line |
|
||||
| P1+P3 | 6/53 | 36× malformed line |
|
||||
| P1+P2 | 6/53 | 44× nested mappings |
|
||||
| **P1+P2+P3** | **6/53** | **44× nested mappings, 3× allowlist** |
|
||||
|
||||
De 6 som passerer med P1 er alle i `acme_retail`; de er 6 av de 9 med
|
||||
flow-sekvens-`tags`, og de tre siste stoppes av allowlisten
|
||||
(`parameters.name` ×2, `not.term` ×1), ikke av `tags`.
|
||||
|
||||
### 5.3 Taket
|
||||
|
||||
Med alle tre predikatene er taket **6/53**. Residualet er 44× topp-nivå
|
||||
blokk-mapping på `generated` og 3× allowlist. En diagnostisk kjøring som også
|
||||
normaliserte topp-nivå blokk-mapping til flow-mapping traff neste vegg med én
|
||||
gang: 44× «a quoted scalar inside a flow mapping is not a supported form» —
|
||||
korpuset skriver `at: '2026-07-10T23:16:06+00:00'` med enkeltfnutter.
|
||||
|
||||
## 6. Tre påstander i punkt 44 er målt feil
|
||||
|
||||
1. **«The line-flat frontmatter parser has no sequence-value type at all.»**
|
||||
Usant siden 1.3.0. `_consume_block_list` (`okf.py:662`) parser blokklister av
|
||||
rene skalarer, og den positive kontrollen i §4 beviser det. Det som mangler
|
||||
er ikke sekvenstypen, men **innrykkskravet**: `_consume_block_list` krever
|
||||
`raw[:1] in (" ", "\t")`, og korpuset skriver `- a` i kolonne 0.
|
||||
Feilklassen «44/53 rejected as malformed frontmatter line» er i dag 12/53,
|
||||
fordi de øvrige treffer `description`-fortsettelsen først.
|
||||
|
||||
2. **«removing `tags` alone lets 4/53 documents pass … removing both … the same
|
||||
4/53.»** Målt i dag: **6/53** med `tags` fjernet, og **6/53** med begge
|
||||
fjernet. Tallet 4 var riktig for 1.2.0-parseren; 1.3.0s `sources`-bærere
|
||||
flyttet to dokumenter til.
|
||||
|
||||
3. **«`tags`/`description` reject before `sources` is ever read» → derfor er
|
||||
dette «det ENESTE residualet som blokkerer hele korpuset».** Den slutningen
|
||||
holder ikke. `description` alene løsner **0/53** — fjerner man bare
|
||||
`description`, passerer ingenting. Og lukker man *alle tre* formene, står
|
||||
**44/53** fortsatt på `generated` som topp-nivå blokk-mapping. Den bindende
|
||||
skranken på dette korpuset er altså **ikke** tags/description, men den
|
||||
topp-nivå blokk-mappingen vi bevisst avviser. Punkt 44 overselger sin egen
|
||||
betydning med en faktor på over sju (6 mot 53).
|
||||
|
||||
Punkt 44 bør skrives om etter at operatøren har bestemt seg — det er en
|
||||
dokumentasjonsendring som hører sammen med predikatvalget, ikke før det.
|
||||
|
||||
## 7. Kandidatpredikatene
|
||||
|
||||
Ordren spurte etter predikatet som slipper inn en ren skalar-flow-sekvens.
|
||||
Det er P1. P2 og P3 tas med fordi målingen viser at P1 alene er en liten
|
||||
gevinst, og beslutningen bør se hva naboene koster.
|
||||
|
||||
### P1 — flow-sekvens av rene skalarer
|
||||
|
||||
Predikat: i `_parse_flow_sequence`, når første ikke-blanke tegn i et element
|
||||
ikke er `{`, les elementet som en plain scalar dersom det ikke inneholder noen
|
||||
av `{ } [ ] : , " ' #` og ikke er tomt. Blandet sekvens (skalar + mapping)
|
||||
avvises, slik blokklisten allerede gjør.
|
||||
|
||||
- **(a) Slipper inn:** `tags: [finance, revenue, headline-metric]` — 9/53 i
|
||||
korpuset. Passeringen går fra 0/53 til **6/53**.
|
||||
- **(b) Avviser fortsatt:** siterte elementer (`['a', 'b']`), elementer med
|
||||
kolon eller komma i seg, tom sekvens `[]`, uavsluttet `[a, b`, blandet
|
||||
`[a, {b: c}]`, og nestet `[[a]]` — alle på tegn-nivå, uten YAML-semantikk.
|
||||
- **(c) Dybde-1:** **bruker den ikke opp.** Elementene er blad; ingen ny
|
||||
nestingsgrad oppstår. Det er samme dybde blokklisten av skalarer allerede har.
|
||||
- **(d) Testen som pinner den:** `tags: [a, b]` → `["a", "b"]`; `tags: ['a']`
|
||||
raiser; `tags: [a, {b: c}]` raiser med blandingsfeilen; `tags: []` raiser;
|
||||
`sources: [{ id: x }]` parser uendret (ingen regresjon på G30-bæreren).
|
||||
|
||||
### P2 — blokksekvens uten innrykk
|
||||
|
||||
Predikat: i `_consume_block_list`, godta også `raw[:1] == "-"` når linjen
|
||||
starter med `- ` og forrige toppnøkkel hadde tom verdi.
|
||||
|
||||
- **(a) Slipper inn:** `tags:` + `- a` i kolonne 0 — 36/53. Men også `sources:`
|
||||
i samme form — 44/53.
|
||||
- **(b) Avviser fortsatt:** alt innholdet i elementet avviser i dag; allowlisten
|
||||
og `_reject_mapping_construct` er uendret.
|
||||
- **(c) Dybde-1:** **bruker den ikke opp** for skalarelementer, men den er ikke
|
||||
gratis: den åpner samtidig den ikke-innrykkede blokk-mapping-bæreren for
|
||||
`sources`, hvor elementenes fortsettelseslinjer *er* innrykket. Det er en
|
||||
større flate enn `tags`, og den bør vurderes for seg.
|
||||
- **(d) Testen:** `tags:\n- a\n- b` → `["a", "b"]`; `sources:\n- id: x\n title: y`
|
||||
→ én dict; en linje `- a` uten forutgående tom toppnøkkel raiser fortsatt;
|
||||
`tags:\n- a\n- {b: c}` raiser med blandingsfeilen.
|
||||
- **Målt effekt alene: 0/53.** Den løsner ingenting uten P1 eller uten at
|
||||
`generated` også åpnes.
|
||||
|
||||
### P3 — foldet plain scalar (fortsettelseslinje)
|
||||
|
||||
Predikat: etter en toppnøkkel med ikke-tom, ikke-`[`/`{` verdi, slå sammen
|
||||
etterfølgende innrykkede linjer som *ikke* starter med `- ` og *ikke* inneholder
|
||||
en uquotet `": "`, med ett mellomrom som skjøt.
|
||||
|
||||
- **(a) Slipper inn:** `description` brutt over to linjer — 29/53.
|
||||
- **(b) Avviser fortsatt:** en innrykket linje som ser ut som `k: v` treffer
|
||||
fremdeles nested-mapping-avvisningen; `- ` treffer fremdeles listeruten.
|
||||
`_reject_dangerous_value` kjører på den sammenslåtte verdien, ikke på
|
||||
fragmentene.
|
||||
- **(c) Dybde-1:** **bruker den ikke opp** — resultatet er én skalar.
|
||||
Men den svekker et vern: i dag er *enhver* innrykket linje uten aktiv listenøkkel
|
||||
et avvist nestet uttrykk. Etter P3 er den regelen betinget av at linjen ikke
|
||||
inneholder `": "` — altså samme heuristikk som `_reject_mapping_construct`,
|
||||
gjenbrukt til å *slippe gjennom* i stedet for til å avvise.
|
||||
- **(d) Testen:** `description: en setning\n som fortsetter` → én streng med
|
||||
ett mellomrom; `description: x\n y: z` raiser fortsatt som nested mapping;
|
||||
`description: x\n - a` raiser fortsatt.
|
||||
- **Målt effekt alene: 0/53.**
|
||||
|
||||
## 8. Anbefaling
|
||||
|
||||
**P1 alene. Ikke P2, ikke P3, ikke nå.**
|
||||
|
||||
Begrunnelsen er tallene, ikke smaken:
|
||||
|
||||
- P1 er den eneste av de tre som flytter passeringstallet i det hele tatt
|
||||
(0 → 6). P2 og P3 gir hver for seg **0/53**, og lagt oppå P1 gir de fortsatt
|
||||
**6/53**. De koster parserflate og kjøper null målt konformans.
|
||||
- P1 er den minste flaten: den er et tegn-nivå-predikat inne i en funksjon som
|
||||
allerede eksisterer, og den bruker ikke opp dybde-1-regelen.
|
||||
- P1 lukker et gap STATE allerede fører som bevisst («`tags: [a, b]`
|
||||
konformansgap»), og den bringer parseren i linje med §4.1s eget skjelett —
|
||||
den ene formen spec-en faktisk skriver ut.
|
||||
- P2 og P3 bør ikke besluttes på dette korpuset, fordi korpuset ikke kan skille
|
||||
dem: **44/53 stopper på `generated` som topp-nivå blokk-mapping uansett.**
|
||||
Å bygge P2 og P3 nå ville være å betale for to predikater og måle null.
|
||||
|
||||
**Den ærlige konsekvensen, som må sies høyt:** P1 tar korpuset fra 0/53 til
|
||||
6/53. Det lukker ikke «hele korpuset». Skal 53/53 nås, er den neste
|
||||
beslutningen en helt annen og mye tyngre en — topp-nivå blokk-mapping (44/53)
|
||||
pluss siterte skalarer (44/53) — og *den* bruker opp dybde-1-regelen. Det er en
|
||||
sikkerhetsbeslutning, ikke en parserdetalj, og den hører ikke i denne ordren.
|
||||
|
||||
## 9. Verifiseringslogg
|
||||
|
||||
| Påstand | Kommando | Exit | Resultat |
|
||||
|---|---|---|---|
|
||||
| Nevner = 53 | `find scratchpad/corpus -name '*.md' ! -name index.md ! -name log.md \| wc -l` | 0 | 53 |
|
||||
| Pinnen er ekte forgjenger | `git -C _okf-upstream merge-base --is-ancestor 3fcbb9f HEAD` | 0 | ja (HEAD = `9a15b13`) |
|
||||
| SPEC-pinnen er ren | `git -C _okf-canonical rev-parse --short HEAD; git status --porcelain` | 0 | `ad30107`, rent tre |
|
||||
| SPEC har ingen dybderegel | `grep -n -i 'nest\|depth\|indent' SPEC.md` | 0 | 13 treff, 0 relevante |
|
||||
| Baseline 0/53 | `PYTHONPATH=src .venv/bin/python scratchpad/measure.py` | 0 | 0 passerer |
|
||||
| Formsensus | `PYTHONPATH=src .venv/bin/python scratchpad/shapes.py` | 0 | tabell §5.1 |
|
||||
| Kandidattall | `PYTHONPATH=src .venv/bin/python scratchpad/candidates.py` | 0 | tabell §5.2 |
|
||||
| Positiv kontroll | inline, se §4 | 0 | parser, `tags == ['finance','revenue']` |
|
||||
| LIMITATIONS-antall | `grep -c '^- \*\*' docs/LIMITATIONS.md` | 0 | 45 |
|
||||
| Upushet ved øktstart | `git rev-list --count origin/main..HEAD` | 0 | **0** |
|
||||
|
||||
**Ikke målt:** om P1 påvirker ytelse eller ReDoS-marginen — predikatet er
|
||||
tegn-for-tegn uten regex, men ingen sveip er kjørt, siden ingen kode er skrevet.
|
||||
Kandidatene er målt ved overflatenormalisering, ikke ved en patchet parser: det
|
||||
er en trofast simulering av *hva* som slippes inn, men den beviser ikke at en
|
||||
implementasjon av P1 avviser nøyaktig (b)-listen. Testene i (d) er det som ville
|
||||
pinne det.
|
||||
|
|
@ -4,7 +4,7 @@
|
|||
especially one converging on Google's Open Knowledge Format (OKF v0.1) — and needs
|
||||
to decide **when** and **where** to add a write-time ingestion guard.
|
||||
|
||||
**Status of the guard:** `v1.0.0`. Stdlib-only core, framework-agnostic. The
|
||||
**Status of the guard:** `v1.3.0`. Stdlib-only core, framework-agnostic. The
|
||||
exported Python surface is frozen under semver — nothing exported is removed,
|
||||
renamed or given a different meaning without a `2.0.0`. Detection behaviour is
|
||||
*not* frozen: severities, thresholds and lexicon entries are calibration and move
|
||||
|
|
@ -142,9 +142,9 @@ live payload:
|
|||
python -m llm_ingestion_guard.coverage # exit 0 = all as documented
|
||||
```
|
||||
|
||||
As of `v1.0.0`: **129 / 129 defended classes demonstrated (recall 100%)** and **6 /
|
||||
As of `v1.3.0`: **130 / 130 defended classes demonstrated (recall 100%)** and **6 /
|
||||
6 documented gaps still hold** (a *closed* gap fails the test, forcing a doc
|
||||
update). The matrix is the single source of truth for the test suite (**792
|
||||
update). The matrix is the single source of truth for the test suite (**868
|
||||
passing**), which also asserts total recall, that every lexicon pattern has a
|
||||
case (so the matrix cannot fall behind the lexicon), the full LLM02 secret-egress
|
||||
set, and the container-layer front-end (CSV formula-injection, zip-slip/bomb,
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
**A reusable, minimal, dependency-light defensive layer for LLM *ingestion*
|
||||
pipelines — the write-time siblings of query-time chatbot guardrails.**
|
||||
|
||||
Status: implemented — v1.0.0, exported surface frozen under semver. This document defines what the repo contains
|
||||
Status: implemented — v1.3.0, exported surface frozen under semver. This document defines what the repo contains
|
||||
and why; the stdlib-only core is built and tested (see `README.md` for usage and
|
||||
`docs/PLAN.md` for the build order).
|
||||
|
||||
|
|
|
|||
|
|
@ -40,38 +40,125 @@ items; this is the full list, each with the mechanism.
|
|||
(an injection in a directory listing is caught) rather than path-rejecting the
|
||||
conformant bundle. A front-end materialising individual uploads keeps the opposite
|
||||
rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused.
|
||||
- **OKF frontmatter is a restricted grammar, and a one-key block-sequence item is
|
||||
silently misparsed.** Gate T2 accepts a line-oriented subset deliberately — full
|
||||
YAML is a larger parse-attack surface than a write-time gate needs. Nested mappings
|
||||
and flow collections (`[a, b]`, `{k: v}`) are *rejected outright*, which fails
|
||||
secure. **All three routes to a mapping fail, each on a different rule** — flow
|
||||
(`{k: v}`) on the disallowed value-start indicator, block (`k:\n sub: v`) on the
|
||||
nested-mapping check, and dotted keys (`k.sub: v`) on the key pattern — so the
|
||||
mapping *class* has no expressible form, rather than one form being preferable to
|
||||
another. What survives is scalars and flat lists of strings. The defect is between
|
||||
those two outcomes: a block sequence whose items carry
|
||||
exactly **one** key parses "successfully" into the wrong type —
|
||||
`sources:\n - uri: https://e.com/a` yields the **string** `'uri: https://e.com/a'`,
|
||||
not a mapping, while the same list with two keys per item hard-rejects. A pointer
|
||||
can therefore ride through in a key the `resource` allowlist never inspects
|
||||
(`attester:\n - resource: attesters/sql_equality.py` → WARN), whereas a top-level
|
||||
`resource:` with a relative path correctly fails secure. The shape is not conformant
|
||||
OKF, so a well-formed bundle will not produce it; a malformed or hostile one can, and
|
||||
mode-b `import_bundle` writes the merged concept verbatim. Note the three block-list
|
||||
shapes are *not* one case: flat scalars parse correctly, one key per item misparses
|
||||
silently, two keys per item hard-rejects.
|
||||
- **OKF frontmatter is a restricted grammar: a mapping is expressible through
|
||||
four carriers, all of them key-allowlisted.** Gate T2 accepts a line-oriented
|
||||
subset deliberately — full YAML is a larger parse-attack surface than a
|
||||
write-time gate needs. The admitted carriers are the flow mapping as a value
|
||||
(`generated: { by: x, at: y }`), the flow mapping as a block-list item, the
|
||||
flow *sequence* of flow mappings (`sources: [{ id: a, resource: x }]`) and the
|
||||
block sequence of block mappings (SPEC.md §5.1's own form). **The routes that
|
||||
still fail, each on a different rule:** a top-level block *mapping*
|
||||
(`k:\n sub: v`) on the nested-mapping check, dotted keys (`k.sub: v`) on the
|
||||
key pattern, the inline second colon (`k: sub: v`) on the mapping-construct
|
||||
check, and a flow sequence of plain *scalars* (`tags: [a, b]`) on the `[`
|
||||
indicator — the sequence carrier is opened for the mapping element and nothing
|
||||
else. **Every carrier is admitted only when every key is on an allowlist**
|
||||
(`by`, `at`, `from`, `to`, `id`, `title`, `author`, `usage_count`,
|
||||
`last_modified` — the keys SPEC.md @ `62432a09` §5.1/§5.2 names inside a mapping,
|
||||
plus `resource` and `usage_window` *under `sources` only*, see below)
|
||||
and every leaf is a plain scalar, itself run through the same value predicates as a
|
||||
top-level scalar. Nested collections, quoted leaves, duplicate keys, an empty or
|
||||
unclosed mapping, and `{a:b}` (which PyYAML 6.0.3 reads as the *key* `a:b`, not as
|
||||
a scalar) all raise. The form is expressible, never trusted: the allowlist
|
||||
inspects every key, which is the property that carried the security when the
|
||||
blanket refusal was doing the enforcing. **A block list may not mix scalar items
|
||||
and mappings** — YAML permits it, but a consumer iterating `sources` and reading
|
||||
`entry.get("id")` gets an `AttributeError` off the first `str`. What else
|
||||
survives is scalars and flat lists of strings. **Two routes used to
|
||||
degrade into a string instead of failing, and that defect is closed in `1.1.0`**:
|
||||
a block-sequence item carrying exactly one key (`sources:\n - uri: https://e.com/a`
|
||||
yielded the *string* `'uri: https://e.com/a'`) and the inline second colon
|
||||
(`attester: resource: attesters/sql_equality.py`, which a real YAML parser refuses
|
||||
outright). Both parsed "successfully" into the wrong *type*, and a pointer parked
|
||||
in one rode through in a key the `resource` allowlist never inspects — mode-b
|
||||
`import_bundle` returned WARN and wrote the merged concept verbatim. Both now
|
||||
FAIL_SECURE at T2, before the allowlist is reached. **What closed is the type
|
||||
confusion, not pointer-smuggling as a class:** T3 still inspects `resource` and
|
||||
nothing else, so an honest *string* under another key rides through exactly as
|
||||
before — `attester: attesters/sql_equality.py` is WARN, while the same path
|
||||
under `resource:` FAIL_SECUREs. That is by design (the string is scanned like
|
||||
any other frontmatter value under T1) and it is not what `1.1.0` changed.
|
||||
**The boundary is where YAML
|
||||
puts it**, ground-truthed against PyYAML 6.0.3: `": "` and a trailing `":"` open a
|
||||
mapping and are refused; a colon carrying neither a space nor a line end
|
||||
(`domain:security`, `https://e.com:8443/a`) does not and still parses, as does a
|
||||
quoted scalar (`- "uri: x"`). Quotes are retained rather than stripped — a
|
||||
divergence from YAML that remains, pinned in `tests/test_okf.py`.
|
||||
- **T2 constrains import, not emission.** The frontmatter grammar runs on
|
||||
`okf.import_bundle` (door C) only — `parse_frontmatter` is referenced nowhere in the
|
||||
door A/B persist path, so frontmatter that fails secure on import passes
|
||||
`screen_output` unremarked. The grammar therefore bounds what a consumer can *receive*,
|
||||
never what a producer can *emit*. Verified identical on 0.2.0 and 0.3.1.
|
||||
- **Consequence: an OKF v0.2 concept cannot traverse the external-import path.** Both
|
||||
of v0.2's backward-breaking migration targets are nested — `timestamp` → `generated.at`,
|
||||
and body `# Citations` → a `sources` block list of mappings — so a conformant v0.2
|
||||
concept fails secure at the frontmatter gate. This is the correct direction but it is
|
||||
a compatibility wall, not a policy: v0.2 support requires a deliberate parse-safety
|
||||
decision about widening the grammar, and the dangling-or-substituted `executor`/
|
||||
`attester` pointer question only becomes live once that decision is made.
|
||||
- **`sources` passes in both of its spec carriers; the per-entry `usage_window`
|
||||
does not.** The wall used to be total: both of v0.2's backward-breaking migration
|
||||
targets are mappings — `timestamp` → `generated.at`, and body `# Citations` → a
|
||||
`sources` block list of mappings — and a consumer measured **0 of 53** upstream
|
||||
concepts through the gate. `generated.at` stopped being a wall in `1.2.0`;
|
||||
`sources` stopped being one in `1.3.0`, which admits both the block sequence of
|
||||
block mappings (SPEC.md §5.1's own example) and the flow sequence of flow
|
||||
mappings (the form the OKF producers emit, measured 2026-09-02 by two consumers
|
||||
independently). **`resource` is allowlisted inside a `sources` entry and nowhere
|
||||
else.** `1.2.0` left it off on the argument that the parser could not tell
|
||||
`sources[].resource` (§5.1, a citation) from `executor.resource` / `attester.resource`
|
||||
(§10, a pointer to code to be run — door C). That premise was measured false: the
|
||||
owning key is in scope at every call site and was simply never threaded through.
|
||||
It is threaded now, so `executor: [{ resource: skills/run.md }]` and
|
||||
`attester:\n - resource: …` are refused on the allowlist through *every* carrier,
|
||||
including the two this opened. **What stays refused: a `usage_window` inside a
|
||||
`sources` entry.** SPEC §5.1 permits it per entry ("A single entry MAY carry its
|
||||
own `usage_window`"), and it is a mapping inside a mapping — depth 2, which this
|
||||
parser admits at no key. A bundle using the per-entry override is refused; the
|
||||
shared sibling `usage_window` (the §5.1 example's own form) passes. This is a
|
||||
registered conformance gap, not an oversight: no-nesting-past-depth-1 is a
|
||||
security property, and spending it was not what the fix was for.
|
||||
- **`sources[].resource` is scanned as text but never validated as a URL.** T3's
|
||||
https allowlist inspects the *top-level* `resource` and nothing else. It cannot
|
||||
be extended to `sources` entries without over-blocking conformant bundles: SPEC
|
||||
§5.1 explicitly permits a bundle-relative path, a path into `references/`, or a
|
||||
scope descriptor a consumer cannot follow at all (the OKF producers' own golden
|
||||
bundle emits `resource: fixture`). So a `sources` entry may carry
|
||||
`file://`, `javascript:` or any other string; it goes through T1's scan like any
|
||||
other frontmatter value, and nothing else. **A consumer that dereferences
|
||||
`sources[].resource` must validate it itself** — `okf.validate_resource_url` is
|
||||
exported for exactly that. The dangling-or-substituted `executor`/`attester`
|
||||
pointer question stays out of reach separately: both are top-level block
|
||||
mappings, a carrier that is still refused.
|
||||
- **`tags` parses in two of the three forms the OKF corpus writes it in;
|
||||
`description`'s folded form still does not — and neither is what caps the
|
||||
corpus.** Re-measured 2026-09-07/08 against the pinned upstream reference
|
||||
bundles (`_okf-upstream/okf` @ `3fcbb9f`, denominator **53** — every
|
||||
non-reserved `.md`, which is SPEC §11.1's own unit) and the pinned SPEC
|
||||
(`_okf-canonical` @ `ad30107`). This entry carried three claims that the
|
||||
measurement showed to be wrong, all three inherited from `1.2.0` and left
|
||||
standing for twelve days after `1.3.0` moved them; they are corrected here.
|
||||
- **A sequence-value type exists** — `_consume_block_list` has parsed block
|
||||
lists of plain scalars since `1.3.0`, and `1.4.0` adds the flow sequence of
|
||||
plain scalars (`tags: [a, b, c]`, SPEC §4.1's own skeleton). What is missing
|
||||
is not the type but the **indentation requirement**: `_consume_block_list`
|
||||
demands a leading space, and the corpus writes `- a` flush at column 0
|
||||
(36/53 of `tags`, 44/53 of `sources`). Shape census of `tags` over the 53:
|
||||
36/53 flush block sequence, 9/53 flow sequence, 8/53 single-line scalar.
|
||||
- **The number is 6/53, not 4/53.** `4` was the `1.2.0` parser's figure.
|
||||
Measured on `1.4.0`, the corpus goes from **0/53 to 6/53** — all six in
|
||||
`acme_retail`, being 6 of the 9 documents whose `tags` is a flow sequence;
|
||||
the other three stop on the mapping-key allowlist (`parameters.name` ×2,
|
||||
`not.term` ×1), not on `tags`.
|
||||
- **`tags`/`description` are not the binding constraint.** `description`
|
||||
alone releases **0/53** — 29/53 write it as a folded plain scalar over an
|
||||
indented second line, which the parser has no continuation-line model for,
|
||||
but removing that obstacle by itself lets nothing through. With all three
|
||||
surface forms closed the corpus still measures **6/53**, because **44/53
|
||||
stop on `generated` written as a top-level block mapping**, and behind that
|
||||
wall sit 44/53 single-quoted scalars inside the mapping. That constraint is
|
||||
refused deliberately: it spends the no-nesting-past-depth-1 rule, which is
|
||||
a security decision and not a parser detail.
|
||||
**Said plainly: `1.4.0` takes this corpus to 6/53, not to 53/53.** The two
|
||||
neighbouring predicates were measured and deliberately **not** built — a
|
||||
flush-left block sequence and a folded plain scalar each release **0/53** on
|
||||
their own, and stacked on top of `1.4.0` they still measure 6/53. They would
|
||||
cost parser surface and buy no measured conformance. The full measurement,
|
||||
including the SPEC quotations and the per-candidate table, is
|
||||
`docs/2026-09-07-limitations-44-maaling.md`.
|
||||
- **A persist gate cannot cover execution risk.** OKF v0.2 introduces concepts whose
|
||||
purpose is to *name code to be run* (`runtime`, `executor.resource`,
|
||||
`attester.resource`). This library answers "is this safe to **store**"; executable
|
||||
|
|
@ -295,6 +382,21 @@ items; this is the full list, each with the mechanism.
|
|||
`test_split_tightens_the_trusted_tier_when_both_carriers_are_present` constructs
|
||||
and pins. Read the zero as "not observed in any of the three populations, each
|
||||
counted against its own denominator", never as "cannot happen".
|
||||
- **The split also LOOSENS the upload door for a lone anchor — the direction it was
|
||||
built for, and the one with a residual worth naming.** Measured as shipped:
|
||||
`<a href="https://ext.example/p">t</a>` on its own emits one
|
||||
`active:raw-html-link` at MEDIUM and disposes `quarantine_review` under
|
||||
`PRESET_USER_UPLOAD` (`warn` under `PRESET_TRUSTED_SOURCE`); through 0.6.1 the
|
||||
same document graded HIGH and `fail_secure`d. An `<img src>` to the same host is
|
||||
untouched — `active:raw-html`, HIGH, `fail_secure`. So an anchor pointing at an
|
||||
attacker-controlled host, arriving on an untrusted upload, is now a human decision
|
||||
rather than a halt. The trade is deliberate and it removes an asymmetry that came
|
||||
from syntax rather than affordance: following an anchor needs a click, exactly like
|
||||
the markdown inline link that has graded MEDIUM since 0.3.1, so the same URL no
|
||||
longer grades two different ways depending on which syntax carries it. It is
|
||||
recorded here so 0.7.0's "frees N documents" is not read as free — what was freed
|
||||
is the click-required class, and MEDIUM is a real grade drop on the door where
|
||||
every finding is trust-escalated.
|
||||
- **Raw-HTML findings no longer count end tags, and that moved a published field.**
|
||||
Through 0.6.1 `</a>` was active by name on its own, so `count` ran roughly 1.6×
|
||||
the opening-tag total (measured on one corpus) and a start/end pair counted 2.
|
||||
|
|
@ -303,6 +405,44 @@ items; this is the full list, each with the mechanism.
|
|||
every document carrying `</a>`, on a field whose meaning did not change. The
|
||||
finding count is unaffected: the class still collapses to one finding per class
|
||||
per document, and `count` was never a document count.
|
||||
- **Which tags the no-URL narrowing may render inert is a judgement about affordance,
|
||||
and no test in this repo can derive it.** `_URL_AFFORDANCE_TAGS` holds the nine
|
||||
names whose entire active affordance *is* the URL they name — `a`, `area`, `img`,
|
||||
`video`, `audio`, `source`, `track`, `frame`, `frameset` — so carrying no URL
|
||||
attribute they name no target and grade inert. Every other name in the active set
|
||||
stays active with no attributes at all, because it does something a URL cannot
|
||||
describe: `<script>` executes its body, `<style>` restyles, `<form>` submits. That
|
||||
boundary is asserted, not measured. A name placed in the set whose affordance does
|
||||
*not* reduce to its URL would go silently invisible, and no corpus can catch it,
|
||||
because what it produces is an absence — the census counts findings, and a tag that
|
||||
stopped firing contributes nothing to count. The fail-secure choice one branch
|
||||
further in holds the other way and is worth reading beside it: a URL attribute whose
|
||||
*value* this module cannot resolve keeps the tag active
|
||||
(`active_content.py:320-324`), a branch the three corpora exercise **0** times.
|
||||
That zero is empirical, so the predicate is written not to depend on it.
|
||||
- **"Clean" means *graded, no finding raised* — never *cleaned bytes* — and at one
|
||||
measured consumer's door, `warn` is the floor a document must clear to be
|
||||
persisted rather than rejected.** The word is the library's own: a WARN
|
||||
disposition with nothing to report carries the reason string `"clean: no
|
||||
findings"` (`disposition.py:264`), and this project has repeated that word in the
|
||||
tables it sends consumers. `screen_output` is a judgement API — its
|
||||
`DispositionResult` carries `assessment` / `disposition` / `max_severity` /
|
||||
`reasons`, with no sanitized-text field to read off it. Defanging lives in a
|
||||
separate, deliberate call — `neutralize` — that a caller must invoke itself;
|
||||
nothing upstream of that call transforms a byte. Measured against
|
||||
`llm-ingestion-okf`'s `0.7.0` pin (2026-08-13): `inbox.py:139` sets its persist
|
||||
floor to `warn`, and `inbox.py:323` persists anything carrying that disposition
|
||||
into the bundle; its adapter (`guard_adapter.py:70`) forwards the original
|
||||
extracted text, because nothing upstream ever handed it a transformed one. Four
|
||||
raw-HTML carrier forms the 0.7.0 no-URL narrowing grades inert — an
|
||||
`<a aria-label>` with no `href`, a bare `</a>`, `<Frame>`, `<video />` — verified
|
||||
here (`screen_output(..., PRESET_USER_UPLOAD)`) to dispose `warn, clean: no
|
||||
findings`; at that consumer's door the same four land written into the bundle,
|
||||
carrier present verbatim. Neither library is wrong: `screen_output` never
|
||||
promised transformed bytes, and the consumer never called `neutralize` for them.
|
||||
The gap is in reading "clean" as "sanitized" rather than "no finding raised" — a
|
||||
reading this project's own reports invite, and one that will mislead any caller
|
||||
that persists on `warn` without calling `neutralize` itself.
|
||||
- **Measured, document by document: a large minority of *benign* documents do not
|
||||
persist unattended at the upload door.** The bullets above bound single rules on
|
||||
single URLs. This one bounds the thing a consumer actually feels — how often an
|
||||
|
|
@ -429,6 +569,23 @@ items; this is the full list, each with the mechanism.
|
|||
HIGH under a low-trust preset is `fail_secure`. Report-only means the text is
|
||||
never mutated — it does not mean the finding cannot block.
|
||||
|
||||
- **The two labels a `<script>` tag raises come from patterns that do not match the
|
||||
same strings.** `hybrid-xss:script-tag` is `<script\b[^><]*>`; `active:raw-html`
|
||||
reads the same tag through `HTML_TAG_RE`, which consumes quoted attribute runs
|
||||
atomically and so tolerates both `<` and `>` inside a quoted value. The lexicon's
|
||||
`<` exclusion is not a modelling choice — it is the 0.3.3 ReDoS fix, and widening
|
||||
it back to `[^>]` restores a quadratic arm (the row below carries the numbers).
|
||||
Measured through both scanners: `<script src="a<b">` raises `active:raw-html` and
|
||||
**no** XSS label, while `<script data-t="a>b">` raises both, the lexicon's match
|
||||
simply ending at the quoted `>`. The disposition never moves — the raw-HTML branch
|
||||
grades `<script>` HIGH with no attributes at all, so every shape here still
|
||||
`fail_secure`s under `PRESET_USER_UPLOAD` — so what the divergence costs is the
|
||||
*label*: a consumer filtering findings on the XSS id sees a subset of the script
|
||||
tags the gate actually caught, and must not read that id as the gate's script-tag
|
||||
census. The residual is practically dead in prose (a `<` inside a script tag's
|
||||
quoted attribute region is not an ordinary shape) and is recorded because the
|
||||
asymmetry is invisible from either scanner alone.
|
||||
|
||||
- **A connection-string password longer than 256 chars is not matched.** The
|
||||
password run in the `*-connstr` egress patterns is bounded by
|
||||
`MAX_CONNSTR_VALUE`; unbounded, it sits in front of a mandatory `@` and makes
|
||||
|
|
@ -445,24 +602,96 @@ items; this is the full list, each with the mechanism.
|
|||
missed; on one preset it is held for review instead of halted.
|
||||
|
||||
- **The ReDoS sweep has a measured sensitivity floor, not a clean bill of
|
||||
health.** All 150 compiled patterns across all eleven regex-bearing modules are
|
||||
health.** All 152 compiled patterns across all eleven regex-bearing modules are
|
||||
swept arm by arm — payloads synthesised per run from each pattern's own
|
||||
skeleton, so `[`, `[system]` and `[system](` are each probed separately rather
|
||||
than relying on generic units, and each pattern is timed in the call mode the
|
||||
production code uses (`.sub()`/`.finditer()` visit every start position where
|
||||
`.match()` cannot). Five patterns were quadratic across 0.3.3 and 0.3.4; all
|
||||
are fixed. But the sweep flags on *timing*, and it ignores measurements below a
|
||||
1.5 ms noise floor at N=8000. A quadratic arm sitting just under that floor
|
||||
would still cost **up to ~23 s** at the 1 000 000-char cap. So the claim this
|
||||
sweep supports is "no arm worse than ~23 s at the cap", not "no quadratic arm
|
||||
remains". The method's blind spot is real and has now been demonstrated twice:
|
||||
a generic-payload pass found only one of 0.3.3's two patterns, and 0.3.2's
|
||||
1.5 ms noise floor at N=8000 — process CPU time, re-derived on that instrument
|
||||
(see the clock bullet below) rather than inherited from the wall clock the
|
||||
script used through 1.1.0. A quadratic arm sitting just under that floor would
|
||||
still cost **up to ~23 s** at the 1 000 000-char cap — that figure is arithmetic
|
||||
and not a measurement: a quadratic arm costs the square of the length ratio, and
|
||||
1.5 ms × 125 × 125 is 23.4 s. So the claim this sweep supports is "no arm worse
|
||||
than ~23 s at the cap", not "no quadratic arm remains". The method's blind spot
|
||||
is real and has now been demonstrated twice: a generic-payload pass found only
|
||||
one of 0.3.3's two patterns, and 0.3.2's
|
||||
hand-written rows missed all three of 0.3.4's — including one on `sanitize`,
|
||||
the first thing every ingested document touches. **Two arm shapes the unit-
|
||||
repetition payloads cannot express** are pinned by hand as a result: a tag that
|
||||
*closes* around a long body, and a run of plain characters carrying no anchor
|
||||
at all.
|
||||
|
||||
- **A green ReDoS row is evidence only if it has been seen red, and three rows in
|
||||
this suite had never been.** The class is not a bad bound but a payload that cannot
|
||||
reach the defect, and it leaves the row passing under the vulnerable form too. The
|
||||
sub-agent row is the clearest case: the seed's unbounded lazy run costs per *prefix
|
||||
match*, not per character — each start position where `spawn an agent that ` matches
|
||||
drives its own O(N) scan to end-of-string looking for a capability keyword the
|
||||
payload never supplies, so K prefix matches cost K×O(N), and the bounded
|
||||
`(?:\S+\s+){0,12}?` port caps each scan at 12 tokens for K×O(1). A payload that
|
||||
matches the prefix **once** and then pads pays a single lazy run and is linear
|
||||
however long the pad is — two earlier shapes did exactly that, and the row sat
|
||||
measured-dead at 1.2× until the payload was rebuilt as
|
||||
`"spawn an agent that " * 3000`. (The nesting an older comment blamed is a red
|
||||
herring: the inner `.*?` sits in an optional group, never a repeated one.) Measured
|
||||
through `scan_lexicon` with the seed form patched back in — exponent **1.92** against
|
||||
the shipped **1.01**, and **4.091s vs 0.190s** at 12 000 words, the seed breaking the
|
||||
2.0s bound outright. Two siblings were dead for different reasons.
|
||||
`lexicon-script-tag` had to be given its own N=200 000: at the shared N=100 000 the
|
||||
vulnerable `[^>]` form measured only ~1.2–1.4s — under the 2.0s assert, so the row
|
||||
was green under both forms and proved nothing. And
|
||||
`test_gate_is_bounded_on_the_long_attribute_arm` was killed by **this repo's own
|
||||
narrowing**: 0.7.0 put `<a>` in `_URL_AFFORDANCE_TAGS`, so its `<a ` + 100k + `>`
|
||||
payload became inert and returned *before* the body ever reached the arm the row
|
||||
exists to guard — separation 1.0×, 0.028s and no findings, against 12.475s for the
|
||||
same payload carried by `<script `. The carrier was moved to `<script `, which is
|
||||
active by name with no attributes, so no future URL-shaped narrowing can hollow it
|
||||
out the same way. The general rule the three share: a payload must **deny** the
|
||||
literal the vulnerable run sits in front of — a unit that supplies it matches
|
||||
immediately and never exercises the run. **Nothing but hand measurement finds this
|
||||
class.** The row is green either way, so
|
||||
the suite cannot report its own blind spot, and every bound in it should be read as
|
||||
"verified red under the vulnerable form" only where a comment says it was.
|
||||
|
||||
- **Every ReDoS bound in the suite is measured on process CPU time, and so is the
|
||||
sweep that sets the published sensitivity floor.** `tests/redos_clock.py` is the one
|
||||
clock all six test files import — `time.process_time()` — because a blowup is spent
|
||||
cycles while a loaded machine steals wall clock without adding any. On
|
||||
`time.monotonic()` two 0.7.0 rows failed at **2.24s / 3.66s** against a 2.0s bound
|
||||
while two census processes held the CPU, and passed 3/3 on an idle machine: they had
|
||||
been descheduled, not slowed. It lives in one module rather than five copies because
|
||||
`test_output.py::test_the_redos_clock_ignores_time_this_process_did_not_spend` pins
|
||||
one implementation, and four unpinned copies would be free to drift back to a wall
|
||||
clock with nothing going red. **What the CPU clock gives up, stated: a scan that
|
||||
BLOCKS forever burns no CPU, so it would hang the suite instead of failing it.**
|
||||
That is acceptable only because every scanner it measures is pure regex over an
|
||||
in-memory string, with no I/O and no locks — the last wall-clock holdout was retired
|
||||
by auditing its path for anything that could block, not by assumption, and a wall
|
||||
clock guarding a mode that cannot occur still charges the false-red premium
|
||||
(measured there at 21.6s against a 10.0s bound, on a scan that spent 7.6s). **The sweep
|
||||
now runs on the same clock, and closing that divergence bought no sensitivity.**
|
||||
`docs/redos-sweep.py` imports `scan_seconds` instead of timing on
|
||||
`time.monotonic()`, so the 1.5 ms floor at N=8000 and the "~23 s at the cap"
|
||||
figure above are finally in the same currency as the bounds they justify. What
|
||||
the move did *not* do is quiet the sweep, and the floor came back unchanged.
|
||||
Measured over **twelve full runs of all 2585 arms**: the median ratio sits at
|
||||
**1.95–2.03** in every size bucket above 50 µs — the whole surface measures
|
||||
linear — while two-point excursions past the 2.6 flag threshold survive at every
|
||||
magnitude, p99 ratio **2.9–3.3 even above 1 ms**. Flagged arms per run by floor:
|
||||
**6.9 at 0.5 ms, 1.1 at 1.0 ms, 0.33 at 1.5 ms** (0–2 per run), so 1.5 ms is
|
||||
still the knee. Descheduling was never what made this sweep noisy — a ratio
|
||||
computed from two points is. Four distinct arms flagged at the shipped floor
|
||||
across those twelve runs, **each in exactly one of them**, and nine of the twelve
|
||||
runs were clean; eight consecutive runs of the shipped script immediately after a
|
||||
full test run flagged 0–3 arms each, so machine load still moves the count even
|
||||
on a CPU clock. Six flagged arms re-measured over six doublings give exponent
|
||||
**0.97–1.09** and at most 1.2 s at the 1 000 000-char cap. **A single clean run
|
||||
of this sweep is therefore not evidence either** — and neither is a single
|
||||
flagged one.
|
||||
|
||||
- **Every surface now bounds its input, but not all of them the same way.**
|
||||
`sanitize`, `fence` and `neutralize` raise `OversizeInputError` above
|
||||
`MAX_INPUT_CHARS` (1 000 000) rather than returning a partially transformed
|
||||
|
|
@ -525,6 +754,23 @@ fails the test, forcing this doc to be updated:
|
|||
names the outward fetch whatever severity it carries — and must not infer
|
||||
"nothing was seen" from a low `Severity`.
|
||||
|
||||
- **Under the default action map the assessment axis carries exactly one judgement
|
||||
the disposition does not.** `DEFAULT_ACTION_MAP` sends `NONE` and `LOW` to `WARN`,
|
||||
`ELEVATED` to `QUARANTINE_REVIEW` and `SEVERE` to `FAIL_SECURE` — the last two 1:1.
|
||||
So for any document that carries a finding at all, `assessment` is a relabelling of
|
||||
`disposition` and nothing more; the only thing it adds is *clean* versus *findings
|
||||
present, none dispositive in this context*, which 0.4.0 rendered identically. That
|
||||
collapse is the point (the map is what keeps the separation additive, so a caller
|
||||
ignoring the new axis sees no change), and it is also the limitation: reading
|
||||
`assessment` buys a consumer nothing until it supplies its own `action_map` or needs
|
||||
the clean/low distinction. **The second consequence is on this document.** The
|
||||
published false-positive rates are counts of documents *disposed non-WARN*, and they
|
||||
are a statement about assessed risk only while `NONE` + `LOW` are exactly the WARN
|
||||
pre-image. `tests/test_corpus.py::test_the_published_fp_metric_is_a_risk_statement`
|
||||
pins that equivalence — but it pins it for `DEFAULT_ACTION_MAP`. A caller running its
|
||||
own map makes "disposed non-WARN" a different claim from the one measured here, with
|
||||
nothing in either repo failing to say so.
|
||||
|
||||
- **A ZWJ hidden between two emoji is exempt, and ZWNJ's own false-positive
|
||||
class is untouched.** U+200D composes emoji (👩💻 is WOMAN + ZWJ + PERSONAL
|
||||
COMPUTER), so testing it on codepoint membership alone flagged *and stripped*
|
||||
|
|
@ -543,6 +789,22 @@ fails the test, forcing this doc to be updated:
|
|||
not pictographic) and no corpus is available here to verify it against, so it
|
||||
is parked as a known false-positive class rather than guessed at.
|
||||
|
||||
- **That context test is one predicate on two surfaces, and the symbol carrying it is
|
||||
private.** `sanitize` owns `_is_joiner_in_emoji_sequence`; `output` imports it
|
||||
(`output.py:74`) instead of restating it, because the same defect had to be fixed on
|
||||
both surfaces and a split would let the input side stop flagging while the output
|
||||
side kept hard-blocking — or the reverse, which is how a carrier reaches a persisted
|
||||
artifact after passing the input gate. The agreement is pinned by
|
||||
`tests/test_output.py::test_output_zwj_narrowing_matches_the_sanitize_side`, which
|
||||
asserts `stripped == flagged` across six shapes — half-context on either side, a
|
||||
leading and a trailing joiner, one genuine in-sequence joiner, and a word split.
|
||||
Two things that pin does not give. The six shapes are hand-written rather than
|
||||
drawn from a corpus, so everywhere outside them
|
||||
the surfaces agree by *shared implementation*, not by test — which is the stronger
|
||||
guarantee only for as long as the import survives. And the leading underscore means
|
||||
the predicate is **not** part of the surface frozen under semver: a consumer that
|
||||
imports it is pinning a private name 1.x makes no promise about.
|
||||
|
||||
## Out-of-scope (documented boundary)
|
||||
|
||||
Embedding/vector-layer defenses (OWASP LLM08, downstream of persist); multimodal
|
||||
|
|
|
|||
|
|
@ -228,7 +228,18 @@ Nøkkelantakelser (+ test) · Verifisering. Testkommando alltid:
|
|||
→ +N grønne; `python -c "import tomllib,pathlib; d=tomllib.loads(pathlib.Path('pyproject.toml').read_text()); assert d['project']['dependencies']==[] and 'pypdf' in ' '.join(d['project']['optional-dependencies']['dev'])"` → exit 0.
|
||||
- **Avhengigheter:** uavhengig; kan gjøres når som helst før G.
|
||||
|
||||
### Session G — v1.0 freeze + release *(FRYSER Python-surfacen — Node-prereq)*
|
||||
### Session G — v1.0 freeze + release *(LANDET 2026-08-13)*
|
||||
|
||||
> **LANDET — `98ebc07`, tag `v1.0.0` pushet.** D1–D6 i `docs/GATE-G-v1.md` §6 tatt av
|
||||
> operatøren: frys på 0.7.0 (D2 i), `:492`/`:460`/`:88` konsedert i 1.x (D3/D5,
|
||||
> `e9d8fb2`), varslingsplikten fyrte ikke (D4 — begrunnelse over ved løfte 1),
|
||||
> `active_tag_class` er IKKE en pinnbar shape (D6). **Åtte flater bumpet for hånd;
|
||||
> den niende — Forge-beskrivelsen — ble VERIFISERT mot API-et og trengte ingen
|
||||
> endring** (178 kodepunkter, ingen versjon, ingen «alpha»). Klassifiseringssveipet
|
||||
> (421 treff) og `git show 98ebc07:README.md` kjørte begge FØR taggen, i den
|
||||
> rekkefølgen; anonym `pip install …@v1.0.0` i rent venv etter. Sveipet fant to
|
||||
> flater lista under ikke navnga: READMEs status-**badge** og ADOPTION-BRIEFs
|
||||
> testtall (`791` mot 792). 792 tester, 129/129, 6/6.
|
||||
|
||||
- **Mål:** shippe v1.0.0; fryse den offentlige surfacen som porten oversetter.
|
||||
- **Scope-grense:** ingen ny feature. Kun versjons-bump, CHANGELOG, tag, push.
|
||||
|
|
@ -428,6 +439,18 @@ ikke en preferanse.**
|
|||
og bygger på 0.3.1-formen; en stille re-stramming lander som produksjonsincident hos
|
||||
dem, ikke som en release-note. Gjelder også 0.4.0: akse-separasjonen skal endre
|
||||
DISPOSISJON, ikke graderingen — viser det seg feil under scoping, fyrer løftet.
|
||||
> **D4, avgjort av operatøren 2026-08-13 ved v1.0-frysen: løftet fyrte IKKE av
|
||||
> rå-HTML-bevegelsen 0.3.1→0.7.0, og intet etterskuddsvarsel gikk ut.**
|
||||
> Begrunnelsen styrer, ikke ordlyden. Løftets formål er navngitt i teksten over:
|
||||
> en stille re-**stramming** lander som produksjonsincident hos dem. Alle fire
|
||||
> ordinære markdown-former er målt IDENTISKE 0.3.1 vs 0.7.0 (WARN/LOW, før/etter i
|
||||
> samme økt, `docs/GATE-G-v1.md` §4); de fire radene som flyttet seg LØSNET alle
|
||||
> (`<a href>` HIGH→MEDIUM, `<a aria-label>` og `</a>` HIGH→rent, ZWJ-emoji
|
||||
> HIGH→rent). En løsning kan ikke produsere incidenten løftet finnes for å hindre.
|
||||
> **Ordlyden («enhver endring») pekte motsatt vei, og det er den reelle
|
||||
> motforestillingen** — hadde den styrt, var varselet uteblitt i tre utgivelser.
|
||||
> Nedtegnet her, ikke i `STATE.md`, nettopp av grunnen seksjonen selv oppgir: et
|
||||
> fravær uten begrunnelse er ikke til å skille fra at vi glemte det.
|
||||
2. **Relativ-mål-asymmetrien** (relative lenker/bilder i en OKF-bundle flagges ikke) —
|
||||
lukkes den, får `llm-ingestion-okf` varsel **før** det shippes. Den er en Door
|
||||
C-egenskap, ikke et guard-gap: et merget konsept skrives VERBATIM, så vi reparerer
|
||||
|
|
|
|||
|
|
@ -37,17 +37,30 @@ import importlib
|
|||
import json
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
SRC = Path(__file__).resolve().parent.parent / "src" / "llm_ingestion_guard"
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
SRC = ROOT / "src" / "llm_ingestion_guard"
|
||||
sys.path.insert(0, str(SRC.parent))
|
||||
sys.path.insert(0, str(ROOT / "tests"))
|
||||
|
||||
from llm_ingestion_guard.lexicon import load_lexicon # noqa: E402
|
||||
from redos_clock import scan_seconds # noqa: E402
|
||||
|
||||
N1, N2 = 4_000, 8_000
|
||||
RATIO_FLAG = 2.6
|
||||
# RE-DERIVED on the CPU clock, not inherited from the wall clock this script used
|
||||
# through 1.1.0. The clock move fixed false REDS in the suite's bounds; it bought
|
||||
# this sweep no sensitivity. Over twelve full runs (2585 arms each) the median
|
||||
# ratio is 1.95-2.03 in every size bucket above 50 us -- the whole surface
|
||||
# measures linear -- yet two-point excursions past RATIO_FLAG survive at every
|
||||
# magnitude (p99 ratio 2.9-3.3 even above 1 ms). Flagged arms per run by floor:
|
||||
# 6.9 at 0.5 ms, 1.1 at 1.0 ms, 0.33 at 1.5 ms. The knee is here. Four arms
|
||||
# flagged across those twelve runs, each in exactly ONE of them, and six arms
|
||||
# that have ever flagged re-measure at exponent 0.97-1.09 over six doublings.
|
||||
# Descheduling was never what made this sweep noisy -- a two-point ratio is, so
|
||||
# read a clean run and a flagged run with the same suspicion.
|
||||
NOISE_FLOOR = 0.0015
|
||||
HARD_CAP = 20.0
|
||||
|
||||
|
|
@ -134,21 +147,35 @@ def build(unit: str, n: int) -> str:
|
|||
|
||||
|
||||
def t(rx: re.Pattern[str], text: str, mode: str = "search") -> float:
|
||||
"""Time one scan of ``text`` in the mode the production code actually uses."""
|
||||
"""Time one scan of ``text`` in the mode the production code actually uses.
|
||||
|
||||
On the SAME clock every ReDoS bound in the suite is measured against --
|
||||
``tests/redos_clock.py``, process CPU time -- imported rather than restated
|
||||
here, for the reason that module gives: a blowup is spent cycles, and a
|
||||
loaded machine steals wall clock without adding any. Until 1.1.0 this timed
|
||||
on ``time.monotonic()``, which made the floor below and the cap figure
|
||||
derived from it numbers from a different instrument than the bounds they
|
||||
justify. The closure is built BEFORE the clock starts, so only the scan is
|
||||
charged.
|
||||
"""
|
||||
mode = mode.rstrip("*")
|
||||
start = time.monotonic()
|
||||
if mode == "finditer":
|
||||
for _ in rx.finditer(text):
|
||||
pass
|
||||
def scan(s: str) -> None:
|
||||
for _ in rx.finditer(s):
|
||||
pass
|
||||
elif mode == "sub":
|
||||
rx.sub("", text)
|
||||
def scan(s: str) -> None:
|
||||
rx.sub("", s)
|
||||
elif mode == "match":
|
||||
rx.match(text)
|
||||
def scan(s: str) -> None:
|
||||
rx.match(s)
|
||||
elif mode == "fullmatch":
|
||||
rx.fullmatch(text)
|
||||
def scan(s: str) -> None:
|
||||
rx.fullmatch(s)
|
||||
else:
|
||||
rx.search(text)
|
||||
return time.monotonic() - start
|
||||
def scan(s: str) -> None:
|
||||
rx.search(s)
|
||||
return scan_seconds(scan, text)
|
||||
|
||||
|
||||
# --- targets ----------------------------------------------------------------
|
||||
|
|
|
|||
7
llms.txt
Normal file
7
llms.txt
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
# llm-ingestion-guard
|
||||
|
||||
> Write-time defensive layer for Python pipelines that persist LLM output: sanitize, fence, tool-less quarantined transform, capability isolation, scan before persist, fail-secure.
|
||||
|
||||
```bash
|
||||
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v1.1.0"
|
||||
```
|
||||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "llm-ingestion-guard"
|
||||
version = "1.0.0"
|
||||
version = "1.4.0"
|
||||
description = "Write-time defensive layer for Python pipelines that persist LLM output: sanitize, fence, tool-less quarantined transform, capability isolation, scan before persist, fail-secure."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
|
|
|
|||
155
scratchpad/candidates.py
Normal file
155
scratchpad/candidates.py
Normal file
|
|
@ -0,0 +1,155 @@
|
|||
"""What each candidate predicate would admit, measured by normalizing the SURFACE
|
||||
form onto a shape the CURRENT parser already accepts. The predicate is imported,
|
||||
never re-implemented: only the input's spelling is rewritten."""
|
||||
import os, re, sys, collections
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "src"))
|
||||
from llm_ingestion_guard import okf
|
||||
|
||||
ROOT = os.path.join(os.path.dirname(__file__), "corpus", "okf", "bundles")
|
||||
|
||||
def docs():
|
||||
out = []
|
||||
for dirpath, _d, fns in os.walk(ROOT):
|
||||
for fn in sorted(fns):
|
||||
if fn.endswith(".md") and fn not in ("index.md", "log.md"):
|
||||
out.append(os.path.join(dirpath, fn))
|
||||
return sorted(out)
|
||||
|
||||
def split_fm(text):
|
||||
lines = text.split("\n")
|
||||
if not lines or lines[0].strip() != "---":
|
||||
return None, None
|
||||
for i in range(1, len(lines)):
|
||||
if lines[i].strip() == "---":
|
||||
return lines[1:i], lines[i + 1:]
|
||||
return None, None
|
||||
|
||||
TOPKEY = re.compile(r"^([A-Za-z0-9_]+):(.*)$")
|
||||
|
||||
def n_flow_scalar_seq(fm):
|
||||
"""P1: `k: [a, b]` where every element is a plain scalar -> indented block list."""
|
||||
out = []
|
||||
for raw in fm:
|
||||
m = TOPKEY.match(raw)
|
||||
if m and m.group(2).strip().startswith("[") and m.group(2).strip().endswith("]"):
|
||||
inner = m.group(2).strip()[1:-1]
|
||||
elems = [e.strip() for e in inner.split(",")]
|
||||
if inner and all(e and "{" not in e and "}" not in e and "[" not in e for e in elems):
|
||||
out.append("%s:" % m.group(1))
|
||||
out.extend(" - %s" % e for e in elems)
|
||||
continue
|
||||
out.append(raw)
|
||||
return out
|
||||
|
||||
def n_flush_block_seq(fm):
|
||||
"""P2: a flush-left `- item` run under a bare `k:` -> the same run, indented."""
|
||||
out, i, n = [], 0, len(fm)
|
||||
while i < n:
|
||||
raw = fm[i]
|
||||
m = TOPKEY.match(raw)
|
||||
if m and m.group(2).strip() == "" and i + 1 < n and fm[i + 1][:1] == "-":
|
||||
out.append(raw)
|
||||
i += 1
|
||||
while i < n and (fm[i][:1] == "-" or fm[i][:1] in (" ", "\t")):
|
||||
out.append(" " + fm[i])
|
||||
i += 1
|
||||
continue
|
||||
out.append(raw)
|
||||
i += 1
|
||||
return out
|
||||
|
||||
def n_folded_scalar(fm):
|
||||
"""P3: `k: text` continued on more-indented plain lines -> one joined line."""
|
||||
out, i, n = [], 0, len(fm)
|
||||
while i < n:
|
||||
raw = fm[i]
|
||||
m = TOPKEY.match(raw)
|
||||
if m and m.group(2).strip() and not m.group(2).strip()[0] in "[{":
|
||||
acc = raw
|
||||
i += 1
|
||||
while i < n and fm[i][:1] in (" ", "\t") and not fm[i].strip().startswith("- "):
|
||||
acc = acc.rstrip() + " " + fm[i].strip()
|
||||
i += 1
|
||||
out.append(acc)
|
||||
continue
|
||||
out.append(raw)
|
||||
i += 1
|
||||
return out
|
||||
|
||||
def parses(fm, body):
|
||||
text = "---\n" + "\n".join(fm) + "\n---\n" + "\n".join(body)
|
||||
try:
|
||||
okf.parse_frontmatter(text)
|
||||
return None
|
||||
except okf.OKFFrontmatterError as exc:
|
||||
return str(exc)
|
||||
|
||||
COMBOS = [
|
||||
("baseline (v1.3.0 as shipped)", []),
|
||||
("P1 scalar flow sequence", [n_flow_scalar_seq]),
|
||||
("P2 flush block sequence", [n_flush_block_seq]),
|
||||
("P3 folded plain scalar", [n_folded_scalar]),
|
||||
("P1+P3", [n_flow_scalar_seq, n_folded_scalar]),
|
||||
("P2+P3", [n_flush_block_seq, n_folded_scalar]),
|
||||
("P1+P2", [n_flow_scalar_seq, n_flush_block_seq]),
|
||||
("P1+P2+P3", [n_flow_scalar_seq, n_flush_block_seq, n_folded_scalar]),
|
||||
]
|
||||
|
||||
def main():
|
||||
paths = docs()
|
||||
print("denominator: %d documents (pin 3fcbb9f, parser HEAD)" % len(paths))
|
||||
for name, fns in COMBOS:
|
||||
ok = 0
|
||||
residual = collections.Counter()
|
||||
for p in paths:
|
||||
fm, body = split_fm(open(p, encoding="utf-8").read())
|
||||
for fn in fns:
|
||||
fm = fn(fm)
|
||||
err = parses(fm, body)
|
||||
if err is None:
|
||||
ok += 1
|
||||
else:
|
||||
residual[err.split(":")[0]] += 1
|
||||
top = "; ".join("%dx %s" % (c, r) for r, c in residual.most_common(3))
|
||||
print(" %-30s %2d/%d residual: %s" % (name, ok, len(paths), top or "-"))
|
||||
|
||||
main()
|
||||
|
||||
# --- diagnostic only (NOT a proposal): size the real binding constraint ---
|
||||
def n_block_mapping(fm):
|
||||
"""D4: a top-level `k:` followed by indented `a: b` lines -> one flow mapping."""
|
||||
out, i, n = [], 0, len(fm)
|
||||
while i < n:
|
||||
raw = fm[i]
|
||||
m = TOPKEY.match(raw)
|
||||
if m and m.group(2).strip() == "" and i + 1 < n and fm[i + 1][:1] in (" ", "\t") \
|
||||
and not fm[i + 1].strip().startswith("- "):
|
||||
pairs = []
|
||||
i += 1
|
||||
while i < n and fm[i][:1] in (" ", "\t") and not fm[i].strip().startswith("- "):
|
||||
pairs.append(fm[i].strip())
|
||||
i += 1
|
||||
out.append("%s: { %s }" % (m.group(1), ", ".join(pairs)))
|
||||
continue
|
||||
out.append(raw)
|
||||
i += 1
|
||||
return out
|
||||
|
||||
print("\n--- diagnostic: what caps the corpus ABOVE the tags/description gap ---")
|
||||
for name, fns in [
|
||||
("D4 top-level block mapping", [n_block_mapping]),
|
||||
("P1+P2+P3+D4", [n_flow_scalar_seq, n_flush_block_seq, n_folded_scalar, n_block_mapping]),
|
||||
]:
|
||||
ok = 0
|
||||
residual = collections.Counter()
|
||||
for p in docs():
|
||||
fm, body = split_fm(open(p, encoding="utf-8").read())
|
||||
for fn in fns:
|
||||
fm = fn(fm)
|
||||
err = parses(fm, body)
|
||||
if err is None:
|
||||
ok += 1
|
||||
else:
|
||||
residual[err.split(":")[0]] += 1
|
||||
top = "; ".join("%dx %s" % (c, r) for r, c in residual.most_common(3))
|
||||
print(" %-30s %2d/53 residual: %s" % (name, ok, top or "-"))
|
||||
45
scratchpad/measure.py
Normal file
45
scratchpad/measure.py
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
"""Measure the pinned OKF corpus against the CURRENT parser. No src/ changes."""
|
||||
import os, sys, collections
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "src"))
|
||||
from llm_ingestion_guard import okf
|
||||
|
||||
ROOT = os.path.join(os.path.dirname(__file__), "corpus", "okf", "bundles")
|
||||
|
||||
def docs():
|
||||
out = []
|
||||
for dirpath, _dirnames, filenames in os.walk(ROOT):
|
||||
for fn in sorted(filenames):
|
||||
if not fn.endswith(".md"):
|
||||
continue
|
||||
if fn in ("index.md", "log.md"):
|
||||
continue
|
||||
out.append(os.path.join(dirpath, fn))
|
||||
return sorted(out)
|
||||
|
||||
def main():
|
||||
paths = docs()
|
||||
ok, fail = [], []
|
||||
reasons = collections.Counter()
|
||||
for p in paths:
|
||||
with open(p, encoding="utf-8") as fh:
|
||||
text = fh.read()
|
||||
try:
|
||||
okf.parse_frontmatter(text)
|
||||
ok.append(p)
|
||||
except okf.OKFFrontmatterError as exc:
|
||||
fail.append((p, str(exc)))
|
||||
reasons[str(exc).split(":")[0]] += 1
|
||||
print("denominator: %d documents" % len(paths))
|
||||
print("parse OK : %d" % len(ok))
|
||||
print("parse FAIL : %d" % len(fail))
|
||||
print("--- failure classes ---")
|
||||
for r, c in reasons.most_common():
|
||||
print("%4d %s" % (c, r))
|
||||
print("--- passing documents (positive control) ---")
|
||||
for p in ok:
|
||||
print(" " + os.path.relpath(p, ROOT))
|
||||
print("--- first 8 failures verbatim ---")
|
||||
for p, e in fail[:8]:
|
||||
print(" %s\n %s" % (os.path.relpath(p, ROOT), e))
|
||||
|
||||
main()
|
||||
59
scratchpad/shapes.py
Normal file
59
scratchpad/shapes.py
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
"""Shape census of the pinned OKF corpus: which surface form each key is written in."""
|
||||
import os, re, sys, collections
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "src"))
|
||||
from llm_ingestion_guard import okf
|
||||
|
||||
ROOT = os.path.join(os.path.dirname(__file__), "corpus", "okf", "bundles")
|
||||
|
||||
def docs():
|
||||
out = []
|
||||
for dirpath, _d, fns in os.walk(ROOT):
|
||||
for fn in sorted(fns):
|
||||
if fn.endswith(".md") and fn not in ("index.md", "log.md"):
|
||||
out.append(os.path.join(dirpath, fn))
|
||||
return sorted(out)
|
||||
|
||||
def split_fm(text):
|
||||
lines = text.split("\n")
|
||||
if not lines or lines[0].strip() != "---":
|
||||
return None, None
|
||||
for i in range(1, len(lines)):
|
||||
if lines[i].strip() == "---":
|
||||
return lines[1:i], lines[i + 1:]
|
||||
return None, None
|
||||
|
||||
def key_shape(fm, key):
|
||||
for i, raw in enumerate(fm):
|
||||
m = re.match(r"^(%s):(.*)$" % re.escape(key), raw)
|
||||
if not m:
|
||||
continue
|
||||
val = m.group(2).strip()
|
||||
follow = fm[i + 1] if i + 1 < len(fm) else ""
|
||||
if val.startswith("["):
|
||||
return "flow-sequence"
|
||||
if val.startswith("{"):
|
||||
return "flow-mapping"
|
||||
if val:
|
||||
if follow[:1] in (" ", "\t") and not follow.strip().startswith("- "):
|
||||
return "scalar+continuation"
|
||||
return "single-line-scalar"
|
||||
if follow.strip().startswith("- "):
|
||||
return "block-seq-indented" if follow[:1] in (" ", "\t") else "block-seq-flush"
|
||||
if follow[:1] in (" ", "\t"):
|
||||
return "block-mapping"
|
||||
return "empty"
|
||||
return "ABSENT"
|
||||
|
||||
def main():
|
||||
paths = docs()
|
||||
print("denominator: %d" % len(paths))
|
||||
for key in ("tags", "description", "generated", "verified", "sources"):
|
||||
c = collections.Counter()
|
||||
for p in paths:
|
||||
fm, _ = split_fm(open(p, encoding="utf-8").read())
|
||||
c[key_shape(fm, key)] += 1
|
||||
print("\n%s:" % key)
|
||||
for shape, n in c.most_common():
|
||||
print(" %-22s %d/%d" % (shape, n, len(paths)))
|
||||
|
||||
main()
|
||||
|
|
@ -63,7 +63,7 @@ from .grounding import (
|
|||
)
|
||||
from . import okf
|
||||
|
||||
__version__ = "1.0.0"
|
||||
__version__ = "1.4.0"
|
||||
|
||||
|
||||
# --- §6 bookends: the two library-side halves around the transform ---------
|
||||
|
|
|
|||
|
|
@ -542,8 +542,17 @@ def _build_cases() -> list[Case]:
|
|||
lambda: okf.parse_frontmatter("---\nkey:\n nested: x\n---\nbody\n"), owasp="LLM10"),
|
||||
_raise_case("okf", "T2 frontmatter block scalar", "OKFFrontmatterError",
|
||||
lambda: okf.parse_frontmatter("---\ndesc: |\n block\n---\nbody\n"), owasp="LLM10"),
|
||||
_raise_case("okf", "T2 frontmatter flow collection", "OKFFrontmatterError",
|
||||
lambda: okf.parse_frontmatter("---\ntags: [a, b]\n---\nbody\n"), owasp="LLM10"),
|
||||
# The plain-scalar flow sequence is ADMITTED as of 1.4.0 (P1, SPEC §4.1's
|
||||
# own skeleton for `tags`); the class this row measures is the element
|
||||
# shape the parser would have to interpret rather than read -- here a
|
||||
# quoted one, which it would have to strip a quote from to return.
|
||||
_raise_case("okf", "T2 frontmatter flow sequence, quoted element",
|
||||
"OKFFrontmatterError",
|
||||
lambda: okf.parse_frontmatter("---\ntags: ['a', 'b']\n---\nbody\n"),
|
||||
owasp="LLM10"),
|
||||
_raise_case("okf", "T2 mapping key off the allowlist", "OKFFrontmatterError",
|
||||
lambda: okf.parse_frontmatter(
|
||||
"---\ngenerated: { by: a, tool: shell }\n---\nbody\n"), owasp="LLM10"),
|
||||
_raise_case("okf", "T3 resource non-https (http)", "OKFResourceError",
|
||||
lambda: okf.validate_resource_url("http://insecure.test/x"), owasp="LLM05"),
|
||||
_raise_case("okf", "T3 resource data: scheme", "OKFResourceError",
|
||||
|
|
|
|||
|
|
@ -7,7 +7,8 @@ and feeds scannable text regions into the existing ``sanitize`` / ``scan_output`
|
|||
|
||||
T2 — frontmatter parse-safety gate. ``parse_frontmatter`` is a *strict,
|
||||
reject-by-default* loader for the minimal OKF frontmatter subset: flat
|
||||
``key: value`` scalars plus block ``- item`` lists. Every construct the
|
||||
``key: value`` scalars, block ``- item`` lists, and one typed, allowlisted
|
||||
mapping form (``{ by: x, at: y }`` — see :func:`_parse_flow_mapping`). Every construct the
|
||||
"block anchor/alias DoS + dangerous type coercion" requirement names is refused
|
||||
*by construction* — you cannot suffer a billion-laughs alias expansion or a
|
||||
``!!python/object`` coercion if anchors, aliases and explicit tags are rejected
|
||||
|
|
@ -16,7 +17,9 @@ philosophy, the frontmatter analogue of the ``resource`` reject-gate (T3).
|
|||
|
||||
Deliberately NOT a general YAML parser. A security tool whose thesis is
|
||||
minimal-dependency should not pull in a full YAML engine whose own features
|
||||
(anchors, tags, merges) are the attack surface being defended against. Quoted
|
||||
(anchors, tags, merges) are the attack surface being defended against. The one
|
||||
mapping form it does admit is admitted key-by-key against an allowlist, not
|
||||
parsed generally: the mapping class is expressible, never trusted. Quoted
|
||||
scalars are kept verbatim (quotes included) rather than unquoted — the value is
|
||||
still scanned as text downstream, so an injection inside a quoted value is not
|
||||
lost; richer scalar forms are a future refinement, not a silent parse.
|
||||
|
|
@ -62,8 +65,56 @@ _KEY_RE = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9_-]*$")
|
|||
# A plain OKF scalar cannot *begin* with a YAML structural indicator. Any value
|
||||
# starting with one signals an anchor (&), alias (*), explicit tag (!), block
|
||||
# scalar (|, >), flow collection ([ ] { }), directive (%) or reserved char
|
||||
# (@ `) — all outside the supported subset and all rejected.
|
||||
# (@ `) — all outside the supported subset and all rejected. `{` and `[` are
|
||||
# tried as the allowlisted mapping form (G3) and the flow sequence of them (G30)
|
||||
# FIRST; they reach this predicate only as a leaf inside one, where a nested
|
||||
# collection is refused before it can be read.
|
||||
_DANGEROUS_VALUE_STARTS = frozenset("&*!|>[]{}%@`")
|
||||
# A quoted scalar is a scalar in YAML however many colons it carries, so the
|
||||
# mapping check steps aside for one. The quotes are retained rather than
|
||||
# stripped — a pre-existing divergence, pinned in tests/test_okf.py.
|
||||
_QUOTE_STARTS = frozenset("\"'")
|
||||
|
||||
# P1 - what disqualifies a flow-sequence element from being a plain scalar
|
||||
# (operator decision, 2026-09-08). Each character is one this parser would have
|
||||
# to interpret rather than read: the two quotes (retained, never stripped), the
|
||||
# two splitters, the two collection openers and their closers, and the comment
|
||||
# indicator. Refusing them is what lets the element be split on commas at the
|
||||
# character level without a YAML quote state machine.
|
||||
_FLOW_SCALAR_REFUSED = "{}[]:,\"'#"
|
||||
|
||||
# G3 - the one mapping form T2 can express (operator decision, 2026-08-21).
|
||||
# Every key inside a mapping must be on this allowlist: the form is safe because
|
||||
# the allowlist inspects each key, not because mappings became trusted. The keys
|
||||
# are the ones OKF v0.2 names inside a mapping - `by`/`at` (SPEC.md @ 62432a09
|
||||
# §5.2 `generated`/`verified`) and `from`/`to` (§5.1 `usage_window`), plus the
|
||||
# §5.1 `sources`-entry labels.
|
||||
_MAPPING_KEY_ALLOWLIST = frozenset({
|
||||
"by", "at", "from", "to", "id", "title", "author", "usage_count",
|
||||
"last_modified",
|
||||
})
|
||||
|
||||
# G30 - the two §5.1 keys admitted inside a `sources` entry and NOWHERE else
|
||||
# (operator decision, 2026-09-02). `resource` is REQUIRED within a `sources`
|
||||
# entry, so leaving it off left the whole provenance family unwritable; but the
|
||||
# same field name in §10 (`executor.resource`, `attester.resource`) names run
|
||||
# instructions and code - the door-C route closed in 1.1.0. 1.2.0 argued the
|
||||
# parser could not tell the two apart without parent-key context it did not
|
||||
# have. That premise was false: the owning key is in scope at every call site
|
||||
# below, it was simply never threaded through. It is threaded now, so the
|
||||
# discrimination is structural rather than a judgement about the value.
|
||||
# `usage_window` is allowlisted here for accuracy of refusal - §5.1 permits it
|
||||
# per entry, and it is then refused on the depth rule (a mapping inside a
|
||||
# mapping, which this parser admits at no key) rather than refused as if the
|
||||
# key were unknown.
|
||||
_SOURCES_ENTRY_KEYS = frozenset({"resource", "usage_window"})
|
||||
|
||||
|
||||
def _allowed_mapping_keys(parent_key):
|
||||
"""The mapping-key allowlist for a mapping owned by ``parent_key``."""
|
||||
if parent_key == "sources":
|
||||
return _MAPPING_KEY_ALLOWLIST | _SOURCES_ENTRY_KEYS
|
||||
return _MAPPING_KEY_ALLOWLIST
|
||||
|
||||
|
||||
class OKFError(Exception):
|
||||
|
|
@ -102,7 +153,10 @@ def parse_frontmatter(document):
|
|||
|
||||
Raises ``OKFFrontmatterError`` on an unterminated fence or any construct
|
||||
outside the minimal flat subset (anchors, aliases, explicit tags, merge
|
||||
keys, block scalars, flow collections, nested mappings).
|
||||
keys, block scalars, flow sequences, nested mappings). The single exception
|
||||
is the typed, allowlisted flow mapping (:func:`_parse_flow_mapping`), which
|
||||
parses into a ``dict`` of allowlisted keys with plain-scalar leaves — every
|
||||
other route to a mapping still raises.
|
||||
"""
|
||||
lines = document.split("\n")
|
||||
if not lines or lines[0].strip() != _FENCE:
|
||||
|
|
@ -145,13 +199,31 @@ def _scannable_regions(frontmatter, body):
|
|||
"""The text regions of a concept that carry attacker-controlled content."""
|
||||
regions = [body]
|
||||
for value in frontmatter.values():
|
||||
if isinstance(value, list):
|
||||
regions.extend(value)
|
||||
elif value:
|
||||
regions.append(value)
|
||||
regions.extend(_value_regions(value))
|
||||
return regions
|
||||
|
||||
|
||||
def _value_regions(value):
|
||||
"""Every scannable leaf of one frontmatter value.
|
||||
|
||||
A mapping value (G3) is a new *shape* on this surface, not a new exemption:
|
||||
its leaves are scanned exactly like a scalar or a list item, so an injection
|
||||
parked in ``generated: { by: ... }`` reaches ``scan_output`` like any other
|
||||
frontmatter text. The same holds for a *list* of mappings (G30, ``sources``),
|
||||
which this function already flattens through its list branch. Mapping *keys*
|
||||
are not scanned because they cannot carry attacker text - the allowlist
|
||||
admits a fixed, per-parent name set and nothing else.
|
||||
"""
|
||||
if isinstance(value, dict):
|
||||
return [leaf for leaf in value.values() if leaf]
|
||||
if isinstance(value, list):
|
||||
regions = []
|
||||
for item in value:
|
||||
regions.extend(_value_regions(item))
|
||||
return regions
|
||||
return [value] if value else []
|
||||
|
||||
|
||||
def validate_concept_path(path, *, allow_reserved=False):
|
||||
"""Validate a bundle-relative concept path and return its concept-ID.
|
||||
|
||||
|
|
@ -571,25 +643,50 @@ def _parse_flat(fm_lines):
|
|||
raise OKFFrontmatterError("invalid frontmatter key: %r" % key)
|
||||
|
||||
if value == "":
|
||||
items, i = _consume_block_list(fm_lines, i + 1)
|
||||
items, i = _consume_block_list(fm_lines, i + 1, key)
|
||||
result[key] = items if items is not None else ""
|
||||
continue
|
||||
|
||||
mapping = _parse_flow_mapping(value, key)
|
||||
if mapping is not None:
|
||||
result[key] = mapping
|
||||
i += 1
|
||||
continue
|
||||
|
||||
sequence = _parse_flow_sequence(value, key)
|
||||
if sequence is not None:
|
||||
result[key] = sequence
|
||||
i += 1
|
||||
continue
|
||||
|
||||
_reject_dangerous_value(value)
|
||||
_reject_mapping_construct(value)
|
||||
result[key] = value
|
||||
i += 1
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _consume_block_list(fm_lines, start):
|
||||
def _consume_block_list(fm_lines, start, parent_key=None):
|
||||
"""Consume `` - item`` lines following a bare ``key:``.
|
||||
|
||||
Returns ``(items, next_index)`` — ``items`` is ``None`` (and ``next_index``
|
||||
Returns ``(items, next_index)`` - ``items`` is ``None`` (and ``next_index``
|
||||
unchanged) when no list item follows, so the caller can treat the key as an
|
||||
empty scalar and let the next line trip the nested-structure guard.
|
||||
|
||||
An item is one of three shapes, decided by the item text alone: a flow
|
||||
mapping (G3), a block mapping (G30 - an unquoted ``key: value`` opening a
|
||||
run of more-indented sibling entries, which is SPEC.md §5.1's own carrier
|
||||
for ``sources``), or a plain scalar. ``parent_key`` is the key that owns the
|
||||
list; it decides the mapping-key allowlist, which is how ``sources[].resource``
|
||||
is admitted while ``executor``/``attester`` ``resource`` stays refused.
|
||||
|
||||
A list may not mix scalars and mappings. YAML permits it, but a consumer
|
||||
iterating ``sources`` and reading ``entry.get("id")`` gets an
|
||||
``AttributeError`` off the first ``str`` - refusing is the cheaper failure.
|
||||
"""
|
||||
items = []
|
||||
kinds = set()
|
||||
i = start
|
||||
n = len(fm_lines)
|
||||
while i < n:
|
||||
|
|
@ -598,21 +695,343 @@ def _consume_block_list(fm_lines, start):
|
|||
if stripped == "" or stripped.startswith("#"):
|
||||
i += 1
|
||||
continue
|
||||
if raw[:1] in (" ", "\t") and stripped.startswith("- "):
|
||||
item = stripped[2:].strip()
|
||||
_reject_dangerous_value(item)
|
||||
items.append(item)
|
||||
if not (raw[:1] in (" ", "\t") and stripped.startswith("- ")):
|
||||
break
|
||||
|
||||
item = stripped[2:].strip()
|
||||
|
||||
mapping = _parse_flow_mapping(item, parent_key)
|
||||
if mapping is not None:
|
||||
items.append(mapping)
|
||||
kinds.add("mapping")
|
||||
i += 1
|
||||
continue
|
||||
break
|
||||
|
||||
entry = _block_mapping_entry(item)
|
||||
if entry is not None:
|
||||
mapping, i = _consume_block_mapping(fm_lines, i + 1, entry, parent_key)
|
||||
items.append(mapping)
|
||||
kinds.add("mapping")
|
||||
continue
|
||||
|
||||
_reject_dangerous_value(item)
|
||||
_reject_mapping_construct(item)
|
||||
items.append(item)
|
||||
kinds.add("scalar")
|
||||
i += 1
|
||||
|
||||
if len(kinds) > 1:
|
||||
raise OKFFrontmatterError(
|
||||
"a block list may not mix scalar items and mappings: %r" % (parent_key,)
|
||||
)
|
||||
if not items:
|
||||
return None, start
|
||||
return items, i
|
||||
|
||||
|
||||
def _block_mapping_entry(text):
|
||||
"""Read ``text`` as one ``key: value`` block-mapping entry, or return ``None``.
|
||||
|
||||
The trigger is deliberately the same shape ``_reject_mapping_construct``
|
||||
uses to *refuse* a scalar: an unquoted ``": "``. What changes in 1.3.0 is
|
||||
only what happens next - the entry is admitted key-by-key against the
|
||||
allowlist instead of refused wholesale. Every shape that is a scalar to
|
||||
PyYAML stays one here: a quoted item, a colon with no space
|
||||
(``domain:security``, ``https://e.com:8443/a``) and a trailing colon all
|
||||
return ``None`` and fall through to the unchanged scalar rules.
|
||||
"""
|
||||
if not text or text[0] in _QUOTE_STARTS:
|
||||
return None
|
||||
key, sep, leaf = text.partition(": ")
|
||||
if not sep:
|
||||
return None
|
||||
key = key.strip()
|
||||
if not _KEY_RE.match(key):
|
||||
return None
|
||||
return key, leaf.strip()
|
||||
|
||||
|
||||
def _consume_block_mapping(fm_lines, start, first_entry, parent_key):
|
||||
"""Consume the sibling entries of a block mapping opened by a ``- `` item.
|
||||
|
||||
Returns ``(mapping, next_index)``. A sibling is an indented line that does
|
||||
not open a new list item; the run ends at a blank line, a comment, a new
|
||||
``- `` item, or a line at column zero. Depth is capped at one by giving the
|
||||
leaves the *unchanged* scalar predicates: a nested collection opens with
|
||||
``{`` or ``[`` and is refused by ``_reject_dangerous_value``, and a further
|
||||
block level is refused by ``_reject_mapping_construct``.
|
||||
"""
|
||||
allowed = _allowed_mapping_keys(parent_key)
|
||||
mapping = {}
|
||||
_admit_mapping_entry(mapping, first_entry[0], first_entry[1], allowed, parent_key)
|
||||
|
||||
i = start
|
||||
n = len(fm_lines)
|
||||
while i < n:
|
||||
raw = fm_lines[i]
|
||||
stripped = raw.strip()
|
||||
if stripped == "" or stripped.startswith("#"):
|
||||
break
|
||||
if raw[:1] not in (" ", "\t") or stripped.startswith("- "):
|
||||
break
|
||||
entry = _block_mapping_entry(stripped)
|
||||
if entry is None:
|
||||
_reject_dangerous_value(stripped)
|
||||
_reject_mapping_construct(stripped)
|
||||
raise OKFFrontmatterError(
|
||||
"a block-mapping entry must be 'key: value': %r" % (raw,)
|
||||
)
|
||||
_admit_mapping_entry(mapping, entry[0], entry[1], allowed, parent_key)
|
||||
i += 1
|
||||
return mapping, i
|
||||
|
||||
|
||||
def _admit_mapping_entry(mapping, key, leaf, allowed, parent_key):
|
||||
"""Admit one mapping entry, or raise. The single gate both carriers pass."""
|
||||
if not _KEY_RE.match(key):
|
||||
raise OKFFrontmatterError("invalid mapping key: %r" % (key,))
|
||||
if key not in allowed:
|
||||
raise OKFFrontmatterError(
|
||||
"mapping key %r is not on the OKF mapping allowlist under %r"
|
||||
% (key, parent_key)
|
||||
)
|
||||
if key in mapping:
|
||||
raise OKFFrontmatterError("duplicate mapping key %r" % (key,))
|
||||
_reject_dangerous_value(leaf)
|
||||
_reject_mapping_construct(leaf)
|
||||
mapping[key] = leaf
|
||||
|
||||
|
||||
def _reject_dangerous_value(value):
|
||||
if value and value[0] in _DANGEROUS_VALUE_STARTS:
|
||||
raise OKFFrontmatterError(
|
||||
"value begins with a disallowed YAML indicator %r: %r"
|
||||
% (value[0], value)
|
||||
)
|
||||
|
||||
|
||||
def _reject_mapping_construct(value):
|
||||
"""Reject a scalar that YAML reads as a mapping rather than as a string.
|
||||
|
||||
T2 gives the mapping *class* exactly one expressible form, the typed
|
||||
allowlisted flow mapping (G3); the nested-block and dotted-key routes still
|
||||
raise, and this predicate is what keeps them raising — both at the top level
|
||||
and on a leaf *inside* an admitted mapping. Two routes used to escape by degrading
|
||||
into a string instead: a block-sequence item carrying exactly one key
|
||||
(``- uri: x``), and an inline second colon (``attester: resource: x``).
|
||||
Both parsed "successfully" into the wrong *type*, and a pointer parked in
|
||||
one rode through in a key the ``resource`` allowlist never inspects.
|
||||
|
||||
``": "`` and a trailing ``":"`` are exactly the two shapes where a plain
|
||||
scalar stops being one — ground-truthed against PyYAML 6.0.3, which reads
|
||||
``- uri: x`` as ``[{'uri': 'x'}]``, ``- uri:`` as ``[{'uri': None}]``, and
|
||||
refuses ``k: sub: v`` outright. A colon carrying neither a space nor a line
|
||||
end opens no mapping (``domain:security``, ``https://e.com:8443/a``) and is
|
||||
left alone, as is a quoted scalar — over-blocking a conformant bundle is
|
||||
itself a failure mode.
|
||||
"""
|
||||
if not value or value[0] in _QUOTE_STARTS:
|
||||
return
|
||||
if ": " in value or value.endswith(":"):
|
||||
raise OKFFrontmatterError(
|
||||
"a mapping is not expressible in OKF frontmatter: %r" % (value,)
|
||||
)
|
||||
|
||||
|
||||
def _parse_flow_mapping(value, parent_key=None):
|
||||
"""Parse ``{ key: value, ... }`` into a typed dict, or refuse it (G3).
|
||||
|
||||
Returns ``None`` when ``value`` does not open a flow mapping, so the caller
|
||||
falls through to the unchanged scalar rules. Otherwise the value either
|
||||
parses into a ``dict`` of allowlisted keys with plain-scalar leaves, or
|
||||
raises - it never degrades into a string, which is the defect closed in
|
||||
1.1.0 and not reopened here.
|
||||
|
||||
Why the mapping class needed *a* form at all: OKF v0.2 writes its whole
|
||||
trust and provenance layer as mappings, and SPEC.md @ ``62432a09`` uses flow
|
||||
form in its own examples (§5.1 ``usage_window``, §5.2 ``generated`` /
|
||||
``verified``). §11 goes further than "should": a consumer *MUST* treat a
|
||||
bare ``verified`` mapping as a one-element list - a rule that presupposes
|
||||
the mapping parses. With no form, 0 of 53 upstream concepts reached the
|
||||
gate, and no threshold would have changed that.
|
||||
|
||||
Why this form is safe: the allowlist inspects **every key**, which is the
|
||||
property that actually carried the security in T2 - the blanket refusal was
|
||||
the enforcement, not the point. Admitted, ground-truthed against PyYAML
|
||||
6.0.3:
|
||||
|
||||
- one flow mapping per value, closed on the same line (``{ a: b }``);
|
||||
- keys on :data:`_MAPPING_KEY_ALLOWLIST` and matching ``_KEY_RE``, no
|
||||
duplicates - PyYAML resolves a duplicate last-wins, which is a way to
|
||||
show one claim and mean another;
|
||||
- plain-scalar leaves only, each run through the *unchanged*
|
||||
``_reject_dangerous_value`` / ``_reject_mapping_construct`` predicates, so
|
||||
a leaf can no more open an anchor, a tag or a nested mapping than a
|
||||
top-level scalar can.
|
||||
|
||||
Refused, each on its own rule: nested collections (``{ a: { b: c } }``,
|
||||
``{ a: [1] }``), quoted leaves, an empty mapping, an unclosed or
|
||||
trailing-junk value (``{ a: b } x``, which PyYAML also refuses), a key
|
||||
outside the allowlist, and ``{a:b}`` - which PyYAML reads as the *key*
|
||||
``a:b``, not as a scalar, and which the required ``": "`` separator catches.
|
||||
|
||||
Two deliberate divergences from PyYAML, both toward refusal: a quoted leaf
|
||||
(``{ title: 'a, b' }``) and a trailing comment (``{ a: b } # note``) are
|
||||
conformant YAML that this rejects. Splitting quoted commas correctly needs a
|
||||
quote state machine whose failure mode is *accepting* something YAML would
|
||||
refuse; refusing is the cheaper side to be wrong on, and the keys that
|
||||
plausibly need a comma (``title``, ``author``) only occur inside ``sources``
|
||||
entries, whose block-sequence carrier is refused anyway.
|
||||
"""
|
||||
if not value or value[0] != "{":
|
||||
return None
|
||||
if not value.endswith("}"):
|
||||
raise OKFFrontmatterError(
|
||||
"a flow mapping must be closed by '}' on the same line: %r" % (value,)
|
||||
)
|
||||
|
||||
inner = value[1:-1].strip()
|
||||
if inner.endswith(","): # a trailing comma is legal YAML; one, and only one
|
||||
inner = inner[:-1].strip()
|
||||
if not inner:
|
||||
raise OKFFrontmatterError("an empty flow mapping carries nothing: %r" % (value,))
|
||||
for char in "{}[]":
|
||||
if char in inner:
|
||||
raise OKFFrontmatterError(
|
||||
"a flow mapping admits scalar leaves only, not %r: %r" % (char, value)
|
||||
)
|
||||
for quote in _QUOTE_STARTS:
|
||||
if quote in inner:
|
||||
raise OKFFrontmatterError(
|
||||
"a quoted scalar inside a flow mapping is not a supported form: %r"
|
||||
% (value,)
|
||||
)
|
||||
|
||||
allowed = _allowed_mapping_keys(parent_key)
|
||||
mapping = {}
|
||||
for entry in inner.split(","):
|
||||
entry = entry.strip()
|
||||
key, sep, leaf = entry.partition(": ")
|
||||
if not sep:
|
||||
raise OKFFrontmatterError(
|
||||
"a flow-mapping entry must be 'key: value': %r" % (entry,)
|
||||
)
|
||||
_admit_mapping_entry(mapping, key.strip(), leaf.strip(), allowed, parent_key)
|
||||
return mapping
|
||||
|
||||
|
||||
def _parse_flow_sequence(value, parent_key=None):
|
||||
"""Parse ``[{ ... }, { ... }]`` into a list of typed dicts, or refuse it (G30).
|
||||
|
||||
Returns ``None`` when ``value`` does not open a flow sequence, so the caller
|
||||
falls through to the unchanged scalar rules - where ``[`` is still a
|
||||
disallowed indicator. This carrier is opened for the flow-mapping element
|
||||
and nothing else: it is the form the OKF producers emit for ``sources``
|
||||
(measured 02.09 against llm-ingestion-okf's golden bundle, where a
|
||||
one-element sequence raised on the ``[`` just as a two-element one did).
|
||||
|
||||
As of 1.4.0 it also carries a sequence of plain *scalars*
|
||||
(``tags: [a, b, c]``) - SPEC.md §4.1's own skeleton for ``tags``, and the
|
||||
one candidate form measured to move the upstream corpus at all (0/53 ->
|
||||
6/53 against ``_okf-upstream`` @ 3fcbb9f, denominator 53; see
|
||||
docs/2026-09-07-limitations-44-maaling.md). The quoting and comma-splitting
|
||||
problem that kept it refused is answered by refusing the characters that
|
||||
create it rather than by parsing them: an element is a plain scalar only if
|
||||
it is non-empty and carries none of ``{ } [ ] : , " ' #``, and it then
|
||||
passes the unchanged scalar indicator rule. Everything needing YAML
|
||||
semantics to split or unquote correctly still raises.
|
||||
|
||||
A mapping element is split on ``}`` rather than on commas, which is sound
|
||||
precisely because ``_parse_flow_mapping`` admits no nested collection: a
|
||||
``}`` inside an element cannot occur, so the first ``}`` after ``{`` always
|
||||
closes it. A scalar element runs to the next comma, which cannot occur
|
||||
inside one. Anything between elements that is not a separating comma is
|
||||
refused, which is what makes trailing junk fail rather than parse.
|
||||
|
||||
A sequence may not mix the two, for the reason the block list may not: a
|
||||
consumer iterating the value and reading ``entry.get("id")`` gets an
|
||||
``AttributeError`` off the first ``str``. The mixing verdict is reached
|
||||
before the element is parsed, so the caller is told about the mix rather
|
||||
than about a key the allowlist would have complained of instead.
|
||||
"""
|
||||
if not value or value[0] != "[":
|
||||
return None
|
||||
if not value.endswith("]"):
|
||||
raise OKFFrontmatterError(
|
||||
"a flow sequence must be closed by ']' on the same line: %r" % (value,)
|
||||
)
|
||||
|
||||
inner = value[1:-1].strip()
|
||||
if not inner:
|
||||
raise OKFFrontmatterError("an empty flow sequence carries nothing: %r" % (value,))
|
||||
|
||||
items = []
|
||||
kinds = set()
|
||||
i = 0
|
||||
n = len(inner)
|
||||
while True:
|
||||
while i < n and inner[i] in " \t":
|
||||
i += 1
|
||||
if i >= n:
|
||||
break
|
||||
if inner[i] == "{":
|
||||
_refuse_mixed_flow_sequence(kinds, "mapping", value)
|
||||
close = inner.find("}", i)
|
||||
if close == -1:
|
||||
raise OKFFrontmatterError(
|
||||
"an unclosed flow mapping inside a flow sequence: %r" % (value,)
|
||||
)
|
||||
items.append(_parse_flow_mapping(inner[i:close + 1], parent_key))
|
||||
i = close + 1
|
||||
else:
|
||||
_refuse_mixed_flow_sequence(kinds, "scalar", value)
|
||||
end = i
|
||||
while end < n and inner[end] != ",":
|
||||
end += 1
|
||||
items.append(_flow_sequence_scalar(inner[i:end].strip(), value))
|
||||
i = end
|
||||
while i < n and inner[i] in " \t":
|
||||
i += 1
|
||||
if i >= n:
|
||||
break
|
||||
if inner[i] != ",":
|
||||
raise OKFFrontmatterError(
|
||||
"trailing junk after a flow-sequence element: %r" % (value,)
|
||||
)
|
||||
i += 1
|
||||
return items
|
||||
|
||||
|
||||
def _refuse_mixed_flow_sequence(kinds, kind, value):
|
||||
kinds.add(kind)
|
||||
if len(kinds) > 1:
|
||||
raise OKFFrontmatterError(
|
||||
"a flow sequence may not mix scalar items and mappings: %r" % (value,)
|
||||
)
|
||||
|
||||
|
||||
def _flow_sequence_scalar(element, value):
|
||||
"""Read one flow-sequence element as a plain scalar, or refuse it (P1).
|
||||
|
||||
Character-level, with no YAML semantics: the element must be non-empty and
|
||||
carry none of :data:`_FLOW_SCALAR_REFUSED`. That set is not a style rule -
|
||||
each member is a character whose meaning this parser would have to guess at.
|
||||
A quote would have to be stripped (this parser retains quotes, so it would
|
||||
hand back a different value than YAML reads); a comma or a colon would have
|
||||
to be split on; a bracket or a brace would open a second collection level,
|
||||
which no carrier here admits; a ``#`` opens a comment. The unchanged
|
||||
indicator rule then applies to what is left, exactly as it does to a
|
||||
block-list item, so an anchor or an alias is no more a scalar here.
|
||||
"""
|
||||
if not element:
|
||||
raise OKFFrontmatterError(
|
||||
"an empty element in a flow sequence carries nothing: %r" % (value,)
|
||||
)
|
||||
for char in _FLOW_SCALAR_REFUSED:
|
||||
if char in element:
|
||||
raise OKFFrontmatterError(
|
||||
"a flow-sequence scalar admits plain scalars only, not %r: %r"
|
||||
% (char, value)
|
||||
)
|
||||
_reject_dangerous_value(element)
|
||||
return element
|
||||
|
|
|
|||
|
|
@ -28,10 +28,12 @@ from pathlib import Path
|
|||
|
||||
import pytest
|
||||
|
||||
import redos_clock
|
||||
from llm_ingestion_guard import Disposition, PRESET_USER_UPLOAD, Risk, screen_output
|
||||
from llm_ingestion_guard import active_content as ac
|
||||
|
||||
_DOCS = Path(__file__).resolve().parent.parent / "docs"
|
||||
_LIMITATIONS = _DOCS / "LIMITATIONS.md"
|
||||
|
||||
|
||||
def _load(filename: str):
|
||||
|
|
@ -41,6 +43,7 @@ def _load(filename: str):
|
|||
spec = importlib.util.spec_from_file_location(name, path)
|
||||
assert spec and spec.loader, f"cannot load {path}"
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[name] = module # @dataclass resolves its own module via sys.modules
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
|
@ -49,6 +52,7 @@ def _load(filename: str):
|
|||
# script's import list fails the whole file loudly instead of one quiet test.
|
||||
fp_sweep = _load("fp-sweep.py")
|
||||
census = _load("rawhtml-census.py")
|
||||
redos_sweep = _load("redos-sweep.py")
|
||||
|
||||
|
||||
# --- docs/fp-sweep.py --------------------------------------------------------
|
||||
|
|
@ -245,6 +249,38 @@ def test_census_production_row_equals_its_shipped_candidate(name, attrs):
|
|||
assert candidate(name, attrs) == ac.active_tag_class(name, attrs)
|
||||
|
||||
|
||||
# --- docs/redos-sweep.py ------------------------------------------------------
|
||||
|
||||
|
||||
def test_redos_sweep_times_on_the_suite_clock_not_a_reimplementation():
|
||||
# Until 1.1.0 this script timed on `time.monotonic()`, a different instrument
|
||||
# than every ReDoS bound in the suite. `t()` must call the shared
|
||||
# `scan_seconds` — imported, not restated — and the module must not import
|
||||
# `time` itself, else a drift back to a wall clock would go unnoticed here.
|
||||
assert redos_sweep.scan_seconds is redos_clock.scan_seconds
|
||||
assert not hasattr(redos_sweep, "time"), "module must not import time itself"
|
||||
|
||||
|
||||
def test_redos_sweep_floor_and_flag_match_the_published_numbers():
|
||||
# docs/LIMITATIONS.md publishes the 1.5 ms floor and the 2.6 flag ratio this
|
||||
# script derives from twelve full runs. Pin both sides: the constants, and
|
||||
# that the doc still states the same numbers — either drifting alone is a bug.
|
||||
assert redos_sweep.NOISE_FLOOR == 0.0015
|
||||
assert redos_sweep.RATIO_FLAG == 2.6
|
||||
text = _LIMITATIONS.read_text(encoding="utf-8")
|
||||
assert "1.5 ms noise floor" in text
|
||||
assert "2.6 flag threshold" in text
|
||||
|
||||
|
||||
def test_redos_sweep_collector_covers_152_patterns_across_11_tables():
|
||||
# The count docs/LIMITATIONS.md carries as "all 152 compiled patterns across
|
||||
# all eleven regex-bearing modules". A pattern added or removed in `src/`
|
||||
# without re-measuring would drift the doc's claim silently otherwise.
|
||||
assert len(redos_sweep.TABLES) == 11
|
||||
total = sum(len(collect()) for collect in redos_sweep.TABLES.values())
|
||||
assert total == 152
|
||||
|
||||
|
||||
# --- both scripts: the argument-less contract --------------------------------
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -116,9 +116,13 @@ def test_rejects_unterminated_frontmatter():
|
|||
|
||||
|
||||
def test_rejects_flow_collection():
|
||||
# inline flow collections are outside the supported subset -> reject, don't
|
||||
# silently mis-parse the bracket string as a scalar.
|
||||
doc = "---\ntype: table\ntags: [pii, customers]\n---\nbody\n"
|
||||
# An inline flow collection this parser cannot read without guessing ->
|
||||
# reject, don't silently mis-parse the bracket string as a scalar. The
|
||||
# QUOTED elements are what makes this row stay red: the plain-scalar form
|
||||
# (`tags: [pii, customers]`) is admitted as of 1.4.0 -- see the P1 block at
|
||||
# the foot of this file -- while a quoted element would have to be stripped
|
||||
# to be read, and this parser retains quotes.
|
||||
doc = "---\ntype: table\ntags: ['pii', 'customers']\n---\nbody\n"
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(doc)
|
||||
|
||||
|
|
@ -548,9 +552,15 @@ _V02_REJECTED = [
|
|||
("generated (nested)", "generated:\n at: 2026-07-26T10:00:00Z\n"),
|
||||
("executor (nested)", "executor:\n resource: skills/run-on-bq.md\n"),
|
||||
("attester (nested)", "attester:\n resource: attesters/sql_equality.py\n"),
|
||||
("sources (block list of mappings)",
|
||||
# The carrier is admitted as of 1.3.0 (G30); this row now measures the KEY
|
||||
# SET - `uri`/`kind` are producer-invented, not SPEC §5.1 - and stays red for
|
||||
# that reason. SPEC's own §5.1 keys parse; see the G30 block at the foot.
|
||||
("sources (block list, off-allowlist keys)",
|
||||
"sources:\n - uri: https://e.com/a\n kind: doc\n"),
|
||||
("flow sequence", "tags: [a, b, c]\n"),
|
||||
# The plain-scalar flow sequence is admitted as of 1.4.0 (P1); this row now
|
||||
# measures the QUOTED element, which stays refused for the same reason it
|
||||
# always did -- reading it would mean stripping a quote this parser retains.
|
||||
("flow sequence, quoted elements", "tags: ['a', 'b']\n"),
|
||||
("flow mapping", "executor: {resource: skills/run.md}\n"),
|
||||
]
|
||||
|
||||
|
|
@ -569,6 +579,8 @@ _V02_ADMITTED = [
|
|||
("status/stale_after", "status: active\nstale_after: 2026-12-01\n"),
|
||||
("verified bool", "verified: true\n"),
|
||||
("block sequence of scalars", "tags:\n - alpha\n - beta\n"),
|
||||
# SPEC §4.1's own skeleton for `tags`, admitted as of 1.4.0 (P1).
|
||||
("flow sequence of scalars", "tags: [alpha, beta]\n"),
|
||||
]
|
||||
|
||||
|
||||
|
|
@ -577,15 +589,61 @@ def test_v02_flat_frontmatter_still_parses(cid, fm):
|
|||
assert parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")[0]["id"] == "x"
|
||||
|
||||
|
||||
def test_one_key_block_sequence_item_is_misparsed_as_a_string():
|
||||
# The documented defect: two keys per item hard-reject (loud, safe), but ONE key
|
||||
# parses "successfully" into the wrong type. A consumer reading
|
||||
# frontmatter["sources"][0].get("uri") gets a string, not a mapping.
|
||||
fm, _ = parse_frontmatter(
|
||||
"---\nid: x\nsources:\n - uri: https://e.com/a\n---\n\nbody\n"
|
||||
)
|
||||
assert fm["sources"] == ["uri: https://e.com/a"], "shape changed — update LIMITATIONS.md"
|
||||
assert not isinstance(fm["sources"][0], dict)
|
||||
# --- the type-confusion defect, closed in 1.1.0 (2026-08-13) ----------------
|
||||
# Was: a mapping construct that the restricted grammar cannot represent degraded
|
||||
# into a STRING instead of failing. Two routes did this, not the one documented.
|
||||
# Ground-truthed against PyYAML 6.0.3: every shape below that we now reject is a
|
||||
# shape a real YAML parser reads as a MAPPING (or refuses outright), and every
|
||||
# shape we still admit is one PyYAML reads as a plain scalar.
|
||||
|
||||
_DEGRADED_TO_STRING = [
|
||||
# (id, frontmatter, what PyYAML 6.0.3 makes of it)
|
||||
# Still red after G30 opened the block-mapping carrier: `uri` is not a §5.1
|
||||
# key, so the item is refused by the allowlist instead of by the grammar.
|
||||
("one key per item", "sources:\n - uri: https://e.com/a\n", "[{'uri': ...}]"),
|
||||
("item, trailing colon", "sources:\n - uri:\n", "[{'uri': None}]"),
|
||||
("inline double colon", "attester: resource: attesters/sql_equality.py\n", "parse error"),
|
||||
("top value, trailing colon", "description: see below:\n", "parse error"),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm,yaml_reads_as", _DEGRADED_TO_STRING,
|
||||
ids=[c[0] for c in _DEGRADED_TO_STRING])
|
||||
def test_a_mapping_construct_never_degrades_into_a_string(cid, fm, yaml_reads_as):
|
||||
# None of these shapes is the one form T2 admits (G3, the allowlisted flow
|
||||
# mapping) — so each must RAISE, never parse "successfully" into the wrong
|
||||
# type. A consumer reading frontmatter["sources"][0].get("uri") must not be
|
||||
# handed a str, and that holds whether the mapping class has no expressible
|
||||
# form or one.
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")
|
||||
|
||||
|
||||
_STILL_SCALARS = [
|
||||
# PyYAML reads every one of these as a plain scalar: the colon carries no
|
||||
# space and no line end, so it never opens a mapping. Over-blocking a
|
||||
# conformant bundle is itself a failure mode (brief principle 5).
|
||||
("colon, no space", "tags:\n - domain:security\n", "tags", ["domain:security"]),
|
||||
("url item", "sources:\n - https://e.com/a\n", "sources", ["https://e.com/a"]),
|
||||
("url item with port", "sources:\n - https://e.com:8443/a\n", "sources",
|
||||
["https://e.com:8443/a"]),
|
||||
("url value with port", "resource: https://e.com:8443/a\n", "resource",
|
||||
"https://e.com:8443/a"),
|
||||
("double-quoted item", 'sources:\n - "uri: https://e.com/a"\n', "sources",
|
||||
['"uri: https://e.com/a"']),
|
||||
("single-quoted item", "sources:\n - 'uri: https://e.com/a'\n", "sources",
|
||||
["'uri: https://e.com/a'"]),
|
||||
("quoted top value", 'description: "Note: careful"\n', "description",
|
||||
'"Note: careful"'),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm,key,expected", _STILL_SCALARS,
|
||||
ids=[c[0] for c in _STILL_SCALARS])
|
||||
def test_scalars_that_merely_contain_a_colon_still_parse(cid, fm, key, expected):
|
||||
# Quotes are retained rather than stripped — a pre-existing divergence from
|
||||
# YAML, pinned here so closing the mapping hole is not read as fixing it.
|
||||
assert parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")[0][key] == expected
|
||||
|
||||
|
||||
def test_relative_resource_pointer_fails_the_allowlist():
|
||||
|
|
@ -595,42 +653,54 @@ def test_relative_resource_pointer_fails_the_allowlist():
|
|||
validate_resource_url(pointer)
|
||||
|
||||
|
||||
def test_pointer_in_one_key_sequence_reaches_the_consumer_tree():
|
||||
# The security-relevant consequence of the misparse above: the pointer never
|
||||
# touches the top-level `resource` key, so the https allowlist never inspects it
|
||||
# and door C admits the concept. Not conformant OKF — a well-formed bundle will
|
||||
# not produce this shape — but mode-b writes the merged concept verbatim.
|
||||
doc = ("---\nid: x\ntype: Attested Computation\n"
|
||||
"attester:\n - resource: attesters/sql_equality.py\n---\n\nbody\n")
|
||||
@pytest.mark.parametrize("cid,carrier", [
|
||||
("block sequence", "attester:\n - resource: attesters/sql_equality.py\n"),
|
||||
("inline double colon", "attester: resource: attesters/sql_equality.py\n"),
|
||||
])
|
||||
def test_pointer_in_a_degraded_mapping_no_longer_reaches_the_consumer_tree(cid, carrier):
|
||||
# The security-relevant consequence, closed at door C. Both carriers put the
|
||||
# pointer in a key the https allowlist never inspects, so while the shape
|
||||
# parsed, mode-b wrote the merged concept verbatim. It now fails secure at T2,
|
||||
# before the allowlist is even reached.
|
||||
doc = f"---\nid: x\ntype: Attested Computation\n{carrier}---\n\nbody\n"
|
||||
result = import_bundle({"computations/x.md": doc})
|
||||
assert result.disposition is Disposition.WARN, "hole closed — update LIMITATIONS.md"
|
||||
assert result.disposition is Disposition.FAIL_SECURE, "hole reopened — see LIMITATIONS.md"
|
||||
|
||||
|
||||
def test_every_route_to_a_mapping_fails_on_a_different_rule():
|
||||
# The v0.2 wall is not a choice between two forms where one is better: ALL three
|
||||
# ways to express a mapping fail, each on its own rule, so the mapping *class* has
|
||||
# no expressible form through T2. v0.2's `generated` IS a mapping (`by` required
|
||||
# when present), so it cannot be expressed at all.
|
||||
def test_the_expressible_mapping_routes_are_the_ones_the_allowlist_inspects():
|
||||
# Was `test_exactly_one_route_to_a_mapping_is_expressible` (1.2.0), and before
|
||||
# that ALL FOUR routes failed so the mapping *class* had no expressible form.
|
||||
# There are four expressible carriers as of 1.3.0 - flow mapping as a value,
|
||||
# flow mapping as a list item, flow sequence of flow mappings, block sequence
|
||||
# of block mappings - and the criterion that admits them is unchanged: each
|
||||
# hands the allowlist every key. The routes below stay shut for the same
|
||||
# reason, each on its own rule: a top-level block MAPPING (not a sequence),
|
||||
# a dotted key, and an inline second colon give the allowlist nothing to
|
||||
# inspect.
|
||||
assert parse_frontmatter("---\nid: x\ngenerated: { by: x, at: y }\n---\n\nbody\n")[0][
|
||||
"generated"] == {"by": "x", "at": "y"}
|
||||
|
||||
routes = {
|
||||
"flow": "generated: { by: x, at: y }\n",
|
||||
"block": "generated:\n by: x\n",
|
||||
"dotted": "generated.by: x\n",
|
||||
"inline": "generated: by: x\n",
|
||||
}
|
||||
errors = {}
|
||||
for name, fm in routes.items():
|
||||
with pytest.raises(OKFFrontmatterError) as exc:
|
||||
parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")
|
||||
errors[name] = str(exc.value)
|
||||
assert "indicator" in errors["flow"]
|
||||
assert "nested mappings" in errors["block"]
|
||||
assert "key" in errors["dotted"]
|
||||
assert "mapping" in errors["inline"]
|
||||
assert len(set(errors.values())) == 3, "routes must fail distinctly, not collapse"
|
||||
|
||||
|
||||
_BLOCK_LIST_ITEM_SHAPES = [
|
||||
# A consumer called all three "the sources block list"; the parser does not.
|
||||
# The one-key-per-item row lived here until 1.1.0, admitted as the string
|
||||
# "id: a"; it now hard-rejects with the two-key row (_DEGRADED_TO_STRING).
|
||||
("flat scalars", "sources:\n - file://x\n - file://y\n", ["file://x", "file://y"]),
|
||||
("one key per item", "sources:\n - id: a\n", ["id: a"]), # silent misparse
|
||||
("single-element", "verified:\n - human:ktg\n", ["human:ktg"]),
|
||||
]
|
||||
|
||||
|
|
@ -642,17 +712,35 @@ def test_block_lists_admitted_by_item_shape(cid, fm, expected):
|
|||
assert parse_frontmatter(f"---\nid: x\n{fm}---\n\nbody\n")[0][key] == expected
|
||||
|
||||
|
||||
def test_two_keys_per_item_is_where_the_block_list_hard_rejects():
|
||||
def test_the_block_list_rejects_on_the_key_set_not_on_arity():
|
||||
# Was `test_two_keys_per_item_is_where_the_block_list_hard_rejects`: a
|
||||
# two-key item was refused because the block mapping had no expressible form
|
||||
# at all. G30 gives it one (SPEC.md §5.1's own carrier), so arity is no
|
||||
# longer the boundary - the key set is. The same two-key item parses when its
|
||||
# keys are §5.1's, and still raises when one of them is not.
|
||||
parsed = parse_frontmatter(
|
||||
"---\nid: x\nsources:\n - id: a\n resource: file://x\n---\n\nbody\n"
|
||||
)[0]
|
||||
assert parsed["sources"] == [{"id": "a", "resource": "file://x"}]
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(
|
||||
"---\nid: x\nsources:\n - id: a\n resource: file://x\n---\n\nbody\n"
|
||||
"---\nid: x\nsources:\n - id: a\n uri: file://x\n---\n\nbody\n"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("fm", [
|
||||
"generated: { by: x, at: y }\n", "sources: [{ id: a }]\n", "tags: [a, b]\n",
|
||||
# The flow row carries a key OFF the G3 allowlist: the shape is admitted, the
|
||||
# key is not, so this stays a T2 rejection and the door A/B half still holds.
|
||||
# The two `sources` rows that lived here until 1.3.0 now PARSE - that is the
|
||||
# G30 fix, not a weakening of this property. Their replacements are the same
|
||||
# carriers with a key off the allowlist, so the row still measures what it
|
||||
# says: the shape is admitted, the key set is not.
|
||||
# The `tags` row moved the same way in 1.4.0: `[a, b]` now PARSES (P1), so
|
||||
# the row carries the quoted form, which is still a T2 rejection.
|
||||
"generated: { by: x, tool: y }\n", "sources: [{ id: a, uri: u }]\n", "tags: ['a', 'b']\n",
|
||||
"generated:\n by: x\n", "generated.by: x\n",
|
||||
"sources:\n - id: a\n resource: file://x\n",
|
||||
"sources:\n - id: a\n uri: file://x\n",
|
||||
"executor: [{ resource: skills/run.md }]\n",
|
||||
])
|
||||
def test_t2_constrains_import_not_emission(fm):
|
||||
# T2 runs on door C only. The same frontmatter that FAIL_SECUREs through
|
||||
|
|
@ -690,3 +778,489 @@ def test_link_extraction_survives_the_redos_fix():
|
|||
assert extract_link_targets("see [x](./a.md) and [y](/b.md)") == ["./a.md", "/b.md"]
|
||||
assert extract_link_targets("[a b](./c.md)") == ["./c.md"]
|
||||
assert extract_link_targets("text [](./t.md)") == ["./i.png"]
|
||||
|
||||
|
||||
# --- G3: the typed, allowlisted mapping form (2026-08-21) --------------------
|
||||
# Door 1 of three (operator decision, 2026-08-21). The mapping *class* had no
|
||||
# expressible form, and OKF v0.2 writes its whole trust and provenance layer as
|
||||
# mappings — SPEC.md @ 62432a09 §5.2 uses flow form in its own examples, and §11
|
||||
# carries a hard MUST that presupposes they parse ("consumers MUST treat a bare
|
||||
# `verified` mapping as a one-element list"). A consumer measured 0 of 53
|
||||
# upstream concepts through the gate. This admits ONE shape: a flow mapping whose
|
||||
# every key is on the allowlist and whose every leaf is a plain scalar.
|
||||
|
||||
def test_spec_flow_mapping_parses_into_a_typed_mapping():
|
||||
# SPEC.md §5.2, verbatim. This is the red test: it must fail before the form
|
||||
# exists and pass after, with a real dict — never a degraded string.
|
||||
doc = (
|
||||
"---\ntype: table\n"
|
||||
"generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z }\n"
|
||||
"---\nbody\n"
|
||||
)
|
||||
assert parse_frontmatter(doc)[0]["generated"] == {
|
||||
"by": "reference_agent/gemini-2.5-pro",
|
||||
"at": "2026-06-20T22:53:05Z",
|
||||
}
|
||||
|
||||
|
||||
def test_spec_bare_verified_mapping_parses():
|
||||
# SPEC.md §5.2's bare form, which §11 turns into a hard MUST for consumers
|
||||
# ("MUST treat a bare `verified` mapping as a one-element list") - a rule that
|
||||
# cannot be obeyed by a consumer that cannot parse the mapping.
|
||||
doc = "---\ntype: table\nverified: { by: human:ahormati, at: 2026-06-25T09:00:00Z }\n---\nb\n"
|
||||
assert parse_frontmatter(doc)[0]["verified"] == {
|
||||
"by": "human:ahormati", "at": "2026-06-25T09:00:00Z"}
|
||||
|
||||
|
||||
def test_spec_verified_list_of_flow_mappings_parses():
|
||||
# §5.2's list form. This is the SAME typed form in list position, not the
|
||||
# block-sequence-with-one-key route (`- uri: x`), which stays shut below.
|
||||
doc = (
|
||||
"---\ntype: table\nverified:\n"
|
||||
" - { by: human:ahormati, at: 2026-06-25T09:00:00Z }\n"
|
||||
" - { by: process:finance-nightly, at: 2026-06-26T02:00:00Z }\n"
|
||||
"---\nbody\n"
|
||||
)
|
||||
assert parse_frontmatter(doc)[0]["verified"] == [
|
||||
{"by": "human:ahormati", "at": "2026-06-25T09:00:00Z"},
|
||||
{"by": "process:finance-nightly", "at": "2026-06-26T02:00:00Z"},
|
||||
]
|
||||
|
||||
|
||||
def test_spec_usage_window_parses():
|
||||
doc = "---\ntype: table\nusage_window: { from: 2026-06-01T00:00:00Z, to: 2026-06-30T00:00:00Z }\n---\nb\n"
|
||||
assert parse_frontmatter(doc)[0]["usage_window"] == {
|
||||
"from": "2026-06-01T00:00:00Z", "to": "2026-06-30T00:00:00Z"}
|
||||
|
||||
|
||||
def test_an_unknown_key_inside_a_mapping_is_still_rejected():
|
||||
# The rejection side of the allowlist. Without this test the allowlist could
|
||||
# silently grow to "anything" - or be emptied - and nothing would fail.
|
||||
with pytest.raises(OKFFrontmatterError) as exc:
|
||||
parse_frontmatter("---\nid: x\ngenerated: { by: a, tool: shell }\n---\n\nbody\n")
|
||||
assert "allowlist" in str(exc.value)
|
||||
|
||||
|
||||
def test_the_allowlist_is_not_empty_and_admits_only_the_spec_keys():
|
||||
# Both directions of the same guard: a shrunk allowlist breaks the first
|
||||
# assertion, a widened one the second.
|
||||
for key in ("by", "at", "from", "to", "id", "title", "author", "usage_count",
|
||||
"last_modified"):
|
||||
assert parse_frontmatter(f"---\nid: x\nk: {{ {key}: v }}\n---\n\nb\n")[0]["k"] == {key: "v"}
|
||||
for key in ("resource", "executor", "attester", "runtime", "command", "uri"):
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(f"---\nid: x\nk: {{ {key}: v }}\n---\n\nb\n")
|
||||
|
||||
|
||||
_FLOW_REJECTED = [
|
||||
# (id, value, what PyYAML 6.0.3 makes of it)
|
||||
("nested mapping", "{ by: { at: x } }", "a nested mapping"),
|
||||
("nested sequence", "{ by: [a, b] }", "a sequence leaf"),
|
||||
("anchor leaf", "{ by: &a x }", "an anchor definition, silently"),
|
||||
("tag leaf", "{ by: !!python/object:os.system x }", "refused outright"),
|
||||
("block scalar leaf", "{ by: | }", "a scanner error"),
|
||||
("nested colon leaf", "{ by: sub: v }", "refused outright"),
|
||||
("no space after colon", "{by:x}", "the KEY 'by:x', not a scalar"),
|
||||
("quoted leaf", "{ title: 'a, b' }", "a scalar - we refuse, deliberately"),
|
||||
("empty mapping", "{}", "an empty mapping"),
|
||||
("empty leaf", "{ by: }", "None"),
|
||||
("unclosed", "{ by: x", "a parse error"),
|
||||
("trailing junk", "{ by: x } more", "a parse error"),
|
||||
("duplicate key", "{ by: a, by: b }", "last-wins, silently"),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,value,yaml_reads_as", _FLOW_REJECTED,
|
||||
ids=[c[0] for c in _FLOW_REJECTED])
|
||||
def test_the_mapping_form_admits_scalar_leaves_on_allowlisted_keys_only(cid, value, yaml_reads_as):
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(f"---\nid: x\ngenerated: {value}\n---\n\nbody\n")
|
||||
|
||||
|
||||
def test_a_rejected_mapping_never_degrades_into_a_string():
|
||||
# The 1.1.0 defect, re-asserted against the NEW form: a refused mapping must
|
||||
# raise, not arrive as a str a consumer will .get() a key out of.
|
||||
for value in ("{ by: { at: x } }", "{ tool: shell }", "{ by: x"):
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(f"---\nid: x\ngenerated: {value}\n---\n\nbody\n")
|
||||
|
||||
|
||||
def test_the_admitted_mapping_is_a_dict_not_a_string():
|
||||
value = parse_frontmatter("---\nid: x\ngenerated: { by: a, at: b }\n---\n\nb\n")[0]["generated"]
|
||||
assert isinstance(value, dict), "a typed form that arrives as a str is the 1.1.0 defect"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm", [
|
||||
("block sequence, one key", "attester:\n - resource: attesters/sql_equality.py\n"),
|
||||
("inline second colon", "attester: resource: attesters/sql_equality.py\n"),
|
||||
("block mapping", "attester:\n resource: attesters/sql_equality.py\n"),
|
||||
("flow mapping, pointer key", "attester: { resource: attesters/sql_equality.py }\n"),
|
||||
])
|
||||
def test_the_pointer_routes_stay_shut(cid, fm):
|
||||
# G3 is additive: none of the routes that put an executable-code pointer in a
|
||||
# key the https allowlist never inspects is reopened. The fourth row is why
|
||||
# `resource` is off the allowlist - the form would otherwise have carried the
|
||||
# door-C pointer through in typed clothes instead of degraded ones.
|
||||
doc = f"---\nid: x\ntype: Attested Computation\n{fm}---\n\nbody\n"
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(doc)
|
||||
assert import_bundle({"computations/x.md": doc}).disposition is Disposition.FAIL_SECURE
|
||||
|
||||
|
||||
def test_injection_in_a_mapping_leaf_is_caught_by_the_scan():
|
||||
# T1 is not weakened by the new shape: a mapping leaf is scanned exactly like a
|
||||
# scalar value or a list item. A typed form that parses but is not scanned would
|
||||
# be a hole, not a fix.
|
||||
doc = f"---\ntype: table\ngenerated: {{ by: {_INJECTION} }}\n---\nclean body\n"
|
||||
assert scan_concept(doc).found is True
|
||||
|
||||
|
||||
def test_injection_in_a_listed_mapping_leaf_is_caught_by_the_scan():
|
||||
doc = f"---\ntype: table\nverified:\n - {{ by: {_INJECTION} }}\n---\nclean body\n"
|
||||
assert scan_concept(doc).found is True
|
||||
|
||||
|
||||
def test_a_conformant_v02_trust_layer_now_reaches_the_gate():
|
||||
# The measured consequence: a consumer reported 0 of 53 upstream concepts through
|
||||
# the gate, because every one of them carries §5.2 trust frontmatter.
|
||||
doc = (
|
||||
"---\n"
|
||||
"type: table\n"
|
||||
"title: Users\n"
|
||||
"resource: https://example.com/users\n"
|
||||
"generated: { by: reference_agent/gemini-2.5-pro, at: 2026-06-20T22:53:05Z }\n"
|
||||
"verified: { by: human:ahormati, at: 2026-06-25T09:00:00Z }\n"
|
||||
"usage_window: { from: 2026-06-01T00:00:00Z, to: 2026-06-30T00:00:00Z }\n"
|
||||
"---\nThe users table.\n"
|
||||
)
|
||||
result = import_bundle({"tables/users.md": doc})
|
||||
assert result.disposition is Disposition.WARN
|
||||
assert result.concepts[0].error is None
|
||||
|
||||
|
||||
# --- G30: the `sources` provenance layer becomes reachable (2026-09-02) ------
|
||||
# Door 2. G3 gave the mapping *class* one expressible form but left `sources`
|
||||
# unreachable: SPEC.md §5.1 writes an entry as a MAPPING carrying a REQUIRED
|
||||
# `resource`, so neither of the two carriers the spec and the producers actually
|
||||
# use could parse. Measured 02.09 by two consumers independently -- a flow
|
||||
# sequence of flow mappings raised on the `[` indicator, a block sequence of
|
||||
# block mappings raised "nested mappings are not supported".
|
||||
#
|
||||
# Why `resource` is admissible now when 1.2.0 argued it was not: the old
|
||||
# argument was that the parser could not tell `sources[].resource` (§5.1, a
|
||||
# citation) from `executor.resource` (§10, a code pointer). That premise was
|
||||
# false -- the owning key is in scope at every call site, it was simply never
|
||||
# threaded through. `resource` is allowlisted for `sources` entries ONLY, so
|
||||
# the door-C routes above stay shut on the same input.
|
||||
|
||||
_SPEC_51_BLOCK = (
|
||||
"sources:\n"
|
||||
" - id: ga4-schema\n"
|
||||
" resource: https://developers.google.com/analytics/bigquery/export-schema\n"
|
||||
" title: GA4 BigQuery Export schema\n"
|
||||
" author: team:ga4-docs\n"
|
||||
" usage_count: 5000\n"
|
||||
" last_modified: 2026-05-30T00:00:00Z\n"
|
||||
)
|
||||
|
||||
|
||||
def test_spec_sources_block_sequence_of_block_mappings_parses():
|
||||
# SPEC.md §5.1's own example block, verbatim. It is the spec's canonical
|
||||
# carrier for a REQUIRED field, so §11.1 ("parseable YAML frontmatter") makes
|
||||
# a bundle written this way conformant -- refusing it refuses a conformant
|
||||
# bundle, which is the failure mode G3 was opened to end.
|
||||
fm, _ = parse_frontmatter(f"---\ntype: table\n{_SPEC_51_BLOCK}---\n\nbody\n")
|
||||
assert fm["sources"] == [{
|
||||
"id": "ga4-schema",
|
||||
"resource": "https://developers.google.com/analytics/bigquery/export-schema",
|
||||
"title": "GA4 BigQuery Export schema",
|
||||
"author": "team:ga4-docs",
|
||||
"usage_count": "5000",
|
||||
"last_modified": "2026-05-30T00:00:00Z",
|
||||
}]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm,expected", [
|
||||
("one entry",
|
||||
"sources: [{ id: golden-v0-2-sales, resource: fixture }]\n",
|
||||
[{"id": "golden-v0-2-sales", "resource": "fixture"}]),
|
||||
("two entries",
|
||||
"sources: [{ id: a, resource: https://e.com/a }, { id: b, resource: https://e.com/b }]\n",
|
||||
[{"id": "a", "resource": "https://e.com/a"},
|
||||
{"id": "b", "resource": "https://e.com/b"}]),
|
||||
])
|
||||
def test_sources_flow_sequence_of_flow_mappings_parses(cid, fm, expected):
|
||||
# The form the producer emits today (llm-ingestion-okf's golden
|
||||
# expected-bundle/ingest-sales.md, measured 02.09). One entry raised too, so
|
||||
# this is not an arity bug: the `[` indicator refused the carrier outright.
|
||||
assert parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")[0]["sources"] == expected
|
||||
|
||||
|
||||
def test_the_two_sources_carriers_parse_to_the_same_value():
|
||||
block = parse_frontmatter(
|
||||
"---\ntype: t\nsources:\n - id: a\n resource: https://e.com/a\n---\n\nb\n")[0]
|
||||
flow = parse_frontmatter(
|
||||
"---\ntype: t\nsources: [{ id: a, resource: https://e.com/a }]\n---\n\nb\n")[0]
|
||||
assert block["sources"] == flow["sources"] == [{"id": "a", "resource": "https://e.com/a"}]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm", [
|
||||
("unknown key, flow", "sources: [{ id: a, uri: https://e.com/a }]\n"),
|
||||
("unknown key, block", "sources:\n - id: a\n uri: https://e.com/a\n"),
|
||||
("unknown key, flow value", "sources: { id: a, kind: doc }\n"),
|
||||
])
|
||||
def test_an_unknown_key_in_a_sources_entry_is_still_rejected(cid, fm):
|
||||
# The negative control. The carrier is admitted; the key set is not. A
|
||||
# producer-invented key gets no free ride on the new shape.
|
||||
with pytest.raises(OKFFrontmatterError) as exc:
|
||||
parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")
|
||||
assert "allowlist" in str(exc.value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm", [
|
||||
("executor, block sequence", "executor:\n - resource: skills/run-on-bq.md\n"),
|
||||
("executor, flow sequence", "executor: [{ resource: skills/run-on-bq.md }]\n"),
|
||||
("attester, flow sequence", "attester: [{ resource: attesters/sql_equality.py }]\n"),
|
||||
("attester, block sequence", "attester:\n - resource: attesters/sql_equality.py\n"),
|
||||
])
|
||||
def test_resource_is_allowlisted_for_sources_entries_only(cid, fm):
|
||||
# The whole reason `resource` can be admitted at all: the owning key decides.
|
||||
# §10's `executor.resource` / `attester.resource` name run instructions and
|
||||
# code -- door C -- and stay refused through EVERY carrier, including the two
|
||||
# this change opens for `sources`.
|
||||
doc = f"---\nid: x\ntype: Attested Computation\n{fm}---\n\nbody\n"
|
||||
with pytest.raises(OKFFrontmatterError) as exc:
|
||||
parse_frontmatter(doc)
|
||||
assert "allowlist" in str(exc.value)
|
||||
assert import_bundle({"computations/x.md": doc}).disposition is Disposition.FAIL_SECURE
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm", [
|
||||
# SPEC §5.1: "A single entry MAY carry its own `usage_window`". That is a
|
||||
# mapping inside a mapping -- depth 2 -- and stays refused: no nesting deeper
|
||||
# than one is a security property this change does not spend. Registered as a
|
||||
# conformance gap in docs/LIMITATIONS.md, not as an oversight.
|
||||
("per-entry usage_window, flow",
|
||||
"sources: [{ id: a, usage_window: { from: x, to: y } }]\n"),
|
||||
("per-entry usage_window, block",
|
||||
"sources:\n - id: a\n usage_window: { from: x, to: y }\n"),
|
||||
# A sequence inside an entry is the same depth violation.
|
||||
("nested sequence", "sources: [{ id: a, title: [x, y] }]\n"),
|
||||
])
|
||||
def test_a_sources_entry_admits_scalar_leaves_only(cid, fm):
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm", [
|
||||
("scalar then mapping", "sources:\n - https://e.com/a\n - id: b\n title: B\n"),
|
||||
("mapping then scalar", "sources:\n - id: a\n title: A\n - https://e.com/b\n"),
|
||||
])
|
||||
def test_a_block_list_may_not_mix_scalars_and_mappings(cid, fm):
|
||||
# A consumer that reads `entry.get("id")` over the list crashes on the str.
|
||||
# One list, one item type -- refuse rather than hand back a mixed tree.
|
||||
with pytest.raises(OKFFrontmatterError) as exc:
|
||||
parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")
|
||||
assert "mix" in str(exc.value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm", [
|
||||
("empty flow sequence", "sources: []\n"),
|
||||
("flow sequence, unclosed", "sources: [{ id: a }\n"),
|
||||
("flow sequence, trailing junk", "sources: [{ id: a }] x\n"),
|
||||
("flow sequence, nested sequence", "sources: [[ id ]]\n"),
|
||||
])
|
||||
def test_the_flow_sequence_refuses_a_carrier_it_cannot_read(cid, fm):
|
||||
# What no element shape rescues: nothing at all, an unclosed mapping, junk
|
||||
# between elements, and a sequence inside a sequence. The plain-scalar
|
||||
# element admitted in 1.4.0 (P1) is a THIRD shape, not a loosening of these
|
||||
# -- each row here still raises through it.
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")
|
||||
|
||||
|
||||
# --- P1: the flow sequence of plain scalars (operator decision, 2026-09-08) --
|
||||
# `tags: [a, b, c]` is the ONE form SPEC.md §4.1's own skeleton writes out for
|
||||
# `tags`, and the only candidate measured to move the upstream corpus at all:
|
||||
# 0/53 -> 6/53 against the pinned reference bundles (`_okf-upstream` @ 3fcbb9f,
|
||||
# denominator 53). The two neighbours measured 0/53 EACH and are deliberately
|
||||
# NOT built -- P2 (a flush-left block sequence) and P3 (a folded plain scalar)
|
||||
# would cost parser surface and buy no measured conformance on this corpus;
|
||||
# see docs/2026-09-07-limitations-44-maaling.md §§7-8.
|
||||
#
|
||||
# It does not spend the depth-1 rule: the elements are leaves, the same depth
|
||||
# the block list of scalars already carries. The binding constraint on the
|
||||
# corpus is elsewhere and stays refused -- 44/53 stop on `generated` written as
|
||||
# a top-level block mapping, which is a security decision, not this one.
|
||||
|
||||
@pytest.mark.parametrize("cid,fm,expected", [
|
||||
("SPEC §4.1 skeleton", "tags: [finance, revenue, headline-metric]\n",
|
||||
["finance", "revenue", "headline-metric"]),
|
||||
("two elements", "tags: [a, b]\n", ["a", "b"]),
|
||||
("one element", "tags: [solo]\n", ["solo"]),
|
||||
("uneven spacing", "tags: [ a ,b ]\n", ["a", "b"]),
|
||||
("one trailing comma, as the flow mapping already allows", "tags: [a, b,]\n", ["a", "b"]),
|
||||
])
|
||||
def test_a_flow_sequence_of_plain_scalars_parses(cid, fm, expected):
|
||||
# The red test for P1: `[a, b]` must come back as a real list of strings,
|
||||
# never a degraded string (the 1.1.0 defect) and never a refusal.
|
||||
assert parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")[0]["tags"] == expected
|
||||
|
||||
|
||||
def test_the_two_scalar_sequence_carriers_parse_to_the_same_value():
|
||||
flow = parse_frontmatter("---\ntype: t\ntags: [a, b]\n---\n\nb\n")[0]
|
||||
block = parse_frontmatter("---\ntype: t\ntags:\n - a\n - b\n---\n\nb\n")[0]
|
||||
assert flow["tags"] == block["tags"] == ["a", "b"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm", [
|
||||
# Quoting is the failure mode this shape is refused for elsewhere: the
|
||||
# parser retains quotes rather than stripping them, so admitting a quoted
|
||||
# element would hand back a value YAML reads differently.
|
||||
("single-quoted element", "tags: ['a']\n"),
|
||||
("double-quoted element", 'tags: ["a", "b"]\n'),
|
||||
# Comma splitting is character-level, so anything that would need YAML
|
||||
# semantics to split correctly is refused rather than guessed at.
|
||||
("colon inside an element", "tags: [a: b]\n"),
|
||||
("comment indicator", "tags: [a #b]\n"),
|
||||
("brace inside an element", "tags: [a{b}]\n"),
|
||||
("bracket inside an element", "tags: [a[b]]\n"),
|
||||
# Depth: a sequence inside a sequence opens a second level and is refused
|
||||
# on the same character rule, with no YAML semantics involved.
|
||||
("nested flow sequence", "tags: [[a]]\n"),
|
||||
("empty element", "tags: [a, , b]\n"),
|
||||
("leading empty element", "tags: [, a]\n"),
|
||||
("empty sequence", "tags: []\n"),
|
||||
("unterminated sequence", "tags: [a, b\n"),
|
||||
# The unchanged scalar indicators still apply to an element, exactly as they
|
||||
# do to a block-list item: an anchor or an alias is not a plain scalar.
|
||||
("anchor element", "tags: [&anchor]\n"),
|
||||
("alias element", "tags: [*alias]\n"),
|
||||
("explicit tag element", "tags: [!!python/object]\n"),
|
||||
])
|
||||
def test_a_flow_sequence_scalar_element_must_be_a_plain_scalar(cid, fm):
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm", [
|
||||
("scalar then mapping", "tags: [a, {b: c}]\n"),
|
||||
("mapping then scalar", "sources: [{ id: a }, plain]\n"),
|
||||
])
|
||||
def test_a_flow_sequence_may_not_mix_scalars_and_mappings(cid, fm):
|
||||
# Same rule, same reason as the block list: a consumer iterating the value
|
||||
# and reading `entry.get("id")` crashes on the first str. The mixing verdict
|
||||
# is reached BEFORE the element is parsed, so it is what the caller sees --
|
||||
# not an allowlist complaint about a key that was never the problem.
|
||||
with pytest.raises(OKFFrontmatterError) as exc:
|
||||
parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")
|
||||
assert "mix" in str(exc.value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm,expected", [
|
||||
("one entry", "sources: [{ id: x }]\n", [{"id": "x"}]),
|
||||
("two entries", "sources: [{ id: a, title: A }, { id: b, title: B }]\n",
|
||||
[{"id": "a", "title": "A"}, {"id": "b", "title": "B"}]),
|
||||
])
|
||||
def test_the_g30_flow_mapping_carrier_is_unchanged_by_the_scalar_element(cid, fm, expected):
|
||||
# The no-regression pin. P1 adds a branch to the same function that carries
|
||||
# `sources`; the mapping element must parse exactly as it did in 1.3.0.
|
||||
assert parse_frontmatter(f"---\ntype: table\n{fm}---\n\nbody\n")[0]["sources"] == expected
|
||||
|
||||
|
||||
# The self-safety row for P1 (OWASP LLM10). The predicate is character-level
|
||||
# with no regex, so there is no backtracking engine to blow up -- but "no regex"
|
||||
# is an argument, and the sweep in docs/redos-sweep.py cannot check it, because
|
||||
# it collects compiled patterns and this predicate compiles none. So it is
|
||||
# measured instead, on the same CPU clock every other bound here uses. Over four
|
||||
# doublings (50k -> 800k, 2026-09-08) the exponent is 0.86-0.99 in the element
|
||||
# LENGTH and 0.97-1.05 in the element COUNT: linear in both, 0.66s at 800_000
|
||||
# elements and 0.03s at an 800_000-character element. The refusal path is the
|
||||
# same shape -- a forbidden character at the very end of an 800_000-character
|
||||
# element is found in 0.04s -- because the element is scanned once per rule, not
|
||||
# rescanned per start position.
|
||||
_FLOW_SCALAR_PERF_N = 400_000
|
||||
|
||||
|
||||
def test_a_long_flow_sequence_of_scalars_stays_bounded():
|
||||
many = "---\ntype: t\ntags: [" + ", ".join(["ab"] * _FLOW_SCALAR_PERF_N) + "]\n---\nbody\n"
|
||||
assert scan_seconds(parse_frontmatter, many) < 2.0
|
||||
one_long = "---\ntype: t\ntags: [" + "a" * _FLOW_SCALAR_PERF_N + "]\n---\nbody\n"
|
||||
assert scan_seconds(parse_frontmatter, one_long) < 2.0
|
||||
|
||||
|
||||
def test_a_long_flow_sequence_element_is_refused_without_a_rescan():
|
||||
# The refusal is the half an attacker controls: a forbidden character parked
|
||||
# at the END of a long element is the worst case for any per-start rescan.
|
||||
doc = "---\ntype: t\ntags: [" + "a" * _FLOW_SCALAR_PERF_N + ":]\n---\nbody\n"
|
||||
|
||||
def refuse(payload):
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(payload)
|
||||
|
||||
assert scan_seconds(refuse, doc) < 2.0
|
||||
|
||||
|
||||
def test_injection_in_a_sources_entry_leaf_is_caught_by_the_scan():
|
||||
# T1 over the new shape: every leaf of every entry reaches scan_output.
|
||||
doc = f"---\ntype: table\nsources: [{{ id: a, title: {_INJECTION} }}]\n---\nclean\n"
|
||||
assert scan_concept(doc).found is True
|
||||
doc_block = f"---\ntype: table\nsources:\n - id: a\n title: {_INJECTION}\n---\nclean\n"
|
||||
assert scan_concept(doc_block).found is True
|
||||
|
||||
|
||||
def test_the_producer_golden_now_passes_the_gate():
|
||||
# llm-ingestion-okf's expected-bundle/ingest-sales.md, the K5 blocker.
|
||||
doc = (
|
||||
"---\n"
|
||||
"type: dataset\n"
|
||||
"title: Regional Sales\n"
|
||||
"source_system: golden-v0-2-sales\n"
|
||||
"ingested_at: 2026-07-16T12:00:00Z\n"
|
||||
"generated: { by: process:okf-ingest, at: 2026-07-16T12:00:00Z }\n"
|
||||
"sources: [{ id: golden-v0-2-sales, resource: fixture }]\n"
|
||||
"---\n\n| region | units |\n| --- | --- |\n| nord | 412 |\n"
|
||||
)
|
||||
result = import_bundle({"datasets/sales.md": doc})
|
||||
assert result.disposition is Disposition.WARN
|
||||
assert result.concepts[0].error is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm", [
|
||||
# A `- ` item whose text begins with a YAML indicator AND carries a `": "`
|
||||
# must fall through to the scalar predicates, not into the block-mapping
|
||||
# route with a half-validated key. Each of these fails `_KEY_RE` on the key
|
||||
# side, so `_reject_dangerous_value` gets the item intact.
|
||||
("anchor", "sources:\n - &anchor id: a\n"),
|
||||
("alias", "sources:\n - *anchor id: a\n"),
|
||||
("tag", "sources:\n - !!str id: a\n"),
|
||||
("directive", "sources:\n - %YAML id: a\n"),
|
||||
("reserved", "sources:\n - `x id: a\n"),
|
||||
("merge key", "sources:\n - id: a\n <<: *base\n"),
|
||||
])
|
||||
def test_an_indicator_in_a_block_item_is_refused_before_the_mapping_route(cid, fm):
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(f"---\ntype: t\n{fm}---\n\nbody\n")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cid,fm", [
|
||||
# The block mapping ends at a blank line and at a line in column zero. Both
|
||||
# hand control back with an index that must not skip or re-read a line: a
|
||||
# dangling `resource:` line left over from a mapping that closed early must
|
||||
# RAISE, never be silently dropped -- a pointer that vanishes rather than
|
||||
# failing is exactly this repo's failure class.
|
||||
("blank line inside the mapping", "sources:\n - id: a\n\n resource: b\n"),
|
||||
("top-level key interleaved", "sources:\n - id: a\ntags: x\n resource: b\n"),
|
||||
])
|
||||
def test_a_line_orphaned_by_the_mapping_boundary_raises_rather_than_vanishing(cid, fm):
|
||||
with pytest.raises(OKFFrontmatterError):
|
||||
parse_frontmatter(f"---\ntype: t\n{fm}---\n\nbody\n")
|
||||
|
||||
|
||||
def test_the_block_list_hands_back_an_index_that_resumes_at_the_next_key():
|
||||
# The return-index contract: a top-level key following a multi-entry block
|
||||
# list is neither swallowed by the list nor re-read as a list item.
|
||||
fm, _ = parse_frontmatter(
|
||||
"---\ntype: t\nsources:\n - id: a\n resource: b\n - id: c\ntitle: T\n---\n\nbody\n")
|
||||
assert fm == {"type": "t", "sources": [{"id": "a", "resource": "b"}, {"id": "c"}],
|
||||
"title": "T"}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue