Kjell Tore Guttormsen
a965e8ac5b
Order 20260902T150716Z from .claude -- a K5 blocker in the OKF programme.
`parse_frontmatter` rejected `sources` in every form the spec and its
producers actually use. Measured 02.09 by two consumers independently:
`sources: [{ id: a, resource: x }]` raised on the `[` indicator (one entry
as well as two), and the block sequence of block mappings -- SPEC.md 5.1's
OWN example -- raised "nested mappings are not supported". `resource` is
REQUIRED within a `sources` entry, so the whole provenance family was
unwritable and a bundle written the way the spec documents it was refused.
Measured against the spec before coding, not reasoned: 5.1's example block
is the canonical carrier for a REQUIRED field and 11.1 defines conformance
as parseable frontmatter, so refusing it refuses a conformant bundle. Both
carriers now parse to the same list of dicts.
The load-bearing change is not the carrier, it is WHO admits `resource`.
1.2.0 left it off the allowlist arguing the parser could not tell
`sources[].resource` (5.1, a citation) from `executor.resource` /
`attester.resource` (10, a pointer to code to be run -- the door-C route
closed in 1.1.0). That premise was false: the owning key is in scope at
every call site and was simply never threaded through. It is threaded now,
so the discrimination is structural, and door C stays shut through EVERY
carrier including the two this adds -- pinned by a new test that drives
`executor`/`attester` through all four.
Refusal stays the default elsewhere. A flow sequence of plain scalars
(`tags: [a, b]`) still raises: the sequence carrier is opened for the flow
mapping element and nothing else. A `sources` entry admits scalar leaves
only, so 5.1's optional PER-ENTRY `usage_window` is refused -- no nesting
past depth 1 is a security property and it was not spent here; registered
as a conformance gap rather than left as an oversight. A block list may not
mix scalars and mappings, because a consumer reading `entry.get("id")` over
one gets an AttributeError off the first str.
New residual registered: `sources[].resource` is scanned as text (T1) but
never URL-validated. T3's https allowlist cannot reach it without
over-blocking conformant bundles -- 5.1 permits bundle-relative paths and
scope descriptors, and the producers' own golden emits `resource: fixture`.
A consumer that dereferences it must call `validate_resource_url` itself.
Suite 834 -> 859 green. 25 new rows; four pre-existing rows changed because
this release changed the behaviour they pinned, two of them renamed since
their names asserted the old invariant (`exactly_one_route_to_a_mapping`,
`two_keys_per_item_is_where_the_block_list_hard_rejects`). Not "unchanged".
130/130 classes, 6/6 gaps hold, 44 -> 45 limitations, ReDoS 0/152 (the
sweep adds no evidence here -- this change adds no regex and the splitting
is linear). Six version surfaces bumped by hand, no sed. Re-measured alone
after the bump.
No exported surface changed; no detector behaviour and no calibration
changed.
75 lines
4.5 KiB
Markdown
75 lines
4.5 KiB
Markdown
# llm-ingestion-pipeline-security
|
|
|
|
## Kontekst
|
|
|
|
Gjenbrukbar, minimal defensiv layer for LLM **ingestion**-pipelines (write-time),
|
|
til forskjell fra query-time chatbot-guardrails. Pakker det arkitektoniske
|
|
kontraktet — sanitize → fence → tool-less karantenert transform → per-stadium
|
|
capability-isolasjon → scan output før commit → fail-secure — som komponerbar,
|
|
framework-agnostisk kode.
|
|
|
|
Referanse-implementasjon: `claude-code-llm-wiki` Stage B (`tools/wiki_ingest/`).
|
|
Lexikon-seed: `injection-patterns.mjs` fra `llm-security`-pluginen.
|
|
|
|
Repoet er på **v1.3.0** — den eksporterte Python-surfacen er frosset under semver
|
|
(deteksjonsatferd er det IKKE; kalibrering flytter seg i 1.x). Stdlib-kjernen er
|
|
bygget og testet (15 moduler +
|
|
topp-nivå wiring, showcase + korpus), inkl. OKF-adapter og aktivt-innhold-
|
|
detektor (EchoLeak-klassen) i output-gaten. OKF-frontmatterens mapping-klasse
|
|
har **fire** uttrykkbare bærere (G3 21.08, G30 02.09): flow-mapping som verdi
|
|
og som blokkliste-element, flow-sekvens av flow-mappinger, og blokk-sekvens av
|
|
blokk-mappinger (SPEC §5.1s egen form). HVER nøkkel i alle fire står på
|
|
allowlisten og hvert blad er en ren skalar. Formen er trygg fordi allowlisten
|
|
inspiserer hver nøkkel; det blanke avslaget var håndhevelsen, ikke poenget.
|
|
**`resource` er allowlistet KUN inne i en `sources`-oppføring** — foreldre-
|
|
nøkkelen avgjør, så `executor`/`attester` sin `resource` (§10, dør C) avvises
|
|
gjennom hver eneste bærer. 1.2.0s begrunnelse for å utelate den (parseren
|
|
manglet foreldre-kontekst) var målt feil: konteksten var der, den var bare
|
|
aldri tredd gjennom. Topp-nivå blokk-mapping, dotted- og inline-kolon-rutene
|
|
raiser fortsatt, flow-sekvens av rene skalarer (`tags: [a, b]`) raiser, en
|
|
blokkliste kan ikke blande skalarer og mappinger, og en avvist mapping raiser —
|
|
den degraderer aldri til en streng (1.1.0-defekten). `sources[].resource`
|
|
URL-valideres ALDRI (T3 ser kun topp-nivå `resource`) — §5.1 tillater
|
|
bundle-relative stier og scope-beskrivelser, så en https-gate ville over-blokkert
|
|
konforme bundles; konsumenten må selv kalle `validate_resource_url`. Mode-b `import_bundle` skanner
|
|
reserverte strukturfiler (`index.md`/`log.md`) i mottatte bundles i stedet for å
|
|
path-avvise dem; upload-front-end beholder shadow-reject (`allow_reserved=False`).
|
|
Output-gatens decode-and-rescan mater dekodet base64-klartekst gjennom BÅDE lexicon
|
|
og secret-egress (LLM02), så en base64-innpakket credential fanges som
|
|
`decoded:egress:*` i stedet for å forsvinne; hex-innpakket er en dokumentert
|
|
restgap (entropy eksponerer kun base64-klartekst). `active:raw-html` krever et
|
|
EKSTERNT mål på URL-attributt-grenen, og `<base>` er ute av det aktive navnesettet;
|
|
scanner og mutator har hver sin predikat (`is_active_tag` / `is_defangable_tag`).
|
|
Rå HTML graderes nå også på BÆRER: `<a>`/`<area>` er klikk-krevende og rapporteres
|
|
som `active:raw-html-link` (MEDIUM), og en tagg hvis hele affordans ER en URL den
|
|
ikke bærer (`</a>`, `<Frame>`, `<video />`) er inert. Klassifisering skjer i
|
|
`active_tag_class`; `is_active_tag` er en tynn wrapper, og census patcher den
|
|
FØRSTE (en boolsk patch kan ikke uttrykke en regradering).
|
|
ZWJ (U+200D) dømmes på KONTEKST, ikke identitet — unntas kun mellom to emoji, på
|
|
begge flater (`sanitize` eier predikatet, `output` importerer det).
|
|
Start med `docs/BRIEF.md` for design, `README.md` for bruk, `docs/PLAN.md` for
|
|
byggerekkefølgen.
|
|
|
|
## Konvensjoner
|
|
|
|
- Norsk for dialog og planer, engelsk for kode og innhold (repoet er publisert).
|
|
- Ingen GitHub — kun Forgejo (`git.fromaitochitta.com`).
|
|
- Remote satt: offentlig `open/`-speil på Forgejo; push hver commit (durabelt autorisert).
|
|
- Minimal-dependency: stdlib-first kjerne; ML/judge-detektorer bak extras.
|
|
|
|
## Communication patterns
|
|
|
|
### Linking to local files
|
|
|
|
When pointing to local files in responses, always use markdown link syntax with a descriptive name:
|
|
|
|
- Use `[Human-friendly name](file:///absolute/path)` — never bare `file:///...` URLs or autolinks `<file://...>`.
|
|
- Always use absolute paths. Never `~/` or relative paths.
|
|
- For multiple files, render as a bullet list of named markdown links.
|
|
|
|
Why: bare `file://` URLs only render the first as clickable across multiple lines. Named markdown links make each entry independently clickable and look cleaner.
|
|
|
|
Example (the path is a placeholder — the checkout root is the reader's own, and
|
|
this file is published, so it must not carry one machine's directory layout):
|
|
|
|
- [Brief](file:///absolute/path/to/llm-ingestion-pipeline-security/docs/BRIEF.md)
|