1
0
Fork 0

feat(okf): a flow sequence of plain scalars parses, and the corpus number is 6/53

`tags: [a, b, c]` is the form SPEC.md 4.1's own frontmatter skeleton writes out,
and 9/53 upstream reference concepts use it. It raised on the `[` indicator.
It parses now, to the same list its block-sequence sibling already produced.

The predicate is character-level, inside `_parse_flow_sequence`: an element is a
plain scalar only if it is non-empty and carries none of `{ } [ ] : , " ' #`,
and it then passes the unchanged scalar-indicator rule. Everything that would
need YAML semantics to split or unquote still raises - a quoted element (quotes
are retained here, never stripped), a colon or comma inside an element, a
sequence inside a sequence, an empty element, an anchor, an alias. A sequence
may not mix scalars and mappings, the rule the block list already carries, and
the mixing verdict is reached before the element is parsed so the caller is told
about the mix rather than about a key the allowlist would have named instead.
The 1.3.0 `sources` flow-mapping carrier is unchanged and pinned against
regression. Depth 1 is not spent: the elements are leaves.

Measured with the denominator, against the pinned corpus (`_okf-upstream` @
3fcbb9f, 53 non-reserved documents) and the pinned SPEC (`_okf-canonical` @
ad30107). Baseline reproduced first, with a known-positive control, at 0/53.
After: 6/53, all six in acme_retail. It does not close the corpus - 44/53 still
stop on `generated` written as a top-level block mapping, which spends the
no-nesting-past-depth-1 rule and is a security decision, out of scope here.

P1 alone, per the operator decision of 08.09. The two neighbouring predicates
were measured and deliberately not built: a flush-left block sequence and a
folded plain scalar release 0/53 each on their own, and stacked on this one they
still measure 6/53. `_consume_block_list`, the `description` continuation and
the allowlist are untouched.

docs/LIMITATIONS.md's tags/description entry is rewritten against the
measurement: three of its claims were wrong. The parser does have a
sequence-value type (since 1.3.0 - what it lacks is the indentation the corpus
omits); the figure is 6/53, not the 1.2.0-era 4/53; and tags/description are
not the residual that blocks the corpus. README gains the sequence carrier in
the paragraph that already describes the mapping one.

Self-safety: the predicate compiles no regex, so docs/redos-sweep.py cannot see
it. Measured instead on the CPU clock - linear in element length (exponent
0.86-0.99) and in element count (0.97-1.05) over four doublings to 800_000 -
and pinned by two bounds in tests/test_okf.py.

Six rows that pinned the old refusal are re-aimed at the class that still
holds - the quoted element - the way the 1.3.0 rows were when the carrier
opened. One of them lives in src/llm_ingestion_guard/coverage.py, which is why
the src diff is three files rather than one.

Version 1.4.0 in the code only. The CHANGELOG entry stays under Unreleased and
no tag is cut: README's badge and install pin must keep naming a tag that
exists.

Gates after `git add`: 893 passed (was 868), coverage 130/130 + 6/6 gaps,
redos-sweep exit 0, LIMITATIONS still 45 entries.
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 05:34:55 +02:00
commit 3e324a1f86
9 changed files with 341 additions and 59 deletions

View file

@ -5,6 +5,53 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added — `tags: [a, b, c]`, the one flow-sequence form SPEC §4.1 writes out
`okf.parse_frontmatter` refused a flow sequence of plain scalars. That is the
form SPEC.md §4.1's own frontmatter skeleton uses for `tags`, and `tags` is
present in 53/53 upstream reference concepts, 9/53 of them in exactly this form.
Admitted now, parsing to the same value its block-sequence sibling already did:
```yaml
tags: [finance, revenue, headline-metric]
```
An element is a plain scalar only if it is non-empty and carries none of
`{ } [ ] : , " ' #`, and it then passes the unchanged scalar-indicator rule.
The refusals that make that a rule rather than a preference: a quoted element
(this parser retains quotes rather than stripping them, so reading one would
hand back a value YAML does not), an element carrying a colon or a comma, a
sequence inside a sequence, an empty element, an anchor or an alias. A sequence
may not **mix** scalars and mappings — the same rule the block list already
carries, for the same reason: a consumer iterating the value and reading
`entry.get("id")` crashes on the first `str`. The `sources` flow-mapping
carrier added in 1.3.0 is unchanged and pinned against regression.
**Measured, with the denominator: this takes the pinned OKF corpus from 0/53 to
6/53** (`_okf-upstream/okf` @ `3fcbb9f`, 53 non-reserved documents; the six are
all in `acme_retail`). It does not close the corpus. **44/53 still stop on
`generated` written as a top-level block mapping** — a form refused on the
no-nesting-past-depth-1 rule, which is a security decision and out of scope
here. Two neighbouring predicates were measured and deliberately not built: a
flush-left block sequence and a folded plain scalar release **0/53** each on
their own and leave the corpus at 6/53 when stacked on this one. Detection
behaviour is not frozen under semver; the exported surface is unchanged.
### Changed
- `docs/LIMITATIONS.md`'s `tags`/`description` entry is rewritten against the
measurement. Three of its claims were wrong: the parser *does* have a
sequence-value type (since 1.3.0 — what it lacks is the indentation the
corpus omits), the figure is 6/53 and not the 1.2.0-era 4/53, and
`tags`/`description` are *not* the residual that blocks the corpus.
- Self-safety: the new predicate is character-level with no regex, so
`docs/redos-sweep.py` cannot see it (it collects compiled patterns). It is
measured instead — linear in both element length (exponent 0.86-0.99) and
element count (0.97-1.05) over four doublings to 800_000, and pinned by two
CPU-clock bounds in `tests/test_okf.py`.
## [1.3.0] — 2026-09-02
### Added — the `sources` provenance family becomes expressible in both spec carriers