fix(okf): a flow sequence admits '#' and ':' where YAML reads them as text
1.4.0 refused a flow-sequence scalar element carrying '#' or ':' anywhere. okf writes `references: [...]` as a flow sequence of doc links, and an entry may carry a #fragment or a scheme://. claude-code-llm-wiki measured parse_frontmatter raising on 2 038 of 5 467 concepts of a bundle okf produced (2026-09-17); reproduced here on the same bundle, 2 038 / 5 467 before and 0 / 5 467 after (0 / 4 466 and 0 / 5 530 on the two other builds). The two characters are now refused by position, where PyYAML 6.0.3 gives them a meaning: '#' opening the element or following whitespace (a comment), ':' opening or ending the element or preceding whitespace (a mapping). Elsewhere they are scalar characters. The rest of the refused set, the indicator rule and the no-mixing rule are unchanged. Chosen over admitting the characters wholesale because `[a: b]` is a mapping and `[a #b]` a comment to YAML; admitting those would hand the consumer a value YAML does not read. The position checks are substring tests, no regex: CPU time stays linear in element count and length (100k -> 800k), about 10 % over 1.4.0, inside the existing 2 s bound. Pinned upstream corpus unmoved at 6 / 53. 910 passed (+17), coverage exit 0, redos-sweep exit 0, 45 LIMITATIONS entries.
This commit is contained in:
parent
79285e1265
commit
639da03f8c
4 changed files with 119 additions and 6 deletions
|
|
@ -81,7 +81,15 @@ _QUOTE_STARTS = frozenset("\"'")
|
|||
# two splitters, the two collection openers and their closers, and the comment
|
||||
# indicator. Refusing them is what lets the element be split on commas at the
|
||||
# character level without a YAML quote state machine.
|
||||
_FLOW_SCALAR_REFUSED = "{}[]:,\"'#"
|
||||
#
|
||||
# 1.4.1 - `:` and `#` are refused only where YAML gives them a meaning, not
|
||||
# anywhere in the element: okf writes `references: [...]` with `#fragment`s and
|
||||
# `scheme://` links, and refusing those raised on 2 038 of 5 467 concepts of a
|
||||
# consumer's bundle (measured 2026-09-17, reproduced 2026-09-23). Ground truth,
|
||||
# PyYAML 6.0.3: `#` opens a comment at the start of an element or after
|
||||
# whitespace; `:` opens a mapping at the start of an element, at its end, or
|
||||
# before whitespace. Elsewhere both are ordinary scalar characters.
|
||||
_FLOW_SCALAR_REFUSED = "{}[],\"'"
|
||||
|
||||
# G3 - the one mapping form T2 can express (operator decision, 2026-08-21).
|
||||
# Every key inside a mapping must be on this allowlist: the form is safe because
|
||||
|
|
@ -937,7 +945,8 @@ def _parse_flow_sequence(value, parent_key=None):
|
|||
docs/2026-09-07-limitations-44-maaling.md). The quoting and comma-splitting
|
||||
problem that kept it refused is answered by refusing the characters that
|
||||
create it rather than by parsing them: an element is a plain scalar only if
|
||||
it is non-empty and carries none of ``{ } [ ] : , " ' #``, and it then
|
||||
it is non-empty and carries none of ``{ } [ ] , " '``, carries ``#`` and
|
||||
``:`` only where YAML reads them as scalar characters (1.4.1), and it then
|
||||
passes the unchanged scalar indicator rule. Everything needing YAML
|
||||
semantics to split or unquote correctly still raises.
|
||||
|
||||
|
|
@ -1019,9 +1028,17 @@ def _flow_sequence_scalar(element, value):
|
|||
A quote would have to be stripped (this parser retains quotes, so it would
|
||||
hand back a different value than YAML reads); a comma or a colon would have
|
||||
to be split on; a bracket or a brace would open a second collection level,
|
||||
which no carrier here admits; a ``#`` opens a comment. The unchanged
|
||||
indicator rule then applies to what is left, exactly as it does to a
|
||||
block-list item, so an anchor or an alias is no more a scalar here.
|
||||
which no carrier here admits. The unchanged indicator rule then applies to
|
||||
what is left, exactly as it does to a block-list item, so an anchor or an
|
||||
alias is no more a scalar here.
|
||||
|
||||
``#`` and ``:`` are refused by position, where YAML reads them (1.4.1):
|
||||
a ``#`` that opens the element or follows whitespace opens a comment, and a
|
||||
``:`` that opens or ends the element, or precedes whitespace, opens a
|
||||
mapping. ``a#b``, ``x://y`` and ``https://e.com:8443/a`` are plain scalars
|
||||
to YAML and to this parser alike. The element has already been cut at the
|
||||
next comma and stripped, so its end is the only place a ``:`` can meet the
|
||||
separator.
|
||||
"""
|
||||
if not element:
|
||||
raise OKFFrontmatterError(
|
||||
|
|
@ -1033,5 +1050,15 @@ def _flow_sequence_scalar(element, value):
|
|||
"a flow-sequence scalar admits plain scalars only, not %r: %r"
|
||||
% (char, value)
|
||||
)
|
||||
if element[0] == "#" or " #" in element or "\t#" in element:
|
||||
raise OKFFrontmatterError(
|
||||
"a '#' at the start of a flow-sequence element or after whitespace "
|
||||
"opens a comment: %r" % (value,)
|
||||
)
|
||||
if element[0] == ":" or element[-1] == ":" or ": " in element or ":\t" in element:
|
||||
raise OKFFrontmatterError(
|
||||
"a ':' at the start or end of a flow-sequence element, or before "
|
||||
"whitespace, opens a mapping: %r" % (value,)
|
||||
)
|
||||
_reject_dangerous_value(element)
|
||||
return element
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue