fix(frontmatter): write a value a YAML reader reads back, and read both forms
K3-22. SPEC SS 11 point 1: "Every non-reserved `.md` file in the tree
contains a parseable YAML frontmatter block." Measured with PyYAML 6.0.3,
okf's own default K2 bundle failed safe_load on 41 of 455 blocks and the
R761 build on 1 of 2 763, every one a block scalar written verbatim.
Block (the profile emitter, every key): a value the K3-19 rule refuses as
plain is written double-quoted, `\` and `"` escaped; every other value keeps
its bytes, and a flow collection or an empty value is written as it stands.
The rule, now `profiles.yaml_block_plain`, agrees with PyYAML on every
top-level value in eleven measured trees (0 refused that it reads verbatim,
0 kept that it does not). Double, never single: 0 values in those trees are
`"`-wrapped and 11 193 are `'`-wrapped.
Flow (`sources`, Door A and Door B, and a run-stated flow value): the pinned
guard refuses ANY quote in a flow mapping (1.3.0, measured), so a leaf PyYAML
needs quoted has no form both read. `yaml_flow_plain` refuses it instead:
`,[]{}`, `?`, a quote, ": ", " #", a trailing `:`, a leading indicator -- a
leading `-` before a non-space excepted, which both readers take. The file
name is checked too, because it is the entry's `title` when the document
declares none. Existing codes: inbox_source_file_unaddressable,
inbox_source_title_unaddressable, source_reference_unquotable,
run_frontmatter_invalid.
Readers: parse_frontmatter, profiles' and structure's copies, and both
read_sources branches unquote a `"`-wrapped value (`\"` and `\\` decoded,
nothing else); `'`-wrapped values are untouched, and structure keeps the
single-quote rule it already had. The flow-mapping split is quote-aware, so
`{ title: "a, b" }` is one pair. The generated SKILL.md header goes through
the same block rule.
TWO K3-19 TESTS MOVED, deliberately: test_run_frontmatter built with
`sources=[{ resource: ...?languageCode=nb, ... }]`, the exact form PyYAML
refused on 2 761 of 2 761 frontmatters of K3-19's flagged build. The two
build tests now write an address without `?`; the flag-grammar test keeps
the `?` address (it only splits), and a new test holds that the build
refuses it with exit 2 and writes nothing.
1753 passed, 1 skipped (OKF_HTML_CORPUS, known). No golden moved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
06e61a5acf
commit
ed0418f228
7 changed files with 288 additions and 79 deletions
|
|
@ -29,7 +29,7 @@ from .manifest import (
|
|||
generated_filename,
|
||||
load_manifest_bytes,
|
||||
)
|
||||
from .profiles import DEFAULT, BundleProfile
|
||||
from .profiles import DEFAULT, BundleProfile, unquote_scalar, yaml_flow_plain
|
||||
from .render import render_fenced_block, render_table
|
||||
|
||||
_LOGGER = logging.getLogger(__name__)
|
||||
|
|
@ -135,7 +135,10 @@ def parse_frontmatter(path: Path) -> dict[str, str]:
|
|||
continue
|
||||
key, sep, value = line.partition(":")
|
||||
if sep:
|
||||
frontmatter[key.strip()] = value.strip()
|
||||
# A `"`-wrapped value is how the emitter writes a scalar a YAML
|
||||
# reader would refuse plain (K3-22); read back as that reader
|
||||
# would. A `'`-wrapped one is returned as it stands.
|
||||
frontmatter[key.strip()] = unquote_scalar(value.strip())
|
||||
return frontmatter
|
||||
|
||||
|
||||
|
|
@ -170,12 +173,6 @@ def _is_ingest_owned(path: Path, manifest_stem: str, *, profile: BundleProfile =
|
|||
return reference.rsplit("@", 1)[0] == manifest_stem
|
||||
|
||||
|
||||
# The characters that terminate or restructure a YAML flow mapping. `:\s`
|
||||
# catches a colon that would open a nested key; a colon inside `https://host`
|
||||
# does not, and stays a plain scalar.
|
||||
_FLOW_UNSAFE_RE = re.compile(r"[,\[\]{}]|:\s")
|
||||
|
||||
|
||||
def _source_locator(source: Source) -> str:
|
||||
"""Where a manifest source points, per source type.
|
||||
|
||||
|
|
@ -238,11 +235,12 @@ def _render_sources(sources: Sequence[Source]) -> str:
|
|||
for source in sources:
|
||||
locator = _source_locator(source)
|
||||
for label, value in (("id", source.id), ("resource", locator)):
|
||||
if _FLOW_UNSAFE_RE.search(value):
|
||||
if not yaml_flow_plain(value):
|
||||
raise MaterializationError(
|
||||
f"the source {label} {value!r} contains a character that would "
|
||||
"restructure the `sources` flow mapping (one of `,[]{}` or a "
|
||||
"colon followed by whitespace) — refusing to emit a provenance "
|
||||
f"the source {label} {value!r} has no plain form in the `sources` "
|
||||
"flow mapping that both a YAML reader and the guard read back "
|
||||
"verbatim (`,[]{}`, `?`, a quote, ': ', ' #', a trailing `:` or a "
|
||||
"leading YAML indicator) — refusing to emit a provenance "
|
||||
"record that parses cleanly into something no one wrote",
|
||||
code="source_reference_unquotable",
|
||||
)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue