feat(propose,consume,profiles,importer): recovery yields to declaration, and 9 % of the corpus that was in no segment
One rule explains every remaining `pdf` miss on the twelve-position reference: where a document DECLARES headings, Arm D's RECOVERED headings are the whole of the excess, and every declared one is a unit the reference wants. `--outline-gate` admits recovery only where the document declares none of its own, plus any one recovered heading covering OUTLINE_SHARE (0.20) of the text. It is `fold_units` clause 2's own principle moved from voting to admission, and it filters at ADMISSION so the text a removed mark opened is carried by the mark above it -- the post-filter form scores identically on all twelve positions and loses that text, which is why only one of them shipped. `--outline-gate` and `--drop-wrapped-outline` become the package default, one decision because neither carries the reference alone: `pdf` 2 of 8 -> 5 of 8 alone, 7 of 8 together; the sheet 5 of 12 -> 10 of 12; `docx` unchanged at 3 of 3. Each keeps an explicit opt-out. The bar the move had to clear was not the reference: hit@8 on a K2 bundle built with it holds 5 of 6 at ranks 1,1,1,1,1,-, no row losing rank 1. `--sheet-section-rows --keep-table-heading` reaches 11 of 12 and does NOT ship, because on a bundle built with it row 1 falls rank 1 -> 2. Cost to a consumer is a re-run: 492 concepts / 944 files -> 425 / 810. DOCUMENT_PRIOR_EXPONENT makes the document prior sublinear (total/n**0.5). A sum measures size and a density is diluted by every unit carrying none of the question, so a document split 1 -> 12 lost its prior by 12. Swept over five values on 18 rows it is at least as good as the delivered density everywhere and strictly better on three. Stated plainly: end to end it moved NOT ONE hit@8 row on any of four bundles, so it did not solve the knot it was adopted for -- what did is that the `pdf` gain never needed `--sheet-section-rows`. `--first-span-from-zero` is off and repairs a measured loss found while chasing one position's 940 characters: 32 of the 32 documents that get a plan leave the text above their first concept in no segment -- 159 704 characters, 9.18 % of the corpus, 45 841 from one document. It changes nothing on the reference. Off because it moves the first span of essentially every bundle with no hit@8 number behind it yet. vegnormal-okf FUNN 2: SPEC section 8's own star row parsed as prose, so every concept behind one was unreachable to the section 9.2 walk. `IndexPolicy.also_reads` carries it for the SEGMENTED profiles, read-only, after the emitted pattern misses -- the asymmetry `sources` already has. DEFAULT and STRICT_V1 untouched (O2). vegnormal-okf FUNN 1: Door C's own outcome was refused at exit 1, `bundle_id_missing`. `import_bundle` now takes `root_frontmatter_values`, keyword-only, rendered before any disk mutation, written only when the index is created -- Door B's mechanism and ordering. Report: docs/2026-09-09-k3-runde6-outline-gaten-og-prioren.md. Suite 1478 passed (1449 before), ruff and mypy clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
b01492b7f5
commit
38104b7df5
16 changed files with 1301 additions and 42 deletions
|
|
@ -564,6 +564,27 @@ class IndexPolicy:
|
|||
root_frontmatter: tuple[str, ...] = ()
|
||||
root_frontmatter_required: frozenset[str] = field(default_factory=frozenset)
|
||||
facets: FacetPolicy | None = None
|
||||
#: Row forms this policy READS but never writes. Consulted by `parse_entry`
|
||||
#: only, after `entry_pattern` misses, so no emitted byte can move: every
|
||||
#: line this library writes still comes from `link_template`.
|
||||
#:
|
||||
#: WHY IT EXISTS (vegnormal-okf, FUNN 1/2, 2026-09-08). OKF SPEC section 8
|
||||
#: shows `* [Title](file.md) - description` in its own example and Google's
|
||||
#: generator writes it, while this library's segmented profiles write
|
||||
#: `- [Title](file.md)`. Measured, the star row parsed as `None` -- curated
|
||||
#: prose -- so the section 9.2 index walk could not reach a single concept
|
||||
#: behind one. A bundle we cannot walk is the silent loss the "arbitrary
|
||||
#: bundle" direction forbids.
|
||||
#:
|
||||
#: Reading a form is NOT a licence to emit it. That asymmetry is this
|
||||
#: repository's existing posture, not a new one: `sources` is read in both
|
||||
#: YAML forms and written in one, for the same reason -- the emitted shape
|
||||
#: is what our own parser must round-trip.
|
||||
#:
|
||||
#: Every member must be anchored at both ends, for the reason
|
||||
#: `_faceted_pattern` states: an unanchored alternative would match a
|
||||
#: target mentioned inside curated prose.
|
||||
also_reads: tuple[re.Pattern[str], ...] = ()
|
||||
sort_key: str | None = None
|
||||
sort_order: str = SORT_ASCENDING
|
||||
sort_missing: str = SORT_MISSING_LAST
|
||||
|
|
@ -591,6 +612,13 @@ class IndexPolicy:
|
|||
+ f"(?:{re.escape(self.facets.separator)}(?P<facets>.+))?$"
|
||||
),
|
||||
)
|
||||
for extra in self.also_reads:
|
||||
if not extra.pattern.startswith("^") or not extra.pattern.endswith("$"):
|
||||
raise ValueError(
|
||||
f"an also-read pattern must be anchored at both ends, got "
|
||||
f"{extra.pattern!r} — an unanchored alternative would read a "
|
||||
"target mentioned inside curated prose as an entry"
|
||||
)
|
||||
stray = sorted(self.root_frontmatter_required - set(self.root_frontmatter))
|
||||
if stray:
|
||||
raise ValueError(
|
||||
|
|
@ -672,8 +700,18 @@ class IndexPolicy:
|
|||
Anything this returns `None` for is curated content and survives
|
||||
verbatim: the index is the one file where this library writes beside
|
||||
somebody else's prose.
|
||||
|
||||
`also_reads` is tried only after the emitted form misses, and only
|
||||
here: index MAINTENANCE keys on `entry_pattern` alone, so a foreign row
|
||||
this reads is never a row this rewrites.
|
||||
"""
|
||||
match = self.entry_pattern.match(line.rstrip("\r\n"))
|
||||
stripped = line.rstrip("\r\n")
|
||||
match = self.entry_pattern.match(stripped)
|
||||
if match is None:
|
||||
for extra in self.also_reads:
|
||||
match = extra.match(stripped)
|
||||
if match is not None:
|
||||
break
|
||||
if match is None:
|
||||
return None
|
||||
groups = match.groupdict()
|
||||
|
|
@ -1265,7 +1303,22 @@ SEGMENTED_V1 = BundleProfile(
|
|||
types=STRUCTURED_V1.types,
|
||||
frontmatter=STRUCTURED_V1.frontmatter,
|
||||
paths=STRUCTURED_V1.paths,
|
||||
index=replace(STRUCTURED_V1.index, per_directory=True, root_frontmatter=("bundle_id",)),
|
||||
index=replace(
|
||||
STRUCTURED_V1.index,
|
||||
per_directory=True,
|
||||
root_frontmatter=("bundle_id",),
|
||||
# SPEC section 8's own row form, read and never written. See
|
||||
# `IndexPolicy.also_reads`. Set on the SEGMENTED profiles alone:
|
||||
# `DEFAULT` states commons' spec and `STRICT_V1` the wiki's ratified
|
||||
# contract, and widening either from here would be this repository
|
||||
# editing another repository's contract (O2).
|
||||
also_reads=(
|
||||
re.compile(
|
||||
r"^\* \[(?P<label>[^\]]*)\]\((?P<target>[^)\s]+)\)"
|
||||
r"(?: - (?P<description>.+))?$"
|
||||
),
|
||||
),
|
||||
),
|
||||
ownership=STRUCTURED_V1.ownership,
|
||||
segmentation=SegmentationPolicy(),
|
||||
)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue