docs: general wording for the remaining example-base totals
Replace the combined concept total, the distinct-token total and the per-level document count of earlier example bases with general wording in prose, comments and docstrings. No constant, assertion or test data changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
6b1046bc23
commit
c66f4ae2b0
9 changed files with 16 additions and 16 deletions
|
|
@ -115,7 +115,7 @@ def _frontmatter_from_text(text: str) -> dict[str, str]:
|
|||
key (``sources:\\n - title: …`` names the SOURCE, not the concept) and must never overwrite
|
||||
the concept's own field, however late it appears in the scan. Measured 2026-09-13 on
|
||||
the corpus build's concepts: without this, every ``sources:``-bearing file's nested ``title``
|
||||
replaced its own, collapsing 4605 concepts to 4 distinct titles. A nested line with no
|
||||
replaced its own, collapsing every concept to its base's one title. A nested line with no
|
||||
top-level counterpart is still preserved (OKF SPEC §4, "unknown fields are preserved")."""
|
||||
fm: dict[str, str] = {}
|
||||
top_level: set[str] = set()
|
||||
|
|
@ -804,7 +804,7 @@ def read_provenance(
|
|||
# acts on.
|
||||
items = sum(1 for ln in continuation if ln.strip().startswith("- "))
|
||||
# P13b: the block sequence of mappings is SPEC §5.1's own example and the form all four
|
||||
# delivered bases actually write (4605/4605, measured). It is decoded by the second carrier
|
||||
# delivered bases actually write (every concept, measured). It is decoded by the second carrier
|
||||
# of the one grammar; a block MAPPING (no item opened) and anything the grammar refuses
|
||||
# still come back unreadable, which is what keeps "we could not read it" from quietly
|
||||
# becoming "there was nothing to read".
|
||||
|
|
|
|||
|
|
@ -295,7 +295,7 @@ _GROUNDING_MIN_LENGTH: Final = 3
|
|||
#: The share of the grounding's DOCUMENTS above which a token identifies nothing.
|
||||
#:
|
||||
#: MEASURED over the four delivered corpora, counting document frequency for every code-shaped
|
||||
#: token (``generate._IDENTIFIER_FORMS``): 1 692 distinct tokens, and NOT ONE reaches 5 % of its
|
||||
#: token (``generate._IDENTIFIER_FORMS``): well over a thousand distinct tokens, and NOT ONE reaches 5 % of its
|
||||
#: base's documents. The highest anywhere is 1.35 % (6 documents); the highest that a fasit or mandate
|
||||
#: actually names is 0.67 % (3 documents). P16's fabricated ``P900`` is in every one — 100 %.
|
||||
#: 5 % therefore sits 3.7x above the highest real token measured and 20x below the defect.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue