test(errors): reunite the segmentation codes with the conformance suite
The registry file says one test per code and that this file IS the conformance suite. During the 1-to-N voyage the byte-stability pin over tests/ was the verification mechanism, so editing this file was forbidden and six new codes were covered in the new segmentation modules instead. That was right then. The end-to-end gate run released the pin, so the convention applies again -- a conformance suite split across two files stops being one quietly, which is why this was carried as an obligation rather than a preference. All seven segmentation codes now have a test here, including segmentation_plan_unmatched from this session. The behavioural tests in test_segmentation.py and test_segmented_inbox.py stay where they are: they exercise the parser and the door, a different question from whether every documented code has a raise site. Measured after: 49 documented codes, 48 covered in this file. The one gap, source_reference_unquotable, predates this work and is tested in tests/test_okf_v0_2_profile.py:351 -- reported, not silently closed. Also records both defects and this migration as closed in the decision record's known-gaps list. Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
parent
f65f5fc342
commit
ac6dffe51e
2 changed files with 152 additions and 12 deletions
|
|
@ -118,23 +118,43 @@ hidden: a consumer meets both mechanisms.
|
||||||
|
|
||||||
## Known gaps, stated rather than left to be discovered
|
## Known gaps, stated rather than left to be discovered
|
||||||
|
|
||||||
1. **`derived` marks a declared title as inferred.** A segmented concept's
|
1. ~~**`derived` marks a declared title as inferred.**~~ **Closed 2026-09-01.**
|
||||||
`title` is DECLARED by the adjudicator, but structure derivation runs over
|
A segmented concept's `title` is DECLARED by the adjudicator, but structure
|
||||||
the segment body and adds `title` to `derived`. The marker is therefore
|
derivation ran over the segment body and added `title` to `derived`. The
|
||||||
wrong in the one direction this repo cares about — a consumer that distrusts
|
marker pointed the wrong way in the one direction this repo cares about — a
|
||||||
derived fields will distrust a fact the producer stated. The golden pins
|
consumer that distrusts derived fields would have distrusted a fact the
|
||||||
today's behaviour; the fix belongs with the code that composes the two.
|
producer stated. `title` is now dropped from `derived` when a segment is
|
||||||
2. **A plan matching no dropped file is silent.** A typo'd `source_sha256`
|
present, and only then. The fix is scoped to `title` alone and pinned that
|
||||||
segments nothing and reports nothing, which is the silent skip this library
|
way: an inferred `number` still marks itself on a segment, because nothing
|
||||||
refuses elsewhere.
|
about segmentation makes a derived document number declared, and blunting
|
||||||
|
the marker would be the opposite defect. The `SEGMENTED_V1` golden moved
|
||||||
|
with it; the four others were measured byte-identical to baseline.
|
||||||
|
2. ~~**A plan matching no dropped file is silent.**~~ **Closed 2026-09-01.**
|
||||||
|
A typo'd `source_sha256` segmented nothing and reported nothing, which is
|
||||||
|
the silent skip this library refuses elsewhere. `process_inbox` now refuses
|
||||||
|
fail-fast with `segmentation_plan_unmatched` before any disk mutation. Two
|
||||||
|
sharper questions had to be settled to make the refusal correct rather than
|
||||||
|
merely loud. It asks whether a covering plan was **found**, not whether
|
||||||
|
every file was examined, so an unreadable drop cannot suppress it. And
|
||||||
|
coverage is recorded at **selection**, not after path validation: the first
|
||||||
|
cut asked the later question, and a plan whose hash matched but whose entry
|
||||||
|
paths were then refused looked identical to a plan that matched nothing —
|
||||||
|
the operator would have been sent to check a hash that was already right.
|
||||||
|
An existing collision test caught it; the case now has its own test,
|
||||||
|
verified red against the earlier form.
|
||||||
3. **`status: deprecated` has no producer.** `status` is named as a facet key
|
3. **`status: deprecated` has no producer.** `status` is named as a facet key
|
||||||
and nothing writes it, so a deprecated parent cannot carry one.
|
and nothing writes it, so a deprecated parent cannot carry one.
|
||||||
4. **An unchanged round still rewrites.** `write_bytes` writes
|
4. **An unchanged round still rewrites.** `write_bytes` writes
|
||||||
unconditionally; the bytes are identical, so no contract is broken, but
|
unconditionally; the bytes are identical, so no contract is broken, but
|
||||||
"an unchanged round writes nothing" is not true today.
|
"an unchanged round writes nothing" is not true today.
|
||||||
5. **Six new error codes live in `tests/test_segmentation.py`, not
|
5. ~~**Six new error codes live in `tests/test_segmentation.py`, not
|
||||||
`tests/test_error_codes.py`.** The baseline pin forbade editing the
|
`tests/test_error_codes.py`.**~~ **Closed 2026-09-01.** The baseline pin
|
||||||
registry file. Migrate once the pin is released.
|
forbade editing the registry file while it was the voyage's verification
|
||||||
|
mechanism. The gate run above released it, so the registry convention — one
|
||||||
|
test per code, this file IS the conformance suite — applies again, and the
|
||||||
|
codes were migrated. A conformance suite split across two files stops being
|
||||||
|
one quietly, which is why this was carried as an obligation rather than a
|
||||||
|
preference.
|
||||||
|
|
||||||
## The end-to-end gate run, and its one failure
|
## The end-to-end gate run, and its one failure
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -8,6 +8,7 @@ conformance suite.
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
import importlib.util
|
import importlib.util
|
||||||
import json
|
import json
|
||||||
import sqlite3
|
import sqlite3
|
||||||
|
|
@ -26,6 +27,7 @@ from llm_ingestion_okf.errors import (
|
||||||
MaterializationError,
|
MaterializationError,
|
||||||
NetworkGateError,
|
NetworkGateError,
|
||||||
RenderError,
|
RenderError,
|
||||||
|
SegmentationError,
|
||||||
SourceError,
|
SourceError,
|
||||||
)
|
)
|
||||||
from llm_ingestion_okf.extract import extract_text
|
from llm_ingestion_okf.extract import extract_text
|
||||||
|
|
@ -45,7 +47,9 @@ from llm_ingestion_okf.inbox import (
|
||||||
)
|
)
|
||||||
from llm_ingestion_okf.manifest import load_manifest, load_manifest_bytes
|
from llm_ingestion_okf.manifest import load_manifest, load_manifest_bytes
|
||||||
from llm_ingestion_okf.materialize import materialize_bundle
|
from llm_ingestion_okf.materialize import materialize_bundle
|
||||||
|
from llm_ingestion_okf.profiles import DEFAULT, SEGMENTED_V1
|
||||||
from llm_ingestion_okf.render import sql_value_to_text
|
from llm_ingestion_okf.render import sql_value_to_text
|
||||||
|
from llm_ingestion_okf.segmentation import assert_plan_applies, parse_segmentation_plan
|
||||||
|
|
||||||
INGESTED_AT = "2026-07-17T12:00:00Z"
|
INGESTED_AT = "2026-07-17T12:00:00Z"
|
||||||
|
|
||||||
|
|
@ -526,3 +530,119 @@ def test_network_opt_in_missing(tmp_path: Path) -> None:
|
||||||
with pytest.raises(NetworkGateError) as excinfo:
|
with pytest.raises(NetworkGateError) as excinfo:
|
||||||
materialize_bundle(src / "manifest.json", tmp_path / "bundle", INGESTED_AT)
|
materialize_bundle(src / "manifest.json", tmp_path / "bundle", INGESTED_AT)
|
||||||
assert code_of(excinfo) == "network_opt_in_missing"
|
assert code_of(excinfo) == "network_opt_in_missing"
|
||||||
|
|
||||||
|
|
||||||
|
# --- SegmentationError codes ---
|
||||||
|
#
|
||||||
|
# These arrived during the 1-to-N voyage, when the byte-stability pin over
|
||||||
|
# `tests/` was the verification mechanism and editing this file would have
|
||||||
|
# broken it, so they were covered in the new segmentation modules instead. The
|
||||||
|
# pin was released by the end-to-end gate run of 2026-09-01, and the registry
|
||||||
|
# convention above applies again: one test per code, and this file is where a
|
||||||
|
# reader confirms the registry is complete. The behavioural tests in
|
||||||
|
# `tests/test_segmentation.py` and `tests/test_segmented_inbox.py` stay where
|
||||||
|
# they are -- they exercise the parser and the door, which is a different
|
||||||
|
# question from whether every documented code has a raise site.
|
||||||
|
|
||||||
|
|
||||||
|
def segmentation_entry(**overrides: Any) -> dict[str, Any]:
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"segment_id": "s1",
|
||||||
|
"path": "krav/3-1/brannkonsept.md",
|
||||||
|
"title": "Brannkonsept",
|
||||||
|
"okf_type": "requirement",
|
||||||
|
"span": [0, 40],
|
||||||
|
"ingested_at": INGESTED_AT,
|
||||||
|
}
|
||||||
|
payload.update(overrides)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def segmentation_payload(**overrides: Any) -> dict[str, Any]:
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"version": "1",
|
||||||
|
"source_sha256": "a" * 64,
|
||||||
|
"extractor_id": "text",
|
||||||
|
"extractor_version": "1.0.0",
|
||||||
|
"adjudicated_at": INGESTED_AT,
|
||||||
|
"entries": [segmentation_entry()],
|
||||||
|
}
|
||||||
|
payload.update(overrides)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def parse_segmentation_fails(payload: dict[str, Any]) -> str:
|
||||||
|
with pytest.raises(SegmentationError) as excinfo:
|
||||||
|
parse_segmentation_plan(payload)
|
||||||
|
return excinfo.value.code
|
||||||
|
|
||||||
|
|
||||||
|
def run_segmented_inbox(tmp_path: Path, *, plan_overrides: dict[str, Any], **kwargs: Any) -> None:
|
||||||
|
inbox = tmp_path / "round"
|
||||||
|
inbox.mkdir(parents=True, exist_ok=True)
|
||||||
|
body = "Brannkonsept: krav til seksjonering.\n"
|
||||||
|
(inbox / "n500.md").write_text(body, encoding="utf-8", newline="")
|
||||||
|
payload = segmentation_payload(
|
||||||
|
source_sha256=hashlib.sha256(body.encode("utf-8")).hexdigest(),
|
||||||
|
entries=[segmentation_entry(span=[0, len(body)])],
|
||||||
|
)
|
||||||
|
payload.update(plan_overrides)
|
||||||
|
process_inbox(
|
||||||
|
inbox,
|
||||||
|
tmp_path / "bundle",
|
||||||
|
INGESTED_AT,
|
||||||
|
okf_type="requirement",
|
||||||
|
gate=lambda text: GateDecision(sanitized_text=text, disposition="warn"),
|
||||||
|
segmentation=parse_segmentation_plan(payload),
|
||||||
|
**kwargs,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_segmentation_plan_invalid() -> None:
|
||||||
|
assert parse_segmentation_fails(segmentation_payload(entries=[])) == "segmentation_plan_invalid"
|
||||||
|
|
||||||
|
|
||||||
|
def test_segmentation_path_invalid() -> None:
|
||||||
|
payload = segmentation_payload(entries=[segmentation_entry(path="/krav/absolutt.md")])
|
||||||
|
assert parse_segmentation_fails(payload) == "segmentation_path_invalid"
|
||||||
|
|
||||||
|
|
||||||
|
def test_segmentation_span_invalid() -> None:
|
||||||
|
payload = segmentation_payload(entries=[segmentation_entry(span=[40, 40])])
|
||||||
|
assert parse_segmentation_fails(payload) == "segmentation_span_invalid"
|
||||||
|
|
||||||
|
|
||||||
|
def test_segmentation_duplicate_id() -> None:
|
||||||
|
payload = segmentation_payload(
|
||||||
|
entries=[segmentation_entry(), segmentation_entry(path="krav/3-2/roemning.md")]
|
||||||
|
)
|
||||||
|
assert parse_segmentation_fails(payload) == "segmentation_duplicate_id"
|
||||||
|
|
||||||
|
|
||||||
|
def test_segmentation_extractor_mismatch() -> None:
|
||||||
|
plan = parse_segmentation_plan(segmentation_payload())
|
||||||
|
with pytest.raises(SegmentationError) as excinfo:
|
||||||
|
assert_plan_applies(
|
||||||
|
plan,
|
||||||
|
source_sha256="b" * 64,
|
||||||
|
extractor_id=plan.extractor_id,
|
||||||
|
extractor_version=plan.extractor_version,
|
||||||
|
)
|
||||||
|
assert code_of(excinfo) == "segmentation_extractor_mismatch"
|
||||||
|
|
||||||
|
|
||||||
|
def test_segmentation_unsupported_profile(tmp_path: Path) -> None:
|
||||||
|
with pytest.raises(SegmentationError) as excinfo:
|
||||||
|
run_segmented_inbox(tmp_path, plan_overrides={}, profile=DEFAULT)
|
||||||
|
assert code_of(excinfo) == "segmentation_unsupported_profile"
|
||||||
|
|
||||||
|
|
||||||
|
def test_segmentation_plan_unmatched(tmp_path: Path) -> None:
|
||||||
|
with pytest.raises(SegmentationError) as excinfo:
|
||||||
|
run_segmented_inbox(
|
||||||
|
tmp_path,
|
||||||
|
plan_overrides={"source_sha256": "0" * 64},
|
||||||
|
profile=SEGMENTED_V1,
|
||||||
|
root_frontmatter_values={"bundle_id": "b-1"},
|
||||||
|
)
|
||||||
|
assert code_of(excinfo) == "segmentation_plan_unmatched"
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue