open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(importer): Door C projects the sender's own facets into the index

Browse source 
vegnormal-okf measured the gap on 2026-08-27: the arm reading DEFAULT's
index.md scored 0 hits of 8, the arm reading a faceted index of the same
frontmatter scored 25 of 29. Same bundle, same concepts, same model. The
DEFAULT arm did not answer wrongly, it abstained -- the metadata is in the
bundle and the index throws it away (30 974 characters over 269
requirements, 0 occurrences of any of the eight facts).

FacetPolicy and STRUCTURED_V1 already did this. They did not reach Door C.

`import_bundle` now takes a keyword-only `profile` defaulting to DEFAULT, so
every existing call site emits the bytes it always did, and `link_in_index`
takes the facets to render.

Door C PROJECTS and never DERIVES, which is the answer to the objection this
work opened with: deriving structure for a document a third party wrote would
put our inference into an index entry ABOUT their bytes, where it reads as
their claim. The concept file was already verbatim; the entry describing it
now is too. Where the sender carries `derived`, THEIR list travels unchanged,
so a reader can still see which of the sender's facts the sender inferred.

The projection asks the policy which keys to carry and never what a key
means. That is what makes it work for a meeting note as well as a numbered
norm -- nothing in it can key off a numbering scheme -- and it is why a
consumer whose concepts are named by UUID can get `title` into the index by
naming the key, with no change here.

Two things measured during the work rather than assumed:

- A value carrying the policy's own joiner cannot be rendered. Door C's
  tolerance is structural and it refuses no sender on form, so the FACET is
  dropped and the concept still merges -- reported per concept and key in
  `ImportResult.unrendered_facets`, never dropped silently.
- A faceted entry can go stale where a flat one never could: the collision
  gate refuses an updated concept, so the operator's only route is to remove
  the merged file and re-import, after which the file said `gjeldende` while
  the index still said `utkast`. A faceted entry for a present target is now
  refreshed in place instead of skipped. Unfaceted callers keep the early
  return byte for byte.

Suite 695 -> 707; ruff and mypy --strict clean.

Order 20260826T224500Z-873805419-from-vegnormal-okf.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-27 10:58:33 +02:00
commit 1f7d3502b8
3 changed files with 491 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

131
src/llm_ingestion_okf/importer.py
View file

@ -27,6 +27,7 @@ obeys the verdict it returns.
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass
from pathlib import Path
from typing import Protocol
@ -41,7 +42,7 @@ from .materialize import (
validate_ingested_at,
write_bytes,
)
from .profiles import DEFAULT
from .profiles import DEFAULT, BundleProfile, FacetPolicy
# An OKF concept is a `.md` document by definition — the guard's path gate
# rejects anything else outright — so nothing else in the source tree is a
@ -183,6 +184,30 @@ class UnverifiedReference:
key: str
@dataclass(frozen=True)
class UnrenderedFacet:
"""A merged concept declares a facet value the index policy cannot render.
The policy refuses a value carrying its own separator or joiner rather than
escaping it (escaping makes the line parse one way here and another way
downstream). At Door B that refuses the DOCUMENT, because the value is one
this library derived and the operator can fix the source. At Door C it must
not: this door judges no shape and refuses no sender on form — the whole
reason it writes concepts verbatim — so refusing a merge over a semicolon in
someone else's frontmatter is precisely the failure the module docstring
names.
So the FACET is dropped and the CONCEPT is merged. Dropping it silently is
the other failure: the sender made a claim our index does not show, and a
reader comparing the two would find no trace of why. Reported, like an
unverified pointer — an advisory over the merged set, never a fifth bucket.
"""
concept_path: str
key: str
reason: str
@dataclass(frozen=True)
class ImportResult:
"""Every concept's outcome, in sorted concept-path order.
@ -193,9 +218,9 @@ class ImportResult:
the guard's log body and `ingested_at` the run's explicit timestamp — the
caller persists them if their reserved-file policy says to.
`unverified_references` is not a fifth bucket and does not partition
anything: every concept it names has already merged. It is an advisory over
the merged set.
`unverified_references` and `unrendered_facets` are not fifth and sixth
buckets and partition nothing: every concept they name has already merged.
Both are advisories over the merged set.
"""
merged: tuple[MergedConcept, ...]
@ -205,9 +230,10 @@ class ImportResult:
log: str
ingested_at: str
unverified_references: tuple[UnverifiedReference, ...] = ()
unrendered_facets: tuple[UnrenderedFacet, ...] = ()
def import_slug(concept_path: str) -> str:
def import_slug(concept_path: str, *, profile: BundleProfile = DEFAULT) -> str:
"""Reduce a bundle-relative concept path to the Phase 1 id grammar.
The whole path reduces, not just its final segment: `tables/users.md` and
@ -215,7 +241,7 @@ def import_slug(concept_path: str) -> str:
collapse them onto one filename. A path that reduces to nothing fails fast
rather than being given an invented name.
"""
concept_id = concept_path[: -len(DEFAULT.paths.concept_suffix)]
concept_id = concept_path[: -len(profile.paths.concept_suffix)]
slug = reduce_to_id_grammar(concept_id)
if not slug:
raise MaterializationError(
@ -226,7 +252,7 @@ def import_slug(concept_path: str) -> str:
return slug
def import_filename(slug: str) -> str:
def import_filename(slug: str, *, profile: BundleProfile = DEFAULT) -> str:
"""The bundle filename for an imported concept.
The `import-` prefix keeps the namespace disjoint from `index.md`, Door A's
@ -234,19 +260,19 @@ def import_filename(slug: str) -> str:
grammar admits.
"""
return check_filename_length(
f"{DEFAULT.paths.import_prefix}{slug}{DEFAULT.paths.concept_suffix}",
f"{profile.paths.import_prefix}{slug}{profile.paths.concept_suffix}",
code="import_path_too_long",
)
def _index_label(concept_path: str) -> str:
def _index_label(concept_path: str, *, profile: BundleProfile = DEFAULT) -> str:
"""The concept-ID, validated as an index link label.
The guard's path gate permits brackets in a concept path; `- [label](target)`
does not. Fail-fast, never repair — the same rule Door A applies to a
manifest title and Door B to a dropped filename.
"""
label = concept_path[: -len(DEFAULT.paths.concept_suffix)]
label = concept_path[: -len(profile.paths.concept_suffix)]
if any(char in label for char in "\n\r[]"):
raise MaterializationError(
f"concept path {concept_path!r} contains '[' or ']', which would break "
@ -256,7 +282,47 @@ def _index_label(concept_path: str) -> str:
return label
def _read_bundle(source: Path) -> tuple[dict[str, str], list[FailedConcept]]:
def _project_facets(
frontmatter: Mapping[str, str], policy: FacetPolicy
) -> tuple[dict[str, str], list[tuple[str, str]]]:
"""The sender's declared values for the keys the policy names, and the drops.
A PROJECTION and never a derivation. Every value here is one the sender
wrote in their own frontmatter; nothing is inferred from their body, their
filename, or their neighbours in the bundle. That is the ownership answer at
this door: the concept file is verbatim, and so is the index entry's account
of what the concept claims. Where the sender carries `derived`, THEIR list
travels unchanged, so a reader can still tell which of the sender's facts
the sender inferred — a distinction this library would erase by adding
inferences of its own beside them.
The loop asks the policy which keys to carry and never what a key means.
That is what makes the door work for a meeting note as well as a numbered
norm: nothing here can key off a numbering scheme, because nothing here
reads a value at all except to check the policy can render it.
Each key is rendered ALONE to find the offender, because the policy reports
a refusal for the entry rather than for one field, and dropping the whole
tail over one bad value would lose the other facts the sender declared.
"""
values: dict[str, str] = {}
dropped: list[tuple[str, str]] = []
for key in policy.keys:
value = frontmatter.get(key)
if not value:
continue
try:
policy.render({key: value})
except ValueError as exc:
dropped.append((key, str(exc)))
continue
values[key] = value
return values, dropped
def _read_bundle(
source: Path, *, profile: BundleProfile = DEFAULT
) -> tuple[dict[str, str], list[FailedConcept]]:
"""Read every concept document under `source`, keyed by POSIX-relative path.
Unreadable and undecodable concepts never reach the gate — they are per-
@ -269,7 +335,7 @@ def _read_bundle(source: Path) -> tuple[dict[str, str], list[FailedConcept]]:
# glob: glob case-sensitivity follows the FILESYSTEM, so `NOTE.MD`
# would be a concept on APFS and not one on ext4 — the same bundle
# importing differently per platform. The guard folds case here too.
if not path.is_file() or path.suffix.lower() != DEFAULT.paths.concept_suffix:
if not path.is_file() or path.suffix.lower() != profile.paths.concept_suffix:
continue
concept_path = path.relative_to(source).as_posix()
try:
@ -296,6 +362,7 @@ def import_bundle(
origin: str,
channel: str,
gate: ImportGate,
profile: BundleProfile = DEFAULT,
) -> ImportResult:
"""Merge the accepted concepts of an external OKF bundle (Door C).
@ -307,6 +374,14 @@ def import_bundle(
INCLUDING a disposition this library does not recognise and a concept the
gate returned no verdict for, fails closed.
`profile` names the filename namespace this door writes into and the shape
of the index it maintains. It is keyword-only and defaults to `DEFAULT`, so
every existing call site emits the bytes it always did — a consumer with
branch bases built through this door is not asked to rebuild them. Where the
profile's index carries facets, each merged concept's OWN frontmatter is
projected onto them; see :func:`_project_facets` for why this door projects
and never derives.
One bad concept never aborts the run. Unreadable concepts, unusable names
and collisions are reported per concept in :class:`ImportResult` while the
rest still merge. Only three conditions fail the whole run, and all three
@ -328,12 +403,13 @@ def import_bundle(
f"source bundle directory does not exist: {source}", code="source_root_missing"
)
documents, failed = _read_bundle(source)
documents, failed = _read_bundle(source, profile=profile)
merged: list[MergedConcept] = []
quarantined: list[RefusedConcept] = []
rejected: list[RefusedConcept] = []
unverified: list[UnverifiedReference] = []
unrendered: list[UnrenderedFacet] = []
log = ""
if documents:
@ -378,8 +454,8 @@ def import_bundle(
slug_owners: dict[str, list[str]] = {}
for concept_path, text in accepted:
try:
name = import_filename(import_slug(concept_path))
_index_label(concept_path)
name = import_filename(import_slug(concept_path, profile=profile), profile=profile)
_index_label(concept_path, profile=profile)
except IngestError as exc:
failed.append(FailedConcept(concept_path=concept_path, error=exc))
continue
@ -463,11 +539,29 @@ def import_bundle(
# §6 index — the last disk mutation, and only when something merged.
if merged:
index_path = bundle / DEFAULT.index.name
index_path = bundle / profile.index.name
if not index_path.is_file():
write_bytes(bundle, DEFAULT.index.name, "")
write_bytes(bundle, profile.index.name, "")
for entry in merged:
link_in_index(bundle, entry.path.name, _index_label(entry.concept_path))
facets: dict[str, str] | None = None
if profile.index.facets is not None:
# Read from the WRITTEN file, like the pointer scan above,
# so the entry describes the concept that exists rather than
# the text that was staged.
facets, dropped = _project_facets(
parse_frontmatter(entry.path), profile.index.facets
)
unrendered.extend(
UnrenderedFacet(concept_path=entry.concept_path, key=key, reason=reason)
for key, reason in dropped
)
link_in_index(
bundle,
entry.path.name,
_index_label(entry.concept_path, profile=profile),
profile=profile,
facets=facets,
)
return ImportResult(
merged=tuple(merged),
@ -477,4 +571,5 @@ def import_bundle(
log=log,
ingested_at=ingested_at,
unverified_references=tuple(unverified),
unrendered_facets=tuple(unrendered),
)

62
src/llm_ingestion_okf/materialize.py
View file

@ -286,26 +286,78 @@ def _update_index_lines(
index_path.write_bytes("".join(updated).encode("utf-8"))
def _refresh_index_entry(
index_path: Path,
target_name: str,
label: str,
facets: Mapping[str, str],
*,
profile: BundleProfile,
) -> None:
"""Re-render the managed line for `target_name`, in place and alone.
Keyed on the policy's entry pattern and on the parsed target, never on a
substring: the index is the one file this library writes beside somebody
else's prose, so a curated line that merely MENTIONS the target has to
survive verbatim, and so does its own line ending.
"""
lines = index_path.read_bytes().decode("utf-8").splitlines(keepends=True)
updated: list[str] = []
changed = False
for line in lines:
content = line.rstrip("\r\n")
ending = line[len(content) :]
match = profile.index.entry_pattern.match(content)
if match is not None and match.group("target") == target_name:
refreshed = profile.index.render_link(label, target_name, facets=facets)
if refreshed != content:
line = refreshed + ending
changed = True
updated.append(line)
if changed:
index_path.write_bytes("".join(updated).encode("utf-8"))
def link_in_index(
bundle_dir: Path, target_name: str, label: str, *, profile: BundleProfile = DEFAULT
bundle_dir: Path,
target_name: str,
label: str,
*,
profile: BundleProfile = DEFAULT,
facets: Mapping[str, str] | None = None,
) -> None:
# §6: idempotent by target — a link whose target is already present in
# the index is never added twice.
#
# `profile` is keyword-only with a default because this function is public
# and called from all three doors (A here, B in inbox.py, C in importer.py).
# Doors B and C keep the default, which is the behaviour they already had;
# which profile THEY should own is a separate question, and answering it by
# changing this signature would have decided it silently.
# Door A and Door B's unfaceted path keep the default; which profile a
# caller should own is a separate question, and answering it by changing
# this signature would have decided it silently.
#
# `facets` also decides what "already present" MEANS, and the split is not a
# convenience. A flat entry carries a label and a target, both stable, so it
# can never disagree with the file it points at and returning early is
# exactly right — a hand-edited label survives. An entry carrying the
# concept's FACTS can go stale, and an index that contradicts the bundle it
# indexes is worse than one that says nothing: the consumer reads the index
# and stops there. So a faceted entry for a target already present is
# REFRESHED in place rather than skipped.
#
# Additive by construction: with `facets=None` nothing below the early
# return runs, so every unfaceted caller emits the bytes it always did.
index_path = safe_resolve(bundle_dir, profile.index.name)
body = index_path.read_bytes().decode("utf-8")
if f"]({target_name})" in body:
if facets is None:
return
_refresh_index_entry(index_path, target_name, label, facets, profile=profile)
return
# An empty index needs no separator: Door A always seeds its index with
# bundle_summary first, but Door B has no summary to invent, so its index
# starts empty and must not open with a blank line.
prefix = body if (body == "" or body.endswith("\n")) else body + "\n"
line = profile.index.render_link(label, target_name)
line = profile.index.render_link(label, target_name, facets=facets)
index_path.write_bytes(f"{prefix}{line}\n".encode())

321
tests/test_import_facets.py Normal file
View file

@ -0,0 +1,321 @@
"""Door C carries the sender's OWN metadata into the index — and nothing else.
The measured defect this closes, as data. vegnormal-okf ran a pre-registered
reasoning bake-off on 2026-08-27 over the same bundle through two index shapes:
the arm reading DEFAULT's `index.md` scored 0 hits of 8, the arm reading a
faceted index of the same frontmatter scored 25 of 29. The DEFAULT arm did not
answer wrongly — it abstained, because the index it was given is a link list
and the metadata never reaches it. Measured on their `index.md`: 30 974
characters over 269 requirements, 0 occurrences of any of the eight facts their
concepts carry.
The mechanism already existed (`FacetPolicy`, `STRUCTURED_V1`); it simply did
not reach Door C's import path.
WHY THIS DOOR NEEDS ITS OWN ANSWER, and not Door B's. Door B DERIVES structure
from documents this library's own operator dropped. Door C merges documents a
THIRD PARTY wrote, verbatim, and the whole door is built on refusing to put
anything of ours inside their bytes. Deriving structure for a sender and
publishing it in our index would break that guarantee in the one place nobody
would look for it: not in the concept file, which stays verbatim, but in the
index entry ABOUT the concept, where our inference would read as their claim.
So Door C PROJECTS and never DERIVES. Every facet in a Door C index entry is a
value the sender declared in their own frontmatter, copied. The projection is
key-agnostic by construction — it asks the policy which keys to carry and never
what a key means — which is also why it does not care whether a document is a
numbered norm or a meeting note.
"""
from __future__ import annotations
from dataclasses import replace
from pathlib import Path
from test_import_flow import EXTERNAL, INGESTED_AT, AUTOMATIC, StubImportGate, place
from llm_ingestion_okf.importer import ImportResult, import_bundle
from llm_ingestion_okf.profiles import DEFAULT, STRUCTURED_V1, BundleProfile, FacetPolicy
# A profile whose facet policy also names `title`. Door C's index LABEL is the
# concept path, which for a sender naming files by UUID carries no title at all
# — so the title has to travel as a facet or not at all. Built here rather than
# added to `STRUCTURED_V1` because that profile is Door B's shipped contract and
# widening its facet set would move Door B's index bytes.
TITLED = replace(
STRUCTURED_V1,
index=replace(
STRUCTURED_V1.index,
facets=FacetPolicy(keys=("title", *STRUCTURED_V1.index.facets.keys)),
),
)
def run_with(
tmp_path: Path,
gate: StubImportGate,
*,
profile: BundleProfile,
) -> tuple[ImportResult, Path]:
bundle = tmp_path / "bundle"
result = import_bundle(
tmp_path / "source",
bundle,
INGESTED_AT,
origin=EXTERNAL,
channel=AUTOMATIC,
gate=gate,
profile=profile,
)
return result, bundle
def index_of(bundle: Path, profile: BundleProfile) -> str:
return (bundle / profile.index.name).read_text(encoding="utf-8")
# --- the projection -------------------------------------------------------
def test_door_c_carries_the_senders_declared_facets_into_the_index(tmp_path: Path) -> None:
# The 0/8 case, in one document: the sender declares the facts, and the
# index the reasoning arm is handed now states them.
place(
tmp_path / "source",
"krav/n500-3-1.md",
"---\ntype: dataset\nnumber: N500\nstatus: gjeldende\ndate: 2024-06-01\n---\n\nBody.\n",
)
_, bundle = run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
assert index_of(bundle, STRUCTURED_V1) == (
"- [krav/n500-3-1](import-krav-n500-3-1.md)"
" — number: N500; status: gjeldende; date: 2024-06-01\n"
)
def test_a_facet_key_the_policy_names_is_projected_whatever_it_means(tmp_path: Path) -> None:
# The projection asks the POLICY which keys to carry. It has no opinion
# about any particular key, which is what lets a consumer whose concepts are
# named by UUID get the title into the index without a change here.
place(
tmp_path / "source",
"0f9a.md",
"---\ntype: dataset\ntitle: Vegtunneler\nnumber: N500\n---\n\nBody.\n",
)
_, bundle = run_with(tmp_path, StubImportGate(), profile=TITLED)
assert index_of(bundle, TITLED) == (
"- [0f9a](import-0f9a.md) — title: Vegtunneler; number: N500\n"
)
# --- what the door refuses to claim ---------------------------------------
def test_door_c_derives_nothing_the_sender_did_not_declare(tmp_path: Path) -> None:
# This body is exactly what Door B's deriver reads a number and a title off.
# Door C must read neither: an inference of ours, printed in an index entry
# about someone else's document, reads as their claim. The concept file is
# verbatim either way — the attribution is what would have been forged.
place(
tmp_path / "source",
"b.md",
"---\ntype: dataset\nstatus: gjeldende\n---\n\n# N500 Vegtunneler\n\nSe N100.\n",
)
_, bundle = run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
index = index_of(bundle, STRUCTURED_V1)
assert index == "- [b](import-b.md) — status: gjeldende\n"
assert "N500" not in index
assert "references" not in index
def test_the_senders_own_derived_list_travels_verbatim(tmp_path: Path) -> None:
# `derived` is the key that says which of the facts before it were INFERRED
# rather than read. When the sender carries one, the index must carry theirs
# unchanged: that is the whole ownership stamp at this door — the reader can
# see which claims are the sender's own inference, and none are ours.
place(
tmp_path / "source",
"c.md",
"---\ntype: dataset\nnumber: N130\nstatus: gjeldende\n"
"derived: [number, status]\n---\n\nBody.\n",
)
_, bundle = run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
assert index_of(bundle, STRUCTURED_V1) == (
"- [c](import-c.md) — number: N130; status: gjeldende; derived: [number, status]\n"
)
# --- content that is not a numbered norm ----------------------------------
def test_an_unnumbered_document_still_gets_the_facets_it_has(tmp_path: Path) -> None:
# Operator directive 2026-08-27: everything built around OKF must work for
# ALL content, so a design that only works for numbered norms is wrong even
# when it scores well on N100/N200/N500. A meeting note has no number, no
# parent and no version — and must still reach the index carrying what it
# does have, with no half-written separator for what it does not.
place(
tmp_path / "source",
"referat-styringsgruppe.md",
"---\ntype: dataset\nstatus: utkast\ndate: 2026-08-27\n---\n\nProse, ingen nummerering.\n",
)
_, bundle = run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
assert index_of(bundle, STRUCTURED_V1) == (
"- [referat-styringsgruppe](import-referat-styringsgruppe.md)"
" — status: utkast; date: 2026-08-27\n"
)
def test_a_document_declaring_none_of_the_facets_renders_the_bare_link(tmp_path: Path) -> None:
# The floor of the same rule: a sender who declares nothing the policy names
# gets the line they would have got without facets at all — never a
# separator with nothing after it.
place(tmp_path / "source", "d.md", "---\ntype: dataset\n---\n\nBody.\n")
_, bundle = run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
assert index_of(bundle, STRUCTURED_V1) == "- [d](import-d.md)\n"
# --- what must not move ---------------------------------------------------
def test_the_default_profile_leaves_door_c_byte_identical(tmp_path: Path) -> None:
# 171 branch bases were built through this door. `profile` defaults to
# DEFAULT, and DEFAULT names no facets, so an existing call site emits the
# bytes it always did — proven against the same frontmatter that WOULD
# produce a facet tail under a faceted profile.
place(
tmp_path / "source",
"e.md",
"---\ntype: dataset\nnumber: N500\nstatus: gjeldende\n---\n\nBody.\n",
)
bundle = tmp_path / "bundle"
import_bundle(
tmp_path / "source",
bundle,
INGESTED_AT,
origin=EXTERNAL,
channel=AUTOMATIC,
gate=StubImportGate(),
)
assert index_of(bundle, DEFAULT) == "- [e](import-e.md)\n"
def test_the_concept_file_is_still_written_verbatim_under_a_faceted_profile(
tmp_path: Path,
) -> None:
# The verbatim guarantee is what makes projection safe, so it is pinned on
# the faceted path too: it is `index.md` that gains bytes, never the file
# the sender wrote.
text = "---\ntype: dataset\nnumber: N500\n---\n\nBody.\n"
place(tmp_path / "source", "f.md", text)
_, bundle = run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
assert (bundle / "import-f.md").read_text(encoding="utf-8") == text
def test_an_unrenderable_facet_value_drops_the_facet_not_the_concept(tmp_path: Path) -> None:
# A value carrying the policy's own joiner cannot be rendered. Door C's
# tolerance is structural — it judges no shape and refuses no sender on form
# — so the concept still merges verbatim. But a claim the sender made that
# our index cannot show is exactly the thing that must not vanish quietly,
# so it is reported per concept and per key, like an unverified pointer.
place(
tmp_path / "source",
"g.md",
"---\ntype: dataset\nnumber: N500\nstatus: utkast; til horing\n---\n\nBody.\n",
)
result, bundle = run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
assert [entry.concept_path for entry in result.merged] == ["g.md"]
assert (bundle / "import-g.md").is_file()
assert index_of(bundle, STRUCTURED_V1) == "- [g](import-g.md) — number: N500\n"
assert [(entry.concept_path, entry.key) for entry in result.unrendered_facets] == [
("g.md", "status")
]
# --- the entry must not outlive the facts it states ------------------------
def test_a_reimported_concept_updates_its_index_entry_rather_than_going_stale(
tmp_path: Path,
) -> None:
# Measured, not assumed. A flat entry carries only a label and a target,
# both stable, so it could never disagree with the file it points at. An
# entry that carries the concept's FACTS can, and this is the path that
# gets there: the collision gate refuses an updated concept outright, so
# the operator's only way to accept an update is to remove the merged file
# (the code's own refusal message says so) and import again. The concept
# then becomes `gjeldende` on disk while the index still says `utkast`.
#
# An index that contradicts the bundle it indexes is worse than one that
# says nothing, because a reasoning arm reads the index and stops.
source = tmp_path / "source"
place(source, "a.md", "---\ntype: dataset\nnumber: N500\nstatus: utkast\n---\n\nBody.\n")
_, bundle = run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
assert index_of(bundle, STRUCTURED_V1) == (
"- [a](import-a.md) — number: N500; status: utkast\n"
)
(bundle / "import-a.md").unlink()
place(source, "a.md", "---\ntype: dataset\nnumber: N500\nstatus: gjeldende\n---\n\nBody.\n")
run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
assert "gjeldende" in (bundle / "import-a.md").read_text(encoding="utf-8")
assert index_of(bundle, STRUCTURED_V1) == (
"- [a](import-a.md) — number: N500; status: gjeldende\n"
)
def test_refreshing_an_entry_leaves_curated_prose_around_it_untouched(tmp_path: Path) -> None:
# The index is the one file this library writes beside somebody else's
# prose. A refresh keyed on the managed pattern must rewrite the one line it
# owns and nothing else — including a line that merely MENTIONS the target.
source = tmp_path / "source"
place(source, "a.md", "---\ntype: dataset\nstatus: utkast\n---\n\nBody.\n")
_, bundle = run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
index_path = bundle / STRUCTURED_V1.index.name
index_path.write_text(
"# Katalog\n\nSe ogsaa import-a.md i teksten.\n\n" + index_path.read_text(encoding="utf-8"),
encoding="utf-8",
)
(bundle / "import-a.md").unlink()
place(source, "a.md", "---\ntype: dataset\nstatus: gjeldende\n---\n\nBody.\n")
run_with(tmp_path, StubImportGate(), profile=STRUCTURED_V1)
assert index_of(bundle, STRUCTURED_V1) == (
"# Katalog\n\nSe ogsaa import-a.md i teksten.\n\n- [a](import-a.md) — status: gjeldende\n"
)
def test_an_unfaceted_profile_keeps_the_early_return_it_always_had(tmp_path: Path) -> None:
# The refresh is scoped to the facets feature by construction: a policy
# naming no facets has nothing that can go stale, so its idempotent-by-
# target behaviour must be byte-identical to what it always was — including
# leaving a hand-edited label alone.
from llm_ingestion_okf.materialize import link_in_index
bundle = tmp_path / "bundle"
bundle.mkdir()
(bundle / "index.md").write_text("- [Hand Edited](import-a.md)\n", encoding="utf-8")
link_in_index(bundle, "import-a.md", "a", profile=DEFAULT)
assert (bundle / "index.md").read_text(encoding="utf-8") == "- [Hand Edited](import-a.md)\n"