feat(okf-v0.2): D4 — door C surfaces the §10 pointers it imports [skip-docs]
V6, settled by the operator today: import and report, not refuse. Door C imports the POINTER to executable code and never the code — it writes concepts verbatim and skips every non-`.md` file. So an imported Attested Computation can name an `executor`/`attester` resource that did not arrive, or one that RESOLVES against a file the destination tree already holds under that path. The second is the outcome worth surfacing: it looks valid. Refusing was the plan's leaning and is not what shipped. §14 forbids a consumer to reject a bundle over a broken cross-link and does not settle whether `executor.resource` is one; §10.5 asks a consumer to surface rather than silently drop. Reporting honours the second without testing the first, and leaves the door's one invariant — verbatim bytes — alone. `ImportResult.unverified_references` is an advisory over the merged set, not a fifth bucket: every concept it names has already merged, the bytes are unchanged, and a refused concept is never named (there is no imported pointer to check). The report is at KEY level, and that is a measured limit rather than a choice. Resolving the resource means reading `executor.resource` — the value the line-oriented parser cannot recover in either canonical form: a block mapping flattens and collides, a flow mapping stays one opaque string. A resource-level report would be empty or wrong on exactly the forms upstream writes. Precision arrives with the structured reader (D1b); the key-level signal is robust in both forms today. [skip-docs] is on the CLAUDE.md half only: README carries the new public surface (`unverified_references`), and the invariant this work put in CLAUDE.md — flow form, never block — landed with the previous commit and needs no restatement. 584 tests, mypy --strict clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KKKMwi7e7PVHoFW6dJK5XP
This commit is contained in:
parent
deeb248091
commit
c6d64c3fd6
4 changed files with 188 additions and 0 deletions
10
README.md
10
README.md
|
|
@ -162,6 +162,16 @@ the question an attestation answers — was this value produced the sanctioned
|
||||||
way — is not this library's. It re-enters scope when upstream specifies the
|
way — is not this library's. It re-enters scope when upstream specifies the
|
||||||
protocol.
|
protocol.
|
||||||
|
|
||||||
|
On the import side, a third-party concept may name an `executor` or `attester`
|
||||||
|
resource pointing at executable code. Door C imports the **pointer** and never
|
||||||
|
the code — it writes concepts verbatim and skips every non-`.md` file — so such
|
||||||
|
a reference may not resolve, or may resolve to a file the destination tree
|
||||||
|
already holds under that path. Each one is reported in
|
||||||
|
`ImportResult.unverified_references`; the concept still merges, because §14
|
||||||
|
forbids rejecting a bundle over a broken cross-link while §10.5 asks a consumer
|
||||||
|
to surface rather than silently drop. The report names the pointer key, not the
|
||||||
|
resource it points at: recovering the resource needs the structured reader.
|
||||||
|
|
||||||
One limit worth knowing before you write such a concept: §10.2 presents
|
One limit worth knowing before you write such a concept: §10.2 presents
|
||||||
`executor` and `attester` as nested block mappings, and this library's
|
`executor` and `attester` as nested block mappings, and this library's
|
||||||
frontmatter parser is line-oriented. It reads inline **flow** mappings
|
frontmatter parser is line-oriented. It reads inline **flow** mappings
|
||||||
|
|
|
||||||
|
|
@ -61,6 +61,7 @@ from .importer import (
|
||||||
ImportResult,
|
ImportResult,
|
||||||
MergedConcept,
|
MergedConcept,
|
||||||
RefusedConcept,
|
RefusedConcept,
|
||||||
|
UnverifiedReference,
|
||||||
import_bundle,
|
import_bundle,
|
||||||
)
|
)
|
||||||
from .manifest import (
|
from .manifest import (
|
||||||
|
|
@ -102,6 +103,7 @@ __all__ = [
|
||||||
"RenderError",
|
"RenderError",
|
||||||
"SourceError",
|
"SourceError",
|
||||||
"SqlSource",
|
"SqlSource",
|
||||||
|
"UnverifiedReference",
|
||||||
"extract_text",
|
"extract_text",
|
||||||
"import_bundle",
|
"import_bundle",
|
||||||
"load_manifest",
|
"load_manifest",
|
||||||
|
|
|
||||||
|
|
@ -36,6 +36,7 @@ from .extract import decode_text
|
||||||
from .materialize import (
|
from .materialize import (
|
||||||
check_filename_length,
|
check_filename_length,
|
||||||
link_in_index,
|
link_in_index,
|
||||||
|
parse_frontmatter,
|
||||||
reduce_to_id_grammar,
|
reduce_to_id_grammar,
|
||||||
validate_ingested_at,
|
validate_ingested_at,
|
||||||
write_bytes,
|
write_bytes,
|
||||||
|
|
@ -57,6 +58,12 @@ from .profiles import DEFAULT
|
||||||
_DISPOSITION_MERGE = "warn"
|
_DISPOSITION_MERGE = "warn"
|
||||||
_DISPOSITION_QUARANTINE = "quarantine_review"
|
_DISPOSITION_QUARANTINE = "quarantine_review"
|
||||||
|
|
||||||
|
# The OKF §10.2 frontmatter keys that point at executable code. Alphabetical
|
||||||
|
# because the report has to be deterministic and these two carry no precedence
|
||||||
|
# over each other — `executor` runs the computation, `attester` checks the
|
||||||
|
# receipt, and a concept naming either has imported a pointer we cannot follow.
|
||||||
|
_ATTESTED_POINTERS = ("attester", "executor")
|
||||||
|
|
||||||
# The guard's Origin/Channel vocabularies, likewise by value. Validated here
|
# The guard's Origin/Channel vocabularies, likewise by value. Validated here
|
||||||
# because `trust_for` compares by enum IDENTITY: a value outside these sets
|
# because `trust_for` compares by enum IDENTITY: a value outside these sets
|
||||||
# would reach the guard as a plain string, miss the identity check, and be
|
# would reach the guard as a plain string, miss the identity check, and be
|
||||||
|
|
@ -147,6 +154,35 @@ class FailedConcept:
|
||||||
error: IngestError
|
error: IngestError
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class UnverifiedReference:
|
||||||
|
"""A merged concept declares an OKF §10 pointer to executable code.
|
||||||
|
|
||||||
|
Door C imports the POINTER and never the code: it writes concepts verbatim
|
||||||
|
and skips every non-`.md` file in the source tree. So an imported Attested
|
||||||
|
Computation can name an `executor` or `attester` resource that did not
|
||||||
|
arrive with it — and if the destination tree already holds, or later gains,
|
||||||
|
a file at that path, the reference does not dangle, it RESOLVES to code the
|
||||||
|
sender never shipped. That second outcome is the one worth surfacing: it
|
||||||
|
looks valid.
|
||||||
|
|
||||||
|
Reported, not refused. §14 forbids a consumer to reject a bundle over a
|
||||||
|
broken cross-link and does not settle whether `executor.resource` is one,
|
||||||
|
while §10.5 asks a consumer to surface rather than silently drop a failing
|
||||||
|
attestation. Reporting honours the second without testing the first.
|
||||||
|
|
||||||
|
`key` names the pointer (`executor` or `attester`); the RESOURCE it names is
|
||||||
|
deliberately absent. Recovering it means reading a value this library's
|
||||||
|
line-oriented parser cannot represent — a nested block mapping is flattened
|
||||||
|
and a flow mapping stays one opaque string — so a resource-level report
|
||||||
|
would be empty or wrong on exactly the canonical forms. It arrives with the
|
||||||
|
structured reader.
|
||||||
|
"""
|
||||||
|
|
||||||
|
concept_path: str
|
||||||
|
key: str
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class ImportResult:
|
class ImportResult:
|
||||||
"""Every concept's outcome, in sorted concept-path order.
|
"""Every concept's outcome, in sorted concept-path order.
|
||||||
|
|
@ -156,6 +192,10 @@ class ImportResult:
|
||||||
would show up as a missing entry rather than as nothing at all. `log` is
|
would show up as a missing entry rather than as nothing at all. `log` is
|
||||||
the guard's log body and `ingested_at` the run's explicit timestamp — the
|
the guard's log body and `ingested_at` the run's explicit timestamp — the
|
||||||
caller persists them if their reserved-file policy says to.
|
caller persists them if their reserved-file policy says to.
|
||||||
|
|
||||||
|
`unverified_references` is not a fifth bucket and does not partition
|
||||||
|
anything: every concept it names has already merged. It is an advisory over
|
||||||
|
the merged set.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
merged: tuple[MergedConcept, ...]
|
merged: tuple[MergedConcept, ...]
|
||||||
|
|
@ -164,6 +204,7 @@ class ImportResult:
|
||||||
failed: tuple[FailedConcept, ...]
|
failed: tuple[FailedConcept, ...]
|
||||||
log: str
|
log: str
|
||||||
ingested_at: str
|
ingested_at: str
|
||||||
|
unverified_references: tuple[UnverifiedReference, ...] = ()
|
||||||
|
|
||||||
|
|
||||||
def import_slug(concept_path: str) -> str:
|
def import_slug(concept_path: str) -> str:
|
||||||
|
|
@ -292,6 +333,7 @@ def import_bundle(
|
||||||
merged: list[MergedConcept] = []
|
merged: list[MergedConcept] = []
|
||||||
quarantined: list[RefusedConcept] = []
|
quarantined: list[RefusedConcept] = []
|
||||||
rejected: list[RefusedConcept] = []
|
rejected: list[RefusedConcept] = []
|
||||||
|
unverified: list[UnverifiedReference] = []
|
||||||
log = ""
|
log = ""
|
||||||
|
|
||||||
if documents:
|
if documents:
|
||||||
|
|
@ -406,6 +448,19 @@ def import_bundle(
|
||||||
MergedConcept(concept_path=concept_path, path=path, reasons=verdict.reasons)
|
MergedConcept(concept_path=concept_path, path=path, reasons=verdict.reasons)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# §10 pointers, surfaced over what actually landed. Read AFTER the merge
|
||||||
|
# decision and never before it: this door's tolerance is structural —
|
||||||
|
# it writes the sender's bytes verbatim and judges no shape — and a
|
||||||
|
# reader consulted earlier is precisely what would start refusing
|
||||||
|
# senders on form. Read from the written file rather than from the
|
||||||
|
# staged text, so the report describes the concept that exists.
|
||||||
|
unverified.extend(
|
||||||
|
UnverifiedReference(concept_path=entry.concept_path, key=key)
|
||||||
|
for entry in merged
|
||||||
|
for key in _ATTESTED_POINTERS
|
||||||
|
if key in parse_frontmatter(entry.path)
|
||||||
|
)
|
||||||
|
|
||||||
# §6 index — the last disk mutation, and only when something merged.
|
# §6 index — the last disk mutation, and only when something merged.
|
||||||
if merged:
|
if merged:
|
||||||
index_path = bundle / DEFAULT.index.name
|
index_path = bundle / DEFAULT.index.name
|
||||||
|
|
@ -421,4 +476,5 @@ def import_bundle(
|
||||||
failed=tuple(sorted(failed, key=lambda entry: entry.concept_path)),
|
failed=tuple(sorted(failed, key=lambda entry: entry.concept_path)),
|
||||||
log=log,
|
log=log,
|
||||||
ingested_at=ingested_at,
|
ingested_at=ingested_at,
|
||||||
|
unverified_references=tuple(unverified),
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -31,6 +31,7 @@ import pytest
|
||||||
|
|
||||||
from llm_ingestion_okf.materialize import parse_frontmatter
|
from llm_ingestion_okf.materialize import parse_frontmatter
|
||||||
from llm_ingestion_okf.profiles import DEFAULT, OKF_V0_2, STRICT_V1, FrontmatterSchema
|
from llm_ingestion_okf.profiles import DEFAULT, OKF_V0_2, STRICT_V1, FrontmatterSchema
|
||||||
|
from test_import_flow import CONCEPT, StubImportGate, place, run
|
||||||
|
|
||||||
ATTESTED = "Attested Computation"
|
ATTESTED = "Attested Computation"
|
||||||
|
|
||||||
|
|
@ -193,3 +194,122 @@ def test_two_nested_block_mappings_sharing_a_key_collide_in_the_scalar_parser(
|
||||||
assert parsed["attester"] == ""
|
assert parsed["attester"] == ""
|
||||||
assert parsed["resource"] == "attesters/revenue.py"
|
assert parsed["resource"] == "attesters/revenue.py"
|
||||||
assert "skills/run-on-bq.md" not in parsed.values()
|
assert "skills/run-on-bq.md" not in parsed.values()
|
||||||
|
|
||||||
|
|
||||||
|
# --- door C surfaces the §10 pointers it imports ---------------------------
|
||||||
|
#
|
||||||
|
# V6, decided by the operator 2026-07-31. Door C imports the POINTER to
|
||||||
|
# executable code while never importing the code: it writes concepts verbatim
|
||||||
|
# and skips every non-`.md` file. So an imported Attested Computation can
|
||||||
|
# reference an executor or attester that did not arrive — or, worse, one that
|
||||||
|
# resolves to a file the destination tree already holds under that name.
|
||||||
|
#
|
||||||
|
# The operator ruled: import and report, not refuse. Refusing sits against §14
|
||||||
|
# (a consumer MUST NOT reject a bundle for broken cross-links, and whether
|
||||||
|
# `executor.resource` is one the spec does not settle), while §10.5 asks a
|
||||||
|
# consumer to surface rather than silently drop. Reporting satisfies the second
|
||||||
|
# without testing the first.
|
||||||
|
#
|
||||||
|
# The report is at KEY level, not resource level, and that is a measured limit
|
||||||
|
# rather than a choice: resolving the resource means reading `executor.resource`,
|
||||||
|
# which is exactly the value this library's parser cannot recover (see above).
|
||||||
|
# Precision arrives with the structured reader (D1b).
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_merged_concept_declaring_an_executor_is_reported(tmp_path: Path) -> None:
|
||||||
|
gate = StubImportGate()
|
||||||
|
place(
|
||||||
|
tmp_path / "source",
|
||||||
|
"computations/margin.md",
|
||||||
|
"---\ntype: Attested Computation\nruntime: bigquery\n"
|
||||||
|
"executor: { resource: skills/run-on-bq.md, receipt: [job_id] }\n---\n\n# Computation\n",
|
||||||
|
)
|
||||||
|
|
||||||
|
result, _ = run(tmp_path, gate)
|
||||||
|
|
||||||
|
assert [(r.concept_path, r.key) for r in result.unverified_references] == [
|
||||||
|
("computations/margin.md", "executor")
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_block_form_is_reported_too(tmp_path: Path) -> None:
|
||||||
|
"""The form whose VALUE the parser loses. Key presence survives it, which is
|
||||||
|
what makes a key-level report robust where a resource-level one would be
|
||||||
|
silently empty on exactly the canonical presentation."""
|
||||||
|
gate = StubImportGate()
|
||||||
|
place(
|
||||||
|
tmp_path / "source",
|
||||||
|
"computations/margin.md",
|
||||||
|
"---\ntype: Attested Computation\nruntime: bigquery\n"
|
||||||
|
"attester:\n resource: attesters/margin.py\n---\n\n# Computation\n",
|
||||||
|
)
|
||||||
|
|
||||||
|
result, _ = run(tmp_path, gate)
|
||||||
|
|
||||||
|
assert [(r.concept_path, r.key) for r in result.unverified_references] == [
|
||||||
|
("computations/margin.md", "attester")
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_both_pointers_are_reported_in_deterministic_order(tmp_path: Path) -> None:
|
||||||
|
gate = StubImportGate()
|
||||||
|
place(
|
||||||
|
tmp_path / "source",
|
||||||
|
"computations/margin.md",
|
||||||
|
"---\ntype: Attested Computation\nruntime: bigquery\n"
|
||||||
|
"executor: { resource: skills/run-on-bq.md }\n"
|
||||||
|
"attester: { resource: attesters/margin.py }\n---\n\n# Computation\n",
|
||||||
|
)
|
||||||
|
|
||||||
|
result, _ = run(tmp_path, gate)
|
||||||
|
|
||||||
|
assert [(r.concept_path, r.key) for r in result.unverified_references] == [
|
||||||
|
("computations/margin.md", "attester"),
|
||||||
|
("computations/margin.md", "executor"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_concept_carrying_neither_pointer_is_not_reported(tmp_path: Path) -> None:
|
||||||
|
gate = StubImportGate()
|
||||||
|
place(tmp_path / "source", "users.md", CONCEPT)
|
||||||
|
|
||||||
|
result, _ = run(tmp_path, gate)
|
||||||
|
|
||||||
|
assert result.unverified_references == ()
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_report_changes_neither_the_verdict_nor_the_bytes(tmp_path: Path) -> None:
|
||||||
|
"""Advisory, and only advisory. The concept merges, and it merges verbatim:
|
||||||
|
Door C's one invariant is that it never rewrites a sender's bytes, and a
|
||||||
|
report that moved a concept out of `merged` would be the refusal the
|
||||||
|
operator did not choose."""
|
||||||
|
document = (
|
||||||
|
"---\ntype: Attested Computation\nruntime: bigquery\n"
|
||||||
|
"executor: { resource: skills/run-on-bq.md }\n---\n\n# Computation\n"
|
||||||
|
)
|
||||||
|
gate = StubImportGate()
|
||||||
|
place(tmp_path / "source", "computations/margin.md", document)
|
||||||
|
|
||||||
|
result, bundle = run(tmp_path, gate)
|
||||||
|
|
||||||
|
assert [entry.concept_path for entry in result.merged] == ["computations/margin.md"]
|
||||||
|
assert result.merged[0].path.read_text(encoding="utf-8") == document
|
||||||
|
assert result.unverified_references != ()
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_refused_concept_is_never_reported(tmp_path: Path) -> None:
|
||||||
|
"""Nothing was written, so there is no imported pointer to surface. A report
|
||||||
|
on a refused concept would tell an operator to check a file that does not
|
||||||
|
exist."""
|
||||||
|
gate = StubImportGate(by_marker={"Attested Computation": "fail_secure"})
|
||||||
|
place(
|
||||||
|
tmp_path / "source",
|
||||||
|
"computations/margin.md",
|
||||||
|
"---\ntype: Attested Computation\nruntime: bigquery\n"
|
||||||
|
"executor: { resource: skills/run-on-bq.md }\n---\n\n# Computation\n",
|
||||||
|
)
|
||||||
|
|
||||||
|
result, _ = run(tmp_path, gate)
|
||||||
|
|
||||||
|
assert result.merged == ()
|
||||||
|
assert result.unverified_references == ()
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue