Door A (manifest -> connector -> deterministic materialization -> index) is no longer implemented here. src/portfolio_optimiser/ingest.py becomes a thin consumer seam over the shared library, git-pinned to v0.3.1 on the same Forgejo channel portfolio-optimiser-claude uses. Net -626/+385; ingest.py 599 -> 145 lines. shared/ingest-spec.md remains the normative spec: the library implements it, it does not replace it. Spec changes continue to go via commons. Acceptance criterion met and proven: all three golden bundles (file/sql/http) are byte-exact before and after, including the idempotence re-run. examples/ and shared/ carry ZERO modifications -- the fasit was not adjusted to fit. The rejection set was verified equivalent, not assumed: all 22 malformations the repo's pydantic models refused are refused by the library, with typed codes (okf_type_reserved, credential_embedded, extraction_id_duplicate, ...). Test rebinding (invariants preserved, vehicle changed): the library has zero runtime dependencies by design, so pydantic is unavailable to it. ManifestV1.model_validate(dict) -> load_manifest_bytes(bytes); ValidationError -> ManifestError; model_fields -> dataclasses.fields; PathSecurityError -> SourceError(path_escape); ValueError -> MaterializationError(ingested_at_invalid). Tests now also pin the refusal `code`, the library's documented stability contract -- a sharper assertion than "some validation error was raised". Two accepted behavioural deltas, recorded rather than silently dropped: - Title whitespace is stored verbatim instead of collapsed at validation, so the frontmatter title and the index label are no longer guaranteed identical for irregular whitespace. Both behaviours are spec-conformant (the spec is SILENT; the old one was a repo-local pinned decision). Queued as a commons-amendment candidate so both stacks pin the same answer. Goldens unaffected. - The section 8 audit log moves to logger llm_ingestion_okf.materialize. Nothing in the repo consumed the old channel. Also: the `type` discriminator is no longer a dataclass field, so the spec cross-check asserts it explicitly -- without that line the swap would have silently narrowed the test. New tests/test_ingest_library_seam.py pins the seam itself: the restated section 5 stamp formula against the stamp the library actually writes (the one place the adapter does not purely delegate, since v0.3.1 exposes no stamp helper), the local-only allow_network default, the list[Path] unwrapping, and a guard that the adapter never regrows local Door A machinery. All four verified RED when detached, as were both golden regressions under a byte-level render mutation. Door A is UNGATED: it calls no guard before writing to disk. Gating untrusted content remains the caller's responsibility (guard wiring still planned). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B4jNN186eVqfe1x5DnTU6r
127 lines
5.6 KiB
Python
127 lines
5.6 KiB
Python
"""Load-bearing checks on the consumer seam over ``llm-ingestion-okf`` (adopted 2026-07-20).
|
||
|
||
Door A is no longer implemented here — ``src/portfolio_optimiser/ingest.py`` is a thin adapter
|
||
over the shared library, so the spec §4–§6 rules are covered by the library's own suite plus the
|
||
repo's golden regressions. What is NOT covered by either is the seam itself: the places where the
|
||
adapter restates something instead of delegating, and the boundary claims the adapter's docstring
|
||
makes. Those are pinned here, because a docstring that no test can falsify is decoration.
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import hashlib
|
||
import json
|
||
from pathlib import Path
|
||
from typing import Any
|
||
|
||
import pytest
|
||
from llm_ingestion_okf import NetworkGateError
|
||
|
||
from portfolio_optimiser import okf
|
||
from portfolio_optimiser.ingest import load_manifest, materialize, materialize_bundle
|
||
|
||
_INGESTED_AT = "2026-07-03T12:00:00Z"
|
||
|
||
_MANIFEST: dict[str, Any] = {
|
||
"manifest_version": 1,
|
||
"source": {"type": "file", "id": "prosjekt-arkiv", "root": "fixture"},
|
||
"bundle_summary": "Cost extracts from the project archive.",
|
||
"extractions": [
|
||
{
|
||
"id": "costs",
|
||
"title": "Project costs",
|
||
"query": "costs.csv",
|
||
"okf_type": "dataset",
|
||
"max_rows": 100,
|
||
}
|
||
],
|
||
}
|
||
|
||
|
||
def _project(tmp_path: Path, source: dict[str, Any] | None = None) -> tuple[Path, Path]:
|
||
fixture = tmp_path / "fixture"
|
||
fixture.mkdir()
|
||
(fixture / "costs.csv").write_bytes(b"item,cost_nok\nled-retrofit,120000\n")
|
||
data = json.loads(json.dumps(_MANIFEST))
|
||
if source is not None:
|
||
data["source"] = source
|
||
manifest_path = tmp_path / "manifest.json"
|
||
manifest_path.write_text(json.dumps(data), encoding="utf-8")
|
||
return manifest_path, tmp_path / "bundle"
|
||
|
||
|
||
def test_adapter_stamp_equals_library_stamp(tmp_path: Path) -> None:
|
||
"""LOAD-BEARING ANTI-DRIFT: the adapter's ``load_manifest`` restates the §5 stamp formula
|
||
because the library (v0.3.1) mints the stamp inside ``materialize_bundle`` and exposes no
|
||
stamp helper. That is the ONE place the adapter is not purely delegating, so the two
|
||
formulas can drift apart silently — this compares the adapter's value against the stamp the
|
||
library actually writes into ``ingest_manifest`` frontmatter. RED the moment either side
|
||
changes how the stamp is computed."""
|
||
manifest_path, bundle_dir = _project(tmp_path)
|
||
|
||
_, adapter_stamp = load_manifest(manifest_path)
|
||
result = materialize_bundle(manifest_path, bundle_dir, _INGESTED_AT)
|
||
|
||
assert adapter_stamp == result.stamp
|
||
# ...and the stamp is what actually landed on disk, not merely an agreeing computation.
|
||
written_stamp = okf.parse_frontmatter(result.written[0])["ingest_manifest"]
|
||
assert adapter_stamp == written_stamp
|
||
# §5 shape, pinned independently of both implementations.
|
||
expected = "manifest@" + hashlib.sha256(manifest_path.read_bytes()).hexdigest()[:16]
|
||
assert adapter_stamp == expected
|
||
|
||
|
||
def test_adapter_returns_written_paths_in_extraction_order(tmp_path: Path) -> None:
|
||
"""The adapter keeps the repo's historical ``list[Path]`` return over the library's
|
||
``IngestResult``. RED if the unwrapping is dropped or reordered."""
|
||
manifest_path, bundle_dir = _project(tmp_path)
|
||
written = materialize(manifest_path, bundle_dir, ingested_at=_INGESTED_AT)
|
||
assert isinstance(written, list)
|
||
assert [p.name for p in written] == ["ingest-costs.md"]
|
||
assert all(p.is_file() for p in written)
|
||
|
||
|
||
def test_local_only_default_holds_at_the_adapter_seam(tmp_path: Path) -> None:
|
||
"""LOAD-BEARING BOUNDARY (§8): the adapter's docstring claims an ``http`` source is refused
|
||
unless a run explicitly opts in — the manifest can never grant itself network. The library
|
||
owns the gate, but the adapter owns the DEFAULT it is called with. RED if the adapter ever
|
||
starts passing ``allow_network=True`` (or forwards a manifest-derived value)."""
|
||
manifest_path, bundle_dir = _project(
|
||
tmp_path, source={"type": "http", "id": "api", "base_url": "https://host/api"}
|
||
)
|
||
|
||
with pytest.raises(NetworkGateError) as exc:
|
||
materialize(manifest_path, bundle_dir, ingested_at=_INGESTED_AT)
|
||
assert exc.value.code == "network_opt_in_missing"
|
||
assert not bundle_dir.exists(), "a refused network source must write NOTHING"
|
||
|
||
# The opt-in is reachable, so the refusal above is a real default rather than a dead path.
|
||
calls: list[str] = []
|
||
|
||
def fake_get(url: str, credential: str | None) -> str:
|
||
calls.append(url)
|
||
return "ok\n"
|
||
|
||
materialize(
|
||
manifest_path,
|
||
bundle_dir,
|
||
ingested_at=_INGESTED_AT,
|
||
allow_network=True,
|
||
http_get=fake_get,
|
||
)
|
||
assert calls == ["https://host/api/costs.csv"]
|
||
|
||
|
||
def test_adapter_does_not_reimplement_door_a(tmp_path: Path) -> None:
|
||
"""The adoption's point: Door A improves in ONE place. This pins the adapter as thin — it
|
||
must not regrow a local connector/renderer/materializer. RED if the module starts carrying
|
||
the machinery it delegates (csv/sqlite/urllib reading, table escaping, frontmatter
|
||
rendering), which is how a 'temporary local fix' silently forks the shared implementation."""
|
||
source = (
|
||
Path(__file__).resolve().parents[1] / "src" / "portfolio_optimiser" / "ingest.py"
|
||
).read_text(encoding="utf-8")
|
||
for forbidden in ("import csv", "import sqlite3", "urlopen", "def render_table", "\\\\|"):
|
||
assert forbidden not in source, (
|
||
f"ingest.py reimplements Door A machinery ({forbidden!r}) — it must delegate to "
|
||
"llm-ingestion-okf, not fork it"
|
||
)
|