open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 1 Activity

feat(okf): conform the context seam to the pulled navigation + stamp-integrity contracts

Browse source 
The commons pull (7aa53fc -> a2b57d2) rewrote method-spec §3 Step 1 and added two §11
seams. Measuring okf.py against the new normative text found six contradictions; this
closes all six, gated by the commons-owned nav-goldens that came with the pull.

method-spec §3 Step 1 (navigate_bundle / bundle_context):
- follow cross-links RECURSIVELY, depth-first in first-seen order (there was no
  recursion at all — only the root index's links were read, so no hierarchy was
  navigable even with the other fixes in place);
- resolve a leading `/` against the BUNDLE ROOT, anything else against the LINKING
  file's directory, and drop the retired "a path separator means out-of-bundle"
  heuristic, which conflated depth with escape and forbade valid nesting;
- de-duplicate on the RESOLVED path (`./a.md` == `a.md` == `/a.md`), which is also
  what terminates cycles;
- exclude index files by BASENAME at every level, so a nested index is navigation and
  never renders as content (flat rendering regardless of depth);
- bind index_summary to the ROOT index alone.

safe_resolve stays the sole in-/out-of-bundle test, fail-closed: a target that fails to
resolve for ANY reason is skipped, never raised.

ingest-spec §3 (write_concept_file): it is the repo's one authoring primitive that
materialises a concept file from caller-supplied frontmatter, so it now refuses the
COMPLETE ownership stamp (`generated: true` + `ingest_manifest`) with IngestStampError,
while permitting either field alone. A validation, never a repair — nothing is written.

Gates (tests/test_okf.py, 529 -> 537):
- nav-golden-hierarchy and nav-golden-escape compared against the shipped
  expected-read-context.md fasit (trailing-whitespace normalisation only, which the
  fixture README explicitly permits; internal blank-line structure stays gated);
- traversal order pinned separately from the rendered output, so a right-looking render
  from a wrong walk still fails;
- unit seams for the recursion in isolation, resolved-path dedup, and the leading-`/`
  rule's breach case (a real out-of-bundle file addressed by its absolute path).

Load-bearing MEASURED, not asserted: seven mutations each go red — detach the recursion,
restore the separator prefilter, dedup on the raw target, read `/` as filesystem-absolute,
render nested index bodies as content, drop the stamp guard, and the fully naive navigator
with no boundary check (which is what makes the `/`-trap test bite). okf.py restored from a
checksum-verified copy after each.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WetWTHpdRbqinN5XHFTaTb
This commit is contained in:
Kjell Tore Guttormsen 2026-07-31 21:07:29 +02:00
commit 0d50ab89d3
3 changed files with 288 additions and 29 deletions
Download patch file Download diff file Expand all files Collapse all files

151
src/portfolio_optimiser/okf.py
View file

@ -1,19 +1,29 @@
"""OKF (Open Knowledge Format) bundle navigation — framework-neutral, D7-portable context seam.
Reads a bundle the way OKF intends (progressive disclosure): start at ``index.md``, follow
intra-bundle cross-links, parse each file's YAML frontmatter, classify by the one required
``type`` field. **NO** ``agent_framework``, **NO** ``mcp`` — pure stdlib, so the SAME navigation
serves both the MAF and the Claude-SDK implementations unchanged (målbilde §4 vendor-neutrality).
intra-bundle cross-links **recursively, depth-first in first-seen link order**, parse each file's
YAML frontmatter, classify by the one required ``type`` field. **NO** ``agent_framework``, **NO**
``mcp`` — pure stdlib, so the SAME navigation serves both the MAF and the Claude-SDK
implementations unchanged (målbilde §4 vendor-neutrality).
Link resolution follows ``shared/method-spec.md`` §3 Step 1: a leading ``/`` denotes the **bundle
root** (NEVER a filesystem-absolute path), any other form is relative to the LINKING FILE's own
directory — so a bundle may be a hierarchy. It is **escape, not depth**, that is forbidden; this
resolve-and-boundary-check replaced the old "a path separator means out-of-bundle" heuristic, which
conflated the two and forbade valid nesting. Repeated links de-duplicate on the RESOLVED path, so
``./a.md`` and ``a.md`` are one entry and cycles terminate.
Robustness is part of the spec (OKF SPEC §4): consumers MUST tolerate broken links and unknown
fields. A link to a missing file — or one escaping the bundle — is silently skipped, never raised.
Path-safety reuses ``retrieval.safe_resolve`` (also pure stdlib): each cross-link is canonicalised
and boundary-checked against the bundle dir, fail-closed.
fields. A target that fails to resolve for ANY reason (missing file, invalid path component,
escape) is silently skipped, never raised. Path-safety reuses ``retrieval.safe_resolve`` (also pure
stdlib): each cross-link is canonicalised and boundary-checked against the bundle dir, fail-closed
— the SOLE in-/out-of-bundle test.
"""
from __future__ import annotations
import json
import posixpath
import re
from dataclasses import dataclass
from pathlib import Path
@ -23,8 +33,8 @@ from portfolio_optimiser.retrieval import PathSecurityError, safe_resolve
_INDEX_NAME = "index.md"
_IR_PROJECTION = "validator-input.json"
# Intra-bundle markdown cross-links: ``](target.md)``. Targets with a path separator (``/``) are
# treated as out-of-bundle and skipped (only same-dir bundle files are navigated).
# Intra-bundle markdown cross-links: ``](target.md)``. A path separator is NOT a rejection reason —
# ``_resolve_target`` decides in-/out-of-bundle, and only escape is refused (method-spec §3 Step 1).
_LINK_RE = re.compile(r"\]\(([^)]+\.md)\)")
@ -76,7 +86,8 @@ class Bundle:
@property
def index_summary(self) -> str:
"""The index body — the progressive-disclosure entry point (not whole-bundle stuffing)."""
"""The ROOT index body — the progressive-disclosure entry point (not whole-bundle stuffing).
Bound to the root alone: a nested ``a/index.md`` is navigation, never summary prose."""
return next((f.body for f in self.files if f.name == _INDEX_NAME), "")
@property
@ -88,8 +99,16 @@ class Bundle:
def context_files(self) -> list[BundleFile]:
"""The non-index, non-``verdict`` concept files — the bodies that form the agent context.
The verdict layer is deliberately EXCLUDED: prior verdicts reach the hypothesis prompt only
through the gated ExpeL fold, never by stuffing them into the read-context (målbilde §4)."""
return [f for f in self.files if f.name != _INDEX_NAME and f.type != "verdict"]
through the gated ExpeL fold, never by stuffing them into the read-context (målbilde §4).
The exclusion is a TYPE CHECK on each reached file, applied at EVERY level — never a
property of the link graph, so a mislabelled or injected navigation edge cannot smuggle a
nested verdict in. Index files are dropped by BASENAME at every level too: a nested
``a/index.md`` is navigation, not content."""
return [
f
for f in self.files
if posixpath.basename(f.name) != _INDEX_NAME and f.type != "verdict"
]
@property
def hypothesis(self) -> BundleFile | None:
@ -98,8 +117,8 @@ class Bundle:
def _load_file(bundle_dir: str, name: str) -> BundleFile | None:
"""Resolve ``name`` within ``bundle_dir`` and read it, or ``None`` if missing / escaping the
bundle (OKF §4 broken-link tolerance + fail-closed path-safety)."""
"""Resolve the bundle-relative ``name`` within ``bundle_dir`` and read it, or ``None`` if
missing / escaping the bundle (OKF §4 broken-link tolerance + fail-closed path-safety)."""
try:
resolved = Path(safe_resolve(bundle_dir, name))
except PathSecurityError:
@ -110,23 +129,60 @@ def _load_file(bundle_dir: str, name: str) -> BundleFile | None:
return BundleFile(name=name, type=fm.get("type", ""), frontmatter=fm, body=_read_body(resolved))
def _resolve_target(bundle_dir: str, from_name: str, target: str) -> tuple[str, str] | None:
"""Resolve one cross-link into ``(bundle-relative posix name, canonical path)``, or ``None``
when it fails to resolve for ANY reason (escape, invalid path component) — the caller skips,
never raises (method-spec §3 Step 1).
A leading ``/`` denotes the BUNDLE ROOT, not the filesystem root: an implementation that let
``os.path.join`` see an absolute target would either escape to the real filesystem path or
refuse a legitimate root-relative link. Any other form resolves against the LINKING FILE's own
directory, so an index links its immediate children one segment at a time."""
if target.startswith("/"):
rel = posixpath.normpath(target.lstrip("/"))
else:
rel = posixpath.normpath(posixpath.join(posixpath.dirname(from_name), target))
try:
return rel, safe_resolve(bundle_dir, rel)
except PathSecurityError:
return None
def _walk(bundle_dir: str, current: BundleFile, files: list[BundleFile], seen: set[str]) -> None:
"""Follow ``current``'s cross-links depth-first in first-seen order, appending each newly
reached file and recursing into it. De-duplication is on the CANONICAL RESOLVED path (so
``./a.md``, ``a.md`` and ``/a.md`` are one entry), which is also what terminates cycles."""
for target in _LINK_RE.findall(current.body):
resolved = _resolve_target(bundle_dir, current.name, target)
if resolved is None:
continue
rel, canonical = resolved
if canonical in seen:
continue
seen.add(canonical)
linked = _load_file(bundle_dir, rel)
if linked is None:
continue # broken link: tolerated, never raised (OKF §4)
files.append(linked)
_walk(bundle_dir, linked, files, seen)
def navigate_bundle(bundle_dir: str) -> Bundle:
"""Navigate the OKF bundle from ``index.md``: parse the index, follow its intra-bundle ``.md``
cross-links, and read each linked file's frontmatter + body. Deterministic: index first, then
links in first-seen order, de-duplicated. Broken / escaping links are skipped (§4). Raises
``ValueError`` only when ``index.md`` itself is unreadable (a bundle has no entry point)."""
"""Navigate the OKF bundle from ``index.md``: parse the root index, then follow intra-bundle
``.md`` cross-links RECURSIVELY, depth-first in first-seen link order, reading each reached
file's frontmatter + body. Fully deterministic. Broken / escaping links are skipped (§4).
Navigation follows LINKS ONLY — a directory is never enumerated. Hence the missing-``index.md``
error binds the bundle ROOT alone (a bundle has no entry point without it); an intermediate
directory reached by a link needs no ``index.md`` of its own, and content nothing links to is
simply unreachable, not an error. Raises ``ValueError`` only for the unreadable root index."""
index = _load_file(bundle_dir, _INDEX_NAME)
if index is None:
raise ValueError(f"OKF bundle has no readable {_INDEX_NAME}: {bundle_dir!r}")
files: list[BundleFile] = [index]
seen = {_INDEX_NAME}
for target in _LINK_RE.findall(index.body):
if "/" in target or target in seen:
continue # only same-dir bundle files; de-dup repeated links
seen.add(target)
linked = _load_file(bundle_dir, target)
if linked is not None:
files.append(linked)
root = _resolve_target(bundle_dir, _INDEX_NAME, _INDEX_NAME)
seen = {root[1]} if root is not None else set()
_walk(bundle_dir, index, files, seen)
return Bundle(dir=bundle_dir, files=tuple(files))
@ -140,15 +196,23 @@ def bundle_context(bundle: Bundle, *, dimension: str | None = None) -> str:
When ``dimension`` is given, only concept files whose frontmatter ``dimension`` matches — or that
carry no ``dimension`` at all (un-scoped knowledge is never dropped) — are rendered; the default
``dimension=None`` renders every concept file, byte-identical to the prior behavior. ``dimension``
is a plain ``str`` (not the ``Dimension`` type) so ``okf`` stays MAF-free and import-cycle-free."""
sections = [bundle.index_summary]
is a plain ``str`` (not the ``Dimension`` type) so ``okf`` stays MAF-free and import-cycle-free.
Rendering is FLAT regardless of nesting depth (method-spec §3 Step 1): directory structure is
navigation, not presentation, so a nested concept file renders as the same
``## {type}: {title}`` section a root file would — there is no level heading, and nested index
bodies do not appear at all. The serialisation (heading, blank line, body; sections separated by
one blank line) is what the commons nav-golden ``expected-read-context.md`` fasit compares
against — see ``tests/test_okf.py`` nav-golden gates."""
sections = [bundle.index_summary.strip("\n")]
for f in bundle.context_files:
if dimension is not None:
file_dim = f.frontmatter.get("dimension")
if file_dim is not None and file_dim != dimension:
continue
title = f.frontmatter.get("title", f.name).strip('"')
sections.append(f"## {f.type or 'document'}: {title}\n{f.body}")
body = f.body.strip("\n")
sections.append(f"## {f.type or 'document'}: {title}\n\n{body}")
return "\n\n".join(s for s in sections if s.strip())
@ -163,11 +227,42 @@ def render_frontmatter(frontmatter: dict[str, str]) -> str:
return "\n".join(f"{key}: {' '.join(str(value).split())}" for key, value in frontmatter.items())
class IngestStampError(ValueError):
"""A curated writer was handed frontmatter carrying the COMPLETE ingest ownership stamp
(``shared/ingest-spec.md`` §3). The stamp is the sole mark separating ingest-owned files from
curated ones, and re-materialization replaces exactly what carries it — so a curated file that
forged it could be silently deleted by a later ingest run."""
def _carries_complete_ingest_stamp(frontmatter: dict[str, str]) -> bool:
"""Whether ``frontmatter`` carries BOTH halves of the ingest ownership stamp: ``generated:
true`` together with a non-empty ``ingest_manifest`` reference (ingest-spec §7).
The test is on the COMPLETE stamp, never on the individual field names — curated content may
legitimately carry a single provenance field, and a verbatim round-trip of one half must keep
working. Values are compared the way ``parse_frontmatter`` yields them (line-oriented strings,
quotes retained), so surrounding quotes and case are normalised away here."""
generated = str(frontmatter.get("generated", "")).strip().strip('"').lower()
manifest = str(frontmatter.get("ingest_manifest", "")).strip().strip('"')
return generated == "true" and bool(manifest)
def write_concept_file(bundle_dir: str, name: str, frontmatter: dict[str, str], body: str) -> Path:
"""Write a typed OKF concept file (``---`` frontmatter + markdown body) into ``bundle_dir``,
path-safe via ``safe_resolve`` (fail-closed: a ``name`` escaping the bundle raises
``PathSecurityError``). Pure stdlib — the D7-portable counterpart of ``navigate_bundle``'s read.
This is the repo's one authoring primitive that materialises a concept file from CALLER-SUPPLIED
frontmatter, so it is the surface ingest-spec §3's "no other writer may forge the stamp" binds:
a frontmatter carrying the complete ingest stamp raises ``IngestStampError`` and NOTHING is
written. A validation, never a repair — the caller is told, not silently corrected.
Returns the written path."""
if _carries_complete_ingest_stamp(frontmatter):
raise IngestStampError(
"refusing to write a curated concept file carrying the COMPLETE ingest ownership stamp "
"(generated: true + ingest_manifest); only the ingest materializer may claim it "
"(ingest-spec §3) — either field alone is permitted"
)
resolved = Path(safe_resolve(bundle_dir, name))
resolved.parent.mkdir(parents=True, exist_ok=True)
resolved.write_text(f"---\n{render_frontmatter(frontmatter)}\n---\n\n{body}", encoding="utf-8")