llm-ingestion-okf/docs/plan/phase-3-configurable-contract.md
Kjell Tore Guttormsen cf1f7984c4 docs(phase-3): the four open questions are answered; required set is eight
The proving consumer answered Q1-Q4 on 2026-07-25, each measured or read out
of their code that day. No row in the mapping moved. Three of the four answers
carry a sharper reason than this table drafted, and one reverses a correction
this table made a day earlier.

Q2 is the substantive change. This table had corrected the plan's eight-key
required set down to four, against the consumer's REQUIRED_KEYS constant.
Their operator ratified eight: the constant is their emit-path minimum, not
their contract, and all eight keys are present on 522/522 of their documents.
STRICT_V1 is therefore deliberately stricter than the consumer's own
validator; if they ever ship a class lacking source_sha they bump the profile
rather than bend the bundle, and they say so before shipping it. The eleven-key
allowlist is unchanged and still a different thing from the required set:
version/date are layer-specific and summary is the one nullable key, so all
three stay optional-when-absent.

Q1, Q3 and Q4 keep their Consumer/Guard assignments, now with the reasons that
justify them: gate 5 escalates on run state (disposition == verbatim) rather
than document class, so a profile structurally cannot see it; gate 4's
changelog rule is a vendor slug list that is not scoped to concepts/ at all;
gates 19 and 22 need external state, one of them by construction never
blocking. Assumption C2 is confirmed at 25 gates and the index shape is
confirmed non-negotiable.

Both reviews - the consumer's and the operator's - are now in, so the port may
begin. The Guard column is unaffected by all of it.
2026-07-25 15:21:59 +02:00

122 lines
7.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Phase 3 plan — Configurable bundle contract
Status: approved roadmap phase (see `CLAUDE.md`); details settled here before code.
Depends on: Phases 12 (their behavior becomes the *default profile* and must not change).
Proving consumer: `claude-code-llm-wiki` with a `strict-v1` profile.
## Goal
Everything the first two phases hard-code about what a valid bundle looks like —
concept types, layers/path prefixes, frontmatter sets and ordering, reserved-file
policy — becomes configuration on a profile object instead of constants. The
observable behavior under the default profile is byte-identical to Phase 1/2;
the golden suite is the regression harness for that claim.
## Design
1. **`BundleProfile`** — a frozen dataclass (stdlib, no config-file DSL in v1;
profiles are constructed in code by the consumer):
- `type_policy`: closed enum vs. open set; the reserved `verdict` exclusion
stays unconditional (it is a spec invariant, not profile config).
- `frontmatter_schema`: three distinct fields, not one — the *required* key
set, the *allowlisted* key set, and the canonical emission *order*.
STRICT_V1 requires eight keys inside an eleven-key allowlist (see
`docs/phase-3-split-table.md`), so a single ordered required/optional pair
cannot express it. Also carries nullable keys and the scalar encoding
subset. Phase 1's seven-key layer and Phase 2's inbox layer are
expressions of this schema.
- `path_policy`: layer/path prefixes, filename namespaces, reserved filename
patterns (`index.md`, `promoted-verdict-*`, …).
- `reserved_files`: per-file policy, THREE values not two — required,
required-absent, and present-but-exempt-from-document-gates (the proving
consumer's `README.md`, which carries no frontmatter and is never indexed
yet is still content-scanned).
- `index_policy`: managed-link format and per-level index expectations. This
is the widest gap between the two profiles: one index per bundle versus
one per directory, `- [label](target)` versus
`* [Title](link) - description`, append-if-absent versus exact match in
both directions, and a root index with or without frontmatter.
2. **Built-in profiles:**
- `DEFAULT` (implicit): exactly the ingest-spec v1 + Phase 2 contract.
- `STRICT_V1`: the `claude-code-llm-wiki` contract — closed type enum
(`Concept`/`Guide`/`Reference`/`Release`), eight required frontmatter keys
(`type`, `title`, `description`, `timestamp`, `layer`, `source_tier`,
`source_url`, `source_sha`) inside an eleven-key allowlist that doubles as
emission order, layer/path/type/source_tier agreement rules, `log.md`
required-absent, `README.md` reserved-exempt, per-directory indexes, and
root-index metadata (`okf_version`, `bundle_profile`, `okf_spec_commit`).
This line was drafted as eight, corrected to four against the consumer's
`REQUIRED_KEYS` constant on 2026-07-25, and settled at eight by their
operator the same day: the four-key constant is their emit-path minimum,
not their contract, and all eight are present on 522/522 of their
documents. STRICT_V1 is therefore deliberately stricter than the
consumer's own validator; the split table records the agreed consequence.
3. **Plumbing change:** materialization, inbox, import, and index generation take
an optional `profile` argument defaulting to `DEFAULT`. No behavior branches
outside what the profile object expresses.
## The split table (settled before code)
The wiki's existing validator mixes three kinds of gates. First implementation
step is a written mapping of every gate to exactly one column — the mapping is
reviewed with the operator before any porting. **Drafted 2026-07-25 in
`docs/phase-3-split-table.md`**, over all 25 gates, and sent to the proving
consumer for review; two gates turned out to bundle checks from different
columns and are decomposed there rather than forced into one.
| Column | Meaning | Goes where |
|---|---|---|
| Profile | Structural bundle contract (types, keys, paths, reserved files) | This phase, `BundleProfile` |
| Guard | Content safety (sanitize, quarantine, fencing, secrets, budgets) | Guard calls — never reimplemented here |
| Consumer | Domain logic (source pipelines, promotion, staging) | Stays in the consumer repo |
## TDD order
1. `BundleProfile` type + `DEFAULT` profile; refactor Phase 1/2 constants onto
it with the full existing test suite as the safety net.
2. Golden suite re-run — byte-for-byte identical under `DEFAULT` (the phase's
load-bearing regression test).
3. `STRICT_V1` profile + fixtures: a wiki-shaped bundle that validates under
`STRICT_V1` and is rejected under `DEFAULT` (extra frontmatter keys are fine
under preserve-unknown; the *rejection* comes from the missing ingest-spec
layer), and an ingest-spec bundle rejected under `STRICT_V1` (missing
required keys, `log.md` policy).
4. Reserved-file policy tests (required-absent enforcement).
5. Split-table review artifact committed under `docs/` and cross-checked against
the implemented profile fields.
## Key assumptions (each with its test)
| # | Assumption | Test |
|---|---|---|
| C1 | Default-profile refactor is behavior-neutral | Phase 1 golden suite byte-identical; full Phase 1/2 test suite green, unmodified |
| C2 | `strict-v1` requirements are fully enumerable from the wiki's validator | The split table is exhaustive: every wiki gate appears in exactly one column; reviewed with the operator. Drafted 2026-07-25 against `validate.py`/`bundle.py` at their `9ee5a8e`, which is the single gate entrypoint — the enumeration holds only against that commit, so re-check before porting. Two gates required decomposition rather than a single column |
| C3 | Verdict reservation must survive every profile | Named test: no profile construction can permit `type: verdict` |
| C4 | Profiles-in-code (no config files) suffice for the proving consumer | CONFIRMED with the operator 2026-07-25 at phase start: profiles are constructed in code, config-file loading stays an extension point |
## Non-goals
- Porting the wiki's security stack — those gates map to the Guard column.
- Config-file/DSL profile loading, profile inheritance chains — extension points.
- Adopting the second-brain (Node-world) conventions — that contract enters in
Phase 4; this phase only ensures the profile object is expressive enough not
to block it (checked against the catalog's spec during the split-table step).
- Any change to commons' ingest-spec; if profile work reveals spec friction, it
is raised in commons, not patched locally.
## Verification
1. `pytest` green; `mypy --strict src/`; `ruff check .` and
`ruff format --check .` clean.
2. Golden suite byte-for-byte under `DEFAULT` — zero fixture changes in the diff
(`git diff --stat examples/` empty for this phase).
3. `STRICT_V1` fixture bundle validates; cross-profile rejection tests pass in
both directions (step 3 above).
4. C3 named test: constructing a profile that admits `verdict` raises at
construction time.
5. The split table exists under `docs/`, every wiki gate row has exactly one
column, and the Guard column contains no rows implemented in this repo
(grep-gate from Phase 2 still clean).
6. Public API check: Phase 1/2 call sites compile unchanged (profile argument is
optional) — verified by leaving existing tests untouched.