open/llm-ingestion-okf
1
0
Fork 0
Code Issues Pull requests Releases Activity

docs(phase-3): split table over all 25 gates, and two corrected premises

Browse source 
Phase 3 starts with a written mapping of the proving consumer's gates to
exactly one of Profile / Guard / Consumer, reviewed before any porting.
Drafted here against claude-code-llm-wiki's validate.py and bundle.py at
their 9ee5a8e, which is their single gate entrypoint, so the enumeration
is exhaustive against that commit and no other.

Reading the validator corrected two things this plan asserted before it
had been read, both of which would have produced a wrong STRICT_V1:

- Required frontmatter keys are FOUR (type, title, description,
  timestamp), not the eight the plan listed. Those eight were a
  truncation of FRONTMATTER_KEY_ORDER, which is an eleven-key allowlist
  doubling as canonical emission order. frontmatter_schema therefore
  needs three fields -- required, allowlisted, ordered -- not one
  required/optional pair.
- Reserved files are THREE basenames, not two: README.md is reserved
  like index.md and log.md but in a third way, present and exempt from
  the document gates while still content-scanned. A two-valued
  required/required-absent policy cannot express it.

Two of the 25 gates bundle checks from different columns
(LLM_OUTPUT_LINT, BUNDLE_URL_ALLOWLIST). They are decomposed per check
rather than assigned wholesale to a dominant column, which would have
moved the residue to the wrong side of the boundary.

Also recorded: index policy is the widest gap between the two profiles
(per-bundle vs per-directory, two link formats, append-if-absent vs
exact match, root index with or without frontmatter), and C4 is
confirmed with the operator -- profiles in code, no config-file loading
in v1.

Four open questions are addressed to the proving consumer in the
artifact; no porting until they and the table come back reviewed.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 12:30:51 +02:00
commit d8e466791c
2 changed files with 200 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

168
docs/phase-3-split-table.md Normal file
View file

@ -0,0 +1,168 @@
# Phase 3 split table — the proving consumer's 25 gates
Status: **draft, awaiting review.** Phase 3's plan requires this mapping before
any porting, reviewed with the operator and with the proving consumer
(`claude-code-llm-wiki`). Nothing here is implemented yet, and nothing in the
consumer's repository is modified by this repository.
Read from the consumer's working tree on 2026-07-25:
`tools/wiki_ingest/validate.py` and `tools/wiki_ingest/bundle.py` at commit
`9ee5a8e` (repository HEAD `20bcdd6`). Their `validate.py` is the single
entrypoint for every per-run gate, so this table is exhaustive over the gate
set as it stands at that commit — assumption C2 is satisfied only as long as
that remains true, and a gate added there after this date is not in this table.
## The three columns
| Column | Meaning | Goes where |
|---|---|---|
| Profile | Structural bundle contract: types, frontmatter keys, paths, reserved files, index shape | This phase, `BundleProfile` |
| Guard | Content safety: carriers, secrets, injection, active content | `llm-ingestion-guard` calls — never reimplemented here |
| Consumer | Domain logic: source pipelines, LLM contract, promotion, run accounting | Stays in the consumer's repository |
## Stage A — detection/acquisition
| # | Gate | Default class | Column | Note |
|---|---|---|---|---|
| 1 | `INGEST_LEDGER_DIVERGENCE` | ALERT (halts) | Consumer | Upstream re-poll reconciliation; no bundle structure involved. |
| 2 | `INGEST_CHANGELOG_PARSE` | ERROR | Consumer | Source-format parsing for one specific upstream. |
| 3 | `INGEST_EXPECTED_SET` | ALERT | Consumer | npm/changelog reconciliation against known deltas. |
| 4 | `INGEST_DOCS_CORPUS_SANITY` | ERROR | Consumer | Corpus size/shape thresholds for one named corpus. Its second half — "a concept must not derive from the changelog page" — is a layer-provenance rule expressed as a URL test; see open question Q3. |
| 5 | `INGEST_INPUT_INJECTION_SCAN` | WARN (ERROR when verbatim) | Guard | Carrier scan plus semantic lexicon on fetched input. The disposition-aware escalation is Consumer policy over a Guard verdict; see open question Q1. |
## Stage B — LLM-output contract
| # | Gate | Default class | Column | Note |
|---|---|---|---|---|
| 6 | `LLM_STOP_REASON` | ERROR | Consumer | |
| 7 | `LLM_SCHEMA` | ERROR | Consumer | Frozen run-record schema plus summary length caps. |
| 8 | `LLM_OUTPUT_LINT` | ERROR | mixed | Decomposed below. |
| 9 | `LLM_GROUNDING_ANCHOR` | ERROR | Consumer | Anchor resolution against their concepts layer. |
| 10 | `LLM_SOURCE_TRACE` | ERROR | Consumer | Verbatim-quote and dropped-citation detectors. |
| 11 | `LLM_SEAM_LINKS` | ERROR | Consumer | Proposed links restricted to existing concept IDs. |
| 12 | `LLM_BUDGET` | ERROR | Consumer | Per-run call/token/cost caps. |
## Stage C — bundle conformance and publish
| # | Gate | Default class | Column | Note |
|---|---|---|---|---|
| 13 | `BUNDLE_FRONTMATTER_PARSEABLE` | ERROR | Profile | Every non-reserved `.md` opens with a complete fence. |
| 14 | `BUNDLE_TYPE_ENUM` | ERROR | Profile | Closed enum `Concept`/`Guide`/`Reference`/`Release`. |
| 15 | `BUNDLE_RESERVED_FILES` | ERROR | Profile | `log.md` required-absent; `index.md` structure. Three reserved basenames, not two — see "What STRICT_V1 must express". |
| 16 | `BUNDLE_YAML_STRICT` | ERROR | Profile | Key allowlist, required keys, and the StrictYAML scalar subset. |
| 17 | `BUNDLE_LAYER_PATH_TYPE` | ERROR | Profile | Path prefix ⇄ `layer` ⇄ `source_tier` ⇄ `type` agreement. |
| 18 | `BUNDLE_LINKS_INTERNAL` | ERROR | Profile | Every relative cross-link resolves inside the bundle. |
| 19 | `BUNDLE_LINKS_EXTERNAL` | WARN (never blocks) | Consumer | Liveness polling; needs network state this library does not collect. |
| 20 | `BUNDLE_URL_ALLOWLIST` | ERROR | mixed | Decomposed below. |
| 21 | `BUNDLE_INDEX_COMPLETE` | ERROR | Profile | Per-directory index exactness plus root-index metadata keys. |
| 22 | `BUNDLE_HASH_REGISTRY` | ERROR | Consumer | Their page-hash snapshotter must cover the concepts layer exactly. |
| 23 | `BUNDLE_SANITIZATION` | ERROR | Guard | Carrier-freedom of published text. |
| 24 | `BUNDLE_BLAST_RADIUS` | ERROR | Consumer | Per-version diff caps from their run manifest. |
| 25 | `BUNDLE_SECRET_SCAN` | ERROR | Guard | Credential patterns over bundle text and run artifacts. |
## The two mixed gates, decomposed
The plan asks for exactly one column per gate. Two gates do not honour that as
written, because each bundles checks belonging to different columns. Splitting
them is the honest mapping; the alternative — assigning the whole gate to its
dominant column — would quietly move the residue to the wrong side.
**Gate 8, `LLM_OUTPUT_LINT`** runs `_lint_text` over enrichment summaries and
verbatim release bodies:
| Check | Column |
|---|---|
| Off-allowlist URL | Consumer (their editorial pinning; see gate 20) |
| HTML tag in prose | Guard — active content is squarely the guard's territory at v0.3.0 |
| base64-like blob | Guard |
| Credential pattern | Guard |
| Consumer-directed imperative | Guard — this is the injection posture, expressed as their house rule |
| Injection lexicon and hex blob | Guard |
**Gate 20, `BUNDLE_URL_ALLOWLIST`** bundles two unrelated rules:
| Check | Column |
|---|---|
| URL host/prefix on the pinned allowlist | Consumer |
| Domain is new versus the previous bundle state | Consumer (drift detection over their own retained state) |
Neither half is Profile: a URL allowlist is editorial policy about which
sources a bundle may cite, not a statement about bundle structure. It is listed
as mixed only because the checks have different natures, and both land in the
same column — noted rather than merged, so a later reviewer can see it was
considered.
## What STRICT_V1 must express
The phase-3 plan sketched this profile before the validator was read. Two of
its claims do not survive contact and are corrected here; the plan file is
corrected in the same commit.
1. **Required keys are four, not eight.** `REQUIRED_KEYS = (type, title,
description, timestamp)`. The eight-key list in the plan was a truncation of
`FRONTMATTER_KEY_ORDER`, which is a different thing: an eleven-key
*allowlist* that doubles as canonical emission order — `type`, `title`,
`description`, `timestamp`, `layer`, `source_tier`, `source_url`,
`source_sha`, `version`, `date`, `summary`. So `frontmatter_schema` needs
three fields, not one: required, allowlisted, and ordered.
2. **Three reserved basenames, not two.** `index.md`, `log.md`, and
`README.md`. README is bundle documentation, reserved like `index.md`: it
carries no frontmatter, is skipped by the per-document gates, is never
listed in a generated index, and is still subject to the link, URL,
sanitization and secret gates. A two-value reserved-file policy
(required-present / required-absent) cannot express it; the third value is
"present-and-exempt-from-document-gates".
Beyond the frontmatter set, `BundleProfile` must carry:
- **Type policy:** closed enum, four values.
- **Nullable keys:** `summary` alone may be `null` — the single deliberate
escape from strings-only frontmatter.
- **Scalar encoding:** the StrictYAML subset (JSON-double-quoted scalars), so
a regeneration over the same data is byte-identical.
- **Layer/path policy:** `bundle/concepts/` ⇄ layer `concepts`, source_tier
`official-docs`, any type except `Release`; `bundle/releases/` ⇄ layer
`releases`, source_tier `changelog-derived`, type `Release` exactly.
- **Index policy**, which differs from this library's default in four ways:
one index per *directory* rather than one per bundle; entries formatted
`* [Title](link) - description` rather than `- [label](target)`; index
contents must match the directory exactly, both directions; and the root
index carries frontmatter pinning `okf_version`, `bundle_profile`,
`okf_spec_commit`.
- **Bundle root:** a `bundle/` subdirectory, where this library's default
treats the bundle directory as the root itself.
The index-policy divergence is the largest single item. Under the default
profile the observable output must stay byte-identical (assumption C1, proven
by the golden suite), so `index_policy` has to express both shapes rather than
generalising one into the other.
## Open questions for the proving consumer
- **Q1.** Gate 5 escalates the injection scan from WARN to ERROR on a
`verbatim` disposition, because no LLM rewrite layer sits in front of the
published text. That is a policy tier chosen per document, over a guard
verdict. Should the profile be able to express "this document class gets a
stricter gate tier", or does that stay entirely in the consumer's hands?
This repository's position is the latter — the profile describes structure,
not security tiers — but it is the consumer's call, since it decides where
their gate lives after the port.
- **Q2.** Is the four-key required set final, or is it four only because seed
triage has not finished? `layer`, `source_tier`, `source_url` and
`source_sha` are allowlisted and, in practice, present on every generated
document — if they are meant to be required, STRICT_V1 should require them.
- **Q3.** Gate 4's second half asserts that no `concepts/` page may derive from
the changelog page. That reads as a layer-provenance rule, which is Profile
territory, but it is expressed as a URL predicate over one specific upstream,
which is Consumer territory. Mapped to Consumer here. Correct?
- **Q4.** Gates 19 and 22 are mapped to Consumer because they need state this
library never collects (link liveness, page hashes). If either is expected to
move, the profile would have to grow a notion of external state, which is a
larger change than phase 3 scopes.
## What this table does not do
It does not port anything. Per the plan, porting starts only after this
mapping is reviewed, and the Guard column must contain no rows implemented in
this repository — the phase-2 grep gate over guard functionality still applies
unchanged.

45
docs/plan/phase-3-configurable-contract.md
View file

@ -18,21 +18,37 @@ the golden suite is the regression harness for that claim.
profiles are constructed in code by the consumer):
- `type_policy`: closed enum vs. open set; the reserved `verdict` exclusion
stays unconditional (it is a spec invariant, not profile config).
- `frontmatter_schema`: ordered required keys + optional keys, per concept
type where needed (Phase 1's seven-key layer and Phase 2's inbox layer are
expressions of this schema).
- `frontmatter_schema`: three distinct fields, not one — the *required* key
set, the *allowlisted* key set, and the canonical emission *order*. The
proving consumer keeps four required keys inside an eleven-key allowlist
(see `docs/phase-3-split-table.md`), so a single ordered required/optional
pair cannot express it. Also carries nullable keys and the scalar encoding
subset. Phase 1's seven-key layer and Phase 2's inbox layer are
expressions of this schema.
- `path_policy`: layer/path prefixes, filename namespaces, reserved filename
patterns (`index.md`, `promoted-verdict-*`, …).
- `reserved_files`: per-file required / optional / required-absent policy
(e.g. a profile may demand that `log.md` is absent).
- `index_policy`: managed-link format and per-level index expectations.
- `reserved_files`: per-file policy, THREE values not two — required,
required-absent, and present-but-exempt-from-document-gates (the proving
consumer's `README.md`, which carries no frontmatter and is never indexed
yet is still content-scanned).
- `index_policy`: managed-link format and per-level index expectations. This
is the widest gap between the two profiles: one index per bundle versus
one per directory, `- [label](target)` versus
`* [Title](link) - description`, append-if-absent versus exact match in
both directions, and a root index with or without frontmatter.
2. **Built-in profiles:**
- `DEFAULT` (implicit): exactly the ingest-spec v1 + Phase 2 contract.
- `STRICT_V1`: the `claude-code-llm-wiki` contract — closed type enum
(`Concept`/`Guide`/`Reference`/`Release`), extended required frontmatter
(`type`, `title`, `description`, `timestamp`, `layer`, `source_tier`,
`source_url`, `source_sha`), layer path prefixes, `log.md` required-absent,
root-index metadata.
(`Concept`/`Guide`/`Reference`/`Release`), four required frontmatter keys
(`type`, `title`, `description`, `timestamp`) inside an eleven-key
allowlist that doubles as emission order, layer/path/type/source_tier
agreement rules, `log.md` required-absent, `README.md` reserved-exempt,
per-directory indexes, and root-index metadata (`okf_version`,
`bundle_profile`, `okf_spec_commit`).
An earlier draft of this line listed eight required keys. That was the
allowlist read as the required set; corrected against the validator on
2026-07-25.
3. **Plumbing change:** materialization, inbox, import, and index generation take
an optional `profile` argument defaulting to `DEFAULT`. No behavior branches
outside what the profile object expresses.
@ -41,7 +57,10 @@ the golden suite is the regression harness for that claim.
The wiki's existing validator mixes three kinds of gates. First implementation
step is a written mapping of every gate to exactly one column — the mapping is
reviewed with the operator before any porting:
reviewed with the operator before any porting. **Drafted 2026-07-25 in
`docs/phase-3-split-table.md`**, over all 25 gates, and sent to the proving
consumer for review; two gates turned out to bundle checks from different
columns and are decomposed there rather than forced into one.
| Column | Meaning | Goes where |
|---|---|---|
@ -69,9 +88,9 @@ reviewed with the operator before any porting:
| # | Assumption | Test |
|---|---|---|
| C1 | Default-profile refactor is behavior-neutral | Phase 1 golden suite byte-identical; full Phase 1/2 test suite green, unmodified |
| C2 | `strict-v1` requirements are fully enumerable from the wiki's validator | The split table is exhaustive: every wiki gate appears in exactly one column; reviewed with the operator |
| C2 | `strict-v1` requirements are fully enumerable from the wiki's validator | The split table is exhaustive: every wiki gate appears in exactly one column; reviewed with the operator. Drafted 2026-07-25 against `validate.py`/`bundle.py` at their `9ee5a8e`, which is the single gate entrypoint — the enumeration holds only against that commit, so re-check before porting. Two gates required decomposition rather than a single column |
| C3 | Verdict reservation must survive every profile | Named test: no profile construction can permit `type: verdict` |
| C4 | Profiles-in-code (no config files) suffice for the proving consumer | Confirmed with the operator at phase start; config-file loading is an extension point |
| C4 | Profiles-in-code (no config files) suffice for the proving consumer | CONFIRMED with the operator 2026-07-25 at phase start: profiles are constructed in code, config-file loading stays an extension point |
## Non-goals