Kjell Tore Guttormsen
9436427520
`IndexPolicy` grew five judging fields and `IndexPolicy.violations`, closing the
gap a test has been pinning since `848e395`. `STRICT_V1.index` is now the proving
consumer's shape rather than DEFAULT's.
The design was settled by a conflict rather than by a preference. The convention
owner reported that an index is an authored count of a directory's children and
that a validator enumerating the directory has implemented the wrong contract.
Checked against the proving consumer before adopting it, the two turn out to be
directly opposed: gate BUNDLE_INDEX_COMPLETE (validate.py:1081-1120, ERROR) builds
its expected set by enumerating the directory and demands an exact bidirectional
match, and bundle.py:498-567 writes every index from a tree walk. Neither is
incoherent inside its own spec, so authored-versus-derived is a policy field in
both directions and a library invariant in neither.
Consequences encoded rather than documented: nothing here enumerates a directory
— the caller supplies the listing, `violations` refuses one when the profile's
index is authored and refuses to run without one when it is derived, so code
written to the wrong contract fails at the call instead of passing every test one
would think to write. Root and nested indexes are asymmetric (confirmed in both
consumers, different key sets). A per-entry description is template-level, so
`render_link` refuses both a missing description and an unwanted one.
DEFAULT keeps upstream's root-only index and judges nothing, for the same measured
reason it carries no required frontmatter key set: upstream binds `index.md` to
the bundle root alone, so a judging default would condemn conforming bundles.
25 new tests, 493 green. C1 re-proven: `git diff --stat examples/` empty.
312 lines
19 KiB
Markdown
312 lines
19 KiB
Markdown
# Phase 3 split table — the proving consumer's 25 gates
|
|
|
|
Status: **reviewed by the proving consumer and by the operator (2026-07-25);
|
|
the port may begin.** Phase 3's plan requires this mapping before any porting.
|
|
Nothing in the consumer's repository is modified by this repository.
|
|
|
|
The proving consumer (`claude-code-llm-wiki`) answered the same questions from
|
|
their own code on 2026-07-25, crossing with this draft rather than responding
|
|
to it. Two independent readings of the same source agreed on every substantive
|
|
point: the required-key set as their code spells it, three reserved basenames,
|
|
`summary` as the only nullable key, the index structure, and the closed sets.
|
|
Their answer added four refinements, each folded in below and marked. Their
|
|
column assignments match this table's, with one wording difference resolved in
|
|
gate 5's row. A later round answered the four open questions outright — see
|
|
"Answered by the proving consumer" — and moved the required set from what their
|
|
code enforces to what their operator ratified.
|
|
|
|
Read from the consumer's working tree on 2026-07-25:
|
|
`tools/wiki_ingest/validate.py` and `tools/wiki_ingest/bundle.py` at commit
|
|
`9ee5a8e` (repository HEAD `20bcdd6`). Their `validate.py` is the single
|
|
entrypoint for every per-run gate, so this table is exhaustive over the gate
|
|
set as it stands at that commit — assumption C2 is satisfied only as long as
|
|
that remains true, and a gate added there after this date is not in this table.
|
|
|
|
## The three columns
|
|
|
|
| Column | Meaning | Goes where |
|
|
|---|---|---|
|
|
| Profile | Structural bundle contract: types, frontmatter keys, paths, reserved files, index shape | This phase, `BundleProfile` |
|
|
| Guard | Content safety: carriers, secrets, injection, active content | `llm-ingestion-guard` calls — never reimplemented here |
|
|
| Consumer | Domain logic: source pipelines, LLM contract, promotion, run accounting | Stays in the consumer's repository |
|
|
|
|
## Stage A — detection/acquisition
|
|
|
|
| # | Gate | Default class | Column | Note |
|
|
|---|---|---|---|---|
|
|
| 1 | `INGEST_LEDGER_DIVERGENCE` | ALERT (halts) | Consumer | Upstream re-poll reconciliation; no bundle structure involved. |
|
|
| 2 | `INGEST_CHANGELOG_PARSE` | ERROR | Consumer | Source-format parsing for one specific upstream. |
|
|
| 3 | `INGEST_EXPECTED_SET` | ALERT | Consumer | npm/changelog reconciliation against known deltas. |
|
|
| 4 | `INGEST_DOCS_CORPUS_SANITY` | ERROR | Consumer | Corpus size/shape thresholds for one named corpus. Its second half — "a concept must not derive from the changelog page" — is a layer-provenance rule expressed as a URL test; see open question Q3. |
|
|
| 5 | `INGEST_INPUT_INJECTION_SCAN` | WARN (ERROR when verbatim) | Guard | Carrier scan plus semantic lexicon on fetched input. Guard *column* means the functionality is the guard's and is never reimplemented — not that the call site moves here. The consumer keeps the call, as they state, since it sits in their acquisition stage. The disposition-aware escalation is Consumer policy over a Guard verdict; see Q1. |
|
|
|
|
## Stage B — LLM-output contract
|
|
|
|
| # | Gate | Default class | Column | Note |
|
|
|---|---|---|---|---|
|
|
| 6 | `LLM_STOP_REASON` | ERROR | Consumer | |
|
|
| 7 | `LLM_SCHEMA` | ERROR | Consumer | Frozen run-record schema plus summary length caps. |
|
|
| 8 | `LLM_OUTPUT_LINT` | ERROR | mixed | Decomposed below. |
|
|
| 9 | `LLM_GROUNDING_ANCHOR` | ERROR | Consumer | Anchor resolution against their concepts layer. |
|
|
| 10 | `LLM_SOURCE_TRACE` | ERROR | Consumer | Verbatim-quote and dropped-citation detectors. |
|
|
| 11 | `LLM_SEAM_LINKS` | ERROR | Consumer | Proposed links restricted to existing concept IDs. |
|
|
| 12 | `LLM_BUDGET` | ERROR | Consumer | Per-run call/token/cost caps. |
|
|
|
|
## Stage C — bundle conformance and publish
|
|
|
|
| # | Gate | Default class | Column | Note |
|
|
|---|---|---|---|---|
|
|
| 13 | `BUNDLE_FRONTMATTER_PARSEABLE` | ERROR | Profile | Every non-reserved `.md` opens with a complete fence. |
|
|
| 14 | `BUNDLE_TYPE_ENUM` | ERROR | Profile | Closed enum `Concept`/`Guide`/`Reference`/`Release`. |
|
|
| 15 | `BUNDLE_RESERVED_FILES` | ERROR | Profile | `log.md` required-absent; `index.md` structure. Three reserved basenames, not two — see "What STRICT_V1 must express". |
|
|
| 16 | `BUNDLE_YAML_STRICT` | ERROR | Profile | Key allowlist, required keys, and the StrictYAML scalar subset. |
|
|
| 17 | `BUNDLE_LAYER_PATH_TYPE` | ERROR | Profile | Path prefix ⇄ `layer` ⇄ `source_tier` ⇄ `type` agreement. |
|
|
| 18 | `BUNDLE_LINKS_INTERNAL` | ERROR | Profile | Every relative cross-link resolves inside the bundle. Consumer's caveat, accepted: link resolution assumes their layer layout, so it generalises only because `path_policy` owns the layout too. |
|
|
| 19 | `BUNDLE_LINKS_EXTERNAL` | WARN (never blocks) | Consumer | Liveness polling; needs network state this library does not collect. |
|
|
| 20 | `BUNDLE_URL_ALLOWLIST` | ERROR | mixed | Decomposed below. |
|
|
| 21 | `BUNDLE_INDEX_COMPLETE` | ERROR | Profile | Per-directory index exactness plus root-index metadata keys. |
|
|
| 22 | `BUNDLE_HASH_REGISTRY` | ERROR | Consumer | Their page-hash snapshotter must cover the concepts layer exactly. |
|
|
| 23 | `BUNDLE_SANITIZATION` | ERROR | Guard | Carrier-freedom of published text. |
|
|
| 24 | `BUNDLE_BLAST_RADIUS` | ERROR | Consumer | Per-version diff caps from their run manifest. |
|
|
| 25 | `BUNDLE_SECRET_SCAN` | ERROR | Guard | Credential patterns over bundle text and run artifacts. |
|
|
|
|
## The two mixed gates, decomposed
|
|
|
|
The plan asks for exactly one column per gate. Two gates do not honour that as
|
|
written, because each bundles checks belonging to different columns. Splitting
|
|
them is the honest mapping; the alternative — assigning the whole gate to its
|
|
dominant column — would quietly move the residue to the wrong side.
|
|
|
|
**Gate 8, `LLM_OUTPUT_LINT`** runs `_lint_text` over enrichment summaries and
|
|
verbatim release bodies:
|
|
|
|
| Check | Column |
|
|
|---|---|
|
|
| Off-allowlist URL | Consumer (their editorial pinning; see gate 20) |
|
|
| HTML tag in prose | Guard — active content is squarely the guard's territory at v0.3.0 |
|
|
| base64-like blob | Guard |
|
|
| Credential pattern | Guard |
|
|
| Consumer-directed imperative | Guard — this is the injection posture, expressed as their house rule |
|
|
| Injection lexicon and hex blob | Guard |
|
|
|
|
**Gate 20, `BUNDLE_URL_ALLOWLIST`** bundles two unrelated rules:
|
|
|
|
| Check | Column |
|
|
|---|---|
|
|
| URL host/prefix on the pinned allowlist | Consumer |
|
|
| Domain is new versus the previous bundle state | Consumer (drift detection over their own retained state) |
|
|
|
|
Neither half is Profile: a URL allowlist is editorial policy about which
|
|
sources a bundle may cite, not a statement about bundle structure. It is listed
|
|
as mixed only because the checks have different natures, and both land in the
|
|
same column — noted rather than merged, so a later reviewer can see it was
|
|
considered.
|
|
|
|
## What STRICT_V1 must express
|
|
|
|
The phase-3 plan sketched this profile before the validator was read. Two of
|
|
its claims did not survive contact and are corrected here; the plan file
|
|
carries the same corrections. One of the two was then corrected a second time
|
|
by the consumer's operator — the sequence is recorded rather than smoothed
|
|
over, because the end state is deliberately neither what the plan drafted nor
|
|
what their code says.
|
|
|
|
1. **Required keys are eight — settled by the consumer, not read off their
|
|
code.** Their `REQUIRED_KEYS` constant is four (`type`, `title`,
|
|
`description`, `timestamp`), and this table first corrected the plan down to
|
|
it. Asked whether four was the contract or an artefact of unfinished triage
|
|
(Q2), they measured the whole bundle and their operator ratified eight:
|
|
`type`, `title`, `description`, `timestamp`, `layer`, `source_tier`,
|
|
`source_url`, `source_sha`, present on 522/522 documents. The four-key
|
|
constant is their *emit-path* minimum, not their contract.
|
|
|
|
Two consequences are carried deliberately:
|
|
|
|
- **STRICT_V1 is stricter than the consumer's own validator.** Should they
|
|
ever ship a document class that legitimately lacks `source_sha` — a
|
|
hand-written Guide, say — this profile rejects what their validator
|
|
passes. The agreed resolution is to bump the profile, not to bend the
|
|
bundle, and they send the requirement before shipping such a class rather
|
|
than after.
|
|
- **The allowlist remains a different thing from the required set.**
|
|
`FRONTMATTER_KEY_ORDER` is eleven keys — the eight above plus `version`,
|
|
`date`, `summary` — and doubles as canonical emission order. `version` and
|
|
`date` are layer-specific (present on 351/522) and `summary` on 349/522,
|
|
so all three stay optional-when-absent and are validated against closed
|
|
sets **only when present**; a profile that required them would reject
|
|
pages the consumer ships today. So `frontmatter_schema` needs three
|
|
fields, not one: required, allowlisted, and ordered.
|
|
2. **Three reserved basenames, not two.** `index.md`, `log.md`, and
|
|
`README.md`. README is bundle documentation, reserved like `index.md`: it
|
|
carries no frontmatter, is skipped by the per-document gates, is never
|
|
listed in a generated index, and is still subject to the link, URL,
|
|
sanitization and secret gates. A two-value reserved-file policy
|
|
(required-present / required-absent) cannot express it; the third value is
|
|
"present-and-exempt-from-document-gates".
|
|
|
|
Beyond the frontmatter set, `BundleProfile` must carry:
|
|
|
|
- **Type policy:** closed enum, four values.
|
|
- **Nullable keys:** `summary` alone may be `null` — the single deliberate
|
|
escape from strings-only frontmatter.
|
|
- **Value shapes:** a string scalar, **or a non-empty list of strings**. This
|
|
is the same block-list shape Door C already refuses to round-trip through
|
|
this library's line-oriented parser, which is why an imported concept is
|
|
written verbatim. A profile that assumes scalars-only rejects the consumer's
|
|
own `Release` pages, and a parser that assumes scalars-only would silently
|
|
destroy them.
|
|
- **Key-name grammar:** `^[a-z_]+$`.
|
|
- **Scalar encoding:** the StrictYAML subset (JSON-double-quoted scalars), so
|
|
a regeneration over the same data is byte-identical.
|
|
- **Emission order — carry this over intact:** the ordered prefix above,
|
|
followed by any remaining keys **sorted**. This is the door-A model settled
|
|
by commons decision D1, and it is what makes regeneration byte-identical.
|
|
A profile that normalises key order any other way breaks the consumer's
|
|
`BUNDLE_HASH_REGISTRY` gate, which is what made that decision matter.
|
|
- **Layer/path policy:** `concepts/` ⇄ layer `concepts`, source_tier
|
|
`official-docs`, any type except `Release`; `releases/` ⇄ layer `releases`,
|
|
source_tier `changelog-derived`, type `Release` exactly. Their constants
|
|
spell these `bundle/concepts` and `bundle/releases` because their validator
|
|
walks a repository whose bundle lives in `bundle/`. That prefix is where
|
|
*they* keep the bundle, not part of the contract — see "No placement field"
|
|
below.
|
|
- **Index policy**, which differs from this library's default in four ways:
|
|
one index per *directory* rather than one per bundle; entries formatted
|
|
`* [Title](link) - description` rather than `- [label](target)`; index
|
|
contents must match the directory exactly, both directions; and the root
|
|
index carries frontmatter pinning `okf_version`, `bundle_profile`,
|
|
`okf_spec_commit`.
|
|
## No placement field — a field the profile must NOT grow
|
|
|
|
The catalog reported on 2026-07-25 (their commit `3ccb5f6`,
|
|
`docs/okf-second-brain/spec.md` §1 and §8) that bundle placement is
|
|
**unmandated**, measured against four consumers rather than taken on report:
|
|
exactly one installs a bundle anywhere; for the other three the bundle
|
|
directory is a caller-supplied argument per run, with no default, no discovery
|
|
and no `$HOME` expansion. Discovery, where it happens, keys on the §3 root
|
|
marker (`index.md` plus `okf_version`), never on a path.
|
|
|
|
Three consequences for `BundleProfile`, recorded here because this phase's
|
|
non-goal is precisely "do not make the profile too narrow for the Node-world
|
|
conventions in phase 4":
|
|
|
|
- No root-location or install-path field. There is no path form to express.
|
|
- "No installation root" is a conforming consumer shape, so a profile that
|
|
assumes every consumer has one would be too narrow in exactly the warned way.
|
|
- Installing and writing are separate axes: a consumer may write into a bundle
|
|
it never placed, so write policy must not be tied to a placement concept.
|
|
|
|
Every path in this document is therefore bundle-relative, and the bundle
|
|
directory stays what it is today in both doors — an argument.
|
|
|
|
The index-policy divergence is the largest single item. Under the default
|
|
profile the observable output must stay byte-identical (assumption C1, proven
|
|
by the golden suite), so `index_policy` has to express both shapes rather than
|
|
generalising one into the other.
|
|
|
|
## Answered by the proving consumer
|
|
|
|
All four questions were answered on 2026-07-25, each measured or read out of
|
|
their code that day rather than recalled. **No row in the mapping moved**; the
|
|
reasons under three of the four are sharper than the ones this table drafted.
|
|
|
|
- **Q1 — gate tiers stay Consumer/Guard.** This repository's position was
|
|
accepted, and the escalation turns out not to be a document-class rule at
|
|
all: gate 5 escalates WARN to ERROR when the per-version *record* carries
|
|
`disposition == "verbatim"` (`validate.py:528-540`), i.e. when the LLM
|
|
rewrite layer was skipped and the input lint is the only filter left. That is
|
|
run state, not bundle shape — a profile could not see it even if it wanted
|
|
to. A profile describes structure; it does not set security tiers.
|
|
- **Q2 — eight, ratified by their operator.** Folded into "What STRICT_V1 must
|
|
express" above, together with the consequence both sides accepted.
|
|
- **Q3 — Consumer, for a sharper reason than "URL predicate".** The check is
|
|
`is_changelog_url()` (`docs_corpus.py:181-184`): it lowercases the last URL
|
|
segment and tests membership in `("changelog", "release-notes")` — a
|
|
vendor-specific slug list for one upstream. Two details worth carrying: the
|
|
gate is **not** scoped to `concepts/` at all (it walks every non-reserved
|
|
document across both layers, `validate.py:480-492`, despite a rationale
|
|
string that says "concepts layer"), and it spares their own release pages
|
|
only because their `source_url` ends in `CHANGELOG.md`, whose last segment is
|
|
`changelog.md` rather than `changelog`. That is a coupling, not a design. A
|
|
structural profile could express "layer X may not carry source_tier Y"; it
|
|
could not express this.
|
|
- **Q4 — both stay Consumer.** Gate 19 sits in their `NEVER_BLOCKING_GATES`
|
|
(`validate.py:86-88`) by construction, precisely because liveness is a flake
|
|
magnet: a profile that owned it would inherit a gate that can never block.
|
|
Gate 22 is their provenance model, the one commons decision D1 settled in
|
|
their favour, and it is why the ordered-prefix emission matters. The profile
|
|
gets no notion of external state from either.
|
|
|
|
Two confirmations arrived unasked and are recorded because later work leans on
|
|
them:
|
|
|
|
- **Assumption C2 holds.** `validate.py` is still the single entrypoint for
|
|
every per-run gate, and the registry at lines 50-78 is the whole set — 5
|
|
`INGEST_*`, 7 `LLM_*`, 13 `BUNDLE_*` = 25. A change reaches this repository
|
|
as a message before it can be read out of their code.
|
|
- **The index shape is not negotiable downward**, which makes "express both
|
|
shapes rather than generalise one into the other" the right call: one index
|
|
per directory, exactly one `# ` heading, entries `* [Title](link) -
|
|
description` and nothing else, directory and index matching in both
|
|
directions, and a root index carrying `okf_version` / `bundle_profile` /
|
|
`okf_spec_commit` frontmatter in that key order.
|
|
|
|
## The index policy, and the conflict that shaped it
|
|
|
|
The convention owner reported on 2026-07-25 that the second-brain spec's §3
|
|
mandates an `index.md` at every directory level, that `IndexPolicy` as it then
|
|
stood could not express that, and — the load-bearing part — that an index is an
|
|
**authored** count of a directory's children and never a filesystem lookup. On
|
|
their reading, an index reader or validator that enumerates a directory to
|
|
build or check an index has implemented the wrong contract, with a silent
|
|
failure mode: code written to the wrong reading passes every test one would
|
|
think to write.
|
|
|
|
That was checked against the proving consumer before it was adopted, and the
|
|
two consumers turn out to be **directly opposed on exactly this point**:
|
|
|
|
- `validate.py:1081-1120`, gate `BUNDLE_INDEX_COMPLETE` (severity ERROR),
|
|
builds its expected set by enumerating the directory and fails unless the
|
|
index matches it exactly in both directions.
|
|
- `bundle.py:498-567` writes every index from what a tree walk finds, and its
|
|
own docstring calls `index.md` machine-generated.
|
|
|
|
So one consumer mandates the derivation the other calls the wrong contract.
|
|
Neither is incoherent inside its own spec, and this repository is not the venue
|
|
to adjudicate between them. What it settles is narrower and sufficient:
|
|
**authored-versus-derived cannot be a library invariant in either direction.**
|
|
It is a policy field (`entries_match_directory`), as are per-directory scope,
|
|
the heading requirement, whether prose is admitted, and the root-index key set.
|
|
|
|
Three consequences that outlive this phase:
|
|
|
|
1. **Nothing in this library enumerates a directory.** The caller supplies the
|
|
listing; `IndexPolicy.violations` refuses one when the profile's index is
|
|
authored, and refuses to run without one when it is derived. The
|
|
convention owner's construction rule is enforced at the call site rather
|
|
than documented and hoped for.
|
|
2. **Root and nested indexes are asymmetric**, confirmed independently in both
|
|
consumers: the root carries frontmatter (`okf_version`/`bundle_profile`/
|
|
`okf_spec_commit` for the wiki, `okf_version`/`okf_layout` for the catalog)
|
|
and nested indexes carry none. Different key sets, same shape — which is
|
|
why it is a profile field and not a constant.
|
|
3. **A per-entry description is template-level, not a separate requirement.**
|
|
The wiki requires `* [Title](link) - description`; the catalog requires
|
|
prose in the index and no per-entry description. A profile that wants none
|
|
simply does not name `{description}` in its template.
|
|
|
|
`DEFAULT` keeps upstream's root-only index and judges nothing, for the same
|
|
measured reason it carries no required frontmatter key set: upstream OKF binds
|
|
`index.md` to the bundle root alone, so a default demanding one per level would
|
|
declare upstream-conforming bundles invalid.
|
|
|
|
## What this table does not do
|
|
|
|
It does not port anything. Per the plan, porting starts only after this
|
|
mapping is reviewed, and the Guard column must contain no rows implemented in
|
|
this repository — the phase-2 grep gate over guard functionality still applies
|
|
unchanged.
|
|
|
|
Both reviews are in as of 2026-07-25 — the proving consumer's (above) and the
|
|
operator's — so the port may begin. The Guard-column constraint is unaffected
|
|
by either.
|