Kjell Tore Guttormsen
2d0328aaa7
U4 escalated tag requirement 1: real v0.2 frontmatter is block YAML, and "emit
accepts a block-list value" had been scoped against a list of strings when
upstream's canonical sources is a list of five-key mappings. Re-sized against
the code rather than against its tests, and the answer runs the other way.
Measured: emit (profiles.py:197) takes Mapping[str, str] and renders one
"key: value" line per key through _render (:211), verbatim but for
source_query's whitespace collapse; _is_legal_value (:97) admits a string or a
non-empty list of strings, never mappings; the only production call sites are
materialize.py:166 and inbox.py:129, both passing dict[str, str].
The earlier "discharged" note read the validator correctly and drew too much
from it. But the gap it left is not U4's either: A-E4 owes at least a resource,
not upstream's five keys, and a manifest source has no author, no last_modified,
and no bundle-internal resource in upstream's sense. A field with no reader is
not written. So the question was never how large a YAML emitter we need -- it
was which form sources takes.
Chosen: sources: [{ id: <id>, resource: <ref> }], the same inline flow form A-E3
already chose for generated. It keeps parse_frontmatter line-oriented, which
matters because a block list pollutes that parser with false keys (- id,
resource) -- the failure mode documented for po-claude -- and _is_ingest_owned,
the one piece of this tag we called risky, reads THROUGH that parser. The block
form would force the collision gate and the parser to be hardened in the same
session. It also satisfies commons' section 5 single-line MUST, so the DEFAULT
seam U4 opened stays closed, and section 11 asks for parseable YAML, not block
YAML. The deviation from upstream's emitted shape is real and is recorded as a
decision, like U1, not as compliance we may claim.
Consequences: requirement 1 is smaller than both earlier sizings -- zero emitter
work -- and D1a drops off the critical path to v0.5.0a1. A-E4 is corrected from
"block list" to the flow form before the pilot receives it; a test spec that
outlives its decision is the A-E6 defect class.
Swept the same defect class while here: the TDD order and three other passages
wrote okf_version: "0.2" QUOTED, contradicting A-E6, which is precisely the form
catalog measured as exit 1. All five now unquoted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
188 lines
10 KiB
Markdown
188 lines
10 KiB
Markdown
# Execution order — the full roadmap, sequenced
|
||
|
||
Status: approved sequencing across all remaining phases. This doc is the
|
||
connective tissue between the per-phase plans; it owns the *order and the
|
||
reasons*, not the phase detail. Each phase's detail and verification live in its
|
||
own doc:
|
||
|
||
- `docs/plan/phase-1-door-a.md` — DONE, shipped at v0.3.2.
|
||
- `docs/plan/phase-2-doors-b-c.md`
|
||
- `docs/plan/phase-3-configurable-contract.md`
|
||
- `docs/plan/okf-v0.2-alignment.md` — a policy track, not a phase; slots between
|
||
Phases 3 and 4 for the reason given below.
|
||
- `docs/plan/phase-4-node-half.md`
|
||
|
||
Headline order: **Stage 0 (guard gate) → Phase 2 → Phase 3 → OKF v0.2 alignment
|
||
decisions → Phase 4 code**, with Phase 2 split guard-independent-first and
|
||
Phase 4's *coordination* started in parallel from the beginning. The quality
|
||
argument for this order is below — it is not merely the roadmap numbering.
|
||
|
||
## Stage 0 — Guard readiness (gate; do first, cheap)
|
||
|
||
Before any Phase 2 code, confirm `llm-ingestion-guard` 0.2 exists with the
|
||
pinned surface (`prepare_input`, `screen_output`, `okf.import_bundle`,
|
||
disposition enum) and is installable in CI. This is Phase 2's assumptions
|
||
B1+B2; B2 is explicitly "not yet decided", and the guard is a separate repo
|
||
(`open/llm-ingestion-pipeline-security`), so its readiness is partly outside
|
||
this repo's control.
|
||
|
||
Why first: it is the single biggest unknown, it is cheap to check (read the
|
||
guard repo), and its answer decides the *shape* of Phase 2 — run straight
|
||
through, or front-load the guard-independent half while readiness is confirmed
|
||
in parallel. Do not build a bearing dependency into the plan on an unverified
|
||
foundation. If the surface has drifted from what the plan assumes, the whole
|
||
persist-gate design must know now, not at the integration step.
|
||
|
||
Outcome recorded in STATE before Phase 2 code starts.
|
||
|
||
### Stage 0 revisited — measuring guard 0.3.1 before widening the pin
|
||
|
||
The pin held is `>=0.2,<0.3`, which can only ever resolve `v0.2.0`. The guard repo
|
||
gates its own 1.0 on our fixture suite being measured against `v0.3.1`, so this
|
||
measurement is owed outward and is that repo's only named blocker. Order is
|
||
*measure first, widen after*: widening the range before measuring would ship a
|
||
window admitting a version the fixtures never ran against.
|
||
|
||
Procedure — do not vary it, each step exists because of a specific failure mode:
|
||
|
||
1. Scratch venv **outside** `.venv`: install the package `--no-deps`, then the
|
||
guard from the `v0.3.1` tag. Do not touch `pyproject.toml`.
|
||
2. **Assert the resolved version** via `importlib.metadata` *before* running
|
||
anything, and carry it in the reported result. An unmodified tree resolves
|
||
`0.2.0` and runs green — a true claim about 0.2.0 published under a heading
|
||
that says 0.3.1. This step is what makes the result honest.
|
||
3. Run `tests/test_guard_adapter.py` (19 tests, real scanner) and the full suite.
|
||
Watch percent-escapes specifically: the slugger builds filenames from titles,
|
||
and the guard has recorded that as its most likely false positive. Report raw
|
||
URLs, not counts.
|
||
4. **Send the result whichever way it goes**, and state explicitly that 19
|
||
fixture tests are not a corpus — their gate is "does 0.3.1 change our
|
||
verdicts", not "how often does the false positive fire".
|
||
5. Green → one release: `>=0.2,<0.4`, uv tag `v0.3.1`, CLAUDE.md and README
|
||
updated together. Red → no bump, and the finding goes back to the guard.
|
||
|
||
Until this lands, consumers of v0.4.0 resolve guard 0.2.0 (`pyproject.toml:25`).
|
||
That is a silent under-defend: no build fails, so nothing surfaces it. The
|
||
release in step 5 is what closes it.
|
||
|
||
## Phase 2 — Doors B/C (split: guard-independent first)
|
||
|
||
Internal order follows the phase-2 doc's TDD steps, deliberately:
|
||
|
||
1. **Guard-independent half (steps 1–2):** extraction registry (stdlib core
|
||
types, fail-fast on unknown extension and on `[extract]` types without the
|
||
extra) + provenance rendering and filename slugging (pure functions). Zero
|
||
external dependency, maximal test-determinism, reuses the Phase 1 renderer.
|
||
Highest-quality-yield work to start with; validates B3 (`html.parser`
|
||
adequacy) early while it is cheap to change.
|
||
2. **Guard seam (steps 3–5):** Door B against a *stub* guard first (nail the
|
||
disposition control flow deterministically), then integration with the *real*
|
||
guard, then Door C. Stub-first means that when the real-guard integration
|
||
test is the only thing that can newly fail, a failure isolates cleanly to the
|
||
integration boundary. Door B before Door C: C reuses B's guard-gate pattern
|
||
and both reuse Phase 1 primitives.
|
||
|
||
This is where `pyproject` runtime deps become exactly the guard pin — a
|
||
semver-worthy event (propose 0.4.0), its own CHANGELOG entry.
|
||
|
||
## Phase 3 — Configurable bundle contract (only after 2)
|
||
|
||
Extract configurability only once Door A **and** Door B/C both concretely use
|
||
the type/layer/frontmatter/reserved-file constants. Phase 3 abstracts a pattern;
|
||
you need ≥2 real consumers before the right seams are visible. Building the
|
||
config object first would be speculative abstraction — an explicit anti-pattern.
|
||
The proving consumer (`claude-code-llm-wiki`, `strict-v1`) is the concrete
|
||
second profile that proves the abstraction is not single-use.
|
||
|
||
The 2→3 boundary has no painful retrofit: Phase 2 already leaves the seam open —
|
||
Door C v1 deliberately does NOT persist the guard log "because reserved-file
|
||
policy differs per consumer and becomes configurable in Phase 3." This is where
|
||
the contract becomes explicit and frozen — the precondition for a clean Node
|
||
port.
|
||
|
||
## OKF v0.2 support — between 3 and 4, and it is real code
|
||
|
||
Everything through Phase 3 targets OKF v0.1. Upstream published v0.2 on
|
||
2026-07-25, and the standing operator policy is that the library always supports
|
||
the current latest OKF version (see `CLAUDE.md`). So this is a committed
|
||
implementation track, not a decision round:
|
||
`docs/plan/okf-v0.2-alignment.md`.
|
||
|
||
**Why it must land before Phase 4 code.** Phase 4 ports a *frozen, explicit*
|
||
contract and uses the shared fixture suite as the cross-runtime conformance
|
||
oracle. Freezing that contract before v0.2 is supported means freezing a v0.1
|
||
shape into two runtimes instead of one, and the fixture suite would then certify
|
||
the drift rather than catch it. Worse, the Node half would need the same v0.2
|
||
work done twice, in two languages, against a contract that had already been
|
||
declared final. The expensive version of this mistake is discovered after the
|
||
Node half exists.
|
||
|
||
**Why it is not a phase of its own.** It adds no new door and no new stage — it
|
||
adds a *profile*, which is precisely what Phase 3 built the seam for. Support is
|
||
additive: `DEFAULT` (commons') and `STRICT_V1` (the wiki's) keep their contracts
|
||
untouched, and v0.2 arrives as a third profile plus an `OKF_LATEST` alias. This
|
||
is the first time the Phase 3 abstraction is forced by something outside this
|
||
repo rather than by a second consumer, which is the better test of it.
|
||
|
||
**Order inside the track.** Characterization and audit first — they add tests and
|
||
change no behavior. Then the frontmatter model that can carry block lists
|
||
(`sources`, multi-verifier `verified`), then the profile, then the golden
|
||
fixtures that make the support claim testable. The `okf_version: 0.2`
|
||
declaration comes last — **unquoted**, per A-E6 — once the shape has earned it. Commons' answer on
|
||
`DEFAULT`'s `generated` runs in parallel and no longer blocks: the additive
|
||
design took it off the critical path.
|
||
|
||
## Phase 4 — Node half (two clocks)
|
||
|
||
**Coordination clock — start early, parallel to Stages 0–3, but only the
|
||
shape-independent agreements.** Phase 4's hard preconditions are agreements, not
|
||
code: okr's reference-impl lift *in principle*, catalog as re-pin owner,
|
||
guard-as-contract at the Node persist seam, and the linkedin-studio
|
||
non-normalization carve-out can all be initiated early — via coord — so they are
|
||
settled when code starts. The one agreement that must WAIT is the exact
|
||
second-brain *contract shape*: the phase-4 doc ties the catalog-spec
|
||
expressiveness check to Phase 3's split-table step, so do not freeze the
|
||
cross-runtime contract before Phase 3 has proven the profile can express it.
|
||
|
||
**Code clock — last.** The Node/ESM implementation ports a *frozen, explicit*
|
||
contract (Phase 3's output) and uses the shared fixture suite as the
|
||
cross-runtime conformance oracle. Porting before the contract is frozen means
|
||
chasing a moving target and guaranteed drift between halves. Halves share
|
||
contract and fixtures, never code.
|
||
|
||
## Quality spine (holds across every stage)
|
||
|
||
- TDD iron law throughout: no production code without a failing test first.
|
||
- The golden/fixture suite is the backbone — every phase extends it; it is the
|
||
regression oracle and, in Phase 4, the cross-runtime conformance oracle. It
|
||
never goes non-byte-exact.
|
||
- Determinism invariant every phase (explicit `ingested_at`, LF-only,
|
||
byte-exact). Phase 2 adds: the guard *version* is part of the input surface —
|
||
pin it.
|
||
- Boundary grep-gate green every phase: `grep -rn "sanitize\|quarantine\|lexicon"
|
||
src/` empty (guard imports only).
|
||
- One phase = one release with a CHANGELOG entry; the guard dependency landing in
|
||
Phase 2 is the architectural milestone.
|
||
|
||
## Verification — the gate between each handoff
|
||
|
||
These are the objective checks that a stage is done and the next may begin:
|
||
|
||
1. **Enter Phase 2:** guard readiness recorded in STATE — either "0.2 API
|
||
matches + installable in CI" or an explicit "front-load guard-independent
|
||
half; guard integration blocked on <named item>".
|
||
2. **Phase 2 → Phase 3:** Phase 1 golden suite still byte-for-byte; `pyproject`
|
||
runtime deps == exactly `llm-ingestion-guard>=0.2,<0.3`; persist-gate proof
|
||
test green (a fail-secure fixture yields zero new files).
|
||
3. **Phase 3 → Phase 4:** golden suite byte-identical under `DEFAULT`
|
||
(`git diff --stat examples/` empty for the phase); `STRICT_V1` cross-profile
|
||
rejection tests pass both directions; split-table artifact under `docs/`
|
||
reviewed with the operator (contract is frozen and expressive enough for the
|
||
second-brain spec).
|
||
4. **Phase 4 code start:** each shape-independent coordination step has a
|
||
recorded sign-off; the contract-shape agreement is signed off *after* the
|
||
Phase 3 split-table, not before; and the OKF v0.2 open questions V1–V5 are
|
||
each answered or explicitly deferred *in writing*, so the frozen contract
|
||
states its relation to upstream on purpose rather than by omission.
|
||
5. **Throughout:** `mypy --strict src/`, `ruff check .`, `ruff format --check .`
|
||
clean; boundary grep-gate empty.
|