llm-ingestion-okf/docs/plan/execution-order.md
Kjell Tore Guttormsen 0e18a6c0a9 docs(plan): add cross-phase execution ordering
Sequences the remaining roadmap (Stage 0 guard gate -> Phase 2 -> Phase 3
-> Phase 4) with the quality rationale for the order: guard readiness
verified before any Phase 2 code, Phase 2 split guard-independent-first,
Phase 3's configurability extracted only after two concrete consumers
exist, Phase 4 code last on a frozen contract with its coordination
started early. Connective tissue over the four per-phase plans; owns the
order and reasons, not the phase detail.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HBbjgS5A55RVavoyjJC4FX
2026-07-24 20:01:45 +02:00

6.5 KiB
Raw Blame History

Execution order — the full roadmap, sequenced

Status: approved sequencing across all remaining phases. This doc is the connective tissue between the per-phase plans; it owns the order and the reasons, not the phase detail. Each phase's detail and verification live in its own doc:

  • docs/plan/phase-1-door-a.md — DONE, shipped at v0.3.2.
  • docs/plan/phase-2-doors-b-c.md
  • docs/plan/phase-3-configurable-contract.md
  • docs/plan/phase-4-node-half.md

Headline order: Stage 0 (guard gate) → Phase 2 → Phase 3 → Phase 4 code, with Phase 2 split guard-independent-first and Phase 4's coordination started in parallel from the beginning. The quality argument for this order is below — it is not merely the roadmap numbering.

Stage 0 — Guard readiness (gate; do first, cheap)

Before any Phase 2 code, confirm llm-ingestion-guard 0.2 exists with the pinned surface (prepare_input, screen_output, okf.import_bundle, disposition enum) and is installable in CI. This is Phase 2's assumptions B1+B2; B2 is explicitly "not yet decided", and the guard is a separate repo (open/llm-ingestion-pipeline-security), so its readiness is partly outside this repo's control.

Why first: it is the single biggest unknown, it is cheap to check (read the guard repo), and its answer decides the shape of Phase 2 — run straight through, or front-load the guard-independent half while readiness is confirmed in parallel. Do not build a bearing dependency into the plan on an unverified foundation. If the surface has drifted from what the plan assumes, the whole persist-gate design must know now, not at the integration step.

Outcome recorded in STATE before Phase 2 code starts.

Phase 2 — Doors B/C (split: guard-independent first)

Internal order follows the phase-2 doc's TDD steps, deliberately:

  1. Guard-independent half (steps 12): extraction registry (stdlib core types, fail-fast on unknown extension and on [extract] types without the extra) + provenance rendering and filename slugging (pure functions). Zero external dependency, maximal test-determinism, reuses the Phase 1 renderer. Highest-quality-yield work to start with; validates B3 (html.parser adequacy) early while it is cheap to change.
  2. Guard seam (steps 35): Door B against a stub guard first (nail the disposition control flow deterministically), then integration with the real guard, then Door C. Stub-first means that when the real-guard integration test is the only thing that can newly fail, a failure isolates cleanly to the integration boundary. Door B before Door C: C reuses B's guard-gate pattern and both reuse Phase 1 primitives.

This is where pyproject runtime deps become exactly the guard pin — a semver-worthy event (propose 0.4.0), its own CHANGELOG entry.

Phase 3 — Configurable bundle contract (only after 2)

Extract configurability only once Door A and Door B/C both concretely use the type/layer/frontmatter/reserved-file constants. Phase 3 abstracts a pattern; you need ≥2 real consumers before the right seams are visible. Building the config object first would be speculative abstraction — an explicit anti-pattern. The proving consumer (claude-code-llm-wiki, strict-v1) is the concrete second profile that proves the abstraction is not single-use.

The 2→3 boundary has no painful retrofit: Phase 2 already leaves the seam open — Door C v1 deliberately does NOT persist the guard log "because reserved-file policy differs per consumer and becomes configurable in Phase 3." This is where the contract becomes explicit and frozen — the precondition for a clean Node port.

Phase 4 — Node half (two clocks)

Coordination clock — start early, parallel to Stages 03, but only the shape-independent agreements. Phase 4's hard preconditions are agreements, not code: okr's reference-impl lift in principle, catalog as re-pin owner, guard-as-contract at the Node persist seam, and the linkedin-studio non-normalization carve-out can all be initiated early — via coord — so they are settled when code starts. The one agreement that must WAIT is the exact second-brain contract shape: the phase-4 doc ties the catalog-spec expressiveness check to Phase 3's split-table step, so do not freeze the cross-runtime contract before Phase 3 has proven the profile can express it.

Code clock — last. The Node/ESM implementation ports a frozen, explicit contract (Phase 3's output) and uses the shared fixture suite as the cross-runtime conformance oracle. Porting before the contract is frozen means chasing a moving target and guaranteed drift between halves. Halves share contract and fixtures, never code.

Quality spine (holds across every stage)

  • TDD iron law throughout: no production code without a failing test first.
  • The golden/fixture suite is the backbone — every phase extends it; it is the regression oracle and, in Phase 4, the cross-runtime conformance oracle. It never goes non-byte-exact.
  • Determinism invariant every phase (explicit ingested_at, LF-only, byte-exact). Phase 2 adds: the guard version is part of the input surface — pin it.
  • Boundary grep-gate green every phase: grep -rn "sanitize\|quarantine\|lexicon" src/ empty (guard imports only).
  • One phase = one release with a CHANGELOG entry; the guard dependency landing in Phase 2 is the architectural milestone.

Verification — the gate between each handoff

These are the objective checks that a stage is done and the next may begin:

  1. Enter Phase 2: guard readiness recorded in STATE — either "0.2 API matches + installable in CI" or an explicit "front-load guard-independent half; guard integration blocked on ".
  2. Phase 2 → Phase 3: Phase 1 golden suite still byte-for-byte; pyproject runtime deps == exactly llm-ingestion-guard>=0.2,<0.3; persist-gate proof test green (a fail-secure fixture yields zero new files).
  3. Phase 3 → Phase 4: golden suite byte-identical under DEFAULT (git diff --stat examples/ empty for the phase); STRICT_V1 cross-profile rejection tests pass both directions; split-table artifact under docs/ reviewed with the operator (contract is frozen and expressive enough for the second-brain spec).
  4. Phase 4 code start: each shape-independent coordination step has a recorded sign-off; the contract-shape agreement is signed off after the Phase 3 split-table, not before.
  5. Throughout: mypy --strict src/, ruff check ., ruff format --check . clean; boundary grep-gate empty.