docs(plan): add detailed phase 1-4 implementation plans
Phase 1 (Door A): ingest-spec v1 implementation + §11 golden fixtures, pinned to the current spec revision, with TDD order and byte-exact verification criteria. Phase 2 (Doors B/C): guard-gated inbox and external bundle import. Phase 3: configurable bundle contract via a profile object, default profile locked to the golden suite. Phase 4: zero-dep Node half, coordination-first with owner sign-offs. Each plan carries explicit key assumptions with tests and a verification section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QeqhJpYQyghASjiJo5EhGg
This commit is contained in:
parent
e57afecf72
commit
b0ad2aedfb
4 changed files with 450 additions and 0 deletions
124
docs/plan/phase-1-door-a.md
Normal file
124
docs/plan/phase-1-door-a.md
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
# Phase 1 plan — Door A: ingest-spec implementation + §11 golden fixtures
|
||||
|
||||
Status: approved roadmap phase (see `CLAUDE.md`); this plan details it before any code.
|
||||
Normative basis: `ingest-spec.md` v1 in `portfolio-optimiser-commons` (RFC-2119).
|
||||
This plan was written against spec revision `sha256:959484e1952d28fa…` — if the spec has
|
||||
changed when implementation starts, re-read it first and update this plan.
|
||||
|
||||
## Goal
|
||||
|
||||
A stdlib-only Python implementation of the full ingest-spec contract:
|
||||
manifest → connector (`file`/`sql`/`http`) → deterministic materialization of
|
||||
`ingest-{id}.md` files → index generation — plus the §11 golden fixture suite
|
||||
(byte-exact) that no repo ships today. Zero model calls; zero runtime dependencies.
|
||||
|
||||
## Deliverables
|
||||
|
||||
1. **Manifest layer** — parse + fail-fast schema validation BEFORE any source call
|
||||
(spec §4): `manifest_version == 1`, required top-level fields, polymorphic
|
||||
`source.type`, extraction list validation (id grammar `[a-z0-9][a-z0-9-]*`,
|
||||
unique ids, single-line titles, positive `max_rows`), and the verdict
|
||||
reservation (spec §3): `okf_type` must not be `verdict` (case-insensitive) and
|
||||
generated names must not fall in `promoted-verdict-*` or be `index.md`.
|
||||
Hand-rolled validation (no pydantic — zero-dep rule); typed error hierarchy
|
||||
rooted in `IngestError`.
|
||||
2. **Connectors** (spec §4):
|
||||
- `file`: CSV under `root`, boundary-checked fail-closed path resolution,
|
||||
`utf-8-sig` decoding, streaming row cap, ragged-row rejection.
|
||||
- `sql`: sqlite via `connection_ref` env-var resolution, single read-only
|
||||
SELECT, bool-value rejection, sqlite errors wrapped in typed errors,
|
||||
streaming row cap.
|
||||
- `http`: GET against `base_url` + path, `credential_ref` env-var resolution,
|
||||
hard network opt-in gate (spec §8): refuse fail-fast unless the per-run flag
|
||||
is set; the flag is a run argument, never a manifest field. Transport behind
|
||||
a small seam so tests inject a mock (spec §11: no live sources in tests).
|
||||
3. **Materialization** (spec §5): explicit required `ingested_at`
|
||||
(ISO-8601 UTC `Z`, regex-validated), frontmatter with exactly the seven keys in
|
||||
spec order, body rendering per source type (cell escaping `\` → `\\`, `|` →
|
||||
`\|`, newline → space; integers plain decimal; floats shortest round-trip;
|
||||
SQL NULL → empty string; any other type is an error), `ingest_manifest` stamp
|
||||
`{stem}@{sha256(manifest bytes)[:16]}`, LF-only output with exactly one
|
||||
trailing newline. Replacement semantics: stage everything in memory, run the
|
||||
collision gate (a generated name colliding with an unstamped existing file is
|
||||
an error) BEFORE any mutation, then remove stamped files, write the new set,
|
||||
update the index.
|
||||
4. **Index generation** (spec §6): create with `bundle_summary` when missing;
|
||||
otherwise preserve every unmanaged line byte for byte; managed links
|
||||
`- [{title}](ingest-{id}.md)` appended in extraction order, idempotent by
|
||||
target; on re-materialization remove only links whose target was an
|
||||
ingest-owned file removed this run.
|
||||
5. **Public API** — one entry point, roughly
|
||||
`materialize_bundle(manifest_path, bundle_dir, ingested_at, *, allow_network=False) -> IngestResult`,
|
||||
plus the typed error hierarchy. Exact naming settled during TDD; the shape
|
||||
(three explicit inputs + opt-in flag) is fixed by spec §5/§8.
|
||||
6. **Golden fixtures** (spec §11): `examples/ingest-golden-file/` and
|
||||
`examples/ingest-golden-sql/` (conformance MUSTs), plus
|
||||
`examples/ingest-golden-http/` against mock payloads (documented as the
|
||||
optional extension exercised). Each case: `manifest.json`, `fixture/`,
|
||||
`ingested-at.txt`, `expected-bundle/`. Runnable offline without credentials.
|
||||
7. **Load-bearing test table** (spec §11): one named test per seam — provenance
|
||||
stamping, navigability, verdict reservation, re-ingest layer safety over a
|
||||
promoted verdict, golden byte regression, network gate. (The spec-integrity
|
||||
seam belongs to commons, which owns the spec.)
|
||||
|
||||
## Implementation baseline
|
||||
|
||||
The stricter behaviors named in `CLAUDE.md` are the floor, not options: streaming
|
||||
row caps, `utf-8-sig`, in-memory staging with pre-mutation collision gate,
|
||||
validated `ingested_at`, typed `IngestError`.
|
||||
|
||||
## TDD order
|
||||
|
||||
Each step starts with a failing test:
|
||||
|
||||
1. Manifest validation (schema, id grammar, verdict reservation, fail-fast order —
|
||||
validation error must occur with zero source calls).
|
||||
2. Body renderers (escaping, number forms, NULL, type rejection) as pure functions.
|
||||
3. `file` connector (fixture catalogue; boundary check; BOM; caps).
|
||||
4. `sql` connector (fixture sqlite file; env-var resolution; read-only; caps).
|
||||
5. Materialization + collision gate + replacement semantics.
|
||||
6. Index generation (fresh, preserve, idempotent, removal rules).
|
||||
7. `http` connector + network gate (mock transport).
|
||||
8. Golden fixtures authored, then the byte-for-byte conformance test.
|
||||
9. Load-bearing table completed; each seam test verified red when its seam is
|
||||
detached (mutation check by hand, noted in the test docstring).
|
||||
|
||||
## Key assumptions (each with its test)
|
||||
|
||||
| # | Assumption | Test |
|
||||
|---|---|---|
|
||||
| A1 | Spec v1 unchanged since this plan | Compare `sha256(ingest-spec.md)` against the hash above at implementation start |
|
||||
| A2 | stdlib suffices (`csv`, `sqlite3`, `json`, `hashlib`, `urllib`, `re`) | Import smoke test + CI assert that `pyproject.toml` declares no runtime dependencies |
|
||||
| A3 | Byte determinism holds across runs/platforms | Run materialization twice → `diff -r` empty; golden suite in CI |
|
||||
| A4 | Float shortest round-trip (`repr`) is stable in Python ≥ 3.10 | Guaranteed since Python 3.1; covered by golden bytes anyway |
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Consumer adoption (`portfolio-optimiser-claude`, then `portfolio-optimiser`)
|
||||
is separate coordinated work in those repos — this phase only makes the library
|
||||
adoption-ready. Spec changes, if any prove necessary, go via commons.
|
||||
- Doors B/C, configurable contracts, Node — later phases.
|
||||
- Per-row file splitting, multi-manifest bundles, incremental re-ingest,
|
||||
approval registries — spec-declared extension points, not v1.
|
||||
|
||||
## Verification
|
||||
|
||||
All must pass before the phase is declared done:
|
||||
|
||||
1. `pytest` green, including the golden byte-for-byte comparison for all three
|
||||
fixture cases.
|
||||
2. `mypy --strict src/` clean; `ruff check .` and `ruff format --check .` clean.
|
||||
3. Determinism: run the materializer twice on the same golden inputs into two
|
||||
fresh directories; `diff -r` between them and against `expected-bundle/` is
|
||||
empty.
|
||||
4. Idempotence: re-run over the just-materialized bundle; `diff -r` still empty.
|
||||
5. Layer safety: fixture bundle containing a `promoted-verdict-*` file and its
|
||||
index link → after re-ingest both survive byte-identical (named test).
|
||||
6. Verdict reservation: manifest with `okf_type: Verdict` rejected at validation,
|
||||
before any source call (named test asserts no source I/O happened).
|
||||
7. Network gate: `http` manifest without the opt-in flag fails fast with a typed
|
||||
error; no socket use (mock transport asserts zero calls).
|
||||
8. No wall-clock: `grep -rn "datetime.now\|time.time\|utcnow" src/` returns
|
||||
nothing.
|
||||
9. Zero deps: `pyproject.toml` has an empty runtime dependency list.
|
||||
10. Offline: full test suite passes with network access disabled.
|
||||
123
docs/plan/phase-2-doors-b-c.md
Normal file
123
docs/plan/phase-2-doors-b-c.md
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
# Phase 2 plan — Door B (bundle inbox) + Door C (external bundle import)
|
||||
|
||||
Status: approved roadmap phase (see `CLAUDE.md`); details settled here before code.
|
||||
Depends on: Phase 1 (materialization + index primitives are reused, never duplicated).
|
||||
This phase adds the library's first — and only permitted — runtime dependency:
|
||||
`llm-ingestion-guard>=0.2,<0.3`.
|
||||
|
||||
## Goal
|
||||
|
||||
Two guard-gated persist paths on top of the Phase 1 plumbing:
|
||||
|
||||
- **Door B — bundle inbox:** convert operator-dropped files into OKF concept
|
||||
files. All file-type→text extraction lives here (the guard is text-only).
|
||||
- **Door C — external bundle import:** assess a third-party OKF bundle per
|
||||
concept via the guard's `okf.import_bundle`; merge, materialize, and index
|
||||
only the non-rejected concepts.
|
||||
|
||||
The boundary rule is absolute: the guard answers "is this content safe to
|
||||
persist?"; this library only connects, extracts, materializes, and indexes.
|
||||
No scanning, sanitizing, or quarantine logic is implemented here.
|
||||
|
||||
## Door B — deliverables
|
||||
|
||||
1. **Extraction registry** — file extension → extractor:
|
||||
- Core (stdlib only): `md`/`txt` passthrough, `csv` → markdown table (reuses
|
||||
the Phase 1 renderer), `json` → verbatim inside a fenced block, `html` →
|
||||
text via `html.parser`.
|
||||
- Optional (`[extract]` extra only): `pdf`, `docx`, `xlsx`. Without the extra
|
||||
installed those types are rejected fail-fast with a typed error naming the
|
||||
extra — never a silent skip, never a bundled parser in core.
|
||||
2. **Inbox flow** — an explicit operator command (mirrors the spec §9 HITL rule:
|
||||
never automatic, no watcher, no scheduler):
|
||||
`process_inbox(inbox_dir, bundle_dir, ingested_at, ...) -> InboxResult`.
|
||||
Per file: read bytes → extract text → guard gate → materialize concept →
|
||||
index link. `ingested_at` explicit as in Phase 1; no wall-clock anywhere.
|
||||
3. **Guard gate (persist gate #1):** extracted text passes through the guard
|
||||
(`prepare_input`/`screen_output` bookends with an upload-grade policy) before
|
||||
any write. Disposition handling: blocking dispositions → the file is NOT
|
||||
persisted and is reported in `InboxResult`; quarantine-grade dispositions →
|
||||
reported for operator review, not persisted (a quarantine directory is an
|
||||
extension point, not v1); pass/warn → persisted with the report attached to
|
||||
the result.
|
||||
4. **Inbox provenance layer** — machine-generated files carry the honesty
|
||||
marker, analogous to spec §7: `type`, `title`, `source_file` (inbox-relative
|
||||
name), `source_sha256` (of the original bytes), `ingested_at`, `generated: true`.
|
||||
Filenames namespaced `inbox-{slug}.md` with the slug reduced to the Phase 1
|
||||
id grammar — disjoint from `index.md`, `promoted-verdict-*`, and `ingest-*`.
|
||||
The verdict reservation and the pre-mutation collision gate apply unchanged.
|
||||
|
||||
## Door C — deliverables
|
||||
|
||||
1. **Import flow** — explicit operator command:
|
||||
`import_bundle(source_dir, bundle_dir, ingested_at, *, origin, channel) -> ImportResult`.
|
||||
Reads the external bundle as `{relative path -> text}`, hands it to the
|
||||
guard's `okf.import_bundle(bundle, origin=…, channel=…, allow_reserved=False)`,
|
||||
and merges ONLY concepts whose per-concept result carries no error.
|
||||
2. **Merge + index:** accepted concepts go through the Phase 1 staging/collision
|
||||
gate and index primitives (idempotent linking, curated content preserved).
|
||||
Rejected concepts are reported per concept with the guard's reason — never
|
||||
partially written.
|
||||
3. **Import report:** the guard's log body (`BundleResult.log()`) is returned to
|
||||
the caller in `ImportResult`; persisting it into the bundle is NOT done in v1
|
||||
(reserved-file policy differs per consumer and becomes configurable in
|
||||
Phase 3).
|
||||
|
||||
## Design decisions fixed by this plan
|
||||
|
||||
- Trust follows `origin`, never `channel` (guard doctrine) — callers must supply
|
||||
both; no defaults.
|
||||
- Extraction failure (corrupt file, missing extra) is a typed error on that file;
|
||||
the run continues with the remaining files and the result reports per-file
|
||||
outcomes. A guard FAIL-grade disposition on one file likewise never aborts the
|
||||
whole inbox run.
|
||||
- Determinism: same inbox bytes + same `ingested_at` + same guard version ⇒
|
||||
byte-identical persisted output. Guard verdicts are part of that input surface.
|
||||
|
||||
## TDD order
|
||||
|
||||
1. Extraction registry: core types (fixtures per type), fail-fast on unknown
|
||||
extensions, fail-fast on optional types without `[extract]`.
|
||||
2. Inbox provenance rendering + filename slugging (pure functions).
|
||||
3. Door B flow against a stub guard (in tests only — a test double standing in
|
||||
for the pinned guard API, so persist/reject branches are exercised
|
||||
deterministically; the real guard is exercised in integration tests).
|
||||
4. Door B integration with the real guard: benign fixture persists; a fixture
|
||||
the guard fails-secure on is not persisted.
|
||||
5. Door C flow: mixed fixture bundle (accepted + rejected concepts) → only
|
||||
accepted concepts merged; collision and curated-preservation semantics reused
|
||||
from Phase 1 tests.
|
||||
|
||||
## Key assumptions (each with its test)
|
||||
|
||||
| # | Assumption | Test |
|
||||
|---|---|---|
|
||||
| B1 | Guard 0.2 API matches the pinned surface (`prepare_input`, `screen_output`, `okf.import_bundle`, disposition enum) | Import-and-signature smoke test that fails on upgrade drift |
|
||||
| B2 | Guard is installable where this library's CI runs | Resolve the distribution channel with the operator before implementation starts (risk: not yet decided) |
|
||||
| B3 | `html.parser`-based extraction is adequate for v1 | Golden fixtures for representative HTML; anything richer is explicitly out of scope |
|
||||
| B4 | Guard verdicts are deterministic for fixed input + version | Same fixture run twice → identical `InboxResult` |
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Any security decision logic (scan, sanitize, lexicons, fencing) — guard only.
|
||||
- Quarantine storage/workflow and re-scan over time — extension points; the
|
||||
guard's companion-expectations list is revisited in Phase 3.
|
||||
- Model-assisted enrichment of inbox content — never in the run path.
|
||||
- Watchers, schedulers, implicit triggers.
|
||||
|
||||
## Verification
|
||||
|
||||
1. `pytest` green with the guard installed; `mypy --strict src/`; `ruff check .`
|
||||
and `ruff format --check .` clean.
|
||||
2. `pdf` file without `[extract]` → typed error naming the extra; with
|
||||
`pip install .[extract]` the same file extracts (integration test, may be
|
||||
skipped in environments without the extra — but must run in CI).
|
||||
3. Persist-gate proof: a fixture the guard fails-secure on results in ZERO new
|
||||
files in the bundle and an explicit rejection entry in the result (named
|
||||
test asserts the bundle directory is byte-identical before/after).
|
||||
4. Door C partial-merge proof: mixed fixture → accepted concepts present,
|
||||
rejected absent, index links only for accepted, curated lines untouched.
|
||||
5. Phase 1 golden suite still passes byte-for-byte (no regression from reuse).
|
||||
6. Grep-gate: `grep -rn "sanitize\|quarantine\|lexicon" src/` shows no local
|
||||
security reimplementation (guard imports only).
|
||||
7. `pyproject.toml` runtime dependencies == exactly `llm-ingestion-guard>=0.2,<0.3`.
|
||||
100
docs/plan/phase-3-configurable-contract.md
Normal file
100
docs/plan/phase-3-configurable-contract.md
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
# Phase 3 plan — Configurable bundle contract
|
||||
|
||||
Status: approved roadmap phase (see `CLAUDE.md`); details settled here before code.
|
||||
Depends on: Phases 1–2 (their behavior becomes the *default profile* and must not change).
|
||||
Proving consumer: `claude-code-llm-wiki` with a `strict-v1` profile.
|
||||
|
||||
## Goal
|
||||
|
||||
Everything the first two phases hard-code about what a valid bundle looks like —
|
||||
concept types, layers/path prefixes, frontmatter sets and ordering, reserved-file
|
||||
policy — becomes configuration on a profile object instead of constants. The
|
||||
observable behavior under the default profile is byte-identical to Phase 1/2;
|
||||
the golden suite is the regression harness for that claim.
|
||||
|
||||
## Design
|
||||
|
||||
1. **`BundleProfile`** — a frozen dataclass (stdlib, no config-file DSL in v1;
|
||||
profiles are constructed in code by the consumer):
|
||||
- `type_policy`: closed enum vs. open set; the reserved `verdict` exclusion
|
||||
stays unconditional (it is a spec invariant, not profile config).
|
||||
- `frontmatter_schema`: ordered required keys + optional keys, per concept
|
||||
type where needed (Phase 1's seven-key layer and Phase 2's inbox layer are
|
||||
expressions of this schema).
|
||||
- `path_policy`: layer/path prefixes, filename namespaces, reserved filename
|
||||
patterns (`index.md`, `promoted-verdict-*`, …).
|
||||
- `reserved_files`: per-file required / optional / required-absent policy
|
||||
(e.g. a profile may demand that `log.md` is absent).
|
||||
- `index_policy`: managed-link format and per-level index expectations.
|
||||
2. **Built-in profiles:**
|
||||
- `DEFAULT` (implicit): exactly the ingest-spec v1 + Phase 2 contract.
|
||||
- `STRICT_V1`: the `claude-code-llm-wiki` contract — closed type enum
|
||||
(`Concept`/`Guide`/`Reference`/`Release`), extended required frontmatter
|
||||
(`type`, `title`, `description`, `timestamp`, `layer`, `source_tier`,
|
||||
`source_url`, `source_sha`), layer path prefixes, `log.md` required-absent,
|
||||
root-index metadata.
|
||||
3. **Plumbing change:** materialization, inbox, import, and index generation take
|
||||
an optional `profile` argument defaulting to `DEFAULT`. No behavior branches
|
||||
outside what the profile object expresses.
|
||||
|
||||
## The split table (settled before code)
|
||||
|
||||
The wiki's existing validator mixes three kinds of gates. First implementation
|
||||
step is a written mapping of every gate to exactly one column — the mapping is
|
||||
reviewed with the operator before any porting:
|
||||
|
||||
| Column | Meaning | Goes where |
|
||||
|---|---|---|
|
||||
| Profile | Structural bundle contract (types, keys, paths, reserved files) | This phase, `BundleProfile` |
|
||||
| Guard | Content safety (sanitize, quarantine, fencing, secrets, budgets) | Guard calls — never reimplemented here |
|
||||
| Consumer | Domain logic (source pipelines, promotion, staging) | Stays in the consumer repo |
|
||||
|
||||
## TDD order
|
||||
|
||||
1. `BundleProfile` type + `DEFAULT` profile; refactor Phase 1/2 constants onto
|
||||
it with the full existing test suite as the safety net.
|
||||
2. Golden suite re-run — byte-for-byte identical under `DEFAULT` (the phase's
|
||||
load-bearing regression test).
|
||||
3. `STRICT_V1` profile + fixtures: a wiki-shaped bundle that validates under
|
||||
`STRICT_V1` and is rejected under `DEFAULT` (extra frontmatter keys are fine
|
||||
under preserve-unknown; the *rejection* comes from the missing ingest-spec
|
||||
layer), and an ingest-spec bundle rejected under `STRICT_V1` (missing
|
||||
required keys, `log.md` policy).
|
||||
4. Reserved-file policy tests (required-absent enforcement).
|
||||
5. Split-table review artifact committed under `docs/` and cross-checked against
|
||||
the implemented profile fields.
|
||||
|
||||
## Key assumptions (each with its test)
|
||||
|
||||
| # | Assumption | Test |
|
||||
|---|---|---|
|
||||
| C1 | Default-profile refactor is behavior-neutral | Phase 1 golden suite byte-identical; full Phase 1/2 test suite green, unmodified |
|
||||
| C2 | `strict-v1` requirements are fully enumerable from the wiki's validator | The split table is exhaustive: every wiki gate appears in exactly one column; reviewed with the operator |
|
||||
| C3 | Verdict reservation must survive every profile | Named test: no profile construction can permit `type: verdict` |
|
||||
| C4 | Profiles-in-code (no config files) suffice for the proving consumer | Confirmed with the operator at phase start; config-file loading is an extension point |
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Porting the wiki's security stack — those gates map to the Guard column.
|
||||
- Config-file/DSL profile loading, profile inheritance chains — extension points.
|
||||
- Adopting the second-brain (Node-world) conventions — that contract enters in
|
||||
Phase 4; this phase only ensures the profile object is expressive enough not
|
||||
to block it (checked against the catalog's spec during the split-table step).
|
||||
- Any change to commons' ingest-spec; if profile work reveals spec friction, it
|
||||
is raised in commons, not patched locally.
|
||||
|
||||
## Verification
|
||||
|
||||
1. `pytest` green; `mypy --strict src/`; `ruff check .` and
|
||||
`ruff format --check .` clean.
|
||||
2. Golden suite byte-for-byte under `DEFAULT` — zero fixture changes in the diff
|
||||
(`git diff --stat examples/` empty for this phase).
|
||||
3. `STRICT_V1` fixture bundle validates; cross-profile rejection tests pass in
|
||||
both directions (step 3 above).
|
||||
4. C3 named test: constructing a profile that admits `verdict` raises at
|
||||
construction time.
|
||||
5. The split table exists under `docs/`, every wiki gate row has exactly one
|
||||
column, and the Guard column contains no rows implemented in this repo
|
||||
(grep-gate from Phase 2 still clean).
|
||||
6. Public API check: Phase 1/2 call sites compile unchanged (profile argument is
|
||||
optional) — verified by leaving existing tests untouched.
|
||||
103
docs/plan/phase-4-node-half.md
Normal file
103
docs/plan/phase-4-node-half.md
Normal file
|
|
@ -0,0 +1,103 @@
|
|||
# Phase 4 plan — the Node half (`node/`)
|
||||
|
||||
Status: approved roadmap phase (see `CLAUDE.md`); details settled here before code.
|
||||
Depends on: Phase 3 (the profile object must be able to express the second-brain
|
||||
contract; that expressiveness is checked during Phase 3's split-table step).
|
||||
Hard precondition: the coordination steps below — this phase does not start with
|
||||
code, it starts with agreements.
|
||||
|
||||
## Goal
|
||||
|
||||
A zero-dependency Node/ESM package under `node/` for the second-brain world:
|
||||
bundle check, index generation, inbox split/frontmatter/write, and document
|
||||
conversion (docx/pdf/eml/html → md). Importable as a library AND invokable as a
|
||||
CLI (`node <path> <args>`, exit code + stdout — the established plugin
|
||||
invocation pattern), consumed by vendoring per plugin, not npm publishing.
|
||||
The Python and Node halves share the OKF contract and the fixture suite, never
|
||||
code.
|
||||
|
||||
## Coordination first (ordered, each with an owner sign-off)
|
||||
|
||||
1. **okr agreement.** The reference implementations (`okf-check.mjs`,
|
||||
`okf-index.mjs`, the inbox libraries and vocabulary/link modules) are lifted
|
||||
only in agreement with okr; their test suite is ported alongside the code as
|
||||
the seed corpus. Known deviations between okr's current behavior and the
|
||||
catalog spec are resolved WITH the convention owner before the lift — the
|
||||
library implements the agreed contract, not one repo's drift.
|
||||
2. **Catalog stays convention owner.** The second-brain spec remains in the
|
||||
marketplace catalog; this repo implements it. After the lift, the catalog
|
||||
re-pins its shared check gate to this repo's copy — acceptance is behavior
|
||||
parity on the shared fixture suite, and the re-pin follows the marketplace's
|
||||
release-tag pinning discipline.
|
||||
3. **linkedin-studio stays plugin-local.** Its `ingest/published/`
|
||||
provenance-record grammar has a different lifecycle by design and is NOT
|
||||
normalized into this library. Explicit non-goal, agreed up front.
|
||||
4. **Persist gate = guard-as-contract.** There is no Node guard today. The Node
|
||||
half documents and enforces the guard's *contract* at its persist seam
|
||||
(fail-closed hook point, no content-safety logic of its own) so a future Node
|
||||
guard from the security repo can drop in. No security reimplementation in
|
||||
either runtime.
|
||||
5. **ms-ai-architect as greenfield consumer.** Its designed-but-unbuilt needs
|
||||
(writer, index generator, checker, retrieval support) are the acceptance
|
||||
sketch for the package's API surface.
|
||||
|
||||
## Deliverables
|
||||
|
||||
1. **`node/` package skeleton:** ESM, `package.json` with NO `dependencies`,
|
||||
`node:` builtins only, `node:test` for tests, no build step (vendorable as
|
||||
plain files).
|
||||
2. **Bundle check** (`okf check`): the catalog spec's rules — `type:` required,
|
||||
per-level `index.md` without frontmatter, root `okf_version`, canonical
|
||||
`resource` field, preserve-unknown / tolerate-broken posture.
|
||||
3. **Index generation** (`okf index`): per-level index maintenance matching the
|
||||
reference behavior.
|
||||
4. **Inbox primitives:** split, frontmatter emit/parse, relations, write — the
|
||||
lifted library modules under this repo's test suite.
|
||||
5. **Document conversion** (`okf convert`): docx/pdf/eml/html → md. Conversion
|
||||
parsers cannot be zero-dep; this mirrors the Python `[extract]` pattern —
|
||||
the core package stays dependency-free, the conversion module lazy-loads
|
||||
optional parsers and rejects those types fail-fast when they are absent.
|
||||
Which parsers, and whether they are vendored or peer-installed, is settled in
|
||||
coordination step 1 (okr had concrete candidates).
|
||||
6. **CLI:** one entry (`node/bin/okf.mjs check|index|inbox|convert …`),
|
||||
deterministic stdout, meaningful exit codes, no interactive prompts.
|
||||
7. **Shared fixture suite:** cross-runtime fixtures for the second-brain
|
||||
contract, consumed by BOTH the Node tests and (via the Phase 3 profile) the
|
||||
Python tests, so the halves cannot silently diverge.
|
||||
|
||||
## Key assumptions (each with its test)
|
||||
|
||||
| # | Assumption | Test |
|
||||
|---|---|---|
|
||||
| D1 | okr's reference code + tests transfer cleanly to this repo's layout | Ported suite green before any behavior change; parity run against okr's copy on the shared fixtures |
|
||||
| D2 | The catalog spec is the single contract both halves implement | Phase 3 profile expresses it; cross-runtime fixture suite passes in both runtimes |
|
||||
| D3 | Zero-dep core is compatible with doc conversion | Lazy-load pattern proven by a test run WITHOUT optional parsers installed: core commands work, `convert` fails fast with a clear message |
|
||||
| D4 | Vendoring workflow works for consumers | Dry-run: copy `node/` into a scratch plugin layout, run the CLI from there, exit codes intact |
|
||||
|
||||
## Non-goals
|
||||
|
||||
- npm publishing (vendoring only), bundling/transpiling, TypeScript build chain.
|
||||
- Normalizing linkedin-studio's published-record grammar.
|
||||
- A Node guard, or ANY content-safety logic — guard-as-contract until the
|
||||
security repo ships a Node implementation.
|
||||
- Python↔Node code sharing of any kind (contract and fixtures only).
|
||||
|
||||
## Verification
|
||||
|
||||
1. `node --test node/` green; ported reference tests included.
|
||||
2. Zero-dep proof: `package.json` has no `dependencies`/`optionalDependencies`;
|
||||
grep for non-`node:` imports in core modules returns nothing (conversion
|
||||
module exempted, lazy-load only).
|
||||
3. Parity: this repo's check gate and the currently vendored reference copy
|
||||
produce identical verdicts on the full shared fixture suite (scripted
|
||||
comparison, part of CI until the catalog re-pins).
|
||||
4. Cross-runtime agreement: Python (Phase 3 profile) and Node checkers give the
|
||||
same accept/reject verdict per shared fixture (scripted matrix run).
|
||||
5. CLI contract: each subcommand exercised via `node node/bin/okf.mjs …` in
|
||||
tests; exit codes asserted (0 clean, non-zero on violation), stdout
|
||||
deterministic (two runs diff-empty).
|
||||
6. D3 test: suite run in an environment without optional parsers — core green,
|
||||
`convert` on a docx fails fast naming the missing parser.
|
||||
7. Coordination artifacts: each of the five steps above has a recorded sign-off
|
||||
before its dependent code lands (tracked in the session state, summarized in
|
||||
the commit history).
|
||||
Loading…
Add table
Add a link
Reference in a new issue