linkedin-studio/docs/second-brain/plan-sb-s3b.md
Kjell Tore Guttormsen d2ec7482be docs(linkedin-studio): SB-S3b brief + plan — supersede arm (light-Voyage hardened)
Second sub-slice of SB-S3 (after S3a's reader landed). Adds an operator-gated,
explicitly-signalled supersede op to the consolidation engine: a temporal-update
candidate retires a stale fact (status: superseded, re-minted to an archival id,
replaced in place, retained as audit) and installs the new winner under the
canonical key-id. Bounded to scripts/brain/ — pure, deterministic, fully
unit-testable.

Operator-confirmed scope: temporal-update→supersede ONLY · winner→dynamic
(re-earn) · minimal `supersedes?: string` · retain superseded as audit.

Light-Voyage-hardened:
- brief-reviewer APPROVE-WITH-FIXES — 8 FIX folded (the load-bearing one: id
  lifecycle → winner-takes-key-id, retired fact re-minted, so mintEntityId(key)
  always points at the live fact).
- plan-critic REVISE → 2 BLOCK + 6 FIX folded: applyDiff must REPLACE-IN-PLACE
  (not copy → would dup the canonical key-id); proposeDiff must value-guard the
  supersede fork (a re-sent stale signal would self-supersede every run); plus
  --gather active-only filter + decay excludes superseded (else retired facts
  leak back as live context / spam stale flags).
- scope-guardian ALIGNED (0 creep / 0 gap).

Also reconciles brief-sb-s3a.md's stale DRAFT status (S3a landed 4fa411f).
No code yet — awaiting go-before-code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RigJBiRFNtFZKCz21qNbQ4
2026-06-23 18:51:25 +02:00

101 lines
14 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Plan — SB-S3b: the supersede arm
> Brief: `docs/second-brain/brief-sb-s3b.md`. Slice: SB-S3b.
> **Operator-confirmed (2026-06-23):** temporal-update→supersede ONLY · winner→dynamic (re-earn) · minimal `supersedes?: string` · retain superseded as audit.
> **Light-Voyage:** brief-review APPROVE-WITH-FIXES (8 folded); plan-critic **REVISE → 2 BLOCK + 6 FIX folded below**; scope-guardian **ALIGNED** (0 creep/0 gap, 1 minor SC9 reconciliation folded).
> Order is TDD: failing engine unit tests land BEFORE the engine logic (iron law); each rule pinned before the CLI is wired (`brief-sb-s2.md:115`).
## Goal
The consolidation engine can **retire** a fact via an operator-confirmed, explicitly-signalled supersede: a temporal-update candidate flips the stale fact to `status: "superseded"` (re-minted to an archival id, **replaced in place**, retained as audit) and installs the new winner under the canonical key-id. Pure, idempotent, deterministic; rides the existing `--propose`/`--apply --confirm` gate; superseded facts never leak back into gather or decay; no grammar change; fully unit-testable.
## Files touched (exhaustive — for scope-guardian)
| File | Change | SC |
|------|--------|-----|
| `scripts/brain/src/consolidate.ts` | `Candidate` (+`supersedes?: string`, 33-39) · `ProfileDiff` (+`supersedes: SupersedeOp[]`, 41-47; `SupersedeOp` carries `provenance`) · `proposeDiff` routing fork before keep-both (~120), **value-guarded** + intra-batch `touched`-guard · `applyDiff` **replace-in-place** supersede branch, state-checked, ordered (144-162) · decay scan excludes `superseded` (132-134) · local `archivalId()` helper · engine docstring `:11` | SC1SC5,7,8,11,12 |
| `scripts/brain/src/cli.ts` | `renderDiffMd` (+`## Supersessions (old → new)`, appended last via `section()`, 132-145) · `validateCandidates` (+optional single-line `supersedes`, 147-157) · **`--gather` profileFacts filtered to `status === "active"`** (`:174`) | SC6, SC10 |
| `scripts/brain/tests/consolidate.test.ts` | unit tests SC1, SC2, SC3a, SC3b, SC4, SC5, SC7, SC8, SC11, SC12 | engine |
| `scripts/brain/tests/consolidate-cli.test.ts` | CLI tests SC6 (render + fidelity non-vacuity + `--confirm` gate), SC10 (gather active-only) | CLI |
| `scripts/test-runner.sh` | bump `BRAIN_TESTS_FLOOR` 82→(82+N) + breakdown comment (`:716`); **`ASSERT_BASELINE_FLOOR` UNCHANGED at 80** | SC9 |
| `docs/second-brain/consolidation-loop.md` | reconcile "no supersede" claims (`:34`, `:71`) to true state + honest-limit line (engine guarantees mechanics; temporal-update-vs-contradictory is session+operator judgement — the operator gate is the only classification net) | doc |
**Not touched (scope fence):** `types.ts` (`FactStatus` already has `superseded`) · `profile.ts` (grammar already round-trips `superseded`) · `id.ts` (reuse `mintEntityId`/`mintContentId`; `archivalId` is local to `consolidate.ts`) · `ingest.ts`/`scaffold.ts`/`dataRoot.ts` · `hooks/scripts/**` · `agents/**` (S3a reader unchanged) · `architecture.md` (its `:69` design promise is fulfilled; `:79` "S2 had no supersede" stays historically true — arc-status note deferred, not load-bearing) · condition-dependent/distractor branches (Q-SCOPE).
## Step 0 — pre-flight (verify baseline, no edits)
`(cd scripts/brain && npm install)` if `node_modules` absent. Confirm `(cd scripts/brain && npm test)` = **82/82** and `bash scripts/test-runner.sh` green (95/0/0, `BRAIN_TESTS_FLOOR=82`, `ASSERT_BASELINE_FLOOR=80`). Confirm `renderDiffMd`/`--gather`/CLI-propose coverage lives in `consolidate-cli.test.ts` (host SC6/SC10 there). Re-read `applyDiff` (`:144-162`) to see how promotions are removed via `filter` (`:156`) — the supersede branch mirrors that.
## Step 1 — (RED) types + failing tests
1. **Type scaffolding only** (suite still compiles, existing 82 green):
- `Candidate` (`:33-39`): `supersedes?: string`.
- `ProfileDiff` (`:41-47`): `supersedes: SupersedeOp[]`, with `interface SupersedeOp { oldId: string; oldValue: string; newId: string; newValue: string; provenance: Provenance }`.
- `proposeDiff` returned object: initialise `supersedes: []` (no routing yet).
- `applyDiff`: unchanged for now (ignores the new array).
2. **Add failing unit tests** (mirror the SC1e keep-both test at `:78-91`): SC1, SC2, SC3b, SC4, SC8, SC11, SC12 → expected FAIL; SC3a, SC5, SC7 → guards (may pass). SC6, SC10 in `consolidate-cli.test.ts` → FAIL.
**RED gate (single, unambiguous):** run **`(cd scripts/brain && npm test)` directly** → the new behavioural tests fail, the 82 existing pass. **Do NOT run `bash scripts/test-runner.sh` at RED** — Section 16b (`test-runner.sh:717`) fails the whole gate on a non-zero brain exit, which is design-noise, not signal. The direct brain-suite RED is the failing-test proof. (The gate is run GREEN only, after Step 4.)
## Step 2 — (GREEN) engine logic
1. **`proposeDiff` routing fork** — BEFORE the keep-both branch (`~:120`), after the ai-draft reject (`:105`, so a supersede can never carry ai-draft provenance — confirmed):
- If `c.supersedes` is set: `targetId = mintEntityId({kind:"observed", key: c.supersedes})`; `target = byId.get(targetId)`.
- Route to `supersedes[]` **only when** `target` exists AND `target.status === "active"` AND `target.value !== c.value` AND `targetId` not `touched` this batch. **The `target.value !== c.value` guard is load-bearing** (plan-critic BLOCK 2): a re-sent stale signal whose target already holds the candidate's value is NOT a supersession — it falls through to the ordinary bump path, so SC3a holds.
- On route: push `{ oldId: targetId, oldValue: target.value, newId: mintEntityId({kind:"observed", key: c.key}), newValue: c.value, provenance: c.provenance }`; `touched.add(targetId)` + `touched.add(newId)` (intra-batch guard — a later same-key candidate degrades; winner id reserved). Do **not** also push to `additions`. `continue`.
- Otherwise (no target / not active / value already equal / touched): fall through to the ordinary add/bump path (SC4 graceful no-target, SC8 intra-batch). A supersede only retires a fact present at batch start (`byId` built once from `current`).
2. **`archivalId()` helper** (local to `consolidate.ts`): `archivalId(oldId, oldValue) = mintContentId(\`superseded:${oldId}::${oldValue}\`)`. Seeding with the retired fact's **pre-archival id** (unique among active facts by the no-dup invariant) makes the archival id collision-free in practice (plan-critic NICE 9 — stronger than a key+date seed).
3. **`applyDiff` supersede branch — REPLACE IN PLACE, not copy** (`:144-162`; plan-critic BLOCK 1). Explicit ordering:
- (a) Compute `passing = diff.supersedes.filter(s => { const f = factById(current, s.oldId); return f && f.status === "active" && f.value === s.oldValue })`. The **`f.value === oldValue` state-check** is the idempotency + stale-diff guard: on re-apply the winner holds `oldId` with `newValue ≠ oldValue` → not passing → skipped (SC3b); a profile mutated under the diff → skipped (R3). Build `retireById = new Map(passing.map(s => [s.oldId, s]))`.
- (b) When mapping `current.static` and `current.dynamic`: a fact whose `id ∈ retireById` is **transformed in place** to `{ ...f, id: archivalId(f.id, f.value), status: "superseded" }` (keeps its layer + its OWN `provenance`/`first_seen` — a `published→human` update must NOT rewrite the archived fact's provenance, plan-critic FIX 3). It is **excluded from bump/promote** (a superseded fact is never bumped or promoted — resolves the ordering hazard, plan-critic FIX 6: supersede-wins over a same-id bump/promote in one diff).
- (c) Append each winner `{ id: newId, value: newValue, first_seen: today, last_seen: today, evidence_count: 1, provenance, status: "active" }` to **dynamic** (Q-LAYER re-earn). Because (b) vacates `oldId` and the winner re-occupies it (when `key===supersedes`), there is exactly one fact at the canonical key-id (the winner) + one archival fact — **no duplicate id**. Winner `first_seen: today` is intentional (a genuinely new observation; the original first-observation date lives on the archived copy — plan-critic FIX 8a).
- `applyDiff` stays **pure** (new `ProfileDoc`); existing bump/promote/addition handling preserved.
4. **Decay scan excludes superseded** (`consolidate.ts:132-134`; plan-critic FIX 8c): add `f.status !== "superseded"` to the `current.dynamic` decay filter so a retained dynamic archival fact never throws perpetual `staleFlags`. (Static archival facts are already decay-exempt — dynamic-only scan.)
**GREEN gate:** `(cd scripts/brain && npm test)` → all engine SCs pass; existing 82 pass.
## Step 3 — (GREEN) CLI render + validation + gather filter
1. `renderDiffMd` (`cli.ts:132-145`): add a `## Supersessions (old → new)` section, **appended LAST** (after Stale, `:143`) via the existing `section()` helper, **only when `supersedes.length > 0`** — one line per entry `- \`<oldValue>\` → \`<newValue>\``. Zero supersessions → no section → byte-identical to today (plan-critic FIX 5).
2. `validateCandidates` (`:147-157`): if `supersedes` present, assert non-empty single-line string; absent is valid.
3. **`--gather` active-only filter** (`cli.ts:174`; plan-critic FIX 8b): `profileFacts` emitted to the extraction session is filtered to `status === "active"` — superseded archival facts must NOT be re-presented as live context (else the slice re-introduces the retired values it exists to retire). One-line filter.
4. **CLI tests** (`consolidate-cli.test.ts`): SC6 — (a) `--propose` writes one `## Supersessions` line per entry; (b) **render-fidelity non-vacuity** — 2 entries → 2 lines, 0 entries → no section (decoy); (c) `--apply` without `--confirm` refuses (exit 1, no write); with `--confirm` writes superseded status. SC10 — `--gather` output excludes superseded facts.
## Step 4 — gate floor + doc reconciliation
1. `test-runner.sh:716`: `BRAIN_TESTS_FLOOR` 82 → **82 + N** (N = the EXACT counted new brain tests from Steps 13 — set after counting, never guessed). Extend the breakdown comment `… + SB-S2 19 + SB-S3b N`. **`ASSERT_BASELINE_FLOOR` stays 80** — no new *unconditional `test-runner.sh` structure-lint* check is added; the render-fidelity check is a brain-suite test counted by `BRAIN_TESTS_FLOOR`. (This reconciles brief SC9/§3.7 "ASSERT bump" wording: the render-fidelity check lives in the brain suite, so `BRAIN_TESTS_FLOOR` is the only floor that moves — see SC9 below.)
2. `consolidation-loop.md`: edit `:34` + `:71` to the true state — temporal-update supersede landed in SB-S3b (explicit signal, operator-gated, replace-in-place, audit-retained); contradictory still keep-both; decay still advisory. Add the honest-limit line (engine guarantees mechanics; the temporal-update-vs-contradictory call is session+operator judgement).
3. `consolidate.ts:11` docstring: update the "no supersede in S2 — that's S3" line to note S3b implements temporal-update supersede.
## Step 5 — Land
STATE "Telling" + "👉 NESTE" updated (S3b done → S3c/S3d remain); brief+plan committed as docs alongside code. Commit (Conventional; `[skip-docs]` on code/feat; brief+plan+consolidation-loop+docstring are docs → feat-dominant `[skip-docs]`). **Push only inside the window** (`date '+%u %H:%M'` first) and **confirm before push** (`origin` is PUBLIC `open/`). No version bump (additive within v0.5.2 dev).
## Verification (testable)
| SC | Check | Expected |
|----|-------|----------|
| SC1 | routing | supersede candidate (target value differs) → 1 `supersedes[]` entry, 0 keep-both |
| SC2 | mutation+ids | winner id=`mintEntityId(c.key)`, active, ev=1, value=new; retired→`status:superseded`, id=archival≠key-id, OWN provenance/first_seen kept; both present (no dup at key-id) |
| SC3a | propose-idempotent | post-land re-propose (value now matches) → 0 `supersedes[]`, 1 bump — incl. when a stale signal persists (value-guard) |
| SC3b | apply-idempotent | re-apply same diff → value-match skip → no double winner, no re-supersede, no dup id |
| SC4 | graceful no-target | signal + no active fact at key-id → plain add, no throw |
| SC5 | round-trip | `parse(serialize(applyDiff(...)))` deep-equals `applyDiff(...)` (superseded fact from the real path) |
| SC6 | CLI render + gate | `## Supersessions` line/entry; 0 entries → no section (byte-identical); `--apply` needs `--confirm` |
| SC7 | no regression | contradictory still keep-both; bump/promote/published-only unchanged |
| SC8 | intra-batch | two candidates same key / supersede-a-batch-created-fact → deterministic, no dup id |
| SC9 | gate green | `bash scripts/test-runner.sh` green; brain suite ≥ new `BRAIN_TESTS_FLOOR`; **`ASSERT_BASELINE_FLOOR` unchanged at 80** |
| SC10 | gather active-only | `--gather` output excludes `status:superseded` facts |
| SC11 | decay excludes superseded | a retained dynamic superseded fact throws no `staleFlags` |
| SC12 | supersede-wins ordering | a diff that bumps/promotes AND supersedes the same id → supersede wins (superseded fact not bumped/promoted) |
| (red proof) | failing-test-first | `(cd scripts/brain && npm test)` BETWEEN Step 1 and Step 2 → new SCs fail, 82 pass |
## Risks
- **R1 — id corruption (plan-critic BLOCK 1).** `applyDiff` must **replace in place** (filter+map, mirroring promotions `:156`), never copy — else two facts share the canonical key-id. Idempotency/stale guard keys off `oldId` AND `oldValue`. Pinned by SC2 + SC3b.
- **R2 — propose self-supersede (plan-critic BLOCK 2).** A persisted signal whose target already holds the candidate's value must NOT route to supersede (value-guard); else spurious archival churn every run. Pinned by SC3a.
- **R3 — superseded facts leak back.** `--gather` (FIX 8b) and decay (FIX 8c) must filter `status:active`/exclude `superseded` — else the slice regresses its own goal / spams stale flags. Pinned by SC10 + SC11.
- **R4 — stale diff vs mutated profile.** Value-matched state-check skips. Pinned by SC3b/SC4.
- **R5 — render JSON↔MD divergence.** SC6 render-fidelity non-vacuity (every entry → a line; 0 → no section).
- **R6 — floor-count drift.** `BRAIN_TESTS_FLOOR` bumped by the EXACT counted N; `ASSERT_BASELINE_FLOOR` deliberately unchanged at 80 (no new unconditional structure-lint check). Verified by the green gate.
- **R7 — unguarded doc.** No lint protects `docs/`; `consolidation-loop.md` reconciled by discipline (as the S3a `:67` edit).