linkedin-studio/docs/second-brain/brief-sb-s3b.md
Kjell Tore Guttormsen d2ec7482be docs(linkedin-studio): SB-S3b brief + plan — supersede arm (light-Voyage hardened)
Second sub-slice of SB-S3 (after S3a's reader landed). Adds an operator-gated,
explicitly-signalled supersede op to the consolidation engine: a temporal-update
candidate retires a stale fact (status: superseded, re-minted to an archival id,
replaced in place, retained as audit) and installs the new winner under the
canonical key-id. Bounded to scripts/brain/ — pure, deterministic, fully
unit-testable.

Operator-confirmed scope: temporal-update→supersede ONLY · winner→dynamic
(re-earn) · minimal `supersedes?: string` · retain superseded as audit.

Light-Voyage-hardened:
- brief-reviewer APPROVE-WITH-FIXES — 8 FIX folded (the load-bearing one: id
  lifecycle → winner-takes-key-id, retired fact re-minted, so mintEntityId(key)
  always points at the live fact).
- plan-critic REVISE → 2 BLOCK + 6 FIX folded: applyDiff must REPLACE-IN-PLACE
  (not copy → would dup the canonical key-id); proposeDiff must value-guard the
  supersede fork (a re-sent stale signal would self-supersede every run); plus
  --gather active-only filter + decay excludes superseded (else retired facts
  leak back as live context / spam stale flags).
- scope-guardian ALIGNED (0 creep / 0 gap).

Also reconciles brief-sb-s3a.md's stale DRAFT status (S3a landed 4fa411f).
No code yet — awaiting go-before-code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RigJBiRFNtFZKCz21qNbQ4
2026-06-23 18:51:25 +02:00

19 KiB
Raw Permalink Blame History

Brief — SB-S3b: the supersede arm

Slice: SB-S3b (second sub-slice of SB-S3). Status: DRAFT — awaiting operator "go" before any code. Light-Voyage: brief-review APPROVE-WITH-FIXES (8 FIX folded; 3 RISK carried to the plan). Predecessors: SB-S0 (id/profile substrate — reserved the superseded status) · SB-S1 (published gold) · SB-S2 (consolidation motor — keep-both, no supersede) · SB-S3a (first reader strategy-advisor — LANDED 4fa411f, gate 95/0/0, ASSERT floor 80). Grounded in: a read-only code map of scripts/brain/ (file:line cited throughout) + a brief-review pass against the same code.

1. Operator decision (2026-06-23)

SB-S3 was decomposed into four sub-slices; after S3a (reader) landed, the recommended order is S3b (supersede) → S3c (id-threading) → S3d (hygiene+ops, last). S3b is next because it has the lowest blast radius (bounded to the scripts/brain/ TS package — pure functions, fully unit-testable, unlike S3a's behavioural SCs), it completes the consolidation motor's one explicit TODO (consolidate.ts:11: "no supersede in S2 — that's S3"), and it strengthens the read path S3a just shipped (a profile with supersede gives the new reader cleaner, non-contradictory facts).

2. Why supersede, and the central design decision

S2's engine handles a value-conflict by keep-both: the old fact is left untouched and a new alt fact is added with a distinct content-id (consolidate.ts:120-129). That is correct for a genuine contradiction (two views coexist), but wrong for a temporal update — when a new fact replaces an outdated one (the user changed roles, a number moved, a stance evolved). Today there is no way to retire the stale fact; it persists forever, and the S3a reader has to weigh a fact the user has already moved past.

The superseded status was reserved at SB-S0 (types.ts:17, :23; grammar profile.ts:58; round-trip proven profile.test.ts:35) precisely for this — but nothing ever sets it. S3b is the code that does.

Central design decision (the "fuzzy heart", brief-sb-s2.md:115) — supersede is driven by an EXPLICIT signal, never engine auto-classification. The S2 architecture splits cleanly: the session does the fuzzy extraction, the engine is deterministic mechanics, the operator gates the diff before apply. Classifying temporal-update vs contradictory is a fuzzy judgement — so it stays where the fuzziness already lives (the session's candidate extraction), not baked into a pure function. A candidate carries an explicit "this supersedes the prior fact for key X" intent; proposeDiff deterministically routes it; the operator sees and confirms the supersession in the rendered diff before --apply --confirm writes it. This honours the S2 promise (brief-sb-s2.md: "reserve supersede for an explicit confirm signal") and keeps the engine deterministic and testable.

Consequence (the engine has no classification safety net). Because the engine never validates that a supersede should be a supersede, the operator gate is the only place a mis-classification is caught — and supersede is the first op that retires state (a mis-call silently retires a fact that should have been kept-both, and status: superseded makes it look intentional). Two obligations follow: SC6's rendering must make every supersession visually unmistakable (old value → new value, both shown, flagged as a retirement), and the doc reconciliation (§3.7) must add an honest-limit line mirroring consolidation-loop.md:64the engine guarantees supersede mechanics; whether a contradiction is really a temporal update is the session's + operator's judgement.

3. Scope — what is IN (S3b)

  1. Candidate supersede signal. Extend the transient Candidate (consolidate.ts:33-39) with an explicit, optional supersede intent — minimal shape: supersedes?: string (the key of the active prior fact this candidate replaces). Set only by the session when it judges a temporal update; absent for ordinary candidates (fully backward-compatible). validateCandidates (cli.ts:147-157) gains an optional single-line check for the new field.

  2. A sixth diff array. ProfileDiff (consolidate.ts:41-47) gains supersedes: { oldId, oldValue, newId, newValue }[] (parallel to the existing conflicts[] — the five-array shape means a new op is a new array, not a union edit).

  3. proposeDiff routing (deterministic), with a precise matching rule. "Matching active prior fact" = the fact returned by byId.get(mintEntityId({kind:'observed', key})) — i.e. the live fact at the canonical key-id (see §3.5: retired facts no longer hold the key-id, so this is unambiguously the active one). When a candidate carries supersedes: <key> AND that lookup finds a fact, route to supersedes[]. When the signal is present but no fact is found at the key-id, degrade gracefully to a plain add (defined, tested behaviour — never a crash). Candidates without the signal are untouched (keep-both/bump/promote/decay exactly as S2).

  4. applyDiff mutation (the first superseded writer), state-checked. Add a branch (consolidate.ts:144-162) that, for each supersedes[] entry: (a) re-mints the retired fact to a distinct archival id and sets its status: "superseded"; (b) installs the new winner under the canonical key-id (see §3.5). The branch is state-checked, not blind: a supersedes[] entry whose oldId is absent from current or already superseded is skipped (this is what makes re-apply idempotent and guards stale diffs — §5). Superseded facts are retained in the doc (audit), per architecture.md:69 ("keep old status: superseded + date") — never deleted. Stays a pure function returning a new doc.

  5. Id discipline — winner takes the key-id; retired fact is re-minted off it. Because mintEntityId is value-independent (id.ts:33), only one fact may hold the canonical key-id, and it must be the live one. So on supersede: the new winner takes mintEntityId({observed, key}) (the canonical key-id), and the retired fact is re-minted to a distinct archival id (mintContentId over an archival seed, e.g. superseded:${key}::${oldValue}::${oldFirstSeen}, so it is stable and collision-free). This preserves the no-duplicate-id invariant AND keeps mintEntityId(key) always pointing at the live fact — so every downstream match (future same-key candidates, the S3a reader) resolves to the winner exactly as before supersede existed. (This inverts the naïve "keep old id, mint winner fresh", which would leave the key-id on a retired fact and resurrect it on the next same-key candidate — the brief-review's [FIX 1].)

  6. CLI rendering, non-vacuously checked. renderDiffMd (cli.ts:132-145) gains a ## Supersessions (old → new) section so the operator sees every retirement before confirming. No new subcommand — supersede rides the existing --propose / --apply --confirm flow; the sole profile.md writer stays cli.ts:215. Because the operator confirms via the JSON diff while reading the MD, a brain-suite test (consolidate-cli.test.ts, repo non-vacuity idiom) asserts every supersedes[] JSON entry produces a corresponding rendered MD line and a 0-entry diff renders no section — not merely that a section exists (RISK C). (This is a brain-suite test, not a test-runner.sh Section — so it is counted by BRAIN_TESTS_FLOOR, not by a new unconditional structure-lint check; see §3.7.)

  7. Gate + doc reconciliation. Verified baseline (S3a landed): BRAIN_TESTS_FLOOR = 82 (test-runner.sh:716), ASSERT_BASELINE_FLOOR = 80 (:913). Bump BRAIN_TESTS_FLOOR by the new brain-test count; ASSERT_BASELINE_FLOOR stays 80 — no new unconditional test-runner.sh structure-lint check is added (the render-fidelity check lives in the brain suite, §3.6). BRAIN_TESTS_FLOOR is the only floor that moves. Reconcile the now-stale "no supersede" claims (consolidation-loop.md:34, :71; consolidate.ts:11 docstring) to the true state, and add the honest-limit line from §2.

  8. --gather active-only filter (plan-critic FIX 8b — completeness). runConsolidate --gather (cli.ts:174) emits profileFacts to the extraction session with no status filter today. After supersede lands, that would re-present status:superseded archival facts as live context — re-introducing the exact retired values this slice exists to retire (§2). The gather output is filtered to status === "active" (one line). Without this the slice regresses its own goal.

  9. Decay excludes superseded (plan-critic FIX 8c — completeness). A retained dynamic superseded fact is still in current.dynamic, so the decay scan (consolidate.ts:132-134) would throw staleFlags for it forever. The decay filter excludes status:superseded (static archival facts are already decay-exempt — dynamic-only scan). Supersede and decay stay orthogonal.

4. Non-goals — what is OUT (deferred)

  • Engine auto-classification of temporal-update vs contradictory. The signal is explicit (§2). The engine never guesses.
  • The other two contradiction-typology branchescondition-dependent (scope) and distractor (drop) (architecture.md:69, research/secondbrain-sota.md:37). S3b ships temporal-update → supersede ONLY; contradictory stays keep-both (S2). The other two are later, purely-additive arrays (deferring them leaves no broken half-state — confirmed by brief-review).
  • Auto-demotion of stale facts. Decay (staleFlags, consolidate.ts:132-134) stays advisory — never mutates status or removes a fact. Supersede and decay stay orthogonal (no shared field). A stale fact is not auto-superseded.
  • Cross-silo id-threading (S3c) · content-history retirement / operations.md / triple-post reconciliation (S3d) · any new reader (S3a's reader unchanged; content-optimizer is later).
  • Chained / multi-fact supersede as a FEATURE. One active prior → one new winner per candidate. Chains (A→B→C in one pass) and one-supersedes-many are out. But the intra-batch SEMANTICS are defined, not undefined (RISK A): the supersede fork carries a touched-style guard (analogous to consolidate.ts:97) so that within one candidate batch, two candidates targeting the same key, or a candidate superseding a fact another candidate just created, resolve deterministically (first-writer-wins + the later one degrades to add/bump) rather than corrupting. Deferring the feature is fine; the batch rule is in scope.
  • Grammar changes. None needed — superseded is already accepted end-to-end.

5. Boundaries / invariants (must hold)

  • Operator-gated, single disk-writer. No new write path to disk; --apply --confirm (cli.ts:215) stays the sole profile.md writer; --apply refuses without --confirm.
  • Pure, idempotent engine. proposeDiff/applyDiff stay pure (no I/O). Idempotency holds at BOTH layers: proposeDiff produces no new supersedes[] entry once a supersession has landed (winner matches by value → bump; old fact already superseded), and applyDiff's supersede branch is state-checked so re-applying the same diff does not double-install the winner or double-flip status.
  • Stale-diff safety (named assumption + guard). A pending diff is meant to be applied against the profile it was proposed from; a profile mutated between propose and apply is operator error. S3b does not silently corrupt in that case: applyDiff skips any supersedes[] entry whose oldId is absent or already superseded in current (the §3.4 state-check), rather than retiring the wrong fact.
  • Audit-preserving. Superseded facts are retained with status: superseded, never deleted — history is inspectable.
  • No-duplicate-id. Every fact keeps a distinct id; the canonical key-id is held only by the live winner; the retired fact moves to a distinct archival id.
  • Backward-compatible. Candidates without the signal behave exactly as S2; a profile with zero supersessions serialises byte-identically to today (serializeProfile unchanged; grammar already round-trips superseded).
  • Decay stays advisory and orthogonal to supersede.
  • No grammar / no cross-silo threading / no new reader in S3b.
  • TDD iron law — each engine rule is pinned by a failing unit test before the CLI is wired (mirrors brief-sb-s2.md:115).

6. Success criteria (testable — deterministic, unit-level)

  • SC1 — routing: a candidate with supersedes: <key> + a fact at mintEntityId({observed, key})proposeDiff emits exactly one supersedes[] entry (oldId/oldValue + newId/newValue) and no conflicts[]/keep-both entry for it. (unit)
  • SC2 — mutation + concrete ids: applyDiff produces a doc where (a) the winner carries id == mintEntityId({observed, key}), status: "active", evidence_count: 1, the new value; (b) the retired fact carries status: "superseded" and an id != mintEntityId({observed, key}) (the archival id); (c) the retired fact is still present (count of facts for the key = 2, one active + one superseded). (unit)
  • SC3a — propose-level idempotency: after a supersession has landed, a second gather→propose cycle with the same now-current value produces no new supersedes[] entry (the winner matches by value → bump). (unit)
  • SC3b — apply-level idempotency: applyDiff of a supersede diff onto a doc where that supersession is already applied does not double-install the winner, double-flip status, or create a duplicate id (the §3.4 state-check skips it). (unit)
  • SC4 — graceful no-target: a supersedes: <key> signal with no fact at that key-id (active or otherwise — superseded facts no longer hold the key-id) degrades to a plain add; no throw. (unit)
  • SC5 — round-trip through the real path: parseProfile(serializeProfile(applyDiff(cur, supersedeDiff))) deep-equals applyDiff(cur, supersedeDiff) — a superseded fact produced by the engine round-trips, not just the hand-built profile.test.ts:35 fixture. (unit)
  • SC6 — CLI gate + render fidelity: --propose renders a ## Supersessions section in pending-diff.md with one line per supersedes[] entry (old → new, both values shown), and a 0-entry diff renders no section (decoy); --apply writes the superseded status ONLY with --confirm (refuses otherwise). (cli — brain suite)
  • SC7 — no regression: contradictory candidates (no signal) still keep-both; bump/promote/decay/published-only all unchanged (regression pins). (unit)
  • SC8 — intra-batch determinism: a batch with two candidates superseding the same key (or one superseding a fact another candidate creates in the same batch) resolves deterministically per the §4 touched-guard rule — no duplicate id, no double-supersede. (unit)
  • SC9 — gate green: brain suite ≥ bumped BRAIN_TESTS_FLOOR; scripts/test-runner.sh green; ASSERT_BASELINE_FLOOR unchanged at 80 (the render-fidelity check is a brain-suite test, so BRAIN_TESTS_FLOOR is the only floor that moves). (plan-critic FIX 7)
  • SC10 — gather active-only: --gather output excludes status:superseded facts (no retired value re-presented as live context). (cli)
  • SC11 — decay excludes superseded: a retained dynamic superseded fact throws no staleFlags. (unit)
  • SC12 — supersede-wins ordering: a diff that bumps/promotes AND supersedes the same id resolves to supersede-wins (the superseded fact is not bumped/promoted). (unit)

7. Verification

  • Deterministic (the whole slice): SC1SC9 are unit/CLI tests in scripts/brain/tests/ + the structure-lint gate. Unlike S3a, S3b has no behavioural-only SC — supersede is pure engine mechanics, fully exercisable in the harness. A key reason it's a good second slice.
  • Run: (cd scripts/brain && npm test) then bash scripts/test-runner.sh.

8. Open questions — RESOLVED (operator 2026-06-23)

The id-lifecycle is resolved — winner-takes-key-id (§3.5), per brief-review [FIX 1]. The four forks are now operator-confirmed:

  1. Q-SCOPE → temporal-update → supersede ONLY (condition-dependent + distractor deferred). Clean cut, no broken half-state.
  2. Q-LAYER → winner enters DYNAMIC and re-earns promotion (N=3). A superseded static fact stays static-but-superseded (decay-exempt — consolidate.ts:132 scans dynamic only — acceptable as retained audit, not live). No static-inherit.
  3. Q-SIGNAL → minimal supersedes?: string (target key) (widen later only if condition-dependent/distractor are built).
  4. Q-RETAIN → retain superseded facts as audit trail (no pruning in S3b; any pruning is an S3d hygiene concern).

9. Brief-review (light-Voyage) — folded

voyage:brief-reviewer verdict: APPROVE-WITH-FIXES (PROCEED_WITH_RISKS). All 8 [FIX] folded above:

  • [FIX 1] post-supersede id lifecycle → §3.5 inverted to winner-takes-key-id + retired-fact-re-minted (the one latent-corruption fix).
  • [FIX 2] SC3 idempotency layer → split into SC3a (propose) + SC3b (apply, state-checked).
  • [FIX 3] SC2 concrete id/status/count assertions (non-vacuous).
  • [FIX 4] stale-diff trust boundary → §5 named assumption + §3.4 skip-guard.
  • [FIX 5] floor baseline → §3.7 states the verified post-S3a baseline (82 / 80; S3a landed 4fa411f).
  • [FIX 6] SC5 round-trips through applyDiff output, not the hand fixture.
  • [FIX 7] Q-LAYER decided default → §3 IN-scope; only static-inherit stays open (§8.2).
  • [FIX 8] SC4 names the scanned fact-set (the key-id lookup).

The 3 [RISK] are carried as explicit plan obligations:

  • [RISK A] intra-batch supersede semantics → defined in §4 + pinned by SC8 (touched-guard).
  • [RISK B] operator gate is the only classification net → §2 consequence + §3.6 unmistakable render + §3.7 honest-limit doc line.
  • [RISK C] renderDiffMd JSON↔MD divergence → §3.6 render-fidelity check + SC6 lint.

Brief-review confirmed (no fix): byte-identical backward-compat, five→six-array idiom, no grammar change, the "temporal-update only" scope cut, and the optional-field backward-compat of Candidate.supersedes?.

Plan-critic pass (folded back into this brief). Reviewing the plan, voyage:plan-critic surfaced two latent-corruption BLOCKs (applyDiff must replace-in-place not copy; proposeDiff must value-guard the supersede fork) — both resolved in the plan — and two completeness gaps that belong in the brief's scope: the --gather active-only filter (§3.8) and decay excludes superseded (§3.9), with SC10/SC11/SC12 added to §6. voyage:scope-guardian: ALIGNED (0 creep / 0 gap). The plan (plan-sb-s3b.md) carries the full BLOCK/FIX ledger.