docs(linkedin-studio): SB-S3b brief + plan — supersede arm (light-Voyage hardened)
Second sub-slice of SB-S3 (after S3a's reader landed). Adds an operator-gated,
explicitly-signalled supersede op to the consolidation engine: a temporal-update
candidate retires a stale fact (status: superseded, re-minted to an archival id,
replaced in place, retained as audit) and installs the new winner under the
canonical key-id. Bounded to scripts/brain/ — pure, deterministic, fully
unit-testable.
Operator-confirmed scope: temporal-update→supersede ONLY · winner→dynamic
(re-earn) · minimal `supersedes?: string` · retain superseded as audit.
Light-Voyage-hardened:
- brief-reviewer APPROVE-WITH-FIXES — 8 FIX folded (the load-bearing one: id
lifecycle → winner-takes-key-id, retired fact re-minted, so mintEntityId(key)
always points at the live fact).
- plan-critic REVISE → 2 BLOCK + 6 FIX folded: applyDiff must REPLACE-IN-PLACE
(not copy → would dup the canonical key-id); proposeDiff must value-guard the
supersede fork (a re-sent stale signal would self-supersede every run); plus
--gather active-only filter + decay excludes superseded (else retired facts
leak back as live context / spam stale flags).
- scope-guardian ALIGNED (0 creep / 0 gap).
Also reconciles brief-sb-s3a.md's stale DRAFT status (S3a landed 4fa411f).
No code yet — awaiting go-before-code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RigJBiRFNtFZKCz21qNbQ4
This commit is contained in:
parent
4fa411f13c
commit
d2ec7482be
3 changed files with 213 additions and 1 deletions
|
|
@ -1,7 +1,7 @@
|
||||||
# Brief — SB-S3a: the first profile.md READER
|
# Brief — SB-S3a: the first profile.md READER
|
||||||
|
|
||||||
> **Slice:** SB-S3a (first sub-slice of SB-S3, the cross-silo + ops-centre arc step).
|
> **Slice:** SB-S3a (first sub-slice of SB-S3, the cross-silo + ops-centre arc step).
|
||||||
> **Status:** DRAFT — awaiting operator "go" before any code. Light-Voyage hardening (brief-review → plan-critic → scope-guardian) pending.
|
> **Status:** LANDED `4fa411f` (2026-06-23) — gate 95/0/0, `ASSERT_BASELINE_FLOOR` 80. Light-Voyage-hardened (brief-review APPROVE · scope-guardian ALIGNED · plan-critic REVISE→folded). SC1/2/3/6 gate-proven; SC4 inspection-pass; SC5 (read-back) deferred to a reloaded session.
|
||||||
> **Predecessors:** SB-S0 (id/profile substrate) · SB-S1 (published-gold ingest) · SB-S2 (consolidation motor — *motor-only, no reader*).
|
> **Predecessors:** SB-S0 (id/profile substrate) · SB-S1 (published-gold ingest) · SB-S2 (consolidation motor — *motor-only, no reader*).
|
||||||
|
|
||||||
## 1. Operator decision (2026-06-23)
|
## 1. Operator decision (2026-06-23)
|
||||||
|
|
|
||||||
111
docs/second-brain/brief-sb-s3b.md
Normal file
111
docs/second-brain/brief-sb-s3b.md
Normal file
|
|
@ -0,0 +1,111 @@
|
||||||
|
# Brief — SB-S3b: the supersede arm
|
||||||
|
|
||||||
|
> **Slice:** SB-S3b (second sub-slice of SB-S3). **Status:** DRAFT — awaiting operator "go" before any code. Light-Voyage: brief-review APPROVE-WITH-FIXES (8 FIX folded; 3 RISK carried to the plan).
|
||||||
|
> **Predecessors:** SB-S0 (id/profile substrate — reserved the `superseded` status) · SB-S1 (published gold) · SB-S2 (consolidation motor — keep-both, *no supersede*) · **SB-S3a (first reader `strategy-advisor` — LANDED `4fa411f`, gate 95/0/0, ASSERT floor 80).**
|
||||||
|
> **Grounded in:** a read-only code map of `scripts/brain/` (file:line cited throughout) + a brief-review pass against the same code.
|
||||||
|
|
||||||
|
## 1. Operator decision (2026-06-23)
|
||||||
|
|
||||||
|
SB-S3 was decomposed into four sub-slices; after S3a (reader) landed, the recommended order is **S3b (supersede) → S3c (id-threading) → S3d (hygiene+ops, last)**. S3b is next because it has the **lowest blast radius** (bounded to the `scripts/brain/` TS package — pure functions, fully unit-testable, unlike S3a's behavioural SCs), it **completes the consolidation motor's one explicit TODO** (`consolidate.ts:11`: *"no supersede in S2 — that's S3"*), and it **strengthens the read path S3a just shipped** (a profile with supersede gives the new reader cleaner, non-contradictory facts).
|
||||||
|
|
||||||
|
## 2. Why supersede, and the central design decision
|
||||||
|
|
||||||
|
S2's engine handles a value-conflict by **keep-both**: the old fact is left untouched and a new alt fact is added with a distinct content-id (`consolidate.ts:120-129`). That is correct for a *genuine contradiction* (two views coexist), but wrong for a **temporal update** — when a new fact *replaces* an outdated one (the user changed roles, a number moved, a stance evolved). Today there is **no way to retire the stale fact**; it persists forever, and the S3a reader has to weigh a fact the user has already moved past.
|
||||||
|
|
||||||
|
The `superseded` status was reserved at SB-S0 (`types.ts:17`, `:23`; grammar `profile.ts:58`; round-trip proven `profile.test.ts:35`) precisely for this — **but nothing ever sets it.** S3b is the code that does.
|
||||||
|
|
||||||
|
**Central design decision (the "fuzzy heart", `brief-sb-s2.md:115`) — supersede is driven by an EXPLICIT signal, never engine auto-classification.** The S2 architecture splits cleanly: the **session** does the fuzzy extraction, the **engine** is deterministic mechanics, the **operator** gates the diff before apply. Classifying *temporal-update vs contradictory* is a fuzzy judgement — so it stays where the fuzziness already lives (the session's candidate extraction), not baked into a pure function. A candidate carries an explicit "this supersedes the prior fact for key X" intent; `proposeDiff` **deterministically** routes it; the operator **sees and confirms** the supersession in the rendered diff before `--apply --confirm` writes it. This honours the S2 promise (`brief-sb-s2.md`: *"reserve supersede for an explicit confirm signal"*) and keeps the engine deterministic and testable.
|
||||||
|
|
||||||
|
**Consequence (the engine has no classification safety net).** Because the engine never validates that a supersede *should* be a supersede, the **operator gate is the only place a mis-classification is caught** — and supersede is the first op that *retires* state (a mis-call silently retires a fact that should have been kept-both, and `status: superseded` makes it look intentional). Two obligations follow: SC6's rendering must make every supersession **visually unmistakable** (old value → new value, both shown, flagged as a retirement), and the doc reconciliation (§3.7) must add an honest-limit line mirroring `consolidation-loop.md:64` — *the engine guarantees supersede mechanics; whether a contradiction is really a temporal update is the session's + operator's judgement.*
|
||||||
|
|
||||||
|
## 3. Scope — what is IN (S3b)
|
||||||
|
|
||||||
|
1. **Candidate supersede signal.** Extend the transient `Candidate` (`consolidate.ts:33-39`) with an explicit, optional supersede intent — minimal shape: `supersedes?: string` (the **key** of the active prior fact this candidate replaces). Set only by the session when it judges a temporal update; absent for ordinary candidates (fully backward-compatible). `validateCandidates` (`cli.ts:147-157`) gains an optional single-line check for the new field.
|
||||||
|
|
||||||
|
2. **A sixth diff array.** `ProfileDiff` (`consolidate.ts:41-47`) gains `supersedes: { oldId, oldValue, newId, newValue }[]` (parallel to the existing `conflicts[]` — the five-array shape means a new op is a new array, not a union edit).
|
||||||
|
|
||||||
|
3. **`proposeDiff` routing (deterministic), with a precise matching rule.** "Matching active prior fact" = the fact returned by `byId.get(mintEntityId({kind:'observed', key}))` — i.e. the live fact at the canonical key-id (see §3.5: retired facts no longer hold the key-id, so this is unambiguously the *active* one). When a candidate carries `supersedes: <key>` AND that lookup finds a fact, route to `supersedes[]`. When the signal is present but **no fact is found at the key-id**, degrade gracefully to a plain **add** (defined, tested behaviour — never a crash). Candidates without the signal are untouched (keep-both/bump/promote/decay exactly as S2).
|
||||||
|
|
||||||
|
4. **`applyDiff` mutation (the first `superseded` writer), state-checked.** Add a branch (`consolidate.ts:144-162`) that, for each `supersedes[]` entry: (a) re-mints the retired fact to a distinct archival id and sets its `status: "superseded"`; (b) installs the new winner under the canonical key-id (see §3.5). The branch is **state-checked, not blind**: a `supersedes[]` entry whose `oldId` is absent from `current` or already `superseded` is **skipped** (this is what makes re-apply idempotent and guards stale diffs — §5). Superseded facts are **retained** in the doc (audit), per `architecture.md:69` (*"keep old `status: superseded` + date"*) — never deleted. Stays a **pure** function returning a new doc.
|
||||||
|
|
||||||
|
5. **Id discipline — winner takes the key-id; retired fact is re-minted off it.** Because `mintEntityId` is value-independent (`id.ts:33`), only one fact may hold the canonical key-id, and it must be the **live** one. So on supersede: the **new winner takes `mintEntityId({observed, key})`** (the canonical key-id), and the **retired fact is re-minted to a distinct archival id** (`mintContentId` over an archival seed, e.g. `superseded:${key}::${oldValue}::${oldFirstSeen}`, so it is stable and collision-free). This preserves the **no-duplicate-id** invariant AND keeps `mintEntityId(key)` always pointing at the live fact — so every downstream match (future same-key candidates, the S3a reader) resolves to the winner exactly as before supersede existed. *(This inverts the naïve "keep old id, mint winner fresh", which would leave the key-id on a retired fact and resurrect it on the next same-key candidate — the brief-review's [FIX 1].)*
|
||||||
|
|
||||||
|
6. **CLI rendering, non-vacuously checked.** `renderDiffMd` (`cli.ts:132-145`) gains a `## Supersessions (old → new)` section so the operator sees every retirement before confirming. **No new subcommand** — supersede rides the existing `--propose` / `--apply --confirm` flow; the sole `profile.md` writer stays `cli.ts:215`. Because the operator confirms via the **JSON** diff while reading the **MD**, a **brain-suite test** (`consolidate-cli.test.ts`, repo non-vacuity idiom) asserts **every `supersedes[]` JSON entry produces a corresponding rendered MD line** and a 0-entry diff renders no section — not merely that a section exists (RISK C). *(This is a brain-suite test, not a `test-runner.sh` Section — so it is counted by `BRAIN_TESTS_FLOOR`, not by a new unconditional structure-lint check; see §3.7.)*
|
||||||
|
|
||||||
|
7. **Gate + doc reconciliation.** **Verified baseline (S3a landed):** `BRAIN_TESTS_FLOOR` = 82 (`test-runner.sh:716`), `ASSERT_BASELINE_FLOOR` = 80 (`:913`). Bump `BRAIN_TESTS_FLOOR` by the new brain-test count; **`ASSERT_BASELINE_FLOOR` stays 80** — no new *unconditional `test-runner.sh` structure-lint* check is added (the render-fidelity check lives in the brain suite, §3.6). `BRAIN_TESTS_FLOOR` is the only floor that moves. Reconcile the now-stale "no supersede" claims (`consolidation-loop.md:34`, `:71`; `consolidate.ts:11` docstring) to the true state, and add the honest-limit line from §2.
|
||||||
|
|
||||||
|
8. **`--gather` active-only filter (plan-critic FIX 8b — completeness).** `runConsolidate --gather` (`cli.ts:174`) emits `profileFacts` to the extraction session with **no status filter** today. After supersede lands, that would re-present `status:superseded` archival facts as live context — re-introducing the exact retired values this slice exists to retire (§2). The gather output is filtered to `status === "active"` (one line). Without this the slice regresses its own goal.
|
||||||
|
|
||||||
|
9. **Decay excludes superseded (plan-critic FIX 8c — completeness).** A retained *dynamic* superseded fact is still in `current.dynamic`, so the decay scan (`consolidate.ts:132-134`) would throw `staleFlags` for it forever. The decay filter excludes `status:superseded` (static archival facts are already decay-exempt — dynamic-only scan). Supersede and decay stay orthogonal.
|
||||||
|
|
||||||
|
## 4. Non-goals — what is OUT (deferred)
|
||||||
|
|
||||||
|
- **Engine auto-classification of temporal-update vs contradictory.** The signal is explicit (§2). The engine never guesses.
|
||||||
|
- **The other two contradiction-typology branches** — *condition-dependent (scope)* and *distractor (drop)* (`architecture.md:69`, `research/secondbrain-sota.md:37`). S3b ships **temporal-update → supersede ONLY**; contradictory stays keep-both (S2). The other two are later, purely-additive arrays (deferring them leaves no broken half-state — confirmed by brief-review).
|
||||||
|
- **Auto-demotion of stale facts.** Decay (`staleFlags`, `consolidate.ts:132-134`) stays **advisory** — never mutates status or removes a fact. Supersede and decay stay orthogonal (no shared field). A stale fact is *not* auto-superseded.
|
||||||
|
- **Cross-silo id-threading (S3c)** · **content-history retirement / operations.md / triple-post reconciliation (S3d)** · **any new reader** (S3a's reader unchanged; `content-optimizer` is later).
|
||||||
|
- **Chained / multi-fact supersede as a FEATURE.** One active prior → one new winner per candidate. Chains (A→B→C in one pass) and one-supersedes-many are out. **But the intra-batch SEMANTICS are defined, not undefined** (RISK A): the supersede fork carries a `touched`-style guard (analogous to `consolidate.ts:97`) so that within one candidate batch, two candidates targeting the same key, or a candidate superseding a fact another candidate just created, resolve deterministically (first-writer-wins + the later one degrades to add/bump) rather than corrupting. Deferring the *feature* is fine; the *batch rule* is in scope.
|
||||||
|
- **Grammar changes.** None needed — `superseded` is already accepted end-to-end.
|
||||||
|
|
||||||
|
## 5. Boundaries / invariants (must hold)
|
||||||
|
|
||||||
|
- **Operator-gated, single disk-writer.** No new write path to disk; `--apply --confirm` (`cli.ts:215`) stays the sole `profile.md` writer; `--apply` refuses without `--confirm`.
|
||||||
|
- **Pure, idempotent engine.** `proposeDiff`/`applyDiff` stay pure (no I/O). Idempotency holds at BOTH layers: `proposeDiff` produces no new `supersedes[]` entry once a supersession has landed (winner matches by value → bump; old fact already superseded), and `applyDiff`'s supersede branch is **state-checked** so re-applying the same diff does not double-install the winner or double-flip status.
|
||||||
|
- **Stale-diff safety (named assumption + guard).** A pending diff is meant to be applied against the profile it was proposed from; a profile mutated between propose and apply is operator error. S3b does not silently corrupt in that case: `applyDiff` **skips** any `supersedes[]` entry whose `oldId` is absent or already `superseded` in `current` (the §3.4 state-check), rather than retiring the wrong fact.
|
||||||
|
- **Audit-preserving.** Superseded facts are retained with `status: superseded`, never deleted — history is inspectable.
|
||||||
|
- **No-duplicate-id.** Every fact keeps a distinct id; the canonical key-id is held only by the live winner; the retired fact moves to a distinct archival id.
|
||||||
|
- **Backward-compatible.** Candidates without the signal behave exactly as S2; a profile with zero supersessions serialises byte-identically to today (`serializeProfile` unchanged; grammar already round-trips `superseded`).
|
||||||
|
- **Decay stays advisory** and orthogonal to supersede.
|
||||||
|
- **No grammar / no cross-silo threading / no new reader** in S3b.
|
||||||
|
- **TDD iron law** — each engine rule is pinned by a failing unit test before the CLI is wired (mirrors `brief-sb-s2.md:115`).
|
||||||
|
|
||||||
|
## 6. Success criteria (testable — deterministic, unit-level)
|
||||||
|
|
||||||
|
- **SC1 — routing:** a candidate with `supersedes: <key>` + a fact at `mintEntityId({observed, key})` → `proposeDiff` emits exactly one `supersedes[]` entry (oldId/oldValue + newId/newValue) and **no** `conflicts[]`/keep-both entry for it. *(unit)*
|
||||||
|
- **SC2 — mutation + concrete ids:** `applyDiff` produces a doc where (a) the **winner** carries `id == mintEntityId({observed, key})`, `status: "active"`, `evidence_count: 1`, the new value; (b) the **retired** fact carries `status: "superseded"` and an id `!= mintEntityId({observed, key})` (the archival id); (c) the retired fact is **still present** (count of facts for the key = 2, one active + one superseded). *(unit)*
|
||||||
|
- **SC3a — propose-level idempotency:** after a supersession has landed, a second `gather→propose` cycle with the *same* now-current value produces **no** new `supersedes[]` entry (the winner matches by value → bump). *(unit)*
|
||||||
|
- **SC3b — apply-level idempotency:** `applyDiff` of a supersede diff onto a doc where that supersession is already applied does **not** double-install the winner, double-flip status, or create a duplicate id (the §3.4 state-check skips it). *(unit)*
|
||||||
|
- **SC4 — graceful no-target:** a `supersedes: <key>` signal with **no fact at that key-id** (active or otherwise — superseded facts no longer hold the key-id) degrades to a plain add; no throw. *(unit)*
|
||||||
|
- **SC5 — round-trip through the real path:** `parseProfile(serializeProfile(applyDiff(cur, supersedeDiff)))` deep-equals `applyDiff(cur, supersedeDiff)` — a superseded fact *produced by the engine* round-trips, not just the hand-built `profile.test.ts:35` fixture. *(unit)*
|
||||||
|
- **SC6 — CLI gate + render fidelity:** `--propose` renders a `## Supersessions` section in `pending-diff.md` with one line per `supersedes[]` entry (old → new, both values shown), and a 0-entry diff renders no section (decoy); `--apply` writes the superseded status ONLY with `--confirm` (refuses otherwise). *(cli — brain suite)*
|
||||||
|
- **SC7 — no regression:** contradictory candidates (no signal) still keep-both; bump/promote/decay/published-only all unchanged (regression pins). *(unit)*
|
||||||
|
- **SC8 — intra-batch determinism:** a batch with two candidates superseding the same key (or one superseding a fact another candidate creates in the same batch) resolves deterministically per the §4 `touched`-guard rule — no duplicate id, no double-supersede. *(unit)*
|
||||||
|
- **SC9 — gate green:** brain suite ≥ bumped `BRAIN_TESTS_FLOOR`; `scripts/test-runner.sh` green; **`ASSERT_BASELINE_FLOOR` unchanged at 80** (the render-fidelity check is a brain-suite test, so `BRAIN_TESTS_FLOOR` is the only floor that moves). *(plan-critic FIX 7)*
|
||||||
|
- **SC10 — gather active-only:** `--gather` output excludes `status:superseded` facts (no retired value re-presented as live context). *(cli)*
|
||||||
|
- **SC11 — decay excludes superseded:** a retained dynamic superseded fact throws no `staleFlags`. *(unit)*
|
||||||
|
- **SC12 — supersede-wins ordering:** a diff that bumps/promotes AND supersedes the same id resolves to supersede-wins (the superseded fact is not bumped/promoted). *(unit)*
|
||||||
|
|
||||||
|
## 7. Verification
|
||||||
|
|
||||||
|
- **Deterministic (the whole slice):** SC1–SC9 are unit/CLI tests in `scripts/brain/tests/` + the structure-lint gate. **Unlike S3a, S3b has no behavioural-only SC** — supersede is pure engine mechanics, fully exercisable in the harness. A key reason it's a good second slice.
|
||||||
|
- **Run:** `(cd scripts/brain && npm test)` then `bash scripts/test-runner.sh`.
|
||||||
|
|
||||||
|
## 8. Open questions — RESOLVED (operator 2026-06-23)
|
||||||
|
|
||||||
|
The id-lifecycle is resolved — winner-takes-key-id (§3.5), per brief-review [FIX 1]. The four forks are now operator-confirmed:
|
||||||
|
|
||||||
|
1. **Q-SCOPE → temporal-update → supersede ONLY** ✅ (condition-dependent + distractor deferred). Clean cut, no broken half-state.
|
||||||
|
2. **Q-LAYER → winner enters DYNAMIC and re-earns promotion** ✅ (N=3). A superseded *static* fact stays static-but-superseded (decay-exempt — `consolidate.ts:132` scans dynamic only — acceptable as retained audit, not live). No static-inherit.
|
||||||
|
3. **Q-SIGNAL → minimal `supersedes?: string` (target key)** ✅ (widen later only if condition-dependent/distractor are built).
|
||||||
|
4. **Q-RETAIN → retain superseded facts as audit trail** ✅ (no pruning in S3b; any pruning is an S3d hygiene concern).
|
||||||
|
|
||||||
|
## 9. Brief-review (light-Voyage) — folded
|
||||||
|
|
||||||
|
`voyage:brief-reviewer` verdict: **APPROVE-WITH-FIXES** (PROCEED_WITH_RISKS). All 8 [FIX] folded above:
|
||||||
|
- **[FIX 1]** post-supersede id lifecycle → §3.5 inverted to winner-takes-key-id + retired-fact-re-minted (the one latent-corruption fix). ✅
|
||||||
|
- **[FIX 2]** SC3 idempotency layer → split into SC3a (propose) + SC3b (apply, state-checked). ✅
|
||||||
|
- **[FIX 3]** SC2 concrete id/status/count assertions (non-vacuous). ✅
|
||||||
|
- **[FIX 4]** stale-diff trust boundary → §5 named assumption + §3.4 skip-guard. ✅
|
||||||
|
- **[FIX 5]** floor baseline → §3.7 states the **verified** post-S3a baseline (82 / 80; S3a landed `4fa411f`). ✅
|
||||||
|
- **[FIX 6]** SC5 round-trips through `applyDiff` output, not the hand fixture. ✅
|
||||||
|
- **[FIX 7]** Q-LAYER decided default → §3 IN-scope; only static-inherit stays open (§8.2). ✅
|
||||||
|
- **[FIX 8]** SC4 names the scanned fact-set (the key-id lookup). ✅
|
||||||
|
|
||||||
|
The 3 [RISK] are carried as explicit plan obligations:
|
||||||
|
- **[RISK A]** intra-batch supersede semantics → defined in §4 + pinned by SC8 (`touched`-guard).
|
||||||
|
- **[RISK B]** operator gate is the only classification net → §2 consequence + §3.6 unmistakable render + §3.7 honest-limit doc line.
|
||||||
|
- **[RISK C]** `renderDiffMd` JSON↔MD divergence → §3.6 render-fidelity check + SC6 lint.
|
||||||
|
|
||||||
|
Brief-review confirmed (no fix): byte-identical backward-compat, five→six-array idiom, no grammar change, the "temporal-update only" scope cut, and the optional-field backward-compat of `Candidate.supersedes?`.
|
||||||
|
|
||||||
|
**Plan-critic pass (folded back into this brief).** Reviewing the plan, `voyage:plan-critic` surfaced two latent-corruption BLOCKs (applyDiff must replace-in-place not copy; proposeDiff must value-guard the supersede fork) — both resolved in the plan — and two completeness gaps that belong in the brief's scope: the **`--gather` active-only filter (§3.8)** and **decay excludes superseded (§3.9)**, with SC10/SC11/SC12 added to §6. `voyage:scope-guardian`: **ALIGNED** (0 creep / 0 gap). The plan (`plan-sb-s3b.md`) carries the full BLOCK/FIX ledger.
|
||||||
101
docs/second-brain/plan-sb-s3b.md
Normal file
101
docs/second-brain/plan-sb-s3b.md
Normal file
|
|
@ -0,0 +1,101 @@
|
||||||
|
# Plan — SB-S3b: the supersede arm
|
||||||
|
|
||||||
|
> Brief: `docs/second-brain/brief-sb-s3b.md`. Slice: SB-S3b.
|
||||||
|
> **Operator-confirmed (2026-06-23):** temporal-update→supersede ONLY · winner→dynamic (re-earn) · minimal `supersedes?: string` · retain superseded as audit.
|
||||||
|
> **Light-Voyage:** brief-review APPROVE-WITH-FIXES (8 folded); plan-critic **REVISE → 2 BLOCK + 6 FIX folded below**; scope-guardian **ALIGNED** (0 creep/0 gap, 1 minor SC9 reconciliation folded).
|
||||||
|
> Order is TDD: failing engine unit tests land BEFORE the engine logic (iron law); each rule pinned before the CLI is wired (`brief-sb-s2.md:115`).
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
The consolidation engine can **retire** a fact via an operator-confirmed, explicitly-signalled supersede: a temporal-update candidate flips the stale fact to `status: "superseded"` (re-minted to an archival id, **replaced in place**, retained as audit) and installs the new winner under the canonical key-id. Pure, idempotent, deterministic; rides the existing `--propose`/`--apply --confirm` gate; superseded facts never leak back into gather or decay; no grammar change; fully unit-testable.
|
||||||
|
|
||||||
|
## Files touched (exhaustive — for scope-guardian)
|
||||||
|
|
||||||
|
| File | Change | SC |
|
||||||
|
|------|--------|-----|
|
||||||
|
| `scripts/brain/src/consolidate.ts` | `Candidate` (+`supersedes?: string`, 33-39) · `ProfileDiff` (+`supersedes: SupersedeOp[]`, 41-47; `SupersedeOp` carries `provenance`) · `proposeDiff` routing fork before keep-both (~120), **value-guarded** + intra-batch `touched`-guard · `applyDiff` **replace-in-place** supersede branch, state-checked, ordered (144-162) · decay scan excludes `superseded` (132-134) · local `archivalId()` helper · engine docstring `:11` | SC1–SC5,7,8,11,12 |
|
||||||
|
| `scripts/brain/src/cli.ts` | `renderDiffMd` (+`## Supersessions (old → new)`, appended last via `section()`, 132-145) · `validateCandidates` (+optional single-line `supersedes`, 147-157) · **`--gather` profileFacts filtered to `status === "active"`** (`:174`) | SC6, SC10 |
|
||||||
|
| `scripts/brain/tests/consolidate.test.ts` | unit tests SC1, SC2, SC3a, SC3b, SC4, SC5, SC7, SC8, SC11, SC12 | engine |
|
||||||
|
| `scripts/brain/tests/consolidate-cli.test.ts` | CLI tests SC6 (render + fidelity non-vacuity + `--confirm` gate), SC10 (gather active-only) | CLI |
|
||||||
|
| `scripts/test-runner.sh` | bump `BRAIN_TESTS_FLOOR` 82→(82+N) + breakdown comment (`:716`); **`ASSERT_BASELINE_FLOOR` UNCHANGED at 80** | SC9 |
|
||||||
|
| `docs/second-brain/consolidation-loop.md` | reconcile "no supersede" claims (`:34`, `:71`) to true state + honest-limit line (engine guarantees mechanics; temporal-update-vs-contradictory is session+operator judgement — the operator gate is the only classification net) | doc |
|
||||||
|
|
||||||
|
**Not touched (scope fence):** `types.ts` (`FactStatus` already has `superseded`) · `profile.ts` (grammar already round-trips `superseded`) · `id.ts` (reuse `mintEntityId`/`mintContentId`; `archivalId` is local to `consolidate.ts`) · `ingest.ts`/`scaffold.ts`/`dataRoot.ts` · `hooks/scripts/**` · `agents/**` (S3a reader unchanged) · `architecture.md` (its `:69` design promise is fulfilled; `:79` "S2 had no supersede" stays historically true — arc-status note deferred, not load-bearing) · condition-dependent/distractor branches (Q-SCOPE).
|
||||||
|
|
||||||
|
## Step 0 — pre-flight (verify baseline, no edits)
|
||||||
|
|
||||||
|
`(cd scripts/brain && npm install)` if `node_modules` absent. Confirm `(cd scripts/brain && npm test)` = **82/82** and `bash scripts/test-runner.sh` green (95/0/0, `BRAIN_TESTS_FLOOR=82`, `ASSERT_BASELINE_FLOOR=80`). Confirm `renderDiffMd`/`--gather`/CLI-propose coverage lives in `consolidate-cli.test.ts` (host SC6/SC10 there). Re-read `applyDiff` (`:144-162`) to see how promotions are removed via `filter` (`:156`) — the supersede branch mirrors that.
|
||||||
|
|
||||||
|
## Step 1 — (RED) types + failing tests
|
||||||
|
|
||||||
|
1. **Type scaffolding only** (suite still compiles, existing 82 green):
|
||||||
|
- `Candidate` (`:33-39`): `supersedes?: string`.
|
||||||
|
- `ProfileDiff` (`:41-47`): `supersedes: SupersedeOp[]`, with `interface SupersedeOp { oldId: string; oldValue: string; newId: string; newValue: string; provenance: Provenance }`.
|
||||||
|
- `proposeDiff` returned object: initialise `supersedes: []` (no routing yet).
|
||||||
|
- `applyDiff`: unchanged for now (ignores the new array).
|
||||||
|
2. **Add failing unit tests** (mirror the SC1e keep-both test at `:78-91`): SC1, SC2, SC3b, SC4, SC8, SC11, SC12 → expected FAIL; SC3a, SC5, SC7 → guards (may pass). SC6, SC10 in `consolidate-cli.test.ts` → FAIL.
|
||||||
|
|
||||||
|
**RED gate (single, unambiguous):** run **`(cd scripts/brain && npm test)` directly** → the new behavioural tests fail, the 82 existing pass. **Do NOT run `bash scripts/test-runner.sh` at RED** — Section 16b (`test-runner.sh:717`) fails the whole gate on a non-zero brain exit, which is design-noise, not signal. The direct brain-suite RED is the failing-test proof. (The gate is run GREEN only, after Step 4.)
|
||||||
|
|
||||||
|
## Step 2 — (GREEN) engine logic
|
||||||
|
|
||||||
|
1. **`proposeDiff` routing fork** — BEFORE the keep-both branch (`~:120`), after the ai-draft reject (`:105`, so a supersede can never carry ai-draft provenance — confirmed):
|
||||||
|
- If `c.supersedes` is set: `targetId = mintEntityId({kind:"observed", key: c.supersedes})`; `target = byId.get(targetId)`.
|
||||||
|
- Route to `supersedes[]` **only when** `target` exists AND `target.status === "active"` AND `target.value !== c.value` AND `targetId` not `touched` this batch. **The `target.value !== c.value` guard is load-bearing** (plan-critic BLOCK 2): a re-sent stale signal whose target already holds the candidate's value is NOT a supersession — it falls through to the ordinary bump path, so SC3a holds.
|
||||||
|
- On route: push `{ oldId: targetId, oldValue: target.value, newId: mintEntityId({kind:"observed", key: c.key}), newValue: c.value, provenance: c.provenance }`; `touched.add(targetId)` + `touched.add(newId)` (intra-batch guard — a later same-key candidate degrades; winner id reserved). Do **not** also push to `additions`. `continue`.
|
||||||
|
- Otherwise (no target / not active / value already equal / touched): fall through to the ordinary add/bump path (SC4 graceful no-target, SC8 intra-batch). A supersede only retires a fact present at batch start (`byId` built once from `current`).
|
||||||
|
2. **`archivalId()` helper** (local to `consolidate.ts`): `archivalId(oldId, oldValue) = mintContentId(\`superseded:${oldId}::${oldValue}\`)`. Seeding with the retired fact's **pre-archival id** (unique among active facts by the no-dup invariant) makes the archival id collision-free in practice (plan-critic NICE 9 — stronger than a key+date seed).
|
||||||
|
3. **`applyDiff` supersede branch — REPLACE IN PLACE, not copy** (`:144-162`; plan-critic BLOCK 1). Explicit ordering:
|
||||||
|
- (a) Compute `passing = diff.supersedes.filter(s => { const f = factById(current, s.oldId); return f && f.status === "active" && f.value === s.oldValue })`. The **`f.value === oldValue` state-check** is the idempotency + stale-diff guard: on re-apply the winner holds `oldId` with `newValue ≠ oldValue` → not passing → skipped (SC3b); a profile mutated under the diff → skipped (R3). Build `retireById = new Map(passing.map(s => [s.oldId, s]))`.
|
||||||
|
- (b) When mapping `current.static` and `current.dynamic`: a fact whose `id ∈ retireById` is **transformed in place** to `{ ...f, id: archivalId(f.id, f.value), status: "superseded" }` (keeps its layer + its OWN `provenance`/`first_seen` — a `published→human` update must NOT rewrite the archived fact's provenance, plan-critic FIX 3). It is **excluded from bump/promote** (a superseded fact is never bumped or promoted — resolves the ordering hazard, plan-critic FIX 6: supersede-wins over a same-id bump/promote in one diff).
|
||||||
|
- (c) Append each winner `{ id: newId, value: newValue, first_seen: today, last_seen: today, evidence_count: 1, provenance, status: "active" }` to **dynamic** (Q-LAYER re-earn). Because (b) vacates `oldId` and the winner re-occupies it (when `key===supersedes`), there is exactly one fact at the canonical key-id (the winner) + one archival fact — **no duplicate id**. Winner `first_seen: today` is intentional (a genuinely new observation; the original first-observation date lives on the archived copy — plan-critic FIX 8a).
|
||||||
|
- `applyDiff` stays **pure** (new `ProfileDoc`); existing bump/promote/addition handling preserved.
|
||||||
|
4. **Decay scan excludes superseded** (`consolidate.ts:132-134`; plan-critic FIX 8c): add `f.status !== "superseded"` to the `current.dynamic` decay filter so a retained dynamic archival fact never throws perpetual `staleFlags`. (Static archival facts are already decay-exempt — dynamic-only scan.)
|
||||||
|
|
||||||
|
**GREEN gate:** `(cd scripts/brain && npm test)` → all engine SCs pass; existing 82 pass.
|
||||||
|
|
||||||
|
## Step 3 — (GREEN) CLI render + validation + gather filter
|
||||||
|
|
||||||
|
1. `renderDiffMd` (`cli.ts:132-145`): add a `## Supersessions (old → new)` section, **appended LAST** (after Stale, `:143`) via the existing `section()` helper, **only when `supersedes.length > 0`** — one line per entry `- \`<oldValue>\` → \`<newValue>\``. Zero supersessions → no section → byte-identical to today (plan-critic FIX 5).
|
||||||
|
2. `validateCandidates` (`:147-157`): if `supersedes` present, assert non-empty single-line string; absent is valid.
|
||||||
|
3. **`--gather` active-only filter** (`cli.ts:174`; plan-critic FIX 8b): `profileFacts` emitted to the extraction session is filtered to `status === "active"` — superseded archival facts must NOT be re-presented as live context (else the slice re-introduces the retired values it exists to retire). One-line filter.
|
||||||
|
4. **CLI tests** (`consolidate-cli.test.ts`): SC6 — (a) `--propose` writes one `## Supersessions` line per entry; (b) **render-fidelity non-vacuity** — 2 entries → 2 lines, 0 entries → no section (decoy); (c) `--apply` without `--confirm` refuses (exit 1, no write); with `--confirm` writes superseded status. SC10 — `--gather` output excludes superseded facts.
|
||||||
|
|
||||||
|
## Step 4 — gate floor + doc reconciliation
|
||||||
|
|
||||||
|
1. `test-runner.sh:716`: `BRAIN_TESTS_FLOOR` 82 → **82 + N** (N = the EXACT counted new brain tests from Steps 1–3 — set after counting, never guessed). Extend the breakdown comment `… + SB-S2 19 + SB-S3b N`. **`ASSERT_BASELINE_FLOOR` stays 80** — no new *unconditional `test-runner.sh` structure-lint* check is added; the render-fidelity check is a brain-suite test counted by `BRAIN_TESTS_FLOOR`. (This reconciles brief SC9/§3.7 "ASSERT bump" wording: the render-fidelity check lives in the brain suite, so `BRAIN_TESTS_FLOOR` is the only floor that moves — see SC9 below.)
|
||||||
|
2. `consolidation-loop.md`: edit `:34` + `:71` to the true state — temporal-update supersede landed in SB-S3b (explicit signal, operator-gated, replace-in-place, audit-retained); contradictory still keep-both; decay still advisory. Add the honest-limit line (engine guarantees mechanics; the temporal-update-vs-contradictory call is session+operator judgement).
|
||||||
|
3. `consolidate.ts:11` docstring: update the "no supersede in S2 — that's S3" line to note S3b implements temporal-update supersede.
|
||||||
|
|
||||||
|
## Step 5 — Land
|
||||||
|
|
||||||
|
STATE "Telling" + "👉 NESTE" updated (S3b done → S3c/S3d remain); brief+plan committed as docs alongside code. Commit (Conventional; `[skip-docs]` on code/feat; brief+plan+consolidation-loop+docstring are docs → feat-dominant `[skip-docs]`). **Push only inside the window** (`date '+%u %H:%M'` first) and **confirm before push** (`origin` is PUBLIC `open/`). No version bump (additive within v0.5.2 dev).
|
||||||
|
|
||||||
|
## Verification (testable)
|
||||||
|
|
||||||
|
| SC | Check | Expected |
|
||||||
|
|----|-------|----------|
|
||||||
|
| SC1 | routing | supersede candidate (target value differs) → 1 `supersedes[]` entry, 0 keep-both |
|
||||||
|
| SC2 | mutation+ids | winner id=`mintEntityId(c.key)`, active, ev=1, value=new; retired→`status:superseded`, id=archival≠key-id, OWN provenance/first_seen kept; both present (no dup at key-id) |
|
||||||
|
| SC3a | propose-idempotent | post-land re-propose (value now matches) → 0 `supersedes[]`, 1 bump — incl. when a stale signal persists (value-guard) |
|
||||||
|
| SC3b | apply-idempotent | re-apply same diff → value-match skip → no double winner, no re-supersede, no dup id |
|
||||||
|
| SC4 | graceful no-target | signal + no active fact at key-id → plain add, no throw |
|
||||||
|
| SC5 | round-trip | `parse(serialize(applyDiff(...)))` deep-equals `applyDiff(...)` (superseded fact from the real path) |
|
||||||
|
| SC6 | CLI render + gate | `## Supersessions` line/entry; 0 entries → no section (byte-identical); `--apply` needs `--confirm` |
|
||||||
|
| SC7 | no regression | contradictory still keep-both; bump/promote/published-only unchanged |
|
||||||
|
| SC8 | intra-batch | two candidates same key / supersede-a-batch-created-fact → deterministic, no dup id |
|
||||||
|
| SC9 | gate green | `bash scripts/test-runner.sh` green; brain suite ≥ new `BRAIN_TESTS_FLOOR`; **`ASSERT_BASELINE_FLOOR` unchanged at 80** |
|
||||||
|
| SC10 | gather active-only | `--gather` output excludes `status:superseded` facts |
|
||||||
|
| SC11 | decay excludes superseded | a retained dynamic superseded fact throws no `staleFlags` |
|
||||||
|
| SC12 | supersede-wins ordering | a diff that bumps/promotes AND supersedes the same id → supersede wins (superseded fact not bumped/promoted) |
|
||||||
|
| (red proof) | failing-test-first | `(cd scripts/brain && npm test)` BETWEEN Step 1 and Step 2 → new SCs fail, 82 pass |
|
||||||
|
|
||||||
|
## Risks
|
||||||
|
|
||||||
|
- **R1 — id corruption (plan-critic BLOCK 1).** `applyDiff` must **replace in place** (filter+map, mirroring promotions `:156`), never copy — else two facts share the canonical key-id. Idempotency/stale guard keys off `oldId` AND `oldValue`. Pinned by SC2 + SC3b.
|
||||||
|
- **R2 — propose self-supersede (plan-critic BLOCK 2).** A persisted signal whose target already holds the candidate's value must NOT route to supersede (value-guard); else spurious archival churn every run. Pinned by SC3a.
|
||||||
|
- **R3 — superseded facts leak back.** `--gather` (FIX 8b) and decay (FIX 8c) must filter `status:active`/exclude `superseded` — else the slice regresses its own goal / spams stale flags. Pinned by SC10 + SC11.
|
||||||
|
- **R4 — stale diff vs mutated profile.** Value-matched state-check skips. Pinned by SC3b/SC4.
|
||||||
|
- **R5 — render JSON↔MD divergence.** SC6 render-fidelity non-vacuity (every entry → a line; 0 → no section).
|
||||||
|
- **R6 — floor-count drift.** `BRAIN_TESTS_FLOOR` bumped by the EXACT counted N; `ASSERT_BASELINE_FLOOR` deliberately unchanged at 80 (no new unconditional structure-lint check). Verified by the green gate.
|
||||||
|
- **R7 — unguarded doc.** No lint protects `docs/`; `consolidation-loop.md` reconciled by discipline (as the S3a `:67` edit).
|
||||||
Loading…
Add table
Add a link
Reference in a new issue