linkedin-studio/docs/second-brain/brief-sb-s2.md

Task Brief — SB-S2 (Evolution loop)

Status: DRAFT for adversarial brief-review (light-Voyage step 2). Not yet operator-approved. Arc: architecture.md (approved 2026-06-23). Predecessors: SB-S0 (scaffold + profile spine) + SB-S1 (ingest + published gold signal), both landed + released (v0.5.1). Operator scope decisions (2026-06-23, settled before this brief): (1) journal-capture DEFERRED — S2 consumes published + tributary deltas only; (2) deterministic CLI engine + the invoking session does the fuzzy extraction (count-neutral, NO new agent); (3) motor-only — S2 evolves brain/profile.md but wiring a reader is S3 (accepted, flagged). Inputs: architecture.md §"evolution loop" + §"invariants"; SB-S0/S1 code (scripts/brain/); the session-start surface (hooks/scripts/session-start.mjs). Review delta (2026-06-23): folded the brief-review (REVISE). B1+B2 (the state-file write idiom is false AND the brain CLI can't reach the HOME-scoped state file) → resolved by storing consolidation state in a brain data-root sidecar (brain/consolidation-state.json), reachable by both --apply (dataRoot) and the session-start nudge (getDataRoot) — no state-file involvement. M3 --propose emits TWO artifacts (md + json). M4 supersede dropped from S2 (keep-both only; supersede→S3). M5 SC1 split per-rule. Minors: SC3 split, candidate-file schema + gate-validation, freshness nudge = new-published-count only (no profile.md parse at session-start), consolidation matches kind:'observed' only (folded profile-field static seeds immutable in S2).

---

1. Objective

Build the compounding mechanism: an operator-invoked, sleep-time consolidation pass that reads the new published gold signal + tributary deltas, proposes a diff to the two-layer brain/profile.md (never a silent overwrite) — with evidence-threshold promotion, contradiction reconciliation (bias: keep-both-timestamped), provenance-gated learning (published/human only, never ai-draft), and temporal decay flagging (in the diff, not at session-start) — plus a "consolidation-due" nudge on the session-start surface (new-published count + days-since-last-run; per-fact stale nudging deferred), and the session-start wiring of the scaffold init. This is the loop that turns the gold signal (SB-S1) into an ever-improving, drift-resistant profile.

2. Background (why this shape, given the surfaces)

• The loop cannot run AI at session-start. session-start.mjs is zero-dep, must NOT spawn tsx, 10s timeout (the B-S3 rule). So the consolidation pass (which needs fuzzy fact-extraction) is operator-invoked ("sleep-time" = when the operator runs it), and session-start only emits a deterministic nudge ("consolidation due / N new published posts"). This is faithful to the architecture's "sleep-time consolidation, not inline cost."
• journal/ has no producer. SB-S0 created the empty dir; nothing writes episodes. Per the operator decision, S2 consumes published (SB-S1, structured, provenance-tagged) + tributary deltas (specifics/trends/analytics — all have real timestamps); journal-capture is its own later sub-slice.
• The fuzzy extraction lives in the command/session layer, not a new agent (the specifics-bank precedent: the deterministic store + CLI ship, the elicitation interview lives upstream). So S2 ships a deterministic engine + CLI; the invoking session reads the gathered deltas and produces candidate facts. Count-neutral (no agents/ file → no reload, no count-lint bump).
• profile.md has no reader yet. S2 evolves it into a (currently) unconsumed artifact; wiring content consumers is S3. Accepted: the value is that the profile compounds NOW so S3's reader inherits rich data. Flagged honestly (§9).
• The diff is operator-gated — matches the longform SendUserFile + [OPERATØR] idiom (editorial-review): proposed, presentable, never silently applied; confirm/edit/roll-back via git. Anti-sycophancy: the profile is evidence to test, presented neutrally.

3. Confirmed decisions (inherited arc + operator scope — not for re-litigation)

1. Boundary: engine → plugin · user data → data dir · cockpit → Maskinrommet.
2. Invariants (architecture §"Invariants"): provenance-weighted learning (published/human only, never ai-draft); episodic/semantic split; evidence-threshold promotion; contradiction → keep-both-timestamped; temporal-validity + freshness nudge; anti-sycophancy default; zero required curation.
3. Operator scope (§Status): journal deferred · deterministic CLI + session extraction (no new agent) · motor-only (no reader until S3).
4. No new seam function — brain code uses the package's own dataRoot; the session-start edit uses the existing getDataRoot (data-root.mjs).
5. No AI at session-start (zero-dep nudge only); the consolidation pass is operator-invoked; no auto-apply of any diff.

Review SHOULD challenge: feasibility, the engine's classification rules (§5.1), the session-start edit's zero-dep safety, the diff/apply data shapes, hidden coupling, testability, scope leak, missing risks. Review SHOULD NOT re-open the §3 decisions unless internally contradictory or technically impossible.

4. Scope — where SB-S2 sits in the arc

Slice	One-line scope	This brief
SB-S0 — Foundation	scaffold + profile fold + id/provenance spine	✅ landed
SB-S1 — Ingest + gold signal	manual import → ingest/published/; voice-trainer published-only	✅ landed (v0.5.1)
SB-S2 — Evolution loop	consolidation engine → profile diff (threshold/contradiction/decay, provenance-gated) + session-start freshness/consolidation-due nudge + scaffold-ensure wiring	← this
SB-S3 — Cross-silo graph + ops centre	thread the id through tributaries; wire profile.md READERS; flesh out operations.md; retire dead content-history.md	later
SB-S4 (optional)	EU/EEA DMA portability auto-tributary	later

5. SB-S2 in detail (the build target)

5.1 Deliverables

1. Deterministic consolidation engine (scripts/brain/src/consolidate.ts) — PURE functions, no IO, heavily tested. Core: proposeDiff({ current: ProfileDoc, candidates: Candidate[], today, opts }) → ProfileDiff. A Candidate = { key, value, provenance, source, observed_date }. Matching: every candidate is keyed mintEntityId({kind:'observed', key}) and matched ONLY against existing observed-kind facts. The SB-S0 folded profile-field static seeds are immutable in S2 — a candidate that semantically overlaps a folded field becomes its own observed fact (the keep-both philosophy; reconciling observed-vs-folded is operator-gated / S3). This avoids the profile-field↔observed id mismatch (fixes brief-review NICE). Classification rules:

   • Reject any provenance: 'ai-draft' candidate outright (model-collapse guard) — it never enters or bumps a fact.
   • Add (no matching observed fact, provenance ∈ {published, human}) → a new dynamic fact, evidence_count: 1, dates = observed_date/today.
   • Evidence-bump (matching fact, SAME value) → evidence_count++, last_seen = today (no other field touched).
   • Promote (a dynamic fact whose evidence_count reaches threshold N, default 3) → move dynamic→static.
   • Conflict (matching key, DIFFERENT value) → keep-both-timestamped: add the new value as its own fact, keep the old fact untouched (no evidence-bump of the old), and record the pair in conflicts[] — the architecture's explicit bias against silent overwrite. No supersede in S2 (flipping status:superseded is deferred to S3; S2 never demotes/supersedes a fact).
   • Decay-flag — dynamic facts whose last_seen is older than DECAY_DAYS (default 90) are listed in staleFlags[] (information only; S2 never auto-demotes). ProfileDiff = { additions[], evidenceBumps[], promotions[], conflicts[], staleFlags[] } (typed; NO supersessions[] — that arm is S3). The engine NEVER mutates profile.md — it returns proposals over a deep-frozen-safe input. applyDiff(current, diff) → ProfileDoc is a separate pure function (used only by the gated apply path); its output round-trips through parseProfile/serializeProfile.

2. brain consolidate CLI (extend scripts/brain/src/cli.ts) — the operator-invoked loop:

   • brain consolidate --gather [--json] — read-only: gather the deltas since the last run (new published records + each tributary's newest-timestamp summary + the current profile) and print them, for the invoking session to read and turn into candidate facts. (The fuzzy extraction lives here, in the session — no new agent.)
   • brain consolidate --propose --candidates <file.json> — validate the candidate file against the Candidate shape (malformed/missing field → non-zero exit, no profile touch — the only enforceable session↔engine contract), run proposeDiff over the current profile + the candidates, and write TWO artifacts to brain/: pending-diff.md (operator-readable, for the SendUserFile gate) AND pending-diff.json (the typed ProfileDiff, machine-applyable). Print both paths. Does NOT touch profile.md.
   • brain consolidate --apply --diff brain/pending-diff.json --confirm — apply the confirmed JSON diff to profile.md (the only path that writes the profile), then update the consolidation sidecar (D3). Operator-gated (requires --confirm); never auto-runs.

3. Last-consolidation state — a brain data-root sidecar brain/consolidation-state.json ({ "last_run": "YYYY-MM-DD" }), written by --apply and read by the session-start nudge. This resolves brief-review B1+B2: both the brain CLI (dataRoot) and the session-start hook (getDataRoot) resolve the data root, so the sidecar is reachable by both — whereas the plugin state file lives in HOME (~/.claude/linkedin-studio.local.md), a SIBLING of the data root the brain CLI cannot reach through its own resolver, and state-updater.mjs has no last_import_date writer to mimic (that scalar is written by the import command via LLM, not a function). Keeping consolidation state in the brain's own data-root territory is also conceptually right (brain state, not plugin state) and zero-dep JSON for the hook to read.

4. Session-start wiring (Edit hooks/scripts/session-start.mjs — existing file, zero-dep, pathguard-safe; all reads via the existing getDataRoot, the trend-nudge twin idiom):

   • Scaffold-ensure (zero-dep): mkdir the brain/ + ingest/ dirs if absent (idempotent, no tsx); if brain/profile.md is absent, emit a nudge to run brain init (seeding needs the tsx fold — kept manual, not inlined).
   • Brain consolidation-due nudge (new-published-count only): read brain/consolidation-state.json last_run (zero-dep JSON parse) + count ingest/published/*.md records (a readdir count — no profile.md parse, to respect the 10s budget + avoid a fragile zero-dep fact parser). Trend-nudge discipline: daysSince(last_run) + threshold + null-safe (a never-consolidated brain with no published records never nags). Emit e.g. "N published posts captured, last consolidation M days ago → run brain consolidate."
   • Out of scope (honest): stale-FACT nudging (parsing profile.md last_seen at session-start) is NOT shipped in S2 — it needs a zero-dep fact parser + cost, and bites harder once a reader exists. S2's brain-wide nudge = the new-published/consolidation-due signal only; per-fact decay is computed inside proposeDiff (staleFlags[]), surfaced in the diff, not at session-start.

5. Operator-gated diff presentation — the --propose artifact is a readable markdown diff (additions/bumps/promotions/conflicts/stale, sorted, neutral/anti-sycophantic framing); the invoking session SendUserFiles it and presents the [OPERATØR] gate before any --apply (the editorial-review idiom).

6. Docs — docs/second-brain/consolidation-loop.md (the loop contract: CLI usage, the engine rules, the operator-gate, the honest limits); this brief + the SB-S2 plan; flip the architecture.md SB-S2 row to landed.

7. Tests + gate — engine unit tests (every classification rule + idempotency + provenance-gating + threshold + keep-both + decay + applyDiff round-trip), CLI tests (gather/propose/apply, no-auto-apply, last-run recording), and a session-start nudge test in the hooks test suite (consolidation-due nudge fires/null-safe). Bump BRAIN_TESTS_FLOOR + the assertion floor (per the SB-S1-established empirical method) + the hook-test count if applicable.

5.2 What "owns the session-start wiring of the scaffold init" means here

S2 makes the brain present + nudged at session-start without running tsx: it zero-dep-ensures the dirs and nudges brain init (profile seeding stays the manual tsx CLI) + nudges brain consolidate when due. It does NOT run the consolidation pass automatically (that needs AI → operator-invoked).

6. Success criteria (SB-S2 — testable)

Engine rules each get their OWN criterion (fixes brief-review M5; N=3, DECAY_DAYS=90 pinned as the test constants — confirm §9 Q1 at "go" first):

• SC1a (add): a candidate with no matching observed fact + provenance ∈ {published,human} populates additions[] as a dynamic fact, evidence_count:1.
• SC1b (reject ai-draft): an ai-draft candidate yields an EMPTY diff — no add, no bump, no promote (the model-collapse guard, in code).
• SC1c (evidence-bump): a candidate matching an existing fact with the SAME value populates evidenceBumps[] (evidence_count++, last_seen=today), no other field changed.
• SC1d (promote): a dynamic fact reaching evidence_count ≥ 3 populates promotions[] (dynamic→static).
• SC1e (conflict→keep-both): a candidate matching a key with a DIFFERENT value populates conflicts[], ADDS the new value as its own fact, and leaves the old fact untouched (no evidence-bump, no supersede) — both retained.
• SC1f (decay-flag): a dynamic fact with last_seen older than 90 days is listed in staleFlags[]; no auto-demotion.
• SC1g (folded immutable): a candidate overlapping a folded profile-field static seed creates a separate observed fact and never mutates the folded one.
• SC2 (immutability): proposeDiff does not mutate its current/candidates inputs (asserted by structural equality of the inputs after the call).
• SC3 (apply round-trip): applyDiff(current, proposeDiff(...)) yields a ProfileDoc that round-trips exactly through parseProfile/serializeProfile.
• SC4 (idempotency): re-running propose→apply with the same candidates over an already-applied profile yields no spurious additions (evidence-bumps/last_seen only; no duplicate facts).
• SC5 (CLI gating + validation): --gather and --propose never write profile.md (asserted byte-unchanged after each); --propose writes pending-diff.{md,json} to brain/ and REJECTS a malformed candidate file (non-zero exit, no profile touch); only --apply --confirm writes profile.md and updates brain/consolidation-state.json. Verified by temp-dir CLI tests.
• SC6 (session-start nudge, zero-dep): the consolidation-due nudge fires when published records exist + last_run is stale/absent, and is silent when the brain is absent / nothing new; the edit adds no tsx/heavy dep and parses no profile.md. Verified by a hook-suite test (HOME + LINKEDIN_STUDIO_DATA set to separate temp dirs — the session-start-trends-staleness.test.mjs harness pattern — asserting the nudge string appears / is absent).
• SC7 (no regression / no new seam / count-neutral): scripts/test-runner.sh green; floors hold (trends ≥24, specifics ≥28, contract ≥33, brain ≥ new floor); hook tests pass; no new agent/command/ref/skill (counts unchanged); no new seam function (twin-sync untouched); compile-hooks.py --check clean if hooks.json is regenerated.

7. Non-goals / scope fence (SB-S2 does NOT)

• No journal-capture (deferred sub-slice) — S2 reads published + tributary deltas only; nothing writes brain/journal/.
• No new agent — the fuzzy extraction is done by the invoking session (no agents/*.md, no reload, no agent-count bump).
• No profile.md READER — no content agent/command is wired to consume brain/profile.md (S3).
• No cross-silo id threading into tributaries (S3); no content-history.md retirement / triple-post reconciliation (S3).
• No AI at session-start — only a deterministic nudge; the consolidation pass is operator-invoked.
• No auto-apply — no diff is ever applied without --apply --confirm; profile.md is mutated by exactly one gated path.
• No new hook .mjs — only the existing session-start.mjs is edited (pathguard: Write of a new hooks/scripts/*.mjs is blocked).
• No new seam function, no connector (S4), no GUI.

8. Constraints / risks

• TDD iron law: no production code without a failing test first.
• Code placement / pathguard: engine + CLI are TS under scripts/brain/ (safe). The session-start change is an Edit of the existing hooks/scripts/session-start.mjs (Edit is unaffected by the Category-6 Write guard) — no new .mjs under hooks/scripts/.
• Zero-dep hook discipline: the session-start edit must NOT import tsx/the brain TS, must stay within the 10s timeout, and must use direct raw reads (the trend-nudge twin idiom). The brain consolidation is NOT run there.
• Hook compile: if the SessionStart entry changes shape, regenerate hooks.json via compile-hooks.py (drift-guarded). Editing only the script body needs no recompile.
• Consolidation state is a brain sidecar, NOT the plugin state file (fixes brief-review B1+B2): brain/consolidation-state.json under the data root, written by --apply (dataRoot) + read by the session-start hook (getDataRoot, zero-dep JSON). Do NOT route it through ~/.claude/linkedin-studio.local.md — the brain CLI cannot reach HOME, and state-updater.mjs has no last_import_date-style writer to mimic (that scalar is an LLM/command write). Reading it at session-start is a JSON.parse(readFileSync(...)), the trend-trends.json-read twin.
• Public repo (open/): confirm before push; window; STATE.md gitignored.
• Count-neutral: no agents/commands/references/skills file added → no count guard fires (absent a version bump). Version bump likely 0.5.1 → 0.5.2 (new capability + session-start behavior) — confirm at "go".
• Risk — engine rule ambiguity: the conflict-vs-supersede call is the fuzzy heart. Mitigation: bias to keep-both-timestamped (deterministic + safe), reserve supersede for an explicit confirm signal; pin each rule with a unit test before wiring the CLI.
• Risk — candidate extraction quality: the value of the loop depends on the session extracting good candidates from the gathered deltas; the engine only guarantees the mechanics. Honest limit (the architecture's "voice fidelity is hard"): S2 ships the machinery, not a guarantee of insight.
• Risk — unconsumed output: profile.md has no reader until S3; S2's value is deferred (compounding for later). Accepted by the operator; stated in consolidation-loop.md.
• Risk — session-start cost: the nudge must stay cheap; counting published/ files + one scalar read is O(files) — bound it (e.g. count via readdir, no parse) to respect the 10s budget.

9. Open questions for the operator (settle at "go")

1. Promotion threshold N + decay window: default N = 3 independent observations to promote dynamic→static, DECAY_DAYS = 90 for stale-flagging. Confirm or adjust. (Recommend N=3 / 90d — pinned as the test constants.)
2. Version bump: 0.5.1 → 0.5.2 for the new consolidation capability + session-start nudge? (Lean 0.5.2, consistent with SB-S1.)

Resolved by the brief-review (no longer open): conflict = keep-both only, no supersede in S2 (supersede→S3); diff artifacts = brain/pending-diff.{md,json} (git-trackable per "roll-back via git").

10. Definition of done (SB-S2)

SC1a–g + SC2–SC7 pass; deliverables 1–7 exist; non-goals untouched; the engine is provenance-gated + keep-both-only (no supersede) in CODE (not just docs); consolidation state is the brain sidecar (not the plugin state file); the consolidation pass is operator-invoked + the diff operator-gated (no auto-apply); session-start nudge zero-dep (no profile.md parse) + tested; committed (push per window, confirm first); architecture.md SB-S2 row → landed; version/CLAUDE.md/README touched only if a bump is taken.