linkedin-studio/docs/second-brain/plan-sb-s3e.md
Kjell Tore Guttormsen fbad29d3d4 docs(linkedin-studio): SB-S3e brief + plan — hygiene + triple-post reconciliation (light-Voyage hardened)
The LAST S3 slice: (b) retire the dead, zero-reader content-history.md across
its 8 plumbing surfaces; (c) a read-side `brain reconcile` joining silo 1
(## Recent Posts, auto-tracked creation) to the silo 2↔3 graph — surfacing the
coverage gap (posts created but never `brain ingest`-ed), without writing the
state silo.

Light-Voyage hardened: scope-guardian ALIGNED (0/0/0); brief-reviewer REVISE
(4 FIX) + plan-critic REVISE (3 blockers + 6 major + 4 minor), all folded. The
three blockers were real factual errors against the code, each verified directly
before folding: listPublished() is body-less (use runAssemble's inline
parsePublishedRecord loader); the core signature must be {recentPosts, records,
graph} (PostGraphNode has no body); the parser tracks the WRITER format
(state-updater.mjs:116), not pruneContentHistory's date-only :145 regex. Plus the
getStateFile() HOME-chain seam, logic-RED, single-literal gate decoys, recorded
gate TOTAL 97.

Go-before-code gate pending (forks: retire vs back-fill; read-side vs write-side).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RigJBiRFNtFZKCz21qNbQ4
2026-06-23 21:51:14 +02:00

23 KiB
Raw Permalink Blame History

Plan — SB-S3e: hygiene (retire dead content-history.md) + triple-post reconciliation (read-side)

Brief: docs/second-brain/brief-sb-s3e.md. Slice: SB-S3e — the LAST S3 slice (arc complete after; S4 optional). Recommendations pending the go-gate: (b) retire content-history · (c) read-side joiner (brain reconcile) · build now (not doc-only) · brain-CLI-only (no new plugin command/agent). Order is TDD: the failing reconcile test + gate self-test land BEFORE the reconcile code (iron law). The retirement is deletion — its proof is grep-zero + the migrate suite staying green.

Goal

Two coupled deliverables: (b) the dead, zero-reader content-history.md is removed across all eight of its plumbing surfaces (writer prose, template, migration entry+test, gitignore, gate guard, ref-doc, comment); (c) a new read-only brain reconcile reconciles silo 1 (## Recent Posts, auto-tracked creation) against the silo 2↔3 graph, surfacing the coverage gap (created posts never ingested into the brain) — without ever writing the state silo. No state-file write, no $-injection surface, no auto-capture.

Files touched (exhaustive — for scope-guardian)

(b) Retirement

File Change SC
hooks/prompts/state-update-reminder.md remove Section 5 "Content History Log" (:67-86) — the sole writer SC1
config/content-history.template.md DELETE SC1
hooks/scripts/migrate-data.mjs remove the B1 MOVE_FILES entry (:33) SC1
hooks/scripts/__tests__/migrate-data.test.mjs drop content-history fixture (:37), its 2 assertions (:56,:61), trim test-(a) description (:50); cases ae stay (count unchanged) SC2
.gitignore remove assets/analytics/content-history.md (:44) SC1
scripts/test-runner.sh narrow SC2_CLASSES (:592) — drop the content-history\.md alternative (check stays → gate count unchanged) SC1/SC5
references/data-path-convention.md remove content-history.md from the data-tree diagram (:26) SC1
hooks/scripts/session-start.mjs drop content-history from the comment list (:94) SC1

(c) Reconciliation (read-side)

File Change SC
scripts/brain/tests/reconcile.test.ts NEW — RED-first: coverage tiers + truncated-hook degradation + empty-state SC3
scripts/brain/src/reconcile.ts NEW — pure reconcileRecentPosts + parseRecentPosts + loadRecentPosts IO seam SC3/SC4
scripts/brain/src/cli.ts add reconcile subcommand (runReconcile) + dispatch (:312) + usage line SC4
scripts/test-runner.sh NEW Section 16f (2 unconditional checks) + ASSERT_BASELINE_FLOOR 82→84 + BRAIN_TESTS_FLOOR 114→recount + header enumeration SC4/SC5
docs/second-brain/architecture.md mark the SB-S3 build-row S3e (arc complete) SC7

Not touched (scope fence): hooks/scripts/state-updater.mjs, ## Recent Posts, pruneContentHistory (S3e reads silo 1, never writes) · assemble.ts join heuristic (reconcile consumes the graph) · consolidate.ts/ingest.ts (no engine change) · consolidation-loop.md · CLAUDE.md command/agent tables (no new command/agent; brain reconcile is an internal CLI subcommand) · README :120 conceptual "content-history alignment" (concept, not the file — verify-then-leave).

Step 1 — (RED) Reconcile test against the absent module

scripts/brain/tests/reconcile.test.ts — model the IO/pure split on assemble.test.ts. The RED is a logic-RED, not an import-RED: land a stub reconcile.ts first that compiles + exports the symbols but mis-classifies (returns [] / a wrong tier), so the named tier assertions go red against logic, not a MODULE_NOT_FOUND. Cases:

  1. parseRecentPosts extracts {date, hook, charCount, topic} from a ## Recent Posts block whose entries match the writer's exact format (updatePostTracking, state-updater.mjs:116: - [YYYY-MM-DD] "hook" (chars) - topic) — the format source of truth, NOT the date-only pruner regex (:145). A golden-string case feeds a literal writer-produced entry and asserts the round-trip. A non-## Recent Posts doc → []; a $-bearing topic/hook round-trips verbatim (capture/match on read — no String.replace → no regex injection).
  2. reconcileRecentPosts({recentPosts, records, graph}) (the pinned 3-arg signature — records carry body, which PostGraphNode lacks) classifies each silo-1 entry:
    • in-graph — a published record matches (date + hook-prefix on record.body) AND its graph node has an analytics match → the full chain is observable.
    • in-brain-only — record matches but its graph node's analytics confidence==='none'.
    • orphaned-in-state — no record matches (created via the plugin, never brain ingest-ed) → the coverage gap.
    • summary counts (created N · in-graph M · orphaned K).
  3. Truncated-hook degradation — a ≤60-char preview ending ... strips the marker before prefix-matching (mirror assemble.ts: stripTrailingEllipsis); a preview too short to discriminate (below a PREFIX_FLOOR-style guard) → orphaned, never a false in-graph.
  4. Empty-state — empty/absent ## Recent Posts → empty report, no throw.

RED proof (recorded — logic-RED): with the mis-classifying stub in place, (cd scripts/brain && npm test) → the named tier assertions fail (e.g. "expected orphaned-in-state, got …"). Capture that transcript as the iron-law artifact in STATE/changelog — not a bare import error. Circuit-breaker: if the RED run is not recorded before the real reconcile.ts logic lands, halt and redo Step 1 (the RED proof is the only circuit breaker in this plan).

Step 2 — (GREEN) reconcile.ts — pure core + parser + IO seam

scripts/brain/src/reconcile.ts:

  • parseRecentPosts(stateText: string): RecentPost[] — regex the ## Recent Posts section against the writer's format (updatePostTracking, state-updater.mjs:116: - [${postDate}] "${hookPreview}" (${charCount}) - ${postTopic}) → e.g. /^- \[(\d{4}-\d{2}-\d{2})\] "(.*)" \((\d+)\) - (.+)$/gm. NB: this is NOT pruneContentHistory's regex — that one (:145) is date-only (/^- \[(\d{4}-\d{2}-\d{2})\] .+$/gm) and captures no hook/chars/topic; parseRecentPosts is the first full-field reader of this section and tracks the writer, not the pruner. Pure.
  • reconcileRecentPosts({recentPosts, records, graph}): ReconcileNode[] + summarize(...) — pure, no FS/clock. records is the full PublishedRecord[] (carry body); graph is PostGraphNode[] (carry contentId + analytics match, NO body). Reuse assemble.ts's normalize / stripTrailingEllipsis idiom (brain-local copy, NOT a new shared util — the repo's accepted duplication, dataRoot.ts header). Match recentPost.hook (ellipsis-stripped, normalized) as a prefix of record.body; confidence by date equality; below the prefix floor → orphaned. Then look up the matched record.id in graph to read its analytics tier (in-graph if the node's match.confidence !== 'none', else in-brain-only).
  • loadRecentPosts(): RecentPost[] — the new cross-seam IO. Resolve the state file with the canonical precedence already exported as data-root.mjs: getStateFile()process.env.STATE_FILE || join(resolveHome(), ".claude", "linkedin-studio.local.md") where resolveHome() = process.env.HOME || process.env.USERPROFILE || homedir() (data-root.mjs:17-19). NOT bare homedir() — that silently ignores a HOME/USERPROFILE override the writer honours (state-updater.mjs:12); and NOT dataRoot() (that is the dir under LINKEDIN_STUDIO_DATA). The brain TS copies getStateFile()'s chain because it cannot import the .mjs. Read read-only; absent → [] (fresh-clone safe). Header caveat: this is a new root-skew seam (state file via STATE_FILE, a different root than the brain dataRoot) — not a mirror of assemble.ts's ANALYTICS_ROOT note.

GREEN gate (brain): (cd scripts/brain && npm test) → reconcile cases pass; record the new live tests total → BRAIN_TESTS_FLOOR 114→that number (recorded, not guessed; update the inline breakdown comment test-runner.sh:718 with "+ SB-S3e N [reconcile]").

Step 3 — (GREEN) Wire brain reconcile into the CLI

scripts/brain/src/cli.ts:

  • Add runReconcile() (model on runAssemble, :161-191). Loader (BLOCKER fix): do NOT use listPublished() — it takes no args and returns body-less PublishedSummary[] (ingest.ts:262-289). Instead reuse runAssemble's inline loader (cli.ts:163-174): readdirSync(pubDir).filter(.md).map(parsePublishedRecord) → real PublishedRecord[] (with body/specifics/trends). Then const analytics = loadAnalyticsRows(), const graph = assemblePostGraph({records, analytics}), const recentPosts = loadRecentPosts(), and call the core by its literal name reconcileRecentPosts({recentPosts, records, graph}) (no alias/re-export wrapper — Section 16f Check B greps cli.ts for the literal reconcileRecentPosts). Print per-post lines (· <date> · <tier> · "<hook>") + summary + nudge ("K created posts are not in the brain graph — brain ingest --file <p> to feed them"). Read-only.
  • Dispatch: insert if (command === "reconcile") return runReconcile(flags); after the assemble dispatch line (:312, i.e. at the new :313 before usage(...)).
  • Usage: add " reconcile" to the usage block after the " assemble" line (:94).

GREEN gate (lint): Section 16f Check B now passes (see Step 5).

Step 4 — (b) Retire content-history.md (deletions)

Apply the eight edits in the (b) table. Order: remove the writer (state-update-reminder.md Section 5) first, then template, migration entry, gitignore, gate guard, ref-doc, comment. Then trim migrate-data.test.mjs — drop the content-history fixture write + the two assertions + the words "content-history moved" from the test-(a) name; cases ae remain.

Gate (hook suite — separate runner), recorded: node --test hooks/scripts/__tests__/migrate-data.test.mjs → cases ae green (capture the transcript — R8 is demonstrated, not asserted). Confirm cases be never expected analytics/content-history.md as a present path (they don't — only the deleted case-(a) fixture/assertions referenced it; be exercise MOVE_DIRS / idempotency / collision / empty), so removing the B1 MOVE entry cannot regress them. SC1: grep -rn "content-history" hooks/ config/ scripts/test-runner.sh .gitignore references/data-path-convention.md → zero (the README :120 concept mention is out of this grep set; confirm it is the concept, leave it).

Step 5 — (RED→GREEN) Gate Section 16f + floors + SC2 narrow

Insert Section 16f after Section 16e (test-runner.sh:885, before Section 17 :886), modelled on 16e's two-check idiom (self-test + real-file grep, grep -qF fixed-string):

  • Literals: RECON_CLI_LIT='command === "reconcile"', RECON_CORE_LIT='reconcileRecentPosts'.
  • reconcile_wired() — wired iff a probe carries BOTH literals (echo twice; grep -qF — both literals contain regex-special chars, so -F is mandatory).
  • Check A (self-test, unconditional, non-vacuous): a fully-wired probe MUST be detected; the rejected probes must each miss exactly one literal (the load-bearing discriminators), plus a specificity decoy:
    • D1 (discriminator): dispatch present, core call absent — command === "reconcile" but no reconcileRecentPosts → must REJECT (proves the core literal is load-bearing).
    • D2 (discriminator): core call present, dispatch absent — reconcileRecentPosts(...) but no command === "reconcile" → must REJECT (proves the dispatch literal is load-bearing).
    • D3 (specificity): a sibling command fully wired to the WRONG domain — command === "assemble" + assemblePostGraph (neither reconcile literal) → must REJECT (proves the predicate is reconcile-specific, not "any command + any core"). → pass/fail "reconcile self-test: full wiring detected; D1/D2 single-literal forms + D3 assemble specificity decoy rejected".
  • Check B (real-file grep, unconditional): grep -qF "$RECON_CLI_LIT" scripts/brain/src/cli.ts && grep -qF "$RECON_CORE_LIT" scripts/brain/src/cli.tspass "brain CLI wired to reconcile (dispatch + core call by literal name)" else fail. Both literals live in cli.ts (the dispatch if + runReconcile's literal-name call to the core — Step 3 pins "no alias"), so one file carries both → deps-free, like 16e Check B.
  • Header enumeration (:33-39): insert the 16f clause between the 16e clause's terminal "…in Section 16e;" (:38) and the "the assertion-count anti-erosion floor (SC6) in Section 18." tail — preserving 16e → 16f → 18 ordering. The range "Sections 818" is unchanged (16f is in range).
  • ASSERT_BASELINE_FLOOR (:966): 82 → 84 (+2 deps-free unconditional checks — they lift the deps-absent minimum, NOT pinned to the deps-present TOTAL); extend the history comment ("+2 for SB-S3e's two Section-16f checks (reconcile self-test + brain-CLI reconcile grep) = 84").
  • SC2_CLASSES (:592): delete the assets/analytics/content-history\.md| alternative (now retired). The SC2 check is still one pass/fail → gate TOTAL change comes only from 16f's +2.

RED proof: between Steps 1 and 3, a gate run shows Section 16f Check A passing (self-test is self-contained) and Check B FAILING (cli.ts not yet wired); after Step 3 both pass.

GREEN gate (full): bash scripts/test-runner.sh → FAIL=0; TOTAL recounted live. Recorded pre-change baseline (verified live this session, 2026-06-23): 97/0/0 → expected 99 after 16f's +2 (recount at land, never pin); ASSERT floor 84 honoured; SC2-dry-run green with the narrowed regex.

Step 6 — Doc reconciliation

Edit the architecture.md SB-S3 build-row: S3e ✅ — "dead content-history.md retired; brain reconcile reconciles post-tracking ↔ published ↔ analytics (read-side; auto-capture = follow-up)" — and note the arc is complete (S4 EØS-connector optional). No other architecture edit.

CLAUDE.md surface (open-Q4, resolved in-plan): brain reconcile is an internal brain-CLI subcommand, the same class as brain assemble/ingest/consolidate — none of which appear in CLAUDE.md's command/agent tables (those count plugin commands/agents). So no CLAUDE.md count change; the brain reconcile mention lives in the architecture.md build-row above. Confirm at the gate, but the default is no-plugin-surface.

Step 7 — Behavioural verification (manual, recorded — SC4/SC6)

Not unit-testable (operator-accepted command-testing gap). Honest procedure, result recorded at land:

  • SC4 (e2e): seed a temp root — a ## Recent Posts block (via STATE_FILE), 12 ingest/published records (one matching a Recent-Posts entry, one not), an analytics batch — run brain reconcile; confirm it prints in-graph / orphaned tiers + the summary + nudge.
  • SC6 (root-skew): with STATE_FILE=<tmp> it reads silo 1 from the temp file; unset → falls back to $HOME/.claude/linkedin-studio.local.md. Honesty hedge (verifiseringsplikt): if the per-call seam can't be exercised, record wiring-inspected-only, NOT a behavioural pass.

Step 8 — Land

STATE "Telling" + "👉 NESTE" updated (S3e done → arc complete; next = strategic checkpoint / product-maturity). Commit. Mixed: reconcile + retirement + gate are code ([skip-docs]); brief+plan+architecture are docs. Push only inside the window (date '+%u %H:%M' first; origin = PUBLIC open/ → confirm before push). No version bump (additive within v0.5.2 dev; release separate).

Verification (testable)

SC Check Command Expected
SC1 content-history retired grep -rn content-history hooks/ config/ scripts/test-runner.sh .gitignore references/ + ls config/content-history.template.md zero hits; file gone
SC2 migration green node --test hooks/scripts/__tests__/migrate-data.test.mjs cases ae pass
SC3 reconcile core (RED→GREEN) (cd scripts/brain && npm test) reconcile cases pass; floor recounted
SC4 reconcile wired bash scripts/test-runner.sh + manual brain reconcile Section 16f Check B passes; report prints
SC5 gate green + floor same run 99/0/0 (confirm live); ASSERT floor 84; SC2-dry-run green
(red proof) failing-test-first gate + brain run BETWEEN Steps 1 and 3 reconcile test fails; 16f Check B fails
SC6 root-skew manual, STATE_FILE set/unset reads the pointed file / falls back (or honest fallback recorded)
SC7 counts reconciled inspect architecture S3e ; STATE telling updated; no CLAUDE.md command/agent count change

Risks

  • R1 — vacuous gate. Mitigated by the 16e-style non-vacuity self-test with the assemble sibling-command decoy.
  • R2 — seam-mismatch (the load-bearing one). loadRecentPosts() MUST resolve the state FILE via getStateFile()'s exact chain — STATE_FILE || join(resolveHome(), '.claude', 'linkedin-studio.local.md'), resolveHome() = HOME||USERPROFILE||homedir() (data-root.mjs:17-19,30-31) — NOT bare homedir() (ignores a HOME/USERPROFILE override) and NOT dataRoot() (the DIR under LINKEDIN_STUDIO_DATA). A divergence silently reads the wrong path. Mitigated: chain copied from the canonical getStateFile() (not state-updater.mjs:12, which falls through to '' — a latent bug data-root.mjs:14-16 warns against); SC6 exercises both STATE_FILE branches.
  • R3 — parse drift. The reconcile entry regex must track the writer updatePostTracking (state-updater.mjs:116); if the writer's format changes, the reader silently misses. pruneContentHistory (:145) is date-only and would NOT catch such drift, so it is not a co-mover. Mitigated: a golden-string test feeds a literal writer-format entry through parseRecentPosts and asserts the round-trip — pinning the exact write format as the contract.
  • R4 — retirement over-reach. Deleting a conceptual "content history" mention (README :120, or pruneContentHistory's name) would break live behaviour. Mitigated: the (b) table is file-plumbing-only; the scope fence + SC1's bounded grep set exclude the concept; pruneContentHistory (operates on ## Recent Posts) is explicitly out.
  • R5 — $-injection on READ. parseRecentPosts uses match/capture (read), never String.replace with an untrusted replacement — so a $-bearing hook/topic round-trips verbatim (Step 1 case 1 pins it). No write path → no Section-13 state-updater surface touched.
  • R6 — floor false-fail on fresh clone. 16f's two checks are deps-free (+2 → 84, correct). BRAIN_TESTS_FLOOR only checked when the brain suite runs (deps-present); a fresh clone warn-skips it.
  • R7 — honest-limit overclaim. The brief/CLI must state plainly that read-side reports the coverage gap but cannot reconstruct un-captured specifics/trends (auto-capture = follow-up). Mitigated: §3 honest-limits + Step 7 verifiseringsplikt hedge.
  • R8 — migrate idempotency. Removing the B1 MOVE entry must not break the .migrated marker / COPY-MOVE classes. Mitigated: cases be (idempotent re-run, collision, empty) stay green in Step 4's run.

Plan-critic — folded

voyage:plan-critic (3 blockers + 6 major + 4 minor) — each verified directly against the code before folding:

  • B1 — body-less loader (load-bearing): listPublished() takes no args and returns PublishedSummary ({id,provenance,published_date,firstLine}, ingest.ts:262-289) — NO body. The reconcile prefix-match AND assemblePostGraph both need PublishedRecord.body. Folded Step 3: use runAssemble's inline readdirSync().map(parsePublishedRecord) loader (cli.ts:163-174), which yields real PublishedRecord[] (and already builds a bodyById Map).
  • B2 — core-signature contradiction: brief said {recentPosts, graph}; PostGraphNode has no body → graph-only can't prefix-match. Folded to the 3-arg {recentPosts, records, graph} in brief §3(c).1 + plan Steps 1/2 — one signature, agreed across brief/test/core.
  • B3 — regex misattribution: plan cited pruneContentHistory's :145 regex as full-field; it is date-only. Folded Step 2 + R3: the parser tracks the writer :116; parseRecentPosts is the first full-field reader; a golden-string test pins the write format.
  • M4 — HOME seam: loadRecentPosts must use getStateFile()'s resolveHome() chain (HOME||USERPROFILE||homedir()), not bare homedir() (ignores an override the writer honours). Folded Step 2 + R2, citing data-root.mjs:17-19,30-31 as canonical (not state-updater.mjs:12, which falls through to '').
  • M5 — weak gate decoys: the both-absent assemble decoy was trivial. Folded Step 5: D1/D2 now miss exactly one literal each (the load-bearing discriminators); D3 (assemble sibling) is reframed as a specificity decoy.
  • M6 — Check-B literal name: reconcileRecentPosts appears in cli.ts only if runReconcile calls the core by its literal name. Folded Step 3 ("no alias/wrapper") + Step 5 Check-B wording.
  • M7 — unrecorded TOTAL: Folded Step 5 — recorded the live pre-change gate 97/0/0 (2026-06-23); 99 is the expected post-16f count, recount at land.
  • M8 — migrate idempotency asserted not shown: Folded Step 4 — require the recorded node --test migrate-data.test.mjs green run; confirm cases be never expected content-history present.
  • M9 — SC1 grep divergence: the brief's --include-filtered grep would skip .gitignore/.md. Folded — brief SC1 unified with the plan's unfiltered explicit-list grep.
  • m10 — header insertion point: Folded Step 5 — insert the 16f clause between the 16e clause terminal and the Section-18 tail.
  • m11 — :312/:94 line refs: Folded Step 3 — "after :312" / "after :94" (insertion, not the existing line).
  • m12 — CLAUDE.md deferred: Folded Step 6 — resolved in-plan: brain reconcile is internal brain-CLI (like assemble), no CLAUDE.md count change.
  • m13 — circuit-breaker: Folded Step 1 — "if RED not recorded before reconcile.ts logic lands, halt and redo Step 1."

voyage:scope-guardian: ALIGNED — 0 creep / 0 gaps / 0 dependency issues. All 14 IN items map to a step; no NON-GOAL file touched; scope fence consistent (slightly broader/defensive); doc reconciliation proportional. Every file/line/symbol verified live.

Cross-checks confirmed sound: the (b) retirement arm was already nearly executable (all 8 surfaces verified); floor arithmetic (ASSERT 82→84 from 16f's +2; SC2-narrow nets 0; BRAIN 114→recount); the "read-side does not close the gap" honesty (auto-capture = the real closer, follow-up) stands; $-injection is absent on the read path (capture/match, never String.replace).