linkedin-studio/docs/second-brain/plan-sb-s3c.md

Plan — SB-S3c: cross-silo id-threading

Brief: docs/second-brain/brief-sb-s3c.md. Slice: SB-S3c. Status: LANDED (operator go 2026-06-23) — brain 113/113, gate 95/0/0, BRAIN_TESTS_FLOOR 94→113. Design: hub-side threading — specifics/trends ids onto the brain's published record + a pure analytics resolver. Tributaries untouched. Light-Voyage: brief-review APPROVE-WITH-FIXES (5 FIX folded into the brief; 3 RISK carried here). plan-critic REVISE → 1 BLOCK + 4 MAJOR + 4 MINOR folded (Step 1 compile-fixup · Step 2.2 non-throwing parse reader · Step 2.3 producer guard · Step 3.3 floor rationale · Step 3.4 deterministic row-sort · Step 3.5 analytics-root caveat · Step 4.2 runIngest signature+call-site · Step 4.3 full-record load + desc sort · R-G sub-count). scope-guardian ALIGNED (0 creep / 0 gap). Order is TDD: failing brain-suite tests land BEFORE the code (iron law); each rule pinned before the CLI is wired.

Goal

A published post records the raw material it was built from (specifics/trends ids, additive + backward-compatible on the post record), and a pure assembler joins published-record ↔ analytics-row (by normalized title-prefix + date, with honest confidence tiers) so the north-star query — which raw material actually performs? (specific → post → measured analytics, architecture.md:17) — is assemblable. All four tributary schemas stay untouched; the only new disk surface is a read-only brain assemble. Fully unit/CLI-testable with fixtures.

Files touched (exhaustive — for scope-guardian)

File	Change	SC
scripts/brain/src/ingest.ts	PublishedRecord (+specifics: string[] +trends: string[], :27-40) · serializePublishedRecord append the two lines after source:, omit-empty (:50-58) · parsePublishedRecord read them via a new list-scalar reader, 12-hex-validate each (reuse the :82 guard idiom), absent→[] (:75-99) · ingestText (+optional specifics?/trends?, :165-180)	SC1,SC2,SC3,SC4,SC11
scripts/brain/src/assemble.ts	NEW. Pure assemblePostGraph({records, analytics}) + local normalize() + matchRow() (tiers) + minimal input type AnalyticsRowInput + output types · thin read-only loadAnalyticsRows() IO (inline raw-JSON read of dataRoot('analytics/posts')/*.json → AnalyticsBatch.posts[]; never imports the analytics package)	SC6,SC7,SC8
scripts/brain/src/cli.ts	collectRepeated(args,key): string[] helper (leaves parseFlags untouched — SC12 by construction) · change runIngest signature (flags)→(rest, flags) AND the main dispatch runIngest(flags)→runIngest(rest, flags) (:92, :238 — BLOCK 1), thread collectRepeated(rest,"specific")/"trend" · runAssemble(flags) (read-only print, full-record load) · dispatch + if (command==="assemble") (:240) · usage text (+assemble, +ingest flags)	SC5,SC9,SC12
scripts/brain/tests/ingest.test.ts	SC1 (round-trip w/ ids) · SC2 (byte-identical old-record fixture) · SC3 (parse + producer malformed-id throw) · SC4 (producer threading) · SC11 (dedup/collision/published-only regress)	unit
scripts/brain/tests/assemble.test.ts	NEW. SC6 (high) · SC7 (none / below-floor / ellipsis-near-miss / different-date→low) · SC8 (pure+total)	unit
scripts/brain/tests/cli.test.ts	SC5 (repeatable flags) · SC9 (read-only assemble print + missing-dir degrade) · SC12 (single-value flags unregressed)	cli
scripts/test-runner.sh	bump BRAIN_TESTS_FLOOR 94→(94+N) + breakdown comment (:716); ASSERT_BASELINE_FLOOR UNCHANGED at 80	SC10
docs/second-brain/consolidation-loop.md · architecture.md · scripts/brain/src/id.ts	reconcile status to true state: S3c threads the cross-silo graph; id.ts:6-7 "SB-S3 will thread" → "S3c threads (post→specifics/trends + analytics resolver)"; architecture.md:80 SB-S3 row note	doc
CLAUDE.md · STATE	telling/counts: brain test count, S3c landed line; reference-doc count unchanged (briefs aren't counted reference docs)	doc

Not touched (scope fence): scripts/trends/** · scripts/specifics-bank/** · scripts/analytics/** (READ-only, via inlined JSON — no import, no schema/field change, no base-36-id reconciliation) · hooks/scripts/** (no .mjs; pathguard untouched) · agents/** (no reader wired this slice) · scripts/brain/src/{consolidate,types,profile,scaffold,dataRoot}.ts (no consolidation/grammar/scaffold change) · brain/profile.md grammar (C-1 — the 7th-token bump stays OUT) · parseFlags (untouched — collectRepeated is additive) · the plugin command surface (29 unchanged — brain assemble is a TS-CLI subcommand, not a /linkedin:* command).

Step 0 — pre-flight (verify baseline, no edits)

(cd scripts/brain && npm install) if node_modules absent. Confirm (cd scripts/brain && npm test) = 94/94 and bash scripts/test-runner.sh green (95/0/0, BRAIN_TESTS_FLOOR=94, ASSERT_BASELINE_FLOOR=80). Re-read ingest.ts:50-99 (serialize/parse round-trip), cli.ts:44-60,92-116,233-245 (parseFlags/runIngest/main). Confirm no structure-lint in test-runner.sh asserts the brain-CLI subcommand set (so assemble adds no lint debt) and that specifics-bank/src/bank.ts:52 normalizeContent is the idiom to copy locally (not import).

Step 1 — (RED) types + failing tests

1. Type scaffolding + compile-fixups (suite still compiles, 94 green):
   • PublishedRecord (ingest.ts:27-40): + specifics: string[]; + trends: string[]; (required).
   • Compile-fixup (plan-critic MINOR 4 — REQUIRED fields, so existing full-PublishedRecord literals must gain them or the suite won't compile): add specifics: [], trends: [] to the literals at tests/ingest.test.ts:11 (baseRec) and tests/publish.test.ts:78 (squatter). Tests that build records via ingestText/parsePublishedRecord need no change.
   • ingestText opts (:165-180): + specifics?: string[]; + trends?: string[];; record literal sets specifics: opts.specifics ?? [], trends: opts.trends ?? [].
   • parsePublishedRecord defaults the new fields to [] from the start (absent → []) — so existing round-trips (records carrying []) stay green at Step 1 (serialize omits empty, parse sets [], deep-equal holds). The new behaviour (emit non-empty, read present values, validate, producer-throw) is Step 2 RED→GREEN.
   • assemble.ts: stub assemblePostGraph returning [] + the types; normalize/matchRow declared.
2. Add failing tests → expected FAIL: SC1 (parse must read non-empty lines), SC2 (byte-identical fixture — fails until omit-empty serialize), SC3 (validate-throw), SC4 (producer threading with non-empty ids), SC6/SC7/SC8 (assembler logic), SC5/SC9/SC12 (CLI). Existing 94 (carrying []) PASS.

RED gate (single, unambiguous): run (cd scripts/brain && npm test) directly → new tests fail, existing 94 compile (with the two added literal fields) and pass (parse defaults []). Do NOT run bash scripts/test-runner.sh at RED (Section 16b fails the whole gate on a non-zero brain exit — design-noise, per S3b plan :38). The direct brain-suite RED is the failing-test proof; the gate runs GREEN-only after Step 4.

Step 2 — (GREEN) the post record carries raw material

1. serializePublishedRecord (ingest.ts:50-58): after the fixed 5-line header array, conditionally append — only when non-empty — specifics: <id,id> then trends: <id,id> (comma-joined, no spaces), before the SENTINEL. Empty array → line omitted → byte-identical 5-line header (SC2). Order fixed: …source: → specifics:(if any) → trends:(if any) → ---.
2. parsePublishedRecord (ingest.ts:75-99): a new non-throwing optional-list reader — NOT headerScalar (plan-critic MAJOR 1: headerScalar :61-65 THROWS missing "<key>:" header on an absent key, which would break every pre-S3c record / SC2 / SC11). Instead: const m = header.match(new RegExp(\^${key}:\s*(.?)\s$`, "m")); if (!m) return [](absent →[]); else split the captured value on ,, trim, drop empties; **validate each id /^[0-9a-f]{12}$/** (the :82hex guard) → a non-12-hex entry **throws**malformed published record: bad <specifics|trends> id <JSON.stringify(entry)>(never silently dropped). Read from the **header slice only** (pre-sentinel), so a body line that looks likespecifics:` cannot leak.
3. Producer-side validation (SC3 symmetric, plan-critic MAJOR 4 — specify the guard): in ingestText (:165-180), BEFORE mintContentId, validate each id of opts.specifics ?? [] and opts.trends ?? [] against /^[0-9a-f]{12}$/; a non-match throws ingest: bad <specifics|trends> id <JSON.stringify(entry)>. Empty/absent arrays skip validation (no throw) — so scanInbox (:204, calls ingestText with no specifics/trends) and every existing caller stay green (SC11). This adds a throw path to ingestText's previously throw-free contract — that is intended and bounded to malformed-id input.

GREEN gate (record): (cd scripts/brain && npm test) → SC1–SC4 + SC11 pass; existing 94 pass.

Step 3 — (GREEN) the assembler (the payoff)

In assemble.ts:

1. normalize(s) (local copy of the normalizeContent idiom — NOT imported): s.normalize? → s.trim().toLowerCase().replace(/\s+/g, " ").
2. Title prep: nt = stripTrailingEllipsis(normalize(title)) where stripTrailingEllipsis removes a trailing … or ... (+ surrounding ws) — this absorbs LinkedIn's truncation marker so a "…"-suffixed export title still prefix-matches the body. (Mid-word truncation without a marker already prefix-matches — body.startsWith(title) stays true — so only the marker needs stripping.)
3. matchRow(record, row) → { confidence: "high"|"low", row } | null:
   • nb = normalize(record.body); nt as above.
   • if (nt.length < PREFIX_FLOOR) return null — PREFIX_FLOOR = 24 (plan-critic MINOR 2 — rationale: the hook quality-rule floor is 110 chars and a LinkedIn export title is the content's opening run; 24 normalized chars (~3–5 words) is the shortest opener specific enough that a prefix-match is not coincidental, while staying well under any real hook. Tunable constant with this one-line justification; below floor → none).
   • if (!nb.startsWith(nt)) return null → none.
   • else: record.published_date === row.publishedDate ? "high" : "low" (prefix match + same date = high; prefix match + different date = low, surfaced for operator eyeball). (Note the field is publishedDate on the analytics side, published_date on the record — RISK B.)
4. assemblePostGraph({records, analytics}): for each record → { contentId: record.id, specifics, trends, match } where match = the best of analytics.map(r => matchRow(record, r)): prefer high over low; tie-break by longest matched nt (plan-critic MINOR 1 — to make the tie fully deterministic regardless of input/file order, the assembler sorts analytics once at entry by (publishedDate desc, title asc) before matching, so an exact length tie resolves stably, not by readdirSync order); none qualifying → match: { confidence: "none" }. Pure — no FS/clock/network; empty records→[]; empty analytics→every post none (SC8). The match carries the whole row reference (FIX 4).
5. loadAnalyticsRows(): AnalyticsRowInput[] (thin IO, read-only): const dir = dataRoot("analytics/posts"); if absent → [] (RISK C, fresh-clone). Read each *.json, JSON.parse, take .posts (an AnalyticsBatch), map each to the minimal { title, publishedDate, metrics } (extra fields ignored). A malformed/unreadable file is skipped in a try/catch (mirrors listPublished ingest.ts:230), never a crash. No import of the analytics package (§5 decoupling). Doc-comment caveat (plan-critic MAJOR 2 / R-B): this resolves the analytics root via the brain's dataRoot (${LINKEDIN_STUDIO_DATA}/analytics/posts); the analytics package additionally honours the deprecated ANALYTICS_ROOT env override (storage.ts:67-72) which the brain read path does NOT — so if ANALYTICS_ROOT is set to a non-default path, brain assemble reads the default root and degrades to every-post-none. This is the accepted cost of the §5 no-import decoupling (the M0 default leaves ANALYTICS_ROOT unset); name it in the loader doc-comment, do not silently skew.

GREEN gate (assembler): (cd scripts/brain && npm test) → SC6/SC7/SC8 pass.

Step 4 — (GREEN) CLI: repeatable producer flags + read-only assemble

1. collectRepeated(args, key): string[] in cli.ts (near parseFlags): scan the raw args for every --key <value> pair (value not starting with --) and collect all values. parseFlags is left untouched → single-value flags parse exactly as today (SC12 by construction).
2. runIngest signature + call-site change (plan-critic BLOCK 1 — this is a CHANGE, not an existing affordance): the current code is function runIngest(flags) (cli.ts:92) dispatched as runIngest(flags) (main, cli.ts:238) — rest is NOT passed today. Change BOTH: the signature to runIngest(rest: string[], flags) AND the main dispatch to runIngest(rest, flags). Then build specifics = collectRepeated(rest, "specific"), trends = collectRepeated(rest, "trend"), thread into ingestText({ …, specifics, trends }). runIngest reads specifics/trends ONLY via collectRepeated, never flags.specific/flags.trend (R-E). Empty → [] (today's behaviour). Update usage (:71): ingest --file <path> [--source <s>] [--date <YYYY-MM-DD>] [--specific <id>]… [--trend <id>]….
3. runAssemble(flags) + dispatch if (command === "assemble") return runAssemble(flags); (:240): load full PublishedRecord[] via the runConsolidate --gather record-load idiom (cli.ts:170-178 — parsePublishedRecord over each dataRoot('ingest/published')/*.md, NOT listPublished which omits .body and sorts ascending — plan-critic MAJOR 3) + analytics = loadAnalyticsRows(); run assemblePostGraph(...), then sort the result newest-first explicitly (published_date desc — do not rely on load order) and print: per post contentId · published_date · <firstLine> where firstLine = record.body.split("\n", 1)[0] (derived locally), then specifics: …, trends: …, analytics: <confidence> [eng <engagementRate>%] (Fork-4: engagementRate headline; whole row available for a future --verbose). Writes nothing (asserted SC9). Missing ingest/published/ or analytics/posts/ → clean empty/partial output, no crash (RISK C). Add assemble to usage.
4. CLI tests (cli.test.ts): SC5 (--specific a --specific c --trend b → record tagged [a,c]/[b] via round-trip read); SC12 (single-value --file/--source/--date + boolean --scan-inbox/--confirm parse as today, across subcommands); SC9 (assemble over a seeded LINKEDIN_STUDIO_DATA tmp dir prints the join + writes nothing; over an empty/missing dir prints clean + no crash).

GREEN gate: (cd scripts/brain && npm test) → all SCs pass.

Step 5 — gate floor + doc reconciliation

1. test-runner.sh:716: BRAIN_TESTS_FLOOR 94 → 94 + N (N = EXACT counted new brain tests from Steps 1–4 — set after counting, never guessed). Extend the breakdown comment … + SB-S3c N [ingest(a)+assemble(b)+cli(c)]. ASSERT_BASELINE_FLOOR stays 80 — no new unconditional structure-lint check (all new tests are brain-suite tests, counted by BRAIN_TESTS_FLOOR; same idiom as S3b §3.7 / SC10).
2. id.ts:6-7: "SB-S3 will thread this id through the tributaries" → "SB-S3c threads it (post record carries specifics/trends ids; analytics joined by resolver)".
3. consolidation-loop.md + architecture.md:80: mark the SB-S3 cross-silo graph as landed for the specific→post→analytics spine via the hub-side design; note the resolver's honest limit (RISK A): the analytics↔post join is a title-prefix+date heuristic with high/low/none tiers — a real-CSV none is a normalization-tightening signal, not a guarantee of no match (mirror the consolidation-loop.md honest-limit idiom).

Step 6 — Land

STATE "Telling" + "👉 NESTE" updated (S3c done → S3d remains, last); brief+plan committed as docs alongside code. GREEN gate proof: (cd scripts/brain && npm test) = (94+N)/(94+N) and bash scripts/test-runner.sh green. Commit (Conventional; code/feat-dominant → [skip-docs]). Push only inside the window (date '+%u %H:%M' first) and confirm before push (origin is PUBLIC open/). No version bump (additive within v0.5.2 dev). Optional documented end-to-end demo (RISK A) recorded in STATE/changelog.

Verification (testable)

SC	Check	Expected
SC1	record round-trip	parse(serialize(rec))===rec w/ non-empty specifics/trends; order preserved
SC2	byte backward-compat	empty arrays → unchanged 5-line header; serialize(parse(oldText))===oldText for fixture old record
SC3	id validation	non-12-hex specifics/trends id throws on parse AND at ingestText producer boundary
SC4	producer threading	ingestText({specifics:[a],trends:[b]}) → round-tripped [a]/[b]; absent → []
SC5	repeatable flags	--specific a --specific c --trend b → record [a,c]/[b]
SC6	assemble high	normalized body starts with row title (≥floor) + same date → high, whole row attached, specifics/trends surfaced
SC7	assemble low/none	no-prefix → none; <floor prefix → none; …-truncated title + same date → high (ellipsis stripped); prefix + different date → low
SC8	pure + total	records:[]→[]; analytics:[]→every post none; no throw
SC9	read-only CLI	brain assemble prints join newest-first + writes nothing; missing dirs → clean, no crash
SC10	gate green	bash scripts/test-runner.sh green; brain ≥ new BRAIN_TESTS_FLOOR; ASSERT_BASELINE_FLOOR unchanged at 80
SC11	no regression	ingest dedup/collision/published-only unchanged
SC12	single-value flags	--file/--source/--date/--scan-inbox/--confirm parse as today
(red proof)	failing-test-first	brain suite BETWEEN Step 1 and Step 2 → new SCs fail, 94 pass

Risks

• R-A (from RISK A) — synthetic-only proof. All SCs use fixtures; only the §Step-6 demo touches a real CSV. A green gate does not prove the resolver matches a real LinkedIn title against a real body. Mitigation: keep the demo; treat a demo none as a normalization-tightening signal (honest-limit doc line), not a slice failure. PREFIX_FLOOR/ellipsis-strip are the tunables.
• R-B (from RISK B) — analytics extraction-point. brain assemble must inline the raw-JSON read (dataRoot('analytics/posts')), not loadAllPosts, and read publishedDate (not date). Pinned in Step 3.5. A try/catch skips malformed batch files. Root-skew caveat (plan-critic MAJOR 2): the brain dataRoot ignores the analytics package's deprecated ANALYTICS_ROOT override (storage.ts:67-72); if set, the join silently degrades to every-post-none. Accepted cost of the §5 no-import decoupling (M0 default leaves it unset); named in the loadAnalyticsRows doc-comment, not silent.
• R-C (from RISK C) — fresh-clone. analytics/posts/ is gitignored/absent on a fresh clone → loadAnalyticsRows() returns [] → every post none (SC8), brain assemble degrades cleanly (SC9). Confirm no throw on missing dir.
• R-D — round-trip byte-break. New header lines MUST be omit-empty + appended after source:; an interleaved or always-emitted line breaks SC2 silently. Pinned by the SC2 fixture (byte-equality, not just deep-equal).
• R-E — collectRepeated vs parseFlags divergence. parseFlags still records the last --specific value (harmless — runIngest ignores flags.specific/flags.trend and uses collectRepeated). Pin: runIngest reads specifics/trends ONLY via collectRepeated, never flags.
• R-F — false-positive match. A generic short opener could prefix-match the wrong post; PREFIX_FLOOR=24 + same-date→high (else low) guards it; low matches are surfaced (not hidden) so the operator eyeballs. No silent high on a weak match.
• R-G — floor-count drift (plan-critic MINOR 3). BRAIN_TESTS_FLOOR bumped by the EXACT counted N; the breakdown comment's per-file sub-counts [ingest(a)+assemble(b)+cli(c)] MUST sum to N (reconcile after counting). ASSERT_BASELINE_FLOOR deliberately unchanged at 80. Verified by the green gate; nothing lints the comment (discipline-only, as S3a/S3b).
• R-H — unguarded doc. No lint protects docs/; reconciled by discipline (as S3a/S3b).