linkedin-studio/docs/second-brain/plan-sb-s3c.md

# Plan — SB-S3c: cross-silo id-threading

> Brief: `docs/second-brain/brief-sb-s3c.md`. Slice: SB-S3c.
> **Status:** **LANDED** (operator go 2026-06-23) — brain 113/113, gate 95/0/0, `BRAIN_TESTS_FLOOR` 94→113. **Design:** hub-side threading — `specifics`/`trends` ids onto the brain's published record + a pure analytics resolver. Tributaries untouched.
> **Light-Voyage:** brief-review APPROVE-WITH-FIXES (5 FIX folded into the brief; 3 RISK carried here). plan-critic **REVISE → 1 BLOCK + 4 MAJOR + 4 MINOR folded** (Step 1 compile-fixup · Step 2.2 non-throwing parse reader · Step 2.3 producer guard · Step 3.3 floor rationale · Step 3.4 deterministic row-sort · Step 3.5 analytics-root caveat · Step 4.2 `runIngest` signature+call-site · Step 4.3 full-record load + desc sort · R-G sub-count). scope-guardian **ALIGNED** (0 creep / 0 gap).
> Order is TDD: failing brain-suite tests land BEFORE the code (iron law); each rule pinned before the CLI is wired.

## Goal

A published post records the **raw material it was built from** (`specifics`/`trends` ids, additive + backward-compatible on the post record), and a **pure assembler** joins published-record ↔ analytics-row (by normalized title-prefix + date, with honest confidence tiers) so the north-star query — *which raw material actually performs?* (`specific → post → measured analytics`, `architecture.md:17`) — is **assemblable**. All four tributary schemas stay untouched; the only new disk surface is a **read-only** `brain assemble`. Fully unit/CLI-testable with fixtures.

## Files touched (exhaustive — for scope-guardian)

| File | Change | SC |
|------|--------|-----|
| `scripts/brain/src/ingest.ts` | `PublishedRecord` (+`specifics: string[]` +`trends: string[]`, `:27-40`) · `serializePublishedRecord` append the two lines **after `source:`**, **omit-empty** (`:50-58`) · `parsePublishedRecord` read them via a new list-scalar reader, **12-hex-validate each** (reuse the `:82` guard idiom), absent→`[]` (`:75-99`) · `ingestText` (+optional `specifics?`/`trends?`, `:165-180`) | SC1,SC2,SC3,SC4,SC11 |
| `scripts/brain/src/assemble.ts` | **NEW.** Pure `assemblePostGraph({records, analytics})` + local `normalize()` + `matchRow()` (tiers) + minimal input type `AnalyticsRowInput` + output types · thin read-only `loadAnalyticsRows()` IO (inline raw-JSON read of `dataRoot('analytics/posts')/*.json` → `AnalyticsBatch.posts[]`; **never** imports the analytics package) | SC6,SC7,SC8 |
| `scripts/brain/src/cli.ts` | `collectRepeated(args,key): string[]` helper (leaves `parseFlags` **untouched** — SC12 by construction) · **change `runIngest` signature `(flags)`→`(rest, flags)` AND the `main` dispatch `runIngest(flags)`→`runIngest(rest, flags)`** (`:92`, `:238` — BLOCK 1), thread `collectRepeated(rest,"specific")`/`"trend"` · `runAssemble(flags)` (read-only print, full-record load) · dispatch `+ if (command==="assemble")` (`:240`) · `usage` text (+`assemble`, +ingest flags) | SC5,SC9,SC12 |
| `scripts/brain/tests/ingest.test.ts` | SC1 (round-trip w/ ids) · SC2 (byte-identical old-record fixture) · SC3 (parse + producer malformed-id throw) · SC4 (producer threading) · SC11 (dedup/collision/published-only regress) | unit |
| `scripts/brain/tests/assemble.test.ts` | **NEW.** SC6 (high) · SC7 (none / below-floor / ellipsis-near-miss / different-date→low) · SC8 (pure+total) | unit |
| `scripts/brain/tests/cli.test.ts` | SC5 (repeatable flags) · SC9 (read-only assemble print + missing-dir degrade) · SC12 (single-value flags unregressed) | cli |
| `scripts/test-runner.sh` | bump `BRAIN_TESTS_FLOOR` 94→(94+N) + breakdown comment (`:716`); **`ASSERT_BASELINE_FLOOR` UNCHANGED at 80** | SC10 |
| `docs/second-brain/consolidation-loop.md` · `architecture.md` · `scripts/brain/src/id.ts` | reconcile status to true state: S3c threads the cross-silo graph; `id.ts:6-7` "SB-S3 will thread" → "S3c threads (post→specifics/trends + analytics resolver)"; `architecture.md:80` SB-S3 row note | doc |
| `CLAUDE.md` · STATE | telling/counts: brain test count, S3c landed line; reference-doc count unchanged (briefs aren't counted reference docs) | doc |

**Not touched (scope fence):** `scripts/trends/**` · `scripts/specifics-bank/**` · `scripts/analytics/**` (READ-only, via inlined JSON — no import, no schema/field change, no base-36-id reconciliation) · `hooks/scripts/**` (no `.mjs`; pathguard untouched) · `agents/**` (no reader wired this slice) · `scripts/brain/src/{consolidate,types,profile,scaffold,dataRoot}.ts` (no consolidation/grammar/scaffold change) · `brain/profile.md` grammar (C-1 — the 7th-token bump stays OUT) · `parseFlags` (untouched — `collectRepeated` is additive) · the plugin command surface (29 unchanged — `brain assemble` is a TS-CLI subcommand, not a `/linkedin:*` command).

## Step 0 — pre-flight (verify baseline, no edits)

`(cd scripts/brain && npm install)` if `node_modules` absent. Confirm `(cd scripts/brain && npm test)` = **94/94** and `bash scripts/test-runner.sh` green (95/0/0, `BRAIN_TESTS_FLOOR=94`, `ASSERT_BASELINE_FLOOR=80`). Re-read `ingest.ts:50-99` (serialize/parse round-trip), `cli.ts:44-60,92-116,233-245` (parseFlags/runIngest/main). Confirm **no structure-lint** in `test-runner.sh` asserts the brain-CLI subcommand set (so `assemble` adds no lint debt) and that `specifics-bank/src/bank.ts:52` `normalizeContent` is the idiom to **copy locally** (not import).

## Step 1 — (RED) types + failing tests

1. **Type scaffolding + compile-fixups** (suite still compiles, 94 green):
   - `PublishedRecord` (`ingest.ts:27-40`): `+ specifics: string[]; + trends: string[];` (required).
   - **Compile-fixup (plan-critic MINOR 4 — REQUIRED fields, so existing full-`PublishedRecord` literals must gain them or the suite won't compile):** add `specifics: [], trends: []` to the literals at `tests/ingest.test.ts:11` (`baseRec`) and `tests/publish.test.ts:78` (`squatter`). Tests that build records via `ingestText`/`parsePublishedRecord` need no change.
   - `ingestText` opts (`:165-180`): `+ specifics?: string[]; + trends?: string[];`; record literal sets `specifics: opts.specifics ?? []`, `trends: opts.trends ?? []`.
   - **`parsePublishedRecord` defaults the new fields to `[]` from the start** (absent → `[]`) — so existing round-trips (records carrying `[]`) stay **green** at Step 1 (serialize omits empty, parse sets `[]`, deep-equal holds). The *new behaviour* (emit non-empty, read present values, validate, producer-throw) is Step 2 RED→GREEN.
   - `assemble.ts`: stub `assemblePostGraph` returning `[]` + the types; `normalize`/`matchRow` declared.
2. **Add failing tests** → expected FAIL: SC1 (parse must read **non-empty** lines), SC2 (byte-identical fixture — fails until omit-empty serialize), SC3 (validate-throw), SC4 (producer threading with non-empty ids), SC6/SC7/SC8 (assembler logic), SC5/SC9/SC12 (CLI). Existing 94 (carrying `[]`) PASS.

**RED gate (single, unambiguous):** run **`(cd scripts/brain && npm test)` directly** → new tests fail, existing 94 **compile** (with the two added literal fields) and **pass** (parse defaults `[]`). **Do NOT run `bash scripts/test-runner.sh` at RED** (Section 16b fails the whole gate on a non-zero brain exit — design-noise, per S3b plan `:38`). The direct brain-suite RED is the failing-test proof; the gate runs GREEN-only after Step 4.

## Step 2 — (GREEN) the post record carries raw material

1. **`serializePublishedRecord`** (`ingest.ts:50-58`): after the fixed 5-line header array, conditionally append — **only when non-empty** — `specifics: <id,id>` then `trends: <id,id>` (comma-joined, no spaces), **before** the `SENTINEL`. Empty array → line omitted → **byte-identical 5-line header** (SC2). Order fixed: `…source:` → `specifics:`(if any) → `trends:`(if any) → `---`.
2. **`parsePublishedRecord`** (`ingest.ts:75-99`): a **new non-throwing optional-list reader** — **NOT `headerScalar`** (plan-critic MAJOR 1: `headerScalar` `:61-65` THROWS `missing "<key>:" header` on an absent key, which would break **every** pre-S3c record / SC2 / SC11). Instead: `const m = header.match(new RegExp(\`^${key}:\\s*(.*?)\\s*$\`, "m")); if (!m) return []` (absent → `[]`); else split the captured value on `,`, trim, drop empties; **validate each id `/^[0-9a-f]{12}$/`** (the `:82` hex guard) → a non-12-hex entry **throws** `malformed published record: bad <specifics|trends> id <JSON.stringify(entry)>` (never silently dropped). Read from the **header slice only** (pre-sentinel), so a body line that looks like `specifics:` cannot leak.
3. **Producer-side validation (SC3 symmetric, plan-critic MAJOR 4 — specify the guard):** in `ingestText` (`:165-180`), BEFORE `mintContentId`, validate each id of `opts.specifics ?? []` and `opts.trends ?? []` against `/^[0-9a-f]{12}$/`; a non-match throws `ingest: bad <specifics|trends> id <JSON.stringify(entry)>`. **Empty/absent arrays skip validation** (no throw) — so `scanInbox` (`:204`, calls `ingestText` with no specifics/trends) and every existing caller stay green (SC11). This adds a throw path to `ingestText`'s previously throw-free contract — that is intended and bounded to malformed-id input.

**GREEN gate (record):** `(cd scripts/brain && npm test)` → SC1–SC4 + SC11 pass; existing 94 pass.

## Step 3 — (GREEN) the assembler (the payoff)

In `assemble.ts`:

1. **`normalize(s)`** (local copy of the `normalizeContent` idiom — NOT imported): `s.normalize?` → `s.trim().toLowerCase().replace(/\s+/g, " ")`.
2. **Title prep:** `nt = stripTrailingEllipsis(normalize(title))` where `stripTrailingEllipsis` removes a trailing `…` or `...` (+ surrounding ws) — this absorbs LinkedIn's truncation marker so a `"…"`-suffixed export title still prefix-matches the body. (Mid-word truncation **without** a marker already prefix-matches — `body.startsWith(title)` stays true — so only the marker needs stripping.)
3. **`matchRow(record, row)` → `{ confidence: "high"|"low", row } | null`:**
   - `nb = normalize(record.body)`; `nt` as above.
   - `if (nt.length < PREFIX_FLOOR) return null` — **`PREFIX_FLOOR = 24`** (plan-critic MINOR 2 — rationale: the hook quality-rule floor is 110 chars and a LinkedIn export title is the content's opening run; 24 normalized chars (~3–5 words) is the shortest opener specific enough that a prefix-match is not coincidental, while staying well under any real hook. Tunable constant with this one-line justification; below floor → `none`).
   - `if (!nb.startsWith(nt)) return null` → `none`.
   - else: `record.published_date === row.publishedDate ? "high" : "low"` (prefix match + same date = high; prefix match + different date = low, surfaced for operator eyeball). *(Note the field is `publishedDate` on the analytics side, `published_date` on the record — RISK B.)*
4. **`assemblePostGraph({records, analytics})`:** for each record → `{ contentId: record.id, specifics, trends, match }` where `match` = the **best** of `analytics.map(r => matchRow(record, r))`: prefer `high` over `low`; tie-break by **longest matched `nt`** (plan-critic MINOR 1 — to make the tie fully deterministic regardless of input/file order, the assembler **sorts `analytics` once at entry** by `(publishedDate desc, title asc)` before matching, so an exact length tie resolves stably, not by `readdirSync` order); none qualifying → `match: { confidence: "none" }`. Pure — no FS/clock/network; empty `records`→`[]`; empty `analytics`→every post `none` (SC8). The match carries the **whole row reference** (FIX 4).
5. **`loadAnalyticsRows(): AnalyticsRowInput[]`** (thin IO, read-only): `const dir = dataRoot("analytics/posts")`; if absent → `[]` (RISK C, fresh-clone). Read each `*.json`, `JSON.parse`, take `.posts` (an `AnalyticsBatch`), map each to the minimal `{ title, publishedDate, metrics }` (extra fields ignored). A malformed/unreadable file is skipped in a try/catch (mirrors `listPublished` `ingest.ts:230`), never a crash. **No import of the analytics package** (§5 decoupling). **Doc-comment caveat (plan-critic MAJOR 2 / R-B):** this resolves the analytics root via the brain's `dataRoot` (`${LINKEDIN_STUDIO_DATA}/analytics/posts`); the analytics package additionally honours the **deprecated `ANALYTICS_ROOT`** env override (`storage.ts:67-72`) which the brain read path does NOT — so if `ANALYTICS_ROOT` is set to a non-default path, `brain assemble` reads the default root and degrades to every-post-`none`. This is the accepted cost of the §5 no-import decoupling (the M0 default leaves `ANALYTICS_ROOT` unset); name it in the loader doc-comment, do not silently skew.

**GREEN gate (assembler):** `(cd scripts/brain && npm test)` → SC6/SC7/SC8 pass.

## Step 4 — (GREEN) CLI: repeatable producer flags + read-only assemble

1. **`collectRepeated(args, key): string[]`** in `cli.ts` (near `parseFlags`): scan the raw `args` for every `--key <value>` pair (value not starting with `--`) and collect all values. **`parseFlags` is left untouched** → single-value flags parse exactly as today (SC12 by construction).
2. **`runIngest` signature + call-site change (plan-critic BLOCK 1 — this is a CHANGE, not an existing affordance):** the current code is `function runIngest(flags)` (`cli.ts:92`) dispatched as `runIngest(flags)` (`main`, `cli.ts:238`) — `rest` is NOT passed today. Change BOTH: the signature to `runIngest(rest: string[], flags)` AND the `main` dispatch to `runIngest(rest, flags)`. Then build `specifics = collectRepeated(rest, "specific")`, `trends = collectRepeated(rest, "trend")`, thread into `ingestText({ …, specifics, trends })`. **`runIngest` reads specifics/trends ONLY via `collectRepeated`, never `flags.specific`/`flags.trend`** (R-E). Empty → `[]` (today's behaviour). Update `usage` (`:71`): `ingest --file <path> [--source <s>] [--date <YYYY-MM-DD>] [--specific <id>]… [--trend <id>]…`.
3. **`runAssemble(flags)`** + dispatch `if (command === "assemble") return runAssemble(flags);` (`:240`): load **full `PublishedRecord[]`** via the `runConsolidate --gather` record-load idiom (`cli.ts:170-178` — `parsePublishedRecord` over each `dataRoot('ingest/published')/*.md`, NOT `listPublished` which omits `.body` and sorts ascending — plan-critic MAJOR 3) + `analytics = loadAnalyticsRows()`; run `assemblePostGraph(...)`, then **sort the result newest-first explicitly** (`published_date` desc — do not rely on load order) and print: per post `contentId · published_date · <firstLine>` where `firstLine = record.body.split("\n", 1)[0]` (derived locally), then `specifics: …`, `trends: …`, `analytics: <confidence> [eng <engagementRate>%]` (Fork-4: `engagementRate` headline; whole row available for a future `--verbose`). **Writes nothing** (asserted SC9). Missing `ingest/published/` or `analytics/posts/` → clean empty/partial output, no crash (RISK C). Add `assemble` to `usage`.
4. **CLI tests** (`cli.test.ts`): SC5 (`--specific a --specific c --trend b` → record tagged `[a,c]`/`[b]` via round-trip read); SC12 (single-value `--file`/`--source`/`--date` + boolean `--scan-inbox`/`--confirm` parse as today, across subcommands); SC9 (`assemble` over a seeded `LINKEDIN_STUDIO_DATA` tmp dir prints the join + writes nothing; over an empty/missing dir prints clean + no crash).

**GREEN gate:** `(cd scripts/brain && npm test)` → all SCs pass.

## Step 5 — gate floor + doc reconciliation

1. `test-runner.sh:716`: `BRAIN_TESTS_FLOOR` 94 → **94 + N** (N = EXACT counted new brain tests from Steps 1–4 — set after counting, never guessed). Extend the breakdown comment `… + SB-S3c N [ingest(a)+assemble(b)+cli(c)]`. **`ASSERT_BASELINE_FLOOR` stays 80** — no new *unconditional structure-lint* check (all new tests are brain-suite tests, counted by `BRAIN_TESTS_FLOOR`; same idiom as S3b §3.7 / SC10).
2. `id.ts:6-7`: "SB-S3 will thread this id through the tributaries" → "SB-S3c threads it (post record carries `specifics`/`trends` ids; analytics joined by resolver)".
3. `consolidation-loop.md` + `architecture.md:80`: mark the SB-S3 cross-silo graph as landed for the `specific→post→analytics` spine via the hub-side design; note the resolver's **honest limit** (RISK A): the analytics↔post join is a title-prefix+date heuristic with `high/low/none` tiers — a real-CSV `none` is a normalization-tightening signal, not a guarantee of no match (mirror the `consolidation-loop.md` honest-limit idiom).

## Step 6 — Land

STATE "Telling" + "👉 NESTE" updated (S3c done → S3d remains, last); brief+plan committed as docs alongside code. **GREEN gate proof:** `(cd scripts/brain && npm test)` = (94+N)/(94+N) and `bash scripts/test-runner.sh` green. Commit (Conventional; code/feat-dominant → `[skip-docs]`). **Push only inside the window** (`date '+%u %H:%M'` first) and **confirm before push** (`origin` is PUBLIC `open/`). No version bump (additive within v0.5.2 dev). Optional documented end-to-end demo (RISK A) recorded in STATE/changelog.

## Verification (testable)

| SC | Check | Expected |
|----|-------|----------|
| SC1 | record round-trip | `parse(serialize(rec))===rec` w/ non-empty specifics/trends; order preserved |
| SC2 | byte backward-compat | empty arrays → unchanged 5-line header; `serialize(parse(oldText))===oldText` for fixture old record |
| SC3 | id validation | non-12-hex specifics/trends id throws on parse AND at `ingestText` producer boundary |
| SC4 | producer threading | `ingestText({specifics:[a],trends:[b]})` → round-tripped `[a]`/`[b]`; absent → `[]` |
| SC5 | repeatable flags | `--specific a --specific c --trend b` → record `[a,c]`/`[b]` |
| SC6 | assemble high | normalized body starts with row title (≥floor) + same date → `high`, whole row attached, specifics/trends surfaced |
| SC7 | assemble low/none | no-prefix → none; `<floor` prefix → none; `…`-truncated title + same date → high (ellipsis stripped); prefix + different date → low |
| SC8 | pure + total | `records:[]`→`[]`; `analytics:[]`→every post `none`; no throw |
| SC9 | read-only CLI | `brain assemble` prints join newest-first + writes nothing; missing dirs → clean, no crash |
| SC10 | gate green | `bash scripts/test-runner.sh` green; brain ≥ new `BRAIN_TESTS_FLOOR`; `ASSERT_BASELINE_FLOOR` unchanged at 80 |
| SC11 | no regression | ingest dedup/collision/published-only unchanged |
| SC12 | single-value flags | `--file`/`--source`/`--date`/`--scan-inbox`/`--confirm` parse as today |
| (red proof) | failing-test-first | brain suite BETWEEN Step 1 and Step 2 → new SCs fail, 94 pass |

## Risks

- **R-A (from RISK A) — synthetic-only proof.** All SCs use fixtures; only the §Step-6 demo touches a real CSV. A green gate does **not** prove the resolver matches a real LinkedIn title against a real body. Mitigation: keep the demo; treat a demo `none` as a normalization-tightening signal (honest-limit doc line), not a slice failure. `PREFIX_FLOOR`/ellipsis-strip are the tunables.
- **R-B (from RISK B) — analytics extraction-point.** `brain assemble` must inline the raw-JSON read (`dataRoot('analytics/posts')`), **not** `loadAllPosts`, and read **`publishedDate`** (not `date`). Pinned in Step 3.5. A try/catch skips malformed batch files. **Root-skew caveat (plan-critic MAJOR 2):** the brain `dataRoot` ignores the analytics package's deprecated `ANALYTICS_ROOT` override (`storage.ts:67-72`); if set, the join silently degrades to every-post-`none`. Accepted cost of the §5 no-import decoupling (M0 default leaves it unset); named in the `loadAnalyticsRows` doc-comment, not silent.
- **R-C (from RISK C) — fresh-clone.** `analytics/posts/` is gitignored/absent on a fresh clone → `loadAnalyticsRows()` returns `[]` → every post `none` (SC8), `brain assemble` degrades cleanly (SC9). Confirm no throw on missing dir.
- **R-D — round-trip byte-break.** New header lines MUST be omit-empty + appended after `source:`; an interleaved or always-emitted line breaks SC2 silently. Pinned by the SC2 fixture (byte-equality, not just deep-equal).
- **R-E — `collectRepeated` vs `parseFlags` divergence.** `parseFlags` still records the *last* `--specific` value (harmless — `runIngest` ignores `flags.specific`/`flags.trend` and uses `collectRepeated`). Pin: `runIngest` reads specifics/trends ONLY via `collectRepeated`, never `flags`.
- **R-F — false-positive match.** A generic short opener could prefix-match the wrong post; `PREFIX_FLOOR=24` + same-date→high (else low) guards it; low matches are surfaced (not hidden) so the operator eyeballs. No silent high on a weak match.
- **R-G — floor-count drift (plan-critic MINOR 3).** `BRAIN_TESTS_FLOOR` bumped by the EXACT counted N; the breakdown comment's per-file sub-counts `[ingest(a)+assemble(b)+cli(c)]` MUST sum to N (reconcile after counting). `ASSERT_BASELINE_FLOOR` deliberately unchanged at 80. Verified by the green gate; nothing lints the comment (discipline-only, as S3a/S3b).
- **R-H — unguarded doc.** No lint protects `docs/`; reconciled by discipline (as S3a/S3b).