linkedin-studio/docs/second-brain/brief-sb-s3c.md

# Brief — SB-S3c: cross-silo id-threading (the "which raw material performs?" graph)

> **Slice:** SB-S3c (third sub-slice of SB-S3). **Status:** **LANDED** — operator go 2026-06-23; TDD complete, brain suite 113/113, gate 95/0/0 (`BRAIN_TESTS_FLOOR` 94→113, `ASSERT_BASELINE_FLOOR` unchanged at 80). Light-Voyage: brief-review **APPROVE-WITH-FIXES** (5 FIX folded §9), plan-critic **REVISE → 1 BLOCK + 4 MAJOR + 4 MINOR folded** (§10), scope-guardian **ALIGNED** (0 creep/0 gap).
> **Predecessors:** SB-S0 (id/provenance substrate — `mintEntityId`/`mintContentId`, `id.ts:6-7` reserved "SB-S3 will thread this id through the tributaries") · SB-S1 (published-gold ingest — the post anchor `ingest/published/<contentId>.md`) · SB-S2 (consolidation motor) · SB-S3a (first reader `strategy-advisor` — LANDED `4fa411f`) · **SB-S3b (supersede arm — LANDED `585f972`, gate 95/0/0, BRAIN floor 94, ASSERT floor 80).**
> **Grounded in:** a read-only code map of all four tributaries + `scripts/brain/` (file:line cited throughout) — Explore sweep + firsthand reads of `id.ts`, `consolidate.ts`, `types.ts`, `profile.ts`, `ingest.ts`.

## 1. Operator decision (2026-06-23)

SB-S3 was decomposed into four sub-slices; standing operator direction is **S3a (reader) → S3b (supersede) → S3c (id-threading) → S3d (hygiene+ops, last)**. S3a + S3b have landed. **S3c is next** — it is the slice the whole arc was built toward: the architecture's single hardest problem (`architecture.md:17`) — *"A published post lands in three non-referencing places with no shared id. The question 'which raw material actually performs?' (specific → post → measured analytics) is **unanswerable today**"* — and the explicit SB-S3 payoff (`architecture.md:54`, `:80`: *"thread the id through tributaries → post ↔ specific ↔ trend ↔ analytics assemblable"*).

The go-before-code gate still applies per slice: this brief + the plan are presented for operator "go" (with the genuine forks in §8) before any code.

## 2. The north-star query, and the central design decision

**What S3c must make answerable:** given a published post, **what raw material fed it** (the specific(s) that grounded it, the trend(s) that inspired it) and **how did it perform** (its measured analytics). The killer second-brain query is `specific → post → measured analytics`: *which raw material actually performs?*

**The grounding surfaced two hard constraints that reshape the slice** (this is why the design below is leaner — and more arc-aligned — than the STATE's anticipated "mutate every tributary schema"):

- **C-1 — the profile grammar is rigid.** `brain/profile.md` is a fixed **6-token line-grammar** (`profile.ts:27-37,57-58`; `FACT_RE`; `parse∘serialize===doc`, SC2). Persisting a fact→post provenance link **onto the fact** would add a 7th token = a **breaking `SCHEMA_VERSION` bump** (`types.ts:26`, "bumped only on a breaking grammar change") + a migration of every existing `profile.md`. That is its own slice, not S3c. **OUT (§4).**
- **C-2 — analytics has no body and no URN.** The LinkedIn CSV export carries only `title` (the post-content column, taken **verbatim** by the parser — `csv-parser.ts:181`; the "first ~100 chars" in `models/types.ts:3` is a property of *LinkedIn's export*, not a code truncation) + `publishedDate` + metrics — **no full body, no permalink/URN anywhere in the codebase**. So analytics **physically cannot compute** the brain's `mintContentId(body)` (`id.ts:51`, body-byte-exact). Nor does its own `id` help: `PostAnalytics.id = hash(title + date)` (`csv-parser.ts:138`) is derived from the **very fields the resolver already joins on**, so it carries **zero independent linking signal**. The analytics↔post link therefore **cannot be a stored shared id** — it must be a **resolver** (join by title-prefix + date). This is an honest heuristic, not a guarantee, and the design names it as such.

**Central design decision — thread the cross-ids onto the HUB-side post record, not into each tributary; assemble analytics by resolver.** The architecture is explicit that **tributaries stay separate and the hub holds the distilled summary + pointers** (`architecture.md:43,49`: *"Keep tributaries; add a thin hub + a cross-silo id"*; *"Forcing one schema would destroy [each silo's] distinctions"*). The temporally-correct anchor is the **post**: trends + specifics exist *before* a post and feed it; analytics measures it *after*. The post's canonical object already exists — `ingest/published/<contentId>.md` — and it already carries the content-id (filename stem + `id:` header). Its **parse** side is tolerant of added header lines (`headerScalar` reads each key by regex, `ingest.ts:61-65`), so threading new fields is round-trip-safe; the **serializer** is a fixed 5-line array (`ingest.ts:51-57`) that S3c **edits** to conditionally append the two new lines (this is a deliberate edit, not an auto-handled field — corrected per brief-review FIX 5). Unlike the rigid profile grammar, this header grammar takes additive optional lines cleanly. So:

1. the **published record** carries the post's raw-material provenance as **additive, optional header lines** (`specifics:` + `trends:`, lists of tributary ids); and
2. a **pure assembler** joins published-record ↔ analytics-row by title-prefix + date (with a confidence tier) to attach measured performance.

This keeps **all four tributary schemas (trends, specifics, analytics, post-tracking) untouched**, needs **no analytics base-36-id reconciliation**, and is strictly additive + backward-compatible — while delivering the exact graph the north-star query needs. *(That S3c's blast radius is bounded to the `scripts/brain/` package, not the tributaries, is a finding that corrects the STATE's "rører tributær-skjemaer" expectation — surfaced as the headline fork, §8 Fork-1.)*

## 3. Scope — what is IN (S3c)

1. **Published record carries raw-material provenance (additive, optional).** Extend `PublishedRecord` (`ingest.ts:27-40`) with `specifics: string[]` + `trends: string[]` (tributary ids the post was built from; default empty). `serializePublishedRecord` (`ingest.ts:50-58`) emits them as header lines **only when non-empty** (omit-empty → old records serialize byte-identically); `parsePublishedRecord` (`ingest.ts:75-99`) reads them (absent → `[]`). Ids are validated to the 12-hex shape (the existing `id` guard idiom, `ingest.ts:82`); a malformed id throws (never silently dropped). The `\n---\n` sentinel split is unaffected (header lines never contain it).

2. **Producer surface — tag raw material at ingest.** `ingestText` (`ingest.ts:165-180`) accepts optional `specifics?: string[]` / `trends?: string[]`, threaded onto the record. The `brain ingest` CLI gains **repeatable** `--specific <id>` / `--trend <id>` flags so the operator can tag a post's raw material at capture time. **This requires a parser change (FIX 1):** the shared `parseFlags` (`cli.ts:44-60`) returns `Record<string,string>` and **overwrites** on a repeated key (`out[key] = next`), so it cannot collect `[a,c]`. S3c extends flag-collection to gather repeated `--specific`/`--trend` into arrays — **without changing single-value flag behaviour** (`--file`/`--source`/`--date`/`--scan-inbox` parse exactly as today; regression-pinned, SC12). Minimal-disruption shape (a dedicated repeatable-flag pass for the two keys, or a `string|string[]` collect) is pinned in the plan. This makes the substrate **immediately usable end-to-end** (not fixture-only): ingest a post tagged `--specific X` → assemble → see the graph. Backward-compatible (flags absent → empty arrays = today's behaviour).

3. **The assembler (the payoff) — a PURE function.** `assemblePostGraph({ records, analytics })` over already-loaded inputs (no I/O in the core) returns, per published post: its `contentId`, its `specifics[]`/`trends[]` ids, and its **matched analytics** — `{ row, confidence }` carrying the **whole `PostAnalytics` row reference** (FIX 4: the assembler attaches the full row; which metric the *CLI renders* is Fork-4, decoupling the data shape from presentation), or `none`.
   **The join rule (named here, not deferred — FIX 3), absorbing the title↔body asymmetry (verbatim CSV title vs byte-exact body, C-2):** normalize **both** sides with the brain's content normalization (`normalizeContent` idiom: lowercase + collapse-whitespace + trim — the plan pins the exact fn / shared helper) and require **`norm(title)` is a prefix of `norm(body)`** with a **minimum prefix-length floor** (a too-short title must not false-match any body — floor value pinned in the plan, e.g. ≥ N chars). **Confidence tiers:** `high` = prefix-match (≥ floor) + same `published_date`; `low` = prefix-match (≥ floor) but date off by ≤ a small tolerance, OR a near-miss where LinkedIn truncated the title mid-word (the body's normalized text starts with `norm(title)` minus a trailing partial token); `none` = no qualifying prefix match. Deterministic, fully unit-testable (incl. a truncation near-miss case, SC7). The assembler reads tributary ids **as opaque 12-hex strings** and takes a **minimal analytics input shape** (`{ title, publishedDate, metrics, … }` — the raw `PostAnalytics` JSON shape, `models/types.ts:1-8`), so it does **not** import the trends/specifics/analytics packages (decoupled, §5).

4. **A thin read-only IO + CLI surface.** A `brain assemble` (or `brain graph`) subcommand: loads published records (`dataRoot('ingest/published')`) + the analytics batches by **inlining a raw-JSON read** of `dataRoot('analytics/posts')/*.json` (each file is an `AnalyticsBatch` with `.posts[]`; storage path `<root>/posts/<date>-<shortId>.json`, `storage.ts:146-152`) — **NOT** by importing the analytics package's `loadAllPosts` (`storage.ts:197`), per the §5 decoupling. It extracts the minimal shape from each `posts[]` entry (note the field is **`publishedDate`**, not `date` — RISK B), runs `assemblePostGraph`, and prints the post → raw-material → performance join (newest first). **Read-only** — it never writes the brain. A missing `analytics/posts/` (gitignored — absent on a fresh clone, RISK C) or missing `ingest/published/` degrades to an empty/partial graph (every post `match: none`), never a throw. *(Fork-3, §8: pure-function-only with CLI deferred is the lighter alternative.)*

5. **Gate + doc reconciliation.** Bump `BRAIN_TESTS_FLOOR` (currently **94**, `test-runner.sh:716`) by the new brain-test count; **`ASSERT_BASELINE_FLOOR` stays 80** — no new *unconditional structure-lint* check (the new tests are brain-suite tests, counted by `BRAIN_TESTS_FLOOR`, same idiom as S3b §3.7). Reconcile the now-true status lines (`consolidation-loop.md`, `architecture.md:80` SB-S3 row, CLAUDE.md/STATE counts, the `id.ts:6-7` "SB-S3 will thread" comment).

## 4. Non-goals — what is OUT (deferred)

- **Persisting the fact→post link on `brain/profile.md` (the profile-grammar bump, C-1).** The post→specifics/trends graph + the analytics resolver answer the north-star query **without** touching the rigid 6-token grammar. A 7th `source`/provenance-id token is a breaking `SCHEMA_VERSION` change with a migration — its own slice. The fact already carries `provenance`; that is enough for S3c.
- **Mutating the tributary schemas (trends/specifics/analytics/post-tracking).** Per `architecture.md:43,49` tributaries stay separate; the cross-ids live hub-side on the post record (§2). No `TrendRecord`/`Specific`/`PostAnalytics` field is added; no analytics base-36-id → sha256 reconciliation. *(Fork-1 is exactly this choice — operator may redirect to the broader mutate-tributaries shape.)*
- **post-tracking (`state-updater.mjs`) content-id.** The loosest silo (regex-mutated markdown, prune-regex risk, `state-updater.mjs:116,145`) and **not on the `specific→post→analytics` critical path**. The post-tracking ↔ published ↔ analytics "triple-post reconciliation" is the named **S3d** concern (`architecture.md:80`). OUT here.
- **A producer that auto-records which specifics/trends a draft used.** S3c gives the substrate + the **manual** `--specific/--trend` tag at ingest (§3.2). Auto-capture from a drafting command (e.g. `/linkedin:newsletter` recording its `livedSpecifics`) is follow-on wiring, like S3a wired one reader.
- **A fuzzy/semantic analytics match beyond title-prefix + date.** No embedding/edit-distance match. If title-prefix+date doesn't resolve it, it's `none` (honest), not a guess. Tightening the matcher is later, additive.
- **Backfilling existing untagged published records.** Old records parse with empty `specifics/trends` and round-trip byte-identically; no migration pass.

## 5. Boundaries / invariants (must hold)

- **Tributaries untouched.** Zero schema/field change to trends, specifics, analytics, post-tracking. The assembler **reads** their ids/rows; it never writes them.
- **Additive + backward-compatible.** A published record with no specifics/trends serializes **byte-identically** to today (omit-empty); `parse(serialize(rec))===rec` holds with the new fields; `serialize(parse(oldText))===oldText` for pre-S3c records. `mintContentId` is **body-only**, so adding headers never changes a record's id/filename (no dedupe disturbance).
- **Pure core, single read-only IO seam.** `assemblePostGraph` is pure (inputs in, graph out — no clock/FS/network). The only IO is the read-only `brain assemble` loader; **no new disk writer** (the `--apply --confirm` consolidation path stays the sole `profile.md` writer; ingest stays the sole `published/` writer).
- **Decoupled packages.** The brain assembler does not import the analytics/trends/specifics TS packages; it takes a minimal analytics input shape and treats tributary ids as opaque 12-hex strings.
- **Honest matching.** The analytics↔post join is a **named heuristic** with explicit confidence tiers and a `none` outcome — never presented as a guaranteed key. The CLI shows the confidence so the operator can eyeball low-confidence matches.
- **Anti-sycophancy / provenance spine intact** — S3c adds a read path; it does not weaken the published-only learning guard or the profile.
- **Fresh-clone safe.** Missing `ingest/published/` or missing analytics → the assembler returns an empty/partial graph and the CLI degrades cleanly (no crash, no nag).
- **TDD iron law** — each rule (round-trip with the new fields, the producer threading, the match-tier logic, the graceful-absence path) is pinned by a failing brain-suite test before the code.

## 6. Success criteria (testable — deterministic, unit-level)

- **SC1 — record round-trip with raw-material ids:** a `PublishedRecord` with non-empty `specifics`/`trends` satisfies `parsePublishedRecord(serializePublishedRecord(rec)) === rec` (deep-equal); ids preserved in order. *(unit)*
- **SC2 — byte-identical backward-compat (byte contract pinned, FIX 2):** the new header lines, when present, are appended **after `source:`** and before the `\n---\n` sentinel; a record with **empty** `specifics`/`trends` emits the **unchanged 5-line header** (omit-empty), so against a fixture pre-S3c record string `oldText`, `serializePublishedRecord(parsePublishedRecord(oldText)) === oldText` (byte-equal). *(unit)*
- **SC3 — id validation (parse + producer, FIX-add):** a `specifics`/`trends` entry that is not 12-hex throws on `parsePublishedRecord` (never silently dropped), mirroring the `id` guard (`ingest.ts:82`); and the symmetric producer-boundary case — `ingestText({ specifics:["XYZ"] })` with a non-12-hex id — is rejected (validated at the producer, or guaranteed to throw on the immediate round-trip; the plan picks the seam). *(unit)*
- **SC4 — producer threading:** `ingestText({ ..., specifics:[a], trends:[b] })` writes a record whose round-tripped header carries exactly `[a]`/`[b]`; absent → `[]` (today's behaviour, regression pin). *(unit)*
- **SC5 — CLI repeatable flags:** `brain ingest --specific a --specific c --trend b` produces a published record tagged `specifics:[a,c] trends:[b]`. *(cli — brain suite)*
- **SC6 — assemble, high confidence (full row, FIX 4):** for a record whose normalized body begins with an analytics row's normalized title (≥ prefix-floor) and shares its `publishedDate`, `assemblePostGraph` returns that post with `match.confidence === "high"`, the **whole `PostAnalytics` row reference** attached, and its `specifics`/`trends` ids surfaced. *(unit)*
- **SC7 — assemble, low / none / truncation near-miss (FIX 3):** (a) no qualifying prefix match → `match: none` (never a false `high`); (b) a too-short title (< prefix-floor) that is a literal prefix → **not** `high` (floor guards false positives); (c) a LinkedIn-truncated-mid-word title (body starts with `norm(title)` minus a trailing partial token) and/or a date off by ≤ tolerance → `"low"`, per the §3.3 rule. *(unit)*
- **SC8 — assemble is pure + total:** empty `records` → empty graph; empty `analytics` → every post present with `match: none`; no throw on either. *(unit)*
- **SC9 — read-only CLI:** `brain assemble` prints the join (post → specifics/trends → matched analytics + confidence), newest-first, and **writes nothing** (no `profile.md`/`published/` mutation — asserted). Missing analytics/published (RISK C, fresh-clone) → clean partial/empty output, no crash. *(cli — brain suite)*
- **SC10 — gate green:** brain suite ≥ bumped `BRAIN_TESTS_FLOOR`; `scripts/test-runner.sh` green; `ASSERT_BASELINE_FLOOR` unchanged at 80. *(gate)*
- **SC11 — no regression:** ingest dedup/collision-safety, the published-only invariant, consolidation, supersede — all unchanged (the new fields are additive). *(unit/regression pins)*
- **SC12 — single-value flags unregressed (FIX 1):** after the repeatable-flag change, the existing single-value flags (`--file`/`--source`/`--date`) and boolean flags (`--scan-inbox`/`--confirm`) parse exactly as today across all subcommands. *(unit/cli)*

## 7. Verification

- **Deterministic (the whole slice):** SC1–SC11 are unit/CLI tests in `scripts/brain/tests/` + the structure-lint gate. **Like S3b (and unlike S3a), S3c has no behavioural-only SC** — the substrate, the producer, and the assembler are pure/CLI mechanics, fully exercisable in the harness with fixtures.
- **Run:** `(cd scripts/brain && npm test)` then `bash scripts/test-runner.sh`.
- **End-to-end demo (documented, optional):** with a real `LINKEDIN_STUDIO_DATA`: import an analytics CSV (existing path) → `brain ingest --specific <id>` a matching published post → `brain assemble` → confirm the post shows its specific + the matched analytics row. Records the north-star query working once, manually.

## 8. Open questions / genuine forks for brief-review + the operator (the go-gate)

1. **Fork-1 (headline — where the cross-ids live).** **RECOMMENDED: hub-side** — thread `specifics`/`trends` ids onto the brain's published record + resolve analytics (§2). Tributaries stay untouched; blast radius is bounded to `scripts/brain/`; most arc-aligned (`architecture.md:43`). **Alternative:** the STATE-anticipated *mutate-every-tributary* shape (add a cross-id field to `TrendRecord`/`Specific`/`PostAnalytics` + reconcile analytics' base-36 id) — higher blast, crosses 3 package boundaries + 3 gates, and (per C-2) still can't give analytics a real shared id without a resolver. The recommendation is to take the leaner hub-side cut; **operator confirms or redirects.**
2. **Fork-2 (analytics↔post join).** **RECOMMENDED: pure resolver by title-prefix + date with confidence tiers** (§3.3) — the only mechanism available (C-2: no body/URN). Accept the heuristic + `none` outcome as honest. **Alternative:** require a manual `analyticsId` tag on the published record too (operator types it at ingest) — more precise but more friction and still operator-supplied. Recommend the resolver; a manual tag can be added later, additively.
3. **Fork-3 (assembler surface).** **RECOMMENDED: pure `assemblePostGraph` + a thin read-only `brain assemble` CLI** (demonstrable end-to-end). **Alternative:** pure function + tests only, CLI deferred to when a reader/command consumes the graph (smaller, but no operator-visible payoff this slice). Recommend including the thin CLI.
4. **Fork-4 (metric *rendered* — presentation only, FIX 4).** The assembler always attaches the **whole `PostAnalytics` row reference** (data shape fixed; SC6 depends on it). Fork-4 governs only what the **CLI renders** as the headline "performance" figure — `engagementRate` (compact) vs a small bundle (impressions + engagementRate + saves). RECOMMEND `engagementRate` as the headline with the row available for a `--verbose` expansion. Plan pins the rendered field; the data shape is not in question.

## 9. Brief-review (light-Voyage) — folded

`voyage:brief-reviewer` verdict: **APPROVE-WITH-FIXES**. The central design (hub-side id-threading + analytics resolver) was confirmed sound, honestly bounded against both C-1 and C-2, with a clean IN/OUT boundary (no broken half-state) and verified gate mechanics (`BRAIN_TESTS_FLOOR=94` `test-runner.sh:716`, `ASSERT_BASELINE_FLOOR=80` `:913`). Fork-1 was judged framed honestly (not a rigged fork — the recommendation concedes the alternative's only real advantage and notes C-2 defeats even that). All 5 [FIX] folded above:

- **[FIX 1]** repeatable-flag blocker — `parseFlags` (`cli.ts:44-60`) returns `Record<string,string>` and overwrites on repeat → §3.2 now scopes the collect-into-arrays change + SC12 regression pin (the most material fix; SC5 was unsatisfiable as written). ✅
- **[FIX 2]** SC2 byte contract pinned — new lines appended **after `source:`**, omit-empty, fixture `oldText` byte-equality (`ingest.ts:51-57`). ✅
- **[FIX 3]** §3.3 join rule named in-brief (not deferred) — both-side `normalizeContent` + minimum prefix-length floor + truncation near-miss tier; SC7 expanded to floor + near-miss cases. ✅
- **[FIX 4]** assembler attaches the **full `PostAnalytics` row reference**; Fork-4 governs only CLI *rendering* — decouples SC6 (data) from the open fork. ✅
- **[FIX 5]** §2 citation corrected — *parse* is forgiving (`headerScalar`, `ingest.ts:61-65`); *serializer* is a fixed 5-line array (`:51-57`) S3c edits; + the C-2 note that `PostAnalytics.id = hash(title+date)` carries no independent linking signal. ✅

The 3 [RISK] are carried as explicit plan obligations (see §3.3/§3.4/§5; the plan's risk ledger pins them):
- **[RISK A]** all SCs use synthetic fixtures; the §7 end-to-end demo is the only real-data exercise of the resolver. A green gate does not prove a real LinkedIn CSV title matches a real captured body — the plan keeps the demo and treats a demo `none` as a *normalization-tightening signal* (honest-limit line, mirroring `consolidation-loop.md`/S3b §3.7), not a slice failure.
- **[RISK B]** `brain assemble` reads `analytics/posts/*.json` by inlining a raw-JSON read (`dataRoot('analytics/posts')`), **not** `loadAllPosts` — and extracts from `publishedDate` (not `date`). Pinned in §3.4.
- **[RISK C]** fresh-clone: `analytics/posts/` is gitignored/absent → the loader must degrade to every-post-`none` (SC8/SC9), never throw. Pinned in §3.4/§5.

## 10. Plan-critic + scope-guardian (light-Voyage) — folded

`voyage:plan-critic`: **REVISE → 1 BLOCK + 4 MAJOR + 4 MINOR**, all folded into `plan-sb-s3c.md` (the brief's design held; these were plan-precision defects):
- **[BLOCK]** the plan misstated `runIngest`/`main` wiring — `runIngest(flags)` is dispatched without `rest` (`cli.ts:92,238`); Step 4.2 now explicitly changes the signature AND the call site so `collectRepeated(rest,…)` has its input.
- **[MAJOR]** parse must NOT reuse `headerScalar` (`ingest.ts:61-65`) — it throws on absent keys, breaking every old record (SC2/SC11); Step 2.2 pins a non-throwing optional reader.
- **[MAJOR]** analytics-root skew — `ANALYTICS_ROOT` override (`storage.ts:67-72`) honoured by the analytics package but not by brain `dataRoot`; Step 3.5 + R-B name it as a doc-comment caveat (degrades to `none`, not silent).
- **[MAJOR]** `runAssemble` loader shape — needs full `PublishedRecord[]` (for `.body`), not `listPublished`'s summary; Step 4.3 pins full-record load, locally-derived `firstLine`, explicit desc sort.
- **[MAJOR]** producer-boundary throw (SC3) under-specified — Step 2.3 pins the guard (regex/message/placement) + empty-array skip so `scanInbox`/SC11 stay green.
- **[MINOR ×4]** tie-break determinism (Step 3.4 sorts rows), `PREFIX_FLOOR=24` rationale (Step 3.3), floor sub-count reconciliation (R-G), Step 1 compile-fixup of the two existing record literals.

`voyage:scope-guardian`: **ALIGNED** — 0 creep / 0 gap. All SC1–SC12 + all 5 FIXes + all 3 RISKs map to concrete plan steps; every brief §4 OUT item is genuinely excluded; the "Not touched" scope fence is accurate (four tributaries untouched, analytics READ-only via inlined JSON, hooks/agents/profile-grammar/`parseFlags` untouched).