linkedin-studio/docs/second-brain/brief-sb-s3c.md

Brief — SB-S3c: cross-silo id-threading (the "which raw material performs?" graph)

Slice: SB-S3c (third sub-slice of SB-S3). Status: LANDED — operator go 2026-06-23; TDD complete, brain suite 113/113, gate 95/0/0 (BRAIN_TESTS_FLOOR 94→113, ASSERT_BASELINE_FLOOR unchanged at 80). Light-Voyage: brief-review APPROVE-WITH-FIXES (5 FIX folded §9), plan-critic REVISE → 1 BLOCK + 4 MAJOR + 4 MINOR folded (§10), scope-guardian ALIGNED (0 creep/0 gap). Predecessors: SB-S0 (id/provenance substrate — mintEntityId/mintContentId, id.ts:6-7 reserved "SB-S3 will thread this id through the tributaries") · SB-S1 (published-gold ingest — the post anchor ingest/published/<contentId>.md) · SB-S2 (consolidation motor) · SB-S3a (first reader strategy-advisor — LANDED 4fa411f) · SB-S3b (supersede arm — LANDED 585f972, gate 95/0/0, BRAIN floor 94, ASSERT floor 80). Grounded in: a read-only code map of all four tributaries + scripts/brain/ (file:line cited throughout) — Explore sweep + firsthand reads of id.ts, consolidate.ts, types.ts, profile.ts, ingest.ts.

1. Operator decision (2026-06-23)

SB-S3 was decomposed into four sub-slices; standing operator direction is S3a (reader) → S3b (supersede) → S3c (id-threading) → S3d (hygiene+ops, last). S3a + S3b have landed. S3c is next — it is the slice the whole arc was built toward: the architecture's single hardest problem (architecture.md:17) — "A published post lands in three non-referencing places with no shared id. The question 'which raw material actually performs?' (specific → post → measured analytics) is unanswerable today" — and the explicit SB-S3 payoff (architecture.md:54, :80: "thread the id through tributaries → post ↔ specific ↔ trend ↔ analytics assemblable").

The go-before-code gate still applies per slice: this brief + the plan are presented for operator "go" (with the genuine forks in §8) before any code.

2. The north-star query, and the central design decision

What S3c must make answerable: given a published post, what raw material fed it (the specific(s) that grounded it, the trend(s) that inspired it) and how did it perform (its measured analytics). The killer second-brain query is specific → post → measured analytics: which raw material actually performs?

The grounding surfaced two hard constraints that reshape the slice (this is why the design below is leaner — and more arc-aligned — than the STATE's anticipated "mutate every tributary schema"):

• C-1 — the profile grammar is rigid. brain/profile.md is a fixed 6-token line-grammar (profile.ts:27-37,57-58; FACT_RE; parse∘serialize===doc, SC2). Persisting a fact→post provenance link onto the fact would add a 7th token = a breaking SCHEMA_VERSION bump (types.ts:26, "bumped only on a breaking grammar change") + a migration of every existing profile.md. That is its own slice, not S3c. OUT (§4).
• C-2 — analytics has no body and no URN. The LinkedIn CSV export carries only title (the post-content column, taken verbatim by the parser — csv-parser.ts:181; the "first ~100 chars" in models/types.ts:3 is a property of LinkedIn's export, not a code truncation) + publishedDate + metrics — no full body, no permalink/URN anywhere in the codebase. So analytics physically cannot compute the brain's mintContentId(body) (id.ts:51, body-byte-exact). Nor does its own id help: PostAnalytics.id = hash(title + date) (csv-parser.ts:138) is derived from the very fields the resolver already joins on, so it carries zero independent linking signal. The analytics↔post link therefore cannot be a stored shared id — it must be a resolver (join by title-prefix + date). This is an honest heuristic, not a guarantee, and the design names it as such.

Central design decision — thread the cross-ids onto the HUB-side post record, not into each tributary; assemble analytics by resolver. The architecture is explicit that tributaries stay separate and the hub holds the distilled summary + pointers (architecture.md:43,49: "Keep tributaries; add a thin hub + a cross-silo id"; "Forcing one schema would destroy [each silo's] distinctions"). The temporally-correct anchor is the post: trends + specifics exist before a post and feed it; analytics measures it after. The post's canonical object already exists — ingest/published/<contentId>.md — and it already carries the content-id (filename stem + id: header). Its parse side is tolerant of added header lines (headerScalar reads each key by regex, ingest.ts:61-65), so threading new fields is round-trip-safe; the serializer is a fixed 5-line array (ingest.ts:51-57) that S3c edits to conditionally append the two new lines (this is a deliberate edit, not an auto-handled field — corrected per brief-review FIX 5). Unlike the rigid profile grammar, this header grammar takes additive optional lines cleanly. So:

1. the published record carries the post's raw-material provenance as additive, optional header lines (specifics: + trends:, lists of tributary ids); and
2. a pure assembler joins published-record ↔ analytics-row by title-prefix + date (with a confidence tier) to attach measured performance.

This keeps all four tributary schemas (trends, specifics, analytics, post-tracking) untouched, needs no analytics base-36-id reconciliation, and is strictly additive + backward-compatible — while delivering the exact graph the north-star query needs. (That S3c's blast radius is bounded to the scripts/brain/ package, not the tributaries, is a finding that corrects the STATE's "rører tributær-skjemaer" expectation — surfaced as the headline fork, §8 Fork-1.)

3. Scope — what is IN (S3c)

1. Published record carries raw-material provenance (additive, optional). Extend PublishedRecord (ingest.ts:27-40) with specifics: string[] + trends: string[] (tributary ids the post was built from; default empty). serializePublishedRecord (ingest.ts:50-58) emits them as header lines only when non-empty (omit-empty → old records serialize byte-identically); parsePublishedRecord (ingest.ts:75-99) reads them (absent → []). Ids are validated to the 12-hex shape (the existing id guard idiom, ingest.ts:82); a malformed id throws (never silently dropped). The \n---\n sentinel split is unaffected (header lines never contain it).

2. Producer surface — tag raw material at ingest. ingestText (ingest.ts:165-180) accepts optional specifics?: string[] / trends?: string[], threaded onto the record. The brain ingest CLI gains repeatable --specific <id> / --trend <id> flags so the operator can tag a post's raw material at capture time. This requires a parser change (FIX 1): the shared parseFlags (cli.ts:44-60) returns Record<string,string> and overwrites on a repeated key (out[key] = next), so it cannot collect [a,c]. S3c extends flag-collection to gather repeated --specific/--trend into arrays — without changing single-value flag behaviour (--file/--source/--date/--scan-inbox parse exactly as today; regression-pinned, SC12). Minimal-disruption shape (a dedicated repeatable-flag pass for the two keys, or a string|string[] collect) is pinned in the plan. This makes the substrate immediately usable end-to-end (not fixture-only): ingest a post tagged --specific X → assemble → see the graph. Backward-compatible (flags absent → empty arrays = today's behaviour).

3. The assembler (the payoff) — a PURE function. assemblePostGraph({ records, analytics }) over already-loaded inputs (no I/O in the core) returns, per published post: its contentId, its specifics[]/trends[] ids, and its matched analytics — { row, confidence } carrying the whole PostAnalytics row reference (FIX 4: the assembler attaches the full row; which metric the CLI renders is Fork-4, decoupling the data shape from presentation), or none. The join rule (named here, not deferred — FIX 3), absorbing the title↔body asymmetry (verbatim CSV title vs byte-exact body, C-2): normalize both sides with the brain's content normalization (normalizeContent idiom: lowercase + collapse-whitespace + trim — the plan pins the exact fn / shared helper) and require norm(title) is a prefix of norm(body) with a minimum prefix-length floor (a too-short title must not false-match any body — floor value pinned in the plan, e.g. ≥ N chars). Confidence tiers: high = prefix-match (≥ floor) + same published_date; low = prefix-match (≥ floor) but date off by ≤ a small tolerance, OR a near-miss where LinkedIn truncated the title mid-word (the body's normalized text starts with norm(title) minus a trailing partial token); none = no qualifying prefix match. Deterministic, fully unit-testable (incl. a truncation near-miss case, SC7). The assembler reads tributary ids as opaque 12-hex strings and takes a minimal analytics input shape ({ title, publishedDate, metrics, … } — the raw PostAnalytics JSON shape, models/types.ts:1-8), so it does not import the trends/specifics/analytics packages (decoupled, §5).

4. A thin read-only IO + CLI surface. A brain assemble (or brain graph) subcommand: loads published records (dataRoot('ingest/published')) + the analytics batches by inlining a raw-JSON read of dataRoot('analytics/posts')/*.json (each file is an AnalyticsBatch with .posts[]; storage path <root>/posts/<date>-<shortId>.json, storage.ts:146-152) — NOT by importing the analytics package's loadAllPosts (storage.ts:197), per the §5 decoupling. It extracts the minimal shape from each posts[] entry (note the field is publishedDate, not date — RISK B), runs assemblePostGraph, and prints the post → raw-material → performance join (newest first). Read-only — it never writes the brain. A missing analytics/posts/ (gitignored — absent on a fresh clone, RISK C) or missing ingest/published/ degrades to an empty/partial graph (every post match: none), never a throw. (Fork-3, §8: pure-function-only with CLI deferred is the lighter alternative.)

5. Gate + doc reconciliation. Bump BRAIN_TESTS_FLOOR (currently 94, test-runner.sh:716) by the new brain-test count; ASSERT_BASELINE_FLOOR stays 80 — no new unconditional structure-lint check (the new tests are brain-suite tests, counted by BRAIN_TESTS_FLOOR, same idiom as S3b §3.7). Reconcile the now-true status lines (consolidation-loop.md, architecture.md:80 SB-S3 row, CLAUDE.md/STATE counts, the id.ts:6-7 "SB-S3 will thread" comment).

4. Non-goals — what is OUT (deferred)

• Persisting the fact→post link on brain/profile.md (the profile-grammar bump, C-1). The post→specifics/trends graph + the analytics resolver answer the north-star query without touching the rigid 6-token grammar. A 7th source/provenance-id token is a breaking SCHEMA_VERSION change with a migration — its own slice. The fact already carries provenance; that is enough for S3c.
• Mutating the tributary schemas (trends/specifics/analytics/post-tracking). Per architecture.md:43,49 tributaries stay separate; the cross-ids live hub-side on the post record (§2). No TrendRecord/Specific/PostAnalytics field is added; no analytics base-36-id → sha256 reconciliation. (Fork-1 is exactly this choice — operator may redirect to the broader mutate-tributaries shape.)
• post-tracking (state-updater.mjs) content-id. The loosest silo (regex-mutated markdown, prune-regex risk, state-updater.mjs:116,145) and not on the specific→post→analytics critical path. The post-tracking ↔ published ↔ analytics "triple-post reconciliation" is the named S3d concern (architecture.md:80). OUT here.
• A producer that auto-records which specifics/trends a draft used. S3c gives the substrate + the manual --specific/--trend tag at ingest (§3.2). Auto-capture from a drafting command (e.g. /linkedin:newsletter recording its livedSpecifics) is follow-on wiring, like S3a wired one reader.
• A fuzzy/semantic analytics match beyond title-prefix + date. No embedding/edit-distance match. If title-prefix+date doesn't resolve it, it's none (honest), not a guess. Tightening the matcher is later, additive.
• Backfilling existing untagged published records. Old records parse with empty specifics/trends and round-trip byte-identically; no migration pass.

5. Boundaries / invariants (must hold)

• Tributaries untouched. Zero schema/field change to trends, specifics, analytics, post-tracking. The assembler reads their ids/rows; it never writes them.
• Additive + backward-compatible. A published record with no specifics/trends serializes byte-identically to today (omit-empty); parse(serialize(rec))===rec holds with the new fields; serialize(parse(oldText))===oldText for pre-S3c records. mintContentId is body-only, so adding headers never changes a record's id/filename (no dedupe disturbance).
• Pure core, single read-only IO seam. assemblePostGraph is pure (inputs in, graph out — no clock/FS/network). The only IO is the read-only brain assemble loader; no new disk writer (the --apply --confirm consolidation path stays the sole profile.md writer; ingest stays the sole published/ writer).
• Decoupled packages. The brain assembler does not import the analytics/trends/specifics TS packages; it takes a minimal analytics input shape and treats tributary ids as opaque 12-hex strings.
• Honest matching. The analytics↔post join is a named heuristic with explicit confidence tiers and a none outcome — never presented as a guaranteed key. The CLI shows the confidence so the operator can eyeball low-confidence matches.
• Anti-sycophancy / provenance spine intact — S3c adds a read path; it does not weaken the published-only learning guard or the profile.
• Fresh-clone safe. Missing ingest/published/ or missing analytics → the assembler returns an empty/partial graph and the CLI degrades cleanly (no crash, no nag).
• TDD iron law — each rule (round-trip with the new fields, the producer threading, the match-tier logic, the graceful-absence path) is pinned by a failing brain-suite test before the code.

6. Success criteria (testable — deterministic, unit-level)

• SC1 — record round-trip with raw-material ids: a PublishedRecord with non-empty specifics/trends satisfies parsePublishedRecord(serializePublishedRecord(rec)) === rec (deep-equal); ids preserved in order. (unit)
• SC2 — byte-identical backward-compat (byte contract pinned, FIX 2): the new header lines, when present, are appended after source: and before the \n---\n sentinel; a record with empty specifics/trends emits the unchanged 5-line header (omit-empty), so against a fixture pre-S3c record string oldText, serializePublishedRecord(parsePublishedRecord(oldText)) === oldText (byte-equal). (unit)
• SC3 — id validation (parse + producer, FIX-add): a specifics/trends entry that is not 12-hex throws on parsePublishedRecord (never silently dropped), mirroring the id guard (ingest.ts:82); and the symmetric producer-boundary case — ingestText({ specifics:["XYZ"] }) with a non-12-hex id — is rejected (validated at the producer, or guaranteed to throw on the immediate round-trip; the plan picks the seam). (unit)
• SC4 — producer threading: ingestText({ ..., specifics:[a], trends:[b] }) writes a record whose round-tripped header carries exactly [a]/[b]; absent → [] (today's behaviour, regression pin). (unit)
• SC5 — CLI repeatable flags: brain ingest --specific a --specific c --trend b produces a published record tagged specifics:[a,c] trends:[b]. (cli — brain suite)
• SC6 — assemble, high confidence (full row, FIX 4): for a record whose normalized body begins with an analytics row's normalized title (≥ prefix-floor) and shares its publishedDate, assemblePostGraph returns that post with match.confidence === "high", the whole PostAnalytics row reference attached, and its specifics/trends ids surfaced. (unit)
• SC7 — assemble, low / none / truncation near-miss (FIX 3): (a) no qualifying prefix match → match: none (never a false high); (b) a too-short title (< prefix-floor) that is a literal prefix → not high (floor guards false positives); (c) a LinkedIn-truncated-mid-word title (body starts with norm(title) minus a trailing partial token) and/or a date off by ≤ tolerance → "low", per the §3.3 rule. (unit)
• SC8 — assemble is pure + total: empty records → empty graph; empty analytics → every post present with match: none; no throw on either. (unit)
• SC9 — read-only CLI: brain assemble prints the join (post → specifics/trends → matched analytics + confidence), newest-first, and writes nothing (no profile.md/published/ mutation — asserted). Missing analytics/published (RISK C, fresh-clone) → clean partial/empty output, no crash. (cli — brain suite)
• SC10 — gate green: brain suite ≥ bumped BRAIN_TESTS_FLOOR; scripts/test-runner.sh green; ASSERT_BASELINE_FLOOR unchanged at 80. (gate)
• SC11 — no regression: ingest dedup/collision-safety, the published-only invariant, consolidation, supersede — all unchanged (the new fields are additive). (unit/regression pins)
• SC12 — single-value flags unregressed (FIX 1): after the repeatable-flag change, the existing single-value flags (--file/--source/--date) and boolean flags (--scan-inbox/--confirm) parse exactly as today across all subcommands. (unit/cli)

7. Verification

• Deterministic (the whole slice): SC1–SC11 are unit/CLI tests in scripts/brain/tests/ + the structure-lint gate. Like S3b (and unlike S3a), S3c has no behavioural-only SC — the substrate, the producer, and the assembler are pure/CLI mechanics, fully exercisable in the harness with fixtures.
• Run: (cd scripts/brain && npm test) then bash scripts/test-runner.sh.
• End-to-end demo (documented, optional): with a real LINKEDIN_STUDIO_DATA: import an analytics CSV (existing path) → brain ingest --specific <id> a matching published post → brain assemble → confirm the post shows its specific + the matched analytics row. Records the north-star query working once, manually.

8. Open questions / genuine forks for brief-review + the operator (the go-gate)

1. Fork-1 (headline — where the cross-ids live). RECOMMENDED: hub-side — thread specifics/trends ids onto the brain's published record + resolve analytics (§2). Tributaries stay untouched; blast radius is bounded to scripts/brain/; most arc-aligned (architecture.md:43). Alternative: the STATE-anticipated mutate-every-tributary shape (add a cross-id field to TrendRecord/Specific/PostAnalytics + reconcile analytics' base-36 id) — higher blast, crosses 3 package boundaries + 3 gates, and (per C-2) still can't give analytics a real shared id without a resolver. The recommendation is to take the leaner hub-side cut; operator confirms or redirects.
2. Fork-2 (analytics↔post join). RECOMMENDED: pure resolver by title-prefix + date with confidence tiers (§3.3) — the only mechanism available (C-2: no body/URN). Accept the heuristic + none outcome as honest. Alternative: require a manual analyticsId tag on the published record too (operator types it at ingest) — more precise but more friction and still operator-supplied. Recommend the resolver; a manual tag can be added later, additively.
3. Fork-3 (assembler surface). RECOMMENDED: pure assemblePostGraph + a thin read-only brain assemble CLI (demonstrable end-to-end). Alternative: pure function + tests only, CLI deferred to when a reader/command consumes the graph (smaller, but no operator-visible payoff this slice). Recommend including the thin CLI.
4. Fork-4 (metric rendered — presentation only, FIX 4). The assembler always attaches the whole PostAnalytics row reference (data shape fixed; SC6 depends on it). Fork-4 governs only what the CLI renders as the headline "performance" figure — engagementRate (compact) vs a small bundle (impressions + engagementRate + saves). RECOMMEND engagementRate as the headline with the row available for a --verbose expansion. Plan pins the rendered field; the data shape is not in question.

9. Brief-review (light-Voyage) — folded

voyage:brief-reviewer verdict: APPROVE-WITH-FIXES. The central design (hub-side id-threading + analytics resolver) was confirmed sound, honestly bounded against both C-1 and C-2, with a clean IN/OUT boundary (no broken half-state) and verified gate mechanics (BRAIN_TESTS_FLOOR=94 test-runner.sh:716, ASSERT_BASELINE_FLOOR=80 :913). Fork-1 was judged framed honestly (not a rigged fork — the recommendation concedes the alternative's only real advantage and notes C-2 defeats even that). All 5 [FIX] folded above:

• [FIX 1] repeatable-flag blocker — parseFlags (cli.ts:44-60) returns Record<string,string> and overwrites on repeat → §3.2 now scopes the collect-into-arrays change + SC12 regression pin (the most material fix; SC5 was unsatisfiable as written). ✅
• [FIX 2] SC2 byte contract pinned — new lines appended after source:, omit-empty, fixture oldText byte-equality (ingest.ts:51-57). ✅
• [FIX 3] §3.3 join rule named in-brief (not deferred) — both-side normalizeContent + minimum prefix-length floor + truncation near-miss tier; SC7 expanded to floor + near-miss cases. ✅
• [FIX 4] assembler attaches the full PostAnalytics row reference; Fork-4 governs only CLI rendering — decouples SC6 (data) from the open fork. ✅
• [FIX 5] §2 citation corrected — parse is forgiving (headerScalar, ingest.ts:61-65); serializer is a fixed 5-line array (:51-57) S3c edits; + the C-2 note that PostAnalytics.id = hash(title+date) carries no independent linking signal. ✅

The 3 [RISK] are carried as explicit plan obligations (see §3.3/§3.4/§5; the plan's risk ledger pins them):

• [RISK A] all SCs use synthetic fixtures; the §7 end-to-end demo is the only real-data exercise of the resolver. A green gate does not prove a real LinkedIn CSV title matches a real captured body — the plan keeps the demo and treats a demo none as a normalization-tightening signal (honest-limit line, mirroring consolidation-loop.md/S3b §3.7), not a slice failure.
• [RISK B] brain assemble reads analytics/posts/*.json by inlining a raw-JSON read (dataRoot('analytics/posts')), not loadAllPosts — and extracts from publishedDate (not date). Pinned in §3.4.
• [RISK C] fresh-clone: analytics/posts/ is gitignored/absent → the loader must degrade to every-post-none (SC8/SC9), never throw. Pinned in §3.4/§5.

10. Plan-critic + scope-guardian (light-Voyage) — folded

voyage:plan-critic: REVISE → 1 BLOCK + 4 MAJOR + 4 MINOR, all folded into plan-sb-s3c.md (the brief's design held; these were plan-precision defects):

• [BLOCK] the plan misstated runIngest/main wiring — runIngest(flags) is dispatched without rest (cli.ts:92,238); Step 4.2 now explicitly changes the signature AND the call site so collectRepeated(rest,…) has its input.
• [MAJOR] parse must NOT reuse headerScalar (ingest.ts:61-65) — it throws on absent keys, breaking every old record (SC2/SC11); Step 2.2 pins a non-throwing optional reader.
• [MAJOR] analytics-root skew — ANALYTICS_ROOT override (storage.ts:67-72) honoured by the analytics package but not by brain dataRoot; Step 3.5 + R-B name it as a doc-comment caveat (degrades to none, not silent).
• [MAJOR] runAssemble loader shape — needs full PublishedRecord[] (for .body), not listPublished's summary; Step 4.3 pins full-record load, locally-derived firstLine, explicit desc sort.
• [MAJOR] producer-boundary throw (SC3) under-specified — Step 2.3 pins the guard (regex/message/placement) + empty-array skip so scanInbox/SC11 stay green.
• [MINOR ×4] tie-break determinism (Step 3.4 sorts rows), PREFIX_FLOOR=24 rationale (Step 3.3), floor sub-count reconciliation (R-G), Step 1 compile-fixup of the two existing record literals.

voyage:scope-guardian: ALIGNED — 0 creep / 0 gap. All SC1–SC12 + all 5 FIXes + all 3 RISKs map to concrete plan steps; every brief §4 OUT item is genuinely excluded; the "Not touched" scope fence is accurate (four tributaries untouched, analytics READ-only via inlined JSON, hooks/agents/profile-grammar/parseFlags untouched).