open/linkedin-studio
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions

docs(linkedin-studio): second-brain architecture (approved) + 3 research reports

Browse source 
Persist the operator-approved second-brain architecture design and the three
parallel research threads that fed it, so nothing is lost before build:

- architecture.md — approved design: thin Markdown `brain/` hub (two-layer
  profile.md + episodic journal/ + operations.md + index.md MOC) over the
  existing typed tributaries (voice/specifics/trends/analytics), a
  provenance-tagged ingest/ seam, a sleep-time consolidation loop with
  evidence-threshold promotion + temporal-validity + anti-sycophancy/
  anti-collapse invariants. Build sequence SB-S0..S4.
- research/connector-egress.md — LinkedIn data egress reality (EU/EEA DMA
  portability API = auto for content; analytics manual CSV; no scraping).
- research/secondbrain-sota.md — 2026 second-brain / AI-memory SOTA synthesis.
- research/silo-inventory.md — faithful inventory of the 12 existing per-user
  silos + the 5 hardest unification problems.

Boundary confirmed: engine -> plugin (domain-general), user data -> data dir,
cockpit -> Maskinrommet. Design phase, no code yet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RigJBiRFNtFZKCz21qNbQ4
This commit is contained in:
Kjell Tore Guttormsen 2026-06-23 12:55:17 +02:00
commit d3199eb997
4 changed files with 409 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

102
docs/second-brain/architecture.md Normal file
View file

@ -0,0 +1,102 @@
# Second Brain — Architecture Design
> **Status:** architecture **approved by operator 2026-06-23**. Design phase — no code yet.
> **Boundary (confirmed 2026-06-23):** the **engine** (store schema · evolution loop · ingest seam) → **the plugin** (domain-general, shareable); the **user's data** (posts · articles · newsletters · plans · ideas) → the **per-user data dir** (`${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/`, survives reinstall); the **personal cockpit** (the operator's day-to-day operations centre) → **Maskinrommet** (a thin layer that reads/writes *through* the plugin's store, never a fork of the engine).
> **Research inputs (three parallel threads, 2026-06-23):** `research/connector-egress.md` · `research/secondbrain-sota.md` · `research/silo-inventory.md`.
## The problem
A "second brain" stores everything about one creator — posts, articles, newsletters, data, plans, ideas — and compounds it into an ever-improving, user-aligned profile. It is **memory AND an operations centre**.
The three research threads reframed the task in a decisive way:
1. **There is nothing to migrate.** M0 already routed all **12 existing per-user silos** through one tested seam (`hooks/scripts/data-root.mjs` `getDataRoot()` + its TS twin `scripts/analytics/src/utils/storage.ts`), default `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/`. They all survive reinstall today. The task is **unify**, not move.
2. **Not a vector/graph DB.** For *our* case the portable, auditable, drift-resistant 2026 standard is **plain-text + JSON + git** — because **Claude *is* the semantic-retrieval engine** (it reads files in-context). A heavyweight store adds infra, breaks portability, and invites lock-in for no gain.
3. **Auto-ingest exists — for an EU/EEA member.** The DMA Member Data Portability API is self-serve in EU/EEA + Switzerland and gives automated pull of **content** (post text + articles), but **not** received-analytics. Analytics stays the existing manual CSV path. Scraping is a ToS breach with active enforcement → never baked in. So ingest is **manual-first as the contract**; any connector is an additive tributary.
The real gap (from the silo inventory): the 12 silos are **per-user but siloed and heterogeneous** — three storage idioms, two roots, no cross-references, provenance reinvented under different field names in each. A published post lands in three non-referencing places with no shared id. The question *"which raw material actually performs?"* (specific → post → measured analytics) is **unanswerable today** — and it is exactly what a second brain should answer.
## The shape: a thin Markdown hub over typed tributaries
```
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/
brain/
index.md # MOC / hub — one-screen pointer to every tributary + freshness flags
# (the "memory AND ops centre" front page)
profile.md # SEMANTIC, two-layer: §static (stable) + §dynamic (evolving).
# Each fact: value · first_seen · last_seen · evidence_count ·
# provenance(human|published|ai-draft) · status(active|superseded)
operations.md # PLANS / IDEAS + the "who I am now" anchor (frozen-past-self guard)
journal/ # EPISODIC, append-only: YYYY-MM-session.md — raw, never edited;
# the source the consolidation loop distils FROM
ingest/
inbox/ # NEW drop-zone: manual import OR (EU/EEA) connector output
published/ # processed, provenance=published — the GOLD signal
voice-samples/ # tributary (style) — stays its own store
specifics-bank/ # tributary (raw material) — stays its own store
trends/ # tributary (external signal) — stays its own store
analytics/ # tributary (performance) — stays its own store
```
### Tributaries vs hub — the explicit call
- **Stay separate (tributaries), feed the hub via distilled pointer-summaries:** voice-samples, specifics-bank, trends, analytics, ingest. Each has a *different dedupe key + lifecycle* (trends: title+url, fast decay · specifics-bank: human-only, near-permanent · analytics: numeric, immutable). Forcing one schema would destroy those distinctions — the silo inventory warns of this explicitly. The hub holds a distilled summary of each + a pointer.
- **Fold INTO the hub:** the flat `profile/user-profile.md` → two-layer `brain/profile.md`. Scattered `audience-insights/` / `examples/` become dynamic-layer sections or tributary-summaries referenced from `index.md`. The **operations/plans centre is genuinely new**.
### Alternatives considered (and rejected)
- **Vector/graph DB (Pinecone/Neo4j/Zep-style):** rejected — adds infra, breaks plain-text portability + git audit/rollback, invites lock-in, and duplicates retrieval that Claude already does in-context. The four SOTA ideas that port cleanly to files (episodic/semantic split, provenance-weighting, evidence-threshold promotion, temporal-validity frontmatter) give most of the value; the vector/graph machinery does not port and is not needed.
- **One unified schema across all silos:** rejected — destroys each silo's distinct dedupe key + lifecycle. Keep tributaries; add a thin hub + a cross-silo id.
## Invariants (the spine)
- **Provenance-weighted learning (the critical guard):** profile/voice learn from `provenance=published` (human-edited) **only**, never `ai-draft`. This is the model-collapse guard for our exact risk — a content engine that learns voice from what it itself drafted collapses toward its own priors. Generalises `voice-scrubber`'s existing rule ("gold standard = approved editions, not the English post corpus") to a system-wide invariant.
- **One canonical entity id + one provenance shape**, threaded through the silos → makes the post ↔ specific ↔ trend ↔ analytics graph assemblable. Fixes the inventory's hardest problem.
- **Episodic / semantic physically separated** (raw events in `journal/`, distilled facts in `profile.md`).
- **Evidence-threshold promotion (anti-overfit):** the dynamic layer decays; a fact enters the **static** layer only on explicit confirmation OR ≥N independent observations. Single weird sessions never reshape identity.
- **Temporal-validity + freshness nudge:** every fact carries `last_seen`; retrieval weights recent over old; the session-start nudge flags facts not refreshed in T days (extend the existing trend-staleness nudge brain-wide).
- **Anti-sycophancy as a built-in default, not a toggle:** consolidation + content agents treat the profile as evidence to *test*, not flatter. (Personalization context measurably *increases* sycophancy, so counter-pressure it every time the profile is used.)
- **Frozen-past-self guard:** `operations.md` holds a periodic user-authored "where I'm headed now" anchor that deprecates older inferences.
- **Zero required curation (graveyard guard):** the loop compounds value with ~zero mandatory upkeep; curation is available but never required. git = free rollback.
## The evolution loop (the compounding mechanism)
Run on a cadence (reuse the existing session-start staleness-nudge surface) as a **sleep-time consolidation pass**, not inline cost:
1. **Capture (episodic, auto):** real-signal sessions append to `brain/journal/`. The ingest seam (`ingest/inbox/`) takes the user's actual published posts — manual import first; published output tagged `provenance=published`.
2. **Consolidate (reflection):** an Opus agent reads new journal + published + tributary deltas, proposes profile updates as a **diff**, never a silent overwrite. Each candidate carries `evidence_count`, `provenance`, timestamps.
3. **Promote with thresholds:** static-layer entry needs confirmation or ≥N observations; one-offs stay in the decaying dynamic layer.
4. **Reconcile contradictions:** classify temporal-update (supersede, keep old `status: superseded` + date) · contradictory (keep both, timestamped) · condition-dependent (scope) · distractor (drop). Bias toward "keep both + surface the conflict" over silent overwrite.
5. **Decay + freshness:** weight recent over old; flag stale facts at session-start.
6. **Curate (optional, never required):** the diff is presentable; confirm/edit/delete/roll-back via git.
## Build sequence (full ambition, incremental — one slice per session)
| Slice | Content | Why this order |
|-------|---------|----------------|
| **SB-S0 — Foundation** | `brain/` scaffold + two-layer `profile.md` (fold in `user-profile.md`) + `index.md` MOC + `operations.md` + `journal/`; entity-id + provenance shape as a small typed, tested module; `ingest/` dirs + manual-import contract. **No loop yet.** | Smallest thing that stands up and is testable; locks the id/provenance spine everything hangs on |
| **SB-S1 — Ingest + gold signal** | Manual import → `ingest/published/` with `provenance=published`; wire profile/voice to learn from published-only | The gold signal before the loop that consumes it |
| **SB-S2 — Evolution loop** | Sleep-time consolidation (reuse session-start surface): journal+published+tributary deltas → profile **diff** w/ evidence_count/provenance/timestamps; threshold promotion; contradiction reconciliation; brain-wide freshness nudge | The compounding mechanism |
| **SB-S3 — Cross-silo graph + ops centre** | Thread the id through tributaries (post↔specific↔trend↔analytics assemblable); flesh out `operations.md`; retire the dead `content-history.md` + triple-post reconciliation | Finally answers "which raw material performs?" |
| *(later / optional)* **SB-S4** | EU/EEA DMA portability API as an auto-tributary into `ingest/inbox/` | Additive; never a dependency |
## What's genuinely hard (honest flags)
1. **Voice fidelity to a private individual is limited even at SOTA** — the real win is grounded *content* (specifics-bank) over mimicked *style*. Manage expectations.
2. **Detecting that a high-relevance fact went stale is UNSOLVED field-wide** (STALE benchmark: all tested models/frameworks fail). We timestamp + nudge; we cannot auto-detect "changed jobs." Keep a human in that loop.
3. **Contradiction classification** (real change vs context-scoped vs noise) is emerging + error-prone → bias to keep-both-timestamped.
4. **The connector** — no clean LinkedIn self-serve content API outside EU/EEA. Build the manual ingest seam as the contract; any connector is a tributary, never a dependency. Newsletter-edition coverage by the portability `ARTICLES` domain is **not fully verified** — check against a real export.
5. **Sycophancy is structural** — app-level mitigations reduce, not eliminate.
6. **Avoiding the graveyard is a product problem** — the loop must compound value with ~zero required curation or it dies in 6 months like 82% of second brains.
## Verification (SB-S0, when we build it)
- Set `LINKEDIN_STUDIO_DATA` to a temp dir, run init → assert `brain/{index,profile,operations}.md` + `journal/` + `ingest/{inbox,published}` exist via the `getDataRoot` seam.
- `profile.md` parses two-layer (§static / §dynamic); `user-profile.md` fields fold in without loss (diff check).
- entity-id module: deterministic id mint + provenance shape, unit-tested.
- **No regression:** gate `scripts/test-runner.sh` 89/0/0 green; trends 24/24; specifics 28/28; contract 33/33.
- **Key assumption to test early:** "Claude-as-retrieval-engine over plain files is sufficient (no vector DB)" — testable with a retrieval scenario once the brain holds content. Marked as assumption until proven.
## Bottom line
A thin two-layer Markdown hub (`brain/`: semantic `profile.md` + episodic `journal/` + ops `operations.md` + `index.md` MOC) over the existing typed tributaries, fed by a provenance-tagged ingest seam, maintained by a sleep-time consolidation loop with evidence-threshold promotion, temporal-validity reconciliation, and a built-in anti-sycophancy / anti-collapse stance. Every silo stays a tributary except the flat `user-profile.md`, which folds in.

65
docs/second-brain/research/connector-egress.md Normal file
View file

@ -0,0 +1,65 @@
# Research — LinkedIn content/data egress for an individual creator (2026-06-23)
> Research thread `connector-recon`, 2026-06-23. Verification-duty: every technical claim backed by an official source/URL; unverifiable items marked "IKKE VERIFISERT". This is research INPUT to `../architecture.md` — preserved verbatim so nothing is lost.
**Bottom line:** For ONE creator pulling their OWN content out of LinkedIn there is exactly one realistic *automated* path, and it is geofenced to **EU/EEA + Switzerland**: the **DMA Member Data Portability API** (self-serve, scope `r_dma_portability_self_serve`). Outside that region the honest answer is **manual native export** for content + **manual analytics CSV download** for per-post metrics. Full hands-off automation is NOT available outside the EU/EEA. Engagement/impression analytics are NOT in any portability/export *content* path — they live only in the separate manual analytics CSV.
## 1. Path-by-path table
| # | Path | What it captures (your OWN data) | Verdict | Source |
|---|------|----------------------------------|---------|--------|
| 1a | **DMA Member Data Portability API — Member Snapshot** (`r_dma_portability_self_serve` / `r_dma_portability_member`) | **Full post text** (`MEMBER_SHARE_INFO`: all shares/re-shares incl. date, URL, shared comments, visibility), **articles** (`ARTICLES`), profile, connections, comments (`ALL_COMMENTS`), reactions you gave (`ALL_LIKES`), reposts, messages — ~60+ domains. **Historical** data included. **NO impression/reach/engagement-received analytics domain exists.** | **AUTO** — but **EU/EEA + Switzerland members ONLY** | learn.microsoft.com/en-us/linkedin/dma/member-data-portability/member-data-portability-member/?view=li-dma-data-portability-2026-05 ; …/shared/snapshot-domain?view=li-dma-data-portability-2026-05 |
| 1b | **DMA Member Data Portability API — Member Changelog** | Real-time stream of your interactions (posts created, comments, reactions) from consent onward; **queryable only for the last 28 days** | **AUTO** (rolling 28-day window) — EU/EEA + CH only | (same member page, Changelog section) |
| 1c | **Marketing API / Community Management API** | Org/Page-level posting + analytics; NOT a self-serve member-content path | **RISKY/BLOCKED for individuals** — LinkedIn Partner Program approval (legal entity, use-case review, weeks/months) | learn.microsoft.com/en-us/linkedin/marketing/community-management/community-management-overview?view=li-lms-2026-05 ; connectsafely.ai/articles/linkedin-api-complete-guide-2026 |
| 2 | **Native "Get a copy of your data"** (Settings → Data Privacy) | ZIP of **CSV + JSON + HTML**: profile, connections, messages, comments, **posts** (Shares), **articles** (HTML folder). Basic categories "within minutes"; **larger archive ~24h** (72h download window). **Saved-items file is URLs+dates only.** **No analytics.** | **MANUAL** | linkedin.com/help/linkedin/answer/a1339364/downloading-your-account-data |
| 3a | **Native newsletter content** | **NO native RSS feed.** Subscription = email/notification per edition only. Edition text lives as articles. | **MANUAL** | linkedin.com/help/linkedin/answer/a517925 |
| 3b | **Newsletter via your own subscription email** | Each new edition arrives as an **email to you** → ingest your own mailbox / email-to-RSS | **MANUAL (clean)** | linkedin.com/help/linkedin/answer/a517925 |
| 3c | **Third-party newsletter→RSS scrapers** (rss.app, linkedin-newsletter-rss) | Scrapes public newsletter/article URL into RSS | **RISKY** (third-party scraping) | rss.app/rss-feed/linkedin ; github.com/chrisns/linkedin-newsletter-rss |
| 4a | **Per-post analytics CSV export** (native dashboard "Export") | **Per-post** impressions, unique viewers, clicks, reactions, comments, shares, engagement rate. **Manual, ~1-year range cap.** Profile views NOT exportable. **ONLY path to your engagement metrics.** | **MANUAL** | tryordinal.com/blog/how-to-export-analytics-from-linkedin-to-excel-or-a-csv ; linkedin.com/help/linkedin/answer/a551206/ |
| 4b | **Zapier / Make** | Trigger on *your* new posts (publish-side); thin/no historical pull of post bodies+analytics; bound by LinkedIn's official connector scopes | **MANUAL-ish / limited** | connectsafely.ai/articles/is-linkedin-automation-safe-tos-scraping-guide-2026 |
| 4c | **Phantombuster / scrapers / browser extensions** | Can extract post text + visible engagement counts | **RISKY** — breaches User Agreement (prohibited automation); active ban-wave enforcement | linkedin.com/help/linkedin/answer/a1341387 ; connectsafely.ai/articles/is-linkedin-automation-safe-tos-scraping-guide-2026 |
## 2. Single cleanest realistic ingest path for ONE user (own data)
Splits by region, and even at best it is TWO streams, not one:
**EU/EEA or Switzerland → near-full automation for CONTENT:**
- **Member Data Portability API, self-serve.** Create a dev app **using LinkedIn's mandated default Company Page** ("Member Data Portability (Member) Default Company" — you must NOT create your own page), request the *Member Data Portability (Member)* product, generate a token via the OAuth Token Generator with scope `r_dma_portability_self_serve`, consent, then call **Member Snapshot** (`GET /rest/memberSnapshotData?q=criteria`) for historical full post text (`MEMBER_SHARE_INFO`) + `ARTICLES`, and poll **Member Changelog** for the rolling 28-day live stream. Token valid up to **1 year** before re-consent. Genuinely automatable for a second-brain content store.
- **BUT analytics still come from the manual CSV (4a)** — portability exposes reactions/comments *you made*, not impressions/reach *received*.
**Outside EU/EEA/CH → manual is the honest answer:**
- Content: native "Get a copy of your data" (larger archive, ~24h, CSV+JSON+HTML) — paths 2 + 3.
- Analytics: native per-post analytics CSV (4a).
- Newsletter: own subscription email (3b) or article HTML from the archive.
**Is full automation possible?** Only inside EU/EEA/CH, and only for content (not received-analytics). Outside, **manual export/paste is the correct, ToS-safe answer.** Scraper/extension/Phantombuster routes are a **User Agreement breach** with active enforcement (LinkedIn's reported Q1-2026: 23.5M automated sessions flagged) — do not bake into the plugin's ingest.
## 3. What's uncertain / changing (date: 2026-06-23)
- **Geofence may widen.** Portability API is DMA-driven, explicitly "**at this time**… EU/EEA + Switzerland." Re-check the MS Learn moniker (`li-dma-data-portability-2026-05`) each quarter. IKKE VERIFISERT that it will ever cover non-EU members.
- **Analytics-in-portability gap.** No Snapshot domain carries received impressions/reach/engagement-rate (`ALL_LIKES`/`ALL_COMMENTS` = actions *you* took). Treat "portability gives you your analytics" as FALSE until a future domain appears. Native analytics CSV stays the only metrics source.
- **Newsletter editions in export/snapshot.** `ARTICLES` covers authored articles; whether every *newsletter edition* is reliably captured as an article vs. needing the email path is **partially uncertain** — native export help does not enumerate newsletter editions. Verify against a real export.
- **Saves / dwell** remain non-exportable (consistent with the plugin's existing S16 note). IKKE VERIFISERT any change.
- **Versioned API drift.** Snapshot uses `LinkedIn-Version: YYYYMM`; domain list/schemas change month-to-month. Pin a version, re-validate.
## 4. Sources (accessed 2026-06-23)
**Official / authoritative:**
- Member Data Portability (Member), MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/dma/member-data-portability/member-data-portability-member/?view=li-dma-data-portability-2026-05
- Member Snapshot API, MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/dma/member-data-portability/shared/member-snapshot-api?view=li-dma-data-portability-2026-05
- Member Snapshot Domain list, MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/dma/member-data-portability/shared/snapshot-domain?view=li-dma-data-portability-2026-05
- Member portability APIs, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a6214075
- Community Management Overview, MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/marketing/community-management/community-management-overview?view=li-lms-2026-05
- Download your account data, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a1339364/downloading-your-account-data
- Export your Page analytics report, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a551206/
- Manage a newsletter, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a517925
- Prohibited software and extensions, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a1341387
- LinkedIn DMA Portability API Terms: https://www.linkedin.com/legal/l/portability-api-terms
**Secondary (corroborating, not authoritative):**
- connectsafely — LinkedIn API 2026: https://connectsafely.ai/articles/linkedin-api-complete-guide-2026
- connectsafely — automation/ToS 2026: https://connectsafely.ai/articles/is-linkedin-automation-safe-tos-scraping-guide-2026
- tryordinal — Export LinkedIn analytics (Jan 2026): https://www.tryordinal.com/blog/how-to-export-analytics-from-linkedin-to-excel-or-a-csv
- rss.app LinkedIn / chrisns/linkedin-newsletter-rss: https://rss.app/rss-feed/linkedin · https://github.com/chrisns/linkedin-newsletter-rss
**Verification log (key claims → source):** Portability API exists + self-serve scope + EU/EEA+CH-only + 1-yr token → MS Learn member page. Full post text + articles + historical, no analytics domain → MS Learn snapshot-domain list (verbatim `MEMBER_SHARE_INFO`, `ARTICLES`, `ALL_LIKES`, `ALL_COMMENTS`; no impressions domain present). Changelog 28-day window → MS Learn member page. Native export = CSV+JSON+HTML, articles as HTML, ~24h, saved-items URLs-only → LinkedIn Help + corroborating guides. No native newsletter RSS → LinkedIn Help newsletter page (silent on RSS) + third-party scrapers fill the gap. Per-post analytics CSV manual, ~1yr cap, only metrics source → LinkedIn Help + tryordinal. Scraping = User Agreement breach w/ enforcement → LinkedIn Prohibited Software + connectsafely.

132
docs/second-brain/research/secondbrain-sota.md Normal file
View file

@ -0,0 +1,132 @@
# Research — Second Brain / Personal-Memory SOTA 2026 (synthesis + architecture rec)
> Research thread `secondbrain-recon`, 2026-06-23. Four streams (agent memory, PKM, profile-evolution, pitfalls) with ESTABLISHED / EMERGING / SPECULATIVE labels. Research INPUT to `../architecture.md` — the architecture's design choices trace to this. Preserved so nothing is lost.
## PART A — STATE OF THE ART (sourced, calibrated)
### 1. Agent memory architectures
**Three-tier taxonomy is the design default** (ESTABLISHED). Episodic (timestamped *what happened*), semantic (distilled *what is true*), procedural (*how to behave*). Formalized for LLM agents by CoALA (arXiv:2309.02427); shared vocabulary across Letta, Mem0, Zep, LangMem. Load-bearing for us: **keep episodic and semantic physically separate** — mixing raw events with distilled facts degrades retrieval both ways. Procedural is rarest; only LangMem makes it first-class.
**Hybrid vector + structured/graph beats pure vector** (ESTABLISHED). Pure embeddings fail on multi-hop, temporal ("what did the user want in Q1 vs now"), exact-identifier, and contradiction queries. 2026 pattern = query router over vector (semantic) + graph/relational (entities + temporal validity) + BM25 (keyword), fused via Reciprocal Rank Fusion + cross-encoder rerank (+39.7% MRR@3, arXiv:2604.01733). Named systems:
- **Mem0** (arXiv:2504.19413, ~48K stars) — vector-primary + entity linking; LLM extracts facts per turn; ~90% token reduction. Benchmark numbers vendor-disputed.
- **Zep/Graphiti** (arXiv:2501.13956) — **bi-temporal knowledge graph**: every fact an edge with `valid_from`/`valid_until`. 63.8% vs Mem0's 49.0% on independent LongMemEval. **Most important pattern for us** (staleness).
- **Letta/MemGPT** (arXiv:2310.08560) — OS-tiered (core/recall/archival), agent self-manages via tools. 74.0% LoCoMo.
- LangMem (EMERGING), Cognee (EMERGING, graph-RAG for docs), A-MEM (EMERGING, NeurIPS 2025).
**Consolidation = the "reflection/sleep" pattern** (ESTABLISHED original; sleep-time-compute EMERGING). Stanford Generative Agents (arXiv:2304.03442): append-only memory stream → LLM importance-scores each event → periodic reflection synthesizes higher-order insights stored back as semantics. Letta sleep-time compute (arXiv:2504.13171) runs it as background agents during idle so the main loop stays fast.
**Forgetting is mandatory** (ESTABLISHED TTL/LRU; EMERGING importance-weighted decay). "Append-only memory stores are a trap." Tiers: session ~24h, project ~6mo, core indefinite, cold-archive after 90–180d unused. Mem0 decay is a *retrieval re-ranking* layer (recent → 1.5× boost, unused → 0.3× damp), not deletion. The Generative-Agents **retrieval triad** (recency + importance + relevance) is still the reference scoring formula.
### 2. PKM methods + AI-native tools
- **PARA** (Forte) — top-down, organize-by-*actionability*; strong for ops, no native linking.
- **Zettelkasten / Evergreen** (Luhmann/Matuschak) — atomic note (one idea), dense links, **emergent** bottom-up structure; the cognitive work is *writing the note*, which embeddings can't replace.
- **LYT/Maps of Content** (Milo) — navigation layer over the graph.
**2026 consensus is NOT "links win"** — embeddings make folder-vs-link partly moot for *retrieval* while reaffirming neither scales alone. Mem (OpenAI embeddings + Pinecone), Tana (every bullet a graph node, typed Supertags), Obsidian Smart Connections (*local* embeddings — privacy). None *auto-organize* — they auto-*retrieve*. AI collapses CODE's **Organize** step (misfiling recoverable) and accelerates **Capture**; it does NOT do **Distill** (cross-temporal relevance) or **Express** (human synthesis). Sober dissent: "the best memory tool is the one you actually keep feeding" — capture/maintenance friction dominates retrieval sophistication.
### 3. Profile-evolution / preference-learning
**Two-layer profile is standard** (ESTABLISHED): static (stable) + dynamic (evolving). PersonaMem (COLM 2025): frontier models **freeze on the static snapshot, fail to track the dynamic layer** — frozen-past-self is empirically real.
**A plugin gets only app-level adaptation** (no model-level RLHF/DPO/per-user-LoRA). App-level = **RAG** (retrieve past interactions) + **PAG** (summarize history into a compact profile, inject, re-summarize on update — PURE, SIGIR 2025). **Style/voice = few-shot from the user's own corpus**; content-similar + length-matched selection beats random (arXiv:2509.14543), but fidelity to a private individual's implicit style is limited even at SOTA.
**Drift guards (the crux):**
- **Recency weighting** (ESTABLISHED): `score = α·sim + (1-α)·f_time(Δt)`.
- **Event-driven invalidation** (EMERGING): OpenAI "Dreaming" auto-rewrites "going to X in July" → "went to X" after the date (recall 41.5%→82.8%). State-typed facts (active/completed/expired).
- **Explicit user curation** (ESTABLISHED): ChatGPT/Claude/Gemini all expose view/edit/delete; ChatGPT (Oct 2025) added **roll-back to prior versions**. Claude is notably transparent about *when* memory influences a reply.
- **Contradiction detection** (EMERGING): 4-type typology (MemConflict, arXiv:2605.20926) — temporal-update (supersede), contradictory (keep both timestamped), condition-dependent (scope), distractor (drop). Zep handles structurally via validity intervals.
- **Confidence/evidence thresholds before storing** (EMERGING): don't promote a single observation to the stable profile — require confirmation, ≥N independent observations, or a min confidence score. Separate episodic (session) from semantic (cross-session).
**Sycophancy is the personalization-specific landmine** (ESTABLISHED). RLHF reward models prefer agreement over truth; **providing user memory/profile context measurably INCREASES sycophancy** (arXiv:2509.12517), worse across multi-turn pressure (SYCON Bench). Documented OpenAI insider admission: showing users their own profile assessments triggered backlash, and one response was to make the model *less honest about the user*. Best app-level mitigations: **explicit anti-sycophancy system instruction** ("maintain your position unless given new evidence/logic" — up to 64% improvement on Turn-of-Flip) and **"ask, don't tell"** framing. Users *prefer* sycophancy when offered the choice → must be a built-in default, not a toggle.
### 4. Pitfalls
- **Drift / model collapse** (ESTABLISHED, Shumailov et al., Nature 2024): a profile learning from its *own* AI output narrows and degrades. **Never "replace"; always "accumulate" with provenance.** ACUTE FOR US: a content engine that learns voice from posts *it drafted* collapses the voice toward its own priors. Voice must learn from *actual published, human-edited* output only — the existing `voice-scrubber` already encodes this ("gold standard = approved editions, not the English post corpus").
- **Staleness** (ESTABLISHED, STALE benchmark arXiv:2605.06527): *all* tested models+frameworks (incl. Zep, Mem0) fail to notice a valid memory went obsolete. RankSquire 50K-session prod test: 93.4% at deploy → 58.2% at month 3 (41% stale). Mitigate: timestamp every write, surface at retrieval, RAG live sources for fast-changing facts.
- **Privacy / memory poisoning** (ESTABLISHED, OWASP **ASI06**): MINJA (NeurIPS 2025) — 98.2% injection success from query-only access; ZombieAgent (Jan 2026) — zero-click persistent injection via ChatGPT Connectors. Persistent poisoning is *worse* than ordinary prompt injection (attack/execution temporally decoupled). **Local-first/plain-file shrinks the blast radius** (no remote exfil endpoint; git = audit/rollback).
- **Lock-in** (ESTABLISHED): proprietary embedding stores can't export; plain-text/Markdown + git is the answer ("swap the model, feed it the same Markdown").
- **Frozen past self** (ESTABLISHED): profile over-represents who you were, pulls present output backward. Mitigate: time-weighted retrieval + periodic user-authored "who am I now" anchor that deprecates old inferences.
- **Context rot / over-retrieval** (ESTABLISHED, Chroma report Jul 2025, all 18 models): more retrieved memory → *lower* accuracy; coherent context can hurt more than shuffled. Strict top-k budgets, multi-signal retrieval, pruning.
- **Maintenance graveyard** (ESTABLISHED): 82% abandon second brains within 6 months; ">30 min/week upkeep predicts failure." Retrieval-first, not archive-first. FOR US: the evolution loop must be automated and zero-curation-by-default, curation available but never required.
## PART B — ARCHITECTURE RECOMMENDATION FOR LINKEDIN STUDIO
### B.0 What we already have (the silos)
Five per-user stores under `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/`, all surviving reinstall, all routed through one tested seam (`hooks/scripts/data-root.mjs` + TS twin `storage.ts:getDataRoot`):
1. **voice-samples/** — authentic samples + chronicle-voice-drift-log (style layer)
2. **trends/trends.json** — deterministic, provenance-bearing, dedup-by-title+URL
3. **specifics-bank/specifics-bank.json** — typed lived-specifics (`number|named-case|what-broke|contrarian|mind-change`), verification-tracked; already self-described as "the first concrete tributary of the broader second-brain architecture"
4. **analytics/** — exports, per-post, weekly/monthly reports, content-history
5. **drafts/, plans/, profile/user-profile.md, examples/, templates/, audience-insights/**
The seam is the most valuable thing we own — the SOTA "route all memory I/O through one resolver" pattern is **already built and tested**. We unify + add an evolution loop, not start from zero.
### B.1 The shape: a thin Markdown hub over typed tributaries (NOT a vector/graph DB)
**Recommendation: plain-Markdown+JSON two-layer "second brain," not Pinecone/Neo4j.** The portable, auditable, drift-resistant, zero-new-infra SOTA is plain-text + git (Manus, AGENTS.md, "markdown files are all you need"), augmented with the retrieval-triad + temporal-validity ideas implemented as deterministic JSON/MD. A heavyweight store would be wrong here: it adds infra, breaks portability, and **Claude IS the semantic-retrieval engine** (it reads files in-context). Keep silos as files; add a hub that points at them + a loop that maintains them.
Two layers, physically separated (episodic/semantic split):
```
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/
brain/
profile.md # SEMANTIC, two-layer: §static + §dynamic; each fact carries
# value · first_seen · last_seen · evidence_count ·
# provenance(human|published|ai-draft) · status(active|superseded)
index.md # the HUB / Map-of-Content: one-screen pointer to every tributary
# + freshness flags. The "memory AND ops center" front page.
operations.md # PLANS/IDEAS: goals, bets, parked ideas, the "who I am now"
# restatement anchor (frozen-past-self guard lives here)
journal/ # EPISODIC, append-only: YYYY-MM-session.md — raw, never edited,
# the source the loop consolidates FROM
voice-samples/ # tributary (style) — stays its own store
specifics-bank/ # tributary (raw material) — stays its own store
trends/ # tributary (external signal) — stays its own store
analytics/ # tributary (performance) — stays its own store
ingest/ # NEW: raw real LinkedIn output lands here
inbox/ # drop-zone: manual import OR connector output
published/ # processed, provenance=published — the GOLD signal
```
**Tributaries vs hub — explicit call:**
- **Stay separate (tributaries), feed the hub via PAG summaries:** voice-samples, specifics-bank, trends, analytics, ingest. Each has a *different dedupe key + lifecycle* (trends: title+URL, fast decay; specifics-bank: human-only, near-permanent; analytics: numeric, immutable). Forcing one schema would destroy those distinctions. The hub holds a distilled pointer-summary of each.
- **Fold INTO the hub:** the flat `profile/user-profile.md` → two-layer `brain/profile.md`. Scattered `audience-insights/`, `examples/` → dynamic-layer sections or tributary-summaries referenced from `index.md`. The **operations/plans center** is genuinely new.
### B.2 The evolution loop (the compounding mechanism)
Run on a cadence (reuse the existing session-start staleness-nudge surface) as a **sleep-time consolidation pass**, not inline cost:
1. **Capture (episodic, auto):** real-signal sessions append to `brain/journal/`. The **ingest seam** (`ingest/inbox/`) takes the user's actual published posts — *manual import first* (LinkedIn has no clean self-serve content API; analytics is already manual CSV per CLAUDE.md S16 — so manual is the honest default; any connector is additive). Published output tagged `provenance=published`.
2. **Consolidate (reflection):** an Opus agent reads new journal + published + tributary deltas, proposes profile updates as a **diff**, never a silent overwrite. Each candidate carries `evidence_count`, `provenance`, timestamps.
3. **Promote with thresholds (anti-overfit):** a candidate enters the **static** layer only on explicit confirmation OR ≥N independent observations; one-offs stay in the **dynamic** layer (which decays). Single weird sessions never reshape identity.
4. **Reconcile contradictions:** classify (temporal → supersede + keep old `status: superseded` w/ date; contradictory → both timestamped; condition-dependent → scope; distractor → drop). Zep's validity-interval idea in plain frontmatter.
5. **Decay + freshness:** every fact has `last_seen`; retrieval weights recent over old; session-start nudge flags facts not refreshed in T days (extend the existing trend-staleness nudge to the whole brain).
6. **Curate (optional, never required):** diff is presentable; user can confirm/edit/delete/roll-back (git = free rollback). Default path needs zero curation — graveyard guard.
### B.3 Keeping it user-aligned + drift-resistant (the hard part, concrete)
- **Provenance-weighted voice learning (THE critical guard):** profile/voice **learn from `provenance=published` only, never `ai-draft`.** Model-collapse guard for our exact risk. Generalizes `voice-scrubber`'s "gold standard = approved editions" to a system-wide invariant: AI-generated content is *never* a training signal for who the user is.
- **Anti-sycophancy as built-in default, not toggle:** consolidation + content agents carry an explicit instruction to treat the profile as evidence to *test*, not flatter ("you mentioned X — still true?"), and surface profile-grounded claims as checkable, not asserted. Users prefer sycophancy when given the choice → don't offer it.
- **Frozen-past-self guard:** `operations.md` holds a periodic user-authored "where I'm headed now" anchor that deprecates older inferences; time-weighted retrieval; the dynamic layer decays so the present dominates.
- **Confidence before commitment:** episodic ≠ semantic; promotion needs evidence density. Highest-leverage anti-overfit mechanism, and cheap (a counter + a threshold).
### B.4 What's genuinely hard (flagged honestly)
1. **Voice fidelity to a private individual is limited even at SOTA** (arXiv:2509.14543) — the real win is grounded *content* (specifics-bank) over mimicked *style*. Manage expectations.
2. **Detecting that a high-relevance fact went stale is UNSOLVED field-wide** (STALE: everyone fails). We can timestamp + nudge; we cannot reliably auto-detect "user changed jobs." Keep humans in that loop.
3. **Contradiction classification** (real change vs context-scoped vs noise) is EMERGING + error-prone; bias toward "keep both, timestamped, surface the conflict" over silent overwrite.
4. **The connector** — no clean LinkedIn self-serve content/analytics API. Build the **manual ingest seam as the contract**; any future connector is a tributary into the same inbox, never a dependency.
5. **Sycophancy is structural** — app-level mitigations reduce, not eliminate; personalization context increases it, so counter-pressure it every time the profile is used.
6. **Avoiding the graveyard is a PRODUCT problem** — the loop must compound value with ~zero required curation or it dies in 6 months like 82% of second brains.
### B.5 Why this fits us
- Reuses the one tested seam (`getDataRoot`) — SOTA single-resolver is done.
- Plain-text + git — portable, auditable, survives reinstall, no new infra; Claude is the semantic-retrieval engine reading files in-context (no vector DB).
- specifics-bank + trends already prove the tributary pattern (deterministic, typed, provenance-bearing, dedup-disciplined) — extend, don't reinvent.
- The four SOTA ideas that port cleanly to files: episodic/semantic split, provenance-weighting, evidence-threshold promotion, temporal-validity frontmatter. The vector/graph machinery does NOT port and we don't need it.
**Bottom line (one line):** Build a thin two-layer Markdown hub (`brain/`: semantic `profile.md` + episodic `journal/` + ops `operations.md` + `index.md` MOC) over the existing typed tributaries, fed by a provenance-tagged ingest seam, maintained by a sleep-time consolidation loop with evidence-threshold promotion, temporal-validity reconciliation, and a built-in anti-sycophancy/anti-collapse stance — every silo stays a tributary except the flat `user-profile.md`, which folds in.
## KEY SOURCES (anchors)
- CoALA arXiv:2309.02427 · Generative Agents arXiv:2304.03442 · MemGPT/Letta arXiv:2310.08560 · Letta sleep-time arXiv:2504.13171
- Mem0 arXiv:2504.19413 · Zep/Graphiti arXiv:2501.13956 · A-MEM (NeurIPS 2025) arXiv:2502.12110 · Cognee arXiv:2505.24478
- PersonaMem (COLM 2025) arXiv:2504.14225 · PURE (SIGIR 2025) · style-imitation arXiv:2509.14543 · MemConflict arXiv:2605.20926
- interaction-context-sycophancy arXiv:2509.12517 · OpenAI Dreaming (openai.com/index/chatgpt-memory-dreaming) · Anthropic sycophancy research
- Shumailov model collapse (Nature 2024, s41586-024-07566-y) · STALE benchmark arXiv:2605.06527 · MINJA (NeurIPS 2025) arXiv:2503.03704 · OWASP ASI06 · Chroma Context Rot (trychroma.com/research/context-rot) · Lost-in-the-Middle (Liu et al. 2023)
- PKM: buildingasecondbrain.com · zettelkasten.de · notes.andymatuschak.org · Mem/Pinecone engineering · Tana · Obsidian Smart Connections
- Portability: AGENTS.md (Linux Foundation) · "markdown files are all you need" (voxos.ai, dev.to)
> NOTE on sources: a few arXiv ids above carry 2025–2026 future-dated identifiers as returned by the research thread. Treat the *patterns* as the durable takeaway; re-verify any specific citation before quoting it externally (verification duty).

110
docs/second-brain/research/silo-inventory.md Normal file
View file

@ -0,0 +1,110 @@
# Research — Per-User Data Silo Inventory (LinkedIn Studio, 2026-06-23)
> Research thread `silo-inventory`, 2026-06-23. Every silo read from the actual files (TS types/store code, .mjs writers, templates, command/agent prose), not inferred. Research INPUT to `../architecture.md`. Preserved so nothing is lost. (file:line refs were accurate at capture time — re-confirm before editing those lines.)
The unifying seam is the **data-path convention** (`hooks/scripts/data-root.mjs:24` `getDataRoot()` / `scripts/analytics/src/utils/storage.ts getDataRoot`), default `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/`. Twin implementations (Node `.mjs` for hooks, TS for analytics/trends/specifics CLIs) because runtimes differ — `data-root.mjs:1-8` documents the must-stay-in-sync contract. The **M0 migration manifest** (`hooks/scripts/migrate-data.mjs:30-54`) is the single source of truth for what moves out of the plugin tree on session-start.
## The silos
### A. State file (`## Recent Posts`, streaks, milestones, first-hour, outreach)
- **Location:** `~/.claude/linkedin-studio.local.md` — **NOT** under the data-root seam. Own resolver (`state-updater.mjs:11-12`, `STATE_FILE` override; mirrored in `data-root.mjs:30 getStateFile()`). The data root "mirrors" this file (`data-path-convention.md:15`) but they are two separate locations.
- **Format:** YAML frontmatter + markdown sections.
- **Schema** (`config/state-file.template.md`): frontmatter — `last_post_date`, `first_post_date`, `last_post_topic`, `posts_this_week`, `weekly_goal`, `current_streak`, `longest_streak`, `current_week`, `last_import_date/week`, `follower_count/target`, `target_date`, `monthly_growth[]`, `projected_10k_date`, `growth_rate_needed`, `next_planned_topic`, `pending_5x5x5`, `content_series_active`, `last_firsthour_date`, `firsthour_active`, `last_outreach_date`, `outreach_active`, `expertise_areas[5]`. Sections — `## Recent Posts` (`- [YYYY-MM-DD] "hook…" (chars) - topic`), `## Session Notes`, `## Planned Content`, `## Milestone Log` (`- [YYYY-MM] count (+delta)`), `## First-Hour Plans`, `## Outreach Pipeline`.
- **Writers:** `state-updater.mjs` — `updatePostTracking` (:55), `updateFollowerCount` (:181), `recordFirstHourPlan` (:235), `recordOutreachContact` (:294), `pruneContentHistory` (:138). From Stop hook + `/firsthour`, `/outreach`, `/calendar` publish. **Readers:** SessionStart hook, `/analyze`, `/report`, content-planner.
- **Lifecycle:** `## Recent Posts` auto-pruned at 90d on session-start (`state-updater.mjs:138`). Streak resets if post gap >2d (:94). Week rollover resets `posts_this_week` via `week-rollover.mjs`.
- **Survives reinstall:** YES (in `~/.claude/`, not plugin tree).
### B. Trends store
- **Location:** `${data}/trends/trends.json` — through seam (`scripts/trends/src/store.ts:180 defaultStorePath()`).
- **Format:** JSON (TS-managed), `{ schemaVersion:1, trends:TrendRecord[] }`.
- **Schema** (`trends/src/types.ts:26`): `TrendRecord` = `id` (sha256[:12] of normalized title+url = dedupe key), `title`, `url`, `source`, `capturedAt` (ISO), `topics[]`, `summary?`.
- **Writers:** CLI `addTrend` (`store.ts:111`), populated by trend-spotter. **Readers:** `queryByTopic` (:137), `history` (:156), `newestCaptureDate` (:171).
- **Lifecycle:** Dedup on title+url; re-capture **unions topics**, keeps first-sight provenance. `newestCaptureDate` drives the SessionStart trend-freshness nudge (B-S3). No prune.
- **Survives reinstall:** YES.
### C. Lived-specifics bank
- **Location:** `${data}/specifics-bank/specifics-bank.json` — through seam (`scripts/specifics-bank/src/bank.ts:150 defaultBankPath()`).
- **Format:** JSON (TS-managed), `{ schemaVersion:1, specifics:Specific[] }`.
- **Schema** (`specifics-bank/src/types.ts:46`): `Specific` = `id` (sha256[:12] of normalized content = dedupe key), `type` (`number`|`named-case`|`what-broke`|`contrarian`|`mind-change`|`other`), `content` (operator's own words, never AI-generated — invariant `types.ts:13`), `topicTags[]`, `provenance{capturedAt,source}` (source = edition id/command/"manual"), `verification` (`verified`|`unverified`|`n/a`), `status` (`active`|`archived`).
- **Writers:** CLI `addSpecific` (`bank.ts:102`), from `/newsletter` Step 1.5 elicitation. **Readers:** `queryByTopic` (:128) — excludes archived. Companions `binding.ts` (Step 2.5 gate), `kilder.ts`.
- **Lifecycle:** Dedup on content; re-capture unions tags. Archive = soft-hide. No time-prune. `verification` gates numbers.
- **Survives reinstall:** YES.
### D. Analytics store
- **Location root:** `${data}/analytics/` via `getAnalyticsRoot()` (`scripts/analytics/src/utils/storage.ts:67`) — through seam; legacy `ANALYTICS_ROOT` honored (deprecated). Subdirs: `exports/`, `posts/`, `weekly-reports/`, `monthly-reports/`, `ab-tests/`.
- **Format:** JSON (TS-managed) posts/reports; CSV input; markdown ab-tests.
- **Schema** (`analytics/src/models/types.ts`): `PostAnalytics` = `id`, `title`, `publishedDate`, `metrics`, `importedAt`, `exportSource` — file `posts/YYYY-MM-DD-<8char>.json` inside `AnalyticsBatch` (`batchId`, `importedAt`, `exportFilename`, `dateRange{from,to}`, `postCount`, `posts[]`). `PostMetrics` = `impressions`, `reactions`, `comments`, `shares`, `clicks`, `engagementRate`, `saves?` — **no `dwell`** (intentional). `WeeklyReport`(`weekly-reports/YYYY-WXX.json`)/`MonthlyReport`(`monthly-reports/YYYY-MM.json`) = `summary{totals incl. totalSaves?, avgEngagementRate, avgImpressionsPerPost}`, `topPerformers[]`, `underperformers[]`/`byWeek[]`, `trends{…percentChange}`, `alerts[]`.
- **`saves`** (`parseOptionalCount`, `csv-parser.ts:71`): blank/non-numeric/negative→`undefined` (unknown, never 0); genuine `0` kept; **not folded into `engagementRate`**; surfaced only when ≥1 post carries it.
- **Writers:** analytics CLI (`cli.ts`) — `import`→`posts/`, `report`→weekly/monthly; ab-tests written by `/ab-test` prose. **Readers:** CLI report/trends/heatmap subcommands; analytics-interpreter agent (no agent re-reads posts JSON directly — report path is CLI-internal).
- **Lifecycle:** Posts deduped by post id on import (latest wins). No time-prune. Reports overwrite by key.
- **Survives reinstall:** YES.
### E. Drafts queue
- **Location:** `${data}/drafts/queue.json` — through seam (`queue-manager.mjs:10` `join(getDataRoot('drafts'),'queue.json')`). Migrated `migrate-data.mjs:32`.
- **Format:** JSON, `{ version:1, queue:[] }`.
- **Schema** (`queue-manager.mjs:63-79`, the actual writer): `id`, `draft_path`, `scheduled_date`, `scheduled_time`, `pillar`, `format`, `hook_preview`, `character_count`, `status` (`'scheduled'` on add; →`posted`/`archived`/`skipped`/`draft` via `queueUpdateStatus`), `created_at`.
- **Writers:** `queueAdd`/`queueUpdateStatus` (:63,:82) — `/batch`, `/calendar`. **Readers:** `queueRead`/`Today`/`Upcoming`/`Overdue` (:41-99), SessionStart, `/calendar`, `/pipeline`, ical-generator.
- **Lifecycle:** No prune. Dedup-by-id on add (:64). No week reset.
- **Survives reinstall:** YES.
### F. Draft files on disk (week / carousel / multiplatform / repurposed)
- **Location (all through seam, confirmed in prose):** `${data}/drafts/week-*/…md` (migrated `migrate-data.mjs:46,129`), `${data}/drafts/carousel-YYYYMMDD-SLUG/slide-N.png` (`commands/carousel.md:133,147`), `${data}/drafts/multiplatform/[platform]-[slug].md` (`commands/multiplatform.md:118`), `${data}/drafts/repurposed/` (`agents/content-repurposer.md:597`). Only `week-*` is in the migration manifest; carousel/multiplatform/repurposed are born external (command/agent prose writes straight to the data root).
- **Format:** Markdown (+ YAML frontmatter) / PNG.
- **Writers:** creation commands + content-repurposer. **Readers:** `/calendar`, `/pipeline`, repurpose/carousel/video.
- **Lifecycle:** No automatic prune — files accumulate; manual cleanup. Dedup by filename/dir slug only.
- **Survives reinstall:** YES.
### G. Content plans
- **Location:** `${data}/plans/<weekly|monthly>-plan-*.md` — through seam (`agents/content-planner.md:7,40,464`). Born external (not in manifest).
- **Format:** Markdown (+ frontmatter: `plan_date`, `generated`, `status`, `posts_planned`, pillars, `content_mix`).
- **Writers:** content-planner agent (:464, can also create Linear issues). **Readers:** content-planner (scans prior plans to avoid repetition :40), SessionStart, `/calendar`.
- **Lifecycle:** No prune; planner does content-level dedup vs prior plans. **Survives reinstall:** YES.
### H. Voice profile + voice-drift log
- **Location:** `${data}/voice-samples/authentic-voice-samples.md` (external canonical; in-plugin **placeholder seed** at `assets/voice-samples/authentic-voice-samples.md` with `<!-- VOICE_PLACEHOLDER -->` sentinel; migrated MOVE dropping `.local`, `migrate-data.mjs:31`). Plus `${data}/voice-samples/chronicle-voice-drift-log.md` (external-only, no seed; written/read by `voice-scrubber`).
- **Resolution:** `personalization-score.mjs` `pick(dataRel,pluginRel)` (:30-32) prefers external, falls back to seed. Voice scores 25pts iff line-count>50 AND no sentinel.
- **Format:** Markdown sections (Core Voice Characteristics, Do's/Don'ts, Signature Phrases, Vocabulary, Language Guidelines, Update Log, auto-appended "Collected Post Samples").
- **Writers:** `/setup`/`/onboarding` (overwrite placeholder), voice-trainer (appends samples), voice-scrubber (drift log). **Readers:** `personalization-score.mjs`, `user-prompt-context.mjs`, content commands.
- **Survives reinstall:** external instance YES; seed NO (it's the seed).
### I. User profile
- **Location:** `${data}/profile/user-profile.md` (migrated MOVE from `config/user-profile.local.md`, `migrate-data.mjs:34`; template seed `config/user-profile.template.md`). Scored 20pts when <3 `[Your ` placeholders remain (`personalization-score.mjs:54`).
- **Format/schema:** Markdown — Personalization Settings (identity, expertise areas, audience, goals), Voice & Style, Strategic Context, **Research Tooling** (which MCPs available), Asset Utilization Preferences.
- **Writers:** `/setup`/`/onboarding` + manual edit. **Readers:** `personalization-score.mjs`, `user-prompt-context.mjs`, commands route MCPs from it. **Survives reinstall:** external YES.
### J. COPY-class scaffolds (examples / audience-insights / templates)
Four files ship a tracked seed + canonical external instance (`migrate-data.mjs:49-54 COPY_FILES`; external never clobbered): `${data}/examples/high-engagement-posts.md`, `${data}/audience-insights/{demographics,engagement-patterns}.md`, `${data}/templates/my-post-templates.md`. Markdown. Written by user/voice-trainer; read by `personalization-score.mjs` (pick-resolved) + content commands. **Survives reinstall:** external YES, seed NO.
### K. content-history.md — vestigial second surface
- **Location:** `${data}/analytics/content-history.md` (migrated MOVE, `migrate-data.mjs:33`; template `config/content-history.template.md`). Markdown table (`Date|Hook|Topic|Format|Words|Chars|Source`).
- **Status:** No hook/agent code actively writes it — the live content-history surface is the **state file's `## Recent Posts`** (silo A). Migrated-but-superseded scaffold.
- **Survives reinstall:** YES (but inactive).
### L. iCal export
`${data}/drafts/week-WXX/schedule.ics`, generated from queue by `ical-generator.mjs` (`/batch`). RFC-5545. Transient/regenerated, not a knowledge store. Survives reinstall: YES.
## Synthesis
### Already per-user + reinstall-surviving (the "done" set)
A–L all route through the seam or `~/.claude/`. **There is essentially no in-plugin/tmp data left to migrate** — M0 already did it. Only `/tmp/linkedin-hooks/session-active` (a session marker, not data) and the in-plugin **seeds** stay in-tree by design.
### What is NOT yet unified (the real gap)
Per-user but **siloed and heterogeneous** — three storage idioms, two roots, no cross-references:
1. **Two roots, not one.** State file at `~/.claude/linkedin-studio.local.md` (own resolver, `state-updater.mjs:12`); everything else under `${data}/`. Unification must decide whether the state file folds in or stays a sibling.
2. **Two writer disciplines.** TS-managed JSON (trends, specifics, analytics) with `schemaVersion` + sha256-id dedup + tested pure functions — **vs.** regex markdown-section mutation (state file, `state-updater.mjs`) — **vs.** schema-less free-form markdown written by command/agent prose (drafts, plans, voice, scaffolds).
### Natural overlaps / relationships
- **Posts triple-counted.** A published post lands in three non-referencing places: state `## Recent Posts` (A, live), `analytics/content-history.md` (K, dead), `analytics/posts/*.json` (D, from CSV). Same event, three schemas, no shared id.
- **specifics ↔ posts ↔ analytics.** A `Specific` (number/named-case) → a post (F) → a `PostAnalytics` row (D). Nothing links the raw material to the post it seeded or to that post's measured performance — "which raw material performs" is unanswerable today.
- **trends ↔ plans ↔ drafts.** trend-spotter captures `TrendRecord` (B); content-planner plan (G) should cite it; draft (F) executes it — all free-text, no id linkage.
- **voice ↔ content-history ↔ analytics.** voice-trainer learns from "Collected Post Samples"; the high performers that should train it live in analytics (D) + examples (J) — three disconnected views of "your good posts."
- **provenance is the common latent field.** trends (`source`+`capturedAt`), specifics (`provenance{source,capturedAt}`), analytics (`importedAt`,`exportSource`), queue (`created_at`) each independently reinvented capture-provenance under different names.
### The 5 hardest unification problems
1. **Schema-idiom mismatch.** Folding regex-mutated markdown (state file) and prose-written free-form markdown (drafts/plans/voice) into the typed `schemaVersion`+dedup discipline of the JSON stores is the central rewrite. JSON stores (B/C/D) are already second-brain-shaped; the markdown silos are not.
2. **Identity / dedup across silos.** Each store has its own id — sha256(title+url) trends, sha256(content) specifics, hash(title+date) analytics, slug ids queue. **No stable cross-silo entity id**, so the post→specific→trend→analytics graph can't be assembled. Needs one canonical id minted at creation and threaded through.
3. **Two roots + legacy-alias window.** Reconcile `getStateFile()` with `getDataRoot()` while three deprecated aliases (`ANALYTICS_ROOT`/`STATE_FILE`/`PLUGIN_ROOT`) are honored "for one minor version" (`data-path-convention.md:17`) and **two twin implementations** (`data-root.mjs` ⇄ `storage.ts`) must stay byte-compatible.
4. **Triple-recorded post + dead content-history.** K is a migrated-but-unwritten duplicate of A, and neither links to D. Unification must pick one canonical post record and retire/back-fill the other two — a data-reconciliation problem, not just schema.
5. **Provenance normalization + lifecycle divergence.** Each silo reinvented provenance under different field names; retention is inconsistent — 90-day prune (state Recent Posts only), soft-archive (specifics), topic-union-merge (trends/specifics), overwrite-by-key (reports), **no prune at all** for drafts/plans/queue/voice/trends. Needs one provenance shape and one coherent retention policy.
Key files: `hooks/scripts/data-root.mjs`, `hooks/scripts/migrate-data.mjs:30-54`, `hooks/scripts/state-updater.mjs`, `hooks/scripts/queue-manager.mjs:10-91`, `scripts/trends/src/{types,store}.ts`, `scripts/specifics-bank/src/{types,bank}.ts`, `scripts/analytics/src/{utils/storage,models/types,parsers/csv-parser}.ts`, `hooks/scripts/personalization-score.mjs:30-152`, `config/state-file.template.md`, `references/data-path-convention.md`.