open/linkedin-studio
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions
linkedin-studio/docs/second-brain/research/connector-egress.md
Kjell Tore Guttormsen d3199eb997 docs(linkedin-studio): second-brain architecture (approved) + 3 research reports 
Persist the operator-approved second-brain architecture design and the three
parallel research threads that fed it, so nothing is lost before build:

- architecture.md — approved design: thin Markdown `brain/` hub (two-layer
  profile.md + episodic journal/ + operations.md + index.md MOC) over the
  existing typed tributaries (voice/specifics/trends/analytics), a
  provenance-tagged ingest/ seam, a sleep-time consolidation loop with
  evidence-threshold promotion + temporal-validity + anti-sycophancy/
  anti-collapse invariants. Build sequence SB-S0..S4.
- research/connector-egress.md — LinkedIn data egress reality (EU/EEA DMA
  portability API = auto for content; analytics manual CSV; no scraping).
- research/secondbrain-sota.md — 2026 second-brain / AI-memory SOTA synthesis.
- research/silo-inventory.md — faithful inventory of the 12 existing per-user
  silos + the 5 hardest unification problems.

Boundary confirmed: engine -> plugin (domain-general), user data -> data dir,
cockpit -> Maskinrommet. Design phase, no code yet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RigJBiRFNtFZKCz21qNbQ4
2026-06-23 12:55:17 +02:00

65 lines
10 KiB
Markdown
Raw Blame History

# Research — LinkedIn content/data egress for an individual creator (2026-06-23)
> Research thread `connector-recon`, 2026-06-23. Verification-duty: every technical claim backed by an official source/URL; unverifiable items marked "IKKE VERIFISERT". This is research INPUT to `../architecture.md` — preserved verbatim so nothing is lost.
**Bottom line:** For ONE creator pulling their OWN content out of LinkedIn there is exactly one realistic *automated* path, and it is geofenced to **EU/EEA + Switzerland**: the **DMA Member Data Portability API** (self-serve, scope `r_dma_portability_self_serve`). Outside that region the honest answer is **manual native export** for content + **manual analytics CSV download** for per-post metrics. Full hands-off automation is NOT available outside the EU/EEA. Engagement/impression analytics are NOT in any portability/export *content* path — they live only in the separate manual analytics CSV.
## 1. Path-by-path table
| # | Path | What it captures (your OWN data) | Verdict | Source |
|---|------|----------------------------------|---------|--------|
| 1a | **DMA Member Data Portability API — Member Snapshot** (`r_dma_portability_self_serve` / `r_dma_portability_member`) | **Full post text** (`MEMBER_SHARE_INFO`: all shares/re-shares incl. date, URL, shared comments, visibility), **articles** (`ARTICLES`), profile, connections, comments (`ALL_COMMENTS`), reactions you gave (`ALL_LIKES`), reposts, messages — ~60+ domains. **Historical** data included. **NO impression/reach/engagement-received analytics domain exists.** | **AUTO** — but **EU/EEA + Switzerland members ONLY** | learn.microsoft.com/en-us/linkedin/dma/member-data-portability/member-data-portability-member/?view=li-dma-data-portability-2026-05 ; …/shared/snapshot-domain?view=li-dma-data-portability-2026-05 |
| 1b | **DMA Member Data Portability API — Member Changelog** | Real-time stream of your interactions (posts created, comments, reactions) from consent onward; **queryable only for the last 28 days** | **AUTO** (rolling 28-day window) — EU/EEA + CH only | (same member page, Changelog section) |
| 1c | **Marketing API / Community Management API** | Org/Page-level posting + analytics; NOT a self-serve member-content path | **RISKY/BLOCKED for individuals** — LinkedIn Partner Program approval (legal entity, use-case review, weeks/months) | learn.microsoft.com/en-us/linkedin/marketing/community-management/community-management-overview?view=li-lms-2026-05 ; connectsafely.ai/articles/linkedin-api-complete-guide-2026 |
| 2 | **Native "Get a copy of your data"** (Settings → Data Privacy) | ZIP of **CSV + JSON + HTML**: profile, connections, messages, comments, **posts** (Shares), **articles** (HTML folder). Basic categories "within minutes"; **larger archive ~24h** (72h download window). **Saved-items file is URLs+dates only.** **No analytics.** | **MANUAL** | linkedin.com/help/linkedin/answer/a1339364/downloading-your-account-data |
| 3a | **Native newsletter content** | **NO native RSS feed.** Subscription = email/notification per edition only. Edition text lives as articles. | **MANUAL** | linkedin.com/help/linkedin/answer/a517925 |
| 3b | **Newsletter via your own subscription email** | Each new edition arrives as an **email to you** → ingest your own mailbox / email-to-RSS | **MANUAL (clean)** | linkedin.com/help/linkedin/answer/a517925 |
| 3c | **Third-party newsletter→RSS scrapers** (rss.app, linkedin-newsletter-rss) | Scrapes public newsletter/article URL into RSS | **RISKY** (third-party scraping) | rss.app/rss-feed/linkedin ; github.com/chrisns/linkedin-newsletter-rss |
| 4a | **Per-post analytics CSV export** (native dashboard "Export") | **Per-post** impressions, unique viewers, clicks, reactions, comments, shares, engagement rate. **Manual, ~1-year range cap.** Profile views NOT exportable. **ONLY path to your engagement metrics.** | **MANUAL** | tryordinal.com/blog/how-to-export-analytics-from-linkedin-to-excel-or-a-csv ; linkedin.com/help/linkedin/answer/a551206/ |
| 4b | **Zapier / Make** | Trigger on *your* new posts (publish-side); thin/no historical pull of post bodies+analytics; bound by LinkedIn's official connector scopes | **MANUAL-ish / limited** | connectsafely.ai/articles/is-linkedin-automation-safe-tos-scraping-guide-2026 |
| 4c | **Phantombuster / scrapers / browser extensions** | Can extract post text + visible engagement counts | **RISKY** — breaches User Agreement (prohibited automation); active ban-wave enforcement | linkedin.com/help/linkedin/answer/a1341387 ; connectsafely.ai/articles/is-linkedin-automation-safe-tos-scraping-guide-2026 |
## 2. Single cleanest realistic ingest path for ONE user (own data)
Splits by region, and even at best it is TWO streams, not one:
**EU/EEA or Switzerland → near-full automation for CONTENT:**
- **Member Data Portability API, self-serve.** Create a dev app **using LinkedIn's mandated default Company Page** ("Member Data Portability (Member) Default Company" — you must NOT create your own page), request the *Member Data Portability (Member)* product, generate a token via the OAuth Token Generator with scope `r_dma_portability_self_serve`, consent, then call **Member Snapshot** (`GET /rest/memberSnapshotData?q=criteria`) for historical full post text (`MEMBER_SHARE_INFO`) + `ARTICLES`, and poll **Member Changelog** for the rolling 28-day live stream. Token valid up to **1 year** before re-consent. Genuinely automatable for a second-brain content store.
- **BUT analytics still come from the manual CSV (4a)** — portability exposes reactions/comments *you made*, not impressions/reach *received*.
**Outside EU/EEA/CH → manual is the honest answer:**
- Content: native "Get a copy of your data" (larger archive, ~24h, CSV+JSON+HTML) — paths 2 + 3.
- Analytics: native per-post analytics CSV (4a).
- Newsletter: own subscription email (3b) or article HTML from the archive.
**Is full automation possible?** Only inside EU/EEA/CH, and only for content (not received-analytics). Outside, **manual export/paste is the correct, ToS-safe answer.** Scraper/extension/Phantombuster routes are a **User Agreement breach** with active enforcement (LinkedIn's reported Q1-2026: 23.5M automated sessions flagged) — do not bake into the plugin's ingest.
## 3. What's uncertain / changing (date: 2026-06-23)
- **Geofence may widen.** Portability API is DMA-driven, explicitly "**at this time**… EU/EEA + Switzerland." Re-check the MS Learn moniker (`li-dma-data-portability-2026-05`) each quarter. IKKE VERIFISERT that it will ever cover non-EU members.
- **Analytics-in-portability gap.** No Snapshot domain carries received impressions/reach/engagement-rate (`ALL_LIKES`/`ALL_COMMENTS` = actions *you* took). Treat "portability gives you your analytics" as FALSE until a future domain appears. Native analytics CSV stays the only metrics source.
- **Newsletter editions in export/snapshot.** `ARTICLES` covers authored articles; whether every *newsletter edition* is reliably captured as an article vs. needing the email path is **partially uncertain** — native export help does not enumerate newsletter editions. Verify against a real export.
- **Saves / dwell** remain non-exportable (consistent with the plugin's existing S16 note). IKKE VERIFISERT any change.
- **Versioned API drift.** Snapshot uses `LinkedIn-Version: YYYYMM`; domain list/schemas change month-to-month. Pin a version, re-validate.
## 4. Sources (accessed 2026-06-23)
**Official / authoritative:**
- Member Data Portability (Member), MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/dma/member-data-portability/member-data-portability-member/?view=li-dma-data-portability-2026-05
- Member Snapshot API, MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/dma/member-data-portability/shared/member-snapshot-api?view=li-dma-data-portability-2026-05
- Member Snapshot Domain list, MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/dma/member-data-portability/shared/snapshot-domain?view=li-dma-data-portability-2026-05
- Member portability APIs, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a6214075
- Community Management Overview, MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/marketing/community-management/community-management-overview?view=li-lms-2026-05
- Download your account data, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a1339364/downloading-your-account-data
- Export your Page analytics report, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a551206/
- Manage a newsletter, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a517925
- Prohibited software and extensions, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a1341387
- LinkedIn DMA Portability API Terms: https://www.linkedin.com/legal/l/portability-api-terms
**Secondary (corroborating, not authoritative):**
- connectsafely — LinkedIn API 2026: https://connectsafely.ai/articles/linkedin-api-complete-guide-2026
- connectsafely — automation/ToS 2026: https://connectsafely.ai/articles/is-linkedin-automation-safe-tos-scraping-guide-2026
- tryordinal — Export LinkedIn analytics (Jan 2026): https://www.tryordinal.com/blog/how-to-export-analytics-from-linkedin-to-excel-or-a-csv
- rss.app LinkedIn / chrisns/linkedin-newsletter-rss: https://rss.app/rss-feed/linkedin · https://github.com/chrisns/linkedin-newsletter-rss
**Verification log (key claims → source):** Portability API exists + self-serve scope + EU/EEA+CH-only + 1-yr token → MS Learn member page. Full post text + articles + historical, no analytics domain → MS Learn snapshot-domain list (verbatim `MEMBER_SHARE_INFO`, `ARTICLES`, `ALL_LIKES`, `ALL_COMMENTS`; no impressions domain present). Changelog 28-day window → MS Learn member page. Native export = CSV+JSON+HTML, articles as HTML, ~24h, saved-items URLs-only → LinkedIn Help + corroborating guides. No native newsletter RSS → LinkedIn Help newsletter page (silent on RSS) + third-party scrapers fill the gap. Per-post analytics CSV manual, ~1yr cap, only metrics source → LinkedIn Help + tryordinal. Scraping = User Agreement breach w/ enforcement → LinkedIn Prohibited Software + connectsafely.
Reference in a new issue View git blame Copy permalink