From 63506f7d5cde3b9862eac50ffc98fd9b2ac48921 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Sat, 25 Jul 2026 15:56:04 +0200 Subject: [PATCH] =?UTF-8?q?feat(linkedin-studio):=20N16=20=E2=80=94=20out-?= =?UTF-8?q?of-network-andel=20+=20patterns-oppdatering=20+=20boundary-map?= =?UTF-8?q?=20[skip-docs]?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reach-splitten (in/out-of-network) er native i LinkedIns post-analytics siden juni 2026, men vises som PROSENT og finnes ikke i CSV-eksporten. Planen antok to manuelle antall; verifiseringen viste prosent, så modellen er ett felt — outOfNetworkPct — og in-network er komplementet. - parseOptionalPercent: egen parser, ikke parseOptionalCount. Komma er desimal (36,5 -> 36.5, aldri 365), og verdi >100 avvises: i én kolonne kan ikke et absolutt antall skilles fra en andel, så svaret er unknown, ikke en gjetning. Blank/ikke-numerisk/negativ -> unknown; ekte 0 beholdes. - Ett lagret halvpart, kryssjekket: In-network godtas og lagres som komplement; et transkribert par som ikke summerer til ~100 (±1 avrunding) forkastes som unknown i stedet for å bli halvveis trodd. - weightedOutOfNetworkPct: impressions-vektet roll-up (avgOutOfNetworkPct, uke + måned). Flatt snitt lar en 50-visnings-post slå en på 10 000; poster uten avlesning ekskluderes, og null vekt gir undefined — aldri 0, aldri NaN. - Reach inngår ALDRI i engagementRate (distribusjon != engasjement). Rapporten leser den som akvisisjon (ut) vs resonans (inn), og sier «ikke ført for denne perioden» framfor å estimere. En reach-innsikt går inn i N15s do-next-kanal. - Step 7c (A2-F11): rapporten tilbyr diff mot brukerens engagement-patterns.md med eksplisitt go — aldri stille skriving, aldri inn i den shippede malen. - Boundary-map (E#9): dwell eksplisitt umålbar, saves partner-gated, reach native men CSV-eksport uverifisert. - Reach-frie importer er byte-identiske med før, på skjerm og på disk. TDD: rødt bevist først (10 feilende), analytics 119 -> 144 tester, tsc ren. test-runner 232 -> 247 (Section 16w, gulv 213 -> 228). Alle suiter grønne. CHANGELOG: N15-oppføringen manglet og er backfilt sammen med N16. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QxvWAjte7vPcF79QeSRvRJ --- CHANGELOG.md | 18 ++ CLAUDE.md | 1 + README.md | 5 +- assets/analytics/README.md | 37 ++++ commands/import.md | 12 +- commands/report.md | 75 ++++++- scripts/analytics/src/cli.ts | 38 +++- scripts/analytics/src/models/types.ts | 25 +++ scripts/analytics/src/parsers/csv-parser.ts | 71 +++++++ scripts/analytics/src/reports/monthly.ts | 6 +- scripts/analytics/src/reports/weekly.ts | 10 +- scripts/analytics/src/utils/stats.ts | 37 ++++ scripts/analytics/tests/csv-parser.test.ts | 186 ++++++++++++++++++ .../tests/fixtures/reach-both-export.csv | 4 + .../tests/fixtures/reach-edge-export.csv | 5 + .../analytics/tests/fixtures/reach-export.csv | 5 + .../fixtures/reach-in-network-export.csv | 3 + scripts/analytics/tests/monthly.test.ts | 25 +++ scripts/analytics/tests/stats.test.ts | 57 ++++++ scripts/analytics/tests/weekly.test.ts | 51 +++++ scripts/test-runner.sh | 183 ++++++++++++++++- 21 files changed, 841 insertions(+), 13 deletions(-) create mode 100644 scripts/analytics/tests/fixtures/reach-both-export.csv create mode 100644 scripts/analytics/tests/fixtures/reach-edge-export.csv create mode 100644 scripts/analytics/tests/fixtures/reach-export.csv create mode 100644 scripts/analytics/tests/fixtures/reach-in-network-export.csv diff --git a/CHANGELOG.md b/CHANGELOG.md index 7401412..0b87587 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **`references/fold-in-loop.md` (A2-F9)** — the capture → classify → promote → enforce loop documented **domain-generally and in-tree**, so an adopter with no external writing contract has the whole loop (that absence "is not a gap" — the same mirror rule `editorial-reviewer` uses). Names the other four accumulation silos (specifics-bank, brain, voice drift log, A/B learnings) that Step 11 surfaces together, because a retro that empties one queue teaches the operator the others are empty too. references 28 → 29. - Suites: test-runner 197 → 217 (Section 16u: 18 unconditional greps + a non-vacuity self-test on the capture predicate; the phase-coverage sweep now spans **17** phases; anti-erosion floor 179 → 198). All nine suites green. +### Added — the do-next contract: a measurement changes the next piece (N15 / A2-F10, A2-F12, A2-F13, A2-F14) + +- **`recordDoNext()` in `state-updater.mjs`** — one channel from measurement to creation. `/linkedin:report`, `/linkedin:analyze`, an `/linkedin:ab-test` **Adopt** verdict and the `post-feedback-monitor`'s 48h learning each persist 2–3 directives (imperative + evidence pointer, tagged with their source) to `## Do-Next Directives` in the state file; `post`/`quick`/`batch`/`create` Step 0 and `newsletter` Step 1 read them before drafting. Before this every analytics surface ended in chat (`grep do.next` across the repo returned 0) and the next drafting session — a fresh context — never saw any of it. +- **Lifetime is replace-by-source + a 60-day age floor** — each writer supersedes only its own rows, so a new weekly report never evicts an A/B Adopt decision, and the section stays a live steering signal instead of a growing backlog. Deterministic, no AI in the write; `--record-do-next` exposes it as a CLI verb. +- **Previous-edition calibration (A2-F14)** — `newsletter` Step 1 matches the previous edition to its measured numbers through the queue id (`drafts/queue.json` → `analytics/posts/*.json` by publish date, disambiguated on the hook preview), with an explicit **honest-miss** branch: an unmeasured edition is stated as unmeasured, never estimated. +- test-runner 213 → 232 (Section 16v: 15 unconditional do-next greps + self-test; anti-erosion floor 198 → 213). Two real bugs found by the tests themselves: a section regex missing `/m` (old rows were never superseded or pruned) and fixtures dated 7 months before their own `today`, which had made the replace-by-source test falsely green. + +### Added — measure-truth: reach split + honest boundary map (N16 / D-3, A2-F11, E#9) + +- **Out-of-network reach as an optional manual column (D-3)** — LinkedIn split a post's impressions into in-network and out-of-network in June 2026 (post analytics → Discovery, under the impressions count): the first native number that says whether a post **acquired** audience or only resonated with the audience it already had. It is shown as a **percentage split** and is **absent from the CSV export** (whether it will ever be exported is unverified), so it follows the saves pattern — add an `Out-of-network` (or `In-network`) column with the percentage read off that panel and `/linkedin:import` ingests it. +- **A separate percent parser, deliberately not the count parser** — `parseOptionalPercent()` reads a comma as a **decimal** mark (`36,5` → 36.5; `parseOptionalCount`'s US-thousands rule would have said 365) and **refuses any value above 100**: in one column an absolute impression count and a share are indistinguishable, so the honest answer is unknown rather than a guess. Blank, non-numeric and negative stay unknown; a genuine `0` is kept (nothing left the network). +- **One stored half, cross-checked** — only `outOfNetworkPct` is stored, because the two halves describe one split and keeping both would let a record contradict itself. An `In-network` cell is stored as its complement, and a transcribed **pair that does not sum to ~100** (one point of rounding slack, since the UI rounds each half independently) is **discarded as unknown** instead of half-trusted. +- **Impressions-weighted roll-ups** — `weightedOutOfNetworkPct()` feeds an optional `avgOutOfNetworkPct` on the weekly and monthly summaries. Weighted, because a flat mean lets a 50-impression post at 90 % outvote a 10,000-impression one at 20 %; posts without a reading are excluded rather than folded in as 0; zero total weight yields `undefined`, never 0 or NaN. Reach-free imports stay byte-identical to pre-N16 output, on screen and on disk. +- **Not part of the engagement rate** — reach is distribution, not engagement, so it never enters `engagementRate` (which stays comparable to every historical import). `/linkedin:report` renders the split as **acquisition (out) vs resonance (in)**, states coverage honestly, and says "not entered for this period" rather than estimating an unentered split. A reach reading routes into the N15 do-next channel so it shapes the next post instead of being admired in the report. +- **Operator-gated baseline updates (A2-F11)** — new `/linkedin:report` **Step 7c**: the report now offers a diff against the user's `audience-insights/engagement-patterns.md` baseline (the file `analytics-interpreter`, `strategy-advisor` and `/linkedin:audit` treat as tracked truth, and which nothing in the pipeline used to write, so it went stale silently). Old → new with the evidence, an explicit go via `AskUserQuestion`, changes labelled *provisional* when they rest on 1–2 posts, an append to the section's update log — and never a silent write, never into the plugin's shipped template. +- **Boundary map brought current (E#9)** — the README boundaries now state all three honestly: dwell **explicitly unmeasurable** (internal to LinkedIn, no count to transcribe, no API), saves partner-gated (`POST_SAVE` on `/memberCreatorPostAnalytics` from v202604), and reach native-but-not-exported with export status **unverified**. +- analytics suite 119 → 144 (25 new tests: percent contract, either-half resolution, split cross-check, weighted roll-up, backward-compat); test-runner 232 → 247 (Section 16w: 15 unconditional greps + self-test; anti-erosion floor 213 → 228); `tsc` clean. + ### Built feedback (awaiting consumer-side proof) - **MR-F9** (demand-sweep «innenfra og ut») — built; the (a)/(b)/(c) evidence gate is a runtime demonstration, proven consumer-side (plugin agents don't resolve in the dev repo). diff --git a/CLAUDE.md b/CLAUDE.md index 3e0c87d..12082a8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -18,6 +18,7 @@ Full-spectrum LinkedIn content engine — short-form feed posts, carousels, vide - **Contract-gate:** `scripts/contract-gate/` (TypeScript, needs `tsx` + `npm install`) — deterministic §B/§C1 rule-gate on the full draft (`/linkedin:newsletter` Step 4.5, before the AI sweeps); ratifies `rules.ts` against the edition's §E-manifest, then gates with BLOCK/WARN - **Fold-in loop:** capture → classify → promote → enforce, documented domain-generally in `references/fold-in-loop.md`. The four `/linkedin:newsletter` steps where operator judgment enters the text (2.5, 3a, 5.5, 6.5) append corrections to `articles.NN.foldIns[]` (`decision: "pending"`); **Step 11 (retro)** empties the queue with an explicit operator JA/NEI — mechanical → an atomic contract-gate promotion (only counts when `--ratify` is green), judgment → the operator's opt-in language-rules file (`${DATA}/language-rules/.md`, template `config/language-rules.template.md`), which Step 4 reads while drafting (prevent) and `language-reviewer` reads at Step 6.5 (catch). Rejected rows are kept, never deleted. Step 11 also reports effort from the **measured** `phaseLog` and asks one friction question whose answer goes back to the operator — the plugin never writes their register - **Do-next contract:** `recordDoNext()` in `hooks/scripts/state-updater.mjs` — the one channel from measurement to creation. `/linkedin:report`, `/linkedin:analyze`, an `/linkedin:ab-test` **Adopt** verdict and the `post-feedback-monitor`'s 48h learning each persist 2–3 directives (imperative + evidence pointer, tagged with their source) to `## Do-Next Directives` in the state file; the create surfaces (`post`/`quick`/`batch`/`create` Step 0, `newsletter` Step 1) read them before drafting, so a recommendation changes the next piece instead of dying in chat. Lifetime is **replace-by-source + a 60-day age floor**: each writer supersedes only its own rows, so the section is a live steering signal, never a backlog. `newsletter` Step 1 additionally calibrates on the **previous edition's measured numbers** via the queue-id match (`drafts/queue.json` → `analytics/posts/*.json` by publish date), with an explicit honest-miss branch — an unmeasured edition is stated as such, never estimated +- **Reach split (N16):** the in-network/out-of-network share is native in post analytics (Discovery, under impressions; global rollout from June 2026) but shown as **percentages** and **absent from the CSV** (export status unverified) — so it follows the saves pattern as an optional manual column. `parseOptionalPercent()` is a separate parser, not `parseOptionalCount`: a comma is a **decimal** mark (`36,5` → 36.5, never 365) and a value **>100 is refused** (a count and a share are indistinguishable in one column). Only the out-of-network half is stored (`outOfNetworkPct`) — in-network is its complement, and a transcribed pair that does not sum to ~100 (±1 rounding) is **discarded as unknown** rather than half-trusted. Roll-ups (`avgOutOfNetworkPct`, weekly + monthly) are **impressions-weighted** via `weightedOutOfNetworkPct()` and stay `undefined` on zero weight — never 0, never NaN. Reach is **not** in `engagementRate` (distribution ≠ engagement); reports read it as acquisition (out) vs resonance (in), and an unentered split is stated as such, never estimated. `/linkedin:report` Step 7c additionally offers an **operator-gated** diff against the per-user `audience-insights/engagement-patterns.md` baseline (A2-F11 — explicit go, never a silent write, never into the shipped template) - **Analytics metrics (S16):** parsed CSV columns + an optional, manually-entered `saves` count (count-only in native LinkedIn analytics since ~Sept 2025, no CSV export; the Marketing API exposes `POST_SAVE` on `/memberCreatorPostAnalytics` from v202604, but access is partner-gated — so manual entry remains the right UX). `parseOptionalCount()`: blank / non-numeric / negative → `undefined` (`unknown`, never 0), a genuine `0` is kept; saves surfaced per-post + as `totalSaves`, but **not** folded into `engagementRate`. `dwell` stays **explicitly unmeasurable** (internal to LinkedIn, no export/API). All analytics I/O routes through the `getAnalyticsRoot()` seam (M0 per-user data-dir). ## Hooks diff --git a/README.md b/README.md index 0db5600..46d593a 100644 --- a/README.md +++ b/README.md @@ -195,13 +195,14 @@ All 30 commands use colon notation: `/linkedin:post`, `/linkedin:quick`, etc. Th --- -## Boundaries (as of 2026-05) +## Boundaries (as of 2026-07) LinkedIn Studio is honest about what it can and cannot do for a **personal profile**: - **Post-level analytics via API** — exists, but is **partner-gated** (a vetted Community Management API app + a verified organization + a Page). Not self-serve for a solo profile, so the practical floor is the **CSV export** you drop into `/linkedin:import`. Per-post **saves** are visible in *native* post analytics (count-only, since ~Sept 2025) but absent from the CSV and have no self-serve API — the tool does **not** auto-track them, but you can add a `Saves` column to the CSV manually and `/linkedin:import` ingests it (omit it and saves stays *unknown*, never 0, never folded into engagement rate). - **Auto-publish** — technically possible via the `w_member_social` scope, so this is a **design choice**, not an API limit: the OAuth/token overhead plus LinkedIn's terms on automated posting make copy-to-clipboard + you-paste the right default. The calendar's "publish" action marks a post **you** posted as published — it never posts on your behalf. -- **Dwell time** — internal to LinkedIn for organic posts; not exportable, not measured. +- **In-network vs out-of-network reach** — LinkedIn shows this split natively in post analytics (**Discovery**, under the impressions count; global rollout from June 2026), but as **percentages** and **not in the CSV export** — whether it will ever be exported is *unverified*. So it follows the saves pattern: add an `Out-of-network` (or `In-network`) column with the percentage you read off that panel and `/linkedin:import` ingests it — omit it and the share stays *unknown*, never 0. It is **not** folded into the engagement rate: a high out-of-network share means the post reached **new** people (acquisition), high in-network engagement means it landed with the audience you already have (resonance). Roll-ups are impressions-weighted. +- **Dwell time** — internal to LinkedIn for organic posts; not exportable, no count to transcribe, no API. **Explicitly unmeasurable** — the plugin does not estimate it. - **Also not covered:** real-time/streaming analytics, automated engagement (ToS), profile editing via API, and team/multi-user workflows. The plugin generates recommendations and drafts; you apply them. --- diff --git a/assets/analytics/README.md b/assets/analytics/README.md index 16047b7..d0b6be7 100644 --- a/assets/analytics/README.md +++ b/assets/analytics/README.md @@ -28,6 +28,43 @@ comparable to older imports). Saves is the strongest organic engagement signal, so the reports surface it as its own line. **Dwell time stays unmeasurable** — it is internal to LinkedIn for organic posts, with no count to transcribe. +### Optional: add the out-of-network reach share (manual) + +LinkedIn splits a post's impressions into **in-network** (people who already +follow or are connected to you) and **out-of-network** (people who found it +through recommendations, reshares or search). It sits in your post analytics +under **Discovery**, beneath the impressions count — a global rollout that began +in June 2026 — and it is shown as **percentages**, not as two counts. It is +**not** in the CSV export, and whether it will ever be exported is unverified. + +To track it, add an `Out-of-network` column and type the percentage you read off +that panel. The `%` sign is optional: + +``` +"Content","Date","Impressions","Reactions","Comments","Shares","Clicks","Out-of-network" +"My post...",2026-03-10,5000,100,30,15,200,37% +``` + +Details worth knowing: + +- **Either half works.** An `In-network` column is accepted instead and stored as + its complement (`In-network 63%` → out-of-network 37). Only the out-of-network + share is kept, because the two halves describe one split — storing both would + let a record contradict itself. +- **Both halves are cross-checked.** If you transcribe both and they do not sum + to ~100 (one point of rounding slack allowed), the reading is discarded as + **unknown** rather than guessing which cell was misread. +- **Percent, not a count.** A value above 100 is refused (`unknown`): in one + column an absolute impression count and a share are indistinguishable, so the + importer will not guess. A comma is read as a decimal mark (`36,5` → 36.5%). +- **Unknown is never 0.** A missing column, a blank cell, or a non-numeric cell + leaves the share unknown. A genuine `0` is kept — nothing left your network. +- **Not part of the engagement rate.** Reach is a distribution signal, not + engagement: a high out-of-network share means the post **acquired new + audience**, while high in-network engagement means it **deepened the audience + you already have**. Roll-ups are impressions-weighted, so a small post with a + high share cannot outvote a large one. + ## Directory Structure ``` diff --git a/commands/import.md b/commands/import.md index 3a892e2..670dd97 100644 --- a/commands/import.md +++ b/commands/import.md @@ -23,11 +23,17 @@ You are a LinkedIn analytics data import assistant. Guide the user through impor For data format details and directory structure, see `assets/analytics/README.md`. -> **Why CSV (as of 2026-05).** Post-level analytics via LinkedIn's API is +> **Why CSV (as of 2026-07).** Post-level analytics via LinkedIn's API is > partner-gated (vetted Community Management app + verified org + Page) and **not > self-serve** for a personal profile, so the CSV export is the practical floor. > Saves are visible in native post analytics (count-only) but have no self-serve -> API pull; dwell is internal-only for organic posts. See the README boundaries. +> API pull; the **in-network/out-of-network reach split** is visible natively too +> (Discovery, under impressions — rolling out globally since June 2026) but as +> percentages, and is absent from the CSV, with export status unverified; dwell is +> internal-only for organic posts. Saves and the reach share are therefore +> **optional manual columns** the user adds to the CSV (`Saves`, +> `Out-of-network` / `In-network`) — see `assets/analytics/README.md` for the +> entry rules, and the README boundaries for the full data boundary. ## Step 1: Check for CSV Files in Exports Directory @@ -135,6 +141,7 @@ The CLI prints (see `cli.ts` `handleImport`): - `Date range:` — earliest to latest post in the batch - `Batch ID:` and `Saved to: posts/` — the batch file written - `Saves entered:` — only when the CSV carried a `Saves` column (manual entry) +- `Reach entered:` — only when the CSV carried an `Out-of-network` (or `In-network`) column; reports the impressions-weighted share **and how many posts carried a reading**, so partial coverage is visible instead of reading as a whole-batch number - An anomaly block — either `Immediate alerts detected:` with 🔴/⚠️/ℹ️ spike/drop lines, or `No anomalies detected in imported data.` **Surface the CLI's output to the user** — for example: @@ -146,6 +153,7 @@ Posts imported: 42 Date range: 2025-12-01 to 2026-01-29 Saved to: posts/2025-12-01-batch-a1b2c3d4.json Saves entered: 1,204 across 18 post(s) (manual) +Reach entered: 31.4% out-of-network across 12 post(s) (manual, impressions-weighted) Immediate alerts detected: ───────────────────────────────────── diff --git a/commands/report.md b/commands/report.md index 25915c8..f8d314c 100644 --- a/commands/report.md +++ b/commands/report.md @@ -141,7 +141,7 @@ cat ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/weekly-repo The report contains: - **week**: ISO week identifier - **generatedAt**: ISO timestamp when the report was generated -- **summary**: Totals and averages — `totalPosts`, `totalImpressions`, `totalReactions`, `totalComments`, `totalShares`, `totalClicks`, optional `totalSaves` (manual entry only), `avgEngagementRate`, `avgImpressionsPerPost` +- **summary**: Totals and averages — `totalPosts`, `totalImpressions`, `totalReactions`, `totalComments`, `totalShares`, `totalClicks`, optional `totalSaves` (manual entry only), optional `avgOutOfNetworkPct` (manual entry only — the impressions-weighted out-of-network share), `avgEngagementRate`, `avgImpressionsPerPost` - **topPerformers** / **underperformers**: Best / weakest posts by engagement rate - **trends**: Week-over-week change — `impressionsTrend`, `engagementTrend`, `comparedTo`, `percentChange` - **alerts**: Anomalies and significant events (intra-week + week-over-week) @@ -272,6 +272,24 @@ Format the data into a readable report using this template: **Posts published:** {summary.totalPosts} **Engagement rate:** {summary.avgEngagementRate}% +### 🌐 Reach Split (only when `summary.avgOutOfNetworkPct` is present) + +| Half | Share of impressions | Reads as | +|------|----------------------|----------| +| Out-of-network | {summary.avgOutOfNetworkPct}% | Acquisition — the post reached people who do not follow you | +| In-network | {100 - summary.avgOutOfNetworkPct}% | Resonance — it landed with the audience you already have | + +State the coverage honestly: the share is impressions-weighted across **only the +posts that carried a manual reading**. If the field is absent, say the split is +**not entered for this period** — never estimate it, and never print 0%. + +Read the two halves against the week's goal, not as a score: +- **Out-of-network up, engagement rate flat/down** — the algorithm distributed it to + strangers who did not act. Reach without resonance; the hook traveled, the body did not. +- **In-network high, out-of-network low** — deepening, not growing. Fine for a trust play, + a problem if the week's goal was audience growth. +- **Both up** — the topic has pull beyond your circle. This is the pattern worth repeating. + ### 🏆 Top Performers **Most Impressions:** @@ -362,6 +380,17 @@ writeState(content => recordDoNext(content, { " ``` +**A reach reading belongs in this channel too.** When `avgOutOfNetworkPct` is present, the +split is a directive about *what to write next*, not a number to admire — so persist it here +rather than leaving it in the report: + +- Out-of-network high on one topic → `'Write the next post on — it travels past your network'`, evidence `'weekly 2026-W23: 58% out-of-network vs 31% weighted average'` +- Out-of-network up but engagement rate down → `'Keep the hook, rebuild the body — strangers saw it and did not act'` +- Out-of-network low across the week → `'Next post targets a problem non-followers search for, not an in-circle reference'` + +Never persist a reach directive when the field is absent — an unentered split is unknown, and a +directive with no measured evidence pointer is exactly what this channel exists to prevent. + **Lifetime (do not hand-manage it):** `recordDoNext` replaces this source's own previous rows on every write and drops any row older than 60 days, so the section stays a live steering signal rather than a growing backlog. Directives from other sources (`ab-test`, `analyze`, @@ -370,6 +399,50 @@ rather than a growing backlog. Directives from other sources (`ab-test`, `analyz Then confirm in one line what was persisted, e.g. "2 do-next directives written to state — the next post/newsletter will read them at Step 0." +## Step 7c: Offer a baseline update to `engagement-patterns.md` (operator-gated) + +The do-next directives above steer the **next** piece. This step maintains the **standing** +baseline every other surface reads: `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/audience-insights/engagement-patterns.md` +— the file `analytics-interpreter`, `strategy-advisor` and `/linkedin:audit` treat as the user's +tracked truth about their own audience. It goes stale silently: nothing in the pipeline writes it, +so it keeps asserting last quarter's best time and topics while the reports say otherwise. + +So after a report, **offer a diff** — never write it silently: + +1. **Read the baseline file.** If it is missing or still all `[Day]`/`[Topic]` placeholders, say so + and offer to seed it from this report instead of diffing. +2. **Compare only what this report actually measured** — best posting windows, top-performing + topics, format performance, and (when entered) the reach split. Leave every other section alone. +3. **Present each proposed change as old → new with the evidence**, e.g.: + +``` +Proposed baseline updates (from weekly 2026-W23, n=11 posts): + +1. Primary posting window + old: Tuesday at 12:00 — avg 1,200 impressions + new: Tuesday at 08:00 — avg 2,050 impressions (4 posts over 3 weeks) + +2. Top-performing topic + old: (empty) + new: agent orchestration — avg 4.1% engagement, 58% out-of-network (3 posts) + +3. Format performance + old: carousel — untracked + new: carousel — 3x shares vs text (1 post — thin, flagged as provisional) +``` + +4. **Ask for an explicit go** with `AskUserQuestion` — options: **Apply all** · **Apply + selected** · **Skip**. Skipping is a normal outcome; a thin sample is a reason to skip. +5. **Only on an explicit go, edit the file** — change the affected lines and append to that + section's `**Update Log:**` with the date and what the number came from. Never rewrite + sections the report did not measure, and never delete the user's own annotations. +6. If a proposed change rests on **1–2 posts**, label it *provisional* in the diff and say plainly + that it is not yet a pattern. Do not let a single post rewrite a baseline. + +Write only to the per-user data dir path above — never to the plugin's shipped seed +(`engagement-patterns-template.md` and its placeholder sibling), which is the template +the data dir was initialized from, not the user's data. + ## Step 8: Offer Deep Dive Options After presenting the report, ask: diff --git a/scripts/analytics/src/cli.ts b/scripts/analytics/src/cli.ts index 2024815..8566024 100644 --- a/scripts/analytics/src/cli.ts +++ b/scripts/analytics/src/cli.ts @@ -6,7 +6,7 @@ import { loadAllPosts, } from "./utils/storage.js"; import { detectAlerts } from "./utils/alerts.js"; -import { mean, standardDeviation } from "./utils/stats.js"; +import { mean, standardDeviation, weightedOutOfNetworkPct } from "./utils/stats.js"; import { generateWeeklyReport, getCurrentISOWeek } from "./reports/weekly.js"; import { generateHeatmap } from "./reports/heatmap.js"; import { generateMonthlyReport } from "./reports/monthly.js"; @@ -30,6 +30,19 @@ function savesSuffix(saves?: number): string { return saves !== undefined ? ` | ${saves.toLocaleString()} saves` : ""; } +/** + * Per-post reach suffix. Empty string when the post carries no manual + * out-of-network share, so reach-free output stays identical to the pre-N16 CLI. + */ +function reachSuffix(outOfNetworkPct?: number): string { + return outOfNetworkPct !== undefined ? ` | ${outOfNetworkPct}% out-of-network` : ""; +} + +/** Round to one decimal — keeps the derived in-network half free of float noise. */ +function round1(value: number): number { + return Math.round(value * 10) / 10; +} + function printUsage() { console.log(` LinkedIn Analytics CLI @@ -90,6 +103,15 @@ async function handleImport(root: string, args: string[]) { console.log(`Saves entered: ${totalSaves.toLocaleString()} across ${savesPosts.length} post(s) (manual)`); } + // Same for the reach split when the CSV carried an Out-of-network (or + // In-network) column. Counting the posts that carry a reading makes partial + // coverage visible instead of implying the whole batch was transcribed. + const reachPosts = batch.posts.filter((p) => p.metrics.outOfNetworkPct !== undefined); + if (reachPosts.length > 0) { + const weighted = weightedOutOfNetworkPct(batch.posts); + console.log(`Reach entered: ${weighted}% out-of-network across ${reachPosts.length} post(s) (manual, impressions-weighted)`); + } + // Run alert detection on imported posts const alerts = detectAlerts(batch.posts, "impressions"); @@ -144,6 +166,10 @@ async function handleReport(root: string, args: string[]) { if (report.summary.totalSaves !== undefined) { console.log(`Total saves: ${report.summary.totalSaves.toLocaleString()} (manual entry — top engagement signal)`); } + if (report.summary.avgOutOfNetworkPct !== undefined) { + console.log(`Out-of-network: ${report.summary.avgOutOfNetworkPct}% of impressions (manual entry — acquisition signal)`); + console.log(`In-network: ${round1(100 - report.summary.avgOutOfNetworkPct)}% of impressions (resonance with the audience you have)`); + } console.log(`Avg engagement: ${report.summary.avgEngagementRate.toFixed(2)}%`); console.log(`Avg impressions: ${Math.round(report.summary.avgImpressionsPerPost).toLocaleString()} per post`); console.log(); @@ -154,7 +180,7 @@ async function handleReport(root: string, args: string[]) { for (const post of report.topPerformers.slice(0, 5)) { const title = post.title.length > 50 ? post.title.substring(0, 47) + "..." : post.title; console.log(`• ${title}`); - console.log(` ${post.metrics.impressions.toLocaleString()} impressions | ${post.metrics.engagementRate.toFixed(2)}% engagement${savesSuffix(post.metrics.saves)} | ${post.publishedDate}`); + console.log(` ${post.metrics.impressions.toLocaleString()} impressions | ${post.metrics.engagementRate.toFixed(2)}% engagement${savesSuffix(post.metrics.saves)}${reachSuffix(post.metrics.outOfNetworkPct)} | ${post.publishedDate}`); } console.log(); } @@ -165,7 +191,7 @@ async function handleReport(root: string, args: string[]) { for (const post of report.underperformers.slice(0, 3)) { const title = post.title.length > 50 ? post.title.substring(0, 47) + "..." : post.title; console.log(`• ${title}`); - console.log(` ${post.metrics.impressions.toLocaleString()} impressions | ${post.metrics.engagementRate.toFixed(2)}% engagement${savesSuffix(post.metrics.saves)} | ${post.publishedDate}`); + console.log(` ${post.metrics.impressions.toLocaleString()} impressions | ${post.metrics.engagementRate.toFixed(2)}% engagement${savesSuffix(post.metrics.saves)}${reachSuffix(post.metrics.outOfNetworkPct)} | ${post.publishedDate}`); } console.log(); } @@ -342,6 +368,10 @@ async function handleMonthlyReport(root: string, month: string) { if (s.totalSaves !== undefined) { console.log(`Saves: ${s.totalSaves.toLocaleString()} (manual entry — top engagement signal)`); } + if (s.avgOutOfNetworkPct !== undefined) { + console.log(`Out-of-network: ${s.avgOutOfNetworkPct}% of impressions (manual entry — acquisition signal)`); + console.log(`In-network: ${round1(100 - s.avgOutOfNetworkPct)}% of impressions (resonance with the audience you have)`); + } console.log(); if (report.byWeek.length > 0) { @@ -359,7 +389,7 @@ async function handleMonthlyReport(root: string, month: string) { for (const post of report.topPerformers.slice(0, 5)) { const title = post.title.length > 50 ? post.title.substring(0, 47) + "..." : post.title; console.log(`• ${title}`); - console.log(` ${post.metrics.impressions.toLocaleString()} impressions | ${post.metrics.engagementRate.toFixed(2)}% eng${savesSuffix(post.metrics.saves)} | ${post.publishedDate}`); + console.log(` ${post.metrics.impressions.toLocaleString()} impressions | ${post.metrics.engagementRate.toFixed(2)}% eng${savesSuffix(post.metrics.saves)}${reachSuffix(post.metrics.outOfNetworkPct)} | ${post.publishedDate}`); } console.log(); } diff --git a/scripts/analytics/src/models/types.ts b/scripts/analytics/src/models/types.ts index c9ed27f..62e9629 100644 --- a/scripts/analytics/src/models/types.ts +++ b/scripts/analytics/src/models/types.ts @@ -23,6 +23,24 @@ export interface PostMetrics { // It is deliberately NOT folded into engagementRate (which stays comparable // to historical, saves-free data) — saves is surfaced as its own signal. saves?: number; + // `outOfNetworkPct` is OPTIONAL and manually entered — the share (0–100) of a + // post's impressions that came from people who did NOT follow or connect with + // the author. LinkedIn shows the in-network/out-of-network split natively in + // post analytics (Discovery section, under Impressions; progressive global + // rollout from June 2026) as a PERCENTAGE split, and does NOT put it in the + // CSV export — so the ingest is a percent cell the user adds to the CSV, read + // off that panel (see csv-parser.ts). + // + // Only the out-of-network half is stored: in-network is its complement by + // definition (they describe one split), so keeping both would allow a + // self-contradicting record. A missing column, a blank cell, a non-numeric + // cell, a negative, or a value above 100 stays undefined — "unknown", never + // coerced to 0; a genuine 0 is kept as 0 (nothing left the network). + // + // NOT folded into engagementRate: reach is a distribution signal, not + // engagement. High out-of-network = the post acquired new audience; high + // in-network engagement = it deepened the existing one. + outOfNetworkPct?: number; // NOTE: `dwell` remains absent and unmeasurable. Dwell time is internal to // LinkedIn for organic posts — not exportable, no UI count to transcribe, no // API. Do not fabricate a dwell field or surface. @@ -48,6 +66,10 @@ export interface WeeklyReport { totalShares: number; totalClicks: number; totalSaves?: number; // optional — present only when ≥1 post carries manual saves data + // Optional — present only when ≥1 post carries a manual out-of-network share. + // Impressions-WEIGHTED, so a small post with a high share cannot outvote a + // large one (see weightedOutOfNetworkPct). + avgOutOfNetworkPct?: number; avgEngagementRate: number; avgImpressionsPerPost: number; }; @@ -122,6 +144,9 @@ export interface MonthlyReport { totalShares: number; totalClicks: number; totalSaves?: number; // optional — present only when ≥1 post carries manual saves data + // Optional — present only when ≥1 post carries a manual out-of-network share + // (impressions-weighted; see weightedOutOfNetworkPct). + avgOutOfNetworkPct?: number; avgEngagementRate: number; avgImpressionsPerPost: number; }; diff --git a/scripts/analytics/src/parsers/csv-parser.ts b/scripts/analytics/src/parsers/csv-parser.ts index 229b21c..c61eea2 100644 --- a/scripts/analytics/src/parsers/csv-parser.ts +++ b/scripts/analytics/src/parsers/csv-parser.ts @@ -84,6 +84,64 @@ function parseOptionalCount(value: string): number | undefined { return parsed; } +/** + * Rounding slack, in percentage points, allowed between the two halves of the + * reach split. LinkedIn rounds each half independently for display, so a + * transcribed "63% / 37%" can legitimately sum to 99 or 101. + */ +const REACH_SPLIT_TOLERANCE = 1; + +/** Round to one decimal — the UI reading is itself a rounded percentage. */ +function round1(value: number): number { + return Math.round(value * 10) / 10; +} + +/** + * Parse an OPTIONAL manually-entered PERCENTAGE share (out-of-network reach). + * Distinct from parseOptionalCount in two ways that matter: + * - a share never carries a thousands separator, so a comma is always the + * DECIMAL mark here ("36,5" → 36.5). parseOptionalCount's US-thousands rule + * would read that as 365. + * - a share above 100 is not a share. It is most likely an absolute + * impression count pasted into a percent column, and one column cannot tell + * a count from a share — so the honest answer is unknown, never a guess. + * Otherwise the same contract as the saves field: + * - blank / absent / non-numeric / negative → undefined ("unknown", never 0) + * - a genuine "0" → 0 (nothing left the network) + * - "37%", "37 %", "37" → 37 + */ +function parseOptionalPercent(value: string): number | undefined { + if (!value) return undefined; + const cleaned = value.replace(/"/g, "").replace(/%/g, "").trim(); + if (cleaned === "") return undefined; + + const parsed = Number(cleaned.replace(/,/g, ".")); + if (!Number.isFinite(parsed) || parsed < 0 || parsed > 100) return undefined; + return round1(parsed); +} + +/** + * Reduce whichever halves of the reach split the user transcribed to the single + * stored value: the out-of-network share. + * - out-of-network only → that value + * - in-network only → its complement (they describe one split) + * - both, consistent → the out-of-network reading + * - both, contradictory → undefined. One cell is a misreading and we cannot + * tell which, so the record stays unknown rather than silently trusting one. + */ +function resolveOutOfNetworkPct( + outOfNetwork: number | undefined, + inNetwork: number | undefined +): number | undefined { + if (outOfNetwork !== undefined && inNetwork !== undefined) { + const sum = outOfNetwork + inNetwork; + return Math.abs(sum - 100) <= REACH_SPLIT_TOLERANCE ? outOfNetwork : undefined; + } + if (outOfNetwork !== undefined) return outOfNetwork; + if (inNetwork !== undefined) return round1(100 - inNetwork); + return undefined; +} + /** * Normalizes date to YYYY-MM-DD format * Handles: DD.MM.YYYY, MM/DD/YYYY, YYYY-MM-DD @@ -225,6 +283,19 @@ export function parseLinkedInCSV( metrics.saves = saves; } + // Optional manual-entry reach split: only when the user augmented this CSV + // with an Out-of-network (or In-network) column, read off the native + // Discovery panel in post analytics — LinkedIn does not export it. Either + // half is accepted and reduced to the out-of-network share; anything + // unreadable or self-contradicting stays undefined ("unknown", never 0). + const outOfNetworkPct = resolveOutOfNetworkPct( + parseOptionalPercent(findColumn(record, ["out-of-network", "out of network", "outofnetwork"])), + parseOptionalPercent(findColumn(record, ["in-network", "in network", "innetwork"])) + ); + if (outOfNetworkPct !== undefined) { + metrics.outOfNetworkPct = outOfNetworkPct; + } + return { id: generatePostId(title, date), title, diff --git a/scripts/analytics/src/reports/monthly.ts b/scripts/analytics/src/reports/monthly.ts index db9f965..0e1540b 100644 --- a/scripts/analytics/src/reports/monthly.ts +++ b/scripts/analytics/src/reports/monthly.ts @@ -1,6 +1,6 @@ import type { PostAnalytics, MonthlyReport } from "../models/types.js"; import { loadAllPosts, loadMonthlyReport, saveMonthlyReport } from "../utils/storage.js"; -import { mean } from "../utils/stats.js"; +import { mean, weightedOutOfNetworkPct } from "../utils/stats.js"; import { detectAlerts } from "../utils/alerts.js"; import { getISOWeek } from "./weekly.js"; @@ -34,6 +34,9 @@ export function generateMonthlyReport(root: string, month: string): MonthlyRepor const totalSaves = savesPosts.length > 0 ? savesPosts.reduce((s, p) => s + (p.metrics.saves ?? 0), 0) : undefined; + // Optional out-of-network share: impressions-weighted, present only when ≥1 + // post carries a reading — keeps reach-free months identical to pre-N16 output. + const avgOutOfNetworkPct = weightedOutOfNetworkPct(monthPosts); const avgEngagementRate = totalPosts > 0 ? parseFloat(mean(monthPosts.map(p => p.metrics.engagementRate)).toFixed(2)) : 0; @@ -108,6 +111,7 @@ export function generateMonthlyReport(root: string, month: string): MonthlyRepor totalShares, totalClicks, ...(totalSaves !== undefined ? { totalSaves } : {}), + ...(avgOutOfNetworkPct !== undefined ? { avgOutOfNetworkPct } : {}), avgEngagementRate, avgImpressionsPerPost, }, diff --git a/scripts/analytics/src/reports/weekly.ts b/scripts/analytics/src/reports/weekly.ts index 52c9901..e325202 100644 --- a/scripts/analytics/src/reports/weekly.ts +++ b/scripts/analytics/src/reports/weekly.ts @@ -1,5 +1,5 @@ import type { PostAnalytics, WeeklyReport } from "../models/types.js"; -import { mean, trendDirection, percentChange } from "../utils/stats.js"; +import { mean, trendDirection, percentChange, weightedOutOfNetworkPct } from "../utils/stats.js"; import { detectAlerts, detectWeeklyAlerts } from "../utils/alerts.js"; import { loadAllPosts, loadWeeklyReport, saveWeeklyReport } from "../utils/storage.js"; @@ -173,6 +173,14 @@ export function generateWeeklyReport(analyticsRoot: string, week?: string): Week report.summary.totalSaves = totalSaves; } + // Same contract for the out-of-network share: impressions-weighted, and only + // present when at least one post carried a reading (reach-free reports stay + // byte-identical to pre-N16 output). + const avgOutOfNetworkPct = weightedOutOfNetworkPct(weekPosts); + if (avgOutOfNetworkPct !== undefined) { + report.summary.avgOutOfNetworkPct = avgOutOfNetworkPct; + } + // Calculate averages const engagementRates = weekPosts.map(post => post.metrics.engagementRate); report.summary.avgEngagementRate = mean(engagementRates); diff --git a/scripts/analytics/src/utils/stats.ts b/scripts/analytics/src/utils/stats.ts index a65112e..86e0230 100644 --- a/scripts/analytics/src/utils/stats.ts +++ b/scripts/analytics/src/utils/stats.ts @@ -50,6 +50,43 @@ export function percentChange(current: number, previous: number): number { return ((current - previous) / previous) * 100; } +/** + * Minimal shape needed to weight a reach share — keeps this helper usable from + * both report builders without dragging in the full PostAnalytics record. + */ +interface ReachWeightable { + metrics: { impressions: number; outOfNetworkPct?: number }; +} + +/** + * Roll per-post out-of-network shares up to one number, WEIGHTED by impressions. + * + * The share is a fraction of a post's own impressions, so a flat mean would let + * a 50-impression post at 90% outvote a 10,000-impression post at 20%. Posts + * without a share are excluded entirely — folding them in as 0 would invent + * data that was never entered. + * + * Returns undefined when no post carries a share, or when the posts that do + * carry one have no impressions to weight (a share of zero impressions has no + * meaning; 0 would be a fabricated reading and NaN a bug). + */ +export function weightedOutOfNetworkPct(posts: ReachWeightable[]): number | undefined { + let totalWeight = 0; + let weightedSum = 0; + let sawShare = false; + + for (const post of posts) { + const pct = post.metrics.outOfNetworkPct; + if (pct === undefined) continue; + sawShare = true; + totalWeight += post.metrics.impressions; + weightedSum += post.metrics.impressions * pct; + } + + if (!sawShare || totalWeight <= 0) return undefined; + return Math.round((weightedSum / totalWeight) * 10) / 10; +} + /** * Calculate how many standard deviations a value is from the mean. * Returns 0 if standard deviation is 0. diff --git a/scripts/analytics/tests/csv-parser.test.ts b/scripts/analytics/tests/csv-parser.test.ts index efc8833..054e08d 100644 --- a/scripts/analytics/tests/csv-parser.test.ts +++ b/scripts/analytics/tests/csv-parser.test.ts @@ -191,3 +191,189 @@ describe("Saves (manual-entry, optional)", () => { ); }); }); + +/** + * Out-of-network reach (manual-entry, optional) — N16. + * + * LinkedIn surfaces the in-network/out-of-network split natively in post + * analytics (Discovery section, under Impressions; progressive global rollout + * from June 2026) as a PERCENTAGE split — not as two absolute counts, and not + * in the CSV export. So the ingest is a percent cell the operator transcribes, + * and the stored field is a single share: `outOfNetworkPct`. In-network is its + * complement by definition, so storing both halves would only invite a + * self-contradicting record. + */ +describe("Out-of-network reach (manual-entry, optional)", () => { + it("should parse an Out-of-network percent cell written with a % suffix", () => { + const filePath = join(fixturesDir, "reach-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-export.csv"); + + assert.equal(batch.postCount, 4, "Should have 4 posts"); + assert.equal( + batch.posts[0].metrics.outOfNetworkPct, + 37, + "'37%' must parse to the number 37" + ); + }); + + it("should leave outOfNetworkPct undefined when the cell is blank (unknown != zero)", () => { + const filePath = join(fixturesDir, "reach-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-export.csv"); + + assert.equal( + batch.posts[1].metrics.outOfNetworkPct, + undefined, + "Blank Out-of-network cell must stay undefined, never coerced to 0" + ); + }); + + it("should treat an explicit '0' as a genuine zero share (nothing left the network)", () => { + const filePath = join(fixturesDir, "reach-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-export.csv"); + + assert.equal( + batch.posts[2].metrics.outOfNetworkPct, + 0, + "Explicit '0' is a real reading — must stay 0, not collapse to undefined" + ); + }); + + it("should read a European decimal comma as a decimal, not a thousands separator", () => { + const filePath = join(fixturesDir, "reach-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-export.csv"); + + // A share never carries a thousands separator, so "36,5" is 36.5 percent. + // parseOptionalCount's US-thousands rule would read this as 365 — wrong here. + assert.equal( + batch.posts[3].metrics.outOfNetworkPct, + 36.5, + "'36,5' must parse to 36.5 percent, never 365" + ); + }); + + it("should leave outOfNetworkPct undefined for a standard export with no reach column (backward-compat)", () => { + const filePath = join(fixturesDir, "sample-export.csv"); + const batch = parseLinkedInCSV(filePath, "sample-export.csv"); + + for (const post of batch.posts) { + assert.equal( + post.metrics.outOfNetworkPct, + undefined, + "Existing CSV exports without a reach column must round-trip unchanged" + ); + } + }); + + it("should leave outOfNetworkPct undefined for a non-numeric cell (unknown, never 0)", () => { + const filePath = join(fixturesDir, "reach-edge-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-edge-export.csv"); + + assert.equal( + batch.posts[0].metrics.outOfNetworkPct, + undefined, + "Non-numeric reach cell must stay undefined — never coerced to 0" + ); + }); + + it("should refuse a value above 100 — a count and a share are undecidable in one column", () => { + const filePath = join(fixturesDir, "reach-edge-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-edge-export.csv"); + + // "1234" in a share column is almost certainly an absolute impression count. + // We cannot tell which, so the honest answer is unknown — never a guess. + assert.equal( + batch.posts[1].metrics.outOfNetworkPct, + undefined, + "A share above 100 must stay undefined, never stored as-is" + ); + }); + + it("should leave outOfNetworkPct undefined for a negative cell", () => { + const filePath = join(fixturesDir, "reach-edge-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-edge-export.csv"); + + assert.equal( + batch.posts[2].metrics.outOfNetworkPct, + undefined, + "A negative share is not a real reading — must stay undefined" + ); + }); + + it("should accept exactly 100 as a real reading (the boundary is inclusive)", () => { + const filePath = join(fixturesDir, "reach-edge-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-edge-export.csv"); + + assert.equal( + batch.posts[3].metrics.outOfNetworkPct, + 100, + "100 percent out-of-network is possible and must be kept" + ); + }); + + it("should derive outOfNetworkPct from an In-network column as its complement", () => { + const filePath = join(fixturesDir, "reach-in-network-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-in-network-export.csv"); + + // The operator transcribed the other half of the same split. + assert.equal( + batch.posts[0].metrics.outOfNetworkPct, + 37, + "'In-network 63%' must store out-of-network 37" + ); + assert.equal( + batch.posts[1].metrics.outOfNetworkPct, + undefined, + "A blank In-network cell leaves neither half known" + ); + }); + + it("should keep the out-of-network half when both columns agree", () => { + const filePath = join(fixturesDir, "reach-both-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-both-export.csv"); + + assert.equal( + batch.posts[0].metrics.outOfNetworkPct, + 37, + "63 + 37 = 100 is consistent — keep the out-of-network reading" + ); + }); + + it("should refuse a contradictory split rather than pick a half", () => { + const filePath = join(fixturesDir, "reach-both-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-both-export.csv"); + + // 63 + 20 = 83. One of the two cells is a misreading and we cannot tell + // which, so the record stays unknown instead of silently trusting one. + assert.equal( + batch.posts[1].metrics.outOfNetworkPct, + undefined, + "A split that does not sum to ~100 must stay undefined" + ); + }); + + it("should tolerate one point of rounding slack between the two halves", () => { + const filePath = join(fixturesDir, "reach-both-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-both-export.csv"); + + // 62 + 37 = 99: the UI rounds each half independently, so a one-point gap + // is rounding, not a misreading. + assert.equal( + batch.posts[2].metrics.outOfNetworkPct, + 37, + "A 99 or 101 sum is rounding slack — keep the out-of-network reading" + ); + }); + + it("should NOT fold out-of-network reach into engagementRate", () => { + const filePath = join(fixturesDir, "reach-export.csv"); + const batch = parseLinkedInCSV(filePath, "reach-export.csv"); + + // Row 1: (100+30+15+200)/5000 * 100 = 6.9. Reach is a distribution signal, + // not engagement — it must not touch the rate. + const expectedRate = ((100 + 30 + 15 + 200) / 5000) * 100; + assert.ok( + Math.abs(batch.posts[0].metrics.engagementRate - expectedRate) < 0.01, + `engagementRate should exclude reach (~${expectedRate}), got ${batch.posts[0].metrics.engagementRate}` + ); + }); +}); diff --git a/scripts/analytics/tests/fixtures/reach-both-export.csv b/scripts/analytics/tests/fixtures/reach-both-export.csv new file mode 100644 index 0000000..d57d9bf --- /dev/null +++ b/scripts/analytics/tests/fixtures/reach-both-export.csv @@ -0,0 +1,4 @@ +"Content","Date","Impressions","Reactions","Comments","Shares","Clicks","In-network","Out-of-network" +"Both halves transcribed and they sum to 100 - consistent, out-of-network wins...",2026-02-28,2900,70,22,9,130,63%,37% +"Both halves transcribed but they contradict each other - refuse to guess which one is right...",2026-02-27,2800,70,22,9,130,63%,20% +"Both halves with rounding slack - 62 + 37 = 99 is within the one-point tolerance...",2026-02-26,2700,70,22,9,130,62%,37% diff --git a/scripts/analytics/tests/fixtures/reach-edge-export.csv b/scripts/analytics/tests/fixtures/reach-edge-export.csv new file mode 100644 index 0000000..b92640e --- /dev/null +++ b/scripts/analytics/tests/fixtures/reach-edge-export.csv @@ -0,0 +1,5 @@ +"Content","Date","Impressions","Reactions","Comments","Shares","Clicks","Out-of-network" +"Non-numeric out-of-network cell - the user jotted a note, not a share; stays unknown...",2026-03-06,3500,70,22,9,130,n/a +"Above 100 - most likely an absolute impression count pasted into a share column; undecidable, so unknown...",2026-03-05,3400,70,22,9,130,1234 +"Negative share - not a real reading; stays unknown...",2026-03-04,3300,70,22,9,130,-5 +"Exactly 100 - a real reading: every impression came from outside the network...",2026-03-03,3200,70,22,9,130,100 diff --git a/scripts/analytics/tests/fixtures/reach-export.csv b/scripts/analytics/tests/fixtures/reach-export.csv new file mode 100644 index 0000000..be59e8c --- /dev/null +++ b/scripts/analytics/tests/fixtures/reach-export.csv @@ -0,0 +1,5 @@ +"Content","Date","Impressions","Reactions","Comments","Shares","Clicks","Out-of-network" +"An out-of-network share the user read off the native Discovery panel, with a percent sign...",2026-03-10,5000,100,30,15,200,37% +"A post where the user left the Out-of-network cell blank - unknown, not zero...",2026-03-09,3000,60,20,8,120, +"Explicit zero out-of-network - a real reading: nothing left the network...",2026-03-08,4000,80,25,10,150,0 +"A share written with a European decimal comma - 36,5 percent, not 365...",2026-03-07,2000,40,10,5,60,"36,5" diff --git a/scripts/analytics/tests/fixtures/reach-in-network-export.csv b/scripts/analytics/tests/fixtures/reach-in-network-export.csv new file mode 100644 index 0000000..d22cad5 --- /dev/null +++ b/scripts/analytics/tests/fixtures/reach-in-network-export.csv @@ -0,0 +1,3 @@ +"Content","Date","Impressions","Reactions","Comments","Shares","Clicks","In-network" +"The user transcribed the in-network half of the split instead - out-of-network is the complement...",2026-03-02,3100,70,22,9,130,63% +"In-network blank - neither half known...",2026-03-01,3000,60,20,8,120, diff --git a/scripts/analytics/tests/monthly.test.ts b/scripts/analytics/tests/monthly.test.ts index 9f2d107..5a8423e 100644 --- a/scripts/analytics/tests/monthly.test.ts +++ b/scripts/analytics/tests/monthly.test.ts @@ -104,6 +104,31 @@ describe("generateMonthlyReport", () => { assert.equal(report.summary.totalSaves, undefined); }); + test("rolls out-of-network shares up as an impressions-weighted average", () => { + const withReach = (p: PostAnalytics, outOfNetworkPct: number): PostAnalytics => ({ + ...p, + metrics: { ...p.metrics, outOfNetworkPct }, + }); + const posts: PostAnalytics[] = [ + withReach(createPost("2026-03-03", 10000, 3.0), 20), + withReach(createPost("2026-03-05", 1000, 4.0), 80), + createPost("2026-03-10", 5000, 3.5), // no share entered — must not dilute + ]; + const root = setupTestRoot(posts); + const report = generateMonthlyReport(root, "2026-03"); + assert.equal( + report.summary.avgOutOfNetworkPct, + 25.5, + "Should weight by impressions (25.5), not average the shares flat (50)" + ); + }); + + test("leaves avgOutOfNetworkPct undefined for reach-free months (backward-compat)", () => { + const root = setupTestRoot(marchPosts); + const report = generateMonthlyReport(root, "2026-03"); + assert.equal(report.summary.avgOutOfNetworkPct, undefined); + }); + test("generates weekly breakdown within month", () => { const root = setupTestRoot(marchPosts); const report = generateMonthlyReport(root, "2026-03"); diff --git a/scripts/analytics/tests/stats.test.ts b/scripts/analytics/tests/stats.test.ts index 5fb5a07..a86ceaf 100644 --- a/scripts/analytics/tests/stats.test.ts +++ b/scripts/analytics/tests/stats.test.ts @@ -6,6 +6,7 @@ import { trendDirection, percentChange, deviationsFromMean, + weightedOutOfNetworkPct, } from "../src/utils/stats.js"; describe("stats", () => { @@ -136,4 +137,60 @@ describe("stats", () => { assert.ok(Math.abs(result) < 0.01); }); }); + + /** + * Out-of-network reach aggregate (N16). The share is per-post, so the only + * honest roll-up is impressions-weighted: a 50-impression post at 90% must + * not outvote a 10,000-impression post at 20%. + */ + describe("weightedOutOfNetworkPct", () => { + const post = (impressions: number, outOfNetworkPct?: number) => ({ + metrics: { impressions, outOfNetworkPct }, + }); + + test("should weight each share by that post's impressions", () => { + // (10000*20 + 1000*80) / 11000 = 25.45… — an unweighted mean would say 50. + const result = weightedOutOfNetworkPct([post(10000, 20), post(1000, 80)]); + assert.equal(result, 25.5, "Should be the impressions-weighted share, not the plain mean"); + }); + + test("should exclude posts that carry no share from the weighting", () => { + // The 5000-impression post has no reading; folding it in as 0 would drag + // the answer to 17.5 and invent data that was never entered. + const result = weightedOutOfNetworkPct([ + post(10000, 20), + post(1000, 80), + post(5000, undefined), + ]); + assert.equal(result, 25.5, "Posts without a share must not dilute the aggregate"); + }); + + test("should keep a genuine 0 share in the weighting", () => { + // (1000*0 + 1000*50) / 2000 = 25 — an explicit zero is data, not absence. + const result = weightedOutOfNetworkPct([post(1000, 0), post(1000, 50)]); + assert.equal(result, 25); + }); + + test("should return undefined when no post carries a share", () => { + const result = weightedOutOfNetworkPct([post(1000), post(2000)]); + assert.equal(result, undefined, "Absent data must stay absent, never 0"); + }); + + test("should return undefined for an empty list", () => { + assert.equal(weightedOutOfNetworkPct([]), undefined); + }); + + test("should return undefined when the carrying posts have no impressions", () => { + // A share of zero impressions has no meaning, and the weights sum to 0 — + // returning 0 here would be a fabricated reading, and NaN a bug. + const result = weightedOutOfNetworkPct([post(0, 40)]); + assert.equal(result, undefined, "Zero total weight must yield undefined, never NaN or 0"); + }); + + test("should round to one decimal (the UI reading is itself rounded)", () => { + // (3000*33.3 + 1000*66.7) / 4000 = 41.65 → 41.7 + const result = weightedOutOfNetworkPct([post(3000, 33.3), post(1000, 66.7)]); + assert.equal(result, 41.7); + }); + }); }); diff --git a/scripts/analytics/tests/weekly.test.ts b/scripts/analytics/tests/weekly.test.ts index dbb5d94..b803923 100644 --- a/scripts/analytics/tests/weekly.test.ts +++ b/scripts/analytics/tests/weekly.test.ts @@ -319,6 +319,57 @@ describe("weekly", () => { assert.equal(report.summary.totalSaves, undefined, "Saves-free data must not introduce a totalSaves field"); }); + test("should roll out-of-network shares up as an impressions-weighted average", () => { + tempDir = setupTempDir(); + + const posts: PostAnalytics[] = [ + createTestPost({ + id: "reach1", + publishedDate: "2026-01-12", // 2026-W03 + metrics: { impressions: 10000, reactions: 500, comments: 100, shares: 50, clicks: 200, engagementRate: 8.5, outOfNetworkPct: 20 }, + }), + createTestPost({ + id: "reach2", + publishedDate: "2026-01-13", // 2026-W03 + metrics: { impressions: 1000, reactions: 50, comments: 10, shares: 5, clicks: 20, engagementRate: 8.5, outOfNetworkPct: 80 }, + }), + createTestPost({ + id: "reach3", + publishedDate: "2026-01-14", // 2026-W03 — no share entered; must not dilute. + metrics: { impressions: 5000, reactions: 250, comments: 50, shares: 25, clicks: 100, engagementRate: 8.5 }, + }), + ]; + + saveBatch(tempDir, createTestBatch({ dateRange: { from: "2026-01-12", to: "2026-01-14" }, posts })); + + const report = generateWeeklyReport(tempDir, "2026-W03"); + + assert.equal( + report.summary.avgOutOfNetworkPct, + 25.5, + "Should weight by impressions (25.5), not average the shares flat (50)" + ); + }); + + test("should leave avgOutOfNetworkPct undefined when no post carries a share (backward-compat)", () => { + tempDir = setupTempDir(); + + const posts: PostAnalytics[] = [ + createTestPost({ id: "noreach1", publishedDate: "2026-01-12" }), + createTestPost({ id: "noreach2", publishedDate: "2026-01-13" }), + ]; + + saveBatch(tempDir, createTestBatch({ dateRange: { from: "2026-01-12", to: "2026-01-13" }, posts })); + + const report = generateWeeklyReport(tempDir, "2026-W03"); + + assert.equal( + report.summary.avgOutOfNetworkPct, + undefined, + "Reach-free data must not introduce an avgOutOfNetworkPct field" + ); + }); + test("should identify top performers and underperformers", () => { tempDir = setupTempDir(); diff --git a/scripts/test-runner.sh b/scripts/test-runner.sh index 7b10854..bee3bd7 100755 --- a/scripts/test-runner.sh +++ b/scripts/test-runner.sh @@ -2537,6 +2537,178 @@ fi echo "" +# --- Section 16w: Measure-Truth - Reach + Boundary Map (N16 / D-3, A2-F11, E#9) --- +echo "--- Measure-Truth: Reach + Boundary Map (N16) ---" + +# LinkedIn split a post's impressions into in-network and out-of-network in June +# 2026 - the first native number that says whether a post ACQUIRED audience or +# only resonated with the one already there. It is shown as a PERCENTAGE split in +# the Discovery panel and is absent from the CSV export, so the ingest is a manual +# percent column, exactly like saves. What is worth linting is the honesty of that +# ingest, because every failure mode here is silent: +# (D-3 contract) percent parsing refuses what it cannot know: unknown is never 0, +# a value above 100 is not a share (a count and a share are +# indistinguishable in one column), and a comma is a DECIMAL mark +# here - parseOptionalCount's US-thousands rule would read "36,5" +# as 365. +# (D-3 shape) one stored half, not two. The halves describe one split, so +# keeping both would let a record contradict itself; a transcribed +# pair is cross-checked and DISCARDED when it does not sum to ~100. +# (D-3 roll-up) the aggregate is impressions-WEIGHTED. A flat mean lets a +# 50-impression post at 90% outvote a 10,000-impression post. +# (D-3 boundary) reach never touches engagementRate - it is distribution, not +# engagement, and folding it in would break comparability with +# every historical import. +# (A2-F11) the report OFFERS a baseline diff and writes engagement-patterns.md +# only on an explicit operator go, to the per-user data dir - never +# silently, never into the plugin's shipped template. +# (E#9) the boundary map states what is measurable and what is not: +# dwell explicitly unmeasurable, saves partner-gated, reach native +# but CSV-export status UNVERIFIED. An unstated boundary gets +# quietly filled with an estimate. +CSVP_N16="scripts/analytics/src/parsers/csv-parser.ts" +TYPES_N16="scripts/analytics/src/models/types.ts" +STATS_N16="scripts/analytics/src/utils/stats.ts" +RPT_N16="commands/report.md" +IMP_N16="commands/import.md" +README_N16="README.md" +ADATA_N16="assets/analytics/README.md" + +patterns_update_gated() { # $1 = text; gated iff it NAMES the baseline file, REQUIRES a go, and FORBIDS silent writing + echo "$1" | grep -qF "audience-insights/engagement-patterns.md" \ + && echo "$1" | grep -qF "AskUserQuestion" \ + && echo "$1" | grep -qF "never write it silently" +} + +PU_SELFTEST_OK=1 +if ! patterns_update_gated "read audience-insights/engagement-patterns.md, offer a diff and never write it silently - ask with AskUserQuestion first"; then + PU_SELFTEST_OK=0; echo " non-vacuity FAIL: a fully-gated patterns-update probe was not detected" +fi +while IFS= read -r probe; do + [ -z "$probe" ] && continue + if patterns_update_gated "$probe"; then + PU_SELFTEST_OK=0; echo " false-positive FAIL: under-gated patterns-update probe accepted -> $probe" + fi +done <<'NEGATIVE16W' +update audience-insights/engagement-patterns.md from the report, asking with AskUserQuestion when unsure +read audience-insights/engagement-patterns.md and never write it silently, applying the diff directly +offer the diff with AskUserQuestion and never write it silently, file left unnamed +NEGATIVE16W +if [ "$PU_SELFTEST_OK" -eq 1 ]; then + pass "patterns-update self-test: predicate needs baseline file + explicit go + no-silent-write (1 accepted, 3 under-gated rejected)" +else + fail "patterns-update self-test failed - the N16 A2-F11 lint is vacuous or over-eager" +fi + +# (D-3 contract) the percent ingest is its OWN parser, not saves' count parser +if grep -qF "function parseOptionalPercent" "$CSVP_N16" 2>/dev/null; then + pass "csv-parser has a dedicated percent parser for the reach share (D-3)" +else + fail "$CSVP_N16 has no parseOptionalPercent - a share is being parsed as a count (D-3)" +fi + +# (D-3 contract) a share above 100 is refused, and a comma is a decimal mark +if grep -qF "parsed > 100" "$CSVP_N16" 2>/dev/null \ + && grep -qF 'cleaned.replace(/,/g, ".")' "$CSVP_N16" 2>/dev/null; then + pass "reach parsing refuses >100 and reads a comma as a decimal mark (never 36,5 -> 365)" +else + fail "$CSVP_N16 reach parsing accepts a non-share value or mis-reads a decimal comma (D-3)" +fi + +# (D-3 shape) both halves reduce to ONE stored value, and a contradictory pair is dropped +if grep -qF "resolveOutOfNetworkPct" "$CSVP_N16" 2>/dev/null \ + && grep -qF "REACH_SPLIT_TOLERANCE" "$CSVP_N16" 2>/dev/null; then + pass "the reach split reduces to one stored half, with a bounded rounding tolerance (D-3)" +else + fail "$CSVP_N16 stores the reach split without a cross-check - a record can contradict itself (D-3)" +fi + +# (D-3 shape) the field is optional in the type, and in-network is documented as derived +if grep -qF "outOfNetworkPct?: number" "$TYPES_N16" 2>/dev/null \ + && grep -qF "complement by" "$TYPES_N16" 2>/dev/null; then + pass "PostMetrics carries the reach share as optional, with in-network documented as its complement" +else + fail "$TYPES_N16 reach field is not optional, or stores both halves (D-3)" +fi + +# (D-3 boundary) reach is kept out of the engagement rate +if grep -qF "NOT folded into engagementRate" "$TYPES_N16" 2>/dev/null; then + pass "reach is explicitly kept out of engagementRate (comparability with historical imports)" +else + fail "$TYPES_N16 does not state that reach stays out of engagementRate (D-3)" +fi + +# (D-3 roll-up) the aggregate is weighted, and zero total weight yields unknown - not 0, not NaN +if grep -qF "export function weightedOutOfNetworkPct" "$STATS_N16" 2>/dev/null \ + && grep -qF "totalWeight <= 0" "$STATS_N16" 2>/dev/null; then + pass "the reach roll-up is impressions-weighted and returns unknown on zero weight (never 0/NaN)" +else + fail "$STATS_N16 has no weighted reach roll-up, or can emit 0/NaN for an unweighted set (D-3)" +fi + +# (rendering) the import surface documents the manual column AND its partial coverage +if grep -qF "Reach entered:" "$IMP_N16" 2>/dev/null \ + && grep -qF "Out-of-network" "$IMP_N16" 2>/dev/null; then + pass "/linkedin:import documents the reach column and the coverage-aware output line" +else + fail "$IMP_N16 does not document the manual reach column (the field can never be filled)" +fi + +# (rendering) the report shows the split, reads it as acquisition-vs-resonance, and refuses +# to invent it when it was not entered +if grep -qF "Reach Split" "$RPT_N16" 2>/dev/null \ + && grep -qF "not entered for this period" "$RPT_N16" 2>/dev/null \ + && grep -qF "never estimate it" "$RPT_N16" 2>/dev/null; then + pass "/linkedin:report renders the reach split with an explicit unknown branch (never estimated)" +else + fail "$RPT_N16 renders no reach split, or may estimate an unentered one (D-3)" +fi + +# (do-next) a reach reading steers the next piece instead of being admired in the report +if grep -qF "Never persist a reach directive" "$RPT_N16" 2>/dev/null; then + pass "a reach insight routes into the do-next channel, and only when it was measured (N15 contract)" +else + fail "$RPT_N16 leaves the reach reading in the report - it never reaches the next draft" +fi + +# (A2-F11) the baseline update is real, gated, and aimed at the user's own copy +if patterns_update_gated "$(cat "$RPT_N16" 2>/dev/null)"; then + pass "/linkedin:report offers an operator-gated engagement-patterns update (A2-F11)" +else + fail "report.md updates the patterns baseline silently, or not at all (A2-F11)" +fi + +if grep -qF "never to the plugin's shipped" "$RPT_N16" 2>/dev/null; then + pass "the patterns update targets the per-user data dir, never the shipped template (A2-F11)" +else + fail "$RPT_N16 does not forbid writing the plugin's shipped patterns template (A2-F11)" +fi + +# (E#9) the boundary map is current on all three metrics: reach, saves, dwell +if grep -qF "In-network vs out-of-network reach" "$README_N16" 2>/dev/null \ + && grep -qF "unverified" "$README_N16" 2>/dev/null; then + pass "boundary map states the reach split as native-but-not-exported, export status unverified (E#9)" +else + fail "$README_N16 boundary map does not cover the reach split honestly (E#9)" +fi + +if grep -qF "Explicitly unmeasurable" "$README_N16" 2>/dev/null \ + && grep -qF "partner-gated" "$README_N16" 2>/dev/null; then + pass "boundary map keeps dwell explicitly unmeasurable and saves partner-gated (E#9)" +else + fail "$README_N16 boundary map lost the dwell/saves boundaries (E#9)" +fi + +# (entry rules) the operator-facing rules cover either-half entry and the cross-check +if grep -qF "Either half works" "$ADATA_N16" 2>/dev/null \ + && grep -qF "Both halves are cross-checked" "$ADATA_N16" 2>/dev/null; then + pass "the data README documents either-half entry and the split cross-check" +else + fail "$ADATA_N16 does not document the reach entry rules (the column will be filled wrong)" +fi + +echo "" + # --- Section 18: Assertion-Count Anti-Erosion (SC6) --- # The lint self-modifies its own checks, so a green run could mask a silently dropped # assertion. Pin the total pass()+fail() invocations as a monotonic floor; the count @@ -2600,12 +2772,19 @@ echo "" # analyze writer + ab-test Adopt writer + 48h-monitor writer + analytics-interpreter # directive-shape grep + four create-surface reader greps + newsletter Step-1 queue-id/ # honest-miss compound grep + recordDoNext export grep + --record-do-next CLI verb grep + -# lifetime replace-by-source/age-floor grep + state-template section/scalar grep) = 213. +# lifetime replace-by-source/age-floor grep + state-template section/scalar grep) = 213; +# +15 for N16's fifteen UNCONDITIONAL Section-16w checks (patterns-update self-test + +# parseOptionalPercent grep + >100/decimal-comma refusal grep + resolve/tolerance +# cross-check grep + optional-field/complement grep + engagementRate-exclusion grep + +# weighted roll-up/zero-weight grep + import reach-column grep + report reach-split/ +# never-estimate compound grep + report reach do-next grep + A2-F11 gated-update +# compound grep + shipped-template write-ban grep + boundary-map reach/unverified grep + +# boundary-map dwell/saves grep + data-README entry-rules grep) = 228. # NB: the floor tracks the deps-absent MINIMUM (conditional TS suites warn-skip and drop # the count), so it is bumped only by UNCONDITIONAL new checks — NOT pinned to the # deps-present TOTAL_CHECKS (that would zero the warn-skip margin and false-fail a fresh # clone). Runs last so TOTAL_CHECKS sees every prior check. -ASSERT_BASELINE_FLOOR=213 +ASSERT_BASELINE_FLOOR=228 TOTAL_CHECKS=$((PASS + FAIL)) if [ "$TOTAL_CHECKS" -ge "$ASSERT_BASELINE_FLOOR" ]; then pass "assertion-count anti-erosion: $TOTAL_CHECKS checks >= baseline floor $ASSERT_BASELINE_FLOOR"