From 060ce8a371061e59788d000b58b75377b72f7ff2 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Fri, 19 Jun 2026 06:18:44 +0200 Subject: [PATCH] =?UTF-8?q?fix(linkedin-studio):=20S15=20harden=20report?= =?UTF-8?q?=20=E2=80=94=20drop=20baselines/metadata=20phantom,=20fix=20eng?= =?UTF-8?q?agement=5Frate=20flag=20+=20JSON=20schema=20fields?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit S15 report through the v2 interactive quality gate (analytics class). 3 findings, 6 surgical edits in commands/report.md, all grep-confirmed + grounded against a throwaway-root CLI run: - FUNN 1 (S14 twin): Step 5c baseline/std-dev mechanism + Reference Files referenced baselines.json/metadata.json that no code writes (storage.ts writes only posts/weekly-reports/monthly-reports). Rewrote Step 5c to read the report's own alerts[] (detectAlerts + detectWeeklyAlerts); fixed posts filename to YYYY-MM-DD-.json; added real monthly-reports file. - FUNN 2: trends --metric engagement_rate (snake_case) β†’ exit 1 'Invalid metric'; CLI accepts only engagementRate (cli.ts:47,202-209). Fixed report.md:172. - FUNN 3: Step 4/Step 6 claimed JSON fields dateRange/postCount/aggregateMetrics that don't exist; real schema is summary/topPerformers/underperformers/trends/alerts. Rewrote to real fields; fixed Deep-Dive post-search path. Gate: test-runner 81/0/0 exit 0; counts 29/19 unchanged (.md-only). Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_016qgzo6rxthw7KuxHjn5vyE --- commands/report.md | 46 ++++++++++++++++++---------------- docs/hardening/log.md | 57 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+), 21 deletions(-) diff --git a/commands/report.md b/commands/report.md index 294ff12..4b69b4c 100644 --- a/commands/report.md +++ b/commands/report.md @@ -139,12 +139,11 @@ cat ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/weekly-repo The report contains: - **week**: ISO week identifier -- **dateRange**: Start and end dates -- **postCount**: Number of posts published -- **aggregateMetrics**: Totals and averages across all metrics -- **topPerformers**: Best posts by each metric -- **alerts**: Anomalies and significant events -- **trends**: Week-over-week changes +- **generatedAt**: ISO timestamp when the report was generated +- **summary**: Totals and averages β€” `totalPosts`, `totalImpressions`, `totalReactions`, `totalComments`, `totalShares`, `totalClicks`, optional `totalSaves` (manual entry only), `avgEngagementRate`, `avgImpressionsPerPost` +- **topPerformers** / **underperformers**: Best / weakest posts by engagement rate +- **trends**: Week-over-week change β€” `impressionsTrend`, `engagementTrend`, `comparedTo`, `percentChange` +- **alerts**: Anomalies and significant events (intra-week + week-over-week) ## Step 5: Run Trend Analysis @@ -169,7 +168,7 @@ After the initial trend data, automatically run trend analysis for the key metri trends --period month --metric impressions "${CLAUDE_PLUGIN_ROOT}/scripts/analytics/node_modules/.bin/tsx" "${CLAUDE_PLUGIN_ROOT}/scripts/analytics/src/cli.ts" \ - trends --period month --metric engagement_rate + trends --period month --metric engagementRate ``` **Present trend summary as a 4-week comparison table:** @@ -218,15 +217,19 @@ Automatically flag these conditions based on the report data and trend analysis: - 🟑 Hook length violation: Posts with hooks >140 chars underperforming (>140 chars truncated on mobile "see more") - 🟒 Engagement velocity improving: First-hour engagement trending up (15+ engagements in first hour unlocks 2nd/3rd degree distribution) -**Detect alerts by comparing current week data against baselines:** +**Surface the alerts the report already computed.** The weekly-report JSON's `alerts[]` +is generated by the CLI itself β€” intra-week anomaly detection across the week's posts +(`detectAlerts`) plus a week-over-week comparison (`detectWeeklyAlerts`) that fires once +the previous week's report exists. There is no separate baselines file to consult; read +`alerts[]` straight from the report you generated in Step 3: ```bash -cat ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/baselines.json 2>/dev/null +cat ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/weekly-reports/.json 2>/dev/null \ + | python3 -c "import json,sys; [print(a['severity'].upper(), 'β€”', a['message']) for a in json.load(sys.stdin).get('alerts', [])]" ``` -Compare current week's `aggregateMetrics` against baseline means and standard deviations. Flag any metric that is: -- >2 standard deviations above mean β†’ 🟒 Positive alert -- >2 standard deviations below mean β†’ πŸ”΄ Critical alert -- Between 1-2 standard deviations below β†’ 🟑 Warning alert +Each alert carries a `severity` (`critical` / `warning` / `info`); map it to an icon +(πŸ”΄ / 🟑 / 🟒) and pair it with an action below. Week-over-week alerts appear only after a +prior week's report has been generated. **Present alerts as:** ``` @@ -253,7 +256,7 @@ Format the data into a readable report using this template: ``` # LinkedIn Performance Report -## Week {week} ({dateRange}) +## Week {week} ### πŸ“Š Key Metrics @@ -265,8 +268,8 @@ Format the data into a readable report using this template: | Shares | {total} | {avg} | {trend} | | Engagement Rate | - | {rate}% | {trend} | -**Posts published:** {postCount} -**Engagement rate:** {totalEngagements / totalImpressions * 100}% +**Posts published:** {summary.totalPosts} +**Engagement rate:** {summary.avgEngagementRate}% ### πŸ† Top Performers @@ -372,7 +375,9 @@ If user wants to analyze specific posts: Read the weekly post data directly: ```bash -cat ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/posts/.json | jq '.posts[] | select(.title | contains("search term"))' +for f in ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/posts/*.json; do + jq '.posts[] | select(.title | contains("search term"))' "$f" +done ``` Show detailed metrics for that post and suggest what made it perform well/poorly. @@ -406,10 +411,9 @@ Read `~/.claude/linkedin-studio.local.md` and suggest: ## Reference Files Reports use data from: -- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/posts/YYYY-WXX.json` - Raw weekly post data -- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/weekly-reports/YYYY-WXX.json` - Computed report -- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/baselines.json` - Statistical baselines for comparison -- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/metadata.json` - Import history and tracking +- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/posts/YYYY-MM-DD-.json` - Raw imported batches (one file per import; rows live under `.posts[]`) +- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/weekly-reports/YYYY-WXX.json` - Computed weekly report +- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/monthly-reports/YYYY-MM.json` - Computed monthly report ## Step 8b: Export Options diff --git a/docs/hardening/log.md b/docs/hardening/log.md index 57d05ae..e05de77 100644 --- a/docs/hardening/log.md +++ b/docs/hardening/log.md @@ -1180,3 +1180,60 @@ functionally safe but mis-labelled "(Step 3b)". Root cause: conflating "the phas post-fix. --- + +### /linkedin:report β€” imported analytics β†’ weekly/monthly/heatmap report, alerts surfaced, interpretation delegated to analytics-interpreter + +**WHAT VERIFIED CLEAN (no change).** +- Path-seam (axis-a): `getAnalyticsRoot()` (`storage.ts:67–72`) β†’ `getDataRoot("analytics")` (`:54–59`) + = the exact `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics` paths the prose reads; + `ANALYTICS_ROOT` override confirmed (the grounded sim ran isolated in `/private/tmp`). +- CLI surface (axis-a): `report --week`/`--month`, `trends --period --metric`, `heatmap` all dispatch + (`cli.ts:447–459`); grounded sim produced verbatim weekly report + trends + heatmap on a 3-row fixture. +- Wiring (axis-d): `subagent_type: linkedin-studio:analytics-interpreter` (`report.md:305`) resolves β€” + `agents/analytics-interpreter.md` present with report-mode (`:10,40,295`); one namespaced `Task`. +- Degradation (axis-d): `trends`/`heatmap` "No posts found. Import some data first." exit 1 + (`cli.ts:245–248,397–400`); weekly report try/catch exit 1 (`:191–194`); Step 1 import-nudge; + npm-install Step 1b for fresh-clone `ERR_MODULE_NOT_FOUND`. +- saves/dwell honesty (axis-c): `totalSaves` surfaced only when present, labelled "(manual entry β€” top + engagement signal)" (`cli.ts:144–146,342–344`; `weekly.ts:172–173`); grounded run β€” FIXTURE B's blank + Saves cell produced **no** saves suffix (unknownβ‰ 0); never folded into engagement. **dwell** absent, + explicit "do not fabricate a dwell field" (`types.ts:26–28`). + +**THE FIX (3 findings, 6 surgical edits, all in `commands/report.md`).** +1. [REWORK Β· axis-a/d] **Over-promise: `baselines.json` + `metadata.json` infrastructure the code never + had** (the S14 import twin). Step 5c told the model to `cat baselines.json` and flag ">2 standard + deviations above/below mean"; Reference Files listed both files. Grep over `scripts/analytics/src` = + **NONE writes either**; `storage.ts` writes only `posts/-.json`, + `weekly-reports/.json`, `monthly-reports/.json`; `ensureDirectories` (`:77–90`) makes no + baselines dir; a full import+report cycle in the fixture created **NEITHER**. The real alerts are + `detectAlerts` (intra-week) + `detectWeeklyAlerts` (WoW) surfaced in the report's own `alerts[]` + (`weekly.ts:222–238`). Fix: Step 5c rewritten to read `alerts[]` straight from the generated + weekly-report JSON; Reference Files dropped both phantoms, fixed the posts filename to + `YYYY-MM-DD-.json`, added the real `monthly-reports/` file. +2. [REWORK Β· axis-a] **Broken metric flag `engagement_rate` (snake_case, `:172`).** Grounded: + `--metric engagement_rate` β†’ exit 1 "Invalid metric"; the CLI accepts only camelCase `engagementRate` + (`cli.ts:47,202–209`). Fix: `:172` snakeβ†’camel (the other call, old `:363`, was already correct). +3. [REWORK Β· axis-a] **Step 4 / Step 6 schema field names that don't exist.** Step 4 claimed JSON fields + `dateRange`, `postCount`, `aggregateMetrics`; the real `WeeklyReport` (`weekly.ts:124–149`) top keys + are `week, generatedAt, summary, topPerformers, underperformers, trends, alerts` (no dateRange, no + aggregateMetrics, no top-level postCount β€” it is `summary.totalPosts`). Fix: Step 4 rewritten to the + real schema; Step 6 template `({dateRange})` dropped, `{postCount}`β†’`{summary.totalPosts}`, + eng-rateβ†’`{summary.avgEngagementRate}`; Deep-Dive post-search fixed from the non-existent + `posts/.json` to iterating the real per-batch `posts/*.json`. + +**VERIFY.** +- Re-grepped the final file: `engagement_rate` / `baselines.json` / `metadata.json` / `aggregateMetrics` + / `({dateRange})` / `{postCount}` / `posts/.json` all GONE; corrected strings present + (`metric engagementRate` `:171,366`; `get('alerts', [])` `:227`; `posts/YYYY-MM-DD-.json` + `:414`; `monthly-reports/YYYY-MM.json` `:416`; Step 4 `summary` schema `:143`; Step 6 + `{summary.totalPosts}`/`{summary.avgEngagementRate}` `:271–272`); no residual `dateRange`/`postCount`. +- `bash scripts/test-runner.sh` β†’ `Passed: 81 Β· Failed: 0 Β· Warnings: 0`, **exit 0**; counts **29/19** + unchanged (.md-only edit). +- Grounded simulation (throwaway `ANALYTICS_ROOT=/private/tmp/claude-report-fixture`, discarded): import + seeded 3 fake W25 posts β†’ `report --week 2026-W25` printed "Total saves: 10 (manual entry…)", FIXTURE B + (blank saves) showed no suffix, empty alerts (no baseline consulted); `trends --metric engagementRate` + ok, `--metric engagement_rate` exit 1; heatmap rendered. +- Disposition: **FIXED** (6 edits, all in `commands/report.md`) Β· 0 deferrals Β· axes a/c/d all PASS + post-fix (b N/A β€” not feed-emitting). + +---