open/linkedin-studio
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions

fix(linkedin-studio): S15 harden report — drop baselines/metadata phantom, fix engagement_rate flag + JSON schema fields

Browse source 
S15 report through the v2 interactive quality gate (analytics class). 3 findings,
6 surgical edits in commands/report.md, all grep-confirmed + grounded against a
throwaway-root CLI run:

- FUNN 1 (S14 twin): Step 5c baseline/std-dev mechanism + Reference Files referenced
  baselines.json/metadata.json that no code writes (storage.ts writes only
  posts/weekly-reports/monthly-reports). Rewrote Step 5c to read the report's own
  alerts[] (detectAlerts + detectWeeklyAlerts); fixed posts filename to
  YYYY-MM-DD-<shortid>.json; added real monthly-reports file.
- FUNN 2: trends --metric engagement_rate (snake_case) → exit 1 'Invalid metric';
  CLI accepts only engagementRate (cli.ts:47,202-209). Fixed report.md:172.
- FUNN 3: Step 4/Step 6 claimed JSON fields dateRange/postCount/aggregateMetrics that
  don't exist; real schema is summary/topPerformers/underperformers/trends/alerts.
  Rewrote to real fields; fixed Deep-Dive post-search path.

Gate: test-runner 81/0/0 exit 0; counts 29/19 unchanged (.md-only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016qgzo6rxthw7KuxHjn5vyE
This commit is contained in:
Kjell Tore Guttormsen 2026-06-19 06:18:44 +02:00
commit 060ce8a371
2 changed files with 82 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

46
commands/report.md
View file

@ -139,12 +139,11 @@ cat ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/weekly-repo
The report contains:
- **week**: ISO week identifier
- **dateRange**: Start and end dates
- **postCount**: Number of posts published
- **aggregateMetrics**: Totals and averages across all metrics
- **topPerformers**: Best posts by each metric
- **alerts**: Anomalies and significant events
- **trends**: Week-over-week changes
- **generatedAt**: ISO timestamp when the report was generated
- **summary**: Totals and averages — `totalPosts`, `totalImpressions`, `totalReactions`, `totalComments`, `totalShares`, `totalClicks`, optional `totalSaves` (manual entry only), `avgEngagementRate`, `avgImpressionsPerPost`
- **topPerformers** / **underperformers**: Best / weakest posts by engagement rate
- **trends**: Week-over-week change — `impressionsTrend`, `engagementTrend`, `comparedTo`, `percentChange`
- **alerts**: Anomalies and significant events (intra-week + week-over-week)
## Step 5: Run Trend Analysis
@ -169,7 +168,7 @@ After the initial trend data, automatically run trend analysis for the key metri
trends --period month --metric impressions
"${CLAUDE_PLUGIN_ROOT}/scripts/analytics/node_modules/.bin/tsx" "${CLAUDE_PLUGIN_ROOT}/scripts/analytics/src/cli.ts" \
trends --period month --metric engagement_rate
trends --period month --metric engagementRate
```
**Present trend summary as a 4-week comparison table:**
@ -218,15 +217,19 @@ Automatically flag these conditions based on the report data and trend analysis:
- 🟡 Hook length violation: Posts with hooks >140 chars underperforming (>140 chars truncated on mobile "see more")
- 🟢 Engagement velocity improving: First-hour engagement trending up (15+ engagements in first hour unlocks 2nd/3rd degree distribution)
**Detect alerts by comparing current week data against baselines:**
**Surface the alerts the report already computed.** The weekly-report JSON's `alerts[]`
is generated by the CLI itself — intra-week anomaly detection across the week's posts
(`detectAlerts`) plus a week-over-week comparison (`detectWeeklyAlerts`) that fires once
the previous week's report exists. There is no separate baselines file to consult; read
`alerts[]` straight from the report you generated in Step 3:
```bash
cat ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/baselines.json 2>/dev/null
cat ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/weekly-reports/<YYYY-WXX>.json 2>/dev/null \
| python3 -c "import json,sys; [print(a['severity'].upper(), '—', a['message']) for a in json.load(sys.stdin).get('alerts', [])]"
```
Compare current week's `aggregateMetrics` against baseline means and standard deviations. Flag any metric that is:
- >2 standard deviations above mean → 🟢 Positive alert
- >2 standard deviations below mean → 🔴 Critical alert
- Between 1-2 standard deviations below → 🟡 Warning alert
Each alert carries a `severity` (`critical` / `warning` / `info`); map it to an icon
(🔴 / 🟡 / 🟢) and pair it with an action below. Week-over-week alerts appear only after a
prior week's report has been generated.
**Present alerts as:**
```
@ -253,7 +256,7 @@ Format the data into a readable report using this template:
```
# LinkedIn Performance Report
## Week {week} ({dateRange})
## Week {week}
### 📊 Key Metrics
@ -265,8 +268,8 @@ Format the data into a readable report using this template:
| Shares | {total} | {avg} | {trend} |
| Engagement Rate | - | {rate}% | {trend} |
**Posts published:** {postCount}
**Engagement rate:** {totalEngagements / totalImpressions * 100}%
**Posts published:** {summary.totalPosts}
**Engagement rate:** {summary.avgEngagementRate}%
### 🏆 Top Performers
@ -372,7 +375,9 @@ If user wants to analyze specific posts:
Read the weekly post data directly:
```bash
cat ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/posts/<YYYY-WXX>.json | jq '.posts[] | select(.title | contains("search term"))'
for f in ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/posts/*.json; do
jq '.posts[] | select(.title | contains("search term"))' "$f"
done
```
Show detailed metrics for that post and suggest what made it perform well/poorly.
@ -406,10 +411,9 @@ Read `~/.claude/linkedin-studio.local.md` and suggest:
## Reference Files
Reports use data from:
- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/posts/YYYY-WXX.json` - Raw weekly post data
- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/weekly-reports/YYYY-WXX.json` - Computed report
- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/baselines.json` - Statistical baselines for comparison
- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/metadata.json` - Import history and tracking
- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/posts/YYYY-MM-DD-<shortid>.json` - Raw imported batches (one file per import; rows live under `.posts[]`)
- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/weekly-reports/YYYY-WXX.json` - Computed weekly report
- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/monthly-reports/YYYY-MM.json` - Computed monthly report
## Step 8b: Export Options

57
docs/hardening/log.md
View file

@ -1180,3 +1180,60 @@ functionally safe but mis-labelled "(Step 3b)". Root cause: conflating "the phas
post-fix.
---
### /linkedin:report — imported analytics → weekly/monthly/heatmap report, alerts surfaced, interpretation delegated to analytics-interpreter
**WHAT VERIFIED CLEAN (no change).**
- Path-seam (axis-a): `getAnalyticsRoot()` (`storage.ts:67–72`) → `getDataRoot("analytics")` (`:54–59`)
= the exact `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics` paths the prose reads;
`ANALYTICS_ROOT` override confirmed (the grounded sim ran isolated in `/private/tmp`).
- CLI surface (axis-a): `report --week`/`--month`, `trends --period --metric`, `heatmap` all dispatch
(`cli.ts:447–459`); grounded sim produced verbatim weekly report + trends + heatmap on a 3-row fixture.
- Wiring (axis-d): `subagent_type: linkedin-studio:analytics-interpreter` (`report.md:305`) resolves —
`agents/analytics-interpreter.md` present with report-mode (`:10,40,295`); one namespaced `Task`.
- Degradation (axis-d): `trends`/`heatmap` "No posts found. Import some data first." exit 1
(`cli.ts:245–248,397–400`); weekly report try/catch exit 1 (`:191–194`); Step 1 import-nudge;
npm-install Step 1b for fresh-clone `ERR_MODULE_NOT_FOUND`.
- saves/dwell honesty (axis-c): `totalSaves` surfaced only when present, labelled "(manual entry — top
engagement signal)" (`cli.ts:144–146,342–344`; `weekly.ts:172–173`); grounded run — FIXTURE B's blank
Saves cell produced **no** saves suffix (unknown≠0); never folded into engagement. **dwell** absent,
explicit "do not fabricate a dwell field" (`types.ts:26–28`).
**THE FIX (3 findings, 6 surgical edits, all in `commands/report.md`).**
1. [REWORK · axis-a/d] **Over-promise: `baselines.json` + `metadata.json` infrastructure the code never
had** (the S14 import twin). Step 5c told the model to `cat baselines.json` and flag ">2 standard
deviations above/below mean"; Reference Files listed both files. Grep over `scripts/analytics/src` =
**NONE writes either**; `storage.ts` writes only `posts/<date>-<shortid>.json`,
`weekly-reports/<week>.json`, `monthly-reports/<month>.json`; `ensureDirectories` (`:77–90`) makes no
baselines dir; a full import+report cycle in the fixture created **NEITHER**. The real alerts are
`detectAlerts` (intra-week) + `detectWeeklyAlerts` (WoW) surfaced in the report's own `alerts[]`
(`weekly.ts:222–238`). Fix: Step 5c rewritten to read `alerts[]` straight from the generated
weekly-report JSON; Reference Files dropped both phantoms, fixed the posts filename to
`YYYY-MM-DD-<shortid>.json`, added the real `monthly-reports/` file.
2. [REWORK · axis-a] **Broken metric flag `engagement_rate` (snake_case, `:172`).** Grounded:
`--metric engagement_rate` → exit 1 "Invalid metric"; the CLI accepts only camelCase `engagementRate`
(`cli.ts:47,202–209`). Fix: `:172` snake→camel (the other call, old `:363`, was already correct).
3. [REWORK · axis-a] **Step 4 / Step 6 schema field names that don't exist.** Step 4 claimed JSON fields
`dateRange`, `postCount`, `aggregateMetrics`; the real `WeeklyReport` (`weekly.ts:124–149`) top keys
are `week, generatedAt, summary, topPerformers, underperformers, trends, alerts` (no dateRange, no
aggregateMetrics, no top-level postCount — it is `summary.totalPosts`). Fix: Step 4 rewritten to the
real schema; Step 6 template `({dateRange})` dropped, `{postCount}`→`{summary.totalPosts}`,
eng-rate→`{summary.avgEngagementRate}`; Deep-Dive post-search fixed from the non-existent
`posts/<YYYY-WXX>.json` to iterating the real per-batch `posts/*.json`.
**VERIFY.**
- Re-grepped the final file: `engagement_rate` / `baselines.json` / `metadata.json` / `aggregateMetrics`
/ `({dateRange})` / `{postCount}` / `posts/<YYYY-WXX>.json` all GONE; corrected strings present
(`metric engagementRate` `:171,366`; `get('alerts', [])` `:227`; `posts/YYYY-MM-DD-<shortid>.json`
`:414`; `monthly-reports/YYYY-MM.json` `:416`; Step 4 `summary` schema `:143`; Step 6
`{summary.totalPosts}`/`{summary.avgEngagementRate}` `:271–272`); no residual `dateRange`/`postCount`.
- `bash scripts/test-runner.sh` → `Passed: 81 · Failed: 0 · Warnings: 0`, **exit 0**; counts **29/19**
unchanged (.md-only edit).
- Grounded simulation (throwaway `ANALYTICS_ROOT=/private/tmp/claude-report-fixture`, discarded): import
seeded 3 fake W25 posts → `report --week 2026-W25` printed "Total saves: 10 (manual entry…)", FIXTURE B
(blank saves) showed no suffix, empty alerts (no baseline consulted); `trends --metric engagementRate`
ok, `--metric engagement_rate` exit 1; heatmap rendered.
- Disposition: **FIXED** (6 edits, all in `commands/report.md`) · 0 deferrals · axes a/c/d all PASS
post-fix (b N/A — not feed-emitting).
---