diff --git a/commands/ab-test.md b/commands/ab-test.md index 8201508..cc589c9 100644 --- a/commands/ab-test.md +++ b/commands/ab-test.md @@ -125,7 +125,7 @@ Generate a posting schedule that alternates A/B on comparable days: - Use the user's typical posting days (from state file or ask) - Alternate variants across the same day slots each week - Default: 3 posts per variant across 2 weeks (6 total) -- Suggest posting times based on algorithm-signals-reference.md (Tue-Thu, 8-9 AM optimal) +- Suggest posting times based on algorithm-signals-reference.md (Tue-Thu mornings) ### 2a.6: Set Success Criteria diff --git a/docs/hardening/log.md b/docs/hardening/log.md index 98c6cd0..f552c9d 100644 --- a/docs/hardening/log.md +++ b/docs/hardening/log.md @@ -1344,3 +1344,62 @@ weaker relevance match → less reach is defensible and "topic-relevance signal" - Disposition: **FIXED** (2 edits, `commands/audit.md`) · 0 deferrals · axes a/c PASS, b′ + d PASS post-fix. --- + +### /linkedin:ab-test — design/log/analyze/history/suggest A/B experiments → managed test files, challenger rewrite delegated to content-optimizer + +**INTENT.** Stateful analytics command — the most composite class in the queue so far. CLI-backed (Bash +`ls`/`cat`/`mkdir -p` + `Write` of `[test-name].md` test files under the data-dir), agent-delegated +(`content-optimizer` via `Task subagent_type: linkedin-studio:content-optimizer`, `:119`), AND routing +(`/linkedin:post` `:448`, `/linkedin:report` `:459`, `/linkedin:analyze` `:467`). Measure-journey +(reachable via `/linkedin:measure`). Five lifecycle modes (2a–2e) map 1:1 to the description: design · +log · analyze · history · suggest. (STATE predicted `analytics-interpreter` delegation; actual is +`content-optimizer`.) + +**SIMULATE.** ICP author → analyze an active hook experiment. Throwaway 2-experiment fixture in +`/private/tmp/claude-abtest-fixture/analytics/ab-tests/` (ACTIVE 6-post + COMPLETED), run via +`LINKEDIN_STUDIO_DATA=` to exercise the real seam. Seam-discovery (`:37`,`:270`) found both +files; analyze arithmetic computed correct (eng-rate `:236` per-post 3.0%/5.0%; variant avg + %-diff +`:278–279` → eng +66.7%, comments +200%, impressions/reposts +0%); directional test (`:311`) cleared +20% AND consistent across all 3 paired posts → "Adopt B (directional)"; significance honesty (`:320–331`) +correctly flagged 3/variant << ~50 → directional, NOT significant. History-parse targets all present in +COMPLETED fixture. Empty `weekly-reports/` + `posts/` → graceful degradation (`:485–488`). Fixture +discarded (`find -delete`). + +**EVALUATE (8 predicates across 4 axes; class = stateful + analytics + agent-wiring + routing).** +- **P1 (axis-d wiring): PASS** — `content-optimizer` resolves; `agents/content-optimizer.md` `name:` + matches the namespaced invocation form. +- **P2 (routing): PASS** — `post`/`report`/`analyze`.md all resolve. +- **P3 (intent refs): PASS** — `ab-testing-framework.md` + `algorithm-signals-reference.md` exist. +- **P4 (axis-b′ SSOT): 1 FUNN.** ② 20%-threshold **verbatim** from framework SSOT (`fw:173` = `ab:311`) + ✅; ③ significance honesty under-claims (a real significance test needs more than ~50/variant → the + framing can only make the "directional not significant" verdict safer) ✅; **① `:128` "(Tue-Thu, 8-9 AM + optimal)"** — sourced (cites the ref) but "optimal" over-claims precision against the SSOT's own `:23` + "trust the *ordering*, test the *number*", and Thu is 9 AM–1 PM in SSOT (`alg:144`), not 8 AM; it is the + default slot held constant while testing another variable, so "optimal" is the wrong word. +- **P5 (data-seam): PASS** — all data via `${LINKEDIN_STUDIO_DATA:-…}/`; `:44` = state file (correct), not + a leak; refs via `${CLAUDE_PLUGIN_ROOT}/`. +- **P6 (graceful degradation): PASS** — empty weekly-reports/posts → proceed test-only (`:485–488`); 4 + error modes (`:474–493`). +- **P7 (saves/dwell honesty): PASS** — no saves/dwell mention; eng-rate `:236` excludes saves (S16-consistent). +- **P8 (stateful artifact trace): PASS** — see SIMULATE; full analyze + history path sound against a + realistic file. + +**HARDEN (1 surgical edit, `commands/ab-test.md`).** +1. [REWORK · axis-b′] `:128` "(Tue-Thu, 8-9 AM optimal)" → "(Tue-Thu mornings)" — dropped the precision + over-claim ("optimal" + the inaccurate 8 AM window); the line stem already cites + `algorithm-signals-reference.md`, so source attribution is retained. + +**Out-of-scope (logged, NOT touched in S18).** `:284` "baseline comparison" cross-ref vs `weekly-reports/` +— echoes the S15 "baselines/metadata phantom" vocabulary, but here it reads an existing artifact (the +report command's weekly reports), optional + graceful-degraded; legitimate soft context, not a phantom. +Left as-is (low, no-change). + +**VERIFY.** +- Re-grepped final file: `grep "8-9 AM optimal"` → **NONE**; `:128` = "...based on + algorithm-signals-reference.md (Tue-Thu mornings)". +- `git diff --stat` = 1 file, **+1 / −1**. +- `bash scripts/test-runner.sh` → `Passed: 81 · Failed: 0 · Warnings: 0`, **exit 0**; counts **29/19** + unchanged (.md-only edit). +- Disposition: **FIXED** (1 edit, `commands/ab-test.md`) · 0 deferrals · P1–P3,P5–P8 PASS, P4 PASS post-fix. + +---