Compare commits
70 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 95b1521ae5 | |||
| 5c6393f2f7 | |||
| 81510297db | |||
| 1a67bd2cb8 | |||
| c4434ed789 | |||
| 776d728d7d | |||
| d67552eab1 | |||
| 882f6eee5e | |||
| 4fd038ad1d | |||
| 4109fe7fd0 | |||
| 69f37ba2b3 | |||
| 2b706609bd | |||
| 5474df50e6 | |||
| 9567689c4a | |||
| da0a16a17c | |||
| e9e183ebb0 | |||
| db8cb8c7e3 | |||
| 9e95222d12 | |||
| 001d76ce99 | |||
| 5b51b4baeb | |||
| ddedb3d1de | |||
| e0b191db0c | |||
| 2a8459c674 | |||
| 5aa7187243 | |||
| 3276e44dbf | |||
| b43757462b | |||
| b185db9a12 | |||
| c40b937856 | |||
| e169c78710 | |||
| 4d3b9f4711 | |||
| 9111f006cc | |||
| fa7551070e | |||
| 88fbbadb1b | |||
| 7a158030b6 | |||
| b4e500fad4 | |||
| 24775f4493 | |||
| 0e95ca8cce | |||
| 68f6283d8a | |||
| fbad29d3d4 | |||
| 974e8d1b25 | |||
| 0061bf2bb5 | |||
| edd3e15ef7 | |||
| 016d823f3b | |||
| 585f972a05 | |||
| d2ec7482be | |||
| 4fa411f13c | |||
| fbfbf71cdd | |||
| 41fa9a5fa5 | |||
| f91ffddc8c | |||
| 2d01dfca52 | |||
| 88356b8a83 | |||
| ff39d14206 | |||
| 204665e90b | |||
| be9c619d9d | |||
| ac532f9218 | |||
| b65eb25328 | |||
| 0d3e4911d7 | |||
| 3e3990f36e | |||
| f549d9fdf3 | |||
| 8c927198f5 | |||
| d6acb5d36c | |||
| d3199eb997 | |||
| 0907b2b92d | |||
| 9f9c4bbd86 | |||
| e75cd42bed | |||
| 94d4e707db | |||
| 05a22d2cc7 | |||
| be12fb8b63 | |||
| b89868e3b1 | |||
| be21788321 |
168 changed files with 22202 additions and 744 deletions
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "linkedin-studio",
|
||||
"version": "0.5.0",
|
||||
"version": "0.6.0",
|
||||
"description": "LinkedIn Studio — full-spectrum LinkedIn content engine: feed posts, carousels, video scripts, and long-form newsletter editions, with the 2026 relevance-ranking model baked in. v4.0.0 is an audit-remediation release (Voyage Phase 0–3): every user-facing claim is made honest or removed, all 11 previously-orphaned agents are wired (→ 19 agents), a `/linkedin:firsthour` post-publish command is added (→ 27 commands), the algorithm-signal claims are reconciled to one sourced statement (no unpublishable model name or date), short-form de-AI and video quality gates are added, and the structure lint is rebuilt to guard the real layout plus version/count/stat consistency. Breaking: the newly-wired agents register only on reinstall/reload, and this consolidates the v3.0.0 identity break (slug, agent namespace `linkedin-studio:<agent>`, state-file path `~/.claude/linkedin-studio.local.md`). v3.1.0 added the cold adversarial review package (`/linkedin:headless-review` + Step 6.5 + `/linkedin:pivot` + per-artifact personas); the `/linkedin:*` commands are unchanged. v4.1.0 adds a journey layer: two guided front-doors (`/linkedin:create`, `/linkedin:measure`) plus a router re-tiered into five journeys (Start · Create · Engage · Measure · Grow), with the 27 existing commands kept as the execution tier (→ 29 commands; additive, reload registers the two new commands).",
|
||||
"author": {
|
||||
"name": "Kjell Tore Guttormsen"
|
||||
|
|
|
|||
5
.gitignore
vendored
5
.gitignore
vendored
|
|
@ -41,7 +41,6 @@ assets/drafts/week-*/
|
|||
assets/analytics/exports/
|
||||
assets/analytics/posts/
|
||||
assets/analytics/weekly-reports/
|
||||
assets/analytics/content-history.md
|
||||
|
||||
# Internal development files (not for public release)
|
||||
BACKLOG.md
|
||||
|
|
@ -57,10 +56,14 @@ docs/**/.session-state.local.json
|
|||
# Node.js
|
||||
scripts/analytics/node_modules/
|
||||
scripts/analytics/build/
|
||||
scripts/brain/node_modules/
|
||||
scripts/brain/build/
|
||||
scripts/contract-gate/node_modules/
|
||||
scripts/contract-gate/build/
|
||||
scripts/specifics-bank/node_modules/
|
||||
scripts/specifics-bank/build/
|
||||
scripts/trends/node_modules/
|
||||
scripts/trends/build/
|
||||
|
||||
# --- session/local state ---
|
||||
# STATE.md is LOCAL-ONLY (gitignored): no private remote exists and STATE must
|
||||
|
|
|
|||
70
CHANGELOG.md
70
CHANGELOG.md
|
|
@ -5,6 +5,76 @@ All notable changes to this project will be documented in this file.
|
|||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [0.6.0] - 2026-07-17
|
||||
|
||||
**Catch-up release.** Everything since the v0.5.3 tag ships here (28 commits, `git log --oneline v0.5.3..HEAD` is the source of truth), and the sections below also document work that shipped **inside** earlier tags but was never changelogged (see the catch-up note at the end).
|
||||
|
||||
### Added — figure pipeline (MR-F4/MR-F8)
|
||||
|
||||
- **`render/build-figur.mjs`** — coded data figures: SVG/HTML → PNG via headless Chrome; three targets (article 1200px wide with content-driven height / carousel 1080×1350 / single 1200×1200); brand tokens from the user data dir's `profile/brand-tokens.json` with neutral defaults (token-seam); Chrome-hang watchdog. Standalone CLI + importable module; 33 tests. (`c4434ed`)
|
||||
- **`/linkedin:newsletter` Step 7.5 — three figure routes** with the coded route PRIMARY for data figures, plus new **`references/figure-design-guidelines.md`** (reference docs 27 → 28). (`1a67bd2`)
|
||||
- **`render/build-html.mjs` parser reconciliation** — multi-line blockquotes (blank `>` line = new `<p>` in the same quote), `[text](url)` links with an http/https/mailto scheme-whitelist, and the `**[FIGUR N — «…»]**` marker → inline SVG from `figurer/figN*.svg` with figcaption (fallback: plain blockquote) + CSS for all three. Byte-identical parser parity with the upstream engine proven by fixture; the import-safe `main()` CLI-guard is kept. 7 new tests. (`8151029`)
|
||||
|
||||
### Added — research engine RE-R3a–e
|
||||
|
||||
- **R3a** — persist the relevance score on the store record + rank the morning brief on it. (`e169c78`)
|
||||
- **R3b** — trend lifecycle: re-score on re-capture, status, seen-log. (`b185db9`)
|
||||
- **R3c** — autonomous trigger: scheduler + headless entry. (`3276e44`)
|
||||
- **R3d** — temporal overlay: first-mover + saturation. (`2a8459c`)
|
||||
- **R3e** — brief history + day-over-day diff (frontmatter + "Nytt siden sist"). (`5b51b4b`)
|
||||
|
||||
### Added — OKF Stage 1
|
||||
|
||||
- The brain store emits an **OKF-compatible bundle** (`brain/` as bundle, `ingest/` excluded; title/description + typed pending-diff). (`9e95222`, `e9e183e`)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Oppgave 1 fix-pass** (from independent cold-review): clipboard heredoc, report refs, calendar queue fields. (`882f6ee`)
|
||||
- **Prune-regex no-op** in state-updater + date-independent calendar fixtures (flake). (`776d728`)
|
||||
|
||||
### Docs
|
||||
|
||||
- **Independent cold-review complete — 29/29 surfaces** (R2a Create emitters, R2b Create orchestrators, R3 Engage, R4 Measure, R5 Grow + Router; `docs/hardening/review*.md`). (`5474df5`…`4fd038a`)
|
||||
- **Truth-pass (this release):** README maturity note + badges (hardening 29/29 + cold-review 29/29; what remains for 1.0.0 is a GUI); CLAUDE.md maturity line + Architecture entries for specifics-bank and contract-gate; **GR-model correction** in `references/algorithm-signals-reference.md` — LinkedIn's ranking model has an official primary-source name, the **Generative Recommender (GR)**, announced 2026-03-12 on LinkedIn's engineering blog (the earlier "likely fabricated" flag rejected a genuine primary source and is retracted in a correction note); **saves-API rationale** updated (Marketing API v202604 exposes `POST_SAVE` on `/memberCreatorPostAnalytics`, partner-gated — manual entry remains the right UX); hardening queue table caught up through S31a/b/c; second-brain architecture header caught up through SB-S3a–e. Plus maturity-status correction (`001d76c`), CLAUDE.md trim (−2,266 always-loaded tokens, `9567689`), ingestion-guard adoption plan (`d67552e`).
|
||||
|
||||
### Catch-up note — work that shipped inside earlier tags, never changelogged
|
||||
|
||||
- **Inside v0.5.1:** **Fix #1 — contract-gate** (`scripts/contract-gate/`, deterministic §B/§C1 rule-gate, `/linkedin:newsletter` Step 4.5) and **Fix #2 — specifics-bank / lived-specifics** (`scripts/specifics-bank/` store + per-edition binding + Step 1.5 elicitation, slices 1–3). Together these took the newsletter pipeline 16 → 18 phases.
|
||||
- **Inside v0.5.3:** **SB-S3a–e** (profile.md reader-wiring, supersede arm, cross-silo id-threading, operations.md ops centre, content-history retirement + read-side reconcile) and **RE-R1–R2b** (item-schema + triage-scorer as tested code; item→store capture bridge with lossless schema v1→v2 migration; dated morning-brief artifact + session-start surfacing).
|
||||
|
||||
## [0.5.3] - 2026-06-24
|
||||
|
||||
### Changed — registration hygiene: agent fasit fixtures moved out of `agents/`
|
||||
|
||||
**Stops Claude Code from registering 6 spurious agents.** The six long-form-review fasit fixtures lived in `agents/fixtures/*-cases.md`; because Claude Code scans `agents/` recursively, each was registered as a junk `fixtures:*` agent (`fixtures:content-reviewer-cases`, …). They are test fixtures, not capabilities. **Count-neutral** (no real agent/command/reference/skill change; `ls agents/*.md` still 19) — pure namespace hygiene.
|
||||
|
||||
- **Moved** the 6 fasit fixtures `agents/fixtures/*-cases.md` → `tests/fixtures/*-cases.md`, and their 6 lint tests `agents/__tests__/*-fixture.test.mjs` → `tests/*-fixture.test.mjs` (co-located; each test's relative fixture path retargeted `../fixtures/` → `./fixtures/`). `agents/fixtures/` and `agents/__tests__/` are now removed.
|
||||
- **Retargeted** the 5 agent fasit-refs (`content`/`language`/`persona`/`editorial`/`fact-reviewer`) from `${CLAUDE_PLUGIN_ROOT}/agents/fixtures/…` to `${CLAUDE_PLUGIN_ROOT}/tests/fixtures/…`.
|
||||
- **Tests.** Fixture-lint 35/35 at the new path (`node --test 'tests/*.test.mjs'`); render 20/20, hooks 139/139 unaffected; gate `scripts/test-runner.sh` green (counts + version-consistency). Reinstall/reload required for the 6 spurious agents to disappear from the registry.
|
||||
|
||||
## [0.5.2] - 2026-06-23
|
||||
|
||||
### Added — SB-S2: the evolution loop (second-brain arc)
|
||||
|
||||
**The compounding mechanism.** An operator-invoked, operator-gated consolidation pass that reads the published gold signal + tributary deltas and proposes a **diff** to the two-layer `brain/profile.md` — never a silent overwrite. **Non-breaking** (additive minor): count-neutral (no command/agent/reference/skill); `voice-trainer` unchanged; one session-start hook edit (zero-dep).
|
||||
|
||||
- **Deterministic engine** (`scripts/brain/src/consolidate.ts`): `proposeDiff`/`applyDiff` (pure) — add / reject-`ai-draft` (model-collapse guard, in code) / evidence-bump / promote at N=3 / conflict→keep-both with **distinct ids** (primary `mintEntityId`, alt `mintContentId` — no duplicate-id corruption; no supersede, that's S3) / decay-flag at 90d (dynamic only; static decay-exempt). `applyDiff` round-trips through the SB-S0 grammar; re-running is idempotent.
|
||||
- **`brain consolidate` CLI**: `--gather` (reads published bodies since last run + the profile, for the invoking session to extract candidates — no new agent), `--propose --candidates <json>` (validates shape + single-line; writes `brain/pending-diff.{md,json}`; never touches profile.md), `--apply --diff <json> --confirm` (the ONLY path that writes profile.md; refuses without `--confirm`; records `brain/consolidation-state.json`).
|
||||
- **Session-start** (`hooks/scripts/session-start.mjs`, zero-dep edit): unconditional scaffold-ensure (fresh-install path) + a consolidation-due nudge (counts published records + reads the sidecar `last_run` via `getDataRoot`; no `profile.md` parse) + a brain-init nudge.
|
||||
- **Operator scope:** journal-capture deferred · the invoking session does the fuzzy extraction (no new agent) · motor-only (no `profile.md` reader until SB-S3). Docs: `docs/second-brain/consolidation-loop.md`.
|
||||
- **Tests.** brain suite 63 → 82 (`consolidate` 12 + `consolidate-cli` 7); hook suite 131 → 136 (new `session-start-brain-consolidation` 5); gate `scripts/test-runner.sh` 93/0/0 (`BRAIN_TESTS_FLOOR` 63→82; assertion floor unchanged at 78 — no new gate section). Light-Voyage-hardened (brief-reviewer REVISE, plan-critic REPLAN, scope-guardian ALIGNED — all folded).
|
||||
|
||||
## [0.5.1] - 2026-06-23
|
||||
|
||||
### Added — SB-S1: ingest + the published gold signal (second-brain arc)
|
||||
|
||||
**The second slice of the second-brain arc.** Captures the user's *actual published posts* into `ingest/published/` tagged `provenance=published` — the gold signal that voice/profile learning is allowed to learn from — and wires the learning surface to that signal **only**, never to AI-drafted content (the model-collapse guard). **Non-breaking** (additive minor): no command/agent/reference/skill count changes; a shipped agent's learning behaviour changes (`voice-trainer`).
|
||||
|
||||
- **Ingest data layer** (`scripts/brain/src/ingest.ts`): a `PublishedRecord` file-per-post grammar (fixed 5-line header + `---` sentinel + verbatim body, no YAML) with a `parse ∘ serialize` identity; a content-hash id `mintContentId = sha256(VERBATIM body)[:12]` — byte-identity dedup, so two structurally-different posts never collide and a differing body is never silently dropped.
|
||||
- **`brain` CLI** grows `ingest --file <path>` / `ingest --scan-inbox` (top-level `*.md`, skips dotfiles, non-destructive, idempotent, create-on-demand) and `published list` (surfaces `provenance` so ai-draft leakage is visible). The existing `init` is preserved.
|
||||
- **Published-only invariant wired + gate-enforced.** `agents/voice-trainer.md` reads `ingest/published/` as its primary gold source, keeps `voice-samples/` as a human tributary (not reshaped), forbids `provenance=ai-draft`, and fences the auto-append trap. A new structure-lint section (Brain Published-Only Invariant) enforces the wiring with exact-literal greps + a non-vacuity self-test.
|
||||
- **Tests.** brain suite 34 → 63 (`ingest` 14 + `publish` 9 + `cli` 6); gate `scripts/test-runner.sh` 90 → 93/0/0 (`BRAIN_TESTS_FLOOR` 34→63, assertion floor 75→78). No automatic `brain/profile.md` mutation (the consolidation loop is SB-S2); no cross-silo threading (SB-S3); no connector (SB-S4).
|
||||
|
||||
## [0.5.0] - 2026-06-18
|
||||
|
||||
### Added — M0: per-user external data dir (architecture workstream)
|
||||
|
|
|
|||
81
CLAUDE.md
81
CLAUDE.md
File diff suppressed because one or more lines are too long
|
|
@ -6,12 +6,12 @@
|
|||
|
||||
*AI-generated: all code produced by Claude Code through dialog-driven development. [Full disclosure →](../../README.md#ai-generated-code-disclosure)*
|
||||
|
||||

|
||||

|
||||

|
||||

|
||||

|
||||

|
||||

|
||||

|
||||

|
||||
|
||||
Most experts know they *should* post on LinkedIn — and quietly don't. The blank editor wins. LinkedIn Studio turns that chore into a system: structured workflows that take you from idea to published, in your own voice, calibrated to how LinkedIn's **topic-relevance** ranking model (2026) actually distributes content. Two engines under one surface — a **feed engine** for short-form posts, carousels, and video scripts, and a **long-form engine** that runs newsletter editions and essays through a serious editorial pipeline before they ever lock.
|
||||
|
|
@ -22,7 +22,7 @@ This is not a shortcut. Hand the wheel to the AI and you land where everyone who
|
|||
> New here? Run `/linkedin:onboarding` — it walks you through profile optimization, personalization, and your first published post in one guided flow (~10 minutes).
|
||||
|
||||
> [!NOTE]
|
||||
> **Pre-1.0 (v0.5.0).** The earlier 1.0.0–4.1.0 numbering reflected ambition, not maturity. Honest about where it stands today: the **architecture workstream (M0) is done** — user data now lives in a per-user data dir *outside* the plugin, with automatic migration — but no command has been through a hardening gate, command testing is incomplete, and there is no GUI yet. See [CHANGELOG.md](CHANGELOG.md).
|
||||
> **Pre-1.0.** The earlier 1.0.0–4.1.0 numbering reflected ambition, not maturity. Honest about where it stands today: the **architecture workstream (M0) is done** — user data lives in a per-user data dir *outside* the plugin, with automatic migration — and all **29 command surfaces have passed both the interactive hardening gate (29/29) and independent cold-review (29/29)**. What remains for 1.0.0 is a GUI. See [CHANGELOG.md](CHANGELOG.md).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -245,7 +245,7 @@ The README is the front door. The detail lives alongside it:
|
|||
| For… | See |
|
||||
|------|-----|
|
||||
| Architecture — agent pipeline & selection, 9 hooks, 6 skills, personalization scoring, configuration, analytics internals | [CLAUDE.md](CLAUDE.md) |
|
||||
| The 25-document knowledge base (algorithm signals, angles, frameworks, strategy guides) | [`references/`](references/) |
|
||||
| The 28-document knowledge base (algorithm signals, angles, frameworks, strategy guides) | [`references/`](references/) |
|
||||
| Full version history and known gaps | [CHANGELOG.md](CHANGELOG.md) |
|
||||
| Maintenance model, fork-and-own, what upstream provides | [GOVERNANCE.md](GOVERNANCE.md) |
|
||||
|
||||
|
|
|
|||
|
|
@ -149,21 +149,21 @@ For a 2-3 post/week cadence (optimal for sustainable growth):
|
|||
|
||||
These rotation rules are enforced at write-time by the `topic-rotation-gate` hook:
|
||||
|
||||
1. **No back-to-back pillars** — Never schedule the same pillar for consecutive posts. If Post 1 is "Azure AI", Post 2 must be a different pillar.
|
||||
1. **No back-to-back pillars** — Never schedule the same pillar for consecutive posts. If Post 1 is "Implementation", Post 2 must be a different pillar.
|
||||
2. **14-day 50% balance cap** — No single pillar may exceed 50% of posts in any rolling 14-day window.
|
||||
3. **Rotation priority** — When selecting the next pillar, prioritize the pillar with the highest gap score (most days since last post + fewest posts in 14-day window).
|
||||
4. **Underrepresented pillars** — Any pillar with 0 posts in the last 14 days should receive a priority slot in the next plan.
|
||||
|
||||
## Step 3: Seasonal & Event Awareness
|
||||
|
||||
### Annual Calendar — Nordic/Tech Focus
|
||||
### Annual Calendar — rhythm, adapt to your field & region
|
||||
|
||||
Check the current date and flag relevant themes:
|
||||
The cadence below is general. Anchor it with **your** field's events (conferences, release cycles, regulatory milestones) and your region's cultural moments — loaded from the user's profile/pillars, not baked in. Where a specific event is named, it is an *example* — swap in the equivalent from your domain. Check the current date and flag relevant themes:
|
||||
|
||||
```
|
||||
JANUARY
|
||||
- New Year goals/reflections → "My [year] priorities" posts
|
||||
- AI predictions for the year
|
||||
- Predictions for the year (in your field)
|
||||
- Q4 retrospective content
|
||||
|
||||
FEBRUARY
|
||||
|
|
@ -172,18 +172,18 @@ FEBRUARY
|
|||
- Valentine's: "Love letters to [profession/tool]" (entertaining)
|
||||
|
||||
MARCH
|
||||
- International Women's Day (Mar 8) → Diversity in tech
|
||||
- End of Q1 → Quarterly reflections
|
||||
- Spring conferences starting (Nordic tech scene)
|
||||
- International Women's Day (Mar 8) → diversity in your field
|
||||
- End of Q1 → quarterly reflections
|
||||
- Spring conference season starting (your field's events)
|
||||
|
||||
APRIL
|
||||
- NDC conferences season begins
|
||||
- AI regulation updates (EU AI Act milestones)
|
||||
- Easter break → Personal reflection posts
|
||||
- Conference season builds (your field's events)
|
||||
- Regulatory/policy updates in your domain
|
||||
- Easter break (region-dependent) → personal reflection posts
|
||||
|
||||
MAY
|
||||
- Microsoft Build (typically May) → AI announcements
|
||||
- 17. mai (Norwegian National Day) → Cultural content
|
||||
- Major release cycles (e.g. a key vendor's annual developer conference) → announcement commentary
|
||||
- Regional national days / cultural moments → cultural content (if it fits your brand)
|
||||
- End of spring conference season wrap-ups
|
||||
|
||||
JUNE
|
||||
|
|
@ -202,19 +202,19 @@ AUGUST
|
|||
- Conference CFP deadlines (fall events)
|
||||
|
||||
SEPTEMBER
|
||||
- Tech conference peak (Ignite, various Nordic events)
|
||||
- New product launches (Apple, Microsoft)
|
||||
- Conference peak in many fields → takeaways and commentary
|
||||
- Autumn product/release launches → commentary
|
||||
- "What I learned this summer" reflection
|
||||
|
||||
OCTOBER
|
||||
- Cybersecurity awareness month
|
||||
- Awareness months relevant to your field (e.g. cybersecurity)
|
||||
- Q3 wrap-ups
|
||||
- Halloween → Creative/entertaining tech content
|
||||
- Halloween → creative/entertaining content
|
||||
|
||||
NOVEMBER
|
||||
- Microsoft Ignite (typically November)
|
||||
- AI recap season begins
|
||||
- Black Friday → "Best [professional tools]" lists
|
||||
- Year-end vendor conferences / recaps → commentary
|
||||
- Recap season begins
|
||||
- Black Friday → "best [professional tools]" lists
|
||||
|
||||
DECEMBER
|
||||
- Year-in-review content
|
||||
|
|
|
|||
|
|
@ -283,6 +283,6 @@ Read these for the contract and the pipeline position:
|
|||
lock (Step 8).
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/longform-quality-rules.md` — the broad quality
|
||||
pass; this agent is the *finer* argument-integrity gate that runs cold after it.
|
||||
- `${CLAUDE_PLUGIN_ROOT}/agents/fixtures/content-reviewer-cases.md` — fasit
|
||||
- `${CLAUDE_PLUGIN_ROOT}/tests/fixtures/content-reviewer-cases.md` — fasit
|
||||
fixture: the Del 4 (Security Champions, Maskinrommet, 2026-05-29) worked cases
|
||||
mapping real argument defects to C1–C5 + severities.
|
||||
|
|
|
|||
|
|
@ -325,5 +325,5 @@ content-planner --> [draft] --> differentiation-checker --> content-optimizer --
|
|||
|
||||
Read these files for detailed methodology:
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/content-angles.md` -- 8 Universal Angles, combinations, red flags, authority value test
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/ai-content-framework.md` -- AI content anti-patterns, differentiation checklist, relevance filter
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/content-framework.md` -- content anti-patterns, differentiation checklist, relevance filter
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/engagement-frameworks.md` -- hook types, contrarian opening patterns, story structures
|
||||
|
|
|
|||
|
|
@ -276,6 +276,6 @@ Read these for the contract and the pipeline position:
|
|||
the gate that runs *after* this one; the role boundary is craft vs. response.
|
||||
- `${CLAUDE_PLUGIN_ROOT}/agents/fact-checker.md` — the Step 5 sweep (truth);
|
||||
this agent runs *after* it on the fact-checked draft.
|
||||
- `${CLAUDE_PLUGIN_ROOT}/agents/fixtures/editorial-reviewer-cases.md` — fasit
|
||||
- `${CLAUDE_PLUGIN_ROOT}/tests/fixtures/editorial-reviewer-cases.md` — fasit
|
||||
fixture: the Del 4 v5 gold-standard (KTG's eight editorial points mapped to the
|
||||
two axes + severities).
|
||||
|
|
|
|||
|
|
@ -349,6 +349,6 @@ Read these for the package, the boundary, and the pipeline position:
|
|||
for the five-archetype cold adversarial-review package.
|
||||
- `${CLAUDE_PLUGIN_ROOT}/commands/newsletter.md` — Step 6.5 (where this agent runs,
|
||||
cold, on the frozen draft) and Step 8 (lock + pivot-detection).
|
||||
- `${CLAUDE_PLUGIN_ROOT}/agents/fixtures/fact-reviewer-cases.md` — fasit fixture:
|
||||
- `${CLAUDE_PLUGIN_ROOT}/tests/fixtures/fact-reviewer-cases.md` — fasit fixture:
|
||||
the six Del 4 (Security Champions) worked cases mapped to F1–F4 + risk sort +
|
||||
the pivot-premise rationale.
|
||||
|
|
|
|||
|
|
@ -314,6 +314,6 @@ Read these for the boundary and the pipeline position:
|
|||
- `${CLAUDE_PLUGIN_ROOT}/references/longform-quality-rules.md` — the broad quality
|
||||
pass; rule 3 (AI-slop ban-list) is `voice-scrubber`'s; your axis is the cold
|
||||
Norwegian-language re-read, not the de-AI ban-list.
|
||||
- `${CLAUDE_PLUGIN_ROOT}/agents/fixtures/language-reviewer-cases.md` — fasit
|
||||
- `${CLAUDE_PLUGIN_ROOT}/tests/fixtures/language-reviewer-cases.md` — fasit
|
||||
fixture: the Del 4 / F5 language blind spots (the «Vi» vs «Vi i Nav» quote
|
||||
error, anglicisms, repetitions) mapped to L1–L5 + severities.
|
||||
|
|
|
|||
|
|
@ -687,7 +687,7 @@ Profile networking signals:
|
|||
|
||||
Headline:
|
||||
Include: What you do + Who you help + Signal (e.g., "Open to collabs")
|
||||
Example: "AI Advisor @ [org] | Helping public sector adopt AI | Speaker & Writer"
|
||||
Example: "Ops Lead @ [org] | Helping manufacturers cut downtime | Speaker & Writer"
|
||||
|
||||
About section:
|
||||
Last paragraph should include:
|
||||
|
|
|
|||
|
|
@ -394,4 +394,4 @@ the skeleton + pitches (NOT prose — there is none yet) and re-runs this sweep.
|
|||
|
||||
Read these files for the persona contract and pipeline position:
|
||||
- `${CLAUDE_PLUGIN_ROOT}/config/personas.template.md` — the reader persona library, five-field contract, primær rule, two-mode usage
|
||||
- `${CLAUDE_PLUGIN_ROOT}/agents/fixtures/persona-reviewer-cases.md` — fasit fixture: one persona + sample draft + six axes + both modes
|
||||
- `${CLAUDE_PLUGIN_ROOT}/tests/fixtures/persona-reviewer-cases.md` — fasit fixture: one persona + sample draft + six axes + both modes
|
||||
|
|
|
|||
|
|
@ -44,10 +44,37 @@ ${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/audience-insights/engagem
|
|||
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/examples/high-engagement-posts.md → proven patterns from top posts
|
||||
${CLAUDE_PLUGIN_ROOT}/references/trajectory-strategy-adjustments.md → trajectory-to-action mappings
|
||||
~/.claude/linkedin-studio.local.md → user state + posting history
|
||||
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/brain/profile.md → evolving second-brain profile (consolidated, evidence-weighted facts about the user)
|
||||
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/brain/operations.md → the operations centre: dated "who I am now" anchor + plans + ideas (user-authored)
|
||||
```
|
||||
|
||||
Use demographics data to compare the user's **intended** audience vs **actual** engagers when making strategic recommendations.
|
||||
|
||||
### Consuming the evolving profile (brain/profile.md)
|
||||
|
||||
The second-brain profile is a consolidated, evidence-weighted picture of the user, grown over time by the brain motor (mutated ONLY via `brain consolidate --apply --confirm` — you never write it). It has two layers:
|
||||
|
||||
- **`## Static`** — stable, high-confidence facts about the user.
|
||||
- **`## Dynamic`** — emerging facts still accruing evidence.
|
||||
|
||||
Each fact line carries `evidence_count` and `last_seen`. Weight by strength and recency: a higher `evidence_count` and a more recent `last_seen` are a stronger, fresher signal than a thin or stale one.
|
||||
|
||||
**Anti-sycophancy (binding):** Treat every profile fact as **evidence to TEST**, not flattery — counter-pressure it against the analytics and state each time you use it (e.g. *the profile says you lean toward X, but your recent imports show Y — test that before recommending*). The profile informs your recommendations; it never dictates them and never flatters the user.
|
||||
|
||||
**Graceful absence:** If `brain/profile.md` is missing or empty (fresh installs have no brain yet), proceed silently on the other context sources — no error, and no note about a missing profile.
|
||||
|
||||
### Consuming the operations centre (brain/operations.md)
|
||||
|
||||
The operations centre is the user-authored other half of the second brain (the brain motor never writes here). It has three sections: `## Who I am now (anchor)`, `## Plans`, and `## Ideas`.
|
||||
|
||||
**The anchor is authoritative — it deprecates older inferences.** The `## Who I am now` anchor is the user's own *dated* declaration of current direction (a `_As of YYYY-MM-DD:_` line). When a `brain/profile.md` Dynamic fact predates the anchor's date or contradicts its stated direction, **prefer the anchor and flag the older fact as possibly-stale** — do not parrot the stale fact. This is the frozen-past-self guard: the user's current self overrides the machine's older read of them.
|
||||
|
||||
**Inversion vs the profile (important):** unlike profile facts — which you treat as **evidence to TEST** (challenge them) — the anchor is user-declared direction, *not* an inference to challenge, so you **honour** it. It is direction, not praise: honouring it never licenses flattery, and it never excuses you from counter-pressuring the *profile* facts.
|
||||
|
||||
**Plans vs Ideas:** `## Plans` are active commitments (current intent — weigh as what the user is actually doing); `## Ideas` are a parking lot (suggestions, not commitments — surface them as options, never as settled plans).
|
||||
|
||||
**Graceful absence:** If `brain/operations.md` (or its anchor) is missing or empty, proceed silently on the other context sources — no error, and no note about a missing operations centre.
|
||||
|
||||
### New Creator Advantage Detection
|
||||
|
||||
From the state file, extract `first_post_date`. Calculate the creator window status:
|
||||
|
|
|
|||
|
|
@ -1,27 +1,34 @@
|
|||
---
|
||||
name: trend-spotter
|
||||
description: |
|
||||
Scan trending topics in AI, Microsoft, and public sector. Score relevance against content pillars,
|
||||
suggest content angles, assess first-mover timing, and generate weekly trend digests
|
||||
Scan trending topics across the user's content pillars and domain. Score relevance against those
|
||||
pillars, suggest content angles, assess first-mover timing, and generate weekly trend digests
|
||||
with opportunity scores.
|
||||
|
||||
Use when the user asks:
|
||||
- "what's trending?", "any hot topics?", "what should I post about?"
|
||||
- "scan for trends", "find trending topics", "content opportunities"
|
||||
- "weekly trend digest", "what's happening in AI this week?"
|
||||
- "weekly trend digest", "what's happening in my field this week?"
|
||||
- "is this topic still timely?", "should I post about this news?"
|
||||
- "first-mover check", "trend report", "opportunity scan"
|
||||
|
||||
Triggers on: "trending", "what should I post about", "scan for trends", "content opportunities",
|
||||
"trend digest", "what's happening in AI", "timely topic", "first-mover", "opportunity scan".
|
||||
"trend digest", "what's new in my space", "timely topic", "first-mover", "opportunity scan".
|
||||
model: sonnet
|
||||
color: white
|
||||
tools: ["Read", "WebSearch", "Glob"]
|
||||
# No `tools:` allowlist by design (research-engine slice 2b). An explicit allowlist would
|
||||
# block every research MCP unless its `mcp__<server>__<tool>` name were hardcoded here —
|
||||
# which breaks the "prefer whatever MCP the user actually connected, hardcode nothing"
|
||||
# routing. Omitting `tools:` inherits every session tool, so a Tavily / Gemini / Perplexity /
|
||||
# other research MCP is reachable when present, with WebSearch + WebFetch as the always-
|
||||
# available floor. `disallowedTools` trims the file-writing tools a read + search + persist
|
||||
# agent never needs; Bash stays (it runs the deterministic trend-store CLI).
|
||||
disallowedTools: Write, Edit, NotebookEdit
|
||||
---
|
||||
|
||||
# Trend Spotter Agent
|
||||
|
||||
You are a LinkedIn trend intelligence agent specialized in identifying timely content opportunities at the intersection of AI, Microsoft technology, and public sector digitalization. You help creators catch waves early enough to establish authority positioning.
|
||||
You are a LinkedIn trend intelligence agent that identifies timely content opportunities within the creator's own domain — defined entirely by their content pillars and expertise areas (loaded from their profile at runtime), never by a beat baked into this agent. You help creators catch waves early enough to establish authority positioning.
|
||||
|
||||
## Your Mission
|
||||
|
||||
|
|
@ -47,71 +54,90 @@ Before scanning, load the user's content pillars and expertise areas:
|
|||
3. **Check recent posts:** `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/posts/` (if available)
|
||||
- Avoid recommending topics they already covered recently
|
||||
|
||||
4. **Read research-tooling declaration:** the `### Research Tooling` block of the same
|
||||
`user-profile.md` — which research MCPs (if any) the user has connected, and any preferred
|
||||
order. This drives how you fetch (see **Research Routing** below). If the block is missing
|
||||
or every option is unchecked, treat the floor (WebSearch + WebFetch) as the only research
|
||||
tool.
|
||||
|
||||
5. **Load prior trend history (de-amnesia):** before polling anything new, query the
|
||||
persistent trend store for what you already captured on the candidate topics, so the digest
|
||||
reasons over accumulated history instead of starting amnesiac each session:
|
||||
```bash
|
||||
cd "${CLAUDE_PLUGIN_ROOT}/scripts/trends" && \
|
||||
node --import tsx src/cli.ts query --topics "<pillar-tag1,pillar-tag2,…>"
|
||||
```
|
||||
Use the hits to avoid re-surfacing a trend you already logged and acted on, and to spot a
|
||||
pattern building across captures. (Adopter note: run `npm install` in
|
||||
`${CLAUDE_PLUGIN_ROOT}/scripts/trends` once. If the store has no deps yet, skip this step
|
||||
and proceed with a live poll — the digest still works, just without memory.)
|
||||
|
||||
## Research Routing (MCP-first, floor-fallback)
|
||||
|
||||
WHERE you look is config (the source list, below); HOW you fetch is routed by the user's
|
||||
declared tooling. Pick the strongest research tool available this session — never bake a tool
|
||||
name into your reasoning, read it from the declaration:
|
||||
|
||||
1. **Prefer a declared research MCP.** If the profile's `### Research Tooling` names an MCP you
|
||||
can see this session (a Tavily / Gemini / Perplexity / other `mcp__…` search or research
|
||||
tool), use it first — a research MCP gives better non-US / regional coverage than WebSearch.
|
||||
Honor the user's "Preferred order" if they set one.
|
||||
2. **Floor: WebSearch + WebFetch.** When no research MCP is declared or connected, discover
|
||||
with WebSearch and read source pages with WebFetch. These are always available, so the
|
||||
engine still works with zero MCPs connected.
|
||||
3. **Fail soft.** If a tool call errors (an MCP that isn't actually connected, a fetch that
|
||||
fails), fall back to the floor and keep going — never abort the scan over one unavailable
|
||||
tool.
|
||||
|
||||
Record which tool you used as the `--source` when persisting (below): the MCP's short name
|
||||
(e.g. `tavily`), `websearch`, or `manual`.
|
||||
|
||||
## Source Scanning Framework
|
||||
|
||||
### Tier 1: Breaking News (daily, respond within 24-48h)
|
||||
**Which sources to poll is config, not code — read the list, do not hardcode a beat.** This is
|
||||
what keeps the engine generic: it serves any niche, because the niche lives in the source list,
|
||||
never in this agent.
|
||||
|
||||
- **OpenAI**, **Anthropic**, **Microsoft AI**, **Google AI** -- blog posts and announcements
|
||||
- **EU/Norwegian government** AI regulatory decisions
|
||||
**Load the source list (user override → shipped default):**
|
||||
|
||||
### Tier 2: Analysis & Research (2-3x/week, post within a week)
|
||||
1. If `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/trends/sources.md` exists, use it —
|
||||
the user's own niche-specific list (their vendors, regulators, outlets), which survives
|
||||
plugin upgrades/reinstalls.
|
||||
2. Otherwise fall back to the shipped generic defaults in
|
||||
`${CLAUDE_PLUGIN_ROOT}/config/trends-sources.template.md` (source *categories*, not one
|
||||
person's beat).
|
||||
|
||||
- **MIT Technology Review**, The Verge AI, Ars Technica AI, **Stratechery**
|
||||
- **Industry reports** from McKinsey, Gartner, Forrester on AI adoption
|
||||
- **ArXiv** top-cited papers in cs.AI, cs.CL, cs.LG
|
||||
Both files group sources into four tiers by cadence; poll on that cadence:
|
||||
|
||||
### Tier 3: Community Signals (weekly, post if pattern emerges)
|
||||
| Tier | What lives here | Cadence | Response window |
|
||||
|------|-----------------|---------|-----------------|
|
||||
| **Tier 1 — Primary / breaking** | first-party announcements, authoritative decisions | daily | react within 24–48h |
|
||||
| **Tier 2 — Analysis & research** | where developments get interpreted, not just reported | 2–3×/week | post within a week |
|
||||
| **Tier 3 — Community signals** | where practitioners surface what matters before the press | weekly | post if a pattern emerges |
|
||||
| **Tier 4 — Niche & seasonal** | slower sources with predictable cadence | monthly | plan ahead |
|
||||
|
||||
- **Hacker News** AI discussions (front page = high signal)
|
||||
- **r/MachineLearning**, **r/LocalLLaMA** trending posts
|
||||
- **LinkedIn** trending topics and viral posts in AI/tech
|
||||
|
||||
### Tier 4: Niche & Seasonal (monthly, plan ahead)
|
||||
|
||||
- **Conference announcements** (Build, Ignite, NeurIPS, AAAI)
|
||||
- **Quarterly earnings** with AI mentions (Microsoft, Google, etc.)
|
||||
- **Seasonal themes:** Q1 predictions/strategy, Q2 conferences, Q3 retrospectives, Q4 reflections
|
||||
|
||||
### Recommended Search Queries
|
||||
|
||||
```
|
||||
"OpenAI announcement" OR "Anthropic release" OR "Microsoft AI" this week
|
||||
"Azure AI" OR "Copilot" OR "Microsoft 365 AI" new features
|
||||
"AI regulation" OR "EU AI Act" OR "AI policy" latest
|
||||
"public sector AI" OR "government AI" latest
|
||||
"AI enterprise" OR "AI implementation" report [year]
|
||||
"AI trend" OR "AI debate" LinkedIn [this week]
|
||||
```
|
||||
Build search queries from the loaded source list + the user's pillars — not from a hardcoded
|
||||
query bank: target a source or topic from the list (`"[Tier-1 source] latest"`,
|
||||
`"[pillar] [this week]"`), fetched via the routed tool (**Research Routing**, above).
|
||||
|
||||
## Relevance Scoring System
|
||||
|
||||
Score each discovered trend on a 1-10 scale across five dimensions.
|
||||
**The scoring rubric is a single source of truth — do not inline a matrix here.** Read it from
|
||||
`${CLAUDE_PLUGIN_ROOT}/references/trend-scoring-modes.md` and apply the matching mode:
|
||||
|
||||
### Scoring Matrix
|
||||
- **kortform** (default) — feed posts. Timing + audience pull carry real weight; the first-mover
|
||||
window is short.
|
||||
- **long-form** — when the caller is producing a chronicle / newsletter / series edition (e.g.
|
||||
invoked from `/linkedin:newsletter`) or asks for it explicitly. Depth potential enters at 25 %
|
||||
and timing drops to 10 % — a chronicle rewards substance and a durable angle over speed.
|
||||
|
||||
| Dimension | Weight | 1-2 (Low) | 3-5 (Medium) | 6-8 (High) | 9-10 (Exceptional) |
|
||||
|-----------|--------|-----------|---------------|-------------|---------------------|
|
||||
| **Pillar Fit** | 30% | Outside all 5 pillars | Tangential to one pillar | Direct hit on one pillar | Intersects 2+ pillars |
|
||||
| **Audience Relevance** | 25% | Wrong audience entirely | Some audience overlap | Core audience cares | Audience actively asking about this |
|
||||
| **Timing** | 20% | >7 days old, saturated | 3-7 days, moderate coverage | 24-72h, early coverage | <24h, you would be among first |
|
||||
| **Angle Potential** | 15% | Only obvious take available | One good angle possible | 2-3 strong angles | Contrarian or unique angle clear |
|
||||
| **Authority Match** | 10% | No credibility on topic | Some related experience | Direct experience | Published authority on this |
|
||||
|
||||
### Composite Score Calculation
|
||||
|
||||
```
|
||||
Opportunity Score = (Pillar Fit x 0.30) + (Audience x 0.25) + (Timing x 0.20) + (Angle x 0.15) + (Authority x 0.10)
|
||||
```
|
||||
|
||||
### Score Interpretation
|
||||
|
||||
| Score | Priority | Action |
|
||||
|-------|----------|--------|
|
||||
| 8.0-10 | **Immediate** | Drop everything and draft a post within 24h |
|
||||
| 6.0-7.9 | **High** | Plan and publish within 48-72h |
|
||||
| 4.0-5.9 | **Medium** | Add to content calendar for this week |
|
||||
| 2.0-3.9 | **Low** | Note for future reference, skip for now |
|
||||
| 0-1.9 | **Skip** | Not relevant to your positioning |
|
||||
Score each candidate's five dimensions 1–10 per the mode's table — that qualitative judgment is
|
||||
yours. The deterministic step that follows is NOT: pipe the scored candidates (JSON on stdin) to the
|
||||
scorer CLI `${CLAUDE_PLUGIN_ROOT}/scripts/trends/src/cli.ts score` (`--mode kortform|long-form
|
||||
[--threshold N]`), the single owner of the weighted composite, the composite→action bands
|
||||
(Immediate / High / Medium / Low / Skip), and the keep/drop threshold. It returns the kept candidates
|
||||
ranked highest-first, each annotated with its composite + band. Do not recompute the composite or
|
||||
restate the band thresholds here — supply the five judgment scores and let the scorer rank and triage.
|
||||
|
||||
## Trend Opportunity Assessment
|
||||
|
||||
|
|
@ -205,8 +231,8 @@ If an angle fails the test, try a different one before including in the digest.
|
|||
|
||||
| Priority | Trigger Types | Response Window |
|
||||
|----------|---------------|-----------------|
|
||||
| **High** | Major model releases, capability breakthroughs, regulatory decisions, major acquisitions, security vulnerabilities, Microsoft platform changes | 24-48 hours |
|
||||
| **Medium** | Research papers, industry reports, tool updates, conference takeaways, strategy shifts, public sector milestones | Within the week |
|
||||
| **High** | Major product/model releases, capability breakthroughs, regulatory decisions, major acquisitions, security vulnerabilities, platform changes in the user's stack | 24-48 hours |
|
||||
| **Medium** | Research papers, industry reports, tool updates, conference takeaways, strategy shifts, sector milestones in the user's domain | Within the week |
|
||||
| **Low** | Incremental updates, minor funding rounds, personnel changes, speculation, vendor marketing | Skip or brief mention |
|
||||
|
||||
**High-priority response formula:** Breaking News + So What? + Now What?
|
||||
|
|
@ -216,7 +242,7 @@ If an angle fails the test, try a different one before including in the digest.
|
|||
Before including any trend in the digest, it must pass at least 2 of 4:
|
||||
|
||||
1. **Expertise fit?** Relevant to my core areas (Yes = proceed, No = skip unless huge)
|
||||
2. **Audience care?** Public sector leaders or enterprise AI implementers would notice
|
||||
2. **Audience care?** The user's target audience (per their profile) would notice and care
|
||||
3. **Unique perspective?** I can add experience-based insight, not just commentary
|
||||
4. **Urgency?** Time-sensitive topic with closing window
|
||||
|
||||
|
|
@ -224,18 +250,14 @@ Before including any trend in the digest, it must pass at least 2 of 4:
|
|||
|
||||
### Step-by-Step Generation
|
||||
|
||||
**Step 1: Scan sources (WebSearch)**
|
||||
**Step 1: Scan sources (routed fetch)**
|
||||
|
||||
Run 4-6 targeted searches covering all tiers:
|
||||
|
||||
```
|
||||
Search 1: "[AI announcement OR release] [current week/month] [year]"
|
||||
Search 2: "Microsoft [AI OR Copilot OR Azure] [news OR update] [year]"
|
||||
Search 3: "[public sector OR government] [AI OR digital] [latest OR news]"
|
||||
Search 4: "[AI regulation OR policy OR governance] [latest]"
|
||||
Search 5: "[AI enterprise OR implementation] [trend OR report] [year]"
|
||||
Search 6: "[AI debate OR controversy OR opinion] LinkedIn [this week]"
|
||||
```
|
||||
Run 4–6 targeted searches covering the tiers in your loaded source list (**Source Scanning
|
||||
Framework**, above), via the routed tool (**Research Routing** — declared MCP first, WebSearch +
|
||||
WebFetch floor). Each query targets a source or topic from the list crossed with a user pillar —
|
||||
e.g. `"[Tier-1 source] latest"`, `"[pillar] [this week]"`, `"[regulator] [recent decision]"`. Do
|
||||
not use a fixed query bank: the niche lives in the source list and the user's pillars, never in
|
||||
this agent.
|
||||
|
||||
**Step 2: Filter and score**
|
||||
|
||||
|
|
@ -257,6 +279,85 @@ Search 6: "[AI debate OR controversy OR opinion] LinkedIn [this week]"
|
|||
- Run Authority Value Test on each recommendation
|
||||
- Discard angles that fail the test
|
||||
|
||||
**Step 4.5: Persist kept trends to the store (de-amnesia)**
|
||||
|
||||
For every trend that cleared the relevance filter (Step 2) — not only the ones that make the
|
||||
final digest — fold it into the persistent trend store, so the next session reasons over it
|
||||
instead of re-discovering it. Build ONE raw-item batch (the same trends you just scored) and pipe
|
||||
it through `capture`: it normalizes each item, dedupes on normalized title+URL, unions topics on
|
||||
re-capture (so re-capturing an existing trend just enriches the tags), persists the source's
|
||||
`publishedAt` for later freshness ranking, and — when you carry the score (below) — persists the
|
||||
relevance assessment so the morning brief ranks on it — one call, not one per trend:
|
||||
|
||||
```bash
|
||||
cd "${CLAUDE_PLUGIN_ROOT}/scripts/trends" && \
|
||||
echo '[
|
||||
{"source":"<tavily|websearch|manual|…>","title":"<verbatim headline>","url":"<source url>",
|
||||
"topics":["<pillar-tag1>","<pillar-tag2>"],"publishedAt":"<YYYY-MM-DD if known>",
|
||||
"summary":"<one-line what-happened>",
|
||||
"score":{"mode":"kortform","dimensions":{"pillar":N,"audience":N,"timing":N,"angle":N,"authority":N}}}
|
||||
]' | node --import tsx src/cli.ts capture
|
||||
```
|
||||
|
||||
`source` is the tool you actually fetched with (**Research Routing**); `publishedAt` is the
|
||||
source's own publish date — omit the key when unknown (the store's `capturedAt` is set
|
||||
automatically and stays distinct from it).
|
||||
|
||||
**Carry the Step-2 scores — do not discard them.** You already scored each candidate's five
|
||||
dimensions 1–10 in **Relevance Scoring** (Step 2); fold those same numbers into the capture batch
|
||||
as the item's `"score"`, so the store persists the relevance assessment and the morning brief
|
||||
ranks on its composite (the store computes the composite + band itself — supply only the judgment).
|
||||
Use `"mode":"kortform"` by default; use `"mode":"long-form"` with the long-form dimension names
|
||||
(`pillar`, `depth`, `angle`, `authority`, `currency`) when the caller is producing a chronicle /
|
||||
newsletter / series edition (e.g. invoked from `/linkedin:newsletter`). The `"dimensions"` keys are
|
||||
the rubric's, the `"topics"` are the user's pillars — nothing vendor- or sector-specific is baked
|
||||
in. Omit the `"score"` key when you genuinely did not score an item; an out-of-range or malformed
|
||||
score is reported in `errors[]` (the valid items still persist) and never crashes the run.
|
||||
|
||||
One `capture` call folds the whole batch and reports
|
||||
`{added, merged, duplicates, errors}`; content-invalid items land in `errors[]`, never failing the
|
||||
run. Skip this step silently if the store has no deps installed (an adopter without the trends
|
||||
store) — the digest still compiles, just without persistence.
|
||||
|
||||
**Re-capture refreshes the score; the operator drives the lifecycle.** Re-capturing a trend already
|
||||
in the store never duplicates it — its topics union in and its relevance `score` is **refreshed**
|
||||
(the newer judgment wins, since the timing dimension decays). The operator marks a trend `acted`
|
||||
(written about) or `skipped` with `act`/`skip --id <id>` (the id is shown in the brief and via
|
||||
`list --json`); the morning brief then **excludes** handled trends so the queue surfaces only
|
||||
unresolved work, and `reset --id` returns one to the queue.
|
||||
|
||||
**Step 4.6: Write the dated morning brief (surfacing)**
|
||||
|
||||
After capturing, render today's dated morning brief over the store so the **next session surfaces
|
||||
it automatically** (the SessionStart hook reads the latest one). Pass the user's content pillars —
|
||||
the same ones you scored against in Step 2 — and the brief ranks the store by pillar-overlap, then
|
||||
recency, into a dated Markdown file:
|
||||
|
||||
```bash
|
||||
cd "${CLAUDE_PLUGIN_ROOT}/scripts/trends" && \
|
||||
node --import tsx src/cli.ts brief --pillars "<pillar1>,<pillar2>,<pillar3>"
|
||||
```
|
||||
|
||||
`--pillars` is the user's pillar list (comma-separated, from their profile/config); the brief is
|
||||
written to `<data-dir>/trends/morning-brief/YYYY-MM-DD.md` and ranks only on persisted fields
|
||||
(pillar overlap + `publishedAt`/`capturedAt` freshness, default 7-day window — tune with
|
||||
`--fresh-days N`). Skip silently if the store has no deps installed — same escape hatch as Step 4.5.
|
||||
|
||||
The brief also applies a **derived temporal overlay** (RE-R3d): within a relevance tier, a fresh,
|
||||
not-yet-surfaced trend is ranked up as a **first-mover** (`· 🥇 først ute`) and a repeatedly-surfaced
|
||||
one is ranked down as **saturated** (`· 🔁 mettet`) — computed at render time from the publish/capture
|
||||
dates + the seen-log, with no new capture step. Tune with `--first-mover-days N` / `--saturation-at N`.
|
||||
|
||||
Each brief also **records the trend ids it showed** (frontmatter `surfaced:`) and renders a
|
||||
**day-over-day diff** — a `## 🆕 Nytt siden sist` section listing what is new since the most recent
|
||||
prior brief (plus a ` N nye siden sist` marker on the one-line summary) — no new capture step; the
|
||||
polling/capture path above is unchanged (RE-R3e).
|
||||
|
||||
The morning brief can also be **scheduled** to regenerate autonomously each morning — deterministic,
|
||||
from the current store — via `src/cli.ts schedule` (print-first: it emits a launchd/cron entry firing
|
||||
the `run-daily.sh` headless wrapper). That nightly run re-renders the brief only; your polling above
|
||||
stays the capture path (autonomous AI polling is a later slice).
|
||||
|
||||
**Step 5: Compile digest**
|
||||
|
||||
- Format using output template below
|
||||
|
|
@ -356,12 +457,14 @@ Search 6: "[AI debate OR controversy OR opinion] LinkedIn [this week]"
|
|||
| Same angle every time | Predictable, audience tunes out | Rotate across 8 angles, track recently used |
|
||||
| Hype without substance | Loses trust, attracts wrong audience | Ground every take in experience or evidence |
|
||||
| Skipping the relevance filter | Wastes creator's time on low-value topics | Always run 4-question filter before scoring |
|
||||
| Generic "AI is changing everything" takes | Adds zero value, damages credibility | Be specific: what, for whom, by when |
|
||||
| Generic "[topic] is changing everything" takes | Adds zero value, damages credibility | Be specific: what, for whom, by when |
|
||||
|
||||
## References
|
||||
|
||||
Read these files for detailed methodology:
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/content-angles.md` - 8 universal angles, selection framework, combination patterns
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/ai-content-framework.md` - Content pillars, trigger framework, source tiers, seasonal calendar
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/content-framework.md` - Content pillars, trigger framework, source tiers, seasonal calendar (domain-general)
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/linkedin-growth-playbook-2025-2026.md` - Trend Translator tactic, first-mover advantage
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/algorithm-signals-reference.md` - Engagement signals and profile/topic-relevance validation
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/trend-scoring-modes.md` - **scoring SSOT** — kortform / long-form rubrics + composite→action bands (do not inline a matrix)
|
||||
- `${CLAUDE_PLUGIN_ROOT}/config/trends-sources.template.md` - shipped generic source-list defaults (user override: data-dir `trends/sources.md`)
|
||||
|
|
|
|||
|
|
@ -214,8 +214,8 @@ Naming convention:
|
|||
video-[YYYY-MM-DD]-[slug]-[type]-[length].md
|
||||
|
||||
Examples:
|
||||
video-2026-01-30-ai-implementation-talking-head-90s.md
|
||||
video-2026-01-30-copilot-demo-screen-recording-60s.md
|
||||
video-2026-01-30-onboarding-walkthrough-talking-head-90s.md
|
||||
video-2026-01-30-dashboard-demo-screen-recording-60s.md
|
||||
```
|
||||
|
||||
Update state in `~/.claude/linkedin-studio.local.md`:
|
||||
|
|
|
|||
|
|
@ -133,7 +133,7 @@ Architecture: [prose/sectioned/framework]
|
|||
|
||||
### Analysis Process
|
||||
|
||||
1. **Gather** — Read all files in `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/voice-samples/`, existing profile from `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/profile/user-profile.md`, and template from `config/user-profile.template.md`
|
||||
1. **Gather (published-only gold signal first)** — Read the user's actual published posts from `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/ingest/published/` (`provenance=published` — the highest-trust source). Then read the human-curated `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/voice-samples/` corpus (kept as a tributary, not reshaped), the existing profile from `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/profile/user-profile.md`, and the template from `config/user-profile.template.md`. **Never learn voice patterns from any content marked `provenance=ai-draft`** — a content engine that learns its own voice from its own drafts collapses toward its priors (this generalises voice-scrubber's "gold standard = approved editions, never the draft corpus"). **Fence:** any future auto-capture into a learning corpus (the voice-samples template's Stop-hook hint) MUST tag provenance and admit `provenance=published` only — never blind-append generated drafts.
|
||||
2. **Analyze** — Apply all six dimensions to each sample. Note dates for temporal analysis. Flag inconsistent samples as outliers or evolution.
|
||||
3. **Synthesize** — Patterns in 70%+ of samples = core traits. 40-70% = situational traits (note context). <40% = experimental traits. Track temporal trends.
|
||||
4. **Build** — Compile into Voice Profile Document format. Include confidence levels (high/medium/low) and concrete examples for every trait.
|
||||
|
|
@ -141,12 +141,15 @@ Architecture: [prose/sectioned/framework]
|
|||
|
||||
### Sample Quality Priorities
|
||||
|
||||
0. **`ingest/published/` records (`provenance=published`) — the gold signal; rank above all else.**
|
||||
1. Published posts with high engagement (audience-validated authenticity)
|
||||
2. Recent samples (last 6 months reflect current voice)
|
||||
3. Author-confirmed samples ("this sounds like me")
|
||||
4. Longer samples (more data points)
|
||||
5. Varied contexts (different content types reveal range)
|
||||
|
||||
**Exclude unconditionally: anything marked `provenance=ai-draft`** (the model-collapse guard — never train voice on the engine's own drafts).
|
||||
|
||||
Flag if: fewer than 5 samples (low confidence), single time period (temporal bias), or contradictory patterns (possible ghostwriting).
|
||||
|
||||
## Voice Drift Detection
|
||||
|
|
@ -326,5 +329,5 @@ Read these files for context and methodology:
|
|||
- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/voice-samples/` — Source samples for analysis
|
||||
- `${CLAUDE_PLUGIN_ROOT}/config/user-profile.template.md` — Profile structure template
|
||||
- `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/profile/user-profile.md` — Current voice profile (if exists)
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/ai-content-framework.md` — AI content anti-patterns and quality checklist
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/content-framework.md` — content anti-patterns and quality checklist
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/engagement-frameworks.md` — Hook psychology and tone guidelines
|
||||
|
|
|
|||
|
|
@ -39,8 +39,15 @@ console.log('=== OVERDUE ===');
|
|||
console.log(queueFormatSummary(queueOverdue()));
|
||||
console.log('=== COUNTS ===');
|
||||
console.log(JSON.stringify(queueCount(), null, 2));
|
||||
console.log('=== ENTRY RECORDS (internal — id / draft_path / character_count etc. for the publish & reschedule actions; do NOT show the user) ===');
|
||||
const _seen = new Set();
|
||||
for (const e of [...queueToday(), ...queueOverdue(), ...queueUpcoming(14)]) {
|
||||
if (_seen.has(e.id)) continue; _seen.add(e.id);
|
||||
console.log(JSON.stringify({ id: e.id, draft_path: e.draft_path, scheduled_date: e.scheduled_date, scheduled_time: e.scheduled_time, hook_preview: e.hook_preview, pillar: e.pillar, format: e.format, character_count: e.character_count }));
|
||||
}
|
||||
"
|
||||
```
|
||||
The `queueFormatSummary` blocks are the human-readable overview; the **ENTRY RECORDS** block is the agent's lookup table for the `id`, `draft_path`, and `character_count` that the action steps need (these fields are not in the readable summary).
|
||||
|
||||
Also read state for context:
|
||||
- `~/.claude/linkedin-studio.local.md` for weekly goal and current progress
|
||||
|
|
@ -110,7 +117,7 @@ No posts scheduled for today.
|
|||
- Run /linkedin:quick for an unplanned quick post
|
||||
```
|
||||
|
||||
**3b. Pick a post.** Use AskUserQuestion to ask which post was published (show the list above).
|
||||
**3b. Pick a post.** Use AskUserQuestion to ask which post was published (show the list above). Map the chosen post to its `id` (and `draft_path`/`character_count` if needed downstream) using the **ENTRY RECORDS** block emitted in Step 1 — that block is the source of the `[post-id]` used below.
|
||||
|
||||
**3c. Update queue status:**
|
||||
```bash
|
||||
|
|
@ -168,8 +175,8 @@ If they choose to reschedule:
|
|||
2. Ask for the new date and time
|
||||
3. Re-add the entry with the **same id** and new date/time — `queueAdd` replaces any
|
||||
existing entry with that id, so the post moves in place (no duplicate). Carry the
|
||||
unchanged fields (draft_path, pillar, format, hook preview, char count) from the
|
||||
entry shown in Step 2:
|
||||
unchanged fields (id, draft_path, pillar, format, hook preview, char count) from the
|
||||
**ENTRY RECORDS** block emitted in Step 1:
|
||||
```bash
|
||||
node --input-type=module -e "import { queueAdd } from '${CLAUDE_PLUGIN_ROOT}/hooks/scripts/queue-manager.mjs'; console.log(queueAdd('[post-id]', '[draft_path]', '[new-YYYY-MM-DD]', '[new-HH:MM]', '[pillar]', '[format]', '[hook preview]', [charCount]));"
|
||||
```
|
||||
|
|
|
|||
|
|
@ -208,9 +208,11 @@ CAPTION
|
|||
|
||||
Then auto-copy the full deck to clipboard silently:
|
||||
```bash
|
||||
printf '%s' '<FULL_DECK_PAYLOAD>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs
|
||||
node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs <<'__LINKEDIN_CLIP_EOF__'
|
||||
<FULL_DECK_PAYLOAD>
|
||||
__LINKEDIN_CLIP_EOF__
|
||||
```
|
||||
Substitute `<FULL_DECK_PAYLOAD>` with the assembled deck above — all slides' copy + the caption. Then confirm: "Full deck — [N] slides + caption — copied to clipboard."
|
||||
Substitute `<FULL_DECK_PAYLOAD>` between the heredoc markers with the assembled deck above — all slides' copy + the caption (a quoted heredoc keeps apostrophes, `%`, `$`, and backticks literal). Only if the helper prints `COPIED`, confirm: "Full deck — [N] slides + caption — copied to clipboard." If it prints `FAILED:<platform>`, tell the user no clipboard tool was found and to copy the deck above manually — do not claim it was copied.
|
||||
|
||||
Offer refinement options as text (no interactive prompt):
|
||||
"Want to refine? Options: adjust slide text / change visual style / regenerate specific slide / different hook / ready for publishing."
|
||||
|
|
|
|||
|
|
@ -138,10 +138,12 @@ Show the post with:
|
|||
|
||||
Auto-copy the post text to clipboard silently:
|
||||
```bash
|
||||
printf '%s' '<POST_TEXT>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs
|
||||
node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs <<'__LINKEDIN_CLIP_EOF__'
|
||||
<POST_TEXT>
|
||||
__LINKEDIN_CLIP_EOF__
|
||||
```
|
||||
|
||||
Then present: "Post copied to clipboard. Go to linkedin.com, click 'Start a post', paste it, and hit Post."
|
||||
Substitute `<POST_TEXT>` with the exact post text between the heredoc markers (a quoted heredoc keeps apostrophes, `%`, `$`, and backticks literal). Only if the helper prints `COPIED`, present: "Post copied to clipboard. Go to linkedin.com, click 'Start a post', paste it, and hit Post." If it prints `FAILED:<platform>`, tell the user no clipboard tool was found and to copy the text above manually — do not claim it was copied.
|
||||
|
||||
## Step 7: State Update
|
||||
|
||||
|
|
|
|||
|
|
@ -69,10 +69,12 @@ Show, in this order:
|
|||
Auto-copy the self-comments + draft replies to clipboard silently (so they're one paste away):
|
||||
|
||||
```bash
|
||||
printf '%s' '<DRAFT_COMMENTS_BLOCK>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs
|
||||
node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs <<'__LINKEDIN_CLIP_EOF__'
|
||||
<DRAFT_COMMENTS_BLOCK>
|
||||
__LINKEDIN_CLIP_EOF__
|
||||
```
|
||||
|
||||
Then confirm: "Copied your draft comments to clipboard."
|
||||
Substitute `<DRAFT_COMMENTS_BLOCK>` with the exact comments block between the heredoc markers (a quoted heredoc keeps apostrophes, `%`, `$`, and backticks literal). Only if the helper prints `COPIED`, confirm: "Copied your draft comments to clipboard." If it prints `FAILED:<platform>`, tell the user no clipboard tool was found and to copy the text above manually — do not claim it was copied.
|
||||
|
||||
## Step 4: Persist the Plan to State
|
||||
|
||||
|
|
|
|||
|
|
@ -118,9 +118,11 @@ After creating the adaptation:
|
|||
- Save to `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/drafts/multiplatform/[platform]-[slug].md`
|
||||
- Auto-copy the adapted content to clipboard silently:
|
||||
```bash
|
||||
printf '%s' '<ADAPTED_CONTENT>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs
|
||||
node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs <<'__LINKEDIN_CLIP_EOF__'
|
||||
<ADAPTED_CONTENT>
|
||||
__LINKEDIN_CLIP_EOF__
|
||||
```
|
||||
- Present the content and confirm: "Copied to clipboard."
|
||||
- Substitute `<ADAPTED_CONTENT>` with the exact adapted text between the heredoc markers (a quoted heredoc keeps apostrophes, `%`, `$`, and backticks literal). Present the content, and only if the helper prints `COPIED`, confirm: "Copied to clipboard." If it prints `FAILED:<platform>`, tell the user no clipboard tool was found and to copy the text above manually — do not claim it was copied.
|
||||
- Note platform-specific publishing tips
|
||||
|
||||
## Reference Files
|
||||
|
|
|
|||
|
|
@ -109,7 +109,7 @@ split; v3.1 / Endring 9 on adversarial independence + framing-bias).
|
|||
| 6 | **Persona sweep — BEFORE lock** | reader jury, primær wins, convergence to clean YES | **`persona-reviewer`** (resonance mode) |
|
||||
| 6.5 | **Headless adversarial review — BEFORE lock** | COLD review package on a frozen draft, no drafting-session context: content-reviewer (argument) + language-reviewer (Norwegian) + fact-reviewer (cold re-verification incl. pivot premises) + persona-reviewer resonance/conversion. Consolidated, operator-gated via `SendUserFile`. The independence layer the in-session gates can't be. | **`content-reviewer` + `language-reviewer` + `fact-reviewer` + `persona-reviewer`** (parallel) + `SendUserFile` |
|
||||
| 7 | **Annotation (optional)** | render annotatable review HTML for a manual pass | `render/build-html.mjs` |
|
||||
| 7.5 | **Visual assets — BEFORE lock** | cover (+ optional inline figures) or carousel deck: behov → per-image brief → generate (mcp-image default / external `cover-raw.png`) → operator-gate (`SendUserFile`) → approve to `cover.png` → credit/caption. Runs before lock so the renderer picks the cover up. | `mcp__mcp-image__generate_image` + `SendUserFile` + (carousel) `render/build-carousel.mjs` |
|
||||
| 7.5 | **Visual assets — BEFORE lock** | cover (+ optional inline figures) or carousel deck: behov → per-image brief → generate (coded `build-figur.mjs` primary for data figures / mcp-image for illustrative / external `cover-raw.png`) → operator-gate (`SendUserFile`) → approve to `cover.png` → credit/caption. Runs before lock so the renderer picks the cover up. | `render/build-figur.mjs` (data figures) + `mcp__mcp-image__generate_image` + `SendUserFile` + (carousel) `render/build-carousel.mjs` |
|
||||
| 8 | **LOCK → delivery** | POST.html "all in one place" | `render/build-linkedin.mjs` |
|
||||
| 9 | **Hook / conversion gate** | persona gate on the distribution text post-lock: "would YOU click?" | **`persona-reviewer`** (conversion mode) |
|
||||
| 10 | **Scheduling** | register the edition in the plugin queue/state for native scheduling | `hooks/scripts/queue-manager.mjs` |
|
||||
|
|
@ -1466,13 +1466,25 @@ operator declares carousel format for it. Branch accordingly:
|
|||
in `edition-state.json` → `articles.NN.visualAssets.cover.brief` and
|
||||
`…figures[].brief`.
|
||||
|
||||
3. **Generate — two routes, no lock-in.** The interface is pluggable (path-in /
|
||||
path-out); `mcp-image` is the default, not a hard dependency:
|
||||
- **Default route — `mcp__mcp-image__generate_image`** (Nano Banana Pro /
|
||||
Gemini 3 Pro Image). Write candidates to
|
||||
`linkedin/NN/cover-v<N>-kandidat.png` (and `fig<N>-kandidat.png` for
|
||||
figures). Candidate naming lets several attempts sit side by side without
|
||||
overwriting an approved file. Record route `"mcp-image"`.
|
||||
3. **Generate — three routes, no lock-in.** The interface is pluggable (path-in /
|
||||
path-out). Route by what the image *is*:
|
||||
- **Coded route — PRIMARY for data figures** (charts, diagrams, before/after
|
||||
comparisons — anything whose content is real numbers or real structure).
|
||||
Author the figure as SVG/HTML per
|
||||
`${CLAUDE_PLUGIN_ROOT}/references/figure-design-guidelines.md` (design
|
||||
rules + brand-token convention), then render to a candidate:
|
||||
```bash
|
||||
node "${CLAUDE_PLUGIN_ROOT}/render/build-figur.mjs" linkedin/NN/fig<N>.svg --target article --out linkedin/NN/fig<N>-kandidat.png
|
||||
```
|
||||
Precision and reproducibility beat generative output for data — numbers,
|
||||
labels, and proportions are exact, and the figure re-renders identically
|
||||
after a correction. Record route `"coded"`.
|
||||
- **Generative route — `mcp__mcp-image__generate_image`** (Nano Banana Pro /
|
||||
Gemini 3 Pro Image) for **illustrative** images (cover art, mood,
|
||||
metaphor). Write candidates to `linkedin/NN/cover-v<N>-kandidat.png` (and
|
||||
`fig<N>-kandidat.png` for figures). Candidate naming lets several attempts
|
||||
sit side by side without overwriting an approved file. Record route
|
||||
`"mcp-image"`.
|
||||
- **External route** — DALL·E, Midjourney, a photographer, a hand-built SVG.
|
||||
The plugin accepts a `linkedin/NN/cover-raw.png` the operator drops in; no
|
||||
tool is mandated. Record route `"external"`. (The raw file may then be
|
||||
|
|
@ -1560,7 +1572,7 @@ Visual assets (BEFORE lock).
|
|||
- Cover: linkedin/NN/cover.png approved (after <N> candidates) (or: N/A — carousel)
|
||||
- Figures: <N> approved → linkedin/NN/figN.png (or: none)
|
||||
- Carousel deck: linkedin/NN/carousel.pdf rendered + approved (or: N/A — standard)
|
||||
- Route: mcp-image | external Credit/caption: recorded in image-credit-caption.md + edition-config.json
|
||||
- Route: coded | mcp-image | external Credit/caption: recorded in image-credit-caption.md + edition-config.json
|
||||
- Operator gate: approved (candidates surfaced via SendUserFile) [OPERATØR]
|
||||
Next: Step 8 — LOCK → delivery.
|
||||
```
|
||||
|
|
@ -1841,3 +1853,5 @@ the honest decision surface; it sells nothing.
|
|||
- `${CLAUDE_PLUGIN_ROOT}/render/build-linkedin.mjs` — POST.html delivery; reads `linkedin/NN/cover.png` + credit/caption (Step 8)
|
||||
- `${CLAUDE_PLUGIN_ROOT}/render/build-html.mjs` — annotatable review renderer (Step 7)
|
||||
- `${CLAUDE_PLUGIN_ROOT}/render/build-carousel.mjs` — carousel deck renderer (`## SLIDE N —` → PDF via weasyprint) — Step 7.5 carousel branch
|
||||
- `${CLAUDE_PLUGIN_ROOT}/render/build-figur.mjs` — coded data-figure renderer (SVG/HTML → PNG via headless Chrome; targets article/carousel/single) — Step 7.5 coded route
|
||||
- `${CLAUDE_PLUGIN_ROOT}/references/figure-design-guidelines.md` — coded-figure design rules + brand-token convention + render targets — Step 7.5
|
||||
|
|
|
|||
|
|
@ -154,6 +154,7 @@ file must contain no `<!-- VOICE_PLACEHOLDER -->`.
|
|||
3. Job title / role
|
||||
4. 5 expertise areas (these become your content pillars)
|
||||
5. Target audience description
|
||||
6. Research MCPs connected (Tavily / Gemini deep research / Perplexity — or "none"; WebSearch + WebFetch are the always-available floor). Store only what they name — don't invent MCP names.
|
||||
|
||||
Save to `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/profile/user-profile.md`.
|
||||
|
||||
|
|
@ -217,9 +218,11 @@ Fix any miss before showing it.
|
|||
|
||||
Show the post with its character count, the hook highlighted, and one alternative hook. Auto-copy the post text to clipboard silently:
|
||||
```bash
|
||||
printf '%s' '<POST_TEXT>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs
|
||||
node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs <<'__LINKEDIN_CLIP_EOF__'
|
||||
<POST_TEXT>
|
||||
__LINKEDIN_CLIP_EOF__
|
||||
```
|
||||
Then say: "Post copied to clipboard. Go to linkedin.com, click 'Start a post', paste it, and hit Post."
|
||||
Substitute `<POST_TEXT>` with the exact post text between the heredoc markers (a quoted heredoc keeps apostrophes, `%`, `$`, and backticks literal). Only if the helper prints `COPIED`, say: "Post copied to clipboard. Go to linkedin.com, click 'Start a post', paste it, and hit Post." If it prints `FAILED:<platform>`, tell the user no clipboard tool was found and to copy the text above manually — do not claim it was copied.
|
||||
|
||||
### 3.5 — Record it
|
||||
|
||||
|
|
|
|||
|
|
@ -136,7 +136,9 @@ Offer to help identify target profiles and draft comments.
|
|||
|
||||
Auto-copy the final post text to clipboard silently before presenting:
|
||||
```bash
|
||||
printf '%s' '<FINAL_POST_TEXT>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs
|
||||
node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs <<'__LINKEDIN_CLIP_EOF__'
|
||||
<FINAL_POST_TEXT>
|
||||
__LINKEDIN_CLIP_EOF__
|
||||
```
|
||||
|
||||
Present the final post as copy-paste ready content:
|
||||
|
|
|
|||
|
|
@ -43,7 +43,7 @@ Check for existing assets:
|
|||
|
||||
## Step 1: Understand the Input
|
||||
|
||||
If the user already provided a clear topic with the command invocation (e.g., `/linkedin:post about AI governance in public sector`), skip asking and proceed directly. Only ask if the input is missing or genuinely vague.
|
||||
If the user already provided a clear topic with the command invocation (e.g., `/linkedin:post about pricing strategy for B2B SaaS`), skip asking and proceed directly. Only ask if the input is missing or genuinely vague.
|
||||
|
||||
Identify the type of raw material:
|
||||
|
||||
|
|
@ -150,9 +150,11 @@ Present ONE draft with:
|
|||
|
||||
Auto-copy the final post text to clipboard silently:
|
||||
```bash
|
||||
printf '%s' '<FINAL_POST_TEXT>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs
|
||||
node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs <<'__LINKEDIN_CLIP_EOF__'
|
||||
<FINAL_POST_TEXT>
|
||||
__LINKEDIN_CLIP_EOF__
|
||||
```
|
||||
Then confirm: "Copied to clipboard."
|
||||
Substitute `<FINAL_POST_TEXT>` with the exact post text between the heredoc markers (a quoted heredoc keeps apostrophes, `%`, `$`, and backticks literal). Only if the helper prints `COPIED`, confirm: "Copied to clipboard." If it prints `FAILED:<platform>`, tell the user no clipboard tool was found and to copy the text above manually — do not claim it was copied.
|
||||
|
||||
Do NOT proactively offer alternative versions. Only generate alternatives if the user asks for them.
|
||||
|
||||
|
|
|
|||
|
|
@ -56,7 +56,7 @@ search. Optimize for both.
|
|||
in every search result and connection suggestion, and renders under your name
|
||||
across the site — so it does the most SEO work per character. Lead with the plain
|
||||
words people actually search (the role, the domain, the audience), not a clever
|
||||
tagline. "AI Advisor · public-sector AI governance · Microsoft Copilot" is more
|
||||
tagline. "Data Engineer · healthcare analytics · HIPAA-compliant pipelines" is more
|
||||
findable than "Turning chaos into clarity ✨".
|
||||
|
||||
**Per-section keyword targets** (place the terms a searcher would type, in the
|
||||
|
|
@ -95,7 +95,7 @@ Guide the user through each section using AskUserQuestion for interactive feedba
|
|||
- [ ] No jargon or vague titles
|
||||
|
||||
**Strong example:**
|
||||
"Helping public sector leaders implement AI that actually works | AI Advisor @ [Company]"
|
||||
"Helping e-commerce teams turn returns data into retention | Retention Strategist @ [Company]"
|
||||
|
||||
**Weak example:**
|
||||
"Digital Transformation Expert | Thought Leader | Speaker"
|
||||
|
|
@ -134,7 +134,7 @@ Guide the user through each section using AskUserQuestion for interactive feedba
|
|||
**Transform each role with impact statements, not task lists.**
|
||||
|
||||
**Bad:** "Responsible for AI initiatives"
|
||||
**Good:** "Deployed first Copilot Studio agent handling 40% of internal inquiries"
|
||||
**Good:** "Cut customer-support response time 40% by automating tier-1 triage"
|
||||
|
||||
**Ask the user:** Describe your current role's key achievements with numbers/impact.
|
||||
|
||||
|
|
|
|||
|
|
@ -159,9 +159,11 @@ Show the post with:
|
|||
|
||||
Auto-copy the final post text to clipboard silently:
|
||||
```bash
|
||||
printf '%s' '<FINAL_POST_TEXT>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs
|
||||
node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs <<'__LINKEDIN_CLIP_EOF__'
|
||||
<FINAL_POST_TEXT>
|
||||
__LINKEDIN_CLIP_EOF__
|
||||
```
|
||||
Then confirm: "Copied to clipboard."
|
||||
Substitute `<FINAL_POST_TEXT>` with the exact post text between the heredoc markers (a quoted heredoc keeps apostrophes, `%`, `$`, and backticks literal). Only if the helper prints `COPIED`, confirm: "Copied to clipboard." If it prints `FAILED:<platform>`, tell the user no clipboard tool was found and to copy the text above manually — do not claim it was copied.
|
||||
|
||||
Do NOT proactively offer alternative versions. Only generate alternatives if the user asks.
|
||||
|
||||
|
|
|
|||
|
|
@ -146,9 +146,11 @@ Show:
|
|||
|
||||
Auto-copy the main draft text to clipboard silently:
|
||||
```bash
|
||||
printf '%s' '<MAIN_DRAFT_TEXT>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs
|
||||
node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs <<'__LINKEDIN_CLIP_EOF__'
|
||||
<MAIN_DRAFT_TEXT>
|
||||
__LINKEDIN_CLIP_EOF__
|
||||
```
|
||||
Then confirm: "Copied to clipboard."
|
||||
Substitute `<MAIN_DRAFT_TEXT>` with the exact draft text between the heredoc markers (a quoted heredoc keeps apostrophes, `%`, `$`, and backticks literal). Only if the helper prints `COPIED`, confirm: "Copied to clipboard." If it prints `FAILED:<platform>`, tell the user no clipboard tool was found and to copy the text above manually — do not claim it was copied.
|
||||
|
||||
Do NOT use AskUserQuestion for refinement. Simply state:
|
||||
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ allowed-tools:
|
|||
- Bash
|
||||
- Read
|
||||
- Glob
|
||||
- Write
|
||||
- AskUserQuestion
|
||||
- Task
|
||||
---
|
||||
|
|
@ -69,7 +70,7 @@ Enter your choice:
|
|||
```
|
||||
|
||||
**If monthly (option 2):** Ask for month (YYYY-MM format, default to current month), then jump to **Step 2b**.
|
||||
**If heatmap (option 3):** Run the heatmap CLI command and jump to **Step 6c**.
|
||||
**If heatmap (option 3):** Run the heatmap CLI command and jump to **Step 2c**.
|
||||
**If weekly (option 1 or default):** Continue below.
|
||||
|
||||
### Weekly: Determine Week
|
||||
|
|
|
|||
|
|
@ -314,7 +314,7 @@ Guide through each section of the profile:
|
|||
- "These should be topics you can consistently create content about for 90+ days"
|
||||
|
||||
3. **Target audience:**
|
||||
- "Who is your primary audience? (e.g., 'Public sector leaders exploring AI')"
|
||||
- "Who is your primary audience? (e.g., 'CFOs at mid-size SaaS companies' or 'public-sector IT leaders')"
|
||||
- "Secondary audience?"
|
||||
- "Geographic focus?"
|
||||
|
||||
|
|
@ -336,8 +336,13 @@ Guide through each section of the profile:
|
|||
- "Current follower count?"
|
||||
- "90-day growth goal?"
|
||||
|
||||
7. Read `config/user-profile.template.md` for structure
|
||||
8. Write the completed profile to `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/profile/user-profile.md`
|
||||
7. **Research tooling:**
|
||||
- "Which research MCPs do you have connected? (e.g., Tavily, Gemini deep research, Perplexity — or none)"
|
||||
- Record exactly what they name. If they have none, that's fine — WebSearch + WebFetch are the always-available floor.
|
||||
- This populates the **Research Tooling** section of the profile; the trend/research engine routes to a declared MCP first and falls back to the floor. Do not invent MCP names — store only what the user declares.
|
||||
|
||||
8. Read `config/user-profile.template.md` for structure
|
||||
9. Write the completed profile to `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/profile/user-profile.md`
|
||||
|
||||
**Important:** This file is gitignored (`.local.md` pattern), so personal data stays private.
|
||||
|
||||
|
|
|
|||
|
|
@ -172,9 +172,11 @@ Style: [minimal / branded / text-heavy]
|
|||
|
||||
Auto-copy the POST CAPTION text to clipboard silently:
|
||||
```bash
|
||||
printf '%s' '<POST_CAPTION_TEXT>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs
|
||||
node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs <<'__LINKEDIN_CLIP_EOF__'
|
||||
<POST_CAPTION_TEXT>
|
||||
__LINKEDIN_CLIP_EOF__
|
||||
```
|
||||
Then confirm: "Post caption copied to clipboard."
|
||||
Substitute `<POST_CAPTION_TEXT>` with the exact caption text between the heredoc markers (a quoted heredoc keeps apostrophes, `%`, `$`, and backticks literal). Only if the helper prints `COPIED`, confirm: "Post caption copied to clipboard." If it prints `FAILED:<platform>`, tell the user no clipboard tool was found and to copy the text above manually — do not claim it was copied.
|
||||
|
||||
## Step 7: Refinement Cycle
|
||||
|
||||
|
|
|
|||
|
|
@ -1,18 +0,0 @@
|
|||
# Content History Log
|
||||
|
||||
Tracks all LinkedIn posts created through the plugin. Append-only — never edit existing entries.
|
||||
|
||||
Auto-initialized from `config/content-history.template.md` when the first post is logged.
|
||||
|
||||
## Format
|
||||
|
||||
| Date | Hook | Topic | Format | Words | Chars | Source |
|
||||
|------|------|-------|--------|-------|-------|--------|
|
||||
|
||||
<!-- Entries are appended below by the Stop hook -->
|
||||
<!-- Format: | YYYY-MM-DD | "Hook text (60 chars max)..." | topic_area | post/quick/react/video | word_count | char_count | original/url/curated | -->
|
||||
|
||||
## Content Log
|
||||
|
||||
| Date | Hook | Topic | Format | Words | Chars | Source |
|
||||
|------|------|-------|--------|-------|-------|--------|
|
||||
76
config/trends-sources.template.md
Normal file
76
config/trends-sources.template.md
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
# Trend Sources (template)
|
||||
|
||||
The **source list** a research-engine pass polls for trend/topic candidates. This file
|
||||
ships **generic, niche-agnostic defaults** (source *categories*, not one person's beat).
|
||||
Override it with your own list — the niche specifics (your vendors, your regulators, your
|
||||
country's outlets) belong in the override, never here.
|
||||
|
||||
## How the override works
|
||||
|
||||
Copy this template into your per-user data dir and edit the copy:
|
||||
|
||||
```bash
|
||||
mkdir -p "${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/trends"
|
||||
cp config/trends-sources.template.md \
|
||||
"${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/trends/sources.md"
|
||||
```
|
||||
|
||||
A pass reads `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/trends/sources.md`
|
||||
if it exists, and falls back to these shipped defaults otherwise. The override lives in the
|
||||
data dir (next to the trend store `trends/trends.json`), so it **survives plugin
|
||||
upgrades and reinstalls**. `LINKEDIN_STUDIO_DATA` overrides the root.
|
||||
|
||||
Format below is read by a person and an agent both: a markdown list, one source per line,
|
||||
grouped by tier. Keep `Name — URL — note` so a poll can cite the URL.
|
||||
|
||||
---
|
||||
|
||||
## Tier 1 — Primary / breaking (poll daily, react within 24–48h)
|
||||
|
||||
*First-party announcements and authoritative decisions in your field.*
|
||||
|
||||
- [Major vendor blog] — [url] — releases & announcements
|
||||
- [Major vendor blog] — [url] — releases & announcements
|
||||
- [Regulator / governing body] — [url] — decisions & guidance
|
||||
|
||||
## Tier 2 — Analysis & research (poll 2–3×/week, post within a week)
|
||||
|
||||
*Where a development gets interpreted, not just reported.*
|
||||
|
||||
- [Respected analysis publication] — [url]
|
||||
- [Analyst house report stream] — [url] — adoption / market reports
|
||||
- [Research index] — [url] — preprints / papers in your area
|
||||
|
||||
## Tier 3 — Community signals (poll weekly, post if a pattern emerges)
|
||||
|
||||
*Where practitioners surface what actually matters before the press does.*
|
||||
|
||||
- [Practitioner forum / aggregator] — [url]
|
||||
- [Topic-specific community] — [url]
|
||||
- [Platform-native trending] — [url]
|
||||
|
||||
## Tier 4 — Niche & seasonal (poll monthly, plan ahead)
|
||||
|
||||
*Slower-moving sources with predictable cadence.*
|
||||
|
||||
- [Key conference series] — [url] — announcement / agenda cycles
|
||||
- [Earnings / report calendar] — [url] — scheduled releases
|
||||
- Seasonal themes: [Q1 …] · [Q2 …] · [Q3 …] · [Q4 …]
|
||||
|
||||
---
|
||||
|
||||
## Your niche additions
|
||||
|
||||
Add the sources specific to your field below — these are the ones the generic defaults
|
||||
above cannot know. (Example shape for a Norwegian public-sector / AI niche: national
|
||||
digitalisation agency, data-protection authority, the relevant ministries, the local
|
||||
tech press — replace with yours.)
|
||||
|
||||
- [Your source] — [url] — [why it matters to your audience]
|
||||
- [Your source] — [url] — [why it matters to your audience]
|
||||
|
||||
---
|
||||
|
||||
*Tip: the [8 universal angles](../references/content-angles.md) and the scoring rubric
|
||||
in [trend-scoring-modes.md](../references/trend-scoring-modes.md) decide what to DO with a
|
||||
candidate once a poll surfaces it. This file only decides WHERE to look.*
|
||||
|
|
@ -116,6 +116,27 @@ cp config/user-profile.template.md config/user-profile.local.md
|
|||
|
||||
---
|
||||
|
||||
### Research Tooling
|
||||
|
||||
**Which research MCPs do you have available?** The trend / research engine routes to a
|
||||
declared MCP **first** and falls back to the always-available floor when none is declared.
|
||||
Check what you actually have connected — leave the rest unchecked. (Names are examples;
|
||||
add whatever you have under "Other" — nothing here is hard-coded downstream.)
|
||||
|
||||
- [ ] Tavily (`tavily_search` / `tavily_research`)
|
||||
- [ ] Gemini deep research (`gemini_deep_research`)
|
||||
- [ ] Perplexity
|
||||
- [ ] Other: [name the MCP + what it's good for]
|
||||
|
||||
**Always-available floor (no MCP needed):** WebSearch + WebFetch. These are used when no
|
||||
research MCP is declared above. (Note: WebSearch is US-biased — a research MCP gives
|
||||
better coverage for non-US / regional sources.)
|
||||
|
||||
**Preferred order (optional):** [e.g. "Tavily for niche/regional, Gemini for deep dives,
|
||||
WebSearch as fallback" — or leave blank to let the engine pick]
|
||||
|
||||
---
|
||||
|
||||
### Asset Utilization Preferences
|
||||
|
||||
**When creating content, Claude should:**
|
||||
|
|
|
|||
|
|
@ -93,6 +93,7 @@ that exercises the command's real path:
|
|||
| S7 batch | S14 import | S21 monetize | S28 ref-consistency B |
|
||||
| S8 pipeline | S15 report | S22 outreach | S29 terminology-scrub |
|
||||
| | | | S30 magnitude-scrub |
|
||||
| | | | S31a/b/c multiplier-scrub |
|
||||
|
||||
*S9 newsletter (16-phase) may split into S9a/S9b. Otherwise one command = one session.
|
||||
|
||||
|
|
@ -135,7 +136,9 @@ carries ~45% *correlational engagement gap* at medium confidence, not a 55% reac
|
|||
intact (officially confirmed, high confidence): engagement-pod + AI-slop "penalized" framing.** Full grep
|
||||
catalog in `log.md` S27 entry, Bucket D. Same discipline; hardening-class.
|
||||
|
||||
Run after S26; order adjustable (S27 ✅ → S28 → S29 → S30). These edit already-hardened files surgically and
|
||||
Run after S26; order adjustable (S27 ✅ → S28 ✅ → S29a–e ✅ → S30 ✅ → S31a/b/c ✅ — queue complete, see `log.md`;
|
||||
S31 was cataloged during S30 as the "Nx"-multiplier + descriptive-% class, amendment followed in practice).
|
||||
These edit already-hardened files surgically and
|
||||
are hardening-class (commit local, no push).
|
||||
|
||||
## End-of-session ritual (every session — STATE.md handoff baked in)
|
||||
|
|
|
|||
207
docs/hardening/review-R2a.md
Normal file
207
docs/hardening/review-R2a.md
Normal file
|
|
@ -0,0 +1,207 @@
|
|||
---
|
||||
type: cold-review
|
||||
batch: R2a
|
||||
journey: "Create — atomic emitters"
|
||||
scope: "FROZEN committed files vs HEAD 9567689 (no pending diff; post-hardening cold pass)"
|
||||
method: "2 independent cold Opus reviewers per surface (intent + correctness), no cross-feed; every mechanical claim tool-grounded (anti-fabrication mandate); reviewers carry NO drafting-session context"
|
||||
surfaces: [post, react, carousel, video, multiplatform]
|
||||
reviewers:
|
||||
- "intent-lens (conformance: intent delivery + cross-ref resolution + class predicates + terminology)"
|
||||
- "correctness-lens (internal consistency + bound-vs-canonical + checklist arithmetic + structure)"
|
||||
status: "COMPLETE — all 5 surfaces reviewed (post, react, carousel, video, multiplatform)"
|
||||
verdict: REWORK
|
||||
counts: { BLOCKER: 0, MAJOR_systemic: 1, MAJOR_surface: 1, MINOR: 13, SUGGESTION: 8 }
|
||||
---
|
||||
|
||||
# Cold review — R2a (Create · atomic emitters)
|
||||
|
||||
Independent post-hoc cold review of the 5 atomic Create-journey emitters, on the FROZEN committed
|
||||
files. Mirrors the S1 `review.md` model (the one cold-review method that did **not** fabricate):
|
||||
read-and-show before assert, every `file:line` tool-confirmed. The per-command interactive gate
|
||||
(S2–S26, `log.md`) already passed these; this pass adds the **independent** axis that gate never had.
|
||||
|
||||
**Independence cross-check worked:** on every WAVE-1 surface the two blind lenses converged on the
|
||||
same real defects (personal-stories band in post; "full angle set below" in react; slide-scaffold +
|
||||
slide-count in carousel) — convergence from two no-cross-feed reviewers is the signal that a finding
|
||||
is real, not an artifact of one reviewer's framing.
|
||||
|
||||
---
|
||||
|
||||
## ★ Cross-cutting finding (systemic — spans all 10 content commands)
|
||||
|
||||
### MAJOR (systemic) — `printf '%s' '<TEXT>'` clipboard pattern silently corrupts content containing an apostrophe
|
||||
- **Pattern (verified by main, independent of reviewers):** `printf '%s' '<PLACEHOLDER>' | node ${CLAUDE_PLUGIN_ROOT}/hooks/scripts/clipboard-helper.mjs`, followed by an **unconditional** `Then confirm: "Copied to clipboard."`
|
||||
- **Blast radius — all 10 content-emitting commands** (grep-confirmed): `post.md:153`, `quick.md:162`, `react.md:149`, `carousel.md:211`, `video.md:175`, `multiplatform.md:121`, `pipeline.md:139`, `first-post.md:141`, `firsthour.md:72`, `onboarding.md:221`.
|
||||
- **Mechanism:** the executing model substitutes the draft into the **single-quoted** bash argument. English LinkedIn drafts almost always contain an apostrophe ("it's", "don't", "here's"); a literal `'` terminates the bash string → printf receives word-split fragments → clipboard gets garbled/truncated text (`printf '%s' 'It's a test'` → `Itsatest`). The command gives **no escaping guidance**.
|
||||
- **Why it matters (trust-breaking):** the step runs "silently" then **unconditionally** reports success, so on the most common content shape the clipboard is corrupt while the user is told the copy worked. It is the headline convenience feature of every content command.
|
||||
- **Severity note:** flagged MAJOR (not BLOCKER) — it doesn't crash the session, and a careful executing model *might* escape; but the instruction's default path fails. Independently surfaced by `rev-react-intent` (MAJOR) and `rev-post-intent` (SUGGESTION, "convention-level").
|
||||
- **Disposition (NOT fixed here — review finds, operator decides; 10-file change = own go):** switch the convention to a no-in-content-quoting form — write the draft to a temp file and feed via stdin (`node …/clipboard-helper.mjs < "$tmp"`), or a quoted heredoc. Fix once, consistently, across all 10. **Recommend treating this as the first fix that comes out of R2a.**
|
||||
|
||||
---
|
||||
|
||||
## post.md — VERDICT: REWORK (1 MAJOR · 2 MINOR · 1 SUGGESTION)
|
||||
|
||||
Class: post-emitting (primary) + guided/stateful (8-step). Both lenses confirm intent delivered;
|
||||
all cross-refs resolve (2 agents, 2 routes, 2 scripts, 7 assets — tool-confirmed); no "thought leadership".
|
||||
|
||||
### MAJOR — Step 3 "Personal stories" band (1,000–1,400) contradicts the file's own Step 5 gate + canonical SSOT (1,200–1,800)
|
||||
- `post.md:87` — Step 3 assigns "Personal stories | Medium text post (1,000-1,400 chars)" — a third band present nowhere else.
|
||||
- `post.md:128` — Step 5 quality checklist requires "Character count: 1,200-1,800 (optimal range)" for the same post.
|
||||
- `hooks/prompts/content-quality-gate.md:17` — canonical: "Standard posts: 1,200-1,800". A personal story is a standard text post (not quick 150–500).
|
||||
- **impact:** a personal-story draft written to Step 3 at ~1,050 chars passes Step 3 but FAILS the Step 5 checklist and the save-time quality-gate hook. The command self-contradicts.
|
||||
- **Both lenses flagged this** (rev-post-correct MAJOR, rev-post-intent MINOR). Recorded at the higher severity: it hits a real gate path, not just advisory text.
|
||||
- **disposition:** raise line 87 floor to 1,200–1,800, OR (if shorter personal stories are intentional) push the sub-band to the canonical SSOT first and reconcile `:128` + `content-quality-gate.md:17` — never leave a divergent band only in this file.
|
||||
|
||||
### MINOR — Step 4 component minimums sum below the 1,200 optimal floor
|
||||
- `post.md:100-104` — Hook 110-140 + Context 200-300 + Insight 400-800 + Implication 200-300 + CTA 50-100. Minimums sum to **960** (110+200+400+200+50); maximums to 1,640.
|
||||
- **impact:** following every section at its minimum yields a 960-char post, below the 1,200 floor asserted at `:7/:86/:128` and canonical `:17`. Loose guidance, not a hard gate, but a writer hugging the low end lands under-length.
|
||||
- **disposition:** accept as-is, or lift Insight/Context minimums so the component floor reaches ~1,200.
|
||||
|
||||
### SUGGESTION — clipboard apostrophe breakage → see ★ cross-cutting finding (`post.md:153`).
|
||||
|
||||
---
|
||||
|
||||
## react.md — VERDICT: REWORK (1 MAJOR · 3 MINOR · 1 SUGGESTION)
|
||||
|
||||
Class: post-emitting + graceful-degradation on bad/empty URL (delivered, `react.md:60,194`). Intent
|
||||
(URL→post pipeline) delivered; all 7 cross-refs resolve; no "thought leadership".
|
||||
|
||||
### MAJOR — clipboard apostrophe breakage → see ★ cross-cutting finding (`react.md:148-151`).
|
||||
(Originally surfaced here by rev-react-intent; promoted to the cross-cutting section.)
|
||||
|
||||
### MINOR — `/linkedin:summarize` trigger has no backing command
|
||||
- `react.md:9` — the frontmatter description lists "/linkedin:summarize" among triggers. `ls commands/summarize.md` → does not exist; not among the 29 commands.
|
||||
- **impact:** advertises a slash-command alias that resolves to nothing; a user typing it gets no command. Dead/aspirational trigger.
|
||||
- **disposition:** remove `/linkedin:summarize` from the trigger list, or add a summarize alias command.
|
||||
|
||||
### MINOR — "the full angle set below" is a dead in-file locator
|
||||
- `react.md:95` — "present 2-3 alternatives from **the full angle set below**." No enumerated full angle set appears below this line; Step 4's table (`:81-86`) lists only 4 preferred + 4 fallback; the 8 universal angles live in `references/content-angles.md` (`:273`), not "below."
|
||||
- **Both lenses flagged this** (rev-react-intent + rev-react-correct).
|
||||
- **impact:** dangling locator on the "try a different angle" path; the model must guess "below" means the reference file.
|
||||
- **disposition:** change "the full angle set below" → "the 8 universal angles in `references/content-angles.md`".
|
||||
|
||||
### MINOR — "medium post" label diverges from canonical "standard" band
|
||||
- `react.md:119` — "Character target: 1,200-1,800 chars (medium post)". The **number is correct** (matches canonical standard band `content-quality-gate.md:17` + CLAUDE.md rule 2), but canonical/CLAUDE.md label it "standard"; no "medium" tier is defined anywhere.
|
||||
- **impact:** cosmetic; the active length gate is unaffected. Risk is reader confusion that a separate "medium" tier exists.
|
||||
- **disposition:** accept as-is, or relabel "(standard post)".
|
||||
|
||||
---
|
||||
|
||||
## carousel.md — VERDICT: ALLOW (0 BLOCKER/MAJOR · 5 MINOR · 4 SUGGESTION)
|
||||
|
||||
Class: guided/stateful (content-emitting). Intent (5–8 slide deck + caption, optional image render,
|
||||
text-only degradation) delivered; all cross-refs resolve (differentiation-checker, templates,
|
||||
algorithm-signals, clipboard + state scripts, mcp-image params valid); hook bound `:97` matches SSOT;
|
||||
no "thought leadership". Clean on all blocking dimensions — findings are polish.
|
||||
|
||||
### MINOR — slide-count minimum stated two ways (6 vs 5)
|
||||
- `carousel.md:41-45` — Step 1 offers all 5 templates as "(6-8 slides)"; `carousel.md:114` — Step 5 gate checks "5-8 slides total (7 is optimal)". Minimum disagrees (6 vs 5).
|
||||
- **Both lenses flagged this.** Mirrors the upstream split in `assets/templates/carousel-templates.md:11` ("5-8") vs per-template headers ("6-8").
|
||||
- **impact:** a 5-slide deck passes Step 5 but was never offered in Step 1. Cosmetic guidance, not a hard break.
|
||||
- **disposition:** align the floor (pick 5 or 6 across Step 1 + Step 5); ideally fix the source too.
|
||||
|
||||
### MINOR — inlined Step 5 checklist drops an item vs the source it cites (7 vs 8)
|
||||
- `carousel.md:109` says "Run against the Carousel Quality Checklist from carousel-templates.md", then inlines 7 items (`:111-117`, `grep -c` = 7). The cited source has 8 (`carousel-templates.md:276-283`); the dropped one is `:283` "Exported as PDF, under 100 MB".
|
||||
- **impact:** the export/size check only resurfaces in Step 6's text-only branch (`:184`); when image generation succeeds, the PDF/100 MB constraint is never surfaced in the gate.
|
||||
- **disposition:** add the PDF/under-100 MB item to the Step 5 list, or stop claiming verbatim fidelity to the source.
|
||||
|
||||
### MINOR — slide body scaffold provides 5 line-slots but the rule permits up to 7
|
||||
- `carousel.md:73-77` models 5 BODY lines (4-5 optional); `carousel.md:87` + `carousel-templates.md:10` permit "5-7 lines".
|
||||
- **Both lenses flagged this.**
|
||||
- **impact:** a slide legitimately needing 6-7 lines has no scaffold slot; the literal template caps generation at 5.
|
||||
- **disposition:** extend the scaffold to 7 optional lines, or tighten the rule/template to "max 5".
|
||||
|
||||
### MINOR — caption voice-guardian safety-net claim doesn't engage in this flow
|
||||
- `carousel.md:123` — "(The voice-guardian hook scores the caption on save.)" The PreToolUse gate fires only on Write|Edit of LinkedIn content, but this command never Writes the caption — Step 6 pipes it to clipboard (`:210-212`), Step 7 mutates state via `node -e` (`:222-231`). No save → hook never scores the caption.
|
||||
- **impact:** overstates a backstop that doesn't fire here; could justify under-doing the in-command De-AI check (Step 5).
|
||||
- **disposition:** drop the parenthetical or qualify it ("only if you later save the caption to a file").
|
||||
|
||||
### MINOR — no-external-link rule (Content Quality Rule #3) absent from caption guidance
|
||||
- `carousel.md:93-105, 121-125` — the caption is feed text on the same reach mechanics, but neither Step 4 nor the De-AI gate mentions the no-body-link rule.
|
||||
- **impact:** a caption with an inline link incurs the documented reach penalty with nothing in this surface catching it.
|
||||
- **disposition:** add "no links in the caption body (put links in first comment)" to the De-AI gate or Step 4.
|
||||
|
||||
### SUGGESTION — orphan provenance comment for an unused capability
|
||||
- `carousel.md:18` — `<!-- MERMAID_CHART_RESOLUTION: UNTESTED -->`. Mermaid is never referenced anywhere in the file (only mcp-image is used).
|
||||
- **disposition:** remove the vestigial marker (or move the note to a design doc).
|
||||
|
||||
### SUGGESTION — dual slide-count framing (6-8 vs 5-8); locally-defined per-slide bounds (header "max 8 words" `:70`, body "max 50 chars" `:73-77`, no canonical SSOT — no overlap with post bounds, noted for completeness); buzzword list 8 words `:123` faithful to CLAUDE.md Rule #4 while canonical hook flags 10 (gap lives between CLAUDE.md + hook, not in this file).
|
||||
|
||||
---
|
||||
|
||||
## video.md — VERDICT: ALLOW (0 BLOCKER/MAJOR · 3 MINOR · 1 SUGGESTION)
|
||||
|
||||
Class: guided/stateful (8-step script build) + post-emitting sub-surface (the 200–400 char caption).
|
||||
Intent (paced 30s/60s/90s/2min video script + on-video captions + thumbnail + post caption + first
|
||||
comment, delegated to `video-scripter`) delivered; `video-scripter` + `differentiation-checker`
|
||||
resolve; word-budget math internally consistent (30/60/90/120s × 2.5 wps = 75/150/225/300, `:64-67`);
|
||||
no "thought leadership".
|
||||
|
||||
### MINOR — muted-watch statistic stated two ways
|
||||
- `video.md:100` "~85% watch without sound" vs `video.md:120` "~80–85% watch muted" — same claim, two figures in one frozen file.
|
||||
- **Both lenses flagged this.** 85% sits inside 80–85% so not a hard contradiction, but reads as unreviewed precision in a quality-gate checklist.
|
||||
- **disposition:** pick one figure (recommend "~80–85%") in both places.
|
||||
|
||||
### MINOR — post caption (200–400) is a third length band vs canonical quick (150–500)
|
||||
- `video.md:121,158` specify "200-400 chars"; `content-quality-gate.md:18` quick = 150–500. 200–400 is a narrower subset (no hard conflict) but a band not present in the SSOT.
|
||||
- **impact:** a reader can't tell from video.md whether 200–400 is intentional or drift.
|
||||
- **disposition:** accept if intentional (captions deliberately shorter), but add a one-line note that 200–400 is a deliberate sub-band of the 150–500 quick range.
|
||||
|
||||
### MINOR — caption has no mobile-cutoff / first-line hook discipline
|
||||
- `video.md:122,129,156-158` — the caption (feed text, truncated at the same "...see more" cutoff as any post) gets a length band + De-AI + no-body-link + buzzword strip, but NO instruction to front-load value within the ~110–140 cutoff. (`:101` "first line reads on-screen" is the muted-autoplay test on the VIDEO's on-screen text, not the feed caption.)
|
||||
- **impact:** vs text-post parity (SSOT hook 110–140) the caption's truncation is unguarded; a buried lede underperforms in-feed.
|
||||
- **disposition:** defensible to accept (video is primary content), or add "caption's first line should land value before the mobile cutoff".
|
||||
|
||||
### SUGGESTION — clipboard apostrophe breakage → see ★ cross-cutting finding (`video.md:175`).
|
||||
|
||||
---
|
||||
|
||||
## multiplatform.md — VERDICT: ALLOW (0 BLOCKER/MAJOR · 1 MINOR · 3 SUGGESTION)
|
||||
|
||||
Class: routing + guided/stateful (content-adaptation). **post-emitting predicate does NOT apply** —
|
||||
this command consumes a LinkedIn post and emits adaptations for OTHER platforms (Twitter/slides/
|
||||
YouTube), so the LinkedIn quality-gate (hook 110–140 / length band / no-body-link / topic→pillars) is
|
||||
out of scope; the buzzword check IS carried (`:46-48`, mirrors CLAUDE.md rule #4 exactly). Intent
|
||||
delivered (3 promised platforms = 3 AskUserQuestion options = 3 templates); routing resolves
|
||||
(`/linkedin:newsletter` at `:6,:34,:36,:132` → `commands/newsletter.md` exists); no subagent refs;
|
||||
graceful degradation present (`:27-29`, forbids fabricating source); no "thought leadership".
|
||||
|
||||
### MINOR — clipboard apostrophe breakage → see ★ cross-cutting finding (`multiplatform.md:121`)
|
||||
- Elevated exposure noted: Twitter threads + YouTube CTAs are contraction-heavy (don't/it's/I'll), so this surface is *more* exposed to the systemic bug than most.
|
||||
|
||||
### SUGGESTION — Twitter "280 chars max" is locally-defined (no LinkedIn SSOT)
|
||||
- `multiplatform.md:53` — the only numeric bound in the file; an X/Twitter limit, correct value, cannot diverge from the LinkedIn SSOT. Noted for completeness.
|
||||
|
||||
### SUGGESTION — "write once, publish everywhere" tagline overstates the command
|
||||
- `multiplatform.md:4-5` — the tagline implies publishing; the command only adapts + saves to drafts (`:118`) + copies to clipboard. No publish action exists.
|
||||
- **disposition:** accept, or soften to "adapt once, post everywhere" / "draft for every platform".
|
||||
|
||||
### SUGGESTION — Step 1 always asks platform even when the trigger already names it
|
||||
- `multiplatform.md:38-41` unconditionally invokes AskUserQuestion, yet triggers include "adapt for twitter"/"turn into thread" (`:7`) that already pin the platform — against the commands-section principle to minimize interactive steps / infer from context.
|
||||
- **disposition:** accept, or "skip if the platform is already evident from the user's request".
|
||||
|
||||
---
|
||||
|
||||
## Gate decision — R2a COMPLETE (5 surfaces)
|
||||
|
||||
| Surface | Verdict | BLOCKER | MAJOR | MINOR | SUGGESTION |
|
||||
|---|---|---|---|---|---|
|
||||
| post | REWORK | 0 | 1 (+systemic) | 1 | — |
|
||||
| react | REWORK | 0 | (systemic) | 3 | — |
|
||||
| carousel | ALLOW | 0 | 0 | 5 | 4 |
|
||||
| video | ALLOW | 0 | 0 | 3 | 1 |
|
||||
| multiplatform | ALLOW | 0 | 0 | 1 | 3 |
|
||||
| **★ cross-cutting** | — | 0 | **1 (systemic, 10 files)** | — | — |
|
||||
|
||||
**Batch verdict: REWORK** — 2 of 5 surfaces (post, react), driven by **2 distinct MAJORs**:
|
||||
(1) the systemic clipboard-`printf` corruption (10 content commands), and (2) post's personal-stories
|
||||
band contradicting its own Step 5 gate + the canonical SSOT. **0 BLOCKER anywhere.** carousel / video /
|
||||
multiplatform are ALLOW (polish only).
|
||||
|
||||
**Independence verdict:** every WAVE-1 REWORK/MINOR was independently surfaced by **both** blind lenses
|
||||
(clipboard, personal-stories band, "full angle set below", slide-scaffold, slide-count, muted-stat) —
|
||||
the convergence signal that these are real defects, not single-reviewer framing.
|
||||
|
||||
Cold review **finds**; it changes no code. Each fix is its own operator-gated decision. Recommended
|
||||
first fix out of R2a: the systemic clipboard pattern (one change, 10 files, highest blast radius).
|
||||
Local-only (hardening-class), not pushed.
|
||||
281
docs/hardening/review-R2b.md
Normal file
281
docs/hardening/review-R2b.md
Normal file
|
|
@ -0,0 +1,281 @@
|
|||
---
|
||||
type: cold-review
|
||||
batch: R2b
|
||||
journey: "Create — orchestrators & front-door"
|
||||
scope: "FROZEN committed files at HEAD 5474df5 (clean tree; post-hardening cold pass)"
|
||||
method: "2 independent cold Opus reviewers per surface (intent + correctness), no cross-feed; every mechanical claim tool-grounded (anti-fabrication mandate); reviewers carry NO drafting-session context"
|
||||
surfaces: [create, batch, pipeline, newsletter]
|
||||
reviewers:
|
||||
- "intent-lens (conformance: intent delivery + cross-ref resolution + class predicates + terminology)"
|
||||
- "correctness-lens (internal consistency + bound-vs-canonical + checklist arithmetic + structure)"
|
||||
status: "COMPLETE — all 4 surfaces reviewed (create, batch, pipeline, newsletter)"
|
||||
verdict: REWORK
|
||||
counts: { BLOCKER: 0, MAJOR: 3, MINOR: 10, SUGGESTION: 6, systemic_patterns: 2 }
|
||||
---
|
||||
|
||||
# Cold review — R2b (Create · orchestrators & front-door)
|
||||
|
||||
Independent post-hoc cold review of the 4 Create-journey orchestrators/front-door, on the FROZEN
|
||||
committed files (HEAD `5474df5`). Mirrors the S1 `review.md` + R2a model (the cold-review method that
|
||||
did **not** fabricate): read-and-show before assert, every `file:line` tool-confirmed, reviewers carry
|
||||
no drafting-session context. The per-command interactive gate (`log.md`) already passed these; this
|
||||
pass adds the **independent** axis that gate never had.
|
||||
|
||||
**Independence cross-check — two outcomes worth noting this batch:**
|
||||
1. **Convergence** (the R2a pattern): both blind lenses independently surfaced the same real defect on
|
||||
several surfaces — batch's bare-vs-prefixed reference path, batch's 3a/3b component-band tension,
|
||||
newsletter's `allowed-tools` omission, create's 8-option `AskUserQuestion`.
|
||||
2. **Divergence resolved by grounding** (new this batch, the strongest argument FOR the two-lens
|
||||
method): on `newsletter` the intent-lens **asserted** the resumption table "maps every currentPhase
|
||||
to the correct resume step"; the correctness-lens **counted** the rows (17) against the canonical
|
||||
`_doc.phases` (18) and found the `contract-gate` row missing. Main re-grounded it independently
|
||||
(below) → the correctness-lens is right. The independence axis caught a real MAJOR that one lens had
|
||||
asserted away.
|
||||
|
||||
---
|
||||
|
||||
## ★ Cross-cutting finding #1 (systemic) — 5-component draft scaffold (960–1,640) cannot satisfy the 1,200–1,800 standard band it is gated against
|
||||
|
||||
- **Pattern (verified by main, independent of reviewers — `grep` blast radius):** the identical
|
||||
5-component "standard post" breakdown — `Hook 110-140` + `Context 200-300` + `Insight 400-800` +
|
||||
`Implication 200-300` + `CTA 50-100` — appears in **exactly 3 files**: `post.md:100-104`,
|
||||
`batch.md:88-92`, `pipeline.md:58-62`. (`grep -rln "Insight.*400" commands/` → those three only;
|
||||
`quick`/`first-post`/`react` carry the lone `Hook 110-140` line, NOT the full scaffold, so are
|
||||
unaffected.)
|
||||
- **Grounded arithmetic:** component **min-sum = 110+200+400+200+50 = 960** · **max-sum =
|
||||
140+300+800+300+100 = 1,640**. Each file then gates the SAME post against the canonical standard band
|
||||
**1,200–1,800** (`content-quality-gate.md:17`; `post.md:128`, `batch.md:95`, `pipeline.md:77`).
|
||||
- **Mechanism (two-sided):** (a) a draft built at the component minimums is **960 chars — 240 below**
|
||||
the 1,200 floor enforced by the next step AND the live PreToolUse `content-quality-gate` hook;
|
||||
(b) the component maximums sum to **1,640 — 160 below** the 1,800 ceiling, so the scaffold literally
|
||||
cannot reach the upper half of its own target band.
|
||||
- **Why it matters:** on the most common path (drafting a standard text post) the drafting recipe and
|
||||
the acceptance test are mutually incompatible, with no transition/connective slack to close the
|
||||
960→1,200 gap. In `batch` the defect is sharper: the 3a scaffold is **format-blind** (always the
|
||||
5-component standard layout) while the 3b gate is **format-aware** (`batch.md:114` `format:
|
||||
[text/carousel/video]`, rotation `:76`), so a `quick`-format post built from the scaffold (960–1,640)
|
||||
also blows the quick band (150–500). In `pipeline` the contradiction is between two **adjacent**
|
||||
steps (Step 2 draft → Step 3 scorecard).
|
||||
- **Severity:** recorded **MAJOR** in `batch` and `pipeline` (real adjacent-step / gate contradiction
|
||||
on every standard post). **Reconciliation note for R2a:** R2a recorded the same scaffold in `post.md`
|
||||
at **MINOR** ("component minimums sum below the 1,200 optimal floor") and flagged only the min-side.
|
||||
The batch/pipeline analysis shows it is a two-sided, gate-hitting contradiction, not merely loose
|
||||
guidance — the post.md instance is arguably under-rated and should be reconciled in the same fix.
|
||||
- **Disposition (NOT fixed here — review finds, operator decides; 3-file change = own go):** one
|
||||
consolidated fix across all 3 files — raise the component floors so min-sum ≥ 1,200 (e.g. widen
|
||||
Context/Insight) AND either lift the ceiling or accept ~1,640, AND scope the standard 5-component
|
||||
scaffold explicitly to standard-format posts where the command is format-aware (`batch`), adding
|
||||
per-format draft guidance for quick/carousel/video. **Recommend treating this as the second
|
||||
consolidated fix out of the sweep, after the R2a clipboard fix.**
|
||||
|
||||
## ★ Cross-cutting finding #2 (recurring) — bare relative reference paths vs `${CLAUDE_PLUGIN_ROOT}/`
|
||||
|
||||
- **Pattern:** several `Read`/reference paths are written **bare** (resolved against the runtime cwd)
|
||||
while the same file's appendix and most other sites prefix `${CLAUDE_PLUGIN_ROOT}/`. `batch.md:43`
|
||||
(`references/content-angles.md`) vs `batch.md:206` (prefixed, same file) — flagged by **both** batch
|
||||
lenses. `pipeline.md:31,55,70,75` bare vs `pipeline.md:28,64,107` + appendix `:204-209` prefixed —
|
||||
`content-angles.md` is bare at `:55` but prefixed at `:204`.
|
||||
- **Impact:** the files exist (not dead refs), but a bare path fails the `Read` when cwd ≠ plugin root,
|
||||
on real paths run every invocation (angle-select, optimize). Self-recoverable via Glob, latent.
|
||||
- **Disposition:** normalize all bare reference/asset paths to `${CLAUDE_PLUGIN_ROOT}/`. Cheap,
|
||||
mechanical; fold into the consolidated fix pass. (Worth a repo-wide grep for the same pattern in the
|
||||
other 25 commands during the eventual fix.)
|
||||
|
||||
---
|
||||
|
||||
## create.md — VERDICT: ALLOW (0 BLOCKER/MAJOR · 1 MINOR · 0 SUGGESTION)
|
||||
|
||||
Class: **routing** (pure delegating front-door). Both lenses confirm intent delivered: Step 0 context →
|
||||
Step 1 intent-ID → Step 2 route, with explicit "you do not draft here / do NOT inline the target's
|
||||
steps" (`create.md:49-51,64-65`) — delegation purity intact, single source of truth preserved. All 8
|
||||
routed targets resolve (`post/quick/react/carousel/video/multiplatform/batch/newsletter`, `:37-44` +
|
||||
`:55-62`, `ls`-confirmed); the three enumerations (description `:7`, menu `:37-44`, route table
|
||||
`:55-62`) are mutually consistent (8/8/8, same order); no `subagent_type` refs (correct — it routes to
|
||||
commands); no "thought leadership". Correctness-lens: **0 findings**.
|
||||
|
||||
### MINOR — Step 1 directs one `AskUserQuestion` carrying 8 options; documented support is 2–4
|
||||
- `create.md:35-44` — "use `AskUserQuestion`" immediately followed by 8 numbered options (`grep -cE
|
||||
"^[0-9]+\. \*\*"` → 8). Grounded against the plugin-dev reference
|
||||
`command-development/.../interactive-commands.md:469` ("2-4 options per question") + `:906`.
|
||||
- **impact:** on the PRIMARY interactive path (user names no format) the front-door instructs a single
|
||||
question with double the documented option range.
|
||||
- **anti-fabrication caveat (carried from the reviewer, honestly):** grounded = (a) 8 options
|
||||
instructed, (b) the documented 2–4 range. NOT grounded = whether the live `AskUserQuestion` runtime
|
||||
hard-rejects >4 vs silently truncates/degrades. **If the runtime hard-rejects, this escalates to
|
||||
MAJOR/BLOCKER** on the no-format-named path; if it only degrades, the picker is over-long. Worth a
|
||||
runtime check before the fix.
|
||||
- **disposition:** group the 8 intents into ≤4 options (e.g. Short-form / Reaction / Visual / Long-form
|
||||
& batch) with a drill-down, or split into two questions.
|
||||
|
||||
---
|
||||
|
||||
## batch.md — VERDICT: REWORK (0 BLOCKER · 1 MAJOR · 3 MINOR · 2 SUGGESTION)
|
||||
|
||||
Class: **guided/stateful + routing** ("create a full week of content"). Intent delivered: the Step 0→5
|
||||
flow traces the frontmatter promise; all cross-refs resolve (`trend-spotter`, `content-planner` →
|
||||
`agents/`; `/linkedin:calendar` → `commands/`; `queue-manager.mjs` + `ical-generator.mjs` exports +
|
||||
the 8-arg `queueAdd` call/signature match; all 6 reference/asset paths + `SKILL.md`); graceful
|
||||
degradation present; no "thought leadership".
|
||||
|
||||
### MAJOR — Step 3a component scaffold contradicts the Step 3b band gate → see ★ cross-cutting #1 (`batch.md:88-92` vs `:95`)
|
||||
Sharper here than elsewhere: 3a is **format-blind** (always the 5-component standard layout, 960–1,640)
|
||||
while 3b is **format-aware** (`:114` `format:[text/carousel/video]`, rotation `:76`), so a `quick`-format
|
||||
post built from 3a also blows the quick band 150–500. **Both lenses flagged this** (intent-lens MINOR,
|
||||
correctness-lens MAJOR — recorded at the higher severity: it hits a real gate on every standard post).
|
||||
|
||||
### MINOR — bare reference path → see ★ cross-cutting #2 (`batch.md:43` vs `:206`). Both lenses.
|
||||
|
||||
### MINOR — `weekly_goal` cadence decoupled from the fixed "3–5 posts" headline
|
||||
- `batch.md:5,65` fix the output at "3–5 posts"; `batch.md:52` schedules against `weekly_goal` slot
|
||||
templates (2x/3x/4x/5x). At `weekly_goal=2x`, 3–5 posts against 2 weekly slots overflow into the next
|
||||
week (`:53` "next available slot after today") — ~2.5 weeks of content under a "full week" label.
|
||||
- **impact:** non-breaking (scheduling rolls forward), but "full week" + "3–5" is internally
|
||||
inconsistent with the 2x cadence.
|
||||
- **disposition:** tie post count to `weekly_goal`, or note that overflow rolls into following weeks.
|
||||
|
||||
### MINOR — orphan sub-step marker `5b` with no `5a`
|
||||
- `batch.md:172` `### 5b. Generate Calendar File`; `grep -n "5a" batch.md` → no match (exit 1). Step 3
|
||||
has 3a/3b/3c/3d; Step 5 jumps straight to 5b.
|
||||
- **disposition:** renumber to `5a`, or drop the letter.
|
||||
|
||||
### SUGGESTION — `weekly_goal` default (3x) lives only in the referenced `scheduling-strategy.md:15`, never stated in `batch.md`. Optional one-line "default 3x" for self-evident degradation.
|
||||
|
||||
### SUGGESTION — `planned_date` metadata never computed
|
||||
- `batch.md:109` writes `planned_date: YYYY-MM-DD` into each draft header, but Step 2 only computes
|
||||
`scheduled_date`/`scheduled_time` (`:54`); `planned_date` is introduced nowhere upstream.
|
||||
- **disposition:** drop `planned_date`, or define where it is derived.
|
||||
|
||||
---
|
||||
|
||||
## pipeline.md — VERDICT: REWORK (0 BLOCKER · 1 MAJOR · 4 MINOR · 2 SUGGESTION + clipboard pointer)
|
||||
|
||||
Class: **post-emitting + guided/stateful + routing** ("full end-to-end pipeline"). Intent delivered:
|
||||
Steps 0–8 map to every named stage. All post-emitting predicates present (hook 110–140 `:58,:76` ·
|
||||
length band `:77` · no-body-link `:78` · buzzword check `:79` · topic→expertise `:49,:55,:80`); all
|
||||
cross-refs + function signatures resolve (`content-planner`, `trend-spotter`; `/linkedin:calendar`,
|
||||
`/linkedin:analyze`; `queueAdd` 8-arg call/signature; `writeState`/`updatePostTracking`); no "thought
|
||||
leadership".
|
||||
|
||||
### MAJOR — Step 2 component scaffold cannot satisfy the Step 3 total-length gate → see ★ cross-cutting #1 (`pipeline.md:58-62` vs `:77`)
|
||||
Adjacent-step contradiction: Step 2 partitions into 960–1,640; the very next step's scorecard asserts
|
||||
"Total 1,200–1,800". Correctness-lens, grounded arithmetic.
|
||||
|
||||
### MINOR — inline buzzword checklist enumerates 8, canonical gate enumerates 10
|
||||
- `pipeline.md:79` lists 8 terms (= CLAUDE.md rule 4); SSOT `content-quality-gate.md:13` adds
|
||||
'actionable insights' + 'best practices' = 10. A draft passing the inline list can still trip the
|
||||
Write hook. (Gate-vs-rule divergence, not unique to this file — also noted on carousel in R2a.)
|
||||
- **disposition:** align to the 10-term canonical list, or reference the gate instead of duplicating.
|
||||
|
||||
### MINOR — bare reference paths → see ★ cross-cutting #2 (`pipeline.md:31,55,70,75`). Intent-lens.
|
||||
|
||||
### MINOR — Step 4 deferred/queued path falls through into the immediate Publish steps
|
||||
- `pipeline.md:97-101` offers "Schedule / Add to queue / Save as draft (no schedule)"; Steps 5–8
|
||||
(`:120` "15-20 min BEFORE posting", `:135` Publish, `:158` first-hour, `:171` post-analysis) then run
|
||||
with **no branch**. A user who queued/deferred is marched through Pre-Engagement → Publish →
|
||||
Monitoring, contradicting the just-made defer decision.
|
||||
- **disposition:** add an early-exit after Step 4 for options 2–4 ("if scheduled/queued, end here; Steps
|
||||
5–8 run at publish time").
|
||||
|
||||
### MINOR — Step 7 inlines a static first-hour checklist instead of routing to the stateful surface
|
||||
- `pipeline.md:158-169` inlines a 5-item plan; the dedicated `/linkedin:firsthour` delegates to
|
||||
`engagement-coach`, persists via `recordFirstHourPlan`, hands off to `post-feedback-monitor` — strictly
|
||||
richer (Step 8 already routes to `/linkedin:analyze`, so the inline first-hour is the inconsistent one).
|
||||
- **disposition:** route to `/linkedin:firsthour`.
|
||||
|
||||
### SUGGESTION — over-provisioned `allowed-tools`: `:13` declares `WebFetch` but no body step fetches (trend-spotter does its own). Drop unless a URL-ingest step is intended.
|
||||
|
||||
### SUGGESTION — hardcoded Norwegian peak times: `:92-95` bakes "European/Norwegian audience" peak windows into the body while `scheduling-strategy.md` (read at `:107`) is the SSOT for slots; conflicts with the domain/audience-general principle. Source from the reference/config.
|
||||
|
||||
### (pointer) clipboard `printf '%s'` systemic bug — `pipeline.md:139` confirmed present (the only R2b surface in the 10-content-command set). Folds into the R2a ★ cross-cutting clipboard finding; no new derivation.
|
||||
|
||||
---
|
||||
|
||||
## newsletter.md — VERDICT: REWORK (0 BLOCKER · 1 MAJOR · 2 MINOR · 2 SUGGESTION)
|
||||
|
||||
Class: **guided/stateful + routing + heavy subagent orchestration** (long-form 18-phase pipeline,
|
||||
~110 KB). Intent delivered: all 18 phases present, ordered, `[GATE]`/`[OPERATØR]`-marked. **18-phase
|
||||
count confirmed by both lenses** (`0,1,1.5,2,2.5,3a,3b,4,4.5,5,5.5,6,6.5,7,7.5,8,9,10`; headline `:25`
|
||||
matches body + template `_doc.phases` + build-status). All 7 longform agents (fact-checker,
|
||||
editorial-reviewer, persona-reviewer, voice-scrubber, content-reviewer, language-reviewer, fact-reviewer)
|
||||
present in `agents/` AND invoked; gate sequence ordered before lock (`:1570`): skeleton 2.5 → spine 3a →
|
||||
fact-check 5 → editorial 5.5 → persona 6 → headless 6.5 → visual 7.5 → LOCK 8 → hook 9. All `subagent_type`
|
||||
carry the `linkedin-studio:` namespace (the 5 prefix-less grep hits are line-wraps). All ~25 cross-refs
|
||||
(agents, commands, scripts, configs, render, docs) resolve. Pivot heuristic, flag caps, step-label
|
||||
5.5/6.5 consistency all clean. No "thought leadership".
|
||||
|
||||
### MAJOR — deterministic resumption table omits the contract-gate phase (Step 4.5) → breaks resume between Step 4 and Step 5
|
||||
- **Verified by main (independent re-grounding of a lens disagreement):** the resumption table
|
||||
`newsletter.md:209-228` has **no `contract-gate` row** (`grep contract-gate` over the table region →
|
||||
none). The canonical `_doc.phases` it claims to mirror (`:230-231`) **does** define it —
|
||||
`config/edition-state.template.json` lists `"contract-gate — … (Step 4.5)"` between
|
||||
`consistency-quality` (Step 4) and `factcheck-sweep` (Step 5). Step 4.5 actually writes it:
|
||||
`newsletter.md:988` "Set `currentPhase: "contract-gate"`".
|
||||
- **Two concrete breakages on the multi-session resume path (the file's core premise, `:200-204`):**
|
||||
1. **Gate skipped on resume.** The rule (`:203-204`) is "run the step AFTER the recorded phase." Row
|
||||
`:219` maps `consistency-quality → Step 5` (Fact-check), but the step after Step 4 is Step 4.5
|
||||
(contract-gate), not Step 5. A session aborting after Step 4 resumes **past** the deterministic
|
||||
contract-gate, never running it.
|
||||
2. **Unrecognized phase on resume.** A session aborting after Step 4.5 has `currentPhase:
|
||||
"contract-gate"`, absent from the table → falls into the `:232-234` fallback ("missing or
|
||||
unrecognized → do NOT guess … confirm with the operator"), defeating the deterministic-resumption
|
||||
guarantee the section is built on.
|
||||
- **Note:** the linear next-pointers are correct (`:918` "next: contract-gate", `:989` "next:
|
||||
fact-check"); only the resume **table** is short one row — the defect surfaces solely on abort/resume
|
||||
between Step 4 and Step 5.
|
||||
- **Independence note:** the intent-lens asserted this table "maps every currentPhase to the correct
|
||||
resume step"; the correctness-lens counted (17 rows vs 18 phases) and found the gap. Main confirmed
|
||||
the correctness-lens. Two-lens method earned its keep here.
|
||||
- **disposition:** insert a `contract-gate → Step 5 — Fact-check sweep` row, and repoint
|
||||
`consistency-quality → Step 4.5 — Contract-gate`.
|
||||
|
||||
### MINOR — Step 1 says the brief is first persisted "in Step 2"; the rest of the file says Step 1.5
|
||||
- `newsletter.md:303` "Record the resolved brief inline (you will persist it to edition-state in **Step
|
||||
2**)" contradicts `:287-289`, `:412-418`, `:494-496` (all: first durable write is the **Step 1.5**
|
||||
checkpoint). Stale "Step 2" — almost certainly predates the Fix #2 Step 1.5 insertion; non-breaking
|
||||
(Step 1 only records inline either way).
|
||||
- **disposition:** change `:303` "in Step 2" → "at the Step 1.5 checkpoint".
|
||||
|
||||
### MINOR — `allowed-tools` omits `SendUserFile` (body-primary operator gate) + `mcp__mcp-image__generate_image` (default image route)
|
||||
- `newsletter.md:11-19` declares `Read, Glob, Grep, WebFetch, Bash, AskUserQuestion, Task, Write`. The
|
||||
body names `SendUserFile` as the **primary** operator gate at Steps 5.5/6.5/7.5 (13 uses, e.g. `:1138`)
|
||||
and mcp-image as the **default** image route (`:1471`); neither is declared. **Both lenses flagged
|
||||
this** (intent-lens MINOR, correctness-lens SUGGESTION — recorded at the higher: the declared "default"
|
||||
path can't execute under the frontmatter as written). Every use guards with a fallback ("`SendUserFile`
|
||||
if available, else a markdown `file://` link"), so it degrades gracefully → not load-bearing.
|
||||
- **disposition:** add `SendUserFile` (+ optionally mcp-image) to `allowed-tools`, or downgrade the body
|
||||
wording from "default/primary" to "if permitted".
|
||||
|
||||
### SUGGESTION — undefined "LTL plugin" acronym: `newsletter.md:36,725` ("the LTL plugin" / "the LTL rule"); repo-wide the bare phrase appears only here, no expansion; the plugin is canonically "LinkedIn Studio". (The env vars `LTL_SERIES_ROOT`/`LTL_BRAND` `:48,154-156` ARE a legit convention consumed by `render/build-*.mjs` — not a defect.) Rename to "LinkedIn Studio plugin", or define once.
|
||||
|
||||
### SUGGESTION — "leveraged" in doc prose (`newsletter.md:1785`, note-only): ordinary verb in the command's own explanatory prose, not generated post content; CLAUDE.md rule 4 targets generated posts. Not a real violation; optionally swap to "drew on / built on".
|
||||
|
||||
---
|
||||
|
||||
## Gate decision — R2b COMPLETE (4 surfaces)
|
||||
|
||||
| Surface | Verdict | BLOCKER | MAJOR | MINOR | SUGGESTION |
|
||||
|---|---|---|---|---|---|
|
||||
| create | ALLOW | 0 | 0 | 1 | 0 |
|
||||
| batch | REWORK | 0 | 1 | 3 | 2 |
|
||||
| pipeline | REWORK | 0 | 1 | 4 | 2 |
|
||||
| newsletter | REWORK | 0 | 1 | 2 | 2 |
|
||||
| **★ cross-cutting #1** (scaffold, 3 files) | — | 0 | (counted in batch + pipeline; spans post.md from R2a) | — | — |
|
||||
| **★ cross-cutting #2** (bare paths) | — | 0 | 0 | (counted in batch + pipeline) | — |
|
||||
|
||||
**Batch verdict: REWORK** — 3 of 4 surfaces (batch, pipeline, newsletter), each with **1 MAJOR**:
|
||||
(1) the systemic 5-component scaffold contradicting the 1,200–1,800 band (batch + pipeline; spans
|
||||
post.md from R2a), and (2) newsletter's resumption table missing the contract-gate phase. **0 BLOCKER
|
||||
anywhere.** `create` is ALLOW (one option-count MINOR with a runtime caveat).
|
||||
|
||||
**Independence verdict:** convergence on batch path-prefix / batch 3a-3b band / newsletter allowed-tools
|
||||
/ create 8-option (both lenses) — plus one **divergence resolved by main's grounding** (newsletter
|
||||
resumption table: intent-lens asserted complete, correctness-lens counted the gap, main confirmed). Both
|
||||
the convergence and the resolved divergence are signals these are real defects, not single-reviewer
|
||||
framing.
|
||||
|
||||
**Systemic findings now span R2a+R2b:** clipboard `printf` (R2a, 10 files) · component scaffold (R2b, 3
|
||||
files incl. post.md from R2a) · bare reference paths (R2b, 2 files, worth a repo-wide grep). Cold review
|
||||
**finds**; it changes no code. Each fix is its own operator-gated decision. Recommended consolidated-fix
|
||||
order out of the sweep so far: (1) clipboard `printf` [R2a, 10 files, highest blast radius], (2)
|
||||
component scaffold [3 files], (3) bare reference paths [grep-driven], then the per-surface items.
|
||||
Local-only (hardening-class), not pushed.
|
||||
218
docs/hardening/review-R3.md
Normal file
218
docs/hardening/review-R3.md
Normal file
|
|
@ -0,0 +1,218 @@
|
|||
---
|
||||
type: cold-review
|
||||
batch: R3
|
||||
journey: "Engage — post-publish & longform-support surfaces"
|
||||
scope: "FROZEN committed files at HEAD 2b70660 (clean tree; post-hardening cold pass)"
|
||||
method: "2 independent cold Opus reviewers for the round (intent + correctness), each covering all 4 surfaces, no cross-feed; every mechanical claim tool-grounded (anti-fabrication mandate); reviewers carry NO drafting-session context"
|
||||
surfaces: [firsthour, calendar, headless-review, pivot]
|
||||
reviewers:
|
||||
- "intent-lens (conformance: intent delivery + cross-ref resolution + class predicates + graceful degradation + terminology)"
|
||||
- "correctness-lens (internal consistency + bound-vs-canonical + checklist/phase arithmetic + allowed-tools completeness)"
|
||||
status: "COMPLETE — all 4 surfaces reviewed (firsthour, calendar, headless-review, pivot)"
|
||||
verdict: REWORK
|
||||
counts: { BLOCKER: 0, MAJOR: 1, MINOR: 3, SUGGESTION: 5 }
|
||||
---
|
||||
|
||||
# Cold review — R3 (Engage · post-publish & longform-support)
|
||||
|
||||
Independent post-hoc cold review of the 4 Engage-journey surfaces, on the FROZEN committed files
|
||||
(HEAD `2b70660`). Mirrors the S1 `review.md` + R2a + R2b model (the cold-review method that did **not**
|
||||
fabricate): read-and-show before assert, every `file:line` tool-confirmed, reviewers carry no
|
||||
drafting-session context. The per-command interactive gate (`log.md`) already passed these; this pass
|
||||
adds the **independent** axis that gate never had.
|
||||
|
||||
**Independence cross-check — both outcomes recurred this batch:**
|
||||
1. **Convergence:** both blind lenses independently surfaced the same real defect on `headless-review`
|
||||
(`SendUserFile` invoked on the primary surfacing path but absent from `allowed-tools`).
|
||||
2. **Divergence resolved by grounding** (the strongest argument FOR the two-lens method, recurring from
|
||||
R2b's newsletter): on `calendar` the intent-lens flagged a **MAJOR** (the publish/reschedule/cancel
|
||||
actions key off `id`/`draft_path`/`character_count` that the queue load never surfaces), while the
|
||||
correctness-lens passed the surface as ALLOW — its structural pass found the step/option arithmetic
|
||||
reconciled but did **not** trace the data-flow from load → display → action placeholders. Main
|
||||
re-grounded `queueFormatSummary`'s actual output independently (below) → the intent-lens is right. The
|
||||
independence axis caught a real MAJOR one lens never probed.
|
||||
|
||||
---
|
||||
|
||||
## Connections to existing systemic findings (no NEW ★ cross-cutting this batch)
|
||||
|
||||
R3 surfaces **connect to** the two systemic patterns already recorded in R2a/R2b rather than adding new
|
||||
ones. Both connections were re-grounded by main on the R3 files:
|
||||
|
||||
- **★ cross-cutting #1 (clipboard `printf '%s'`, R2a, 10 files) — firsthour confirmed present.**
|
||||
`firsthour.md:72` `printf '%s' '<DRAFT_COMMENTS_BLOCK>' | node …/clipboard-helper.mjs` + `:75` the
|
||||
unconditional "Copied your draft comments to clipboard." This is the exact systemic pattern: a
|
||||
single-quoted shell string corrupts any draft text containing an apostrophe (`it's`, `don't` — common
|
||||
in natural comment copy), and the "Copied" confirmation is unconditional. firsthour is one of the 10
|
||||
files STATE already lists; **no new derivation — folds into the R2a ★ #1 consolidated fix.**
|
||||
- **★ cross-cutting #2 (bare relative reference paths vs `${CLAUDE_PLUGIN_ROOT}/`, R2b) — firsthour adds
|
||||
3 sites.** `firsthour.md:110` (prose parenthetical), `:118`, `:119` (Reference-Files pointer list) are
|
||||
bare `references/…` while the same file's **executable** blocks correctly prefix `${CLAUDE_PLUGIN_ROOT}/`
|
||||
(`:72`, `:84`) and sibling commands prefix their Reference-Files lists too (`calendar.md:206-207`).
|
||||
**Lower impact than the R2b instances** (firsthour's bare paths are in a pointer list + one prose
|
||||
mention, not inside an executable `Read`), so latent rather than active — but a real parity break worth
|
||||
catching in the same repo-wide grep pass. Counted as a per-surface MINOR below.
|
||||
|
||||
**Recurring (SUGGESTION-class, NOT elevated to ★) — `allowed-tools` over-declaration.** Three of the four
|
||||
surfaces declare a tool the body never invokes: `firsthour` (`Glob`/`Grep`), `calendar` (`Write`/`Edit`),
|
||||
`pivot` (`Grep`). Harmless (over-declaration widens permission surface but breaks nothing;
|
||||
*under*-declaration would be the real risk and there is none). Noted per-surface; optional minimal-surface
|
||||
trim, fold into the consolidated fix if touched.
|
||||
|
||||
---
|
||||
|
||||
## firsthour.md — VERDICT: ALLOW (0 BLOCKER/MAJOR · 1 MINOR · 2 SUGGESTION + clipboard pointer)
|
||||
|
||||
Class: **guided/stateful + subagent orchestration** ("post-publish first-hour / reply-loop sprint").
|
||||
Intent delivered: Step 0 load → Step 1 identify post → Step 2 delegate to `engagement-coach` → Step 3
|
||||
present (timeline / targets / drafts / velocity) → Step 4 `recordFirstHourPlan` persist → Step 5
|
||||
`post-feedback-monitor` handoff. Both subagent targets carry the `linkedin-studio:` namespace and resolve
|
||||
(`agents/engagement-coach.md`, `agents/post-feedback-monitor.md`); `recordFirstHourPlan` signature
|
||||
(`planDate, postTopic, targets, draftComments, plan`) matches the call (`:85-91` vs `state-updater.mjs:235`);
|
||||
6 steps (`grep -cE '^## Step'` = 6), sequential, no orphan markers; Step 2→3 value-flow reconciles (coach
|
||||
asked for target-list / self-comments / timeline / velocity, Step 3 presents exactly those four); no
|
||||
"thought leadership". Empty-state degradation present.
|
||||
|
||||
### MINOR — bare reference paths → see Connections (★ #2) (`firsthour.md:110, :118, :119` vs prefixed `:72, :84`). Intent-lens.
|
||||
|
||||
### SUGGESTION — Step 0 voice-samples read has no stated fallback
|
||||
`firsthour.md:29` reads voice-samples "so every draft comment is in the user's voice," but no path is
|
||||
specified when the file is absent (progressive onboarding suppresses voice until 5+ samples, CLAUDE.md rule
|
||||
7). Non-breaking (the coach can still draft), but the degradation is unstated. Add "if absent, draft in a
|
||||
neutral first-person register and skip voice-matching."
|
||||
|
||||
### SUGGESTION — `allowed-tools` over-declares `Glob`/`Grep` (`:12-13`); body invokes neither. Correctness-lens. See Recurring note.
|
||||
|
||||
### (pointer) clipboard `printf '%s'` systemic bug — `firsthour.md:72, :75` confirmed present → folds into ★ #1 (R2a). No new derivation.
|
||||
|
||||
---
|
||||
|
||||
## calendar.md — VERDICT: REWORK (0 BLOCKER · 1 MAJOR · 0 MINOR · 1 SUGGESTION)
|
||||
|
||||
Class: **guided/stateful + routing** ("view/manage scheduling queue + publish action"). The **view** side
|
||||
(14-day calendar, format mix, pillar balance) delivers; the **action** side has a load-bearing data gap.
|
||||
Step/sub-step/option arithmetic all reconcile (correctness-lens: Steps 1–4 sequential; sub-markers 3a–3f
|
||||
present and ordered; 5 options offered with 4 handlers + explicit no-op, no dangling branch; Quick-Routing
|
||||
anchor `:89` exists); empty/missing-queue degradation present and correct (`queue-manager.mjs:12-27`
|
||||
returns `[]`; body 3a routes "no posts" → `/linkedin:batch`/`quick`); no "thought leadership".
|
||||
|
||||
### MAJOR — the queue load surfaces none of the `id`/`draft_path`/`character_count` the publish/reschedule/cancel actions require (`calendar.md:31-43, :117, :169-174, :185` vs `queue-manager.mjs:112-122`)
|
||||
- **Verified by main (independent re-grounding of the lens divergence):** Step 1 loads the queue
|
||||
**exclusively** through `queueFormatSummary` (`:31-43`). `queueFormatSummary` (`queue-manager.mjs:112-122`,
|
||||
read in full) emits only ` {date} {time} | {hook…} | {pillar} ({fmt}) [{status}]` — it exposes **no**
|
||||
`id`, **no** `draft_path`, **no** `character_count`. The Step 2 display (`:52-71`) mirrors that field set.
|
||||
- **Three concrete breakages on the action paths:**
|
||||
1. **Mark-as-published (the PRIMARY route — Quick-Routing `:25` jumps straight here)** calls
|
||||
`queueUpdateStatus('[post-id]', 'published')` (`:117`) — `[post-id]` was never surfaced. Step 3d
|
||||
also needs `charCount: NNNN` (`:129`), likewise un-surfaced.
|
||||
2. **Reschedule** (`:174`) calls `queueAdd('[post-id]','[draft_path]', …, [charCount])` (8-arg signature
|
||||
confirmed `queue-manager.mjs:63`) and is **explicitly told** to "carry the unchanged fields
|
||||
(draft_path, pillar, format, hook preview, char count) from **the entry shown in Step 2**"
|
||||
(`:169-172`) — but Step 2 provably shows none of `draft_path`/`char count`/`id`. A direct
|
||||
contradiction: the instruction points at a view that lacks the fields it says to carry.
|
||||
3. **Cancel** (`:185`) likewise needs the un-surfaced `[post-id]`.
|
||||
- **Self-recovery caveat (honest):** `queue.json` is in Reference Files (`:208`) and `Read` is allowed, so
|
||||
a capable agent *could* read raw entries to recover `id`/`draft_path`/`char_count`. But the body never
|
||||
instructs that, and the reschedule text actively **mis-directs** to Step 2. Latent-but-real on the
|
||||
primary route → MAJOR, not MINOR.
|
||||
- **Independence note:** intent-lens flagged MAJOR; correctness-lens passed the surface ALLOW (its
|
||||
arithmetic/structure pass reconciled but did not trace load→display→action data-flow). Main grounded
|
||||
`queueFormatSummary`'s output → intent-lens confirmed. Two-lens method earned its keep (same shape as
|
||||
R2b's newsletter resumption table).
|
||||
- **disposition:** in Step 1 also dump raw entries (e.g. `console.log(JSON.stringify(queueUpcoming(14)))`,
|
||||
or a `queueRead()` dump exposing `id`/`draft_path`/`character_count`), and re-point the reschedule text
|
||||
from "the entry shown in Step 2" to "the raw queue entry loaded in Step 1." Surface the display ordinal →
|
||||
queue-`id` mapping so 3b/reschedule/cancel can fill `[post-id]`.
|
||||
|
||||
### SUGGESTION — `allowed-tools` over-declares `Write`/`Edit` (`:13-14`); every mutation routes through `Bash` node one-liners, "View draft" uses `Read`. Correctness-lens. See Recurring note.
|
||||
|
||||
---
|
||||
|
||||
## headless-review.md — VERDICT: ALLOW (0 BLOCKER/MAJOR · 2 MINOR · 1 SUGGESTION)
|
||||
|
||||
Class: **guided/stateful + heavy subagent orchestration + routing** (cold 5-archetype package on a frozen
|
||||
draft → one operator-gated report). Intent delivered: Step 1 resolve-from-disk → Step 2 freeze (`cp`
|
||||
snapshot) → Step 3 parallel fan-out (the `--type`→`subagent_type` table `:141-145` maps to the 5 real cold
|
||||
review modes: content / language / fact / persona-resonance / persona-conversion) → Step 4 consolidate →
|
||||
Step 5 surface + optional `edition-state.json` persist. All reviewer agents resolve; the writing-contract
|
||||
fallback chain terminates in `references/longform-quality-rules.md` (present); degradation well-handled
|
||||
(missing `--draft` → edition-state or ask; `cp` unavailable → live draft + note; degraded reviewer
|
||||
re-runs). "five archetypes" reconciles with the 5-row `--type` table; 5 flags all consumed, no orphan; no
|
||||
"thought leadership".
|
||||
|
||||
### MINOR — `SendUserFile` invoked on the primary surfacing path but absent from `allowed-tools` (`:208, :221` vs `:19-25`). BOTH lenses.
|
||||
- `allowed-tools` (`:19-25`) = Read, Glob, Grep, Bash, AskUserQuestion, Task, Write — no `SendUserFile`;
|
||||
body uses it 2× (`grep -c` = 2), as the documented **primary** operator-gated delivery ("operator-gated
|
||||
via SendUserFile"). Held at MINOR (not MAJOR) by two guards: `:208` carries an in-text fallback ("else a
|
||||
markdown `file://` link") and the report is independently persisted via the declared `Write` (`:206`), so
|
||||
surfacing degrades rather than breaks.
|
||||
- **disposition:** add `SendUserFile` to `allowed-tools` (if a real tool in the target harness), or soften
|
||||
the body wording from "primary/operator-gated via SendUserFile" to "surface via a `file://` link (or
|
||||
`SendUserFile` if available)."
|
||||
|
||||
### MINOR — `v3.1.0` reload anchor misleads on the post-reset version line (`:81-82`)
|
||||
- **Verified by main:** `:81-82` says the three cold archetypes "were added in **v3.1.0** — if the session
|
||||
predates them, reload." Current `plugin.json` version = **0.5.3** (`:3`); CHANGELOG `[0.4.0]` (2026-05-31)
|
||||
records the **honest version reset 4.1.0 → 0.4.0**, so `v3.1.0` is a *pre-reset* tag no longer on the
|
||||
current line. A reader on 0.5.3 comparing numerically (0.5.3 < 3.1.0) would wrongly conclude they
|
||||
"predate" the agents and must reload — when 0.5.3 is post-reset and already ships all three (they are in
|
||||
CLAUDE.md's 19-agent list). Harmless if followed (an unnecessary reload), but the version anchor misleads.
|
||||
- **disposition:** anchor by event/date, not the dead tag — e.g. "added with the cold-review package
|
||||
(CHANGELOG 3.1.0, pre-reset); reload if your session predates those agents."
|
||||
|
||||
### SUGGESTION — fan-out N-count unit left implicit: `persona-resonance` issues "one call per active persona" (`:144`) while the header counts "<N> archetypes" / "<N> run in parallel" (`:172, :219`). Pin whether N counts review-modes (5) or Task-calls (≥5). Non-breaking. Correctness-lens.
|
||||
|
||||
---
|
||||
|
||||
## pivot.md — VERDICT: ALLOW (0 BLOCKER/MAJOR · 0 MINOR · 1 SUGGESTION)
|
||||
|
||||
Class: **guided/stateful + routing (no subagent orchestration by design)** ("re-open a long-form edition so
|
||||
cleared gates re-run before lock"). Intent delivered: Step 1 load+locate (stops if `articles.NN` absent) →
|
||||
Step 2 measure scope + classify → Step 3 append `pivots[]`, reset `currentPhase`, un-lock, invalidate
|
||||
downstream verdicts → Step 4 write `STATE.md` + point at `/linkedin:newsletter`. **Unusually
|
||||
well-reconciled** (correctness-lens, all main-checkable): 4 steps sequential; the >20%/>2-sections
|
||||
heuristic stated identically in 3 places (`:8-9, :54-55, :92`); the worked example's arithmetic checks out
|
||||
(+42% = (1992−1400)/1400 ✓; "added 2 sections … at the boundary of '>2'" correctly attributes the trigger
|
||||
to the 20% arm since `2` is not `>2`); the off-by-one phase map is explicitly reconciled (`:102-108`,
|
||||
`to-phase` = last *completed* phase, newsletter resumes at the step after); `gatesToRerun` (4 entries `:123`)
|
||||
matches the summary + STATE line (`:163, :149-151`). `allowed-tools` correctly **omits `Task`** (delegates
|
||||
gate-running to `/linkedin:newsletter`, never spawns). All 3 Reference-File targets resolve; degradation
|
||||
present (Step 1 stop-on-missing-article; Step 2.2 absent-baseline → ask operator). No "thought leadership".
|
||||
|
||||
### SUGGESTION — `allowed-tools` over-declares `Grep` (`:18`); the only grep in the body is a *shell* `grep -c '^## '` inside a `Bash` block (`:89`), not the `Grep` tool. `Glob` plausibly resolves the series root — keep it. Correctness-lens. See Recurring note.
|
||||
|
||||
---
|
||||
|
||||
## Gate decision — R3 COMPLETE (4 surfaces)
|
||||
|
||||
| Surface | Verdict | BLOCKER | MAJOR | MINOR | SUGGESTION |
|
||||
|---|---|---|---|---|---|
|
||||
| firsthour | ALLOW | 0 | 0 | 1 | 2 |
|
||||
| calendar | REWORK | 0 | 1 | 0 | 1 |
|
||||
| headless-review | ALLOW | 0 | 0 | 2 | 1 |
|
||||
| pivot | ALLOW | 0 | 0 | 0 | 1 |
|
||||
| **★ #1 clipboard** (pointer, firsthour) | — | — | — | (folds into R2a) | — |
|
||||
| **★ #2 bare paths** (firsthour, 3 sites) | — | 0 | 0 | (counted in firsthour) | — |
|
||||
|
||||
**Batch verdict: REWORK** — 1 of 4 surfaces (calendar) carries **1 MAJOR**: the queue load surfaces none of
|
||||
the `id`/`draft_path`/`character_count` that publish/reschedule/cancel require, and the reschedule step's
|
||||
"carry from the entry shown in Step 2" is a direct contradiction. **0 BLOCKER anywhere.** firsthour /
|
||||
headless-review / pivot are ALLOW (pivot notably clean — every count, the heuristic boundary case, and the
|
||||
off-by-one phase map reconcile).
|
||||
|
||||
**Independence verdict:** convergence on headless-review `SendUserFile` (both lenses) + one **divergence
|
||||
resolved by main's grounding** (calendar: intent-lens flagged the data-gap MAJOR, correctness-lens passed
|
||||
it on structural arithmetic, main grounded `queueFormatSummary`'s output and confirmed the MAJOR). Lens-B
|
||||
also uniquely caught the `v3.1.0` dead anchor + the over-declaration pattern; Lens-A uniquely caught the
|
||||
bare paths + the clipboard pointer. Both lenses earned their keep.
|
||||
|
||||
**Systemic findings now span R2a+R2b+R3:** clipboard `printf` (R2a, 10 files incl. firsthour) · component
|
||||
scaffold (R2b, 3 files) · bare reference paths (R2b+R3, now 3 files incl. firsthour's 3 sites). **New this
|
||||
batch (SUGGESTION-class, not ★):** `allowed-tools` over-declaration on 3 of 4 R3 surfaces. Cold review
|
||||
**finds**; it changes no code. Each fix is its own operator-gated decision. Recommended consolidated-fix
|
||||
order unchanged: (1) clipboard `printf` [R2a, 10 files, highest blast radius], (2) component scaffold [3
|
||||
files], (3) bare reference paths [grep-driven, now incl. firsthour], then the per-surface items (calendar
|
||||
queue-data MAJOR, headless `SendUserFile`/`v3.1.0`, over-declaration trims). Local-only (hardening-class),
|
||||
pushed per the 2026-06-30 operator delegation (public catalog, no secrets).
|
||||
|
||||
**Cumulative cold-review coverage: 17/29** (review.md S1=4 · R2a=5 · R2b=4 · R3=4).
|
||||
293
docs/hardening/review-R4.md
Normal file
293
docs/hardening/review-R4.md
Normal file
|
|
@ -0,0 +1,293 @@
|
|||
---
|
||||
type: cold-review
|
||||
batch: R4
|
||||
journey: "Measure — analytics & performance surfaces"
|
||||
scope: "FROZEN committed files at HEAD 69f37ba (clean tree; post-hardening cold pass)"
|
||||
method: "2 independent cold Opus reviewers for the round (intent + correctness), each covering all 6 surfaces, no cross-feed; every mechanical claim tool-grounded (anti-fabrication mandate); reviewers carry NO drafting-session context. Divergences re-grounded by main before registration."
|
||||
surfaces: [import, report, analyze, audit, ab-test, measure]
|
||||
reviewers:
|
||||
- "intent-lens (conformance: intent delivery + cross-ref resolution + analytics class predicates + graceful degradation + terminology)"
|
||||
- "correctness-lens (internal consistency + bound-vs-canonical + step/phase arithmetic + allowed-tools completeness + metric-definition cross-check)"
|
||||
class: "analytics — extra predicate: graceful degradation present · saves/dwell honesty intact (parseOptionalCount → unknown/never 0; dwell unmeasurable; saves NOT folded into engagementRate; analytics I/O via getAnalyticsRoot seam)"
|
||||
status: "COMPLETE — all 6 surfaces reviewed (import, report, analyze, audit, ab-test, measure)"
|
||||
verdict: REWORK
|
||||
counts: { BLOCKER: 0, MAJOR: 1, MINOR: 4, SUGGESTION: 6 }
|
||||
---
|
||||
|
||||
# Cold review — R4 (Measure · analytics & performance)
|
||||
|
||||
Independent post-hoc cold review of the 6 Measure-journey surfaces, on the FROZEN committed files
|
||||
(HEAD `69f37ba`). Mirrors the S1 `review.md` + R2a + R2b + R3 model (the cold-review method that did **not**
|
||||
fabricate): read-and-show before assert, every `file:line` tool-confirmed, reviewers carry no
|
||||
drafting-session context. The per-command interactive gate (`log.md`) already passed these; this pass
|
||||
adds the **independent** axis that gate never had. This is the largest batch (6 surfaces) and the only
|
||||
**analytics-class** batch, so the round carries the extra class predicate (graceful degradation +
|
||||
saves/dwell honesty) alongside the standard intent/correctness lenses.
|
||||
|
||||
**Analytics-class predicate — PASSES across all 6 surfaces (the headline R4 result).** Both blind lenses
|
||||
independently confirmed the honesty contract holds wherever a surface touches the metric: **saves** are
|
||||
consistently framed as native-only / count-only / ~Sept 2025-onward / no self-serve API / manual-entry
|
||||
(`report.md:143,:241`; `import.md:30,:138,:148`; matches `cli.ts:144-146` + `csv-parser.ts:71` where
|
||||
`parseOptionalCount` → blank/non-numeric/negative becomes `undefined` = unknown, never 0) and are **never
|
||||
folded into `engagementRate`** (`csv-parser.ts:205-208` numerator = reactions+comments+shares+clicks, no
|
||||
saves); **dwell** is consistently called unmeasurable/internal-to-LinkedIn (`report.md:241`,
|
||||
`import.md:30`); no surface claims to import or compute either. The `getAnalyticsRoot()` per-user data-dir
|
||||
seam (`storage.ts`) is described accurately wherever quoted. **No analytics-honesty violation anywhere in
|
||||
R4.**
|
||||
|
||||
**Independence cross-check — both outcomes recurred this batch (the case for two lenses, again):**
|
||||
1. **Convergence (×2):** both blind lenses independently surfaced (a) `import.md` Step 6a's invalid
|
||||
`trends` flags (`--period 4w` / `--metric engagement_rate`) and (b) `ab-test.md:236`'s manual
|
||||
engagement-rate formula excluding clicks. Two real defects, found twice without cross-feed.
|
||||
2. **Divergence resolved by grounding (×3, in BOTH directions):**
|
||||
- **intent over-rated, main corrected down** — on `import` the intent-lens flagged the Step 6a CLI block
|
||||
**MAJOR** (→ REWORK), the correctness-lens flagged the same defect **MINOR** (→ ALLOW, "off-primary,
|
||||
descriptive"). Main re-grounded `import.md:194-217`: the executable instruction is the **delegation**
|
||||
(`:200` "Run /linkedin:report"); the bash block (`:207-210`) is import's *description* of report's
|
||||
internals, not import's own step → latent, errors only if copy-run → **MINOR**, import = ALLOW.
|
||||
- **correctness uniquely caught, main confirmed** — on `report` the correctness-lens flagged a **MAJOR**
|
||||
(heatmap branch routes to a nonexistent "Step 6c"); the intent-lens was silent (its lens probes
|
||||
agent/command cross-refs, not internal step-jump arithmetic). Main grounded the step inventory → no
|
||||
`6c` exists, real handler is `2c` → **MAJOR confirmed**. This is the batch's load-bearing defect.
|
||||
- **correctness uniquely caught, main confirmed** — on `analyze` the correctness-lens flagged a **MINOR**
|
||||
(two non-reconciling severity scales); the intent-lens was silent → main grounded `:155-178` vs
|
||||
`:227-231` → confirmed **MINOR**.
|
||||
Both lenses earned their keep: intent over-rated one finding (corrected by grounding), correctness
|
||||
uniquely caught the two structural defects intent's lens never traced.
|
||||
|
||||
---
|
||||
|
||||
## Connections to existing systemic findings (no NEW ★ cross-cutting this batch)
|
||||
|
||||
All connections re-grounded by main against the R4 files:
|
||||
|
||||
- **★ #1 (clipboard `printf '%s'`, R2a, 10 files) — R4 adds nothing.** `grep -nE "printf '%s'|clipboard-helper"`
|
||||
across all 6 R4 files → NONE. The analytics surfaces do not auto-copy to clipboard (they ingest/report
|
||||
data, they don't emit post text), so this systemic pattern simply does not reach the Measure journey.
|
||||
- **★ #2 (5-component scaffold band-mismatch, R2b, 3 files) — R4 adds nothing.** The length-band tokens
|
||||
that appear (`analyze.md:201` "1,200-1,500", `:217` "1,500-1,800"; `report.md:331` example impressions;
|
||||
`ab-test.md:80` test-variable "Short (500) vs standard (1,200-1,800) vs long (2,500+)") are
|
||||
recovery-protocol guidance / illustrative numbers / a test variable — none is a component scaffold that
|
||||
sums outside the standard band. No defect.
|
||||
- **★ #3 (bare reference paths vs `${CLAUDE_PLUGIN_ROOT}/`, R2b+R3) — R4 connects lightly (lowest impact).**
|
||||
`analyze.md:22,:23,:93,:259,:260,:261` and `report.md:241` carry bare `references/…` — but **none is
|
||||
inside an executable `Read`/`cat`** (`grep -nE "(Read|cat) .*references/"` → NONE executable); all are
|
||||
pointer-list entries or prose mentions, the same lowest-impact class as R3's firsthour bare paths.
|
||||
Latent parity break worth catching in the same repo-wide `${CLAUDE_PLUGIN_ROOT}/` grep pass; not elevated
|
||||
to a per-surface finding (neither lens raised it; cosmetic on these surfaces).
|
||||
|
||||
**Recurring (SUGGESTION-class, NOT elevated to ★) — `allowed-tools` over-declaration now spans R3+R4.**
|
||||
Four of six R4 surfaces declare a tool the body never invokes: `import`/`report` (`Glob` — listing done via
|
||||
Bash `ls`/`find`), `audit` (`Grep` — no grep call). Combined with R3's 3-of-4, the pattern now touches ~7
|
||||
surfaces. Harmless (over-declaration widens the permission surface but breaks nothing; *under*-declaration
|
||||
is the real risk and there is one true instance this batch — `report` Step 8b, recorded as MINOR below).
|
||||
Optional minimal-surface trim; fold into the consolidated fix if touched.
|
||||
|
||||
**New R4 cluster (not ★, analytics-specific) — sibling-command interface/metric-definition drift.** Two
|
||||
of the four MINORs are the same shape: an analytics surface quotes another surface's CLI interface or a
|
||||
shared metric definition and drifts from the SSOT — `import.md` Step 6a's stale `trends` flags vs
|
||||
`report.md`'s correct ones, and `ab-test.md:236`'s manual engagement-rate (clicks excluded) vs the CLI's
|
||||
`engagementRate` (clicks included, `csv-parser.ts:205`). Both are latent (cross-reference paths, not
|
||||
primary execution) but both are real consistency debt between siblings. Worth a single reconciliation note
|
||||
in the consolidated fix: pin the canonical `engagementRate` definition + CLI flag vocabulary once, and make
|
||||
the descriptive blocks point at it rather than restate it.
|
||||
|
||||
---
|
||||
|
||||
## import.md — VERDICT: ALLOW (0 BLOCKER/MAJOR · 1 MINOR · 1 SUGGESTION)
|
||||
|
||||
Class: **analytics (import orchestrator)**. Intent delivered: primary artifact is the structured JSON batch
|
||||
written by `cli.ts import` (Step 4 invokes it; output surfaced Step 5), then analysis delegated to
|
||||
`/linkedin:report` (Step 6). Step inventory `1·1b·2·3·4·5·5b·6·6a·6b·7·8` sequential, no gaps; Step 1b/Step 3
|
||||
option lists each carry a Skip/Cancel disposition. `allowed-tools` (`:10-15` Bash/Read/Glob/Write/AskUserQuestion)
|
||||
— Bash/Read/Write/AskUserQuestion all invoked. Cross-refs resolve (`report.md`, `setup.md`, `quick-import.mjs`,
|
||||
`assets/analytics/README.md` all exist; no `subagent_type`). Degradation present (no-CSV, nothing-anywhere,
|
||||
missing-deps `npm install`, skipped rows on empty-title/unparseable-date matching `csv-parser.ts:187,193`).
|
||||
saves/dwell honesty intact (`:30,:138,:148`). No "thought leadership".
|
||||
|
||||
### MINOR — Step 6a documents report's CLI calls with invalid period + metric, contradicting the real owner (`import.md:200, :206, :207-210`)
|
||||
- **Verified by main (the intent/correctness severity divergence, re-grounded `:194-217`):** `:207-210`
|
||||
shows `trends --period 4w --metric impressions` and `--metric engagement_rate` inside a bash fence. The
|
||||
CLI accepts period `week|month|quarter|all` (`cli.ts:217`, validated → `process.exit(1)` `:219-221`) and
|
||||
metric `…|engagementRate` camelCase (`cli.ts:202-209`, validated → `process.exit(1)` `:233-234`) — both
|
||||
`4w` and `engagement_rate` would error. The real owner `report.md:153,:171` uses the correct
|
||||
`--period month --metric engagementRate`. `:200` "(period: 4w)" and `:206` "Read expertise_areas" also
|
||||
mis-describe report (it takes no period arg, never reads expertise_areas).
|
||||
- **Why MINOR not MAJOR (intent-lens rated MAJOR; main grounds down):** the **executable** instruction on
|
||||
this path is the delegation at `:200` ("Run /linkedin:report") → routes to `report.md`, which is correct.
|
||||
The bash block (`:207-210`) is import's *narrative description* of report's internals ("`/linkedin:report`
|
||||
will: … 2. Call `trends`…"), not a step import itself runs. Latent (errors only if a reader copy-runs the
|
||||
illustrative block) and the primary delegation path is unaffected → MINOR, not MAJOR. Both lenses
|
||||
converged on the defect's existence; only the severity diverged.
|
||||
- **disposition:** drop the illustrative bash + the "(period: 4w)"/"expertise_areas" description, or mirror
|
||||
report's real invocations (`--period month --metric engagementRate`). Fold into the sibling-drift
|
||||
reconciliation note.
|
||||
|
||||
### SUGGESTION — `allowed-tools` over-declares `Glob` (`:13`); directory listing uses Bash `ls`/`find` (`:37,:47`). Both lenses. See Recurring note.
|
||||
|
||||
---
|
||||
|
||||
## report.md — VERDICT: REWORK (0 BLOCKER · 1 MAJOR · 1 MINOR · 1 SUGGESTION)
|
||||
|
||||
Class: **analytics (report orchestrator)**. Intent delivered: produces weekly/monthly/heatmap report JSON
|
||||
via `cli.ts report`/`heatmap` + a formatted presentation (Step 6) + an analytics-interpreter handoff
|
||||
(`subagent_type: linkedin-studio:analytics-interpreter` `:308` → `agents/analytics-interpreter.md` ✓, `Task`
|
||||
declared `:14`). Trends flags `--period month --metric engagementRate` (`:153,:171,:366`) all valid vs CLI.
|
||||
Degradation present (no-data, npm install, week-not-found/empty-week/ERR_MODULE_NOT_FOUND `:387-400`).
|
||||
saves/dwell honesty exemplary (`:143,:241`). No "thought leadership".
|
||||
|
||||
### MAJOR — the heatmap report type routes to a nonexistent "Step 6c" (`report.md:72`)
|
||||
- **Verified by main (correctness-lens caught it; intent-lens silent — lens gap, not contradiction):** the
|
||||
step inventory (`grep -nE '^### Step|^## Step'`) is `1·1b·2·2b·2c·3·4·5·5b·5c·6·7·8·8b` — **there is no
|
||||
Step 6c**. `:72` ("If heatmap (option 3): Run the heatmap CLI command and jump to **Step 6c**") points a
|
||||
reader nowhere. The real heatmap handler is **Step 2c** (`:106`, immediately below the monthly Step 2b),
|
||||
which itself "jump[s] to Step 7" (`:114`). The two sibling branches are correct (monthly `:71`→2b;
|
||||
weekly→inline), so heatmap — one of three top-level report types in Step 2 — is the lone misroute.
|
||||
- **Severity:** a provably-wrong cross-reference on a **primary menu branch** (top-level report-type
|
||||
selection, not a deep-dive). Recoverable (`:72` also says "Run the heatmap CLI command," and 2c sits
|
||||
right under 2b, so a capable agent recovers by proximity) — hence MAJOR, not BLOCKER — but it is the
|
||||
batch's one load-bearing navigation defect.
|
||||
- **disposition:** retarget `:72` from "Step 6c" to "Step 2c".
|
||||
|
||||
### MINOR — Step 8b markdown export under-declares its write tool (`report.md:429-431` vs `:9-15`)
|
||||
- **Verified by main:** Step 8b (reached via Step 8 option 4, "Export report as markdown file") instructs
|
||||
"Format the data using this template and **write to file**" / "Save to: …`-report.md`" (`:429-431`).
|
||||
Frontmatter `allowed-tools` (`:9-15`) = Bash/Read/Glob/AskUserQuestion/Task — **no `Write`**. This is the
|
||||
one genuine *under*-declaration this batch (the real-risk class). Held at MINOR by two mitigations: the
|
||||
step is an optional deep-dive (not the primary path), and the declared `Bash` can satisfy the write via a
|
||||
heredoc/`cat >`. Sibling `ab-test.md:12-19` declares `Write` for the same `.md`-save, so the omission
|
||||
reads as an oversight/parity break.
|
||||
- **disposition:** add `Write` to `allowed-tools` (parity with ab-test) or rephrase Step 8b to write via the
|
||||
declared `Bash`.
|
||||
|
||||
### SUGGESTION — `allowed-tools` over-declares `Glob` (`:12`); listing uses Bash `ls` (`:30,:197`). Correctness-lens. See Recurring note.
|
||||
|
||||
---
|
||||
|
||||
## analyze.md — VERDICT: ALLOW (0 BLOCKER/MAJOR · 1 MINOR · 1 SUGGESTION)
|
||||
|
||||
Class: **analytics-adjacent (read-only diagnostic; no CLI)**. Intent delivered: diagnosis + recovery plan
|
||||
from reference files + AskUserQuestion (Steps Load-Context·1-8). Cross-refs resolve
|
||||
(`subagent_type: linkedin-studio:analytics-interpreter` `:41` → agent ✓, `Task` declared `:12`;
|
||||
`/linkedin:profile` `:188` → `commands/profile.md` ✓). Degradation present (analytics delegation is
|
||||
conditional `:41` with a self-report fallback; functions with zero data). No saves/dwell claims → nothing to
|
||||
contradict. No "thought leadership".
|
||||
|
||||
### MINOR — two non-reconciling severity scales in one command (`analyze.md:155-178` vs `:227-231`)
|
||||
- **Verified by main (correctness-lens; intent-lens silent — lens gap):** Step 5 grades reach drop on four
|
||||
percentage bands (`<25` / `25-50` / `50-75` / `75%+`, `:155-178`). Step 7's timeline table (`:227-231`)
|
||||
uses three rows on a *different* axis — "Moderate (link/off-topic)", "Moderate (partial reach loss)",
|
||||
"Severe (sharp reach loss)". The two scales share no common key, so a user holding a Step 5 result (e.g.
|
||||
"Down 50-75% → algorithmic suppression likely") cannot map it to a Step 7 timeline row. Advisory/usability
|
||||
inconsistency, low-confidence; nothing breaks.
|
||||
- **disposition:** cross-label the Step 7 rows to the Step 5 bands, or state explicitly that they are
|
||||
independent axes.
|
||||
|
||||
### SUGGESTION — the `:41` existence check ("If imported analytics data exists `…/analytics/`") implies Glob/Bash, neither declared (`allowed-tools` `:9-13` = Read/AskUserQuestion/Task). Softer than report's Write gap: no explicit tool call is written at `:41` (it's a conditional prose phrase) and the real data access is delegated to `analytics-interpreter` via `Task`. Correctness-lens.
|
||||
|
||||
---
|
||||
|
||||
## audit.md — VERDICT: ALLOW (0 BLOCKER/MAJOR/MINOR · 1 SUGGESTION)
|
||||
|
||||
Class: **analytics-adjacent (read-only strategy auditor; no CLI)**. Intent delivered: audit report (Step 7
|
||||
template) + action items (Step 8). Steps `0·1·2·3·4·5·5.5·6·7·8` sequential. Routing resolves
|
||||
(`/linkedin:strategy` → `commands/strategy.md` ✓, `/linkedin:profile` ✓; no `subagent_type`, `Task`
|
||||
correctly absent from `allowed-tools`). Degradation present (Step 0 checks for analytics data + asks for
|
||||
screenshots/metrics; milestone block self-skips with no data `:140`). **Correctness cross-check (both
|
||||
lenses):** the state fields the milestone block reads — `follower_count`, `monthly_growth`,
|
||||
`growth_rate_needed` (`:105,:109,:128`) — all exist in `config/state-file.template.md:23,26,28`, so it reads
|
||||
real fields. No saves/dwell claims. No "thought leadership".
|
||||
|
||||
### SUGGESTION — `allowed-tools` over-declares `Grep` (`:11`); Step 0 uses Read/Glob, no grep call in the body. Both lenses (convergence). See Recurring note.
|
||||
|
||||
---
|
||||
|
||||
## ab-test.md — VERDICT: ALLOW (0 BLOCKER/MAJOR · 1 MINOR · 2 SUGGESTION)
|
||||
|
||||
Class: **analytics (experiment manager; manual metric entry, no CLI)**. Intent delivered: primary artifact
|
||||
is the test-plan markdown written to `analytics/ab-tests/[name].md` (Step 2a.8) + running comparison /
|
||||
analysis (2b/2c). Step inventory `0·1·2a(.1-.8)·2b·2c·2d·2e·3` consistent; Step 1's 6 intents map to 2a-2e +
|
||||
option 6 self-handles (`:57`). Post-count arithmetic coherent ("3 per variant / 6 total" `:127`; 6-row
|
||||
execution table `:155-162`; "X of 6" `:255`). `allowed-tools` (`:12-19`
|
||||
Read/Glob/Write/Bash/AskUserQuestion/Task) — Read/Write/Bash/AskUserQuestion/Task(→`content-optimizer` `:119`
|
||||
✓) all invoked. Degradation present (Error Handling: No-Tests-Directory, Incomplete-Data, Missing-Analytics,
|
||||
Corrupted-files `:472-493`). Statistical honesty notably correct — 2c.4/Confidence-Level (`:311,:320-331`)
|
||||
explicitly demotes small-sample results to "directional, not significant." No saves/dwell claims. No
|
||||
"thought leadership".
|
||||
|
||||
### MINOR — manual engagement-rate formula excludes clicks, diverging from the canonical `engagementRate` (`ab-test.md:236`)
|
||||
- **Verified by main (both lenses converged):** `:236` computes ER as
|
||||
`(reactions + comments + reposts) / impressions * 100` — clicks excluded (and 2b.3 `:228-234` never
|
||||
collects clicks). The CLI's `engagementRate` includes clicks: `totalEngagement = reactions + comments +
|
||||
shares + clicks` (`csv-parser.ts:205-208`). Step 2c.3 (`:282-288`) cross-references the A/B numbers against
|
||||
the CLI weekly reports, where the two rates will not match.
|
||||
- **Severity:** internally consistent within a single test (same formula on both variants, and A/B is
|
||||
variant-relative), so the verdict-within-a-test is unaffected → MINOR, latent on the cross-reference path.
|
||||
- **disposition:** add a one-line "clicks excluded by design" caveat, or align the manual formula to the
|
||||
canonical definition. Fold into the sibling-drift reconciliation note.
|
||||
|
||||
### SUGGESTION — bare agent name in a user-facing suggestion (`ab-test.md:468`): "use the `content-optimizer` agent" surfaces a bare name where the canonical form (correctly used at `:119`) is the namespaced `subagent_type`. Cosmetic. Intent-lens.
|
||||
|
||||
### SUGGESTION — `allowed-tools` over-declares `Glob` (`:14`); scans use Bash `ls` (`:37,:209`). 2b.4/2c.5 append/update are edit-shaped but the declared `Write` covers them via full rewrite (no `Edit` gap). Correctness-lens. See Recurring note.
|
||||
|
||||
---
|
||||
|
||||
## measure.md — VERDICT: ALLOW (0 findings — clean)
|
||||
|
||||
Class: **front-door router (delegate-only)**. Intent delivered **and structurally enforced** (both lenses
|
||||
converged): `allowed-tools` (`:10-13`) = `Glob` + `AskUserQuestion` only — no Bash/Read/Task — so the
|
||||
command *cannot* run analysis itself; it identifies intent (Step 1, 5 intents `:30-34`) and routes (Step 2
|
||||
table `:41-47`, one row per intent). All five routes — `/linkedin:import`, `/linkedin:report`,
|
||||
`/linkedin:analyze`, `/linkedin:audit`, `/linkedin:ab-test` — resolve to existing command files. Degradation
|
||||
present (Step 0 glob optional, "Do not block on it"; order-note `:49-50` routes to import first when nothing
|
||||
imported). No dangling branch, no analysis logic to drift, no saves/dwell claims, no "thought leadership".
|
||||
The thinnest, cleanest surface in the batch — the delegate-only contract is enforced by the tool whitelist,
|
||||
not just asserted.
|
||||
|
||||
---
|
||||
|
||||
## Gate decision — R4 COMPLETE (6 surfaces)
|
||||
|
||||
| Surface | Verdict | BLOCKER | MAJOR | MINOR | SUGGESTION |
|
||||
|---|---|---|---|---|---|
|
||||
| import | ALLOW | 0 | 0 | 1 | 1 |
|
||||
| report | REWORK | 0 | 1 | 1 | 1 |
|
||||
| analyze | ALLOW | 0 | 0 | 1 | 1 |
|
||||
| audit | ALLOW | 0 | 0 | 0 | 1 |
|
||||
| ab-test | ALLOW | 0 | 0 | 1 | 2 |
|
||||
| measure | ALLOW | 0 | 0 | 0 | 0 |
|
||||
| **TOTAL** | **REWORK** | **0** | **1** | **4** | **6** |
|
||||
|
||||
**Batch verdict: REWORK** — 1 of 6 surfaces (report) carries **1 MAJOR**: the heatmap report type routes to
|
||||
a nonexistent "Step 6c" (real handler Step 2c) — a provably-wrong cross-reference on a primary menu branch.
|
||||
**0 BLOCKER anywhere.** The other five are ALLOW (measure notably clean — its delegate-only contract is
|
||||
enforced by the `allowed-tools` whitelist, not merely asserted).
|
||||
|
||||
**Analytics-class predicate verdict: PASS.** The saves/dwell honesty contract holds on every surface that
|
||||
touches the metric — saves framed as manual/count-only/no-API and never folded into `engagementRate`, dwell
|
||||
called unmeasurable, neither claimed as imported; `parseOptionalCount` semantics (`csv-parser.ts:71`) and the
|
||||
`getAnalyticsRoot` seam described accurately wherever quoted. Graceful degradation present on all six. This
|
||||
was the batch-specific axis and it is clean.
|
||||
|
||||
**Independence verdict:** two convergences (import Step 6a flags; ab-test ER-omits-clicks) + three
|
||||
divergences resolved by main's grounding in **both directions** — intent over-rated import's Step 6a
|
||||
(MAJOR→MINOR on grounding that the executable path delegates), while correctness uniquely caught report's
|
||||
Step 6c misroute (MAJOR) and analyze's twin severity scales (MINOR) that intent's lens never traced. The
|
||||
two-lens method earned its keep again: had only the intent-lens run, report would have shipped ALLOW with a
|
||||
broken primary branch; had only the correctness-lens run, import would have over-escalated to REWORK.
|
||||
|
||||
**Systemic findings now span R2a+R2b+R3+R4:** clipboard `printf` (R2a, 10 files — **R4 adds none**) ·
|
||||
component scaffold (R2b, 3 files — **R4 adds none**) · bare reference paths (R2b+R3+R4 — R4 adds 7
|
||||
lowest-impact pointer/prose sites in analyze/report, not executable). **New this batch:** (a) `allowed-tools`
|
||||
over-declaration now spans R3+R4 (~7 surfaces, SUGGESTION-class, harmless) with one true *under*-declaration
|
||||
(report Step 8b `Write`, MINOR); (b) a small **sibling interface/metric-definition drift** cluster (import's
|
||||
stale `trends` flags + ab-test's clicks-excluded ER) — reconcile once against the CLI SSOT. Cold review
|
||||
**finds**; it changes no code. Each fix is its own operator-gated decision. Recommended consolidated-fix
|
||||
order unchanged: (1) clipboard `printf` [R2a, 10 files, highest blast radius], (2) component scaffold [3
|
||||
files], (3) bare reference paths [grep-driven], then the per-surface items — now including **report Step 6c
|
||||
misroute (the one R4 MAJOR)**, report Step 8b `Write`, the sibling-drift reconciliation (import 6a +
|
||||
ab-test ER), analyze severity scales, and the over-declaration trims. Local-only (hardening-class), pushed
|
||||
per the 2026-06-30 operator delegation (public catalog, no secrets).
|
||||
|
||||
**Cumulative cold-review coverage: 23/29** (review.md S1=4 · R2a=5 · R2b=4 · R3=4 · R4=6). Remaining: **R5
|
||||
(Grow+Router)** — strategy · competitive · monetize · outreach · profile · linkedin (6 surfaces) → 29/29.
|
||||
258
docs/hardening/review-R5.md
Normal file
258
docs/hardening/review-R5.md
Normal file
|
|
@ -0,0 +1,258 @@
|
|||
---
|
||||
type: cold-review
|
||||
batch: R5
|
||||
journey: "Grow + Router — growth/authority surfaces + the command router (FINAL round)"
|
||||
scope: "FROZEN committed files at HEAD 4109fe7 (clean tree; post-hardening cold pass)"
|
||||
method: "2 independent cold Opus reviewers for the round (intent + correctness), each covering all 6 surfaces, no cross-feed; every mechanical claim tool-grounded (anti-fabrication mandate); reviewers carry NO drafting-session context. Divergences re-grounded by main before registration."
|
||||
surfaces: [strategy, competitive, monetize, outreach, profile, linkedin]
|
||||
reviewers:
|
||||
- "intent-lens (conformance: intent delivery + cross-ref resolution + class predicates + graceful degradation + thought-leadership terminology ban)"
|
||||
- "correctness-lens (internal consistency + bound-vs-canonical + step/phase arithmetic + allowed-tools completeness + dead-ref / executable-path checks)"
|
||||
class: "guided/stateful (strategy·competitive·monetize·outreach·profile — primary artifact produced · subagent targets resolve · graceful degradation) + routing (linkedin — every emitted /linkedin:Y resolves)"
|
||||
status: "COMPLETE — all 6 surfaces reviewed (strategy, competitive, monetize, outreach, profile, linkedin). R5 completes 29/29 cold-review coverage."
|
||||
verdict: MINOR
|
||||
counts: { BLOCKER: 0, MAJOR: 0, MINOR: 3, SUGGESTION: 2 }
|
||||
---
|
||||
|
||||
# Cold review — R5 (Grow + Router) · FINAL round → 29/29 coverage
|
||||
|
||||
Independent post-hoc cold review of the 6 remaining surfaces — the five Grow-journey
|
||||
guided/stateful commands (`strategy`, `competitive`, `monetize`, `outreach`, `profile`) plus the
|
||||
`linkedin` router — on the FROZEN committed files (HEAD `4109fe7`). Mirrors the S1 `review.md` +
|
||||
R2a + R2b + R3 + R4 model (the cold-review method that did **not** fabricate): read-and-show before
|
||||
assert, every `file:line` tool-confirmed, reviewers carry no drafting-session context. The
|
||||
per-command interactive gate (`log.md`) already passed these; this pass adds the **independent**
|
||||
axis that gate never had. **R5 completes the sweep: 23/29 + 6 = 29/29 cold-review coverage.**
|
||||
|
||||
**Resolution integrity — PASS across all 6 surfaces (the headline R5 result).** Both blind lenses
|
||||
independently confirmed, target-by-target against their own `ls`/`test -f`:
|
||||
- **2/2** `subagent_type: linkedin-studio:X` refs resolve — `strategy-advisor` (`strategy.md:153`),
|
||||
`network-builder` (`outreach.md:171`).
|
||||
- **28/28** unique `/linkedin:Y` route tokens resolve to `commands/Y.md` (router + cross-command
|
||||
suggestions).
|
||||
- **11/11** router-suggested agents (named in `linkedin.md`) resolve to `agents/*.md`.
|
||||
- **2/2** helper-script invocations resolve: `outreach.md`'s `state-updater.mjs --record-outreach`
|
||||
(flags `--date/--track/--partner/--stage/--next/--due` match `state-updater.mjs:394-411`
|
||||
byte-for-byte) and `linkedin.md`'s `queue-manager.mjs` import (`queueUpcoming`/`queueOverdue`/
|
||||
`queueFormatSummary` exist at `queue-manager.mjs:52,94,112`).
|
||||
- **0 under-declared tools** — every body-invoked tool (Task/Read/Write/Edit/Bash/WebSearch/
|
||||
AskUserQuestion) is in the surface's `allowed-tools`; only harmless `Glob`/`Grep` over-declarations.
|
||||
- **0 dead executable Read/Bash targets** — every `references/*`, `${CLAUDE_PLUGIN_ROOT}/skills/…`,
|
||||
and script path on an executable path verified present.
|
||||
|
||||
**No broken invocation, no missing primary artifact, no failing gate, no runtime-breaking
|
||||
contradiction anywhere in R5 → 0 MAJOR, verdict MINOR (advisory, not REWORK).** This is the
|
||||
cleanest batch of the sweep.
|
||||
|
||||
**Independence cross-check — both outcomes recurred (the case for two lenses, again):**
|
||||
1. **Convergence:** both blind lenses independently surfaced the same `monetize.md` description↔body
|
||||
scope mismatch and the same `thought leader` terminology cluster — high confidence these are real.
|
||||
2. **Divergence (re-grounded by main):**
|
||||
- **Terminology severity** — intent-lens called the `thought leader` hits MINOR (it owns the
|
||||
conformity predicate; memory `no-thought-leadership-phrase` makes this a *standing plugin rule*,
|
||||
not a nicety), correctness-lens called them SUGGESTION (no runtime break). **Main ruling:
|
||||
MINOR** — a direct violation of an explicit plugin terminology rule in user-facing strings is a
|
||||
conformity defect; it is not MAJOR (no runtime break).
|
||||
- **`monetize` scope mismatch severity** — intent-lens SUGGESTION ("description under-claims
|
||||
body"), correctness-lens MINOR ("description contradicts its own 0-1K Stage 1"). **Main ruling:
|
||||
MINOR** — it is a genuine self-contradiction about the command's scope, not a mere under-claim.
|
||||
- **Lens-unique:** correctness-lens alone caught the `monetize` Audience-Size scorecard
|
||||
arithmetic; intent-lens alone caught the bare-relative-path robustness gap. Both re-grounded by
|
||||
main below and kept.
|
||||
|
||||
---
|
||||
|
||||
## strategy.md — VERDICT: MINOR
|
||||
|
||||
- **Intent delivered:** yes. Phase auto-detect from state (`:32-44`), phase strategy + delegation
|
||||
to `strategy-advisor` (`:153`), trajectory overlay (`:250-284`), authority building Phase 2+
|
||||
(`:286-419`), stall points (Step 4), 90-day plan (Step 5), metrics (Step 6).
|
||||
- **Resolution:** all resolve — `subagent_type: linkedin-studio:strategy-advisor` (`:153`) →
|
||||
`agents/strategy-advisor.md` ✓; routes `/linkedin:profile` (`:292,:360`) ✓; `Task` declared.
|
||||
- **Class predicate (guided/stateful):** growth-plan artifact produced ✓; subagent resolves ✓;
|
||||
graceful degradation present ✓ (`follower_count` 0/missing handled `:42`; "If no milestone data:
|
||||
Skip this step" `:284`; authority skipped in Phase 0-1 `:290`).
|
||||
- **Arithmetic (correctness-lens, grounded):** 5 phases (0–4) consistent between Step 0.5
|
||||
auto-detect ranges and Step 2 headers; step numbering monotonic (0.5,1,2,3,3.5,3.6,4,5,6), no
|
||||
gaps/dupes. PASS.
|
||||
- **allowed-tools:** declared {Read, Glob, Grep, AskUserQuestion, Task}; under-declared {} ✓;
|
||||
over {Glob, Grep} (harmless).
|
||||
- **Findings:**
|
||||
1. **[MINOR]** `strategy.md:371` "Engaging with other thought leaders" — `thought leader`
|
||||
terminology-ban hit (off-primary checklist label). Part of the cross-cutting cluster.
|
||||
2. **[MINOR]** bare relative paths for all file loads; **0×** `${CLAUDE_PLUGIN_ROOT}` (grounded
|
||||
`grep -c` = 0, vs 1–10× in competitive/monetize/outreach/linkedin) → `Read` resolves against
|
||||
cwd, not plugin root. Has `Glob` in allowed-tools as a fallback so it degrades, not fatal.
|
||||
Folds into systemic finding #3 (bare ref-paths).
|
||||
|
||||
## competitive.md — VERDICT: MINOR
|
||||
|
||||
- **Intent delivered:** yes. Competitor analysis template (Step 2), landscape map (Step 3),
|
||||
gap/opportunity matrix (Step 4), differentiation plan (Step 5), inspired takeaways (Step 6),
|
||||
ethics note.
|
||||
- **Resolution:** fully self-contained — **no** `subagent_type`, **no** `/linkedin:` routes;
|
||||
`allowed-tools` (Read/Glob/WebSearch/AskUserQuestion) correctly omits `Task`. ✓
|
||||
- **Class predicate (guided/stateful):** competitive-analysis artifact produced ✓; no subagents to
|
||||
resolve ✓; graceful degradation ✓ (Step 1 user-input/WebSearch-driven, runs with no state).
|
||||
- **Arithmetic:** Steps 0–6 linear/monotonic; no stated totals to miscount. PASS.
|
||||
- **allowed-tools:** declared {Read, Glob, WebSearch, AskUserQuestion}; under {} ✓; over {Glob}.
|
||||
- **Findings:**
|
||||
1. **[MINOR]** `thought leader(s)` appears **4×** — including the frontmatter `description`
|
||||
(`:4`, **user-facing**, shows in command listings) and the opening promise (`:17`), plus
|
||||
`:31,:34`. **Worst terminology offender of the six.** Conformity defect, no runtime break.
|
||||
|
||||
## monetize.md — VERDICT: MINOR
|
||||
|
||||
- **Intent delivered:** yes. All 8 steps present: scorecard (1), stage strategy (2), lead-magnet
|
||||
blueprint (3), funnel calendar (4), CTA + A/B variants (5), Featured optimization (6), revenue
|
||||
model (7), tracking dashboard (8).
|
||||
- **Resolution:** all resolve — no `subagent_type`; routes `/linkedin:post` + `/linkedin:pipeline`
|
||||
(`:367`) both exist. ✓
|
||||
- **Class predicate (guided/stateful):** monetization-plan artifact produced ✓; no subagents ✓;
|
||||
graceful degradation ✓ ("Stage 1: Visibility (0-1K followers)" `:92` serves brand-new users).
|
||||
- **Gating honesty:** does NOT hard-gate; serves 0-1K. The real state-read gate is the router's
|
||||
soft prepend (`linkedin.md:175`, `<1000` → prepend, continue anyway) — consistent with outreach.
|
||||
- **Findings:**
|
||||
1. **[MINOR]** **description↔body scope self-contradiction** — `monetize.md:6` "Works from 1K+
|
||||
followers" contradicts its own Stage 1 "Visibility (**0-1K** followers, score 0-30)" (`:92`,
|
||||
`:81`) and the router's "they work at any follower count" (`linkedin.md:126`). The body
|
||||
genuinely covers sub-1K; the description misstates scope. Off-primary (a frontmatter string)
|
||||
but user-facing. **Strongest non-terminology R5 finding.** Fix: align `:6` to "any follower
|
||||
count (compounds at 1K+)".
|
||||
2. **[MINOR]** `thought leaders` in the frontmatter `description` (`:4`, user-facing). Part of the
|
||||
cross-cutting cluster.
|
||||
3. **[SUGGESTION]** Audience-Size scorecard arithmetic (`:51-56`): sub-items +5/+10/+15/+5/+5 sum
|
||||
to **+40** against a **`/25`** cap if read additively, whereas the other three categories each
|
||||
sum to exactly 25 (e.g. Engagement Quality 5+5+10+3+2=25). The follower tiers (1K/5K/10K) are
|
||||
clearly intended mutually-exclusive (one tier) but aren't marked as such → a literal additive
|
||||
read overflows. AI-interpreted (not machine-summed) so intent is recoverable; cosmetic, but it
|
||||
feeds the band that selects the stage. Fix: mark the three follower tiers "(pick one)".
|
||||
|
||||
## outreach.md — VERDICT: PASS
|
||||
|
||||
- **Intent delivered:** yes, thoroughly. Two-track (collab + speaking) orchestrator with a
|
||||
Capability Checklist mapping every predecessor function to a step (`:31-62`) + 10 steps + state
|
||||
persistence.
|
||||
- **Resolution:** all resolve — `subagent_type: linkedin-studio:network-builder` (`:171`) ✓;
|
||||
routes `/linkedin:strategy` (`:112`), `/linkedin:firsthour`+`/linkedin:outreach` (`:1078`),
|
||||
`/linkedin:calendar` (`:1097`) all exist ✓; **Bash executable path sound** — Step 8c's
|
||||
`state-updater.mjs --record-outreach` (`:1084-1092`) matches the script's handler
|
||||
(`state-updater.mjs:294,:394-411`) byte-for-byte. ✓
|
||||
- **Class predicate (guided/stateful):** outreach-plan + persisted pipeline produced ✓; subagent +
|
||||
script resolve ✓; graceful degradation ✓ (Step 2a "Not ready: <3 met → build foundation first" +
|
||||
recommend `/linkedin:strategy` `:112`).
|
||||
- **Arithmetic (correctness-lens, grounded):** "12 collab formats" → 12 (`FORMAT 1–12`); "4 talk
|
||||
templates" → A–D; "5 phases" → PHASE 1–5; scorecards 4×/25=/100 and 5×/5=/25 check out; step
|
||||
numbering monotonic. PASS.
|
||||
- **allowed-tools:** declared {Read, Glob, WebSearch, AskUserQuestion, Task, Bash}; under {} ✓;
|
||||
over {Glob}.
|
||||
- **Gating honesty:** "1K+ followers" self-report (`:100,:143`) consistent with router's `~1K` soft
|
||||
gate (`linkedin.md:120,:176`) and monetize's 1K. ✓
|
||||
- **Findings:**
|
||||
1. **[SUGGESTION]** `thought leader` inside a WebSearch query template (`:229`,
|
||||
`"[your niche] linkedin thought leader"`). It is a *search string* targeting how others
|
||||
self-label (intentional — to find such profiles), so the **lowest-priority** instance of the
|
||||
terminology cluster — but the literal string is still in the plugin.
|
||||
|
||||
## profile.md — VERDICT: PASS
|
||||
|
||||
- **Intent delivered:** yes. Relevance-model context (`:20-44`), Profile SEO + per-section keyword
|
||||
targets (`:46-79`), 7-section audit walkthrough (`:82-199`), profile-content alignment check
|
||||
(`:200-212`), prioritized action plan (`:214-231`), alignment test (`:232-238`).
|
||||
- **Resolution:** self-contained — no `subagent_type`, no `/linkedin:` routes; `allowed-tools`
|
||||
Read/AskUserQuestion (tightest frontmatter of the six). ✓
|
||||
- **Class predicate (guided/stateful + topic-relevance-audit):** the topic-relevance audit **is**
|
||||
actually performed (the body *is* that audit) ✓; artifact (audit + action plan) produced ✓;
|
||||
graceful degradation ✓ (every step AskUserQuestion-driven, runs with zero state). Notable
|
||||
verification discipline: explicitly refuses to fabricate a scoring breakdown (`:28,:32-36,:179`).
|
||||
- **Arithmetic:** Sections 1–7 monotonic; profile-field limits (headline 220, About 2,600) are
|
||||
LinkedIn field limits, NOT post hook/length bounds — no canonical contradiction. PASS.
|
||||
- **Terminology — IMPORTANT NON-VIOLATION:** the two `thought leader` hits (`:79,:101`) are
|
||||
**legitimate negative examples** — the command explicitly tells the user to AVOID the phrase
|
||||
(`:79` lists it with "guru"/"ninja" as keyword-wasters; `:101` is a "Weak example"). Both lenses
|
||||
agree; main confirmed by reading both lines. **profile.md models the correct behavior.**
|
||||
- **Findings:** none of defect class.
|
||||
1. **[Note — systemic #3]** bare relative paths; **0×** `${CLAUDE_PLUGIN_ROOT}` and no `Glob`
|
||||
fallback → most-exposed instance of the bare-path robustness item. BUT correctness-lens
|
||||
verified all three referenced files exist (`test -f` OK), so this is a cwd-robustness concern,
|
||||
not a dead ref. Folds into systemic #3; profile.md is otherwise the cleanest surface of R5.
|
||||
|
||||
## linkedin.md (router) — VERDICT: MINOR
|
||||
|
||||
- **Intent delivered:** yes. Status line (`:19-29`), upcoming/overdue posts via queue (`:31-57`),
|
||||
five-journey menu with front-doors (`:59-127`), gating rule (`:122-127`), interactive menu
|
||||
(`:129-146`), direct-routing table (`:156-198`).
|
||||
- **Resolution (routing class):** **all 28 unique route tokens resolve** to `commands/*.md`
|
||||
(verified token-by-token vs `ls commands/`); all 11 suggested agents resolve; `queue-manager.mjs`
|
||||
node call (`:35-42`) uses exports that all exist (`:52,:94,:112`). **The router advertises nothing
|
||||
that doesn't exist.** ✓ Correctly has no `Task` (delegate-only).
|
||||
- **Class predicate (routing):** every route resolves ✓; graceful degradation ✓ (missing state
|
||||
"No LinkedIn state tracked yet" `:28`; empty queue `:57`; follower segment only if
|
||||
`follower_count > 0` `:30`).
|
||||
- **Gating honesty/consistency:** `:175-176` give monetize + outreach the **same** soft state-read
|
||||
gate (`<1000` → prepend, continue anyway); `:122-127` document the soft-gate design honestly
|
||||
("they work at any follower count… competitive is **not** gated"). Consistent with both command
|
||||
bodies. ✓
|
||||
- **Findings:**
|
||||
1. **[MINOR]** `linkedin.md:118` "Competitive analysis of other thought leaders" — `thought
|
||||
leader` terminology-ban hit in the **user-facing** routing-menu table. Part of the cluster.
|
||||
|
||||
---
|
||||
|
||||
## Summary table
|
||||
|
||||
| Surface | Verdict | MAJOR | MINOR | SUGGESTION |
|
||||
|---|---|---|---|---|
|
||||
| strategy.md | MINOR | 0 | 2 | 0 |
|
||||
| competitive.md | MINOR | 0 | 1 | 0 |
|
||||
| monetize.md | MINOR | 0 | 2 | 1 |
|
||||
| outreach.md | PASS | 0 | 0 | 1 |
|
||||
| profile.md | PASS | 0 | 0 | 0 (1 systemic-#3 note) |
|
||||
| linkedin.md | MINOR | 0 | 1 | 0 |
|
||||
| **R5 total** | **MINOR** | **0** | **3 distinct** | **2 distinct** |
|
||||
|
||||
(Per-surface MINOR counts include shared cross-cutting findings; the 3 *distinct* R5 MINOR findings
|
||||
are: the terminology cluster, the monetize scope contradiction, and the bare-path robustness item.)
|
||||
|
||||
## Cross-cutting findings (for the consolidated fix-pass)
|
||||
|
||||
1. **[MINOR · NEW R5 systemic] `thought leader` terminology cluster — 5 surfaces, 9 instances.**
|
||||
competitive `:4`(description, user-facing)/`:17`/`:31`/`:34` · monetize `:4`(description,
|
||||
user-facing) · linkedin `:118`(menu, user-facing) · strategy `:371`(checklist) · outreach
|
||||
`:229`(WebSearch string, lowest priority). **profile `:79,:101` are NON-violations** (correct
|
||||
negative examples — do not "fix" them). Memory `no-thought-leadership-phrase` confirms this is a
|
||||
standing plugin terminology rule. **Recommended fix:** one terminology sweep replacing the
|
||||
user-facing instances first ("thought leaders" → "creators"/"experts"/"voices in your niche"),
|
||||
leaving profile's avoid-list intact and the outreach search-string as lowest priority. No runtime
|
||||
risk — the buzzword gate scopes to post content, not command markdown.
|
||||
|
||||
2. **[MINOR · per-flate] `monetize.md:6` description↔body scope self-contradiction.** "Works from
|
||||
1K+ followers" vs the body's own 0-1K Stage 1 (`:92`) and the router's "any follower count"
|
||||
(`linkedin.md:126`). Fix: align the description to the body's actual sub-1K-onward range.
|
||||
|
||||
3. **[MINOR · folds into existing systemic #3] bare ref-paths — strategy + profile (0×
|
||||
`${CLAUDE_PLUGIN_ROOT}`).** profile most exposed (no `Glob` fallback). Refs verified to exist, so
|
||||
cwd-robustness not a dead ref. Reconcile in the repo-wide path-style pass already scoped by
|
||||
systemic #3 (R3/R4 found the same pattern in batch/pipeline/firsthour/analyze/report).
|
||||
|
||||
4. **[SUGGESTION · per-flate] `monetize.md:51-56` Audience-Size scorecard sums to +40 vs `/25`
|
||||
cap.** Follower tiers intended mutually-exclusive but unmarked. Fix: annotate "(pick one)".
|
||||
|
||||
**R5 adds NO new ★ systemic finding to the existing three** (clipboard, scaffold-band, bare-paths) —
|
||||
the only genuinely new cross-cutting item is the terminology cluster, which is a conformity sweep,
|
||||
not a code-behavior defect.
|
||||
|
||||
## Verification
|
||||
|
||||
- Spot-checks re-grounded by main (this file): `grep -rniE 'thought.?leader'` over the 6 (9 hits,
|
||||
classifications above); `sed -n '1,8p'`/`'92p'`/`'48,64p' monetize.md` (scope + scorecard);
|
||||
`grep -c CLAUDE_PLUGIN_ROOT` per surface (strategy 0, competitive 4, monetize 10, outreach 10,
|
||||
profile 0, linkedin 1).
|
||||
- `bash scripts/test-runner.sh` → see STATE telling (expected 138 passed / 0 failed, floor 123) —
|
||||
unchanged by this review (review is read-only; no command edits).
|
||||
|
||||
## Sweep status after R5
|
||||
|
||||
**29/29 cold-review coverage reached** (S1 `review.md` + R2a + R2b + R3 + R4 + R5). The independent
|
||||
cold-review phase is COMPLETE; the v1.0.0 review blocker is lifted. Remaining v1.0.0 work: the
|
||||
consolidated fix-pass (operator-gated — review FINDS, fix is a separate decision) and the GUI.
|
||||
209
docs/ingestion-guard/plan.md
Normal file
209
docs/ingestion-guard/plan.md
Normal file
|
|
@ -0,0 +1,209 @@
|
|||
# Ingestion-guard integration plan — when & where to wire `llm-ingestion-guard`
|
||||
|
||||
> **Status: `planned` (2026-07-16).** Plan only — nothing wired yet, per the adoption brief's
|
||||
> "don't implement now" instruction. This document is the repo's durable answer to *when* and *where* a
|
||||
> write-time ingestion guard earns its place at our persist gates.
|
||||
>
|
||||
> **Guard:** `llm-ingestion-guard` `v0.2` (alpha, public API may change). Write-time sibling of query-time
|
||||
> chatbot guardrails; hardens *untrusted content flowing through an LLM step into a persisted,
|
||||
> downstream-read store*. Python, stdlib-only core, 3.10+.
|
||||
>
|
||||
> **Adoption brief (authoritative, self-contained):**
|
||||
> `https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security/raw/branch/main/docs/ADOPTION-BRIEF.md`
|
||||
> (§7 planning checklist, §3 the 8-step contract, §4 the OKF adapter, §8 honest limitations).
|
||||
>
|
||||
> Grounded against `882f6ee` by two independent read-only surveys (ingest surface + persist seams),
|
||||
> 2026-07-16. Every seam below carries a `file:line` anchor.
|
||||
|
||||
## 1. Decision
|
||||
|
||||
**Status = `planned`.** The plugin has live untrusted-ingest paths (a react-to-URL command, an auto-fetching
|
||||
trend agent, an external-research newsletter fan-out), and it persists their output into stores a downstream
|
||||
agent later reads as trusted context. That satisfies the brief's §7 decisive condition. Nothing is wired.
|
||||
The guard is registered here as a **dependency to add before the first *automated* external-origin ingest
|
||||
path goes fully live** — and one such path (trends → session re-injection, §5.1) is arguably already live,
|
||||
so it is the first integration candidate, not a distant one.
|
||||
|
||||
We do **not** implement now. The brief is explicit ("kun planlegg og oppdater planene"), and wiring is gated
|
||||
on an unresolved interop decision (§7).
|
||||
|
||||
## 2. What the guard is (and is not) for us
|
||||
|
||||
It is **not** a query-time guardrail between the user and the model. It is a **write gate**: the last place
|
||||
the provenance of a piece of content is still known before it is committed to a store that a *later* agent
|
||||
reads as trusted. For this plugin that store is the second-brain (`brain/profile.md`, `ingest/published/`),
|
||||
the trends store, the specifics-bank, the post queue, and the state file's `## Recent Posts` — all of which
|
||||
are re-surfaced into future model context. A poisoned concept committed at write time is read back later with
|
||||
its origin forgotten; the write gate is the only place to catch it.
|
||||
|
||||
## 3. Ingest-surface analysis (untrusted vs first-party)
|
||||
|
||||
Scored against the brief's §7 checklist. "Downstream trusted reader" = a command/agent/hook that later reads
|
||||
the store **as context**, which is what turns a write into a poisoning surface.
|
||||
|
||||
| # | Path | Origin | Untrusted? | Persist target | Downstream trusted reader | Live? |
|
||||
|---|------|--------|-----------|----------------|---------------------------|-------|
|
||||
| 1 | **trend-spotter agent** (`agents/trend-spotter.md`) → trends CLI `capture` (`scripts/trends/src/cli.ts:294`) | Auto-fetched web/vendor/regulator content (WebSearch/WebFetch + research MCPs) | **YES — external** | `trends/trends.json` — `source`/`title`/`url`/`summary` stored **verbatim** (`scripts/trends/src/item.ts:24-48`) | **`session-start.mjs:38-78` auto-reinjects into the next session's context — no human in the loop** | **LIVE** |
|
||||
| 2 | **`brain ingest` / `scanInbox`** (`scripts/brain/src/ingest.ts:172-192,:247`) | User's own published posts, dropped into `ingest/inbox/` (manual) | Origin first-party today; **untrusted-*capable*** (drop-zone accepts any file; SB-S4 connector would automate it) | `ingest/published/<id>.md` | `voice-trainer` gold source (`agents/voice-trainer.md:136,144`); `brain consolidate` → `brain/profile.md`; `assemble`/`reconcile` | LIVE (manual) |
|
||||
| 3 | **`/linkedin:newsletter`** research fan-out (`commands/newsletter.md:461-475`) | Open-web research agents (WebSearch/WebFetch) | **YES — external** | specifics-bank `ekstern` bindings (`scripts/specifics-bank/src/kilder.ts:51-52`) + `NN-kilder.md` + `queue.json` | Edition prose + sources ledger; queue readers | LIVE |
|
||||
| 4 | **`/linkedin:react`, `:post`, `:pipeline`, `:batch`** URL ingest (`commands/react.md:52`, `post.md:62`, `pipeline.md:13`, `batch.md:12`) | External URL (news, blog, YouTube, social threads) via WebFetch | **YES — external** | react/post: clipboard + state `## Recent Posts` **metadata only** (raw content not persisted); pipeline/batch: draft files + `queue.json` | state `## Recent Posts` re-injected (`user-prompt-context.mjs:109-117`, `session-start.mjs:341`); queue readers | LIVE |
|
||||
| 5 | **`/linkedin:import`** CSV (`scripts/analytics/src/parsers/csv-parser.ts`) | User's own LinkedIn analytics CSV export | First-party origin; **container-layer** parse surface | `analytics/posts/*.json` (`storage.ts:143-160`) | `report`/`audit`/`analyze`, `analytics-interpreter`, `brain assemble` | LIVE |
|
||||
| 6 | **`/linkedin:competitive`, `:outreach`** (`commands/competitive.md:33`, `outreach.md:223`) | WebSearch of competitor/partner/event content | YES — external | **Nothing durable** (inline report only) | — (no persist) | LIVE |
|
||||
| 7 | Received **third-party OKF bundle** | External bundle | YES — external | *does not exist* | — | **FUTURE / not built** |
|
||||
| 8 | `setup`, `onboarding`, `first-post`, `quick`, `specifics-bank`, voice-samples | User's own typed/pasted content | No — first-party | state file / voice-samples / specifics-bank | content commands | LIVE |
|
||||
|
||||
## 4. Where the guard applies — and where it deliberately does not
|
||||
|
||||
**Applies (wire here):** the untrusted boundaries — rows 1–5. Trust follows the data's *origin*, not the
|
||||
insertion channel (brief §7): a manual paste of an external article (`/linkedin:react`) is still external.
|
||||
|
||||
**Does not apply (out of scope by design):**
|
||||
- **First-party authoring** (row 8): onboarding, `setup`, typed post ideas, `specifics-bank` (human-only),
|
||||
voice-samples, the user's own profile edits. The guard's threat model does not target trusted-author
|
||||
in-place edits.
|
||||
- **Fetch-but-no-persist** (row 6): `competitive`/`outreach` fetch external content but write nothing
|
||||
durable — there is no downstream-trusted store to poison, so the *write-time* guard has no seam. (Their
|
||||
risk is query-time, a different tool's job.)
|
||||
- **Row 2 today** is first-party by origin (the user's own posts). It becomes an untrusted boundary the
|
||||
moment SB-S4 (the EU/EEA DMA connector) or any received-bundle path automates the inbox — see §9.
|
||||
|
||||
## 5. Integration points (persist gates) + minimal wiring
|
||||
|
||||
**Key structural finding:** every external fetch in this plugin goes through the *model's* WebFetch/WebSearch/
|
||||
MCP tools — there is **no raw-HTTP `your_model()` seam in plugin code**. The fetch → transform → persist
|
||||
pipeline is: *model tool call → agent reasoning → deterministic CLI write*. So the guard's classic two-bookend
|
||||
model (`prepare_input` → `your_model` → `screen_output`) only **half-maps**: `screen_output`
|
||||
(scan-before-persist) wires in cleanly at the deterministic CLI write points; `prepare_input` (sanitize+fence
|
||||
before the model) has no clean code seam because the fetch and transform happen *inside the model's turn*
|
||||
(§6).
|
||||
|
||||
**Coverage gap to know:** `hooks/scripts/content-gatekeeper.mjs` is the only `PreToolUse(Write|Edit)`
|
||||
choke-point, but it inspects the file **path, never the content bytes**, and only fires for the **Write/Edit
|
||||
tool** on drafts/assets paths. The four durable trusted-context stores below are all written via
|
||||
**Bash-invoked `node`/`tsx`**, so they **bypass the gate entirely**. A byte-level guard must wire at the CLI
|
||||
write points, not solely at `content-gatekeeper`.
|
||||
|
||||
Ranked by automated-reinjection risk (highest first):
|
||||
|
||||
### 5.1 trends `capture` — the one live automated poison→reinject loop *(priority 1)*
|
||||
- **Seam:** before `store.ts` persists to `trends/trends.json`, in `scripts/trends/src/cli.ts` `capture`.
|
||||
- **Wiring:** `screen_output(item.title + "\n" + item.summary, PRESET_USER_UPLOAD)`; on
|
||||
`Disposition.FAIL_SECURE`, route the item into the CLI's existing `errors[]` channel
|
||||
(`cli.ts:294-320`) instead of persisting — a drop, not a crash.
|
||||
- **Why first:** external content, stored verbatim, auto-surfaced back into future model context by
|
||||
`session-start.mjs` with no human gate. This is the sharpest write→trusted-read loop in the plugin.
|
||||
|
||||
### 5.2 brain `ingest` / `scanInbox` — the voice/profile gold seam *(priority 2)*
|
||||
- **Seam:** `writePublished(rec)` in `scripts/brain/src/ingest.ts:172-192`; especially the `scanInbox`
|
||||
(`:247`) path that reads user-dropped `ingest/inbox/*.md`.
|
||||
- **Wiring:** `screen_output(body, ...)` before write; `FAIL_SECURE` → do not promote to
|
||||
`ingest/published/`, log to a rejects sidecar. Once a bundle-shaped receive lands, switch to
|
||||
`okf.import_bundle` (§8).
|
||||
- **Why:** `ingest/published/` is the ranked-#1 gold source for `voice-trainer` and feeds `profile.md` via
|
||||
consolidation — the highest-trust downstream read in the plugin. Note the existing `provenance=published`
|
||||
guard here is an **anti-model-collapse** control (authorship axis), **not** an anti-injection control
|
||||
(origin axis) — the two are orthogonal; this seam has the former, not the latter.
|
||||
|
||||
### 5.3 newsletter research → specifics-bank `ekstern` bindings *(priority 3)*
|
||||
- **Seam:** before an external research finding persists as an `ekstern` binding
|
||||
(`scripts/specifics-bank/src/kilder.ts:51-52`).
|
||||
- **Wiring:** `screen_output` on the claim text + source; `FAIL_SECURE` → quarantine, surface to the operator
|
||||
in the fact-check sweep the newsletter pipeline already runs.
|
||||
|
||||
### 5.4 analytics CSV import — the container-layer gate *(priority 4)*
|
||||
- **Seam:** `saveBatch` in `scripts/analytics/src/utils/storage.ts:143-160` (or in `parseLinkedInCSV`).
|
||||
- **Wiring:** row-content scan (CSV formula-injection `= + - @`, active-content) complementing the existing
|
||||
**filename-only** sanitization (`sanitizeDate/Id/Week/Month` + `verifyPathWithinDirectory`,
|
||||
`storage.ts:110-137`). Lower priority: origin is first-party and the JSON is never re-emitted to a
|
||||
spreadsheet, so the practical blast radius is small — but there is currently **no** row-content
|
||||
sanitization layer, so it is a real (if narrow) gap.
|
||||
|
||||
## 6. The `prepare_input` caveat (honest scope of what we can wire)
|
||||
|
||||
The input-side bookend (`sanitize` + fence before the model, contract steps 1–2) has **no clean code seam**
|
||||
here: the untrusted content is fetched by the model's WebFetch tool and transformed by the drafting agent
|
||||
*within the same model turn*, so plugin code never holds the raw input to wrap. Partial mitigations exist —
|
||||
a sanitize pass on fetched text inside the URL commands' prompts, or reading `tool_input.content` in
|
||||
`content-gatekeeper` for the Write-tool draft path — but neither is the clean `prepare_input(untrusted)`
|
||||
call the library assumes. What we **can** wire cleanly and fully is `screen_output` at the persist gates (§5).
|
||||
|
||||
Partly, the contract's *real* security already holds structurally: the drafting agents largely reason over
|
||||
fetched text (close to "tool-less transform"), and the persist step is a deterministic CLI ("output as data,
|
||||
parsed to a schema"). The lexicon/entropy scan is defense-in-depth on top of that, not the wall.
|
||||
|
||||
## 7. Python ↔ Node interop — the real integration cost
|
||||
|
||||
The guard is **Python** (stdlib, 3.10+). This plugin's runtime is **Node ESM `.mjs` hooks (deliberately
|
||||
zero-npm-dep) + TypeScript engine via `tsx`**. The only Python in the repo is one **build-time** script
|
||||
(`hooks/scripts/compile-hooks.py`), never on a data path. So wiring the guard means crossing a subprocess
|
||||
boundary. Options:
|
||||
|
||||
| Option | Shape | Trade-off |
|
||||
|--------|-------|-----------|
|
||||
| **(a) `spawnSync('python3', …)` at the engine CLI write points** *(recommended)* | The `scripts/{trends,brain,specifics-bank,analytics}` CLIs (already `tsx`, not the zero-dep hook hot path) shell out to a `python -m llm_ingestion_guard` scan | Adds a `python3` + `pip install llm-ingestion-guard` runtime dependency to the *engine layer only*; keeps the zero-dep Node **hooks** untouched. Cleanest fit. |
|
||||
| (b) Port minimal `sanitize` + `scan_output` to a `.mjs` twin | Reimplement in Node | Defeats the point of adopting a *maintained* guard; the coverage matrix (126/126) would not apply to the port. Rejected unless a hard no-Python constraint appears. |
|
||||
| (c) Bash step inside the command, not the hook | Command invokes the scan before the CLI write | Non-deterministic (depends on the agent running the step); weaker than a code-enforced gate. |
|
||||
|
||||
**Blocker to resolve first:** whether a `python3` + one-package runtime dependency is acceptable given the
|
||||
plugin's zero-dep design value. This is the gate on any wiring work.
|
||||
|
||||
## 8. OKF `import_bundle` — future / conditional
|
||||
|
||||
The plugin is **export-only** toward OKF: its brain *emits* OKF-compatible form (`type:` + per-level
|
||||
`index.md` + root `okf_version`, landed 2026-06-26, `docs/okf-convergence-brief.md`). There is **no
|
||||
`import_bundle` and no third-party-bundle receive path** anywhere in the repo, and inbox auto-classify/convert
|
||||
is explicitly deferred (brief §11). So the guard's `okf.import_bundle(bundle, origin=EXTERNAL,
|
||||
channel=AUTOMATIC)` adapter has **no seam today**.
|
||||
|
||||
It becomes relevant if/when either lands: **(i)** SB-S4 — the EU/EEA DMA portability connector auto-feeding
|
||||
`ingest/inbox/`; or **(ii)** a cross-plugin shared retrieval skill (a *separate standalone plugin*, per the
|
||||
convergence brief §8) that merges *other* plugins' brains. At that point, wire `okf.import_bundle` at
|
||||
`scanInbox` with `allow_reserved` chosen per channel (received bundle → `True`; materialised individual
|
||||
uploads → `False`, per brief §4).
|
||||
|
||||
## 9. When — roadmap triggers
|
||||
|
||||
1. **Now:** `planned`. No wiring. (This doc + the STATE.md marker.)
|
||||
2. **First wiring candidate — trends `screen_output` (§5.1):** the trends→session-reinjection loop is already
|
||||
live, so per the brief's "include it before the first untrusted ingest path goes live," this is the
|
||||
earliest concrete target once §7 is resolved.
|
||||
3. **Hard trigger (not optional) — before SB-S4 or any received-bundle path:** the EU/EEA DMA connector, or a
|
||||
cross-plugin shared skill, turns `ingest/inbox/` from "user's own manual paste" into an automated
|
||||
external-origin ingress. Wire §5.2 (and §8's `import_bundle`) **before** that path goes live — this is the
|
||||
brief's "when, not if" moment.
|
||||
4. **Opportunistic:** §5.3 (newsletter) and §5.4 (CSV container-layer) can ride whichever hardening session
|
||||
touches those CLIs.
|
||||
|
||||
Sequencing note: none of this is on the current `docs/plan-2026-07/` roadmap (N1–N32). It is a new,
|
||||
security-scoped work item to slot in after §7 is decided — most naturally as its own hardening slice, not by
|
||||
displacing the agreed N-plan.
|
||||
|
||||
## 10. Honest limitations (carried from brief §8 — a green scan is not "safe")
|
||||
|
||||
- **Semantic / factual poisoning is invisible** to lexicon + entropy — a plausible-but-wrong concept (wrong
|
||||
metric, wrong runbook step) carries no suspicious token and passes clean. **Highest impact for a
|
||||
second-brain.** Our existing anti-sycophancy / evidence-threshold / keep-both-timestamped stance in the
|
||||
consolidation loop is the human-in-the-loop mitigation; the deterministic guard does not judge semantics.
|
||||
- **Dormant / broken-link injection:** a link to a not-yet-existing target passes a write-time scan; payload
|
||||
planted later. Relevant to the brain's cross-links.
|
||||
- **A document that *describes* attacks is a false positive** — security notes documenting injection payloads
|
||||
trip carrier-strip. Matters if the plugin ever ingests security content.
|
||||
- **Text-only, extracted-text-only** — no file parsing in the core; extract text first, scan with high-untrust
|
||||
upload provenance.
|
||||
|
||||
## 11. Existing write-time precedents the guard complements (not replaces)
|
||||
|
||||
The plugin already has three narrow write-gate defenses; the guard generalizes the class rather than
|
||||
duplicating them:
|
||||
- **`state-updater.mjs`** uses a replacement *function* (not string) on every section-append to neutralize
|
||||
`$&`/`` $` ``/`$'`/`$$`/`$n` from `$`-bearing user topics (`:14-25,:117-125`) — defends the regex mechanics,
|
||||
not content semantics.
|
||||
- **`analytics/storage.ts`** sanitizes filenames + `verifyPathWithinDirectory` (`:110-137`) — path-traversal
|
||||
defense, not row content.
|
||||
- **brain `provenance=published`** — model-collapse guard (authorship), orthogonal to injection (origin).
|
||||
|
||||
## 12. Out of scope for this plan
|
||||
|
||||
No guard code, no `pip install`, no CLI wiring, no hook changes. This is the *when/where* map and the
|
||||
dependency registration only, per the adoption brief. Implementation is a separate, operator-approved work
|
||||
item gated on §7.
|
||||
190
docs/okf-convergence-brief.md
Normal file
190
docs/okf-convergence-brief.md
Normal file
|
|
@ -0,0 +1,190 @@
|
|||
# Brief — Cross-plugin second-brain convergence on OKF-compatible form
|
||||
|
||||
> Created 2026-06-26. **Reference-design brief — not an implementation order.** Captures the operator-locked
|
||||
> direction for converging three plugins' user-owned second brains onto one shared, interoperable form,
|
||||
> with **linkedin-studio's brain as the reference design** and **Google OKF as a thin interop layer only**.
|
||||
> Cross-cutting: most rollout lands in sibling repos and requires its own per-repo go (see §8). State-of-play
|
||||
> in `STATE.md`. Companion design docs (read alongside): `okr/docs/okf-second-brain-note-2026-06.md`,
|
||||
> `ms-ai-architect/docs/okf-second-brain-brief-2026-06.md`.
|
||||
|
||||
## 1. Locked decision (operator, 2026-06-26)
|
||||
|
||||
Converge on **the user's own context** (not the plugins' domain reference files), driven by **interop** —
|
||||
**not** standard-adoption for its own sake.
|
||||
|
||||
- **linkedin-studio's brain is the reference design** — the most mature of the three (provenance-weighted
|
||||
learning, episodic/semantic split, evidence-threshold promotion). The siblings rise toward its maturity;
|
||||
it is **not** levelled down to bare OKF.
|
||||
- **OKF is the thin interop veneer** — add `type:` + per-level `index.md`; keep all rich fields as extension
|
||||
keys (OKF consumers MUST preserve unknown keys). No capability is sacrificed.
|
||||
- **Staged:** ship a shared **spec/convention first**; build a shared **skill only if measured divergence
|
||||
justifies it** (okr's retrieval already works — see §4).
|
||||
|
||||
## 2. Premise corrections (verified — these overturn the old STATE/memory framing)
|
||||
|
||||
The pre-existing framing ("greenfield shared ingest skill; inbox→classify→convert→emit OKF; mdcode is the
|
||||
key tool") rested on three premises that ground-truth checks **disproved**. Verified against the live
|
||||
`GoogleCloudPlatform/knowledge-catalog` repo (research agent, 2026-06-26, file+URL log retained) and the
|
||||
sibling repos.
|
||||
|
||||
1. **`mdcode` is NOT an OKF tool.** It is a **Google Cloud Dataplex** git-sync tool whose on-disk `kb`
|
||||
markdown carries a *different* frontmatter schema (`id`/`resource.name`/`createTime`/`links`) than OKF
|
||||
(`type`/`title`/`description`/`tags`/`timestamp`). They are not interchangeable. Do **not** plan `kcmd`
|
||||
to emit or sync OKF bundles.
|
||||
2. **"OKF has no ingest" is true of the *format*, not the *repo*.** The repo ships an OKF *producer*
|
||||
(`okf/src/reference_agent`, BigQuery+web→OKF) — but it reads a BigQuery dataset + seed URLs, **not** a
|
||||
document folder, and is Gemini/GCP-bound. The genuinely reusable, GCP-free parts are the **SPEC**, the
|
||||
**emit/serialize/validate** core (`OKFDocument`), and the `index.md` synthesis.
|
||||
3. **Classify/convert of arbitrary documents is exactly what the repo provides *nothing* for** — those
|
||||
stages are 100% build-yourself. And — decisive — **the sibling design docs never asked for them.** Both
|
||||
frame the work as *OKF as the storage format for a user-owned second-brain wiki* + a **retrieval skill**
|
||||
+ a **maintenance mechanism**, with ingest being light ("onboarding writes OKF-conformant"), not
|
||||
auto-classification.
|
||||
|
||||
## 3. Landscape — the three consumers have already diverged
|
||||
|
||||
| Plugin | Second-brain status | Maturity |
|
||||
|---|---|---|
|
||||
| **okr** | **Built.** `scripts/okf-index.mjs` + `okf-check.mjs` (conformance checker) + `lib/frontmatter.mjs` + skill `okr-second-brain-search` **v1.6.0** ("OKF-compatible markdown wiki") + tests + fixtures (`okf-minimal/`, `okf-realistic/`) + `inject-okr-context.mjs`. | Structured + retrieval (built) |
|
||||
| **ms-ai-architect** | **Designed, not built.** `docs/okf-second-brain-brief-2026-06.md` (operator-confirmed) + `ref-kb-direction-note` + `ref-kb-workflow-plan`. No retrieval skill yet. | Designed |
|
||||
| **linkedin-studio** | **Built, richer non-OKF schema.** `brain/` hub + `ingest/{inbox,published}` + `journal/` (episodic) + two-layer `profile.md` (semantic), provenance-weighting, evidence-threshold promotion, temporal validity. Engine: `scripts/brain/`. | Provenance-weighted learning system (most mature) |
|
||||
|
||||
**Reading of the siblings' own docs:** they chose OKF because their second brains lived in ad-hoc `org/*.md`
|
||||
**with no retrieval mechanism** — for them OKF (really: *structured markdown + a retrieval skill*) was an
|
||||
upgrade from nothing. linkedin-studio is already past that point. So the convergence is "siblings rise to
|
||||
the reference," not "everyone adopts a new format."
|
||||
|
||||
## 4. okr already supplies the reference checker
|
||||
|
||||
`okr/scripts/okf-check.mjs` implements **exactly the minimal contract** this brief recommends, and is the
|
||||
de-facto reference implementation to align the shared spec with:
|
||||
|
||||
- **Only `type:` is required** on a concept file (`.md` except `index.md`); ≥1 file without `type` → fail.
|
||||
- Recommended fields (`resource`/`title`/`description`/`timestamp`) → **warnings, not errors**.
|
||||
- Root `index.md` carries an `okf_version` marker, echoed for human comparison — **no auto-fetch** (hooks
|
||||
are no-network).
|
||||
|
||||
This means okr has **both** a writer and a checker in production. The shared artifact should generalize
|
||||
okr's checker semantics, not reinvent them. (Reading okr's code is fine; **writing** okr is a separate go.)
|
||||
|
||||
## 5. OKF v0.1 — verified core contract
|
||||
|
||||
Source: `github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md` (v0.1, 12 June 2026,
|
||||
"a starting point, not a finished standard").
|
||||
|
||||
- **Bundle** = a directory tree of markdown files, **one concept per file**. **Concept ID** = file path
|
||||
minus `.md`.
|
||||
- **Frontmatter:** required `type` (free string); recommended `title`, `description`, `resource`
|
||||
(canonical source URI), `tags`, `timestamp`. **Consumers MUST preserve unknown keys and tolerate unknown
|
||||
`type` values.** (Note: the Google *reference producer* is stricter than the spec — it also requires
|
||||
`title`/`description`/`timestamp`. Build to the spec; supply the rest where cheap.)
|
||||
- **Reserved filenames:** `index.md` (directory enumeration, **no frontmatter**, progressive disclosure)
|
||||
and `log.md` (change log). Optional `okf_version` lives in the bundle-root `index.md`.
|
||||
- **Cross-links:** plain markdown links (bundle-relative `/...` or relative); relation type is conveyed by
|
||||
prose. **Consumers MUST tolerate broken links.**
|
||||
- **Permissiveness is the whole point for us:** OKF is a *minimal superset-friendly* contract. Conforming
|
||||
costs `type` + `index.md`; our rich fields ride along untouched as extension keys.
|
||||
|
||||
## 6. The deliverable — "OKF-compatible second-brain form"
|
||||
|
||||
A spec (document, not code) that all three plugins' user-data conforms to:
|
||||
|
||||
1. **Minimal contract:** every concept file carries `type:`; each directory level has an `index.md`;
|
||||
bundle-root `index.md` carries `okf_version`. Recommended fields where cheap. (= okr's `okf-check`
|
||||
semantics, generalized.)
|
||||
2. **Rich fields survive as extension keys.** linkedin-studio's brain keeps `provenance`, `first_seen`,
|
||||
`last_seen`, `evidence_count`, `status`, episodic/semantic distinction — all as extra frontmatter keys
|
||||
OKF must preserve. The model-collapse guard (`provenance=published` only) is unaffected.
|
||||
3. **Mapping for our brain** (illustrative; verify writers in §10):
|
||||
- `brain/index.md` → bundle-root index + `okf_version`.
|
||||
- `brain/profile.md`, `operations.md`, `journal/*.md`, tributary summaries → concept files; each gains a
|
||||
`type` (e.g. `Profile`, `Operations`, `JournalEntry`, `TributarySummary`) + retains its existing rich
|
||||
frontmatter.
|
||||
- `ingest/inbox/` stays the **manual drop-zone** (already exists) — the "inbox folder" mechanism, with
|
||||
no heavy auto-classifier built now.
|
||||
|
||||
## 7. Staged plan
|
||||
|
||||
- **Stage 1 — Shared spec/convention (cheap, delivers interop).** Author "OKF-compatible second-brain form"
|
||||
as a cross-cutting document; align it with okr's `okf-check`. Each plugin's user-data conforms; one
|
||||
reader can traverse all three. **This alone meets the interop goal.**
|
||||
- **Stage 2 — Measure divergence.** Do the per-plugin retrieval skills (okr's built one; architect's
|
||||
planned one; linkedin-studio's in-context reads) diverge enough to hurt? Only a *measured* yes justifies
|
||||
Stage 3 (operator anti-pattern: "ambitious initiatives where a config tweak suffices").
|
||||
- **Stage 3 — Conditional shared skill.** If justified: extract/generalize okr's working
|
||||
`second-brain-search` into one home (see §9), with a discovery convention for where each plugin's brain
|
||||
lives.
|
||||
|
||||
## 8. Home decisions
|
||||
|
||||
- **The spec** is cross-cutting → **catalog/marketplace level** (owned by no single plugin).
|
||||
- **A future shared skill** (Stage 3 only) → a **standalone marketplace plugin** (own repo, release-tagged,
|
||||
catalog-pinned), installable alongside the others, serving **consumer (a) — the user's own context —
|
||||
directly**. Rejected alternatives: duplicate-per-plugin (drift risk); user-level `~/.claude/skills/`
|
||||
(unversioned, outside the catalog).
|
||||
|
||||
## 9. Per-repo scope boundaries (each its own explicit go)
|
||||
|
||||
| Repo | This initiative's work | Status |
|
||||
|---|---|---|
|
||||
| **linkedin-studio** (here) | (1) Be the reference design (mostly exists in `docs/second-brain/architecture.md`). (2) Make our own brain emit OKF-compatible form (`type` + per-level `index.md` + root `okf_version`) without dropping rich fields. | **In scope — (2) ✅ LANDED 2026-06-26** |
|
||||
|
||||
> **Stage-1 outcome (2026-06-26).** Brain writers now emit OKF-compatible form: `serializeProfile`
|
||||
> leads with `type: Profile` frontmatter (constant → round-trip-safe), `operations.md` seed leads with
|
||||
> `type: Operations`, `brain/index.md` carries an `okf_version: 0.1` marker, and `brain/journal/index.md`
|
||||
> is scaffolded (per-level index). **Premise refinement (verified):** the brain is *deliberately*
|
||||
> YAML-free with a byte-exact round-trip invariant on `ingest/published/*.md` (SC2) that a frontmatter
|
||||
> block would break — so the OKF concept-bundle is scoped to **`brain/` only**; the round-trip-critical
|
||||
> **`ingest/` tributary is excluded** and pointed to from the hub index. We **emit** frontmatter, adding
|
||||
> no YAML parser. 5 new tests (`tests/okf-conform.test.ts`); full brain suite **132/132**; cross-tool
|
||||
> proof — `okr/scripts/okf-check.mjs` validates `brain/` (exit 0). **Finish (same day):** the cheap
|
||||
> recommended fields `title`/`description` added to the concept frontmatter (`timestamp`/`resource` stay
|
||||
> out — a timestamp would break the pure serializer, `resource` is N/A for an internal concept); and the
|
||||
> transient `brain/pending-diff.md` now carries `type: PendingDiff` so the bundle passes `okf-check` even
|
||||
> mid-propose (re-verified exit 0 with a pending-diff present). Brain suite **134/134**.
|
||||
| **okr** | Optional form-conformance alignment (already has writer + checker). | **Separate go** |
|
||||
| **ms-ai-architect** | Build its retrieval skill against the shared spec. | **Separate go** |
|
||||
| **catalog** | Host the shared spec. | **Separate go** (catalog only via `release-plugin.mjs`) |
|
||||
| **new standalone plugin** | Stage-3 shared skill, if justified. | **Separate go** |
|
||||
|
||||
Per scope-guard + "never write in other repos without explicit instruction": this session touches
|
||||
**linkedin-studio only**.
|
||||
|
||||
## 10. Key assumptions + tests (plan-quality mandate)
|
||||
|
||||
| Assumption | Test (before relying on it) |
|
||||
|---|---|
|
||||
| OKF preserves unknown keys → our rich brain fields survive conformance | **✅ Verified:** `okf-check.mjs` exits 0 on `brain/`; `profile.md` round-trips (`parseProfile` skips the frontmatter, `parse ∘ serialize` identity holds). |
|
||||
| Our brain is already near-OKF (conformance is a small writer change) | **⚠️ Refined → verified:** brain is *deliberately* YAML-free and `ingest/published` is round-trip-critical, so a literal frontmatter target conflicts there → bundle scoped to `brain/`, `ingest/` excluded (tributary). 4 surgical writer touchpoints (3 scaffold seeds + `serializeProfile`); we EMIT frontmatter, add no parser. |
|
||||
| okr's `okf-check` semantics generalize as the shared conformance contract | Diff okr's contract (only-`type`-required, recommended=warnings, `okf_version` echo) against OKF SPEC §9 conformance → confirm it is a faithful, slightly-laxer subset. |
|
||||
| A shared skill is *not yet* justified | Stage-2 measurement, deferred — do not build Stage 3 before it. |
|
||||
|
||||
## 11. Open choices (resolve in `/trekbrief` or measurement, not now)
|
||||
|
||||
- **Retrieval mechanism:** native Grep/Glob/Read (skill instruction "search the wiki first, open only
|
||||
relevant") vs. a dedicated fileskb MCP server. Both sibling docs lean **native** (Claude Code's
|
||||
Grep/Glob/Read already cover OKF's list/search/read). Genuine doubt → "build both, measure" candidate.
|
||||
- **Degree of OKF formalism:** full v0.1 conformance vs. "OKF-compatible form" (frontmatter + `index.md`
|
||||
only). Lean to the lightest that yields smart retrieval.
|
||||
- **Inbox auto-classify/convert:** **defer.** OKF gives nothing for it; the manual inbox seam already
|
||||
exists. Build only on demonstrated need.
|
||||
- **Discovery convention:** how a shared skill finds each plugin's brain root.
|
||||
- **OKF version-bump tracking:** how to catch v0.1 → later without manual polling (hooks are no-network).
|
||||
|
||||
## 12. Success criterion (operator, inherited from both sibling tracks)
|
||||
|
||||
Measured against **user value** (does the plugin retrieve the right personal/org context in chat and
|
||||
commands?) + **maintenance reliability** — **not** against formal OKF conformance for its own sake.
|
||||
|
||||
## 13. References
|
||||
|
||||
- OKF SPEC v0.1: `github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md`
|
||||
- Sibling design docs: `okr/docs/okf-second-brain-note-2026-06.md`,
|
||||
`ms-ai-architect/docs/okf-second-brain-brief-2026-06.md`,
|
||||
`ms-ai-architect/docs/ref-kb-direction-note-2026-06.md`
|
||||
- okr reference implementation: `okr/scripts/okf-check.mjs`, `okr/scripts/okf-index.mjs`,
|
||||
`okr/skills/okr-second-brain-search/SKILL.md`
|
||||
- Our reference design: `docs/second-brain/architecture.md`; engine `scripts/brain/`
|
||||
- Verified-OKF research log (files+URLs fetched on `main`): retained in session transcript, 2026-06-26
|
||||
- Memory: `google-okf-open-knowledge-format`, `profile-evolution-second-brain`,
|
||||
`plugin-vs-maskinrommet-division`, `plugin-is-domain-general`
|
||||
89
docs/research-engine/brief-re-r1.md
Normal file
89
docs/research-engine/brief-re-r1.md
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
# Brief — RE-R1: lift the research engine's item-schema + triage-scorer from agent prose to tested code
|
||||
|
||||
> **Slice:** RE-R1 (research-engine rung-2, slice 1) — the first build slice of the daily research motor.
|
||||
> **Status:** drafted, awaiting go-gate. **Predecessors:** "slice 2a" = the scoring SSOT `references/trend-scoring-modes.md`; "slice 2b" = the trend store `scripts/trends/` (24/24). This slice adds the **scoring/normalization code those two anticipate** (`trend-scoring-modes.md:96-100` names "any future research-engine pass that scores candidates before writing them to the trend store" as a consumer).
|
||||
> **Substrate:** `docs/research-engine-concepts.local.md` §1, §3 (B1/B2), §5 ("anbefalt minste neste slice"). **TDD-order:** RED tests + RED gate self-test land before code.
|
||||
|
||||
## 1. Operator decision context (2026-06-23)
|
||||
|
||||
The operator lifted the **research engine to Tier-1**. Rationale, verbatim: *"hele min arbeidsflyt hviler på at jeg får en jevn strøm av gode forslag til tema å skrive om."* A/moat (lived-specifics · positioning · shareability · analytics→loop) remains the long-term nordstjerne but is built **interactively in Maskinrommet from lived use**, not as a TDD slice now.
|
||||
|
||||
This slice is exactly the one `research-engine-concepts.local.md` §5 named as the minimal next move *if research is lifted*: **B1 (uniform item-schema, tested) + B2 (triage-scorer as code with threshold config)**. It moves "the only untestable layer" from agent prose into gated code, and is the foundation every later rung (brief artifact, surfacing, freshness, deepening) hangs on.
|
||||
|
||||
## 2. The gap — grounded in code
|
||||
|
||||
- **Storage exists; scoring does not.** `scripts/trends/src/store.ts` (183 LOC) stores/dedupes/queries `TrendRecord {id,title,url,source,capturedAt,topics[],summary?}` (`types.ts:26-41`). Its header states scoring "lives in the agent/command layer; this module only stores, dedupes, and serves." There is **no scoring/routing/saturation/digest code anywhere** in the repo.
|
||||
- **The arithmetic is prose the LLM runs in its head.** `references/trend-scoring-modes.md` (SSOT, 101 lines) defines two modes — **kortform** (Pillar .30 / Audience .25 / Timing .20 / Angle .15 / Authority .10) and **long-form** (Pillar .30 / Depth .25 / Angle .20 / Authority .15 / Currency .10) — a weighted composite on a shared 0–10 scale, and a composite→band map (8.0–10 Immediate · 6.0–7.9 High · 4.0–5.9 Medium · 2.0–3.9 Low · 0–1.9 Skip), each band carrying a kortform + long-form action. `agents/trend-spotter.md:124-137` tells the agent to "score each dimension 1–10, take the weighted composite, rank highest-first, apply the bands" — **deterministic arithmetic with no test and no code.**
|
||||
- **Ingress is ad hoc.** Nothing normalizes a source's raw output into one item envelope before it reaches `addTrend`; downstream would branch on source-type. B1 is the "one schema downstream never branches on" the concepts doc calls "the most load-bearing idea — build it first."
|
||||
|
||||
## 3. Scope — what is IN (RE-R1)
|
||||
|
||||
### B1 — canonical item schema + normalizer (`scripts/trends/src/item.ts`, NEW)
|
||||
A `TrendItem` **ingress envelope** `{ source, title, url, publishedAt?, topics[], summary? }` — what a source/adapter emits — plus a pure `normalizeItem(raw): { ok: true, item } | { ok: false, errors }` and `normalizeItems(raw[]): { items, errors }` (batch partition). Deterministic: required-field validation (`source/title/url` present and non-empty → else a structured `{ok:false}` error naming the field, never a silent partial), whitespace normalization (reuse `normalizeField` from `store.ts`), topic normalize + dedupe, `publishedAt` ISO-8601-validate-**if-present** (the source's publish date — carried for forward-compat, **not persisted in R1**, and distinct from the store's `capturedAt`). **No `id` field:** the envelope carries none and the store already derives it via `addTrend`→`trendId`. Vocabulary kept as `topics` (≡ the concepts doc's `tags`; see §8). **The item→store bridge** (injecting `capturedAt`, persisting `publishedAt`) is **R2 orchestration — out of scope here**; R1 delivers the validated envelope + normalizer only.
|
||||
|
||||
### B2 — deterministic triage-scorer (`scripts/trends/src/score.ts`, NEW)
|
||||
`ScoreMode = 'kortform' | 'long-form'`; per-mode weight constants **encoded from the SSOT** (with the SSOT's "ordering is the signal, not a measured coefficient" caveat as a comment). `DimensionScores` (mode-specific keys). `composite(scores, mode): number` — weighted sum, validates each dimension ∈ [1,10]. `band(composite): { priority, kortformAction, longformAction }` — the five-band map. `triage(scored, { mode, threshold }): { kept, dropped }` — kept = composite ≥ threshold, ranked composite-desc, each annotated with composite + band; dropped below. `threshold` is a single config parameter (default — see §8). The module owns **only** the arithmetic / bands / threshold; producing the five dimension scores stays model judgment (the deterministic/judgment split from the concepts doc §5 / SSOT analysis).
|
||||
|
||||
### CLI surface (`scripts/trends/src/cli.ts`, EDIT existing)
|
||||
Two new subcommands over the new modules — both **read their JSON payload from stdin** (a raw item/batch for `normalize`; items-with-dimension-scores for `score`) and **emit JSON to stdout**, so they do **not** overload the existing `--json` *output* toggle (`cli.ts:79`). `score` takes `[--mode kortform|long-form] [--threshold N]` flags. Exit codes: **2** for a malformed invocation (missing required flag, unparseable stdin JSON) — matching `cli.ts`'s usage-error precedent (`cli.ts:54-63`); **0** for a well-formed call, even when the payload carries content-invalid items (reported as `{ok:false,...}` entries in the output, never via the exit code).
|
||||
|
||||
### Wiring + gate
|
||||
- `agents/trend-spotter.md` (EDIT): replace the "do the composite + bands yourself" prose at L124-137 with a pointer to the scorer CLI as the owner of the composite/band/threshold step (the lift becomes real and grep-able; the agent still supplies the five judgment scores). Kept domain-general.
|
||||
- `scripts/test-runner.sh` (EDIT): bump `TRENDS_TESTS_FLOOR` (recount live at land; it **stays inside the `if [ -x …/tsx ]` deps guard** — conditional, never hoisted out). Add a new **grep-only** Section **16g**, placed after Section 17 (de-niche) and **before** Section 18 (anti-erosion, which must run last so it sees every prior check), with deps-absent-safe checks (no `tsx`): (1) `grep` that `score.ts` encodes both mode weight-sets; (2) `grep -qF "src/cli.ts score"` that `trend-spotter.md` references the scorer CLI; (3) a non-vacuity self-test for those greps, per the house pattern (every sibling section 16c–17 has one). These are **unconditional** → bump `ASSERT_BASELINE_FLOOR` 84 → **live recount** (expected ~+3, not a pinned number). The arithmetic proof (`composite(all-tens)=10.0` ⇒ weights sum to 1.0) lives as a **unit test** in `score.test.ts` (conditional, counts toward the trends floor), since it needs the `tsx` runtime.
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **The dated brief artifact / morning-brief file (B3)** — R2. *This is the rung that makes the stream visible to the operator; R1 is the foundation beneath it, not the stream itself.*
|
||||
- **Session-start surfacing** of the brief (hull 4) — R2.
|
||||
- **Store-schema migration / new persisted fields** (publishedAt, relevance, saturation, status — hull 3/5) — R2. R1 keeps `SCHEMA_VERSION = 1` untouched (no migration risk).
|
||||
- **Freshness window / dedup-vs-seen-log / autonomous trigger (B4)** — R3.
|
||||
- **Research-deepening (plan → isolated workers → synthesis, A1–A4)** — R3.
|
||||
- **Adapter sub-agents / MCP fetch fan-out** — R2/R3.
|
||||
- **Producing the five dimension scores in code** — stays model judgment, by design.
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **TDD iron law:** the failing `item`/`score` tests **and** the failing gate self-test land BEFORE the implementation. RED proofs recorded (logic-RED, not import-RED).
|
||||
- **No store-schema change:** `SCHEMA_VERSION` stays 1; `TrendRecord` untouched; no data migration.
|
||||
- **Domain-general:** de-niche guard (Section 17) stays green; no vendor/sector tokens enter the edited `trend-spotter.md` prose.
|
||||
- **SSOT discipline:** `trend-scoring-modes.md` remains the human source of truth; `score.ts` mirrors it; a test pins **the exact per-mode weights, the four band thresholds (8.0/6.0/4.0/2.0), and the ten band action strings** against the SSOT values, and asserts each mode sums to 1.0 — so silent drift in *any* of them (not just the weights) fails loudly. (A markdown-parsing cross-check of the SSOT table itself is deferred — out of scope.)
|
||||
- **House conventions:** ESM + `node:test` + `tsx`; data-seam stays inline (no new shared util — the documented idiom); no new `.mjs` under `hooks/scripts/`; `.gitignore` already covers `scripts/trends/{node_modules,build}`.
|
||||
- **No new `references/*.md`** (would trip the named-additions guard) and **no new agent/command** (counts stay 19/29/27). Brief+plan live under `docs/` (uncounted), TRACKED like `docs/second-brain/*` (general feature design).
|
||||
- **Counts recounted live at land**, never pinned/guessed.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1** — `normalizeItem` returns a canonical item from a well-formed raw item; a missing/empty required field returns a structured `{ok:false}` error naming the field (not a silent partial); whitespace + topic dedupe applied; `publishedAt` validated-if-present, undefined-if-absent. `normalizeItems` partitions a batch into `{items, errors}`.
|
||||
- **SC2** — `composite()` equals the SSOT formula exactly for both modes: all-tens → **10.0** (proves Σweights=1), and the **asymmetric golden vector `{10, 8, 6, 4, 2}` (in dimension order) → 7.0** for both modes (`10·.30 + 8·.25 + 6·.20 + 4·.15 + 2·.10`) — asymmetric so a weight↔dimension transposition is caught; a dimension outside [1,10] throws.
|
||||
- **SC3** — `band()` returns the correct priority + **the exact mode-appropriate action string** (pinned against the SSOT) at every boundary (8.0, 6.0, 4.0, 2.0 edges).
|
||||
- **SC4** — `triage()` keeps composite ≥ threshold, drops below, ranks kept composite-desc, annotates each kept item with composite + band.
|
||||
- **SC5** — CLI `normalize`/`score` read JSON from stdin, emit valid JSON; exit **2** on a malformed invocation (missing flag / unparseable stdin), **0** on a well-formed call (content-invalid items reported in the payload, never via exit code).
|
||||
- **SC6** — `(cd scripts/trends && npm test)` green at the bumped floor; gate's new unconditional section green; `ASSERT_BASELINE_FLOOR` bumped to the live recount; overall gate `FAIL=0`.
|
||||
- **SC7** — `trend-spotter.md` references the scorer for the deterministic step (gate grep green); de-niche guard green; structure counts unchanged (19/29/27).
|
||||
|
||||
## 7. Verification
|
||||
|
||||
**Deterministic (gate):** `bash scripts/test-runner.sh` → `FAIL=0`; trends suite ≥ new floor; new self-test + wiring-grep pass; Section 17 de-niche green; ref/agent/command counts unchanged.
|
||||
**Behavioural (manual):** run `npm run start -- normalize --json '<sample batch>'` and `... score --json '<scored sample>' --mode kortform --threshold 4.0`; eyeball that kept/dropped/bands match a hand-computed expectation on 3–4 items.
|
||||
|
||||
## 8. Open questions for the go-gate
|
||||
|
||||
1. **Default threshold.** Propose **4.0** — the SSOT's Medium-band floor and the agent's existing "score 4.0+" angle-mapping cutoff (`trend-spotter.md:179,274`). Confirm or set otherwise.
|
||||
2. **Vocabulary `topics` vs `tags`.** The concepts doc envelope says `tags`; the codebase standardized on `topics`. Propose **keep `topics`** (consistency, no rename) and note `tags ≡ topics`. Confirm.
|
||||
3. **CLI tests now?** Trends has no `cli.test.ts` today; siblings (brain, contract-gate) do. Propose **add light cli tests** for the two new subcommands (happy path + exit codes). Confirm vs defer.
|
||||
4. **Wire `trend-spotter.md` now?** Propose **yes** — a prose pointer so the lift is real and gate-grep-able; full orchestration (fan-out, brief assembly) stays R2.
|
||||
|
||||
## 9. Light-Voyage review — folded
|
||||
|
||||
Three Opus reviewers ran on the drafts. **scope-guardian: ALIGNED** (every SC1–SC7 traces to a step; zero scope creep; NON-goals fully respected). **brief-reviewer: PROCEED_WITH_RISKS.** **plan-critic: REVISE (1 blocker, 6 majors).** All findings folded:
|
||||
|
||||
- **[BLOCKER, folded]** `TrendItem`→`TrendInput` does not map directly — live `TrendInput` requires `capturedAt` and has no `publishedAt`; the item has the opposite. → §3 B1 now scopes R1 to the validated envelope only; the item→store bridge (`capturedAt` injection, `publishedAt` persistence) is **R2**. A scope-tightening, not an addition.
|
||||
- **[MAJOR, folded]** CLI `--json` is already an *output* toggle → §3 now reads payload from **stdin**, `--json` untouched.
|
||||
- **[MAJOR, folded]** Gate-section placement + anti-erosion-last → §3 pins **Section 16g, before Section 18**; `TRENDS_TESTS_FLOOR` stays inside the deps guard.
|
||||
- **[MAJOR, folded]** `ASSERT_BASELINE_FLOOR` was pinned to 86 against the recount-live rule, and the house self-test pattern makes it +3 → §3 now says **live recount (~+3), not pinned**.
|
||||
- **[MAJOR, folded]** Only weights were drift-guarded → §5 SSOT discipline now pins **weights + band thresholds + the ten action strings**.
|
||||
- **[MAJOR, folded]** Wiring grep literal unpinned → §3 pins `grep -qF "src/cli.ts score"`.
|
||||
- **[testability, folded]** SC2 unnamed vector → **pinned `{10,8,6,4,2}`→7.0**; SC5 exit-code ambiguity → **explicit 2-vs-0 contract**; `id` hedge → **envelope carries no id**; `normalizeItems` batch shape → **SC1 extended**.
|
||||
- **[minor, folded in plan]** RED-stub split per assertion type; explicit `node --import tsx` invocation (not `npm run start`); commit grouping fixed; STATE.md noted as land-bookkeeping; `config/trends-sources.template.md` added to the scope fence.
|
||||
|
||||
See `plan-re-r1.md` §Plan-critic — folded for the per-step resolution.
|
||||
208
docs/research-engine/brief-re-r2a.md
Normal file
208
docs/research-engine/brief-re-r2a.md
Normal file
|
|
@ -0,0 +1,208 @@
|
|||
# Brief — RE-R2a: capture bridge (item→store) + publishedAt persistence
|
||||
|
||||
> **Slice:** RE-R2a (research-engine rung-2, slice 2 — the data layer of R2). The first of the two
|
||||
> halves R2 ("the visible topic-stream") was split into at the 2026-06-24 go-gate: **R2a** = the pure
|
||||
> `scripts/trends/` data layer (this brief); **R2b** = the dated morning-brief artifact + session-start
|
||||
> surfacing (separate brief, after R2a lands). The split was chosen because (b)+(c) are code-independent
|
||||
> of (a), R2a mirrors R1's pure-TDD shape exactly (lowest risk), and it closes the capture loop the store
|
||||
> was built for.
|
||||
> **Predecessor:** RE-R1 (`brief-re-r1.md`) delivered the validated ingress envelope (`item.ts`:
|
||||
> `normalizeItem`/`normalizeItems`) and the triage scorer (`score.ts`) behind a CLI seam, and **explicitly
|
||||
> deferred the item→store bridge** ("injecting the store's `capturedAt` and persisting `publishedAt` — is
|
||||
> R2 orchestration", `brief-re-r1.md` §3 B1). R2a builds exactly that deferred bridge.
|
||||
> **Substrate:** `docs/research-engine-concepts.local.md` §1 (hull 3 "store-schema mangler felt" / the
|
||||
> item→store gap), §3 B1 (one schema downstream never branches on). **TDD-order:** RED tests land before code.
|
||||
|
||||
## 1. Operator decision context (2026-06-24)
|
||||
|
||||
The research engine is **Tier-1** (operator, 2026-06-23): *"hele min arbeidsflyt hviler på at jeg får en
|
||||
jevn strøm av gode forslag til tema å skrive om."* At the R2 go-gate (2026-06-24) the operator chose to
|
||||
build R2 as **two sequenced slices, foundation-first**: R2a (this — the bridge + schema, pure trends/) before
|
||||
R2b (the visible brief + surfacing). R2a delivers no *visible* change; it closes the capture loop so the
|
||||
store accumulates publish-dated history that R2b's brief and R3's freshness window will rank on.
|
||||
|
||||
## 2. The gap — grounded in code
|
||||
|
||||
- **`normalize` (R1) and the store are disconnected.** `item.ts` validates a raw source item into the
|
||||
canonical `TrendItem` envelope `{source,title,url,publishedAt?,topics[],summary?}` (`item.ts:21-38`), but
|
||||
**nothing folds that envelope into the store.** The only ingress to the store is `cli.ts add` (`cli.ts:111-135`),
|
||||
which builds a `TrendInput` from raw flags — it never calls `normalizeItem`, so a capture path that polls →
|
||||
normalizes → stores **does not exist**. The bridge is the missing link R1 named and deferred.
|
||||
- **`publishedAt` is carried but dropped.** The envelope carries the source's own publish date
|
||||
(`item.ts:28-33`, "NOT persisted in R1"), but `TrendRecord` (`types.ts:26-41`) has no `publishedAt` field and
|
||||
`addTrend` (`store.ts:111-130`) never persists it. So every capture **loses** the publish date — the field
|
||||
B4 freshness (R3) and R2b's "fresh today" sort will both need. R2a persists it now, before history accumulates
|
||||
undated, so the loop is publish-dated from this slice forward.
|
||||
- **`SCHEMA_VERSION` is pinned at 1 with no migration path exercised.** `loadStore` (`store.ts:74-81`) reads
|
||||
`parsed.schemaVersion ?? SCHEMA_VERSION` but the store has never had a second version, so the forward-migration
|
||||
path is untested. Adding `publishedAt` is the first additive-optional schema bump (v1→v2) and the right moment
|
||||
to prove a lossless migrate-on-load.
|
||||
|
||||
## 3. Scope — what is IN (RE-R2a)
|
||||
|
||||
### B-bridge — `itemToInput` (pure mapping, `scripts/trends/src/item.ts`, EDIT)
|
||||
A pure `itemToInput(item: TrendItem, capturedAt: string): TrendInput` that maps the validated envelope to a
|
||||
store input by **injecting `capturedAt`** (the store's "when WE saw it", supplied by the caller — never derived
|
||||
inside the pure function) and **carrying `publishedAt`/`topics`/`summary`/`title`/`url`/`source` through verbatim**.
|
||||
It does **not** re-validate (the item is already validated by `normalizeItem`) and does **not** derive an `id`
|
||||
(the store owns id via `addTrend`→`trendId`). Lives in `item.ts` (it is the item's mapping to the store; `item.ts`
|
||||
already imports from `store.ts`) — **not** a new single-function `bridge.ts` (single-use abstraction).
|
||||
|
||||
### Schema migration — `publishedAt` persisted (`types.ts` + `store.ts`, EDIT)
|
||||
- `types.ts`: `SCHEMA_VERSION 1 → 2`; add `publishedAt?: string` to `TrendRecord` (after `capturedAt`), with a
|
||||
comment distinguishing it from `capturedAt` (source publish-date; forward-compat for B4 freshness).
|
||||
- `store.ts`: `TrendInput` gains `publishedAt?: string`; `addTrend` persists it when present (same
|
||||
conditional-spread idiom as `summary`, `store.ts:126`). **On re-capture/merge it is NOT overwritten** —
|
||||
it joins the store's documented "FIRST sighting's source + capturedAt are kept (provenance of first sight)"
|
||||
rule (`store.ts:108-110`); no back-fill (a deferred enhancement, §4).
|
||||
- `loadStore`: **forward migrate-on-load** — stamp the in-memory store to `Math.max(onDisk, SCHEMA_VERSION)`,
|
||||
where `onDisk = typeof parsed.schemaVersion === "number" ? parsed.schemaVersion : SCHEMA_VERSION` (a string /
|
||||
`NaN` / absent version falls back to current, never crashes). v1→v2 is purely additive-optional (old records are
|
||||
already valid v2 records that simply lack the optional field), so the migration is the stamp alone — **no record
|
||||
rewrite; lossless + idempotent for any well-formed v1/v2 store**. A subsequent `saveStore` persists the v2 stamp.
|
||||
**The existing non-array `trends` coercion (`store.ts:79`, `Array.isArray(parsed.trends) ? … : []`) is UNCHANGED
|
||||
and out of R2a's migration scope** — losslessness is claimed only for well-formed stores, not for a corrupt
|
||||
`trends` field. (No downgrade branch: a hypothetical future on-disk > current can't happen yet — Karpathy #2, no
|
||||
handling for impossible scenarios; noted as a deferred risk.)
|
||||
|
||||
### CLI `capture` subcommand (`scripts/trends/src/cli.ts`, EDIT)
|
||||
`echo '<raw item | batch>' | node --import tsx src/cli.ts capture [--store <path>] [--json]` — reads the raw
|
||||
JSON payload **from stdin** (same stdin contract as `normalize`/`score`, so `--json` stays an *output* toggle),
|
||||
runs it through `normalizeItem`/`normalizeItems`, maps each valid item via `itemToInput(item, today())`, folds
|
||||
each into the store via `addTrend`, and `saveStore`s once. Emits a summary — `{added, duplicates, merged, errors}`
|
||||
+ store path/count — human-readable by default, JSON with `--json`. Content-invalid items are reported in the
|
||||
summary (`errors[]`), **never via the exit code**; exit **2** only on a malformed invocation (unparseable/empty
|
||||
stdin), matching `normalize`/`score`. Note the contract difference from `add`: `capture` items must carry `source`
|
||||
(the normalizer requires it — no "manual" default), because a capture is from a real source.
|
||||
|
||||
### Wiring + gate (see Open Question #1 — included by default, trimmable to minimal)
|
||||
- `agents/trend-spotter.md` (EDIT): re-point **Step 4.5** (`trend-spotter.md:282-301`) from N× flag-based `add` to
|
||||
a single batch `capture` — the agent builds a raw-item JSON batch (it already builds JSON for `score`) and pipes
|
||||
it to `capture` in one call. Strictly better than `add` (it normalizes + carries `publishedAt`; one call, not N).
|
||||
The replacement prose carries the literal `src/cli.ts capture`. Kept domain-general (no vendor/sector tokens).
|
||||
- `scripts/test-runner.sh` (EDIT): bump `TRENDS_TESTS_FLOOR` (62 → **live recount**, stays inside the deps guard).
|
||||
Add **Section 16h** ("Trends Capture Wiring"), mirroring Section 16g's three-check shape, placed **after Section
|
||||
17 / before Section 18** (anti-erosion last): (1) a non-vacuity self-test; (2) `grep` that `cli.ts` has a `capture`
|
||||
handler (deps-absent proof the path exists); (3) `grep -qF "src/cli.ts capture" agents/trend-spotter.md`. These are
|
||||
**unconditional** → bump `ASSERT_BASELINE_FLOOR` 87 → **live recount** (expected ~90).
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **The dated morning-brief artifact (B3)** — R2b. *R2a closes the capture loop; R2b makes the stream visible.*
|
||||
- **Session-start surfacing** of the brief (hull 4) — R2b. R2a does **not** touch `hooks/**`.
|
||||
- **`publishedAt` back-fill on re-capture** (fill an absent existing `publishedAt` from a later sighting) — deferred;
|
||||
first-sight provenance is kept, matching the existing merge rule. Revisit if undated-first-sight proves common.
|
||||
- **`add --published-at` flag** — deferred (was a proposed "minor"; folded out at light-Voyage). The flag-based
|
||||
`add` is the *manual* single-trend path; it stays publish-date-free for now. The store layer still GAINS
|
||||
`publishedAt` (on `TrendInput` + `addTrend`) — that is what `capture` needs — but exposing it on the `add` CLI
|
||||
(and the `isValidIso`-export-vs-inline decision it would force) is out of R2a. Trivial to add any later slice.
|
||||
- **`relevance` / `saturation` / `status` / lifecycle fields** (hull 5) — R3. R2a adds **only** `publishedAt`.
|
||||
- **Freshness window / dedup-vs-seen-log / autonomous trigger (B4)** — R3.
|
||||
- **Research-deepening (A1–A4), adapter sub-agents, MCP fetch fan-out** — R2b/R3.
|
||||
- **A store-reading brief / ranking on accumulated relevance** — R2b/R3 (needs persisted scores).
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **TDD iron law:** the failing migration / `itemToInput` / `capture` tests land BEFORE the implementation.
|
||||
RED proofs recorded (logic-RED, not import-RED). **NB:** only the **v1→v2** load + round-trip are genuinely RED
|
||||
(old `loadStore` returns 1); v2-idempotence + missing/non-numeric-`schemaVersion` pass against old code, so they
|
||||
are **GREEN-only regression guards**, not RED cases (see plan Step 1).
|
||||
- **Lossless migration (well-formed stores):** every existing **well-formed** v1 store loads as v2 with **records
|
||||
intact** (no `publishedAt` invented, no topic/summary/capturedAt/url/title change); idempotent (v2 → v2). A
|
||||
corrupt `trends` field is coerced by the existing (unchanged) `Array.isArray` guard — explicitly **out of R2a's
|
||||
losslessness claim**, not a regression introduced here.
|
||||
- **First-sight provenance preserved:** re-capturing an existing trend never overwrites its `publishedAt`,
|
||||
`capturedAt`, or `source` — only topics union (unchanged from R1).
|
||||
- **No scoring change:** `score.ts` and `references/trend-scoring-modes.md` (the SSOT) are untouched — R2a is the
|
||||
data layer, not the scorer.
|
||||
- **Domain-general:** de-niche guard (Section 17) stays green; no vendor/sector tokens enter the edited
|
||||
`trend-spotter.md` prose.
|
||||
- **House conventions:** ESM + `node:test` + `tsx`; data-seam stays inline; **no new `.mjs` under `hooks/scripts/`**
|
||||
(R2a touches no hook); `.gitignore` already covers `scripts/trends/{node_modules,build}`.
|
||||
- **No new `references/*.md`**, **no new agent/command** (counts stay 19/29/27); no new `.ts` source file
|
||||
(bridge lives in `item.ts`). Brief+plan live under `docs/` (uncounted), TRACKED like `docs/second-brain/*`.
|
||||
- **Counts recounted live at land**, never pinned/guessed.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1 (bridge)** — `itemToInput(item, "2026-06-24")` returns a `TrendInput` with `capturedAt` injected (`=== "2026-06-24"`),
|
||||
`publishedAt`/`topics`/`summary`/`title`/`url`/`source` carried verbatim, and **no `id`**. Absent `publishedAt`
|
||||
on the item → absent on the input (key omitted, not `undefined`-valued). **Field-confusion guard:** for an item
|
||||
whose `publishedAt` differs from the injected `capturedAt`, the result's `capturedAt !== result.publishedAt`
|
||||
(proves the bridge never confuses the two dates — the whole point of the slice).
|
||||
- **SC2 (persist)** — `addTrend` with a `publishedAt` persists it on the new record; a record without it omits the
|
||||
key. On a re-capture (same title+url) the existing `publishedAt` is **unchanged** (first-sight kept), and only
|
||||
topics union — `merged` reflects topic change alone. **No back-fill:** a re-capture carrying a `publishedAt` onto
|
||||
a record that **lacked** one does **not** add it (Open Q#2) — the absent→present case is the one where "first-sight
|
||||
kept" is counterintuitive, so it is pinned by a test.
|
||||
- **SC3 (migrate)** — `loadStore` on a v1 store fixture (`{schemaVersion:1, trends:[…without publishedAt]}`) returns
|
||||
`schemaVersion === 2` with every record intact and **no `publishedAt` invented**. Also pinned: a store with
|
||||
**missing** `schemaVersion` → stamped 2 (records intact); an **empty/absent** store → `{schemaVersion:2, trends:[]}`;
|
||||
a **non-numeric** `schemaVersion` (`"weird"`/`NaN`) → falls back to 2 (records intact); idempotent on a v2 store;
|
||||
a round-trip `loadStore`→`saveStore` writes `schemaVersion: 2`.
|
||||
- **SC4 (capture CLI)** — `echo '<batch>' | … capture` normalizes + folds: a well-formed item is added (or reported
|
||||
duplicate/merged), a content-invalid item appears in the summary `errors[]`, the store file is written once, and
|
||||
the summary counts are correct — **`added + merged + duplicates + errors.length === payload.length`** (the tally
|
||||
is derived from `AddResult {added, merged}`, which has no `duplicates` field: `added`=`res.added`,
|
||||
`merged`=`!res.added && res.merged`, `duplicates`=`!res.added && !res.merged`). The captured record's `capturedAt`
|
||||
is a today-shaped ISO date **distinct from the item's `publishedAt`** (field-confusion guard at the ingress; the
|
||||
exact-value assertion lives in `item.test.ts` with an injected date, since `capture` reads the wall clock). Exit
|
||||
**2** on empty/unparseable stdin; **0** on a well-formed call even with content-invalid items. `--json` emits the
|
||||
summary object.
|
||||
- **SC5 (gate)** — `(cd scripts/trends && npm test)` green at the bumped `TRENDS_TESTS_FLOOR`; new Section 16h green;
|
||||
`ASSERT_BASELINE_FLOOR` bumped to the live recount; overall gate `FAIL=0`.
|
||||
- **SC6 (wiring + de-niche)** — `trend-spotter.md` references `src/cli.ts capture` (Section 16h grep green); de-niche
|
||||
guard (Section 17) green; structure counts unchanged (19/29/27). *(If Open Q#1 → minimal: SC6 drops the wiring/16h
|
||||
clauses; de-niche + counts still asserted.)*
|
||||
|
||||
## 7. Verification
|
||||
|
||||
**Deterministic (gate):** `bash scripts/test-runner.sh` → `FAIL=0`; trends suite ≥ new floor; new Section 16h
|
||||
self-test + greps pass; Section 17 de-niche green; ref/agent/command counts unchanged.
|
||||
**Behavioural (manual):** `echo '<2-item batch incl. one publishedAt + one invalid>' | node --import tsx src/cli.ts
|
||||
capture --store /tmp/r2a-trends.json --json`; confirm the valid item lands with `publishedAt` persisted, the invalid
|
||||
one is in `errors[]`, and `node … list --store /tmp/r2a-trends.json --json` shows the persisted `publishedAt`.
|
||||
Then load an authored v1 fixture and confirm `status --json` reports it migrated (count intact).
|
||||
|
||||
## 8. Open questions for the go-gate
|
||||
|
||||
1. **Wire `trend-spotter.md` + add Section 16h, or keep R2a minimal? — RESOLVED at go-gate (operator, 2026-06-24): WIRE.**
|
||||
Build R2a with the agent re-point (Step 4.5 `add`→`capture`) + Section 16h grep/self-test + ASSERT-floor bump. The
|
||||
minimal alternative (bridge code + tests only, no agent edit) was declined — wiring mirrors R1's "the lift is real
|
||||
and grep-able" discipline and makes the bridge actually used (one normalizing `capture` vs N× `add`).
|
||||
2. **`publishedAt` merge policy.** Proposed **keep first-sight** (no back-fill), matching the existing source/capturedAt
|
||||
provenance rule (now pinned by the absent→present SC2 test). Confirm, or request back-fill-if-absent (adds a branch
|
||||
+ a `merged` semantic question).
|
||||
3. **`add --published-at` flag — RESOLVED at light-Voyage: deferred** (see §4). The manual `add` stays publish-date-free;
|
||||
`capture` is the path that carries `publishedAt`. Noted here only so the resolution is traceable; no go-gate action.
|
||||
|
||||
## 9. Light-Voyage review — folded
|
||||
|
||||
Three Opus reviewers ran on the drafts, each verifying claims against live code. **scope-guardian: ALIGNED**
|
||||
(every SC1–SC6 traces to a step; zero creep; all §4 non-goals held; 2 minors). **brief-reviewer:
|
||||
PROCEED_WITH_RISKS** (2 majors, 4 minors). **plan-critic: REVISE** (1 blocker, 5 majors, 5 minors). All findings
|
||||
folded; see `plan-re-r2a.md` §Plan-critic — folded for the per-step resolution:
|
||||
|
||||
- **[BLOCKER, folded]** "v2 load idempotent" cannot be a RED case — old `loadStore` (`store.ts:78`) returns
|
||||
`parsed.schemaVersion ?? SCHEMA_VERSION`, so a v2 fixture already loads as 2 against unchanged code. → §6 SC3 +
|
||||
§5 now split the migration cases: **only v1→2 + round-trip are RED**; v2-idempotence + missing/non-numeric are
|
||||
**GREEN-only regression guards**.
|
||||
- **[MAJOR, folded]** "lossless / byte-for-byte intact" over-claimed vs `loadStore`'s non-array `trends` coercion
|
||||
(`store.ts:79`). → §5 + §3 scope losslessness to **well-formed** v1/v2 stores; the coercion is unchanged + out of
|
||||
scope (not a regression).
|
||||
- **[MAJOR, folded]** `capturedAt` injection unverified at the new `capture` ingress (the field-confusion the slice
|
||||
exists to prevent). → SC1 + SC4 now pin `capturedAt !== publishedAt` (item.test exact-value; cli.test shape +
|
||||
distinct).
|
||||
- **[MAJOR, folded]** `capture` tally `{added, duplicates, merged}` mis-maps onto `AddResult {added, merged}` (no
|
||||
`duplicates` field). → SC4 pins the exact derivation + `sum === payload.length`.
|
||||
- **[MAJOR, folded]** `add --published-at` validation referenced the non-exported `isValidIso` (`item.ts:51`). →
|
||||
**`add --published-at` deferred entirely** (§4), removing the export-vs-inline decision and keeping `item.ts`'s
|
||||
edit to `itemToInput` only.
|
||||
- **[MAJOR, folded]** README "Capture … `add`" framing would contradict the agent's `add`→`capture` re-point. →
|
||||
plan Step 5 now corrects the README framing (`add` = manual single-trend; `capture` = normalizing batch), not just
|
||||
appends.
|
||||
- **[MAJOR, folded]** empty "folded" placeholders shipped in the doc bodies. → this section + plan §Plan-critic now
|
||||
filled.
|
||||
- **[MINOR, folded in plan]** missing-/non-numeric-`schemaVersion` test, absent→present back-fill test, stub-replacement
|
||||
note, `import type { TrendInput }` line, `grep -qF` for the 16h literal, `TRENDS_TESTS_FLOOR` breakdown-comment update,
|
||||
README in the code commit, headless per-step clauses N/A (in-session execution).
|
||||
309
docs/research-engine/brief-re-r2b.md
Normal file
309
docs/research-engine/brief-re-r2b.md
Normal file
|
|
@ -0,0 +1,309 @@
|
|||
# Brief — RE-R2b: dated morning-brief artifact + session-start surfacing
|
||||
|
||||
> **Slice:** RE-R2b (research-engine rung-2, slice 2 — the *visible* layer of R2). R2 ("the visible
|
||||
> topic-stream") was split at the 2026-06-24 go-gate into **R2a** (the pure `scripts/trends/` data layer —
|
||||
> capture bridge + `publishedAt` persistence, landed `7a15803`) and **R2b** (this — the dated morning-brief
|
||||
> artifact + session-start surfacing). R2a delivered **no visible change**; it closed the capture loop so the
|
||||
> store now accumulates publish-dated history. R2b makes that stream visible: a deterministic, dated Markdown
|
||||
> brief ranked over the store, surfaced at session-start.
|
||||
> **Predecessor:** RE-R2a (`brief-re-r2a.md`) persisted `publishedAt` first-sight and shipped the `capture`
|
||||
> CLI; it **explicitly deferred** the dated brief artifact (B3) and session-start surfacing (hull 4) to R2b
|
||||
> (`brief-re-r2a.md` §4). R2b builds exactly those two deferred pieces.
|
||||
> **Substrate:** `docs/research-engine-concepts.local.md` §3 **B3** (dated-digest as a flat plain-text
|
||||
> artifact: `YYYY-MM-DD.md`, diffbar/grep-bar/lastbar; delivery is a separate later step) + **hull 4**
|
||||
> (surfacing at session-start) + §1 hull list (2)(4)(7). **TDD-order:** RED tests land before code.
|
||||
|
||||
## 1. Operator decision context (2026-06-24)
|
||||
|
||||
The research engine is **Tier-1** (operator, 2026-06-23): *"hele min arbeidsflyt hviler på at jeg får en jevn
|
||||
strøm av gode forslag til tema å skrive om."* At the R2 go-gate the operator chose foundation-first: R2a (the
|
||||
bridge + schema) before R2b (the visible brief + surfacing). **R2b is the slice that delivers the visible
|
||||
value** — the operator opens a session and *sees* a dated brief of fresh, on-pillar topic signals, without
|
||||
running anything. The autonomous nightly trigger that would *regenerate* it unattended stays R3 (hull 1); in
|
||||
R2b the brief is produced **on demand** (a trend scan, or the `brief` CLI) and **surfaced** whenever one
|
||||
exists. Confirmed at format sign-off (2026-06-24): **D1** on-demand generation + session-start surfaces the
|
||||
latest (no tsx in the hook to regenerate); **D2** deterministic ranking on pillar-overlap → recency (no AI /
|
||||
no `score.ts` in the brief path — scores aren't persisted yet); **D3** freshness window = 7 days.
|
||||
|
||||
## 2. The gap — grounded in code
|
||||
|
||||
- **The store accumulates publish-dated history nobody reads.** Post-R2a, `TrendRecord` carries
|
||||
`publishedAt?` (`types.ts:43`) and the `capture` CLI folds polled trends in (`cli.ts:237-263`), but **no
|
||||
artifact ranks or presents the accumulated store.** The read surfaces today are `query` (topic-scoped,
|
||||
`cli.ts:143-164`) and `list` (time-scoped, `cli.ts:166-185`) — both are interactive CLI dumps, neither is a
|
||||
persisted, dated, surfaced brief. The "morgen-brief" rung the engine is aimed at (substrate §1, hull 2) does
|
||||
not exist.
|
||||
- **`publishedAt` is persisted but never ranked on.** `queryByTopic` ranks `topicOverlap desc → capturedAt
|
||||
desc` (`store.ts:155-157`) — it sorts on *when WE saw it*, never on *when the source published*. So even the
|
||||
freshest source item is ordered by capture time, not publish time. The field R2a persisted specifically for
|
||||
freshness ranking has no reader. R2b's `rankForBrief` is that reader.
|
||||
- **Session-start surfaces staleness, not signal.** The SessionStart hook already reads the store directly
|
||||
(zero-tsx) for the B-S3 staleness *nudge* (`session-start.mjs:38-52, 376-380`) — "trend signals are N days
|
||||
old, scan to refresh." It tells the operator the store is *stale*; it never tells them *what is in it*.
|
||||
Hull 4 (surfacing the actual brief) is unbuilt — the seam (a direct store/data-dir read in the hook) is
|
||||
already proven and reusable.
|
||||
|
||||
## 3. Scope — what is IN (RE-R2b)
|
||||
|
||||
### B-rank + B-render — `scripts/trends/src/brief.ts` (NEW `.ts`)
|
||||
Two **pure** functions + the brief's own format-version const + its result types. A new module is justified
|
||||
(cohesive brief layer — ranking + rendering + types — not a single-use function; contrast R2a's `itemToInput`,
|
||||
which belonged inside `item.ts`). No fs, no clock, no AI: `today` and `pillars` are injected by the caller.
|
||||
|
||||
- `rankForBrief(store: TrendStore, pillars: string[], today: string, opts?: { freshDays?: number }): BriefRanking`
|
||||
— for each trend computes `overlap` (count of `pillars` present in `trend.topics`, **case-insensitive, the
|
||||
same idiom as `queryByTopic` `store.ts:151-152`** — computed inline, `queryByTopic` is NOT refactored),
|
||||
`matchedPillars` (the actual matched names), `effectiveDate = publishedAt ?? capturedAt`, and
|
||||
`ageDays = Math.floor((Date.parse(today) - Date.parse(effectiveDate)) / 86400000)` (a **local** day-delta,
|
||||
**not** imported from `cli.ts`'s `daysBetween` `cli.ts:107-109` — that would invert the dependency direction,
|
||||
since `cli.ts` imports `brief.ts`, not the reverse). Groups: **`topMatches`** (`overlap ≥ 2` AND `ageDays ≤
|
||||
freshDays`), **`singleMatches`** (`overlap === 1` AND fresh), **`olderMatched`** (`overlap ≥ 1` AND NOT
|
||||
fresh). `overlap === 0` is **excluded entirely** (off-pillar noise). Within each group: sort `overlap desc`,
|
||||
then `effectiveDate desc` (freshest first), then `title asc`, then **`url asc`** — a **total order** (two
|
||||
records can share title+effectiveDate+overlap but never title+url, since title+url is the dedupe id
|
||||
`store.ts:66-68`; the `url` key makes order independent of store insertion / V8 sort stability). Returns
|
||||
`{ today, freshDays, totals: { trends, matched, fresh }, topMatches, singleMatches, olderMatched }`.
|
||||
`freshDays` default **7** (D3).
|
||||
- `renderBrief(ranking: BriefRanking): string` — produces the full Markdown artifact: a YAML frontmatter block
|
||||
(`date`, a one-line **`summary`** the hook surfaces verbatim, `store: { trends, matched, fresh }`, a
|
||||
`ranking:` descriptor, `schemaVersion: <BRIEF_SCHEMA_VERSION>`) + the body. **Body entry line (pinned):**
|
||||
`### <n>. <title>` then ``- Kilde: <source> · Publisert: <effectiveDate> (<ageDays>d) · Pillarer: <matched,
|
||||
joined> `` then optional summary then `🔗 <url>` (single-matches/older render as one-line bullets:
|
||||
`- <title> — «<pillar>» · <effectiveDate> (<ageDays>d) · 🔗 <url>`). The empty case (no fresh matches) still
|
||||
renders a valid brief with a "no fresh on-pillar signals" `summary`. The frontmatter `summary` is produced by
|
||||
the shared **`briefSummary(ranking)`** (below), NOT re-derived — one source for the frontmatter line and the
|
||||
CLI `--json`. **Deterministic:** same `(store, pillars, today, freshDays)` → byte-identical output (total-order
|
||||
sort, no clock/env inside the pure functions).
|
||||
- `briefSummary(ranking: BriefRanking): string` — the **single** source of the one-line summary, used by
|
||||
`renderBrief` (frontmatter) AND the CLI `--json`. Output is a **single line, column-0 in frontmatter, with no
|
||||
embedded `"` and no newline** (titles in «», fields separated by `·`) — so the hook's `extractYaml` regex
|
||||
(`^summary: *"?([^"\n]*)"?`, `session-start.mjs:20`) captures it whole. Fresh>0 →
|
||||
`<fresh> ferske tema-signaler matcher pillarene dine. Topp: «<top title>» (<top pillar> · <age>d).`;
|
||||
else → `Ingen ferske tema-signaler på pillarene dine (av <trends> i lager).`
|
||||
- `defaultBriefDir(): string` — **derives from `defaultStorePath()`** (`store.ts:190-193`): `join(dirname(
|
||||
defaultStorePath()), "morning-brief")` → `<root>/trends/morning-brief`. **One** root resolution (reuses
|
||||
`defaultStorePath`, imported from `store.js`); no independent re-resolution of `LINKEDIN_STUDIO_DATA` in
|
||||
`brief.ts` (M4 — kills the duplication the first draft introduced). Colocated with the store the brief reads
|
||||
(Open Q#2). Pure path computation, no fs.
|
||||
- `BRIEF_SCHEMA_VERSION = 1` — the artifact format version (distinct from the store's `SCHEMA_VERSION`).
|
||||
|
||||
### CLI `brief` subcommand (`scripts/trends/src/cli.ts`, EDIT)
|
||||
`node --import tsx src/cli.ts brief [--pillars <a,b,c>] [--fresh-days <N>] [--store <path>] [--out <dir>]
|
||||
[--json]` — flag-driven (reads the **store**, not stdin — unlike `normalize`/`score`/`capture`). Resolves
|
||||
`pillars = splitTopics(flags.pillars)` (the caller supplies the user's pillars — same edge-injection pattern as
|
||||
`capture` injecting `today()`; resolving pillars from the profile is a thin caller concern, §4), `freshDays`
|
||||
from `--fresh-days` (default 7; non-numeric → `usage()`→exit 2, mirroring `--limit`/`--threshold`), `store =
|
||||
loadStore(storePath)`, `outDir = flags.out && flags.out !== "true" ? flags.out : defaultBriefDir()` (the
|
||||
`!== "true"` guard is **required** — a bare `--out` with no value yields the string `"true"` via `parseFlags`
|
||||
`cli.ts:52-53`, so `flags.out ?? …` would write to `./true/`). Runs `rankForBrief(store, pillars, today(),
|
||||
{ freshDays })` → `renderBrief(ranking)` → writes `<outDir>/<today()>.md` (`mkdirSync({recursive})` +
|
||||
`writeFileSync` at the CLI edge — `brief.ts` stays pure). Human output: the written path + `(M matched, K
|
||||
fresh)`; `--json` emits `{ path, date, totals, summary }` where `summary = briefSummary(ranking)` (the **same**
|
||||
source as the frontmatter). Exit **2** only on malformed invocation; **0**
|
||||
otherwise, **including empty `--pillars`** (writes a valid no-match brief — the operator who hasn't set pillars
|
||||
still gets a dated, surfaceable file telling them to set pillars). Add a `brief …` line to `usage()` and the
|
||||
header doc comment.
|
||||
|
||||
### Session-start surfacing (`hooks/scripts/session-start.mjs`, EDIT — zero-tsx)
|
||||
A module-private `latestMorningBrief(briefDir)` (mirroring `trendsNewestCapture` `session-start.mjs:38-52` and
|
||||
`brainLastRun` `:57-65`): absent dir → `null`; else `readdirSync` → keep `/^\d{4}-\d{2}-\d{2}\.md$/` → sort
|
||||
desc → read the newest → extract `date` + `summary` via the existing `extractYaml` (`:19-23`) → return
|
||||
`{ date, summary, file }` (or `null` on any read failure — never throws). `extractYaml`'s capture is
|
||||
`[^"\n]*` + `.trim()` (`:20-23`), so the extracted `date`/`summary` are **newline-free by construction** →
|
||||
the surfaced block needs **no** `.replace(/\n/g,'\\n')` treatment (unlike the multi-line state sections
|
||||
`:320`); the static structure uses the literal `\\n` idiom (`:309-321`). **Pure fs + regex; never spawns
|
||||
`tsx`** (the analytics fresh-clone-crash invariant; the store schema/brief frontmatter are stable, so a direct
|
||||
read is safe — identical reasoning to the B-S3 comment `:32-37`). Injected as its own block **after the
|
||||
brain-missing nudge (`:500-504`)**, unconditional on a brief existing (so it surfaces on the fresh-install
|
||||
branch too, like the brain nudge): `## Morning Brief (<date>)` + the `summary` + `→ Full brief: <file>`. The
|
||||
brief dir is `join(getDataRoot('trends'), 'morning-brief')` — the **twin** of `defaultBriefDir()` (same
|
||||
established pattern as the store-path twin, `:376`). Heading kept English to match the existing 14 hook
|
||||
sections (Open Q#3); the `summary` body stays Norwegian (operator-facing).
|
||||
|
||||
### Wiring + gate (Open Q#1 — WIRE by default, mirrors R2a Open Q#1)
|
||||
- `agents/trend-spotter.md` (EDIT): after the Step 4.5 `capture` (re-pointed in R2a), add a step that runs
|
||||
**`node --import tsx scripts/trends/src/cli.ts brief --pillars <the pillars the agent already scans>`** so a
|
||||
trend scan *produces* today's dated brief — closing the visible loop **scan → capture → brief → surfaced next
|
||||
session**. Replacement prose carries the literal `src/cli.ts brief`. Domain-general (pillars are the user's
|
||||
config; no vendor/sector tokens). Keep the "skip silently if no deps" escape hatch.
|
||||
- `scripts/trends/README.md` (EDIT): document the `brief` subcommand + the `trends/morning-brief/YYYY-MM-DD.md`
|
||||
artifact + its frontmatter shape (honest CLI/artifact doc).
|
||||
- `scripts/test-runner.sh` (EDIT): bump `TRENDS_TESTS_FLOOR` — set it to the `tests N` line reported by
|
||||
`(cd scripts/trends && npm test)` after Steps 1–3 (it stays **inside** the deps guard; 79 today, `store.ts`
|
||||
comment `:697` is per-slice), and **append** `+ RE-R2b: brief +N + cli +M (morning-brief)` to that inline
|
||||
breakdown comment so number and comment can't drift. Add **Section 16i** ("Trends Brief Wiring"): insert it
|
||||
**immediately after Section 16h** (currently the **last** section before Section 18 — file order is 17→16g→
|
||||
16h→18, `:943/:1010/:1074/:1118`), i.e. after 16h's closing `fi`/`echo ""` (~`:1116`), **before** the Section
|
||||
18 block (`:1118`). Mirror 16h's shape: **unconditional**, deps-absent-safe `grep -qF` + a non-vacuity
|
||||
self-test — (1) self-test; (2) `command === "brief"` in `cli.ts`; (3) `src/cli.ts brief` in
|
||||
`agents/trend-spotter.md`; (4) `latestMorningBrief` in `session-start.mjs` (surfacing is wired, not merely
|
||||
documented). These are unconditional → bump `ASSERT_BASELINE_FLOOR` 90 → **live recount** (expected ~94).
|
||||
Update the header-enumeration **prose chain** (`:33-46`) by inserting the 16i clause between the 16h clause
|
||||
(`:43-45`) and the Section-18 clause (`:46`), preserving sentence flow.
|
||||
- **Hook suite (a SEPARATE gate, not run by `test-runner.sh`):** new
|
||||
`hooks/scripts/__tests__/session-start-morning-brief.test.mjs` (mirrors
|
||||
`session-start-trends-staleness.test.mjs`: subprocess + `LINKEDIN_STUDIO_DATA` fixture), green under
|
||||
`node --test hooks/scripts/__tests__/` (the command that runs the 136-test hook suite). `test-runner.sh` has
|
||||
no `HOOK_TESTS_FLOOR` and no root `package.json` — so SC6 asserts the hook test under its **own** command,
|
||||
never as part of `bash scripts/test-runner.sh`.
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **Autonomous nightly trigger** (cron/launchd, hull 1) — **R3**. R2b's brief is generated on demand (a scan /
|
||||
the CLI); the hook only *surfaces* the latest. *No scheduler enters the repo in R2b.*
|
||||
- **Freshness as a persisted seen-log / dedup-vs-seen (B4)** — R3. R2b's freshness is a **read-time filter**
|
||||
(`effectiveDate ≤ freshDays` at rank time), not an append-only seen-log.
|
||||
- **Relevance / saturation / status / first-mover scoring fields (hull 5)** — R3. R2b ranks on **pillar-overlap
|
||||
+ recency only**; the B2 triage scorer (`score.ts`) stays **out of the brief path** (its output isn't
|
||||
persisted on records yet — that's R3). No AI in the brief path (D2).
|
||||
- **Research-deepening (A1–A4), adapter sub-agents, MCP fetch fan-out** — R3.
|
||||
- **Pillar resolution from the state file** (`expertise_areas`) — OUT; pillars arrive via `--pillars` (the
|
||||
agent/caller supplies them). Wiring state→pillars is a thin follow-up, not this slice.
|
||||
- **Brief history surfacing / diffing ("yesterday vs today")** — OUT. The artifact is dated and accumulates on
|
||||
disk (hull 7 storage is satisfied), but R2b surfaces only the **latest**; diffing is later.
|
||||
- **Delivery (Slack/email)** — OUT. B3 keeps delivery a separate step; R2b's only "delivery" is session-start
|
||||
surfacing. No delivery channel, no push-window guard (that mechanism is R3/B4).
|
||||
- **Store schema change** — none. R2b is **read-only** over the store; `types.ts`/`store.ts` record shape and
|
||||
`SCHEMA_VERSION` (2) are untouched (only a pure `defaultBriefDir` path helper is added, in `brief.ts`).
|
||||
- **No new agent/command** — counts stay 19/29/27. `brief` is a CLI subcommand; surfacing is a hook edit;
|
||||
generation is wired into the existing `trend-spotter`.
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **TDD iron law:** the failing `brief.ts` tests (`rankForBrief` grouping/freshness/sort + `renderBrief`
|
||||
frontmatter/`summary`/empty-case + `defaultBriefDir`) and the `cli.ts brief` tests land **BEFORE** the
|
||||
implementation. RED is logic-RED (throwing stub for the not-yet-existing exports so cases fail on assertion,
|
||||
not module-not-found).
|
||||
- **Purity:** `rankForBrief`/`renderBrief`/`defaultBriefDir` touch no fs, no clock, no env-at-call, no AI —
|
||||
`today`/`pillars`/`freshDays` are injected. All fs lives at the CLI edge (`cli.ts`) and in the hook.
|
||||
- **Determinism:** same `(store, pillars, today, freshDays)` → byte-identical brief (stable three-key sort).
|
||||
- **Hook stays tsx-free:** surfacing is a plain `readdir` + `readFile` + `extractYaml` (the established
|
||||
zero-dep pattern); it **never** shells out to `tsx` (analytics fresh-clone-crash invariant). A fixture run
|
||||
with no `node_modules/tsx` in `scripts/trends` must still surface the brief (SC5).
|
||||
- **Domain-general:** Section 17 de-niche guard stays green; the `trend-spotter.md` edit + the brief artifact
|
||||
carry **no vendor/sector tokens**; pillars are the user's config, never hardcoded.
|
||||
- **No scoring change:** `score.ts` + `references/trend-scoring-modes.md` (SSOT) untouched.
|
||||
- **No store schema change:** `types.ts`/`store.ts` record shape + `SCHEMA_VERSION` unchanged; `queryByTopic`
|
||||
NOT refactored (overlap is recomputed in `brief.ts`).
|
||||
- **Pathguard:** `brief.ts` is a NEW `.ts` (allowed); **NO new `.mjs` under `hooks/scripts/`** (surfacing edits
|
||||
the existing `session-start.mjs`); `.gitignore` already covers `scripts/trends/{node_modules,build}`.
|
||||
- **House conventions:** ESM + `node:test` + `tsx`; brief+plan docs live under `docs/` (uncounted, TRACKED like
|
||||
`docs/second-brain/*`).
|
||||
- **Counts** (refs/agents/commands 27/19/29) unchanged; `brief.ts` is the only new source file. **Recounted
|
||||
live at land**, never pinned/guessed.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1 (rank/group)** — `rankForBrief(fixtureStore, pillars, today)` puts only `overlap ≥ 2 & fresh` in
|
||||
`topMatches`, `overlap === 1 & fresh` in `singleMatches`, `overlap ≥ 1 & NOT fresh` in `olderMatched`;
|
||||
excludes `overlap === 0`; within each group orders `overlap desc → effectiveDate desc → title asc`;
|
||||
`matchedPillars` lists the actual matched names (case-insensitive match, original-case pillar preserved);
|
||||
`totals.matched` = sum of the three groups, `totals.fresh` = top+single, `totals.trends` = `store.trends.length`.
|
||||
- **SC2 (freshness)** — `effectiveDate = publishedAt ?? capturedAt`; an item whose `publishedAt` is within
|
||||
`freshDays` but whose `capturedAt` is older is **fresh** (and the reverse via the fallback when `publishedAt`
|
||||
is absent); the boundary `ageDays === freshDays` is **fresh** (`≤`); `freshDays` is configurable (a non-7
|
||||
value re-buckets).
|
||||
- **SC3 (render/frontmatter)** — `renderBrief(ranking)` output begins with a YAML frontmatter block carrying
|
||||
`date`, a **column-0, single-line `summary` with no embedded `"` and no newline** (so `extractYaml` reads it
|
||||
whole), `store: { trends, matched, fresh }`, and `schemaVersion: 1`; `renderBrief`'s frontmatter `summary`
|
||||
equals `briefSummary(ranking)` byte-for-byte (one source); the body has the three sections in order with the
|
||||
pinned entry-line shape (§3); the **empty-matches** ranking renders a valid brief whose `summary` is the
|
||||
"no fresh on-pillar signals" line (still surfaceable); identical input → identical bytes (determinism, total
|
||||
order). `briefSummary(emptyRanking)` returns the no-fresh line; `briefSummary(freshRanking)` names the top
|
||||
entry.
|
||||
- **SC4 (CLI brief)** — `… brief --pillars a,b --store <tmp> --out <tmpdir>` writes `<tmpdir>/<today>.md`
|
||||
(today-shaped `/^\d{4}-\d{2}-\d{2}$/`) and prints the path; `--json` emits `{ path, date, totals, summary }`;
|
||||
`--fresh-days xyz` → exit **2**; **empty/absent `--pillars`** → writes a no-match brief, exit **0**; `--out`
|
||||
overrides the dir; the default dir (no `--out`) is `defaultBriefDir()`.
|
||||
- **SC5 (surfacing)** — running `session-start.mjs` (subprocess) with `LINKEDIN_STUDIO_DATA` pointing at a
|
||||
fixture root containing `trends/morning-brief/<date>.md` → `additionalContext` contains `## Morning Brief
|
||||
(<date>)`, the brief's `summary`, and `→ Full brief: <file>`, and carries **no raw newline** inside that
|
||||
block (single-escaped `\n` idiom held); an absent brief dir → **no** block and **no crash** (`continue:
|
||||
true`); the surfacing works with **no `tsx`/`node_modules` present** (zero-dep proof). **Path cross-check:** a
|
||||
file written by the CLI at `defaultBriefDir()` (under a given `LINKEDIN_STUDIO_DATA`) is the one the hook
|
||||
finds via `join(getDataRoot('trends'),'morning-brief')` — the CLI-write/hook-read paths resolve equal (the
|
||||
store-path twin equivalence already guarded by `__tests__/data-root.test.mjs`).
|
||||
- **SC6 (gate + wiring + de-niche) — TWO separate gate commands:**
|
||||
**(a)** `bash scripts/test-runner.sh` → `FAIL=0`: trends suite green at the bumped `TRENDS_TESTS_FLOOR`; new
|
||||
**Section 16i** green (`command === "brief"` in `cli.ts`, `src/cli.ts brief` in `trend-spotter.md`,
|
||||
`latestMorningBrief` in `session-start.mjs`, non-vacuity self-test); `ASSERT_BASELINE_FLOOR` = live recount;
|
||||
Section 17 de-niche green; counts 27/19/29.
|
||||
**(b)** `node --test hooks/scripts/__tests__/` → the new `session-start-morning-brief.test.mjs` green (the
|
||||
hook suite is **not** part of `test-runner.sh`).
|
||||
*(If Open Q#1 → minimal: SC6(a) drops the `trend-spotter.md`/16i-wire clause; brief.ts + cli + surfacing +
|
||||
de-niche + counts still asserted.)*
|
||||
|
||||
## 7. Verification
|
||||
|
||||
**Deterministic (two gates):** (a) `bash scripts/test-runner.sh` → `FAIL=0`; trends suite ≥ new floor; new
|
||||
Section 16i self-test + greps pass; Section 17 de-niche green; ref/agent/command counts unchanged. (b)
|
||||
`node --test hooks/scripts/__tests__/` → the new `session-start-morning-brief.test.mjs` green (hook suite is a
|
||||
separate command, not part of `test-runner.sh`).
|
||||
**Behavioural (manual):**
|
||||
1. `echo '[{"source":"tavily","title":"A","url":"https://e/a","topics":["ai","gov"],"publishedAt":"<~2d
|
||||
ago>"},{"source":"tavily","title":"B","url":"https://e/b","topics":["ai"],"publishedAt":"<~20d ago>"}]' |
|
||||
node --import tsx src/cli.ts capture --store /tmp/r2b.json` (seed the store).
|
||||
2. `node --import tsx src/cli.ts brief --pillars ai,gov --store /tmp/r2b.json --out /tmp/r2b-brief --json` →
|
||||
confirm A in `topMatches` (overlap 2, fresh), B in `olderMatched` (overlap 1, stale), the written path, and a
|
||||
`summary` naming A.
|
||||
3. Place that brief at `<root>/trends/morning-brief/<today>.md` and run `LINKEDIN_STUDIO_DATA=<root> node
|
||||
hooks/scripts/session-start.mjs` → confirm the `## Morning Brief` block + the `summary` appear in
|
||||
`additionalContext`, with `tsx` absent.
|
||||
|
||||
## 8. Open questions for the go-gate
|
||||
|
||||
1. **Wire `brief` generation into `trend-spotter.md` (after capture) + Section 16i, or keep R2b to
|
||||
machinery+surfacing only?** **Recommend WIRE** (mirrors R2a Open Q#1): a scan then *produces* the brief, so
|
||||
the surfacing isn't surfacing an artifact nothing creates — it closes the scan→capture→brief→surfaced loop.
|
||||
Minimal alt: `brief.ts` + CLI + surfacing + tests, no agent edit (the operator runs `brief` by hand).
|
||||
2. **Brief dir = `trends/morning-brief/`** (colocated with the store it reads) — recommend. **Load-bearing**
|
||||
(not a cheap toggle): it is baked into `defaultBriefDir()`, the hook twin, and SC5's path assertions.
|
||||
Alternatives `research/morning-brief/` (the `docs/research-engine` naming the mock showed) or `morning-brief/`
|
||||
at the data-root (substrate §3 B3 literal) would re-touch `brief.ts` + the hook + SC5. Confirm the path; a
|
||||
different choice means updating those three places.
|
||||
3. **Surfacing heading: English `## Morning Brief`** (matches the existing 14 hook section headings —
|
||||
"Posting Reminders", "Queue Summary", "Brain") with a Norwegian `summary` body — recommend. Alt: the mock's
|
||||
Norwegian `## 🌅 Morgen-brief`. (The block is `additionalContext` for the model, not direct user UI; the
|
||||
user-facing *artifact* body stays Norwegian either way.) The one genuinely-cosmetic question here.
|
||||
4. **`summary:` frontmatter field on the artifact** — **confirm** (this is a decision baked in, not a free
|
||||
choice): the entire zero-tsx surfacing (SC3/SC5, `extractYaml(content,'summary')`) depends on it; without a
|
||||
`summary` frontmatter line the hook would have to parse the body (which the no-tsx invariant forbids), so the
|
||||
slice cannot ship without it. The approved visible *body* is unchanged; this only adds one frontmatter line
|
||||
the hook reads.
|
||||
|
||||
## 9. Light-Voyage review — folded
|
||||
|
||||
Three Opus reviewers ran on the drafts, each verifying claims against live code. **scope-guardian: ALIGNED**
|
||||
(every SC1–SC6 traces to a step; zero creep; all nine §4 non-goals held; counts 27/19/29 verified live; 0
|
||||
findings). **brief-reviewer: PROCEED_WITH_RISKS** (1 real risk + minors). **plan-critic: REVISE** (2 blockers,
|
||||
5 majors, 4 minors). All findings folded; see `plan-re-r2b.md` §Plan-critic — folded for per-step resolution:
|
||||
|
||||
- **[BLOCKER, folded]** `--json` summary source was an unresolved either/or ("re-derive OR expose"). → a
|
||||
committed **`briefSummary(ranking)`** export is now the single source for both `renderBrief`'s frontmatter and
|
||||
the CLI `--json` (§3; SC3 pins equality). Step 1 asserts against the named export.
|
||||
- **[BLOCKER, folded]** Section 16i placement was ambiguous. → pinned: immediately after Section 16h (the last
|
||||
section before 18; file order 17→16g→16h→18), before the Section 18 block (`:1118`).
|
||||
- **[MAJOR, folded]** the throwing-stub RED claim contradicted "fail on assertion, not module-not-found". →
|
||||
Step 1 now creates **wrong-but-non-throwing** stubs (empty buckets / `""`) so `brief.test.ts` fails on
|
||||
*assertion*; `cli.test.ts` brief cases are logic-RED against the existing dispatch (unknown command → exit 2).
|
||||
- **[MAJOR, folded]** brief §3 `outDir = flags.out ?? defaultBriefDir()` would write `./true/` for a bare
|
||||
`--out`. → corrected to the `flags.out !== "true"` guard (`parseFlags` `cli.ts:52-53`); a bare-`--out`
|
||||
cli.test case added.
|
||||
- **[MAJOR, folded]** `TRENDS_TESTS_FLOOR` recount was ambiguous (full `tests N` vs additive). → pinned: the
|
||||
`tests N` line after Steps 1–3; the inline `:697` comment appends `+ RE-R2b: brief +N + cli +M`.
|
||||
- **[MAJOR, folded]** `defaultBriefDir` re-resolved the data root independently (triple-twin drift). → it now
|
||||
**derives from `defaultStorePath()`** (`join(dirname(defaultStorePath()), "morning-brief")`) — one root
|
||||
resolution; the hook-vs-CLI path equivalence is the store-path twin already guarded by `data-root.test.mjs`;
|
||||
SC5 adds a write-then-read path cross-check.
|
||||
- **[MAJOR, folded]** the surfacing newline-escape was unproven. → §3 + SC5 state `extractYaml`'s `[^"\n]*` +
|
||||
`.trim()` makes `date`/`summary` newline-free → no `.replace` needed; SC5 asserts no raw newline in the block.
|
||||
- **[brief-reviewer MAJOR, folded]** SC6 attributed "hook-suite at recount" to `bash scripts/test-runner.sh`,
|
||||
which neither runs nor counts the hook suite (no `HOOK_TESTS_FLOOR`, no root `package.json`). → SC6 + §7 now
|
||||
split into **two** gate commands: `test-runner.sh` (trends/16i/ASSERT/de-niche/counts) and
|
||||
`node --test hooks/scripts/__tests__/` (the new hook test).
|
||||
- **[MINOR, folded]** non-total sort → added `url asc` final tie-break (true total order, §3/SC1). **[MINOR,
|
||||
folded]** local `ageDays` math stated as a deliberate non-import (dependency direction `cli.ts → brief.ts`).
|
||||
**[MINOR, folded]** `.md$`-anchored filename filter confirmed. **[MINOR, folded]** header-enumeration is a
|
||||
prose chain → insert the 16i clause between the 16h and Section-18 clauses (`:43-46`). **[brief-reviewer
|
||||
MINOR, folded]** body entry-line age format pinned in §3 + SC3. **[brief-reviewer MINOR, folded]** Open Q#2/#4
|
||||
reframed as load-bearing confirmations (§8).
|
||||
357
docs/research-engine/brief-re-r3a.md
Normal file
357
docs/research-engine/brief-re-r3a.md
Normal file
|
|
@ -0,0 +1,357 @@
|
|||
# Brief — RE-R3a: persist the relevance score + rank the morning brief on it
|
||||
|
||||
> **Slice:** RE-R3a (research-engine rung-2, R3 slice 1 — research-*deepening*). R3 ("deepen the
|
||||
> research engine") is an **arc** of 5 open hulls (substrate §1: autonomous trigger · freshness-as-seen-log ·
|
||||
> relevance/saturation/status scoring · brief history+diff · A1–A4 fan-out). R3a takes the first: the
|
||||
> **relevance** half of hull 5 (and the remainder of hull 3 — "the store schema lacks fields a brief ranks
|
||||
> on"). It persists the composite relevance score the `trend-spotter` agent ALREADY computes, onto the store
|
||||
> record, and makes `rankForBrief` order on it.
|
||||
> **Predecessor:** RE-R1 (`score.ts`, B2 triage-scorer — built, tested, deterministic) + RE-R2a (`capture`
|
||||
> bridge + `publishedAt`, schema v1→v2) + RE-R2b (`brief.ts` dated artifact + surfacing). R2b explicitly
|
||||
> deferred this in its §4: *"the B2 triage scorer stays out of the brief path — its output isn't persisted on
|
||||
> records yet — that's R3."* R3a is exactly that R3 step.
|
||||
> **Substrate:** `docs/research-engine-concepts.local.md` §1 hull (3) (schema fields a brief ranks on:
|
||||
> relevance/...) + (5) (relevance scoring) + §B2 ("scoring/filtering as a gate before expensive work — the
|
||||
> output is the rank key") + §A2 ("curate/score before synthesis — the writer sees ranked material").
|
||||
> **TDD-order:** RED tests land before code — but as **two phases** (light-Voyage BLOCKER fold): the
|
||||
> store/brief/cli tests are true logic-RED against the pre-edit code (they build fixtures inline, import no new
|
||||
> symbol); the score/item tests reference not-yet-existing `score.ts` exports, so under Node16 ESM a missing
|
||||
> named import throws at module-load (not on assertion) — they are RED against **non-throwing stubs** landed as
|
||||
> the first GREEN-prep sub-step. See plan Step 1.
|
||||
|
||||
## 1. Operator decision context (2026-06-24)
|
||||
|
||||
The research engine is **Tier-1** (operator, 2026-06-23): *"hele min arbeidsflyt hviler på at jeg får en jevn
|
||||
strøm av gode forslag til tema å skrive om."* RE-R2 made the stream **visible** (a dated morning brief surfaced
|
||||
at session-start). R2b ranks that brief on **pillar-overlap + recency only** — a coarse proxy for "good topic
|
||||
to write about." The actual relevance judgment (audience pull, timing, angle potential, authority, depth) lives
|
||||
in the five 1–10 dimension scores the `trend-spotter` agent produces in Step 2 and pipes to the `score` CLI —
|
||||
and is then **thrown away** before the trend reaches the store (Step 4.5 builds a *separate*, score-free capture
|
||||
batch). R3a stops discarding it: persist the composite + band on the record, and rank the brief on composite
|
||||
first. **The slice the operator chose** ("scoring inn i briefen", 2026-06-24) — the highest-leverage next step
|
||||
on the core need (better-ordered suggestions), built on already-shipped-but-dormant code (`score.ts` is tested
|
||||
and unused on records). The bigger R3 arcs (autonomous trigger / seen-log / saturation+status / A1–A4 fan-out)
|
||||
stay later slices.
|
||||
|
||||
**Go-gate resolutions — CONFIRMED (operator "Go", 2026-06-24; baked into the plan):** **D1** persist the
|
||||
**4-field** `TrendScore { mode, dimensions, composite, priority }` (composite+priority to rank/display, mode to
|
||||
disambiguate the instrument, dimensions for audit + lossless re-weight). **D2** composite is the **primary
|
||||
within-bucket sort** (buckets still assigned by overlap+freshness; composite orders *inside* a bucket). **D3**
|
||||
score is **first-sight** (set on add, never updated on re-capture — matches the store's provenance discipline;
|
||||
re-score-on-recapture pairs with the R3b seen-log/status slice). **D4** ship persist+rank as **one** slice (the
|
||||
operator named the visible payoff; splitting would land an invisible schema-only cut like R2a).
|
||||
|
||||
## 2. The gap — grounded in code
|
||||
|
||||
- **The score the agent computes never reaches the store.** `trend-spotter.md` scores each candidate's five
|
||||
dimensions and pipes them to the `score` CLI (`agents/trend-spotter.md:134-140`), which returns
|
||||
`{composite, band}` per candidate (`score.ts:110-122` `triage`). But Step 4.5's capture batch
|
||||
(`agents/trend-spotter.md:291-298`) is **built separately and carries no score** — `source/title/url/topics/
|
||||
publishedAt/summary` only. `TrendItem` (`item.ts:22-39`) and `TrendRecord` (`types.ts:26-48`) have **no score
|
||||
field**. The relevance judgment is recomputed for the digest and discarded for persistence.
|
||||
- **`score.ts` is built, tested, deterministic — and unconsumed on records.** It exports `composite()`
|
||||
(`score.ts:77-88`) and `band()` (`score.ts:91-97`) as pure functions, pinned to the SSOT
|
||||
(`references/trend-scoring-modes.md`, by `score.test.ts:12-30` weights + the band-string assertions). Nothing
|
||||
persists their output. `TrendRecord`'s own doc-comment anticipates the field: *"can gain fields (relevance
|
||||
score, first-mover timing, status) in a later slice"* (`types.ts:21-23`).
|
||||
- **The brief ranks on a proxy.** `rankForBrief` sorts each bucket `overlap desc → effectiveDate desc →
|
||||
title asc → url asc` (`brief.ts:94-104`). Overlap (a hard pillar count) is *part* of what the composite
|
||||
already weights (Pillar Fit 30 %, `trend-scoring-modes.md:43`), but the composite also captures audience/
|
||||
timing/angle/authority — signal the brief currently can't see. `brief.ts`'s own header already names this as
|
||||
the next slice: *"A persisted relevance/saturation score … (R3)"* (`brief.ts:11-14`).
|
||||
|
||||
## 3. Scope — what is IN (RE-R3a)
|
||||
|
||||
### S-score — `scripts/trends/src/score.ts` (EDIT)
|
||||
- **`export interface TrendScore { mode: ScoreMode; dimensions: DimensionScores; composite: number; priority:
|
||||
Priority }`** — the persist-ready envelope. Lives in `score.ts` (the score domain owns it); `types.ts` imports
|
||||
it (one-way: `score.ts` imports nothing — verified leaf, `:1-17` — so no cycle).
|
||||
- **`export function requiredDimensions(mode: ScoreMode): string[]`** — `Object.keys(WEIGHTS[mode])`
|
||||
(`score.ts:37-40`). **Contract: ordered** — the keys come back in the SSOT weight-literal order (kortform
|
||||
`["pillar","audience","timing","angle","authority"]`, long-form `["pillar","depth","angle","authority",
|
||||
"currency"]`, `score.ts:20-35`); SC1 deep-equals that ordered array, and `score.test` pins the order so a
|
||||
silent SSOT reorder fails loudly. `normalizeItem` consumes it as a **set** (membership), which is order-safe
|
||||
either way. `WEIGHTS` stays private; the keys are exposed via this function.
|
||||
- **`export function scoreEnvelope(mode: ScoreMode, dimensions: DimensionScores): TrendScore`** — composes the
|
||||
existing pure functions: `const c = composite(dimensions, mode); return { mode, dimensions, composite: c,
|
||||
priority: band(c).priority }`. **No new arithmetic** — `composite()`+`band()` stay the single owners (SSOT
|
||||
discipline). It throws (via `composite`, `score.ts:83`) on an out-of-range dimension — that is its
|
||||
**contract**, exercised directly by SC1/SC2; on the capture path it is unreachable because `normalizeItem`
|
||||
pre-validates (below).
|
||||
|
||||
### S-types — `scripts/trends/src/types.ts` (EDIT)
|
||||
- `import type { TrendScore } from "./score.js";`
|
||||
- `TrendRecord` gains **`score?: TrendScore;`** (optional — pre-R3a records simply lack it; the `add` manual
|
||||
path and unscored adopters omit it). Doc-comment updated to mark `score` as the now-realized field the
|
||||
`:21-23` note anticipated.
|
||||
- **`SCHEMA_VERSION = 2 → 3`** (`types.ts:62`). The bump is the only schema signal; the record shape change is
|
||||
additive-optional, so the migration is the version-stamp alone (below).
|
||||
|
||||
### S-store — `scripts/trends/src/store.ts` (EDIT)
|
||||
- `TrendInput` (`store.ts:26-35`) gains **`score?: TrendScore;`** (imported from `score.js`).
|
||||
- `addTrend` (`store.ts:120-140`): on a **new** record, persist `score` first-sight via the existing
|
||||
conditional-spread idiom (`...(input.score !== undefined ? { score: input.score } : {})`, mirroring
|
||||
`publishedAt`/`summary` `:134,136`). On a **duplicate**, score is **NOT** updated (D3 — first-sight, like
|
||||
`source`/`capturedAt`/first `publishedAt`); topics still union (`:124-126`, unchanged). `AddResult` is
|
||||
unchanged (no new flag).
|
||||
- `loadStore` migrate comment (`:79-84`): extend to *"v1→v2→v3 are all purely additive-optional (an old record
|
||||
is already a valid record that simply lacks the optional field), so the migration is the version stamp alone —
|
||||
records pass through untouched."* **No code change** to the migration logic (`Math.max(onDisk, SCHEMA_VERSION)`
|
||||
`:87` already does v2→v3 correctly; `saveStore` `JSON.stringify` `:95` preserves the `score` field — no field
|
||||
stripping); only `SCHEMA_VERSION` (in `types.ts`) and the comment move.
|
||||
|
||||
### S-item — `scripts/trends/src/item.ts` (EDIT)
|
||||
- `TrendItem` (`item.ts:22-39`) gains **`score?: { mode: ScoreMode; dimensions: DimensionScores };`** — the
|
||||
ingress envelope carries the agent's *judgment* (five scores + mode), **not** a precomputed composite (the
|
||||
store computes it, so the composite has one owner). `import type { ScoreMode, DimensionScores } from
|
||||
"./score.js"` + `import { requiredDimensions } from "./score.js"`.
|
||||
- `normalizeItem` (`:86-119`): if `r.score` present, **validate structurally** (returns a structured error into
|
||||
`errors[]`, never throws — the existing discipline, like the `publishedAt` ISO check `:99-106`): `score` is a
|
||||
**non-array** object; `mode ∈ {kortform, long-form}`; `dimensions` is a **non-array** object; **each key in
|
||||
`requiredDimensions(mode)` is present and a number in [1,10]**. On any failure → `errors.push("invalid score:
|
||||
…")`. On success carry the **validated** `score = { mode, dimensions }` forward (the validated dimensions
|
||||
object, not raw `r.score.dimensions`). Absent/null/invalid → key omitted. This guarantees the *capture path*
|
||||
(`cli.ts:246-254`: `normalizeItems` → `itemToInput`) never reaches `composite` with bad dims.
|
||||
- `itemToInput` (`:129-139`): if `item.score` present → add `score: scoreEnvelope(item.score.mode,
|
||||
item.score.dimensions)` to the returned `TrendInput` (conditional spread, key omitted when absent). The
|
||||
item→store bridge is the natural place to turn judgment into the persisted envelope. `itemToInput` is a public
|
||||
function: called directly (e.g. in a test) with unvalidated dims it **throws by contract** (defense-in-depth);
|
||||
the no-throw guarantee is a property of the *capture path*, not of `itemToInput` in isolation (§5).
|
||||
|
||||
### S-brief — `scripts/trends/src/brief.ts` (EDIT)
|
||||
- `rankForBrief` (`:72-114`): **composite becomes the primary within-bucket sort key** (D2). The comparator
|
||||
(`:94-98`) gains a leading term:
|
||||
`(b.trend.score?.composite ?? -1) - (a.trend.score?.composite ?? -1) || <existing overlap desc → effectiveDate
|
||||
desc → title asc → url asc>`. **Sentinel `-1`, not `-Infinity`** — composite is a weighted sum of [1,10]
|
||||
dims so it is always ≥ 1.0 (min = 1×Σweights = 1.0, verified); `-1` sorts every unscored record below every
|
||||
scored one and subtracts cleanly (`-Infinity - -Infinity = NaN` would corrupt the comparator). **Buckets are
|
||||
UNCHANGED** — assignment stays `overlap≥2 & fresh` / `overlap==1 & fresh` / `!fresh` (`:100-104`); composite
|
||||
only re-orders *within* a bucket. Total order preserved: the `(title,url)` pair is unique per store (it is the
|
||||
dedupe id, `store.ts:66-68`), so the final `url asc` tie-break makes the order insertion-independent even for
|
||||
equal composites.
|
||||
- `renderBrief` (`:152-191`): surface the band **and mode** where a record is scored (so a reader can tell a
|
||||
kortform "High" from a long-form "High" — the two are different instruments). **Pinned line shapes:**
|
||||
- Top-entry meta line (`renderTopEntry`, `:135`), scored:
|
||||
`- Kilde: <source> · Publisert: <date> (<age>d) · <priority> (<mode>) · Pillarer: <matched>`
|
||||
(the `· <priority> (<mode>)` token sits between `(<age>d)` and `· Pillarer`); **unscored: unchanged** (no
|
||||
token).
|
||||
- Bullet line (`renderBulletEntry`, `:144`), scored:
|
||||
`- **<title>** — «<matched>» · <date> (<age>d) · <priority> (<mode>) · 🔗 <url>`
|
||||
(token **before** `· 🔗`); **unscored: unchanged**.
|
||||
- The `ranking:` frontmatter descriptor (`:160`) → the **exact** string
|
||||
`composite desc, then pillar-overlap desc, then publishedAt desc (capturedAt fallback); freshDays <N>`
|
||||
(pinned verbatim; `brief.test` asserts it byte-for-byte).
|
||||
- `briefSummary` (`:122-130`): the top mention names the **band only** (mode stays a body-entry detail to keep
|
||||
the one-line headline clean) — fresh>0 with a **scored** top → `… Topp: «<title>» (<pillar> · <priority> ·
|
||||
<age>d).`; fresh>0 with an **unscored** top → `… Topp: «<title>» (<pillar> · <age>d).` (no token). **Still one
|
||||
line, no `"`, no `\n`** — the `extractYaml` contract (`brief.ts:118-120`) holds; the band strings
|
||||
(`Immediate`/`High`/…) are bare words.
|
||||
- `BRIEF_SCHEMA_VERSION` stays **1** (no frontmatter *field* added/removed; the surfacing hook still reads
|
||||
`date`+`summary`; only the `ranking:` descriptor *string* and body content change). Bumping is an Open Q (§8),
|
||||
not required for correctness.
|
||||
|
||||
### S-cli — `scripts/trends/src/cli.ts` (EDIT, doc-only behavior)
|
||||
- The `capture` branch (`:243-269`) folds via `itemToInput` (`:254`) — so once `item.ts` threads `score`,
|
||||
capture **automatically** persists it with **no logic change**. Update only the header doc-comment
|
||||
(`:15-21`) to note capture now persists an optional relevance score. The `score` CLI (`:218-241`, the digest
|
||||
path) and the `add` manual path (`:123-147`, score-free) are unchanged. *(Capture's `{added, merged,
|
||||
duplicates, errors}` tally is left unchanged — a `scored` count is an Open-Q nice-to-have, §8.)*
|
||||
|
||||
### Wiring (D-default — WIRE, mirrors R2a/R2b Open Q#1)
|
||||
- `agents/trend-spotter.md` (EDIT): Step 4.5's capture batch (`:291-298`) gains a per-item **`"score": {"mode":
|
||||
"kortform", "dimensions": {"pillar": N, "audience": N, "timing": N, "angle": N, "authority": N}}`** — the same
|
||||
five judgment scores the agent computed in Step 2 (`:134`), carried into capture so the store persists them and
|
||||
the brief ranks on them. Prose explains the carry ("don't discard the Step-2 scores — fold them into the
|
||||
capture batch"). Mode defaults `kortform`; `long-form` when invoked from `/linkedin:newsletter` (long-form
|
||||
dims `pillar/depth/angle/authority/currency`). Domain-general (dimensions are the rubric's, pillars are the
|
||||
user's config; no vendor/sector tokens). Keep the "skip silently if no deps" escape hatch. **Verified
|
||||
non-vacuous:** `agents/trend-spotter.md` does NOT currently contain the literal `"dimensions"`, so the
|
||||
Section 16j grep passes only after the wire is added.
|
||||
- `scripts/trends/README.md` (EDIT): document the item `score` field (judgment in), the persisted `TrendScore`
|
||||
(composite/priority out), and that the brief now ranks on composite.
|
||||
- `scripts/test-runner.sh` (EDIT): bump `TRENDS_TESTS_FLOOR` (`:701`, currently 104) to the `tests N` line
|
||||
reported after Steps 1–6, **append** `+ RE-R3a: score +N` to the inline breakdown comment (`:701`). Add
|
||||
**Section 16j** ("Trends Score Wiring", RE-R3a) **after Section 16i's closing `echo ""` (~`:1171`), before the
|
||||
Section 18 block (`:1173`)** (16i is the last 16x before anti-erosion; file order 17→16g→16h→16i→18,
|
||||
`:947/:1014/:1078/:1122/:1173`). Mirror 16i's shape: **unconditional**, deps-absent-safe `grep -qF` + a
|
||||
non-vacuity self-test emitting **one** pass/fail (so the count is exact) — (1) self-test; (2) `export
|
||||
interface TrendScore` in `score.ts`; (3) `score?: TrendScore` in `types.ts`; (4) `"dimensions"` in
|
||||
`agents/trend-spotter.md`; (5) `score?.composite` in `brief.ts`. **5 unconditional emitters → bump
|
||||
`ASSERT_BASELINE_FLOOR` 94 → exactly 99** (`:1193`; "live recount" is the safety net, but the expected value
|
||||
is the pinned 94 + 5 = 99). Update the header-enumeration **prose chain** by inserting the 16j clause between
|
||||
the 16i clause (`:46-49`) and the Section-18 clause (`:49`), preserving sentence flow.
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **Re-score on re-capture** (refresh the score when a trend is re-seen) — **R3b**. R3a is first-sight only
|
||||
(D3). Re-score pairs naturally with the seen-log/status slice (the Timing dimension decays, so a refresh is a
|
||||
real improvement — but it expands `addTrend`'s mutation surface and wants the status/lifecycle model alongside).
|
||||
- **Mode-segmented / mode-normalized ranking** — OUT. R3a ranks **all** records by composite regardless of mode;
|
||||
a kortform composite and a long-form composite are different instruments (different dimensions,
|
||||
`trend-scoring-modes.md:50,68`), so the ranking is **mode-blind by design for R3a**. This is acceptable because
|
||||
(a) almost all records are `kortform` (the default), and (b) the body entry line **shows the mode** (`<priority>
|
||||
(<mode>)`) so the operator can see when two adjacent entries were scored on different instruments. A
|
||||
mode-segmented brief (separate sections per mode) or a `--mode` filter is a later refinement.
|
||||
- **Saturation / status (acted/skipped) / first-mover-as-a-field** (the rest of hull 5) — **R3b+**. R3a does the
|
||||
**relevance** half of hull 5 only.
|
||||
- **Autonomous nightly trigger** (cron/launchd, hull 1) — **R3 later**. No scheduler enters the repo.
|
||||
- **Freshness as a persisted seen-log / dedup-vs-seen (B4)** — **R3 later**.
|
||||
- **Brief history surfacing / diff ("yesterday vs today", hull 7)** — **R3 later**.
|
||||
- **Research-deepening A1–A4** (plan → isolated parallel workers → gap loop → curate), adapter sub-agents, MCP
|
||||
fetch fan-out — **R3 later** (the big slice).
|
||||
- **A new `score` field in the `add` manual CLI path** — OUT. `add` stays the raw, score-free manual path; only
|
||||
the normalizing `capture` path carries scores.
|
||||
- **`BRIEF_SCHEMA_VERSION` bump** — OUT by default (no frontmatter field changes); Open Q#5.
|
||||
- **New source file / new agent / new command** — none. R3a is all edits to the six existing `src/*.ts` +
|
||||
one agent + README + gate. Counts stay 27/19/29.
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **TDD iron law (two-phase RED):** the failing tests land **BEFORE** the implementation. `store`/`brief`/`cli`
|
||||
tests are true logic-RED against the pre-edit code (inline fixtures, no new import). `score`/`item` tests
|
||||
reference new `score.ts` exports → under Node16 ESM a missing named import throws at module-load, so they are
|
||||
RED against **non-throwing stubs** landed first (the stubs return wrong-but-present values; the value
|
||||
assertions then fail). The plan records the RED proof in two phases (Step 1); it does NOT claim a single
|
||||
"all five fail on assertion before any code" run.
|
||||
- **One composite owner:** `composite()` + `band()` (`score.ts`) stay the sole arithmetic; `scoreEnvelope`
|
||||
*composes* them, never re-derives. The agent supplies judgment, the code computes the composite (SSOT
|
||||
discipline, `score.test.ts:12-30` pins the weights/bands).
|
||||
- **Purity:** `scoreEnvelope`/`requiredDimensions`/`rankForBrief`/`renderBrief` touch no fs, no clock, no env,
|
||||
no AI. All fs stays at the CLI edge.
|
||||
- **No throw on the capture path (not "everywhere"):** `normalizeItem` fully validates the score before
|
||||
`itemToInput`, so the capture loop (`cli.ts:246-258`) never reaches `composite` with bad dims and never
|
||||
crashes (a bad score → `errors[]`). `itemToInput`/`scoreEnvelope`/`composite` called **directly** with bad
|
||||
dims throw by contract — that is the defense-in-depth boundary, asserted (SC2), not a leak.
|
||||
- **Determinism:** same `(store, pillars, today, freshDays)` → byte-identical brief (the composite sort is a
|
||||
total order via the unique `(title,url)` final tie-break; `-1` sentinel for unscored is deterministic).
|
||||
- **Lossless additive migration (both directions):** a v2 store loads as v3 with records **untouched** (no
|
||||
`score` invented); round-trip writes `schemaVersion: 3`; a v3 store is idempotent; a v3 store's new optional
|
||||
`score` field **survives a load+resave** (no field stripping, `JSON.stringify` `store.ts:95`). Mirrors the R2a
|
||||
v1→v2 proof (`store.test.ts:403-476`) + a new field-preservation case.
|
||||
- **Hook unaffected:** the SessionStart surfacing reads `date`+`summary` only and **never shells out to tsx**
|
||||
(analytics fresh-clone-crash invariant) — R3a touches neither the hook nor the frontmatter schema, so the
|
||||
zero-tsx surfacing is unchanged. (No hook test added; the existing hook suite must still pass untouched as a
|
||||
regression sanity.)
|
||||
- **Domain-general:** Section 17 de-niche stays green; the `trend-spotter.md` edit carries the rubric's
|
||||
dimension names + the user's pillars, **no vendor/sector tokens**.
|
||||
- **No SSOT change:** `references/trend-scoring-modes.md` (weights/bands/actions) untouched; `score.ts` mirrors
|
||||
it exactly as today.
|
||||
- **No store-query change:** `queryByTopic`/`history`/`newestCaptureDate` untouched; the brief recomputes
|
||||
overlap as before (`queryByTopic` NOT refactored).
|
||||
- **Pathguard:** all edits are to **existing** files (no new `.mjs` under `hooks/scripts/`; no new `.ts` —
|
||||
R3a adds *no* source file). `.gitignore` already covers `scripts/trends/{node_modules,build}`.
|
||||
- **Counts** (refs/agents/commands 27/19/29) unchanged. **Recounted live at land**, never pinned/guessed.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1 (score envelope)** — `requiredDimensions("kortform")` **deep-equals (ordered)** `["pillar","audience",
|
||||
"timing","angle","authority"]`; `requiredDimensions("long-form")` deep-equals `["pillar","depth","angle",
|
||||
"authority","currency"]` (the `WEIGHTS` literal order, `score.ts:20-35`), and `score.test` pins the order so a
|
||||
SSOT reorder fails. `scoreEnvelope("kortform", {pillar:8,audience:7,timing:9,angle:6,authority:5})` returns
|
||||
`{ mode:"kortform", dimensions:<the five>, composite: composite(dims,"kortform"), priority: band(composite).
|
||||
priority }` — composite/priority equal the existing functions' output byte-for-byte (one owner); a bad
|
||||
dimension makes `scoreEnvelope` throw (via `composite`).
|
||||
- **SC2 (item validation + bridge + the throw contract)** — `normalizeItem` on an item with a valid `score`
|
||||
carries the **validated** dims; with a bad `mode`, a missing dimension, a dimension out of [1,10], a non-object
|
||||
`score`, or an **array** `dimensions` → `{ ok:false, errors:["invalid score: …"] }` (structured, **never
|
||||
throws**); absent `score` → key omitted. `itemToInput(validItemWithScore, capturedAt)` returns a `TrendInput`
|
||||
whose `score` is `scoreEnvelope(mode, dimensions)` (composite/priority computed); without a score → no `score`
|
||||
key; **`itemToInput` called directly with an out-of-range dim throws** (the defense-in-depth contract).
|
||||
- **SC3 (first-sight persist)** — `addTrend(store, inputWithScore)` on a **new** title+url persists `score` on
|
||||
the record; re-`addTrend` of the same title+url with a **different** score does **NOT** change the stored
|
||||
score (first-sight, D3) while topics still union; an input **without** a score adds a score-free record.
|
||||
- **SC4 (migration v2→v3, both directions)** — a `schemaVersion:2` store with records lacking `score` loads as
|
||||
**v3**, records intact, **no `score` invented**; round-trip `loadStore→saveStore` writes `schemaVersion:3`; a
|
||||
v3 store with `score` on records loads idempotent; **a v3 store's `score` field survives load+resave** (field
|
||||
preservation of a new optional field — not covered by the mirrored v1→v2 block). Mirrors
|
||||
`store.test.ts:403-476`, **retitled `(RE-R3a / score v2→v3)` with every `schemaVersion` assertion literal
|
||||
flipped `2`→`3`.**
|
||||
- **SC5 (brief ranks on composite)** — within a bucket, `rankForBrief` orders **composite desc** first
|
||||
(a composite-9 record ahead of a composite-6 record at the **same overlap**); an **unscored** record sorts
|
||||
**after** every scored record in its bucket (the `-1` sentinel) and then by the existing keys; buckets are
|
||||
unchanged (still overlap+freshness); the order is a **total order** (same-title/diff-url, both unscored →
|
||||
fixed by `url asc`); same input → byte-identical brief (determinism).
|
||||
- **SC6 (render surfaces band + mode)** — `renderBrief` emits the **full pinned line shapes** (§3): a scored
|
||||
top-entry shows `· <priority> (<mode>)` between `(<age>d)` and `· Pillarer`; a scored bullet shows `·
|
||||
<priority> (<mode>)` before `· 🔗`; an **unscored** entry renders the **unchanged** line (no token) — both
|
||||
asserted as **full lines, not substrings**. `briefSummary` names the band (no mode) on a scored top, omits the
|
||||
token on an unscored top, and stays one line with no `"`/`\n` **even when the top title contains a guillemet/
|
||||
quote** (the only new code path touching the summary). The `ranking:` frontmatter descriptor equals the pinned
|
||||
string verbatim. A store whose only fresh match is a **single-pillar unscored** record → `briefSummary` renders
|
||||
with no `· <priority>` token, one line.
|
||||
- **SC7 (CLI persists score end-to-end)** — `echo '[{…,"score":{"mode":"kortform","dimensions":{…valid…}}}]'
|
||||
| … capture --store <tmp>` then `… list --store <tmp> --json` shows the record carrying a `score` with the
|
||||
computed composite/priority; a batch with one **bad** score → that item in `errors[]`, the valid ones added,
|
||||
**exit 0** (the run isn't failed).
|
||||
- **SC8 (gate + wiring + de-niche)** — `bash scripts/test-runner.sh` → `FAIL=0`: trends suite green at the
|
||||
bumped `TRENDS_TESTS_FLOOR`; new **Section 16j** green (`TrendScore` in `score.ts`, `score?: TrendScore` in
|
||||
`types.ts`, `"dimensions"` in `trend-spotter.md`, `score?.composite` in `brief.ts`, non-vacuity self-test);
|
||||
`ASSERT_BASELINE_FLOOR` = **99** (94 + 5); Section 17 de-niche green; counts 27/19/29.
|
||||
|
||||
## 7. Verification
|
||||
|
||||
**Deterministic:** `bash scripts/test-runner.sh` → `FAIL=0`; trends suite ≥ new floor; new Section 16j
|
||||
self-test + greps pass; `ASSERT_BASELINE_FLOOR` = 99; Section 17 de-niche green; ref/agent/command counts
|
||||
unchanged. **Regression sanity:** `node --test hooks/scripts/__tests__/` → still green untouched (R3a touches no
|
||||
hook; adds no hook test).
|
||||
**Behavioural (manual):**
|
||||
1. `echo '[{"source":"tavily","title":"A","url":"https://e/a","topics":["ai","gov"],"publishedAt":"<~2d ago>",
|
||||
"score":{"mode":"kortform","dimensions":{"pillar":9,"audience":8,"timing":9,"angle":7,"authority":6}}},
|
||||
{"source":"tavily","title":"B","url":"https://e/b","topics":["ai","gov"],"publishedAt":"<~2d ago>",
|
||||
"score":{"mode":"kortform","dimensions":{"pillar":6,"audience":5,"timing":6,"angle":5,"authority":5}}}]'
|
||||
| node --import tsx src/cli.ts capture --store /tmp/r3a.json` — both overlap-2 & fresh, A scored higher.
|
||||
2. `node --import tsx src/cli.ts list --store /tmp/r3a.json --json` → confirm both records carry `score`
|
||||
with computed composite/priority.
|
||||
3. `node --import tsx src/cli.ts brief --pillars ai,gov --store /tmp/r3a.json --out /tmp/r3a-brief --json` →
|
||||
confirm **A precedes B** in `topMatches` (higher composite, same overlap+freshness), the entry line shows
|
||||
`· <priority> (kortform)`, and the `summary` names A with its band.
|
||||
4. Append a bad-score item (`"timing":99`) to the batch and re-`capture` → confirm it lands in `errors[]`,
|
||||
the valid items still added, exit 0.
|
||||
|
||||
## 8. Open questions for the go-gate — RESOLVED
|
||||
|
||||
D1–D4 confirmed by the operator ("Go", 2026-06-24): **D1** 4-field `TrendScore`; **D2** composite primary within
|
||||
bucket; **D3** first-sight; **D4** one slice (data-then-visible commit order within it). Two residual cosmetics,
|
||||
both baked to the recommended default:
|
||||
- **D5 — `BRIEF_SCHEMA_VERSION` 1→2?** No (no frontmatter field added/removed; the hook reads only
|
||||
`date`+`summary`). Re-open only if the artifact should self-announce the ranking change.
|
||||
- **D6 — mode in the per-entry render?** YES (folded from plan-critic #3): the body entry shows `<priority>
|
||||
(<mode>)`; the summary shows the band only. This makes the mode-blind ranking honest (the reader can see the
|
||||
instrument).
|
||||
|
||||
## 9. Light-Voyage review — folded
|
||||
|
||||
Three Opus reviewers ran on the drafts, each verifying claims against live code. **scope-guardian: ALIGNED**
|
||||
(every SC1–SC8 traces to a step; zero creep; all §4 non-goals held; counts 27/19/29 verified live; "no new
|
||||
source file" verified — exactly 6 `src/*.ts` + 5 `tests/*.test.ts`, all edited, none added; 0 findings).
|
||||
**brief-reviewer: PROCEED_WITH_RISKS** (all four load-bearing claims — score.ts-is-a-leaf/no-cycle, composite ≥
|
||||
1.0, version-stamp-only migration, single-owner arithmetic — verified TRUE; 6 MINOR). **plan-critic: REVISE**
|
||||
(1 BLOCKER, 4 MAJOR, 4 MINOR). All findings folded; see `plan-re-r3a.md §Plan-critic — folded` for per-step
|
||||
resolution. Headlines:
|
||||
|
||||
- **[BLOCKER, folded]** the "all five test files fail on assertion after Step 1" RED claim is false for
|
||||
`score`/`item` under Node16 ESM (a missing named import throws at module-load, not on assertion). → RED is now
|
||||
**explicitly two-phase**: logic-RED for `store`/`brief`/`cli` against pre-edit code; stub-first then
|
||||
assertion-RED for `score`/`item` (§5; plan Step 1; the header blockquote).
|
||||
- **[MAJOR, folded]** the no-throw guarantee was overstated ("unreachable" — but `itemToInput` is public and
|
||||
throws on direct bad-dim calls). → reworded **path-specific** (no throw on the capture path; direct calls throw
|
||||
by contract); SC2 asserts both (§5, §6).
|
||||
- **[MAJOR, folded]** mode-mixing was waved away and "mode shown per entry" contradicted the render spec (which
|
||||
only showed priority). → the render now shows `<priority> (<mode>)` per body entry (D6); §4 states mode-blind
|
||||
ranking is accepted for R3a with the mode visible; SC6 asserts the full line incl. mode.
|
||||
- **[MAJOR, folded]** `requiredDimensions` order contract was ambiguous (SC1 hard-coded arrays vs membership
|
||||
use). → pinned **ordered** (SC1 deep-equals the SSOT-order array; `score.test` pins order; `normalizeItem` uses
|
||||
membership) (§3 S-score, SC1).
|
||||
- **[MAJOR, folded]** `ASSERT_BASELINE_FLOOR` "~99" was not pinned. → pinned **99** (94 + 5 unconditional 16j
|
||||
emitters; self-test emits one pass/fail like 16i) (§3 wiring, SC8).
|
||||
- **[MINOR, folded]** SC4 ref `:403-471` stale + pointed at v2 assertions → `:403-476` + "flip every
|
||||
`schemaVersion` literal 2→3" note (SC4). **[MINOR, folded]** R1 SSOT-pin cite was the doc-comment → now
|
||||
`score.test.ts:12-30` (§2, §5; plan R1). **[MINOR, folded]** bullet `· <priority>` placement was substring-only
|
||||
→ full pinned line shape, priority+mode before `🔗`, asserted as a full line (§3, SC6). **[MINOR, folded]**
|
||||
three diverging `ranking:` descriptor strings → one verbatim target, asserted byte-for-byte (§3, SC6). **[MINOR,
|
||||
folded]** unscored single-match-top summary path untested → added as an SC6 case. **[MINOR, folded]**
|
||||
`normalizeItem` non-array object case understated → "non-array" added to both object checks + SC2. **[MINOR,
|
||||
folded]** header-chain line-ref tightened to the 16i clause `:46-49` / Section-18 `:49`. **[MINOR, folded]** R9
|
||||
DAG now lists the three new one-way `score.ts ←` edges. **[MINOR, folded]** SC4 forward-compat /
|
||||
score-survives-round-trip added. **[MINOR, folded]** SC6 quote-safety regression (scored top title with a
|
||||
guillemet) added.
|
||||
416
docs/research-engine/brief-re-r3b.md
Normal file
416
docs/research-engine/brief-re-r3b.md
Normal file
|
|
@ -0,0 +1,416 @@
|
|||
# Brief — RE-R3b: trend lifecycle — re-score on re-capture · status (acted/skipped) · seen-log
|
||||
|
||||
> **Slice:** RE-R3b (research-engine rung-2, R3 slice 2 — the **lifecycle** slice: what happens to a trend
|
||||
> AFTER first capture). R3 ("deepen the research engine") is an **arc** of 5 open hulls (substrate §1). R3a took
|
||||
> the **relevance** half of hull 5 (persist the score, rank on it). R3b takes the rest of the *lifecycle* of a
|
||||
> trend: **(i) re-score on re-capture** (R3a's explicit deferral — hull 3 remainder), **(ii) a status lifecycle**
|
||||
> `new`/`acted`/`skipped` (hull 5), and **(iii) a seen-log** — `surfacedCount`/`lastSurfacedAt` accumulated on
|
||||
> each record as the temporal foundation slices (c)+(b) build on (hull 5, B4 dedup-state).
|
||||
> **Predecessor:** RE-R3a (`score?: TrendScore` persisted first-sight; `rankForBrief` orders on composite;
|
||||
> `renderBrief` surfaces band+mode) + RE-R2b (`brief.ts` dated artifact + surfacing) + RE-R2a (`capture` bridge).
|
||||
> R3a §4 deferred this exactly: *"Re-score on re-capture … R3b. R3a is first-sight only (D3). Re-score pairs
|
||||
> naturally with the seen-log/status slice."* — R3b is that paired slice.
|
||||
> **Substrate:** `docs/research-engine-concepts.local.md` §1 hull (5) (status/lifecycle: acted/skipped) +
|
||||
> remainder of (3) (status as a schema field) + §B4 (*"freshness window + dedup-state (append-only seen-log →
|
||||
> don't re-surface the same item)"*). The freshness window already exists (`freshDays`, R2b); R3b adds the
|
||||
> dedup-state (status as the hard dedup; surfacedCount as the soft signal).
|
||||
> **TDD-order:** RED before code, **two phases** (light-Voyage BLOCKER fold, inherited from R3a): the re-score +
|
||||
> migration parts of `store.test`, all of `brief.test`, and `cli.test` are true logic-RED against the pre-edit
|
||||
> code (inline fixtures / behaviour change / subprocess — no new import); the `setStatus`/`markSurfaced`/
|
||||
> `effectiveStatus` tests reference not-yet-existing `store.ts` exports, so under Node16 ESM a missing named
|
||||
> import throws at module-load (not on assertion) — they are RED against **non-throwing stubs** landed first. See
|
||||
> plan Step 1.
|
||||
|
||||
## 1. Operator decision context (2026-06-25)
|
||||
|
||||
The research engine is **Tier-1** (operator, 2026-06-23). R1→R3a built the deterministic spine: item-schema +
|
||||
triage scorer (R1) → capture bridge (R2a) → dated morning brief + surfacing (R2b) → persisted relevance score +
|
||||
composite ranking (R3a). What the spine still lacks is **memory of a trend's life after first sight**: the score
|
||||
is frozen at first capture even as timing decays; a trend the operator already wrote about (or deliberately
|
||||
passed on) **re-tops tomorrow's brief unchanged**; and nothing records that a trend has been *surfaced* N times
|
||||
without action. The morning brief is meant to be a **work queue**, but today it is amnesiac — it cannot tell a
|
||||
fresh unhandled signal from one the operator dealt with yesterday.
|
||||
|
||||
R3b closes that gap with the **lifecycle layer** the operator chose as slice (a) of the full-R3 build-out
|
||||
(2026-06-24, *"ALLE gjenstående R3-slices … i rekkefølge (a) → (c) → (b) → (d) → (e)"*). It is **the fundament
|
||||
for everything temporal**: the autonomous trigger (c) must *never automate a loop that re-surfaces handled
|
||||
items* — so it depends on (a)'s status+seen-log; saturation/first-mover (b) is *only meaningful with accumulated
|
||||
seen-data* — which (a) starts accumulating. R3b is deliberately first in the sequence: correctness of the
|
||||
lifecycle model before any automation reads it.
|
||||
|
||||
**Architectural decisions — CONFIRMED (operator, AskUserQuestion 2026-06-25; baked into the plan):**
|
||||
- **A1 — seen-log form = on-record + the brief records surfacing.** Three new optional fields on `TrendRecord`
|
||||
(`status`, `surfacedCount`, `lastSurfacedAt`); the `brief` CLI, **after** the pure `rankForBrief` computes the
|
||||
ranking, records surfacing on the rendered trends and re-saves the store. `rankForBrief` stays **pure**
|
||||
(mutation only at the CLI edge). The store stays the **single source of truth** — no separate `seen-items.md`.
|
||||
A `--no-mark` flag gives a side-effect-free dry run. *(This is exactly what slice (c) will automate and slice
|
||||
(b) will read.)*
|
||||
- **A2 — re-score on re-capture = last-score-wins.** On a duplicate capture carrying a fresh `score`, the stored
|
||||
`score` is **replaced** by the freshly-computed envelope (composite re-derived by the one owner,
|
||||
`composite()`+`band()`). `score` becomes **the one deliberately-mutable field**; provenance (`source`,
|
||||
`capturedAt`, first `publishedAt`) stays first-sight. A re-score **does NOT reset status** — an `acted`/`skipped`
|
||||
decision sticks. *(Rationale: the Timing dimension decays, so the newer judgment — even a lower one — is the
|
||||
truer one; monotone "only if higher" would freeze stale optimism.)*
|
||||
- **A3 — acted/skipped are EXCLUDED from the brief.** `rankForBrief` drops every record whose effective status
|
||||
is not `new` from all three buckets — the brief is a work queue, not an archive. Full history stays available
|
||||
via `list`/`query`.
|
||||
|
||||
## 2. The gap — grounded in code
|
||||
|
||||
- **The score is frozen at first sight, even as timing decays.** `addTrend`'s duplicate branch
|
||||
(`store.ts:127-131`) unions topics and returns — it **never touches `score`** (R3a's D3, first-sight only).
|
||||
The capture path already carries a fresh score on every re-capture (`item.ts:192` `itemToInput` →
|
||||
`scoreEnvelope`; `cli.ts:257` folds it through `addTrend`), so the fresh judgment **reaches `addTrend` and is
|
||||
silently discarded** for any trend already in the store. A trend re-polled a week later still ranks on its
|
||||
week-old Timing score.
|
||||
- **A handled trend re-tops the brief unchanged.** `rankForBrief` (`brief.ts:82-92`) iterates **every** store
|
||||
record, dropping only off-pillar ones (`overlap === 0`, `:89`). There is no notion of "I already wrote about
|
||||
this" — an `acted` trend with a high composite re-sorts to the top of `topMatches` tomorrow exactly as it did
|
||||
today. `TrendRecord` has **no `status` field** (`types.ts:29-59`); the doc-comment anticipates it: *"can gain
|
||||
fields (…, status) in a later slice"* (`types.ts:22`).
|
||||
- **Nothing records that a trend has been surfaced.** The brief is a **pure read** (`brief.ts:1-15`: *"No fs, no
|
||||
clock, no AI"*); generating it leaves no trace on the store. There is no `surfacedCount`/`lastSurfacedAt` —
|
||||
so a future autonomous loop (slice c) has **no way to know** a trend was already shown, and saturation (slice b)
|
||||
has **no accumulated signal** to read. B4's dedup-state (`docs/research-engine-concepts.local.md:63`) does not
|
||||
exist yet.
|
||||
- **The CLI has no lifecycle verbs.** `cli.ts` exposes `add`/`query`/`list`/`status`/`normalize`/`score`/
|
||||
`capture`/`brief` (`:5-13`) — all capture/read. There is **no way for the operator to mark** a trend `acted`
|
||||
or `skipped`.
|
||||
|
||||
## 3. Scope — what is IN (RE-R3b)
|
||||
|
||||
### S-types — `scripts/trends/src/types.ts` (EDIT)
|
||||
- **`export type TrendStatus = "new" | "acted" | "skipped";`** — the lifecycle states.
|
||||
- `TrendRecord` gains **three optional fields** (all absent on pre-R3b records, all additive):
|
||||
- **`status?: TrendStatus;`** — lifecycle. **Absent ⇒ `"new"`** (back-compat); set only by `act`/`skip`/`reset`,
|
||||
**never on capture** (a freshly-captured trend is implicitly `new`).
|
||||
- **`surfacedCount?: number;`** — the seen-log count: how many distinct days this trend has appeared in a
|
||||
generated brief. **Absent ⇒ 0.** Incremented (per-day-idempotent) by the `brief` CLI.
|
||||
- **`lastSurfacedAt?: string;`** — ISO date of the most recent surfacing. **Absent ⇒ never.** The per-day
|
||||
idempotency key (re-running today's brief does not re-increment).
|
||||
- Doc-comment: mark `status`/`surfacedCount`/`lastSurfacedAt` as the now-realized lifecycle fields the `:22`
|
||||
note anticipated.
|
||||
- **`SCHEMA_VERSION = 3 → 4`** (`types.ts:73`). Additive-optional; the migration is the version-stamp alone
|
||||
(below), identical to v1→v2→v3.
|
||||
|
||||
### S-store — `scripts/trends/src/store.ts` (EDIT)
|
||||
- **`export function effectiveStatus(t: TrendRecord): TrendStatus`** — `return t.status ?? "new";`. The single
|
||||
reader of the absent-⇒-new convention (pure; consumed by `addTrend` audit, `brief`, and the CLI). Imports
|
||||
`TrendStatus` from `./types.js` (type-only).
|
||||
- **Re-score in `addTrend`'s duplicate branch (`:127-131`, A2):** after the topic union, if `input.score !==
|
||||
undefined` **and it differs from `existing.score`** (compared via `JSON.stringify` — the envelope is built in a
|
||||
fixed key order by `scoreEnvelope`, so the compare is stable), set `existing.score = input.score` and mark the
|
||||
record changed. `AddResult.merged` is **broadened** to *"the existing record was mutated — topics unioned and/or
|
||||
score refreshed"*; `merged` is true iff **either** changed (a re-capture with an identical score → `merged:false`,
|
||||
no false-positive). `status`/`surfacedCount`/`lastSurfacedAt` are **NOT touched** on re-capture (A2: re-score
|
||||
doesn't reset status; surfacing is the brief's job, not capture's). The **new-record** branch (`:132-144`) is
|
||||
unchanged — a new record omits all three lifecycle fields (status absent ⇒ new; never surfaced; no input.status
|
||||
exists on the capture path).
|
||||
- **`export function setStatus(store: TrendStore, id: string, status: TrendStatus): { store: TrendStore; found:
|
||||
boolean }`** — find the record by `id`; if absent return `{ store, found: false }` (no throw); else set
|
||||
`t.status = status` (set **explicitly**, including `"new"` for a `reset`) and return `{ store, found: true }`.
|
||||
Mutates in place + returns the same store (the `addTrend` idiom). Pure (no fs).
|
||||
- **`export function markSurfaced(store: TrendStore, ids: string[], today: string): { store: TrendStore; marked:
|
||||
number }`** — for each record whose `id` is in `ids` **and** whose `lastSurfacedAt !== today` (per-day
|
||||
idempotent), set `surfacedCount = (surfacedCount ?? 0) + 1` and `lastSurfacedAt = today`; count it. Records
|
||||
already surfaced today, or not in `ids`, are untouched. Pure (no fs; `today` injected by the caller, like
|
||||
`capturedAt`). Returns the count actually incremented.
|
||||
- `AddResult` keeps its **2-flag shape** `{ store, added, merged }` (no new flag — `merged` is broadened, not
|
||||
joined). `TrendInput` is **unchanged** (no `status`/`surfaced*` input — lifecycle is set post-capture, not
|
||||
ingested).
|
||||
- `loadStore` migrate comment (`:82-88`): extend the enumeration to *"v1→v2→v3→v4 are all purely
|
||||
additive-optional"*. **No code change** (`Math.max(onDisk, SCHEMA_VERSION)` `:91` already stamps v4;
|
||||
`saveStore` `JSON.stringify` `:99` preserves the three new fields). Only `SCHEMA_VERSION` (in `types.ts`) and
|
||||
the comment move.
|
||||
|
||||
### S-brief — `scripts/trends/src/brief.ts` (EDIT)
|
||||
- **`rankForBrief` excludes handled trends (A3):** in the entry loop (`:82-92`), add **`if (effectiveStatus(trend)
|
||||
!== "new") continue;`** immediately before the `overlap === 0` check (so acted/skipped never enter any bucket).
|
||||
Import `effectiveStatus` from `./store.js` (brief.ts already imports `defaultStorePath` from there — `:19`; the
|
||||
edge stays one-way, no cycle). `totals.trends` **still counts the full inventory** (`store.trends.length`,
|
||||
`:116`) — honest "of N in store"; `totals.matched`/`fresh` naturally reflect the post-filter `entries`.
|
||||
- **`renderBrief`/`renderTopEntry`/`renderBulletEntry` surface the trend `id` + a surfaced marker** (so the
|
||||
operator can act on an entry, and a re-surfaced item is honest). **Pinned line shapes:**
|
||||
- A shared **`surfacedToken(e)`** helper (mirrors `scoreToken`, `:142-145`): ` · sett <surfacedCount>x` when
|
||||
`surfacedCount >= 2`, else `""` (only a genuinely re-surfaced item is flagged; this is a saturation **hint**,
|
||||
not the saturation **scoring** of slice b). **Semantic (folded — plan-critic #3): the count is PRIOR-DAY** —
|
||||
the brief renders from `surfacedCount` **before** the CLI records today's surfacing (the mutation runs after
|
||||
`renderBrief`), so `· sett Nx` means *"shown on N prior distinct days"* (today's appearance is recorded but
|
||||
not yet counted in this render). The `>= 2` floor therefore means "already shown on ≥2 earlier days". This is
|
||||
documented in the README + asserted by a unit test that sets `surfacedCount` directly (the cross-day behaviour
|
||||
is exercised by behavioural step §7).
|
||||
- Top-entry meta line (`renderTopEntry`, `:150`): append **` · \`<id>\``** at the end (after `Pillarer: …`),
|
||||
and `surfacedToken(e)` after the `scoreToken`:
|
||||
`- Kilde: <source> · Publisert: <date> (<age>d)<scoreToken><surfacedToken> · Pillarer: <matched> · \`<id>\``
|
||||
- Bullet line (`renderBulletEntry`, `:159`): append **` · \`<id>\``** at the end (after `🔗 <url>`), with
|
||||
`surfacedToken` after `scoreToken`:
|
||||
`- **<title>** — «<matched>» · <date> (<age>d)<scoreToken><surfacedToken> · 🔗 <url> · \`<id>\``
|
||||
- The id is rendered in backticks so it is copy-paste-ready for `act --id <id>` / `skip --id <id>`.
|
||||
- **`export function surfacedIds(ranking: BriefRanking): string[]`** — the ids of the entries `renderBrief`
|
||||
**actually shows**: `topMatches ∪ singleMatches ∪ olderMatched.slice(0, 5)` (mirrors the `:199` `.slice(0, 5)`
|
||||
older cap), mapped to `e.trend.id`. The CLI feeds this to `markSurfaced` so the seen-log records exactly what
|
||||
the operator saw. Pure.
|
||||
- **`ranking:` frontmatter descriptor (`:175`)** → the **exact** string
|
||||
`composite desc, then pillar-overlap desc, then publishedAt desc (capturedAt fallback); freshDays <N>; excludes
|
||||
acted/skipped` (pinned verbatim; `brief.test` asserts byte-for-byte). The trailing `; excludes acted/skipped`
|
||||
is the only descriptor change.
|
||||
- `briefSummary` (`:129-139`) is **unchanged** (the headline still names the top fresh match's band + age; status
|
||||
exclusion happens upstream in the ranking, so the summary already reflects only `new` trends). `BRIEF_SCHEMA_
|
||||
VERSION` stays **1** (no frontmatter *field* added/removed — `date`/`summary`/`store`/`ranking`/`schemaVersion`
|
||||
unchanged; only the `ranking:` *string* and body content change; the surfacing hook still reads `date`+`summary`).
|
||||
|
||||
### S-cli — `scripts/trends/src/cli.ts` (EDIT)
|
||||
- **`act` / `skip` / `reset` subcommands** (set lifecycle status by id):
|
||||
- `act --id <id> [--store <path>]` → `setStatus(store, id, "acted")`; `skip …` → `"skipped"`; `reset …` →
|
||||
`"new"`. Each: load → setStatus → if `found` save + print `Marked <id> <status>` (exit 0); if **not found**
|
||||
print `error: no trend with id: <id>` to stderr + **exit 2**. A missing/`true` `--id` → `usage('<cmd> needs
|
||||
--id <id>')` (exit 2). **Exit-code contract broadened (folded — plan-critic #2):** a not-found id is exit 2,
|
||||
which the existing contract documents as "usage error". Update the header doc-comment (`cli.ts:33`) to read
|
||||
*"0 on success, 2 on usage error or a not-found id (act/skip/reset)"* — a wrong `--id` value is an
|
||||
argument-class error, distinct from `capture`'s data-stream items (which stay in `errors[]`, never the exit
|
||||
code). A new exit code is **not** introduced (the CLI keeps its two codes).
|
||||
- **`brief` records surfacing (A1):** **hoist the load** (folded — plan-critic #1 / brief-reviewer #1): replace
|
||||
the inline `rankForBrief(loadStore(storePath), …)` (`cli.ts:286`) with **`const store = loadStore(storePath);
|
||||
const ranking = rankForBrief(store, pillars, day, { freshDays });`** — `cli.ts:286` does **not** currently bind
|
||||
a `store` variable (verified), so the surfacing edit needs this hoist or it references an undefined identifier.
|
||||
Then after `writeFileSync(path, md, …)` (`:290`), **unless `--no-mark`**: `markSurfaced(store, surfacedIds
|
||||
(ranking), day)` then `saveStore(storePath, store)` — the **hoisted `store`** holds the full inventory, so
|
||||
acted/skipped records (filtered from the ranking but still in the store) are preserved on resave; the `.md` is
|
||||
rendered from the pure `ranking` **before** the mutation. `const mark = flags["no-mark"] !== "true";` (a bare
|
||||
`--no-mark` → `"true"` → mark off). The `--json` output gains a **`marked`** count (trends whose seen-log this
|
||||
run incremented; `0` when `--no-mark`). `rankForBrief`/`renderBrief` are untouched — the mutation is purely at
|
||||
the edge.
|
||||
- **`capture` tally comment (`cli.ts:251-252`)** (folded — plan-critic #4): the broadened `AddResult.merged`
|
||||
(topics ∪ score-refresh) makes the existing comment *"a fold is … `merged` (existing gained topics)"* stale →
|
||||
update it to *"`merged` (existing gained topics and/or a refreshed score)"*. No tally-logic change (the loop
|
||||
already counts `res.merged`).
|
||||
- **Usage + header doc:** add the three new verbs + `[--no-mark]` to the `usage()` block (`:82-91`) and the
|
||||
header synopsis (`:5-13`); a one-line header note that `act`/`skip`/`reset` set a trend's lifecycle status, the
|
||||
brief excludes handled trends and records surfacing, and re-capture refreshes the score.
|
||||
|
||||
### Wiring (D-default — WIRE, mirrors R3a)
|
||||
- `agents/trend-spotter.md` (EDIT, **prose-only, minimal**): Step 4.5 already emits the per-item `score` (R3a);
|
||||
re-score is **automatic** (capture re-folds an existing trend with a fresh score → `addTrend` now refreshes it),
|
||||
so **no batch-shape change**. Add one prose line: re-capturing a known trend now **refreshes** its relevance
|
||||
score (timing decays), and the operator marks trends `acted`/`skipped` via the CLI so the brief stops
|
||||
re-surfacing handled work. Domain-general (no vendor/sector tokens). *(If a Section-16k grep targets the agent,
|
||||
it must be verified non-vacuous first; the recommended 16k greps target src files only — see gate below.)*
|
||||
- `scripts/trends/README.md` (EDIT): document the status lifecycle (`new`/`acted`/`skipped` + `act`/`skip`/`reset`),
|
||||
the seen-log (`surfacedCount`/`lastSurfacedAt`, per-day idempotent, brief-recorded), re-score-on-recapture
|
||||
(last-wins), and the brief's exclude-handled behaviour + `--no-mark`.
|
||||
- `scripts/test-runner.sh` (EDIT): bump `TRENDS_TESTS_FLOOR` (`:705`, currently 146) to the `tests N` line
|
||||
reported after Steps 1–6, **append** `+ RE-R3b: lifecycle +N` to the inline breakdown comment. Add
|
||||
**Section 16k** ("Trends Lifecycle Wiring", RE-R3b) **after Section 16j's closing block, before Section 18**
|
||||
(16j is the last 16x before the anti-erosion Section 18; preserve that order). Mirror 16j's shape:
|
||||
**unconditional**, deps-absent-safe (`grep -qF` + a non-vacuity self-test emitting **one** pass/fail). Recommended
|
||||
**6 emitters** (all on tracked src — no `tsx`): (1) self-test; (2) `export type TrendStatus` in `types.ts`;
|
||||
(3) `surfacedCount` in `types.ts` (seen-log field); (4) `export function markSurfaced` in `store.ts` (seen-log
|
||||
writer); (5) `effectiveStatus` in `brief.ts` (the brief excludes handled); (6) `command === "act"` in `cli.ts`
|
||||
(the lifecycle verb). **6 unconditional emitters → bump `ASSERT_BASELINE_FLOOR` 99 → exactly 105** (`:1259`;
|
||||
"live recount" is the safety net; the expected value is the pinned 99 + 6 = 105). Update the header-enumeration
|
||||
prose chain by inserting the 16k clause between the 16j clause and the Section-18 clause.
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **Saturation scoring / first-mover-as-a-field** (the quantitative *use* of `surfacedCount`) — **slice (b)**.
|
||||
R3b **accumulates** the seen-log and shows a minimal `· sett Nx` hint, but it does **not** compute a saturation
|
||||
score, decay the composite by surfacings, or add a first-mover field. (b) reads R3b's accumulated data.
|
||||
- **Autonomous nightly trigger** (cron/launchd, headless entry — hull 1+6) — **slice (c)**. R3b adds no scheduler;
|
||||
it builds the lifecycle (c) will safely automate.
|
||||
- **Brief history surfacing / diff** ("what's new since yesterday" — hull 7) — **slice (d)**. The seen-log records
|
||||
*that* a trend was surfaced; the cross-brief **diff** is (d). R3b's `· sett Nx` is a per-record count, not a
|
||||
day-over-day diff.
|
||||
- **Research-deepening A1–A4** (plan → isolated workers → gap loop → curate) — **slice (e)**, behind the post-(d)
|
||||
re-evaluation gate.
|
||||
- **Mode-segmented ranking / `--mode` filter** — still OUT (R3a non-goal, unchanged).
|
||||
- **Re-score semantics other than last-wins** (monotone / timing-only refresh) — OUT (A2 chose last-wins).
|
||||
- **A `status`/`surfaced*` input on the capture/`add` path** — OUT. Lifecycle is set **post-capture** by
|
||||
`act`/`skip`/`reset`; capture never ingests a status. `TrendInput` is unchanged.
|
||||
- **`act`/`skip` by title/url** (deriving the id) — OUT for R3b; `--id` only (the id is shown in the brief +
|
||||
`list --json`). A title/url alias is a later ergonomic nice-to-have.
|
||||
- **Auto-acting on publish** (wiring `act` into `/linkedin:post` / the post-tracking flow) — OUT. R3b ships the
|
||||
CLI verbs; auto-marking from the content commands is a separate plugin-surface slice.
|
||||
- **`BRIEF_SCHEMA_VERSION` bump** — OUT (no frontmatter field changes); Open Q.
|
||||
- **New source file / new agent / new command** — none. R3b is edits to **four** existing `src/*.ts` (`types`,
|
||||
`store`, `brief`, `cli`) + their tests + one agent (prose) + README + gate. `score.ts` + `item.ts` are
|
||||
**untouched** (re-score reuses the R3a capture path). Counts stay 27/19/29.
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **TDD iron law (two-phase RED):** failing tests land **BEFORE** implementation. Phase A — true logic-RED for
|
||||
the re-score + migration parts of `store.test` (existing `addTrend`/`loadStore`, inline fixtures), all of
|
||||
`brief.test` (behaviour change to existing `rankForBrief`/`renderBrief`), and `cli.test` (subprocess: `act`/`skip`
|
||||
print a usage/unknown-command error today → assertion-RED). Phase B — `setStatus`/`markSurfaced`/`effectiveStatus`
|
||||
reference new `store.ts` exports → land non-throwing stubs first (Node16 ESM throws a missing named import at
|
||||
module-load), then record value-assertion RED against the stubs. The plan does **not** claim a single
|
||||
"everything fails before any code" run.
|
||||
- **`rankForBrief` stays pure (A1):** no fs, no clock, no env, no AI, **no store mutation**. The status filter is
|
||||
a pure read of `effectiveStatus`. The seen-log **write** lives only in the `brief` CLI edge (after the pure
|
||||
ranking), guarded by `--no-mark`. `markSurfaced`/`setStatus`/`effectiveStatus`/`surfacedIds` are all pure.
|
||||
- **One composite owner (unchanged):** re-score reuses the **already-built** capture path
|
||||
(`itemToInput`→`scoreEnvelope`→`composite`+`band`); R3b adds **no new arithmetic** and does not touch `score.ts`.
|
||||
- **Provenance discipline (A2):** `source`, `capturedAt`, and the first `publishedAt` stay **first-sight**; only
|
||||
`score` is mutable on re-capture; `status`/`surfacedCount`/`lastSurfacedAt` are mutated only by their own
|
||||
owners (`setStatus`/`markSurfaced`), never by `addTrend`.
|
||||
- **Per-day-idempotent surfacing:** running `brief` twice on the same `today` increments `surfacedCount` **at most
|
||||
once** (`markSurfaced` skips records whose `lastSurfacedAt === today`). Re-generating today's brief is a no-op on
|
||||
the seen-log. *(This is the determinism guarantee for the autonomous loop: an idempotent daily mark.)*
|
||||
- **No false-merge on re-capture:** a re-capture with a **byte-identical** score → `merged:false` (the
|
||||
`JSON.stringify` compare); only a genuine topic-union or score-change flips `merged`.
|
||||
- **Determinism (brief):** same `(store, pillars, today, freshDays)` → byte-identical `renderBrief` output (the
|
||||
status filter + `surfacedToken` + id are deterministic reads of the store; the composite sort total order from
|
||||
R3a holds). The CLI's surfacing mutation is **outside** the pure render.
|
||||
- **Lossless additive migration (both directions):** a v3 store loads as v4 with records **untouched** (no
|
||||
`status`/`surfaced*` invented); round-trip writes `schemaVersion: 4`; a v4 store is idempotent; the three new
|
||||
optional fields **survive a load+resave**. Mirrors the R3a v2→v3 proof (`store.test.ts`, `(RE-R3a / score
|
||||
v2→v3)` block) with the literals flipped `3`→`4`.
|
||||
- **Hook unaffected:** the SessionStart surfacing reads `date`+`summary` only and **never shells out to tsx**.
|
||||
R3b touches neither the hook nor the frontmatter schema (`BRIEF_SCHEMA_VERSION` stays 1), so surfacing is
|
||||
unchanged. The existing hook suite must still pass untouched (regression sanity; R3b adds no hook test).
|
||||
- **Domain-general:** Section 17 de-niche stays green; the `trend-spotter.md` prose carries only generic
|
||||
lifecycle wording (`acted`/`skipped`/"refresh the score"), no vendor/sector tokens.
|
||||
- **No SSOT change:** `references/trend-scoring-modes.md` untouched (R3b changes no scoring math).
|
||||
- **No store-query change:** `queryByTopic`/`history`/`newestCaptureDate` untouched. *(The CLI `status`
|
||||
subcommand — the staleness reader — is unrelated to the new `TrendStatus` lifecycle type; the name collision is
|
||||
pre-existing and not reconciled here.)*
|
||||
- **Pathguard:** all edits are to **existing** files (no new `.mjs` under `hooks/scripts/`; no new `.ts` — R3b
|
||||
adds no source file). `.gitignore` already covers `scripts/trends/{node_modules,build}`.
|
||||
- **Counts** (refs/agents/commands 27/19/29) unchanged. **Recounted live at land**, never pinned/guessed.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1 (status field + effectiveStatus + setStatus)** — `effectiveStatus({…no status})` is `"new"`;
|
||||
`effectiveStatus({…status:"acted"})` is `"acted"`. `setStatus(store, id, "skipped")` on a present id sets the
|
||||
record's `status` and returns `{ found:true }`; on an absent id returns `{ found:false }` (no throw, store
|
||||
unchanged); a `reset` sets `status:"new"` explicitly.
|
||||
- **SC2 (re-score last-wins, no false-merge, status/provenance untouched)** — `addTrend(store, dupInput)` where
|
||||
`dupInput` has the same title+url and a **different** `score` → the stored `score` is **replaced**, `merged:true`,
|
||||
`added:false`, topics still unioned, and `source`/`capturedAt`/`publishedAt`/`status`/`surfacedCount` are
|
||||
**unchanged**. A re-capture with a **byte-identical** score (and no new topics) → `merged:false`. A duplicate
|
||||
with **no** `score` → stored score unchanged. A re-capture of an **acted** trend with a new score → score
|
||||
updated, **status stays `acted`**. **At the CLI edge (folded — plan-critic #4):** a `capture` of a scored item,
|
||||
then a `capture` of the same title+url with a **changed** score → the second `capture --json` reports
|
||||
`merged:1`, and `list --json` shows the **updated** composite (a subprocess test, not only the manual §7 step).
|
||||
- **SC3 (markSurfaced + per-day idempotency)** — `markSurfaced(store, [idA, idC], "2026-06-25")` increments
|
||||
`surfacedCount` (absent⇒0→1) and sets `lastSurfacedAt:"2026-06-25"` on A and C only (B untouched), returns
|
||||
`marked:2`; a second `markSurfaced` with the **same `today`** → `marked:0`, counts unchanged; a third with a
|
||||
**later** `today` → increments again, `lastSurfacedAt` advances; an id not in the store is silently skipped.
|
||||
- **SC4 (migration v3→v4, both directions)** — a `schemaVersion:3` store with records lacking the lifecycle
|
||||
fields loads as **v4**, records intact, **no field invented**; round-trip `loadStore→saveStore` writes
|
||||
`schemaVersion:4`; a v4 store with lifecycle fields loads idempotent; **the three new fields survive
|
||||
load+resave** (byte-identical). Mirrors the R3a `(RE-R3a / score v2→v3)` block, retitled `(RE-R3b / lifecycle
|
||||
v3→v4)`, every `schemaVersion` literal flipped `3`→`4`.
|
||||
- **SC5 (brief excludes acted/skipped)** — given a store with `new`, `acted`, and `skipped` records all matching
|
||||
pillars + fresh: `rankForBrief` places **only** the `new` ones in `topMatches`/`singleMatches`/`olderMatched`;
|
||||
`totals.trends` still equals the **full** store count; a store whose only matches are `acted`/`skipped` →
|
||||
empty buckets + the `briefSummary` "no fresh signals" line; the order among the surviving `new` records is the
|
||||
R3a composite total order (unchanged).
|
||||
- **SC6 (brief render: id + surfaced marker + descriptor)** — `renderBrief` emits the **full pinned line shapes**
|
||||
(§3): a top entry ends with `· \`<id>\`` (after `Pillarer: …`); a bullet ends with `· \`<id>\`` (after `🔗
|
||||
<url>`); a record with `surfacedCount >= 2` shows `· sett <N>x` (after the score token), one with
|
||||
`surfacedCount` 0/1/absent shows **no** surfaced token — both asserted as **full lines**. The `ranking:`
|
||||
descriptor equals the pinned string ending `; excludes acted/skipped` verbatim. `surfacedIds(ranking)` returns
|
||||
exactly the ids of `topMatches ∪ singleMatches ∪ olderMatched.slice(0,5)`. Two `renderBrief` calls on the same
|
||||
input are byte-identical.
|
||||
- **SC7 (CLI act/skip/reset)** — `act --id <id> --store <tmp>` then `list --store <tmp> --json` shows the record
|
||||
with `status:"acted"`; `skip` → `"skipped"`; `reset` → `"new"`; an **unknown** id → stderr error + **exit 2**,
|
||||
store unchanged; a missing `--id` → usage + exit 2.
|
||||
- **SC8 (CLI brief marks surfaced + --no-mark + exclusion end-to-end)** — `brief --pillars … --store <tmp>` on a
|
||||
store with fresh matches → the written `.md` **omits** any acted/skipped record; a following `list --store <tmp>
|
||||
--json` shows the surfaced trends with `surfacedCount:1` + today's `lastSurfacedAt`, and the `--json` output
|
||||
carries `marked:<n>`; a **second** `brief` the same day → `marked:0`, counts unchanged (idempotent);
|
||||
`brief --no-mark --store <tmp>` on a fresh store → `marked:0`, **no `surfacedCount` written** (store's trends
|
||||
unchanged save for nothing).
|
||||
- **SC9 (gate + wiring + de-niche)** — `bash scripts/test-runner.sh` → `FAIL=0`: trends suite green at the bumped
|
||||
`TRENDS_TESTS_FLOOR`; new **Section 16k** green (`TrendStatus` + `surfacedCount` in `types.ts`, `markSurfaced`
|
||||
in `store.ts`, `effectiveStatus` in `brief.ts`, `command === "act"` in `cli.ts`, non-vacuity self-test);
|
||||
`ASSERT_BASELINE_FLOOR` = **105** (99 + 6); Section 17 de-niche green; counts 27/19/29; the hook suite still
|
||||
green untouched.
|
||||
|
||||
## 7. Verification
|
||||
|
||||
**Deterministic:** `bash scripts/test-runner.sh` → `FAIL=0`; trends suite ≥ new floor; Section 16k self-test +
|
||||
greps pass; `ASSERT_BASELINE_FLOOR` = 105; Section 17 de-niche green; ref/agent/command counts unchanged.
|
||||
**Regression sanity:** `node --test hooks/scripts/__tests__/*.test.mjs` → still green untouched (R3b touches no
|
||||
hook; adds no hook test).
|
||||
**Behavioural (manual):**
|
||||
1. `echo '[{"source":"tavily","title":"A","url":"https://e/a","topics":["ai","gov"],"publishedAt":"<~2d ago>",
|
||||
"score":{"mode":"kortform","dimensions":{"pillar":9,"audience":8,"timing":9,"angle":7,"authority":6}}}]'
|
||||
| node --import tsx src/cli.ts capture --store /tmp/r3b.json` — adds A.
|
||||
2. Re-`capture` A with a **lower** timing (`"timing":3`) → `list --store /tmp/r3b.json --json` shows A's
|
||||
composite **dropped** (re-score last-wins); the capture tally reports `merged:1`.
|
||||
3. `node --import tsx src/cli.ts brief --pillars ai,gov --store /tmp/r3b.json --out /tmp/r3b-brief --json` →
|
||||
confirm `marked:1`; `list --json` shows A with `surfacedCount:1` + today's `lastSurfacedAt`; the entry line
|
||||
shows `· \`<id>\``.
|
||||
4. Re-run the **same** `brief` → `marked:0` (idempotent); `surfacedCount` still 1.
|
||||
5. `node --import tsx src/cli.ts act --id <A's id> --store /tmp/r3b.json` → re-run `brief` → A is **absent** from
|
||||
the written `.md`; the summary reports no fresh signals (if A was the only match).
|
||||
6. `node --import tsx src/cli.ts reset --id <A's id> --store /tmp/r3b.json` → A reappears in the brief.
|
||||
7. `brief --no-mark` on a fresh store → `marked:0`, `surfacedCount` not written.
|
||||
|
||||
## 8. Open questions for the go-gate
|
||||
|
||||
Three architectural decisions are **CONFIRMED** (operator, AskUserQuestion 2026-06-25): **A1** on-record seen-log,
|
||||
the `brief` CLI records surfacing (`rankForBrief` pure, `--no-mark` dry-run); **A2** re-score last-wins (score the
|
||||
one mutable field; status not reset); **A3** acted/skipped excluded from the brief. Residual decisions, all baked
|
||||
to the recommended default — confirm or redirect with "Go":
|
||||
- **D1 — status values `new`/`acted`/`skipped`, absent⇒new (omit on add)?** YES (rec). A 3-state lifecycle; a
|
||||
freshly-captured trend is implicitly `new` (field omitted); `reset` sets `"new"` explicitly. Re-open only if a
|
||||
`published`/`drafted` distinction is wanted (the plugin tracks posts elsewhere — kept out of the trend store).
|
||||
- **D2 — `AddResult.merged` broadened (topics ∪ score-refresh), no new flag?** YES (rec). Keeps the 2-flag shape;
|
||||
the capture tally's "N merged" honestly means "N existing records updated". Re-open only if `rescored` must be
|
||||
counted **separately** from topic-merges in the CLI tally.
|
||||
- **D3 — include `reset` (un-skip → new)?** YES (rec). Symmetric + cheap; the operator changes their mind. Drop
|
||||
only to keep the verb set to two.
|
||||
- **D4 — show the trend `id` in brief entries?** YES (rec). The status feature is **inoperable** otherwise — the
|
||||
operator needs the id to `act`/`skip`. Shown in backticks for copy-paste. Alternative: omit, and require
|
||||
`list --json` to find ids (clunky).
|
||||
- **D5 — minimal `· sett Nx` marker when `surfacedCount >= 2`?** YES (rec). Keeps the seen-log **honest/visible**
|
||||
in R3b (otherwise it is an invisible schema-only accumulation — the anti-pattern R3a warned of) without
|
||||
straying into (b)'s saturation scoring or (d)'s diff. The `>= 2` floor means a first/second sighting is silent.
|
||||
Drop only if any visible surfaced signal should wait for (b).
|
||||
- **D6 — `act`/`skip` identify by `--id` only?** YES (rec). Store-native; the id is shown in the brief +
|
||||
`list --json`. A title/url alias is a deferred nice-to-have.
|
||||
- **D7 — which entries count as "surfaced"?** The entries `renderBrief` **actually shows**: `topMatches ∪
|
||||
singleMatches ∪ olderMatched.slice(0,5)` (rec). Matches what the operator saw; the older-bucket cap mirrors the
|
||||
render's `.slice(0,5)`.
|
||||
- **D8 — `BRIEF_SCHEMA_VERSION` 1→2?** NO (rec). No frontmatter field added/removed (the hook reads only
|
||||
`date`+`summary`). Re-open only if the artifact should self-announce the exclude-handled change.
|
||||
- **D9 — commit split?** Single code commit (rec) — R3b's lifecycle (re-score/status/seen-log) is tightly
|
||||
coupled; the R3a data-then-visible split would land an invisible cut. Docs commit first, then one code commit.
|
||||
|
||||
## 9. Light-Voyage review — folded
|
||||
|
||||
Three Opus reviewers ran on the drafts, each verifying claims against live code. **scope-guardian: ALIGNED**
|
||||
(every SC1–SC9 traces to a step; zero creep, zero gaps; all §4 non-goals held; counts 27/19/29 verified live;
|
||||
`score.ts`/`item.ts`-untouched verified — `itemToInput` already builds the envelope on every capture incl.
|
||||
re-capture; A1/A2/A3 consistent across every step; the R3a-block reconcile is a necessary prerequisite, not creep;
|
||||
2 MINOR plan line-cite nits). **brief-reviewer: PROCEED_WITH_RISKS** (all nine load-bearing claims verified TRUE —
|
||||
incl. the v3→v4 reconcile complete for **every** breaking literal, enumerated; 1 MEDIUM + 3 LOW). **plan-critic:
|
||||
PROCEED_WITH_RISKS (78/B)** (the two-phase RED, the atomic bump+reconcile, the `merged` broadening's
|
||||
non-regression, the `surfacedIds` formula, and the gate arithmetic all verified correct; 1 MAJOR + 5 MINOR).
|
||||
All findings folded; see `plan-re-r3b.md §Plan-critic — folded` for per-finding resolution. Headlines:
|
||||
|
||||
- **[MAJOR/MEDIUM, folded — both reviewers] the `brief` CLI's `store` binding does not exist.** `cli.ts:286`
|
||||
inlines `rankForBrief(loadStore(storePath), …)` — there is no `const store`, so the `markSurfaced(store, …)` /
|
||||
`saveStore(storePath, store)` edit referenced an undefined identifier. → §3 S-cli + plan Step 5 now **hoist**
|
||||
`const store = loadStore(storePath)` and pass it to `rankForBrief`; R5 wording corrected.
|
||||
- **[MINOR, folded — plan-critic #2] not-found id → exit 2 contradicted the documented exit-code contract.** →
|
||||
the `cli.ts:33` doc-comment is **broadened** to *"2 on usage error or a not-found id (act/skip/reset)"* (a wrong
|
||||
`--id` is an argument-class error, distinct from `capture`'s data items); no third exit code introduced (§3 S-cli).
|
||||
- **[MINOR, folded — plan-critic #3] `· sett Nx` off-by-one.** Render precedes the surfacing mutation, so the
|
||||
token reflects the **prior-day** count. → the **prior-day semantic** is now stated explicitly (§3 S-brief + the
|
||||
README): `· sett Nx` = "shown on N prior distinct days".
|
||||
- **[MINOR, folded — plan-critic #4] `capture` tally comment stale + the re-score CLI tally untested.** → §3 S-cli
|
||||
updates the `cli.ts:251-252` comment (`merged` = topics ∪ score-refresh); SC2 adds a subprocess assertion that a
|
||||
re-captured changed-score item reports `merged:1` with the updated composite.
|
||||
- **[MINOR, folded — plan-critic #5] Step 2 used `TrendStatus` before Step 3 defined it.** → the plan is
|
||||
reordered: Step 2 adds the `TrendStatus` type + the three fields to `types.ts` **first** (then the `store.ts`
|
||||
functions); Step 3 isolates the atomic `SCHEMA_VERSION` bump + the R3a-block reconcile.
|
||||
- **[LOW, folded — brief-reviewer #4] forward-debt: the new R3b migration block hard-coded `4`** (perpetuating the
|
||||
reconcile-cycle this slice pays for R3a). → the new block's **target + idempotent** assertions commit against
|
||||
`SCHEMA_VERSION` (the hard-`4` is the Step-1 RED device only; the v3 **input** fixtures stay literal `3`),
|
||||
breaking the cycle so R3c won't pay it.
|
||||
- **[LOW, folded] cosmetic literal/title drift** — `store.test.ts:571`/`:598` titles + `:570` comment flipped to
|
||||
"the current version"; `cli.test.ts:247`'s inert `schemaVersion:2` fixture added to the scope-fence enumeration;
|
||||
the two plan line-cites corrected to `~:1235` (after 16j's block) / `:1237` (Section 18 header).
|
||||
424
docs/research-engine/brief-re-r3c.md
Normal file
424
docs/research-engine/brief-re-r3c.md
Normal file
|
|
@ -0,0 +1,424 @@
|
|||
# Brief — RE-R3c: autonomous trigger — scheduler + headless entry point
|
||||
|
||||
> **Slice:** RE-R3c (research-engine rung-2, R3 slice 3 — the **autonomy** slice: the trigger that makes the
|
||||
> daily loop *closed* and the headless entry that runs the deterministic morning brief with **no interactive
|
||||
> session**). R3 ("deepen the research engine") is an **arc** of 5 open hulls (substrate §1). R3a took relevance,
|
||||
> R3b took the lifecycle (status + seen-log + re-score). R3c takes hulls **(1) no autonomous trigger** + **(6) no
|
||||
> headless entry point** — the *mechanism* that runs the existing deterministic brief on a schedule, built and
|
||||
> tested deterministically **before** the autonomous AI fan-out (slice e) plugs into it.
|
||||
> **Predecessor:** RE-R3b (`status` exclusion + per-day-idempotent `surfacedCount`/`lastSurfacedAt` — the
|
||||
> dedup-state a nightly loop **depends on** so it never re-surfaces handled work) + RE-R2b (`brief.ts` dated
|
||||
> artifact + the SessionStart surfacing the nightly run feeds) + RE-R3a (composite ranking).
|
||||
> **Substrate:** `docs/research-engine-concepts.local.md` §1 hull (1) (*"ingen autonom trigger … zero cron/launchd/
|
||||
> scheduler i hele repoet"*) + (6) (*"ingen headless entry point"*) + §B4 (*"behavioral scheduling … a push/delivery
|
||||
> window that gates delivery separately from the sweep"*) + §B3 (the dated digest as a flat plain-text artifact
|
||||
> *"skrevet av Stop-hook eller cron-trigget headless-sesjon"*). R3c builds the cron-triggered headless path B3
|
||||
> anticipated and the scheduling-window discipline B4 names.
|
||||
> **TDD-order:** RED before code, **two phases** (light-Voyage discipline, inherited): Phase A — assertion-RED via
|
||||
> subprocess against the **existing** CLI (`schedule` is an unknown command today → `usage` exit 2; the wrapper
|
||||
> file is absent → exit 127) — true assertion-RED on the exit-code/stdout assertions, not module-not-found. Phase B
|
||||
> — `schedule.ts` is a NEW module whose exports the tests import; under Node16 ESM a missing named import throws at
|
||||
> module-load, so land **non-throwing stubs** (`launchdPlist → ""`, etc.) first, then record value-assertion RED
|
||||
> against them. See plan Step 1.
|
||||
> **Architectural decisions — CONFIRMED (operator, AskUserQuestion 2026-06-26; baked into the plan):**
|
||||
> - **C1 — deterministic brief-only.** The nightly headless run regenerates the dated brief from the **current
|
||||
> store** (freshness-aging drops stale trends; `surfacedCount` accumulates per distinct day → feeds slice b).
|
||||
> **NO AI capture.** Polling stays operator-driven; the autonomous AI fan-out is **slice (e)**, which plugs into
|
||||
> (c)'s headless seam. Faithful to the operator's `(a)→(c)→(b)→(d)→(e)` sequence: build the trigger mechanism +
|
||||
> headless plumbing (deterministic, testable) **before** the AI sweep it will eventually drive. *Honest framing:
|
||||
> the visible autonomous-research payoff lands with (e); (c) is the mechanism.*
|
||||
> - **C2 — print-first installer.** `schedule` **emits** the launchd plist (macOS) / crontab line (Linux) + the
|
||||
> exact install command; the operator runs it. `--install` writes only the inert launchd plist FILE (never runs
|
||||
> `launchctl`; never touches `crontab`). Matches the global `[voyage]` cron-persistence guard, the push-policy's
|
||||
> operator-authorization, and the "confirm outward-facing/persistent actions" rule.
|
||||
|
||||
## 1. Operator decision context (2026-06-26)
|
||||
|
||||
The research engine is **Tier-1** (operator, 2026-06-23). R1→R3b built the deterministic spine **and** the
|
||||
trend's life after capture: item-schema + triage (R1) → capture bridge (R2a) → dated morning brief + surfacing
|
||||
(R2b) → persisted relevance + composite ranking (R3a) → status lifecycle + seen-log + re-score (R3b). The spine is
|
||||
complete and the lifecycle is correct — **but nothing runs it on its own.** The morning brief exists only when the
|
||||
operator interactively invokes the `brief` CLI (via the `trend-spotter` agent or by hand); the SessionStart hook
|
||||
*surfaces* the latest dated brief (`session-start.mjs:534`) but **never generates one**. There is **zero
|
||||
scheduler** in the repo (verified live: only `scripts/test-runner.sh` exists; no plist, no cron, no launchd in any
|
||||
`.ts`/`.mjs`/`.sh`/config). The loop is open: a brief is only as fresh as the last time the operator remembered to
|
||||
ask for one.
|
||||
|
||||
R3c closes hulls **(1)** and **(6)** — the **autonomous trigger** and the **headless entry point** — which the
|
||||
operator chose as slice (c) of the full-R3 build-out (2026-06-24, *"ALLE gjenstående R3-slices … i rekkefølge
|
||||
(a) → (c) → (b) → (d) → (e)"*). It is sequenced **after** R3b for a load-bearing reason the operator named: an
|
||||
autonomous loop **must never re-surface handled work**, so it depends on R3b's status-exclusion (acted/skipped
|
||||
dropped from the brief) and its **per-day-idempotent** surfacing (a double-fire doesn't double-count). R3b made
|
||||
the nightly regeneration *safe to automate*; R3c automates it.
|
||||
|
||||
**What R3c is — and is not (C1).** R3c is the **mechanism**, not the AI sweep. The nightly run is the *existing
|
||||
deterministic* `brief` generation — load store → rank → write the dated `.md` → record surfacing — run with no
|
||||
interaction by a scheduler. It does **not** poll new sources (that is the AI fan-out, slice e). Its honest value
|
||||
without (e): the brief is regenerated every morning from the current store, so SessionStart surfacing is always
|
||||
fresh; freshness-aging drops trends past the window automatically; and `surfacedCount` accumulates day-over-day —
|
||||
the temporal signal slice (b) reads — **without the operator running anything**. (e) later plugs an AI capture
|
||||
step into the documented pre-brief seam to close the full `poll→score→capture→brief` loop.
|
||||
|
||||
## 2. The gap — grounded in code
|
||||
|
||||
- **No autonomous trigger (hull 1).** Repo-wide there is no scheduler: no launchd plist, no crontab artifact, no
|
||||
`launchctl`/`cron` reference in any source or config (verified). Every brief is born of an interactive session.
|
||||
- **No headless entry point (hull 6) — *almost*.** The `brief` subcommand (`cli.ts:297-328`) is **already
|
||||
non-interactive**: it reads flags, writes `<outDir>/<day>.md`, records surfacing, and exits 0 — no prompts. What
|
||||
is missing is a **robust invocation wrapper** that makes it runnable from a scheduler's *minimal* environment:
|
||||
a launchd/cron job inherits **no shell profile** (no `PATH` from `~/.zshenv`, so a bare `node` is unresolvable),
|
||||
has **no working directory** set to the repo (tsx resolves modules only from `scripts/trends/`), and has **no
|
||||
logging**. Today nothing bridges that gap.
|
||||
- **The brief is operator-pulled, never machine-pushed.** `session-start.mjs:60-77`/`:534` *reads* the latest
|
||||
dated brief (`date`+`summary`, zero-tsx) and surfaces it — it is a pure consumer. Generation lives only in the
|
||||
CLI, invoked by a human. B3's *"cron-trigget headless-sesjon"* writer does not exist.
|
||||
- **The CLI has no scheduling verb.** `cli.ts` exposes `add`/`query`/`list`/`status`/`act`/`skip`/`reset`/
|
||||
`normalize`/`score`/`capture`/`brief` (`cli.ts:5-14`, `:134-330`) — capture/read/lifecycle, all interactive.
|
||||
There is **no way to emit or install a daily schedule** for the brief.
|
||||
- **The data-dir seam is solved, but only for two runtimes.** `store.ts:252` (`defaultStorePath`) and
|
||||
`hooks/scripts/data-root.mjs:24` (`getDataRoot`) are *twins* of the one seam (`LINKEDIN_STUDIO_DATA ?? ~/.claude/
|
||||
linkedin-studio`). A scheduler entry running in **shell** needs the same seam for its log path — a **third
|
||||
sanctioned twin**, exactly the inline `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/…` form
|
||||
`references/data-path-convention.md` rule 1 prescribes. It does not exist yet.
|
||||
|
||||
## 3. Scope — what is IN (RE-R3c)
|
||||
|
||||
### S-schedule — `scripts/trends/src/schedule.ts` (NEW, pure module)
|
||||
|
||||
Pure string emitters for the schedule artifacts — no clock, no fs, no env, no AI (the CLI injects every resolved
|
||||
value). Mirrors `brief.ts`'s `renderBrief` purity → fully testable, byte-deterministic given inputs.
|
||||
|
||||
- **`export interface ScheduleSpec`** — `{ platform: "launchd" | "cron"; label: string; nodeBin: string;
|
||||
wrapperPath: string; args: string[]; hour: number; minute: number; logPath: string; workingDir: string;
|
||||
env: Record<string, string>; }`. All paths are absolute, resolved by the CLI at generation time on the
|
||||
operator's machine. **`env`** is the injected environment map (the CLI builds it — always `NODE_BIN` +
|
||||
a **resolved-absolute** `LINKEDIN_STUDIO_DATA`); the emitter only *renders* it, so it reads no env itself
|
||||
(folded — brief-reviewer #4 / plan-critic #3: the field is canonical, not mid-step).
|
||||
- **`export function launchdPlist(spec: ScheduleSpec): string`** — the plist XML: `Label`, `ProgramArguments`
|
||||
(`["/bin/bash", wrapperPath, ...args]`), `StartCalendarInterval` (`{ Hour: spec.hour, Minute: spec.minute }`),
|
||||
`EnvironmentVariables` (rendered from `spec.env` only — purity), `WorkingDirectory` (`spec.workingDir`),
|
||||
`StandardOutPath`/`StandardErrorPath` (`spec.logPath`), `RunAtLoad` false. A pinned, well-formed template
|
||||
(`<?xml … !DOCTYPE plist …>`); `schedule.test` asserts both key-completeness **and** well-formedness
|
||||
(balanced-tag/parse), `plutil -lint` is the deps-present manual check (folded — brief-reviewer #7).
|
||||
- **`export function crontabLine(spec: ScheduleSpec): string`** — one line:
|
||||
`<minute> <hour> * * * <env-prefix> /bin/bash <wrapperPath> <args…> >> <logPath> 2>&1 # <label>` where
|
||||
`<env-prefix>` is `spec.env` rendered as cron's inline `K=V K=V` form. **The function returns the line as a
|
||||
STRING; it never executes `crontab`** (the execution guard + C2; the literal `crontabLine` does not match the
|
||||
guard's `\bcrontab\b` word-boundary pattern — §5).
|
||||
- **`export function installInstructions(spec: ScheduleSpec, plistTargetPath?: string): string`** — the exact
|
||||
operator commands. launchd: *"written to `<plistTargetPath>` — activate with `launchctl bootstrap gui/$(id -u)
|
||||
<plistTargetPath>`"*. cron: *"add the line above with `(crontab -l 2>/dev/null; echo '<line>') | crontab -`"*.
|
||||
Print-first surfaces these so the operator runs them.
|
||||
- **`export function uninstallInstructions(spec, plistTargetPath?): string`** — symmetric removal (launchd:
|
||||
`launchctl bootout …` + `rm <plist>`; cron: the line-removal `grep -v` recipe).
|
||||
- **`export function defaultLabel(): string`** — `"com.linkedin-studio.trends.daily"` (the **plugin namespace**,
|
||||
domain-general — not the user's domain; no vendor/sector token).
|
||||
|
||||
### S-wrapper — `scripts/trends/run-daily.sh` (NEW, headless invocation wrapper)
|
||||
|
||||
The single tested headless entry, invoked identically by **both** the launchd plist and the crontab line (one
|
||||
entry → one test). Bash 3.2-compatible (operator's macOS: no `declare -A`, no `mapfile`, all expansions quoted,
|
||||
ASCII-only).
|
||||
|
||||
- Resolves **its own directory** (`DIR="$(cd "$(dirname "$0")" && pwd)"`) so it is relocatable — no hard-coded
|
||||
repo path — and **`cd "$DIR"`** so `--import tsx` resolves `node_modules` from the package even under cron's
|
||||
`$HOME` CWD (folded — brief-reviewer #1: the plist sets `WorkingDirectory`, but cron does not — `cd` makes the
|
||||
one wrapper scheduler-agnostic).
|
||||
- Resolves **node** from a minimal scheduler env: `NODE_BIN="${NODE_BIN:-$(command -v node 2>/dev/null)}"`;
|
||||
if still empty, fall back to common locations; exit 127 with a logged error if none. (The scheduler bakes
|
||||
`NODE_BIN=<process.execPath>` so resolution always succeeds; the fallback is for a manual invocation.)
|
||||
- Resolves the **log path** from the canonical inline seam — `LOG="${LINKEDIN_STUDIO_DATA:-$HOME/.claude/
|
||||
linkedin-studio}/trends/cron.log"` — the **fourth sanctioned data-path twin** (shell), the exact form
|
||||
`references/data-path-convention.md` rule 1 prescribes (documented as such, like `data-root.mjs`'s twin comment).
|
||||
The scheduler **always bakes a resolved-absolute `LINKEDIN_STUDIO_DATA`** into the artifact env, so a scheduled
|
||||
run never evaluates `$HOME` (sidesteps the `set -u` `HOME`-unset edge — folded — plan-critic #9 / brief-reviewer
|
||||
#9); the `$HOME` fallback is only for a manual invocation, where `HOME` is set. `mkdir -p` its dir.
|
||||
- Runs the **deterministic** brief: `OUT="$("$NODE_BIN" --import tsx "$DIR/src/cli.ts" brief "$@" --json 2>&1)";
|
||||
CODE=$?` then **compacts** `OUT` to one line (`tr '\n' ' ' | tr -s ' '`) — `brief --json` is **pretty-printed**
|
||||
(`cli.ts:323` `JSON.stringify(…, null, 2)`), so the structured log line must collapse the newlines (folded —
|
||||
plan-critic #1). The scheduler bakes `--pillars … --fresh-days N` into `"$@"`; the wrapper hard-codes the
|
||||
`brief` subcommand and adds `--json`. Appends **one** line `<ISO-ts> exit=<CODE> <compact-OUT>` to `$LOG`;
|
||||
`exit $CODE`. **No AI** — `brief` is the deterministic store→artifact path (C1).
|
||||
- **The (e) seam (documented, not built):** a one-line comment marks where slice (e) will insert a pre-brief
|
||||
capture step (`claude -p … trend-spotter | cli.ts capture`) before the `brief` call. R3c builds **only** the
|
||||
deterministic path.
|
||||
|
||||
### S-cli — `scripts/trends/src/cli.ts` (EDIT) — the `schedule` subcommand
|
||||
|
||||
- **`schedule --pillars <a,b> [--at HH:MM] [--fresh-days N] [--platform auto|launchd|cron] [--install]
|
||||
[--uninstall] [--store <path>]`**:
|
||||
- Resolves **platform**: `auto` (default) → `process.platform === "darwin" ? "launchd" : "cron"`.
|
||||
- Resolves **time** from `--at` (default `07:00`); validates `HH ∈ 0–23`, `MM ∈ 0–59` → `usage` exit 2 on bad
|
||||
input. `--pillars` is **required** (a schedule with no pillars is meaningless) → `usage` exit 2 if absent.
|
||||
- Resolves the absolute paths **from the runtime**, never hard-coded: `nodeBin = process.execPath` (absolute);
|
||||
`wrapperPath = join(dirname(fileURLToPath(import.meta.url)), "..", "run-daily.sh")` (`cli.ts` is at
|
||||
`scripts/trends/src/`, so `..` → `scripts/trends/`); `workingDir = join(dirname(fileURLToPath(import.meta.url)),
|
||||
"..")`; **`logPath = join(dirname(defaultStorePath()), "cron.log")`** — derived from `defaultStorePath()`
|
||||
(`<root>/trends/trends.json` → `<root>/trends/cron.log`), **NOT** from the `--store` override, so it matches
|
||||
the wrapper's data-root-anchored log exactly (folded — all three reviewers: a `--store` outside the data dir
|
||||
must not split the plist `StandardOutPath` from the wrapper's own log file).
|
||||
- Builds **`env`** (always): `{ NODE_BIN: process.execPath, LINKEDIN_STUDIO_DATA: <resolved-absolute root> }`
|
||||
where the root = `process.env.LINKEDIN_STUDIO_DATA ?? join(homedir(), ".claude", "linkedin-studio")` — baked
|
||||
so the scheduled run is pinned to the install-time root **and** never evaluates `$HOME` (the wrapper's
|
||||
`set -u` `HOME`-unset edge).
|
||||
- Builds **`args`** = `["--pillars", <p>, "--fresh-days", String(N)]` (+ `["--store", storePath]` when an explicit
|
||||
non-default `--store` was given, so the scheduled run targets the same store). **No leading `"brief"`** — the
|
||||
wrapper hard-codes the `brief` subcommand (folded — plan-critic #8 / brief-reviewer #5: avoids
|
||||
`cli.ts brief brief …`).
|
||||
- Builds the `ScheduleSpec` and dispatches:
|
||||
- **default / `--print`** → print the artifact (`launchdPlist` or `crontabLine`) **+** `installInstructions`
|
||||
to stdout. **No fs.** Exit 0.
|
||||
- **`--install`** → launchd: `mkdirSync` + `writeFileSync` the plist to `~/Library/LaunchAgents/<label>.plist`
|
||||
(an **inert** file; reversible) and print the single `launchctl bootstrap` command — **the tool never runs
|
||||
`launchctl`**. cron: print the line + the `crontab -` install command — **the tool never runs `crontab`**
|
||||
(the global guard + C2). Exit 0.
|
||||
- **`--uninstall`** → launchd: print the `launchctl bootout` command + (if the plist file exists) `rm` it;
|
||||
cron: print the line-removal recipe. Exit 0.
|
||||
- **Exit-code contract unchanged** (0 success / 2 usage). `schedule` introduces **no new exit code**: an autonomy
|
||||
install never *runs* the system mutation, so there is no install-failure path to encode — the operator runs the
|
||||
one printed command. (Update the header doc-comment `cli.ts:36-37` to note `schedule` is print-first and never
|
||||
shells `launchctl`/`crontab`.)
|
||||
- **Imports** `launchdPlist`, `crontabLine`, `installInstructions`, `uninstallInstructions`, `defaultLabel` from
|
||||
`./schedule.js`; **adds `dirname` to the `node:path` import** (`cli.ts:41` imports only `join` today — folded —
|
||||
plan-critic #5), `homedir` from `node:os`, `fileURLToPath` from `node:url` (`defaultStorePath` is already imported,
|
||||
`cli.ts:45`). The DAG stays acyclic: `schedule.ts` is a **leaf** (imports nothing from the package); `cli.ts`
|
||||
is the existing root.
|
||||
- **Usage + header synopsis** (`cli.ts:5-14`, `:86-100`): add the `schedule …` line + a one-line header note that
|
||||
`schedule` emits/installs a daily headless brief (print-first; deterministic — no AI capture; that is slice e).
|
||||
|
||||
### Wiring (D-default — WIRE, mirrors R3a/R3b)
|
||||
|
||||
- `agents/trend-spotter.md` (EDIT, **prose-only, minimal**): one line — the morning brief can now be **scheduled**
|
||||
to regenerate autonomously (deterministic, from the store) via `schedule`; the agent's polling remains the
|
||||
capture path (autonomous AI polling is a later slice). No batch-shape change. Domain-general (no vendor/sector
|
||||
token).
|
||||
- `scripts/trends/README.md` (EDIT): document the headless wrapper + the `schedule` subcommand (print-first,
|
||||
launchd/cron, `--install`/`--uninstall`), the **deterministic-brief-only boundary (C1)** and the (e) AI-capture
|
||||
seam, the `cron.log`, and the R3b per-day idempotency that makes a double-fire safe.
|
||||
- `scripts/test-runner.sh` (EDIT): bump `TRENDS_TESTS_FLOOR` (`:709`, currently 171) to the `tests N` line reported
|
||||
after Steps 1–5, **append** `+ RE-R3c: scheduler +N` to the inline breakdown comment. Add **Section 16l**
|
||||
("Trends Scheduler / Headless Wiring", RE-R3c) **after Section 16k's closing block (`~:1305`), before Section 18
|
||||
(`:1307`)** (16k is the last 16x before the anti-erosion Section 18; preserve that order). Mirror 16k's shape:
|
||||
**unconditional**, deps-absent-safe (`grep -qF` + a non-vacuity self-test emitting **one** pass/fail).
|
||||
Recommended **6 emitters** (all on tracked source — no `tsx`): (1) self-test; (2) `export function launchdPlist`
|
||||
in `schedule.ts`; (3) `export function crontabLine` in `schedule.ts`; (4) `command === "schedule"` in `cli.ts`
|
||||
(the verb); (5) `cli.ts" brief` in `run-daily.sh` (the wrapper invokes the deterministic brief — the sentinel
|
||||
matches the literal `…cli.ts" brief`, folded — plan-critic #8); (6) the data-path twin in `run-daily.sh`
|
||||
(`LINKEDIN_STUDIO_DATA:-`). **6 unconditional emitters → bump `ASSERT_BASELINE_FLOOR` 105 → exactly 111**
|
||||
(`:1329`; "live recount" is the safety net; the expected value is the pinned 105 + 6). Insert the 16l clause into
|
||||
the **header-enumeration prose chain at `:57`** (before "…the assertion-count anti-erosion floor (SC6) in Section
|
||||
18"), and **append the R3b (→105) + R3c-16l (→111) narration** to the Section-18 floor-history comment
|
||||
(`~:1310-1324`, which still stops at "= 99" — folded — scope-guardian #7).
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **AI capture in the nightly run** (`poll→score→capture` via a headless `claude -p` trend-spotter) — **slice (e)**,
|
||||
behind the post-(d) re-evaluation gate. R3c builds the deterministic headless path + the documented (e) seam; it
|
||||
adds **no** AI invocation, no `claude -p`, no API dependency in the scheduler context.
|
||||
- **Running `launchctl` / `crontab` autonomously** — OUT (C2 print-first). `schedule` prints the activation
|
||||
command; `--install` writes only the inert launchd plist FILE. The operator runs the one system-mutating command.
|
||||
- **A `/linkedin:schedule` command wrapper** (plugin surface) — OUT for R3c (would change the command count). R3c
|
||||
ships the CLI subcommand + README; a command front-door is a later ergonomic slice. Counts stay **29/19/27**.
|
||||
- **Windows Task Scheduler** — OUT. launchd (macOS) + cron (Linux) cover the plugin's runtimes; a Windows emitter
|
||||
is a later portability add.
|
||||
- **A lock / mutex / run-marker** — unneeded. R3b's per-day-idempotent surfacing + the per-day brief filename make
|
||||
a double-fire a safe no-op; B4's separate *delivery* window is not needed for a once-daily calendar job.
|
||||
- **A config-file pillar source** — OUT. Pillars are `--pillars`, **baked into the schedule artifact** at
|
||||
generation (the operator supplies them once at install). A config/profile-resolved pillar source is a later
|
||||
nicety.
|
||||
- **Brief history / day-over-day diff** ("what's new since yesterday" — hull 7) — **slice (d)**.
|
||||
- **Re-scoring / time-decay recompute on a schedule** — OUT. Re-score is on **re-capture** (R3b); R3c does no
|
||||
capture, so the nightly run re-ranks the *unchanged* scores against the *current* freshness window only.
|
||||
- **Schema bumps** — none. R3c touches **no** store field and **no** brief frontmatter field
|
||||
(`SCHEMA_VERSION` stays 4; `BRIEF_SCHEMA_VERSION` stays 1). It adds a new *module* + a *wrapper* + a *CLI verb* —
|
||||
no data shape changes.
|
||||
- **New agent / new command / new reference doc** — none. R3c adds **two source files** (`schedule.ts`,
|
||||
`run-daily.sh`) + their tests, and EDITs `cli.ts` + one agent (prose) + README + gate. `store.ts`/`brief.ts`/
|
||||
`item.ts`/`score.ts`/`types.ts` are **untouched** (the nightly run reuses the existing deterministic `brief`
|
||||
path). Counts stay 29/19/27.
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **TDD iron law (two-phase RED):** failing tests land **BEFORE** implementation. Phase A — subprocess
|
||||
assertion-RED against the existing CLI (`schedule` unknown → exit 2; `run-daily.sh` absent → exit 127) on the
|
||||
exit-0/stdout assertions. Phase B — `schedule.ts` exports are imported by the test; land non-throwing stubs
|
||||
first (Node16 ESM throws a missing named import at module-load), then record value-assertion RED against them.
|
||||
The plan does **not** claim a single "everything fails before any code" run.
|
||||
- **`schedule.ts` is pure** (no clock, no fs, no env, no AI): every value the emitters use is injected via
|
||||
`ScheduleSpec`. The CLI is the only edge that reads `process.execPath`/`import.meta.url`/`defaultStorePath`.
|
||||
Mirrors `renderBrief`'s purity.
|
||||
- **Determinism of the nightly run:** the wrapper invokes the **deterministic** `brief` (whose byte-determinism
|
||||
R2b/R3a/R3b proved); given `(store, pillars, day, freshDays)` the written `.md` is byte-identical. The wrapper
|
||||
adds only a timestamped log line + an exit code.
|
||||
- **No autonomous system mutation (C2):** `schedule` (default) writes **nothing**; `--install` writes only an
|
||||
inert launchd plist file (reversible `rm`); the tool **never** runs `launchctl` or `crontab`. The global guard is
|
||||
an **execution** guard (`voyage` `pre-bash-executor.mjs`, pattern `\bcrontab\b|>\s*/etc/cron` — verified live), so
|
||||
it inspects **bash commands**, not file content: the new files' printed strings (`crontab -`, `launchctl
|
||||
bootstrap`) are written by `Write`/emitted by the CLI and are **fine**, and the 16l grep uses `crontabLine`
|
||||
(no `\bcrontab\b` word-boundary match). **No code path — source or test — ever *executes* a command containing the
|
||||
bare word `crontab` or `launchctl …`**; the install commands are printed STRINGS the operator runs, and tests
|
||||
assert those strings on **stdout/the written file** without executing them (a test that *ran* `crontab` would trip
|
||||
the guard and mutate the real system — explicitly forbidden).
|
||||
- **One data-dir seam, four sanctioned runtime twins:** `store.ts:253` (TS store), `data-root.mjs:25` (hooks
|
||||
`.mjs`), `analytics/src/utils/storage.ts:54` (TS analytics — the existing third, named in `data-root.mjs:44`),
|
||||
`run-daily.sh` (shell — NEW fourth). The shell form is the canonical inline `${LINKEDIN_STUDIO_DATA:-$HOME/
|
||||
.claude/linkedin-studio}` expansion (`references/data-path-convention.md` rule 1), **not** a new seam; documented
|
||||
as a twin (like `data-root.mjs`'s comment) and asserted behaviorally (SC8 newly binds `store.ts`'s
|
||||
`defaultStorePath` into the consistency check — `dirname(defaultStorePath()) == getDataRoot('trends') == the
|
||||
wrapper's `${…}/trends`).
|
||||
- **Domain-general:** no hard-coded user/repo path in the committed **source** (`schedule.ts`/`run-daily.sh`/the
|
||||
`cli.ts` edit) — every concrete path is resolved at generation/run on the operator's machine and lives only in
|
||||
the **generated** artifact (outside the repo, in `~/Library/LaunchAgents` / the crontab). The launchd Label is
|
||||
the plugin namespace; pillars are args. Section 17 de-niche stays green.
|
||||
- **Bash 3.2-compatible wrapper** (operator's macOS): no `declare -A`/`mapfile`/`|&`; all expansions quoted;
|
||||
ASCII-only (a multibyte char crashes under `set -u` on bash 3.2).
|
||||
- **Minimal-env robustness:** the wrapper must run from launchd/cron's profile-less env — node resolved via baked
|
||||
`NODE_BIN` (absolute `process.execPath`) with a `command -v` fallback; `WorkingDirectory`/`cd` set so tsx
|
||||
resolves; log dir `mkdir -p`'d.
|
||||
- **Hook unaffected:** the SessionStart surfacing reads `date`+`summary` only and never shells to tsx; R3c touches
|
||||
neither the hook nor the frontmatter schema, so surfacing is unchanged. The hook suite must still pass untouched
|
||||
(regression sanity; R3c adds no hook test).
|
||||
- **No schema/SSOT change:** `references/trend-scoring-modes.md`, `types.ts`, `store.ts`, `brief.ts` untouched
|
||||
(R3c changes no data shape and no scoring/render math).
|
||||
- **Pathguard:** the two NEW files are under `scripts/trends/` — **write-allowed** (the global pre-write-pathguard
|
||||
allowlists `~/repos/*`; `cli.ts`/`test-runner.sh` were added there with no friction — folded — scope-guardian #4:
|
||||
the earlier ".mjs-under-hooks/scripts-only" phrasing was a fabricated mechanism; the *conclusion* holds). EDITs
|
||||
are to existing files. *(Implementation risk, not a docs-step blocker: if any Write is nonetheless blocked, the
|
||||
operator authorizes via the R2b `!cp` fallback — see plan Risk R4.)*
|
||||
- **Counts** (refs/agents/commands 27/19/29) unchanged. **Recounted live at land**, never pinned/guessed.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1 (launchd plist emit)** — `schedule --pillars ai,gov --platform launchd --at 07:30 --print` → stdout is a
|
||||
**key-complete + well-formed** plist (balanced-tag/parse asserted, not just substring greps — folded —
|
||||
brief-reviewer #7) containing `Label` = `com.linkedin-studio.trends.daily`, `ProgramArguments` invoking
|
||||
`run-daily.sh` with `--pillars ai,gov` (the wrapper supplies `brief`), `StartCalendarInterval` `Hour 7`/`Minute
|
||||
30`, `StandardOutPath`/`StandardErrorPath` = the resolved `cron.log` path, `NODE_BIN` +
|
||||
`LINKEDIN_STUDIO_DATA` in `EnvironmentVariables`; exit 0. Two `--print` runs (same args) → byte-identical (the
|
||||
emitter is pure). `plutil -lint` is the deps-present manual check (Step 7).
|
||||
- **SC2 (crontab line emit — string only)** — `schedule --pillars ai,gov --platform cron --at 07:30 --print` →
|
||||
stdout contains `30 7 * * * NODE_BIN=… /bin/bash …/run-daily.sh brief --pillars ai,gov >> <log> 2>&1 #
|
||||
com.linkedin-studio.trends.daily` **+** the `(crontab -l 2>/dev/null; echo '<line>') | crontab -` install
|
||||
instruction; exit 0. The test asserts the **emitted string** and **never executes `crontab`**.
|
||||
- **SC3 (platform auto)** — `schedule --pillars ai --print` (no `--platform`) → launchd on darwin, cron elsewhere;
|
||||
asserted against `process.platform` (the subprocess inherits the host platform; the assertion branches on it).
|
||||
- **SC4 (print-first writes nothing)** — `schedule --pillars ai --platform launchd --print` with `HOME=<tmp>` →
|
||||
exit 0, stdout has the plist, **and `<tmp>/Library/LaunchAgents` is absent/empty** (no fs write); crontab never
|
||||
invoked.
|
||||
- **SC5 (`--install` launchd: inert plist file, no launchctl)** — `schedule --pillars ai --platform launchd
|
||||
--install` with `HOME=<tmp>` → `<tmp>/Library/LaunchAgents/com.linkedin-studio.trends.daily.plist` **exists**
|
||||
with the SC1 plist content; stdout prints the `launchctl bootstrap` command; **`launchctl` is never run** (the
|
||||
test asserts only the file + stdout; no system job is created); exit 0.
|
||||
- **SC6 (`--install` cron: never self-installs)** — `schedule --pillars ai --platform cron --install` → stdout has
|
||||
the line + the `crontab -` instruction; exit 0; **`crontab` is never invoked** (no system mutation; asserted by
|
||||
stdout only).
|
||||
- **SC7 (headless wrapper runs the deterministic brief + logs)** — `run-daily.sh --pillars ai --store <tmp>/
|
||||
s.json --out <tmp>/mb` (the wrapper supplies `brief`) with `LINKEDIN_STUDIO_DATA=<tmp>` on a seeded fresh store →
|
||||
writes `<tmp>/mb/<today>.md` (the dated brief), appends **exactly one** `<ISO-ts> exit=0 {…compact-json…}` line
|
||||
to `<tmp>/trends/cron.log` (the multi-line `brief --json` collapsed — folded — plan-critic #1), exit 0. A
|
||||
**second** run the same day → the brief `.md` is byte-identical (idempotent re-render), the seen-log is **not**
|
||||
double-counted (R3b per-day idempotency), the log gains a second line. **CWD-independence:** the same invocation
|
||||
with `cwd=<tmp-unrelated>` (not the package dir) still resolves `tsx` and succeeds (the wrapper's `cd "$DIR"` —
|
||||
folded — brief-reviewer #1). The test invokes via `bash run-daily.sh …` so the absent-file RED is exit 127
|
||||
(folded — plan-critic #7). **No AI** is invoked (C1).
|
||||
- **SC8 (data-path twin consistency)** — the wrapper's `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/
|
||||
trends` resolves to the **same** directory as `dirname(defaultStorePath())` (TS — the base the CLI's `logPath`
|
||||
uses, **not** the `--store` override) and `getDataRoot('trends')` (hooks `.mjs`), for both the default root and
|
||||
an overridden `LINKEDIN_STUDIO_DATA`. Asserted behaviorally (resolve all three for a temp override; assert
|
||||
equal). Because `logPath` derives from `defaultStorePath()`, a custom `--store` never splits the plist
|
||||
`StandardOutPath` from the wrapper's log (folded — all three reviewers).
|
||||
- **SC9 (usage / validation)** — `schedule` with **no** `--pillars` → `usage` exit 2; `--at 25:00` / `--at 7:99` /
|
||||
`--at noon` → `usage` exit 2; `--platform bogus` → `usage` exit 2. Each leaves the fs untouched.
|
||||
- **SC10 (gate + wiring + de-niche)** — `bash scripts/test-runner.sh` → `FAIL=0`: trends suite green at the bumped
|
||||
`TRENDS_TESTS_FLOOR`; new **Section 16l** green (`launchdPlist`/`crontabLine` in `schedule.ts`, `command ===
|
||||
"schedule"` in `cli.ts`, the `cli.ts brief` + `LINKEDIN_STUDIO_DATA:-` sentinels in `run-daily.sh`, non-vacuity
|
||||
self-test); `ASSERT_BASELINE_FLOOR` = **111** (105 + 6); Section 17 de-niche green; counts 29/19/27; the hook
|
||||
suite still green untouched (`node --test hooks/scripts/__tests__/*.test.mjs`).
|
||||
|
||||
## 7. Verification
|
||||
|
||||
**Deterministic:** `bash scripts/test-runner.sh` → `FAIL=0`; trends suite ≥ new floor; Section 16l self-test +
|
||||
greps pass; `ASSERT_BASELINE_FLOOR` = 111; Section 17 de-niche green; ref/agent/command counts unchanged.
|
||||
**Regression sanity:** `node --test hooks/scripts/__tests__/*.test.mjs` → still green untouched (R3c touches no
|
||||
hook; adds no hook test).
|
||||
**Behavioural (manual):**
|
||||
1. Seed a store: `echo '[{"source":"tavily","title":"A","url":"https://e/a","topics":["ai","gov"],
|
||||
"publishedAt":"<~2d ago>"}]' | node --import tsx src/cli.ts capture --store /tmp/r3c.json`.
|
||||
2. `node --import tsx src/cli.ts schedule --pillars ai,gov --platform launchd --at 07:00 --print` → inspect the
|
||||
plist; `node --import tsx src/cli.ts schedule … --platform cron --print` → inspect the crontab line + install
|
||||
instruction.
|
||||
3. **Lint the plist (macOS):** pipe the `--print` plist to `plutil -lint -` → "OK" (a malformed plist won't load).
|
||||
4. `LINKEDIN_STUDIO_DATA=/tmp/r3c-data ./run-daily.sh brief --pillars ai,gov --store /tmp/r3c.json --out
|
||||
/tmp/r3c-data/trends/morning-brief` → confirm `/tmp/r3c-data/trends/morning-brief/<today>.md` written + a line
|
||||
appended to `/tmp/r3c-data/trends/cron.log`; exit 0.
|
||||
5. Re-run step 4 same day → `.md` byte-identical; `cron.log` gains a second line; `list --json` shows
|
||||
`surfacedCount:1` (not 2 — per-day idempotent).
|
||||
6. `schedule --pillars ai --platform launchd --install` with a throwaway `HOME` → confirm the plist file is
|
||||
written under `<HOME>/Library/LaunchAgents/` and the `launchctl bootstrap` command is printed (do **not** run it
|
||||
against the real system unless intentionally activating).
|
||||
|
||||
## 8. Open questions for the go-gate
|
||||
|
||||
Two architectural decisions are **CONFIRMED** (operator, AskUserQuestion 2026-06-26): **C1** deterministic
|
||||
brief-only (no AI capture — that is slice e); **C2** print-first installer (emit + the operator runs the system
|
||||
mutation; `--install` writes only the inert launchd plist file). Residual decisions, all baked to the recommended
|
||||
default — confirm or redirect with "Go":
|
||||
- **D1 — schedule time default `07:00`, `--at HH:MM` overrides?** YES (rec). A morning brief wants a pre-workday
|
||||
fire; the operator tunes it. Re-open only for a different default hour.
|
||||
- **D2 — launchd Label = `com.linkedin-studio.trends.daily` (plugin namespace)?** YES (rec). Reverse-DNS, the
|
||||
plugin's own namespace (domain-general; no user-domain token). Drop only for a different naming scheme.
|
||||
- **D3 — `--install` writes the launchd plist FILE but never runs `launchctl`/`crontab`?** YES (rec). The strictest
|
||||
honest print-first: the tool prepares the inert artifact, the operator activates it. Re-open only to make
|
||||
`--install` a pure no-op (print-only, no file write).
|
||||
- **D4 — `--platform auto` defaults via `process.platform`?** YES (rec). darwin→launchd, else→cron. Explicit
|
||||
`--platform` overrides (e.g. to emit a crontab line on a Mac for a Linux box). Drop only to require `--platform`.
|
||||
- **D5 — pillars baked into the artifact at generation (no config-file source)?** YES (rec). The operator supplies
|
||||
`--pillars` once at install; the schedule carries them. A profile-resolved pillar source is a later nicety.
|
||||
- **D6 — `cron.log` under `<data>/trends/cron.log`?** YES (rec). Colocated with the store + morning-brief under the
|
||||
data-dir seam, survives reinstalls. Drop only for a different log location.
|
||||
- **D7 — a single `.sh` wrapper invoked by BOTH launchd + cron (vs a CLI `run` subcommand)?** YES (rec). A shell
|
||||
wrapper handles the launchd/cron minimal-env robustness (node resolution, `cd`, logging) the CLI cannot; the CLI
|
||||
`brief` stays the deterministic core. One wrapper → one tested entry. Re-open only to push the robustness into a
|
||||
CLI `run` verb (more TS, but then the plist must still bake node).
|
||||
- **D8 — no `/linkedin:schedule` command (CLI + README only)?** YES (rec). Keeps the command count; a command
|
||||
front-door is a later ergonomic slice. Re-open only if the scheduler should be operator-facing via a slash
|
||||
command now.
|
||||
- **D9 — commit split?** Docs commit first, then **one** code commit (rec) — the scheduler (module + wrapper + CLI
|
||||
verb + wiring) is one coherent feature. Re-open only for a module-then-wiring split.
|
||||
|
||||
## 9. Light-Voyage review — folded
|
||||
|
||||
Three Opus reviewers ran on the drafts, each verifying claims against live code. **scope-guardian: ALIGNED**
|
||||
(0 creep / 0 gaps; every SC1–SC10 traces to a step; no AI/capture, no schema bump, counts 29/19/27 + untouched-
|
||||
files claim verified live; 1 MAJOR line-cite + 6 MINOR accuracy/precision). **brief-reviewer: PROCEED_WITH_RISKS**
|
||||
(RED premises TRUE, the data-path trio genuinely consistent, C2 no-execution path confirmed; 2 MAJOR + 5 MEDIUM/LOW).
|
||||
**plan-critic: REVISE → 73/C** (the two-phase RED, de-niche safety, bash-3.2 wrapper, path resolution, and the
|
||||
+6→111 gate arithmetic all verified correct against live code; 4 MAJOR + 5 MINOR; the C grade is largely the
|
||||
legacy-manifest-format penalty — these are hand-authored slice docs, not trekexecute manifests). **All findings
|
||||
folded** (per-finding resolution in `plan-re-r3c.md §Plan-critic — folded`). Headlines:
|
||||
|
||||
- **[MAJOR, folded — plan-critic #1] `brief --json` is pretty-printed** (`cli.ts:323` `JSON.stringify(…, null, 2)`),
|
||||
so the wrapper's "one log line" / SC7 contract was false. → the wrapper **compacts** `OUT` (`tr '\n' ' ' | tr -s
|
||||
' '`) after capturing `CODE`; SC7 asserts exactly one line.
|
||||
- **[MAJOR, folded — brief-reviewer #1] the wrapper never `cd`s to its package dir**, so the cron path (CWD `$HOME`)
|
||||
would fail to resolve `tsx`. → the wrapper adds **`cd "$DIR"`**; SC7 gains a CWD-independence assertion.
|
||||
- **[MAJOR/MINOR, folded — all three] the `logPath` expression was self-contradicting and used the `--store`
|
||||
override base.** → pinned to **`join(dirname(defaultStorePath()), "cron.log")`** (the data-root anchor, matching
|
||||
the wrapper); the `".."` and `storePath` variants removed.
|
||||
- **[MAJOR, folded — brief-reviewer #4 / plan-critic #3] `ScheduleSpec.env` was stated three ways.** → `env:
|
||||
Record<string,string>` is now **canonical** in §3, always carrying `NODE_BIN` + a resolved-absolute
|
||||
`LINKEDIN_STUDIO_DATA`; the emitters render it only (purity holds), and the always-baked root also fixes the
|
||||
`HOME`-unset `set -u` edge.
|
||||
- **[MAJOR, folded — all three] `ASSERT_BASELINE_FLOOR` line-cite `:1259` → `:1329`** (verified live; value 105→111
|
||||
correct). `TRENDS_TESTS_FLOOR` `:705` → `:709`; header-enum insertion `:49-53` → `:57`; the Section-18
|
||||
floor-history comment (still "= 99") gets the R3b(→105)+R3c(→111) narration appended.
|
||||
- **[MINOR, folded — plan-critic #5] `dirname` is not imported in `cli.ts`** (`:41` is `join` only). → the plan adds
|
||||
`dirname` to the `node:path` import (+ `homedir`/`fileURLToPath`).
|
||||
- **[MINOR, folded — plan-critic #8 / brief-reviewer #5] stray double `brief`** (`cli.ts brief brief …`). → the
|
||||
baked `args` drops the leading `"brief"`; the wrapper owns the subcommand; the 16l sentinel matches `cli.ts" brief`.
|
||||
- **[MINOR, folded — scope-guardian #4] the pathguard justification was a fabricated mechanism.** → restated:
|
||||
`~/repos/*` is allowlisted; the conclusion (new files write-allowed) holds.
|
||||
- **[LOW, folded — brief-reviewer #7] SC1 "lint-valid" was only grep-checked.** → SC1 asserts **well-formed**
|
||||
(balanced-tag/parse) + key-complete; `plutil -lint` stays the deps-present manual check.
|
||||
- **[LOW, folded — brief-reviewer #8] the twin census undercounted (3 → 4).** → `analytics/storage.ts` named as the
|
||||
existing third; `run-daily.sh` is the fourth.
|
||||
401
docs/research-engine/brief-re-r3d.md
Normal file
401
docs/research-engine/brief-re-r3d.md
Normal file
|
|
@ -0,0 +1,401 @@
|
|||
# Brief — RE-R3d: temporal overlay — first-mover + saturation (R3 slice b)
|
||||
|
||||
> **Slice:** RE-R3d (research-engine rung-2, R3 slice **(b)** in the operator's `(a)→(c)→(b)→(d)→(e)`
|
||||
> sequence — the doc is numbered r3d by creation order, the concept is slice (b)). It closes the **rest of
|
||||
> hull #3** (substrate §1): the store-schema fields a brief *ranks on* were `relevance` (✅ R3a) · `status`
|
||||
> (✅ R3b) · **first-mover** · **saturation** · `angle`. `angle` is **already** a scored dimension
|
||||
> (`score.dimensions.angle`, weight 15 %/20 % — `score.ts:24`/`:32`), folded into the R3a composite; so the
|
||||
> rest of #3 is exactly **saturation + first-mover**, confirming §5's framing.
|
||||
> **Predecessor:** RE-R3a (persisted relevance composite — the PRIMARY within-bucket rank key, `brief.ts:102`)
|
||||
> + RE-R3b (the seen-log: per-day-idempotent `surfacedCount`/`lastSurfacedAt`, `types.ts:71-78`, whose comment
|
||||
> names this slice — *"the temporal foundation slices (c)+(b) read"*) + RE-R3c (the autonomous trigger that
|
||||
> makes `surfacedCount` **accumulate day-over-day without the operator running anything** — the dependency that
|
||||
> makes saturation a real signal, the load-bearing reason (b) follows (c)).
|
||||
> **Substrate:** `docs/research-engine-concepts.local.md` §1 hull (3) (*"store-schema mangler felt en brief
|
||||
> rangerer på (relevance/first-mover/saturation/angle/status)"*) + §B4 (*"freshness-vindu … + dedup-state
|
||||
> (append-only seen-log → ikke re-overflate samme sak)"* — the seen-log IS the saturation substrate). R3d turns
|
||||
> the seen-log + the publish/capture dates into a **live temporal overlay** the brief ranks on.
|
||||
> **The core decision (operator-confirmed, AskUserQuestion 2026-06-26 — baked):**
|
||||
> - **SB1 — derived at brief time, NOT a stored field.** first-mover/saturation are pure functions of fields the
|
||||
> store ALREADY persists (`publishedAt`/`capturedAt` → age; `surfacedCount` → repeat-exposure) + the injected
|
||||
> `today` — exactly like `ageDays`/`effectiveDate` are derived on `BriefEntry`, never stored. **No
|
||||
> `SCHEMA_VERSION` bump (stays 4); no `BRIEF_SCHEMA_VERSION` bump (stays 1).** The signal recomputes every run,
|
||||
> so it can never go stale (a stored field would, as `surfacedCount` grows). Respects the discipline that an
|
||||
> *avledbar* value is not persisted (`ageDays` is derived; `score` is persisted only because it is a frozen
|
||||
> model judgment — `types.ts:54-64`).
|
||||
> - **SB2 — refine recency WITHIN the composite tier; do not override it.** The R3a relevance composite stays
|
||||
> the PRIMARY sort key (`brief.ts:103`). The temporal signal enters `cmp` as a NEW key **after** pillar-overlap,
|
||||
> **before** `effectiveDate` — a richer recency class than the raw date it sits in front of. A
|
||||
> saturated-but-higher-composite trend still outranks a fresh-but-lower one (composite dominates across tiers);
|
||||
> the overlay only re-orders *within* the same (composite, overlap) tier. Honors *"felt en brief rangerer på"*
|
||||
> without shadowing the SSOT-pinned relevance score.
|
||||
> **TDD-order (two-phase RED, light-Voyage discipline, inherited from R3c):** Phase A — `temporalSignal` is a NEW
|
||||
> named export of the EXISTING `brief.ts`; under Node16 ESM a missing named import throws at module-load (every
|
||||
> `brief.test.ts` test would error, not assert), so land a **non-throwing stub** (`temporalSignal → {tier:"fresh",
|
||||
> …}`, `BriefEntry.temporal` populated by it) FIRST, then record value-assertion RED against it (the stub returns
|
||||
> "fresh" always → the first-mover/saturated/ordering/token assertions fail on values, true assertion-RED). Phase
|
||||
> B — the CLI flag tests are value-RED against the existing `brief` handler (the new flags are silently ignored by
|
||||
> `parseFlags` today → tuned-threshold behaviour is unchanged → RED). See plan Step 1.
|
||||
|
||||
## 1. Operator decision context (2026-06-26)
|
||||
|
||||
The research engine is **Tier-1** (operator, 2026-06-23). R1→R3c built the deterministic spine, the trend's life
|
||||
after capture, **and** the autonomy that runs it: item-schema + triage (R1) → capture bridge (R2a) → dated
|
||||
morning brief + surfacing (R2b) → persisted relevance + composite ranking (R3a) → status lifecycle + seen-log +
|
||||
re-score (R3b) → autonomous trigger + headless entry (R3c). The brief now **regenerates itself every morning**
|
||||
(R3c), so `surfacedCount` accumulates day-over-day on its own — but the brief still treats a trend the operator
|
||||
has seen surfaced for five straight days **identically** to one captured an hour ago, as long as their frozen
|
||||
relevance composites tie. The temporal axis the seen-log records is **logged but unread by the ranking**.
|
||||
|
||||
R3d closes the **rest of hull #3** — the **first-mover** and **saturation** signals — which the operator chose as
|
||||
slice **(b)** of the full-R3 build-out (2026-06-24, *"ALLE gjenstående R3-slices … i rekkefølge (a) → (c) → (b)
|
||||
→ (d) → (e)"*). It is sequenced **after** R3c for a load-bearing reason: saturation reads `surfacedCount`, and
|
||||
`surfacedCount` only accumulates *autonomously* because R3c now fires the brief daily. (c) makes (b) meaningful;
|
||||
without the daily trigger, the seen-log would only grow on the days the operator happened to ask for a brief.
|
||||
|
||||
**The concrete value — keeping a frozen composite honest over time.** R3a *froze* the relevance composite at first
|
||||
capture (`types.ts:54-59`: *"first-sight, never updated on re-capture"*). A trend captured fresh scores
|
||||
`timing` 9 ("you'd be among first") → a high composite → "Immediate" → sorts top (`brief.ts:103`). Six days and
|
||||
five surfacings later it is stale and over-exposed, yet the **frozen** composite still says "Immediate" and still
|
||||
sorts top. R3d is the **live temporal overlay** that demotes it — *without mutating the stored composite* (which
|
||||
stays the SSOT-pinned, re-score-on-re-capture value R3b owns). `surfacedToken` (`brief.ts:154`) is already a
|
||||
*hint* of this — the `· sett Nx` marker explicitly labelled *"Not the saturation SCORING of slice (b)"*. R3d
|
||||
promotes that placeholder hint into the real, ranked signal.
|
||||
|
||||
## 2. The gap — grounded in code
|
||||
|
||||
- **The seen-log is written but never ranked on (hull 3, saturation).** R3b records `surfacedCount`/
|
||||
`lastSurfacedAt` per trend (`store.ts:187-202`, `types.ts:71-78`) and renders a display hint (`brief.ts:154-157`,
|
||||
the `· sett Nx` token at `>=2`), but `rankForBrief`'s comparator (`brief.ts:102-107`) **never reads it** — sort
|
||||
order is `composite → overlap → effectiveDate → title → url`. A trend surfaced ten times sorts identically to
|
||||
one surfaced zero times at the same composite+overlap.
|
||||
- **Recency is read coarsely (hull 3, first-mover).** The comparator's only temporal key is `effectiveDate`
|
||||
(`brief.ts:105`) — a raw date tiebreaker. There is no notion of *"this is genuinely fresh AND uncrowded — you'd
|
||||
be first"* vs *"this is just the most recent of a stale set"*. The capture-time `timing` dimension
|
||||
(`score.ts:23`, kortform 20 %) captures a first-mover *judgment*, but it is **frozen at capture** (R3a) — it
|
||||
cannot reflect that the same trend is no longer fresh a week later.
|
||||
- **`angle` is already covered (hull 3, no work needed).** Hull #3 lists `angle` among the rank fields, but
|
||||
`angle` is one of the five scored dimensions (`score.ts:24` kortform 15 %, `:32` long-form 20 %), persisted in
|
||||
`score.dimensions` (R3a) and already folded into the composite the brief ranks on. No separate field is needed;
|
||||
§5's *"saturation + first-mover (resten av #3)"* is exact.
|
||||
- **The brief's `ranking:` frontmatter would misrepresent itself.** The descriptor line (`brief.ts:187`) states
|
||||
the exact sort; once the overlay enters `cmp`, that line must name the new key or the artifact lies about its
|
||||
own ordering.
|
||||
|
||||
## 3. Scope — what is IN (RE-R3d)
|
||||
|
||||
**Zero new source/test files** (the two tracked slice docs aside). R3d is pure EDITs: the derived signal lives beside `ageDaysBetween` in `brief.ts` (the module
|
||||
that already owns the derived-temporal ranking logic — surgical, no new module to wire into the gate), its unit
|
||||
tests in `brief.test.ts`, its CLI flags in `cli.ts`/`cli.test.ts`, plus the wiring docs + gate.
|
||||
|
||||
### S-signal — `scripts/trends/src/brief.ts` (EDIT) — the derived temporal overlay
|
||||
|
||||
- **`export type TemporalTier = "first-mover" | "neutral" | "warming" | "saturated"`** — the four ordinal classes
|
||||
of the temporal axis (best → worst opportunity). **`neutral`** (the draft called this "fresh" — renamed: a tier
|
||||
named "fresh" collided with the `freshDays` bucketing concept AND mislabelled a 30-day-old-but-unsurfaced trend
|
||||
as "fresh"; `neutral` is honestly "no exposure signal, no recency edge" — folded, all three reviewers).
|
||||
- **`export interface TemporalSignal { tier: TemporalTier; firstMover: boolean; surfacings: number; rank: number }`**
|
||||
— `firstMover` = recent AND unsurfaced; `surfacings` = prior-day surfacings (`surfacedCount ?? 0`, the
|
||||
self-exposure level); `rank` = the descending sort integer (`first-mover` 3 > `neutral` 2 > `warming` 1 >
|
||||
`saturated` 0).
|
||||
- **`export function temporalSignal(ageDays: number, surfacedCount: number | undefined, opts: { firstMoverDays:
|
||||
number; saturationAt: number }): TemporalSignal`** — pure (no clock/fs/env; takes the already-computed
|
||||
`ageDays` + the persisted `surfacedCount` + the injected thresholds). Logic:
|
||||
- `surfacings = surfacedCount ?? 0`; `const at = Math.max(1, opts.saturationAt)` — a **defensive clamp**: a
|
||||
direct caller (the function is a public export the gate greps for) passing `saturationAt 0` would otherwise
|
||||
make `surfacings >= 0` always true → every non-first-mover trend "saturated". The CLI also guards `< 1`, but
|
||||
the pure function must not trust its caller (folded — plan-critic m3).
|
||||
- `firstMover = ageDays >= 0 && ageDays <= firstMoverDays && surfacings === 0` — recent AND never surfaced on a
|
||||
prior day. The **`ageDays >= 0`** guard keeps a *future*-dated trend (data glitch) OUT of the "act now, you're
|
||||
early" headline (folded — plan-critic m5). The window maps to the SSOT "<24-72h, you would be among first"
|
||||
band, evaluated on the LIVE date rather than frozen at capture.
|
||||
- tier: `first-mover` if `firstMover`; else `saturated` if `surfacings >= at`; else `warming` if `surfacings >=
|
||||
1`; else `neutral`. (`firstMover` requires `surfacings === 0`; `saturated`/`warming` both require `surfacings
|
||||
>= 1` — so first-mover can collide with neither; the four branches are disjoint and total.)
|
||||
- `rank` derived from tier as above.
|
||||
- **`RankOptions` gains two knobs** (`brief.ts:51-54`): `firstMoverDays?: number` (default **2**) +
|
||||
`saturationAt?: number` (default **3** — the existing `· sett Nx` hint fires at `>=2`, so default 3 keeps
|
||||
`surfacedCount 2` a "warming · sett 2x" FYI and escalates `>=3` to "saturated"). Defaults documented like
|
||||
`freshDays`'s default 7.
|
||||
- **`BriefEntry` gains `temporal: TemporalSignal`** (`brief.ts:26-36`) — populated in `rankForBrief` right after
|
||||
`ageDays` is computed: `temporalSignal(ageDays, trend.surfacedCount, { firstMoverDays, saturationAt })`.
|
||||
- **`rankForBrief` threads the two new opts** (`brief.ts:78`): `const firstMoverDays = opts.firstMoverDays ?? 2;
|
||||
const saturationAt = opts.saturationAt ?? 3;`. The bucketing (`isFresh`/`freshDays`) is **unchanged** —
|
||||
saturation does NOT move a trend between top/single/older buckets; it only re-orders *within* a bucket via the
|
||||
comparator (a soft signal, not a filter — staleness still owns the older bucket via `freshDays`). **Note** the
|
||||
intended in-bucket effect this exposes (folded — plan-critic m4): inside `olderMatched`, a `neutral`
|
||||
(unsurfaced) trend ranks *above* a `warming`/`saturated` one — "you have not been shown this stale item yet"
|
||||
ranks above "you have seen and skipped this stale item N times." That is the saturation signal working, not a
|
||||
bug; the `neutral` (not "fresh") name keeps it honest.
|
||||
- **`cmp` gains the temporal key (SB2)** (`brief.ts:102-107`): insert `b.temporal.rank - a.temporal.rank` **after**
|
||||
`b.overlap - a.overlap` and **before** `b.effectiveDate.localeCompare(a.effectiveDate)`. Composite stays
|
||||
PRIMARY; the chain stays a total order (`rank` is an integer; ties fall through to the unchanged
|
||||
`effectiveDate → title → url` tail, whose `(title,url)` pair is the unique id).
|
||||
- **`temporalToken` replaces `surfacedToken`** (`brief.ts:154-157`): the R3b placeholder hint is promoted to the
|
||||
ranked signal's badge — `first-mover` → `· 🥇 først ute`; `saturated` → `· 🔁 mettet (Nx)`; `warming` →
|
||||
`· sett Nx` **only when `surfacings >= 2`** (this **exactly preserves the R3b ≥2 badge contract**: the live
|
||||
`surfacedToken` fires only at `c >= 2`, `brief.ts:156`, and `brief.test.ts:408` pins `!md.includes("sett 1x")`;
|
||||
the warming *tier* still demotes a surfaced-once trend in `rank`, but its *badge* stays suppressed at 1 — folded,
|
||||
all three reviewers: the draft's "preserves the hint" was inaccurate because warming covers `>= 1`; gating the
|
||||
badge at `>= 2` makes it true); `neutral` → `""`. Used in both `renderTopEntry` (`brief.ts:162`) and
|
||||
`renderBulletEntry` (`brief.ts:171`), replacing the `surfacedToken(e)` call. (The `scoreToken` is untouched.)
|
||||
- **`briefSummary` carries the first-mover marker** (`brief.ts:131-141`): when the top entry is a first-mover,
|
||||
append `· 🥇 først ute` inside the headline parens (`(${pillar}${band} · 🥇 først ute · ${top.ageDays}d)`) —
|
||||
the one-line surfacing the SessionStart hook shows then says *"act now, you're early"*. The marker carries **no
|
||||
double-quote and no newline**, so the hook's `^summary: *"?([^"\n]*)"?` regex still captures it whole
|
||||
(`brief.ts:128-130` invariant preserved).
|
||||
- **`renderBrief`'s `ranking:` descriptor names the new key** (`brief.ts:187`): *"composite desc, then
|
||||
pillar-overlap desc, **then temporal (first-mover↑/saturated↓)**, then publishedAt desc (capturedAt fallback);
|
||||
freshDays N; excludes acted/skipped"* — so the artifact self-documents its true sort.
|
||||
|
||||
### S-cli — `scripts/trends/src/cli.ts` (EDIT) — two `brief` threshold flags
|
||||
|
||||
- **`brief … [--first-mover-days N] [--saturation-at N]`** (mirror `--fresh-days`, `cli.ts:312-317`):
|
||||
`--first-mover-days` parses a **non-negative** integer (default 2; bad → `usage` exit 2); `--saturation-at`
|
||||
parses a **positive** integer (default 3; `< 1` → `usage` exit 2, since `saturationAt 0` would mark every
|
||||
surfaced trend saturated). Both passed into `rankForBrief(store, pillars, day, { freshDays, firstMoverDays,
|
||||
saturationAt })` (`cli.ts:323`).
|
||||
- **Usage + header synopsis** (`cli.ts:14`, `:110`): add the two optional flags to the `brief` line + a one-line
|
||||
header note that the brief applies a derived temporal overlay (first-mover↑/saturated↓) at rank time.
|
||||
- **`schedule` is untouched** — the nightly run uses the **defaults** (2/3). Baking tunable thresholds into the
|
||||
schedule artifact is OUT for R3d (keeps `schedule.ts`/`run-daily.sh` and the R3c tests untouched — no
|
||||
regression surface).
|
||||
|
||||
### Wiring (D-default — WIRE, mirrors R3a/R3b/R3c)
|
||||
|
||||
- `agents/trend-spotter.md` (EDIT, **prose-only, minimal**): one line — the morning brief now applies a **live
|
||||
temporal overlay** at rank time (first-mover ranked up, repeatedly-surfaced/saturated ranked down), **derived**
|
||||
from the publish/capture dates + the seen-log — no new capture step; the agent's polling/capture path is
|
||||
unchanged. Domain-general (no vendor/sector token).
|
||||
- `references/trend-scoring-modes.md` (SSOT, EDIT — **one-line note only**): under "Consumers", note that the
|
||||
morning brief applies a *brief-time* temporal overlay (first-mover/saturation, RE-R3d) as a
|
||||
**within-composite-tier** ranking refinement, and that **this does not change the capture-time weights** above.
|
||||
The five-dimension capture-scoring math is **untouched** (R3d changes no weight, no band, no formula) — the note
|
||||
prevents the SSOT from being misread as the *whole* ranking story (verifiseringsplikt; honesty).
|
||||
- `scripts/trends/README.md` (EDIT): add a `## Temporal overlay (RE-R3d)` section between the R3c scheduler
|
||||
section (`README.md:135-154`) and `## Tests` (`:155`): the first-mover/saturation definitions, the
|
||||
**derived-not-stored** boundary (no schema bump), the `--first-mover-days`/`--saturation-at` flags + defaults,
|
||||
the `cmp` integration (composite stays primary), and the badge set.
|
||||
- `scripts/test-runner.sh` (EDIT): bump `TRENDS_TESTS_FLOOR` (**:713**, currently **192**) to the `tests N` line
|
||||
reported after the suite runs, **append** `+ RE-R3d: brief +N, cli +N (temporal overlay)` to the inline
|
||||
breakdown comment. Add **Section 16m** ("Trends Temporal Overlay", RE-R3d) **between Section 16l's closing
|
||||
`echo ""` (`:1374`) and the Section 18 header (`:1376`)** (anti-erosion must stay last). Mirror 16l's shape:
|
||||
**unconditional**, deps-absent-safe (`grep -qF` + a non-vacuity self-test emitting **one** pass/fail).
|
||||
**6 emitters** (all on tracked source, no `tsx`, all literals **ASCII** — test-runner.sh must stay ASCII-only,
|
||||
so the badge sentinels grep the ASCII tier/flag literals, never the emoji): (1) self-test; (2) `export function
|
||||
temporalSignal` in `brief.ts`; (3) the cmp key `b.temporal.rank` in `brief.ts`; (4) the tier literal
|
||||
`"first-mover"` in `brief.ts`; (5) the flag key `first-mover-days` in `cli.ts`; (6) the flag key `saturation-at`
|
||||
in `cli.ts`. **6 unconditional emitters → bump `ASSERT_BASELINE_FLOOR` 111 → exactly 117** (**:1403**; "live
|
||||
recount" is the safety net; the expected value is the pinned 111 + 6). Insert the 16m clause into the
|
||||
**header-enumeration prose chain (`:57-62`)** before "…the assertion-count anti-erosion floor (SC6) in Section
|
||||
18", and **append the R3d (→117) narration** to the Section-18 floor-history comment (`:1376-1402`, which ends
|
||||
"= 111").
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **A stored `saturation`/`firstMover` field + schema bump** — OUT (SB1). The signals are **derived** each run;
|
||||
persisting an *avledbar* value would go stale as `surfacedCount` grows and would violate the
|
||||
`ageDays`-is-derived discipline. `SCHEMA_VERSION` stays **4**; `types.ts` is **untouched**.
|
||||
- **Market/external saturation** (how crowded the topic is *across the web* — how many others have posted) — OUT,
|
||||
needs external polling = **slice (e)** (the AI fan-out). R3d's saturation is **self-surfacing** only: a proxy
|
||||
for *"you keep seeing this and not acting → the window is closing for you"*, derived from OUR seen-log. The
|
||||
brief is honest about which it measures (the README + the badge wording say "seen N×", not "covered N× online").
|
||||
- **Mutating / re-scoring the relevance composite** — OUT. The R3a composite stays frozen + PRIMARY (SB2). R3d
|
||||
adds a SEPARATE sort key; it never recomputes, decays, or overwrites `score.composite`. The displayed composite
|
||||
is always the stored value.
|
||||
- **Saturation as a filter (auto-dropping / hiding saturated trends)** — OUT. Saturation **demotes within a
|
||||
bucket**; it never removes a trend. Dropping is owned by `status` (acted/skipped, R3b) and the `freshDays`
|
||||
staleness window (the older bucket); R3d's overlay is a soft *re-ordering* signal, not a gate.
|
||||
- **Tunable thresholds in the scheduled run** — OUT, and a **known limitation** (not merely a later nicety —
|
||||
folded — plan-critic m6 / brief-reviewer). `brief` gets `--first-mover-days`/`--saturation-at` for an
|
||||
interactive/tuned run; the **nightly `schedule` run is the primary saturation consumer** (it is what makes
|
||||
`surfacedCount` accumulate, §1) yet is locked to the **defaults** (2/3) — the operator cannot retune the signal
|
||||
*where it actually fires* without re-running `schedule`. Accepted for R3d to keep `schedule.ts`/`run-daily.sh` +
|
||||
the R3c tests untouched (no regression surface); baking the two flags into the schedule artifact is a small,
|
||||
clearly-scoped follow-up.
|
||||
- **Brief history / day-over-day diff** ("what changed since yesterday" — hull 7) — **slice (d)**.
|
||||
- **A new module / new test file** — none. The signal lives in `brief.ts`; its tests in `brief.test.ts`. (No
|
||||
pathguard surface either — all EDITs.)
|
||||
- **New agent / new command / new reference doc** — none. R3d EDITs `brief.ts` + `cli.ts` + their tests + one
|
||||
agent (prose) + the SSOT (one note) + README + gate. Counts stay **29/19/27**. `BRIEF_SCHEMA_VERSION` stays
|
||||
**1** (the frontmatter *fields* are unchanged — only the `ranking:` descriptor string + body badge tokens
|
||||
change; R3a/R3b added tokens without bumping it either — `brief.ts:23`).
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **TDD iron law (two-phase RED):** failing tests land **BEFORE** implementation. Phase A — `temporalSignal` is a
|
||||
new named export of the existing `brief.ts`; Node16 ESM throws a missing named import at module-load, so land a
|
||||
non-throwing **stub** first (returns `{tier:"fresh",firstMover:false,surfacings:0,rank:2}`; `BriefEntry.temporal`
|
||||
populated by it), THEN record value-assertion RED (the stub's constant "fresh" fails the first-mover/saturated/
|
||||
ordering/token assertions). Phase B — the CLI flag tests are value-RED against the existing handler (the new
|
||||
flags are silently ignored today). The plan does **not** claim a single "everything fails before any code" run.
|
||||
- **`temporalSignal` is pure** (no clock, no fs, no env, no AI): every input is injected (`ageDays` already
|
||||
computed, `surfacedCount` from the record, thresholds from the CLI edge). Mirrors `ageDaysBetween`/`renderBrief`
|
||||
purity. Same inputs → same output.
|
||||
- **Determinism of the brief:** given `(store, pillars, today, freshDays, firstMoverDays, saturationAt)` the
|
||||
rendered `.md` is **byte-identical** (R2b/R3a/R3b/R3c proved the byte-determinism of the rest; the overlay adds
|
||||
only a pure, injected-threshold sort key + deterministic tokens).
|
||||
- **Composite stays PRIMARY + frozen (SB2 + R3a):** `cmp`'s first key is unchanged (`b.trend.score?.composite ??
|
||||
-1`); the temporal key sits *after* overlap. A saturated higher-composite trend still outranks a fresh
|
||||
lower-composite one — the overlay re-orders only WITHIN a (composite, overlap) tier. The stored
|
||||
`score.composite` is never read for mutation, only for sorting.
|
||||
- **No schema / SSOT-math change:** `types.ts` untouched (`SCHEMA_VERSION` 4); `brief.ts`'s
|
||||
`BRIEF_SCHEMA_VERSION` stays 1; `score.ts`/`store.ts`/`item.ts`/`schedule.ts` untouched. The SSOT
|
||||
(`trend-scoring-modes.md`) gets a **one-line consumer note** only — **no weight, band, or formula changes** (the
|
||||
capture-scoring instrument is unchanged; the overlay is a separate brief-time layer).
|
||||
- **Hook unaffected:** the SessionStart surfacing reads `date`+`summary` only (`session-start.mjs` per R3c brief
|
||||
§2). The summary's new `· 🥇 først ute` marker carries no double-quote/newline, so the extractYaml regex still
|
||||
captures it whole. R3d touches neither the hook nor the frontmatter field set; the hook suite must still pass
|
||||
untouched (regression sanity; R3d adds no hook test).
|
||||
- **ASCII-only gate literals:** `scripts/test-runner.sh` must stay ASCII (a multibyte char crashes bash 3.2 under
|
||||
`set -u`). The Section-16m sentinels grep the **ASCII** tier/flag literals (`"first-mover"`, `first-mover-days`,
|
||||
`saturation-at`, `b.temporal.rank`, `export function temporalSignal`) — **never** the emoji badges (which live
|
||||
only in `brief.ts` source + rendered output, asserted by the TS tests, not by the shell gate).
|
||||
- **Domain-general:** no hard-coded user/repo path, no vendor/sector token in any edit. The tier labels +
|
||||
Norwegian badge wording (`først ute`, `mettet`, `sett Nx`) are domain-general UI copy (the brief's existing
|
||||
language); pillars/topics remain config. Section 17 de-niche stays green.
|
||||
- **Pathguard:** R3d adds **no new files** — every change is an EDIT of an existing file (write-allowed). (No
|
||||
`.mjs`-under-`hooks/scripts/` surface, no new `scripts/` file.)
|
||||
- **Counts** (refs/agents/commands 27/19/29) unchanged — **recounted live at land**, never pinned/guessed.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1 (first-mover detection)** — `temporalSignal(ageDays, surfacedCount, {firstMoverDays:2, saturationAt:3})`:
|
||||
`(1, 0)`, `(1, undefined)`, `(2, 0)`, `(0, 0)` → `{tier:"first-mover", firstMover:true, rank:3}`; `(3, 0)` (past
|
||||
the window) → NOT first-mover (`tier:"neutral", rank:2`); `(1, 1)` (recent but already surfaced) → NOT
|
||||
first-mover (`tier:"warming", rank:1`); **`(-1, 0)`** (future publishedAt) → NOT first-mover (`tier:"neutral"` —
|
||||
the `ageDays >= 0` guard). Pure: same inputs → same output.
|
||||
- **SC2 (saturation grading + clamp)** — same thresholds: `(5, 3)` and `(5, 4)` → `{tier:"saturated", rank:0}`;
|
||||
`(5, 2)` and `(5, 1)` → `{tier:"warming", rank:1}`; `(5, 0)` → `{tier:"neutral", rank:2}`. `surfacedCount ===
|
||||
saturationAt` is saturated (inclusive `>=`). **Defensive clamp:** `temporalSignal(5, 5, {…, saturationAt:0})`
|
||||
classifies via the clamped `at=1` (NOT "every trend saturated") — the function does not trust an out-of-range
|
||||
threshold.
|
||||
- **SC3 (ranking — overlay re-orders within tier, composite dominates, temporal↔date DISAGREE)** — the fixture
|
||||
**forces the temporal key and `effectiveDate` to disagree**, so the test is genuinely RED in Phase A and the new
|
||||
key is what decides (folded — plan-critic M1: `surfacedCount` correlates with age, so a naive "first-mover vs
|
||||
saturated" fixture would already be ordered correctly by the existing `effectiveDate`-desc key — a vacuous test).
|
||||
Three entries, **same** overlap: **A** = `neutral` (surfaced 0, **older** date, e.g. 5d), composite 7.0; **B** =
|
||||
`warming` (surfaced 2, **newer** date, e.g. 1d), composite 7.0; **Z** = `saturated` (surfaced 4), composite
|
||||
**8.5**. Expected order **`[Z, A, B]`**: Z first (higher composite — PRIMARY, SB2); then A **above** B even
|
||||
though B is newer — the temporal key (`neutral` rank 2 > `warming` rank 1) overrides the `effectiveDate`-desc
|
||||
tiebreaker that would have put the newer B first. In Phase A (stub, constant rank) the order is `[Z, B, A]`
|
||||
(effectiveDate decides A vs B) → RED. `cmp` remains a total order.
|
||||
- **SC4 (render badges + the ≥2 badge boundary)** — `renderBrief`/the bullet path: a first-mover entry contains
|
||||
`· 🥇 først ute`; a saturated entry (`surfacedCount 3`, default `saturationAt 3`) contains `· 🔁 mettet (3x)`; a
|
||||
warming entry with `surfacedCount 2` contains `· sett 2x`; a warming entry with **`surfacedCount 1` contains NO
|
||||
badge** (the preserved R3b ≥2 contract — `brief.test.ts:408` stays green, unchanged); a `neutral` entry contains
|
||||
**none** of the three. The R3b `· sett 3x` assertion (`brief.test.ts:407`) is **updated** to `· 🔁 mettet (3x)`
|
||||
(its surfacedCount-3 trend is now saturated). **Also updated** (folded — brief-reviewer MEDIUM-2): the two tests
|
||||
pinning the `ranking:` descriptor verbatim (`brief.test.ts:325-331` and the regex `:410-416`) gain the new
|
||||
`then temporal (first-mover↑/saturated↓), ` segment; and the `effectiveDate`-isolation test (`brief.test.ts:
|
||||
96-102`) is **re-based** so both entries share a temporal tier (both `neutral`) — else the new key, not
|
||||
`effectiveDate`, would silently decide it (coverage erosion — folded — brief-reviewer MEDIUM-3).
|
||||
- **SC5 (summary first-mover marker)** — when the top entry is a first-mover, `briefSummary` (and the frontmatter
|
||||
`summary:` line) contains `· 🥇 først ute` inside the headline parens; when it is not, the marker is absent. The
|
||||
summary contains **no** `"` and **no** `\n` (the hook-regex invariant).
|
||||
- **SC6 (CLI flags)** — `brief --pillars ai --first-mover-days 1 --saturation-at 2 …` changes the tiers vs the
|
||||
defaults (a 2-day-old trend is first-mover at default 2 but `fresh` at `--first-mover-days 1`; `surfacedCount 2`
|
||||
is `warming` at default 3 but `saturated` at `--saturation-at 2`); absent flags use defaults 2/3;
|
||||
`--first-mover-days -1` / `--first-mover-days x` / `--saturation-at 0` / `--saturation-at x` → `usage` exit 2.
|
||||
- **SC7 (determinism)** — two `brief` runs with the same `(store, pillars, today, freshDays, firstMoverDays,
|
||||
saturationAt)` → byte-identical `.md`.
|
||||
- **SC8 (no schema / no score mutation)** — `SCHEMA_VERSION` 4; `BRIEF_SCHEMA_VERSION` 1; `types.ts` untouched; a
|
||||
`brief` run does **not** change any record's `score.composite` (assert the store's scores are unchanged after a
|
||||
brief, only `surfacedCount`/`lastSurfacedAt` move — the existing R3b behaviour).
|
||||
- **SC9 (purity)** — `temporalSignal` reads no clock/fs/env; a property check over a grid of `(ageDays,
|
||||
surfacedCount)` gives stable, threshold-consistent tiers (first-mover ⊆ recent∧unsurfaced; saturated ⇔
|
||||
`surfacings >= saturationAt`).
|
||||
- **SC10 (gate + wiring + de-niche)** — `bash scripts/test-runner.sh` → `FAIL=0`: trends suite green at the
|
||||
bumped `TRENDS_TESTS_FLOOR`; new **Section 16m** green (the six ASCII sentinels + non-vacuity self-test);
|
||||
`ASSERT_BASELINE_FLOOR` = **117** (111 + 6); Section 17 de-niche green; counts 29/19/27; the hook suite still
|
||||
green untouched (`node --test hooks/scripts/__tests__/*.test.mjs`).
|
||||
|
||||
## 7. Verification
|
||||
|
||||
**Deterministic:** `bash scripts/test-runner.sh` → `FAIL=0`; trends suite ≥ new floor; Section 16m self-test +
|
||||
greps pass; `ASSERT_BASELINE_FLOOR` = 117; Section 17 de-niche green; ref/agent/command counts unchanged.
|
||||
**Regression sanity:** `node --test hooks/scripts/__tests__/*.test.mjs` → still green untouched (R3d touches no
|
||||
hook). The R3c suite (`schedule.test.ts`/`run-daily.test.ts`) still green untouched (`schedule.ts`/`run-daily.sh`
|
||||
not edited).
|
||||
**Behavioural (manual):**
|
||||
1. Seed a store with three trends sharing topics/pillars: one fresh+unsurfaced (`publishedAt` ~1d ago,
|
||||
`surfacedCount` absent), one warming (`surfacedCount` 2), one saturated (`surfacedCount` 4) — same composite.
|
||||
2. `node --import tsx src/cli.ts brief --pillars ai,gov --out /tmp/r3d-mb --store /tmp/r3d.json` → inspect the
|
||||
`.md`: the fresh+unsurfaced entry sorts first with `· 🥇 først ute`; the saturated one sorts last with `· 🔁
|
||||
mettet (4x)`; the `ranking:` descriptor names the temporal key.
|
||||
3. Re-run with `--first-mover-days 0 --saturation-at 2` → the first-mover badge disappears (0-day window) and the
|
||||
`surfacedCount 2` entry escalates to `mettet (2x)`.
|
||||
4. `--first-mover-days x` → exit 2 (`usage`); fs untouched.
|
||||
5. Confirm the seeded records' `score.composite` values are unchanged after the brief (only `surfacedCount`/
|
||||
`lastSurfacedAt` advance) — the overlay never mutates the relevance score.
|
||||
|
||||
## 8. Open questions for the go-gate
|
||||
|
||||
Two architectural decisions are **CONFIRMED** (operator, AskUserQuestion 2026-06-26): **SB1** derived-at-brief
|
||||
(no schema bump); **SB2** refine recency within the composite tier (composite stays primary). Residual decisions,
|
||||
all baked to the recommended default — confirm or redirect with "Go":
|
||||
- **D1 — `firstMoverDays` default `2`?** YES (rec). The tight end of the SSOT "<24-72h, you would be among first"
|
||||
band; `--first-mover-days N` tunes it. Re-open only for a different default (e.g. 3 = the full 72h "early"
|
||||
band).
|
||||
- **D2 — `saturationAt` default `3`?** YES (rec). The existing `· sett Nx` hint fires at `>=2`, so default 3
|
||||
keeps `surfacedCount 2` an FYI ("warming · sett 2x") and escalates `>=3` to "saturated". `--saturation-at N`
|
||||
tunes it. Re-open only for a different default.
|
||||
- **D3 — four tiers (`first-mover`/`fresh`/`warming`/`saturated`)?** YES (rec). A first-mover top, a fresh
|
||||
baseline, a warming FYI (preserves the R3b `sett Nx` hint), a saturated demotion. Drop only to collapse
|
||||
warming into fresh (a 3-tier model) or to add a fifth class.
|
||||
- **D4 — temporal key sits AFTER overlap, BEFORE effectiveDate in `cmp`?** YES (rec, = SB2). Composite then
|
||||
overlap stay primary; the overlay is the coarse recency class, `effectiveDate` the fine tiebreaker beneath it.
|
||||
Re-open only to move the key (e.g. before overlap — stronger overlay).
|
||||
- **D5 — saturation NEVER moves a trend between top/single/older buckets (soft re-order only)?** YES (rec). A
|
||||
soft signal; bucketing stays `overlap`+`freshDays`. Re-open only to let a saturated trend drop a bucket.
|
||||
- **D6 — badges `🥇 først ute` / `🔁 mettet (Nx)` / `sett Nx` (warming) / none (fresh)?** YES (rec). Promotes the
|
||||
R3b `sett Nx` hint into a graded set. Re-open for different wording/emoji (the gate sentinels are ASCII, so
|
||||
emoji changes are test-only).
|
||||
- **D7 — summary line carries `· 🥇 først ute` when the top is a first-mover?** YES (rec). The one-line surfacing
|
||||
then signals "act now, you're early". Drop only to keep the summary minimal (no marker).
|
||||
- **D8 — add a one-line overlay note to the SSOT (`trend-scoring-modes.md`)?** YES (rec). Honest cross-reference
|
||||
so the SSOT is not misread as the whole ranking story; **no** weight/formula change. Drop only to document the
|
||||
overlay solely in `brief.ts` + README + the brief's `ranking:` descriptor.
|
||||
- **D9 — `schedule` untouched (nightly run uses default thresholds)?** YES (rec). Keeps `schedule.ts`/
|
||||
`run-daily.sh` + the R3c tests untouched (no regression surface). Re-open only to bake `--first-mover-days`/
|
||||
`--saturation-at` into the schedule artifact now.
|
||||
- **D10 — commit split?** Docs commit first, then **one** code commit (rec) — the overlay (signal + flags +
|
||||
wiring) is one coherent feature. Re-open only for a signal-then-wiring split.
|
||||
|
||||
## 9. Light-Voyage review — folded
|
||||
|
||||
Three Opus reviewers ran on the drafts, each verifying claims against live code. They **converged on the same two
|
||||
defects** (the strongest signal): **scope-guardian: MIXED** (0 hard creep, both confirmed decisions honored,
|
||||
every SC1–SC10 traces to a step; 1 MAJOR + 1 line-cite + discretionary MINORs). **brief-reviewer:
|
||||
PROCEED_WITH_RISKS** (all seven RED-premise/correctness claims HOLD; the risk is GREEN-completeness — the plan
|
||||
listed 1 of 4 breaking test assertions; 1 MAJOR + 2 MEDIUM + 2 LOW). **plan-critic: APPROVE_WITH_NOTES, 78/B** (the
|
||||
floor arithmetic, line-cites, grep sentinels, cmp total-order, and two-phase-RED structure all verified correct;
|
||||
2 MAJOR + 4 MINOR). **All findings folded** (per-finding resolution in `plan-re-r3d.md §Plan-critic — folded`).
|
||||
Headlines:
|
||||
|
||||
- **[MAJOR, folded — all three] the warming badge fired at `surfacings >= 1`, but the live R3b `surfacedToken`
|
||||
fires only at `>= 2`** (`brief.ts:156`), and `brief.test.ts:408` pins `!md.includes("sett 1x")`. The draft's
|
||||
"preserves the R3b hint" was false (it broadened `>=2` to `>=1`). → `temporalToken`'s warming badge is gated at
|
||||
**`surfacings >= 2`** (R3b contract preserved exactly; `:408` stays green); the warming *tier* still demotes
|
||||
surfaced-once in `rank`. SC4 gains the `surfacedCount 1 → no badge` boundary.
|
||||
- **[MAJOR, folded — plan-critic M1 / brief-reviewer MEDIUM-3] the ordering test was not genuinely RED + vacuous.**
|
||||
`surfacedCount` correlates with age, so a "first-mover vs saturated" fixture is *already* ordered by the existing
|
||||
`effectiveDate`-desc key — Phase A would be GREEN and GREEN proves nothing. → SC3's fixture now **forces
|
||||
temporal↔date disagreement** (older-`neutral` A vs newer-`warming` B at equal composite; the temporal key, not
|
||||
the date, must decide A>B). The coverage-eroded `effectiveDate`-isolation test (`:96-102`) is re-based to a
|
||||
shared tier.
|
||||
- **[MEDIUM, folded — brief-reviewer MEDIUM-2] the `ranking:` descriptor change breaks two more pinned tests**
|
||||
(`brief.test.ts:325-331` + the regex `:410-416`). → the test inventory (SC4 + plan Step 1) now enumerates **all
|
||||
four** touch points, not one.
|
||||
- **[MINOR, folded — plan-critic m3] `temporalSignal` was undefensive against `saturationAt < 1`.** → a
|
||||
`Math.max(1, saturationAt)` clamp inside the pure function (the CLI guard is not enough — the function is a
|
||||
public, gate-grepped export). SC2 gains a clamp case.
|
||||
- **[MINOR, folded — plan-critic m4] the "fresh" tier name was a misnomer** (a 30-day-old unsurfaced trend is not
|
||||
"fresh"; collides with `freshDays`). → renamed **`neutral`** ("no exposure signal"); the in-bucket effect
|
||||
(unsurfaced ranks above seen-and-skipped within `olderMatched`) is documented as intended.
|
||||
- **[MINOR, folded — plan-critic m5] a future `publishedAt` (ageDays < 0) became a first-mover** "act now"
|
||||
headline. → the `ageDays >= 0` guard excludes it (it falls to `neutral`). SC1 gains the `(-1, 0)` case.
|
||||
- **[MINOR, folded — plan-critic m6 / brief-reviewer] the nightly run (the primary saturation consumer) is locked
|
||||
to default thresholds.** → reframed in §4 as a **known limitation**, not a "later nicety."
|
||||
- **[LOW, folded — all three] long-form `angle` weight cite `:34` → `:32`** (`:34` is `currency`; the substance —
|
||||
angle is a scored dimension in both modes — holds).
|
||||
430
docs/research-engine/brief-re-r3e.md
Normal file
430
docs/research-engine/brief-re-r3e.md
Normal file
|
|
@ -0,0 +1,430 @@
|
|||
# Brief — RE-R3e: brief history + day-over-day diff (R3 slice d)
|
||||
|
||||
> **Slice:** RE-R3e (research-engine rung-2, R3 slice **(d)** in the operator's `(a)→(c)→(b)→(d)→(e)`
|
||||
> sequence — the doc is numbered r3e by creation order, the concept is slice (d)). It closes **hull #7**
|
||||
> (substrate §1 hull list: *"ingen brief-historikk"*): the dated morning brief already writes one Markdown
|
||||
> file per day (`morning-brief/YYYY-MM-DD.md`), but the file is **prose-only** — nothing records *which*
|
||||
> trends a brief showed in a machine-readable form, and no run reads yesterday's brief, so the engine cannot
|
||||
> answer the one question a daily motor exists to answer: **"what is new since I last looked?"**
|
||||
> **Predecessor:** RE-R2b (the dated brief artifact + `surfacedIds(ranking)` — the exact set a brief shows:
|
||||
> `brief.ts:305`) + RE-R3b (the per-day-idempotent seen-log `surfacedCount`/`lastSurfacedAt`, which already
|
||||
> records *that* a trend was surfaced but not *with which cohort*) + RE-R3c (the autonomous trigger that makes
|
||||
> the dated files **accumulate day-over-day on their own** — the dependency that makes a day-over-day diff a real
|
||||
> signal, not a once-in-a-while comparison) + RE-R3d (the temporal overlay — the within-brief recency/saturation
|
||||
> class the diff is orthogonal to).
|
||||
> **Substrate:** `docs/research-engine-concepts.local.md` §1 hull **#7** (*"ingen brief-historikk"*) + §B3
|
||||
> (*"Dated-digest som flat plain-text-artefakt … diffbar, grep-bar, lenkbar … Senere sesjon laster «gårsdagens
|
||||
> brief» trivielt"* — the dated file is **explicitly designed to be diffed**, R3e is the diff B3 anticipated) +
|
||||
> §B4 (*"append-only seen-log → ikke re-overflate samme sak"* — R3e is the per-cohort complement: not "have I
|
||||
> seen this ever" but "was this in the PRIOR brief").
|
||||
> **The core decisions (operator-confirmed, AskUserQuestion 2026-06-26 — baked):**
|
||||
> - **SD1 — persist membership in the brief's own frontmatter, NOT a sidecar.** Each brief writes a single
|
||||
> `surfaced: <id-csv>` line into its YAML frontmatter — the ids it actually showed (`surfacedIds(ranking)`).
|
||||
> This keeps **one self-describing, grep-bar artifact** per day (B3), mirrors the existing `store: { … }`
|
||||
> frontmatter idiom, and is **hook-safe** (the SessionStart `extractYaml` is `^summary:`-anchored and
|
||||
> line-scoped — a new `surfaced:` line cannot perturb it). The diff reads the prior brief's `surfaced:` line
|
||||
> via one pure regex. **No sidecar `.json`; no second artifact.** `BRIEF_SCHEMA_VERSION` bumps **1 → 2** (the
|
||||
> frontmatter gained a field — the first bump since R2b; the store's `SCHEMA_VERSION` stays **4**).
|
||||
> - **SD2 — `added` with titles + `dropped` as a count; `brief.ts` stays store-free.** The diff is the symmetric
|
||||
> set difference of today's surfaced ids against the prior brief's: **added** (in today, not prior — the
|
||||
> headline "what's new", rendered with titles resolved from the ranking the brief already holds), **carried**
|
||||
> (in both), **dropped** (in prior, not today). `added` is the value; `dropped`/`carried` render as a one-line
|
||||
> tally (counts). The dropped ids are **not** resolved against the store for an acted/skipped/aged *reason* —
|
||||
> that would require injecting store records into the render and is the one explicit follow-up (§4). The render
|
||||
> needs **only the ranking** it already has → `brief.ts` stays pure (no store, no fs). The framing is **honest**:
|
||||
> "ikke vist i dag" (not shown today), never "resolved" (which the count cannot prove).
|
||||
> **TDD-order (two-phase RED, light-Voyage discipline, inherited from R3c/R3d):** Phase A — `diffSurfaced`,
|
||||
> `parseSurfacedFrontmatter`, `selectPriorBriefFile` (+ the `BriefDiff` type) are NEW named exports of the
|
||||
> EXISTING `brief.ts`; under Node16 ESM a missing named import throws at module-load (every `brief.test.ts` test
|
||||
> would error, not assert), so land **non-throwing stubs** FIRST (`diffSurfaced → {priorDate:null,added:[],
|
||||
> carried:[],dropped:[]}`, `parseSurfacedFrontmatter → []`, `selectPriorBriefFile → null`; `renderBrief` gains an
|
||||
> optional `diff` param it ignores in the stub), then record value-assertion RED against them (the stubs' constant
|
||||
> returns fail the diff/parse/select/section/marker assertions — true assertion-RED). Phase B — the CLI two-day
|
||||
> diff test is value-RED against the existing `brief` handler (today it writes no `surfaced:` line, reads no prior
|
||||
> brief, and its `--json` carries no `diff` key → the day-2 diff assertions fail). See plan Step 1.
|
||||
|
||||
## 1. Operator decision context (2026-06-26)
|
||||
|
||||
The research engine is **Tier-1** (operator, 2026-06-23). R1→R3d built the deterministic spine, the trend's life
|
||||
after capture, the autonomy that runs it, and the within-brief temporal overlay: item-schema + triage (R1) →
|
||||
capture bridge (R2a) → dated morning brief + surfacing (R2b) → persisted relevance + composite ranking (R3a) →
|
||||
status lifecycle + seen-log + re-score (R3b) → autonomous trigger + headless entry (R3c) → temporal overlay
|
||||
(first-mover + saturation, R3d). The brief now **regenerates itself every morning** (R3c) and ranks each trend by
|
||||
its frozen relevance composite, its pillar overlap, and a live first-mover/saturation class (R3d). But every
|
||||
morning's brief is a **standalone snapshot**: it cannot say *"these three are new since yesterday; the two you
|
||||
saw yesterday are gone."* The accumulated dated files are a pile of snapshots, not a **history with a diff**.
|
||||
|
||||
R3e closes **hull #7** — the **brief history + day-over-day diff** — which the operator chose as slice **(d)** of
|
||||
the full-R3 build-out (2026-06-24, *"ALLE gjenstående R3-slices … i rekkefølge (a) → (c) → (b) → (d) → (e)"*). It
|
||||
is sequenced **after** R3c for a load-bearing reason: a day-over-day diff is only meaningful when a brief is
|
||||
**produced every day on its own** — R3c's nightly trigger is what makes "yesterday's brief" reliably exist. (c)
|
||||
makes (d) a real signal; without the daily trigger, "since last brief" could mean "since whenever the operator
|
||||
last happened to ask."
|
||||
|
||||
**The concrete value — turning a pile of snapshots into a feed.** A daily motor's job is to surface the *delta*:
|
||||
the operator does not want to re-read the full ranked list every morning and diff it in their head — they want
|
||||
the engine to say **"3 nye siden i går"** at the top of the brief (and on the one-line SessionStart surfacing,
|
||||
for free). R3e makes the dated file a genuine **history rung**: each brief records what it showed
|
||||
(`surfaced:` frontmatter), and the next brief reads the most recent prior one and renders **"Nytt siden sist."**
|
||||
This is the smallest honest step from *"a brief is written daily"* (R3c) to *"the brief tells me what changed"*
|
||||
(the point of a feed) — and it is exactly the diffable dated-digest §B3 said the artifact was designed to be.
|
||||
|
||||
## 2. The gap — grounded in code
|
||||
|
||||
- **The dated brief is prose-only; membership is not machine-readable.** `cli.ts:341-343` writes
|
||||
`morning-brief/${day}.md` from `renderBrief(ranking)`; the body embeds each trend's `id` inside a rendered
|
||||
bullet (`brief.ts:242`, `:251` — `` `${e.trend.id}` ``), but there is **no structured record** of *the set a
|
||||
brief showed*. `surfacedIds(ranking)` (`brief.ts:305`) computes that set and feeds it to `markSurfaced`
|
||||
(`cli.ts:348`), but it is **never persisted to the artifact** — so a later run that wants "what did yesterday's
|
||||
brief show" would have to scrape prose. The seen-log (`surfacedCount`/`lastSurfacedAt`, R3b) records *that* and
|
||||
*how many days* a trend was surfaced, but **not which cohort it appeared with** — it cannot reconstruct
|
||||
"yesterday's brief contained {A, B, C}."
|
||||
- **No run reads a prior brief.** The only consumer of the dated files is the SessionStart hook's
|
||||
`latestMorningBrief` (`session-start.mjs:60-77`), which reads the **single newest** file's `date`+`summary` and
|
||||
surfaces it verbatim. Nothing reads the **second-newest** to compare. There is no diff, anywhere.
|
||||
- **"New since last" is not derivable from the store alone.** The seen-log gives "never surfaced ever"
|
||||
(`surfacedCount` absent ⇒ a first-ever sighting) — but that is **not** "new since the last brief": a trend
|
||||
surfaced once three days ago, absent from yesterday's brief, reappearing today is *new to yesterday's reader*
|
||||
yet has `surfacedCount 1` (not 0). Only a **per-brief membership record** (the prior `surfaced:` set) answers
|
||||
"was this in the immediately-prior brief," and only it can compute **dropped** (in the prior cohort, gone
|
||||
today) — which the store cannot express at all. This is precisely the gap §B3's "diffbar … dated-digest" and
|
||||
§1 hull #7 name.
|
||||
- **The brief's own summary cannot signal a delta.** `briefSummary` (`brief.ts:204-216`) describes today's top
|
||||
match in isolation; the SessionStart surfacing (`session-start.mjs:534-536`) shows that line verbatim. There is
|
||||
no "N nye siden sist" the operator could see *without opening the file* — the one number a feed leads with.
|
||||
|
||||
## 3. Scope — what is IN (RE-R3e)
|
||||
|
||||
**Zero new source/test files** (the two tracked slice docs aside). R3e is pure EDITs: the diff lives beside
|
||||
`surfacedIds` in `brief.ts` (the module that already owns the brief's pure read logic — surgical, no new module
|
||||
to wire into the gate), its unit tests in `brief.test.ts`, its CLI wiring (prior-file discovery) in
|
||||
`cli.ts`/`cli.test.ts`, plus the wiring docs + gate.
|
||||
|
||||
### S-history — `scripts/trends/src/brief.ts` (EDIT) — the pure diff + the persisted membership
|
||||
|
||||
- **`BRIEF_SCHEMA_VERSION` bumps 1 → 2** (`brief.ts:23`) — the frontmatter gained the `surfaced:` field. (The
|
||||
store's `SCHEMA_VERSION` is untouched at **4** — R3e adds **no store field**; the membership lives in the
|
||||
artifact, the diff is derived.)
|
||||
- **`export interface BriefDiff { priorDate: string | null; added: string[]; carried: string[]; dropped: string[] }`**
|
||||
— `priorDate` = the date of the brief diffed against (`null` ⇒ no prior brief, i.e. the first ever / a fresh
|
||||
data dir); `added`/`carried`/`dropped` = the three partitions of the set difference, each **order-stable**
|
||||
(added/carried preserve today's `surfacedIds` order; dropped preserves the prior set's order).
|
||||
- **`export function diffSurfaced(currentIds: string[], priorIds: string[], priorDate: string | null): BriefDiff`**
|
||||
— pure (no clock/fs/env; both id lists + the prior date are injected by the CLI edge). `added` = `currentIds`
|
||||
not in `priorIds`; `carried` = `currentIds` in `priorIds`; `dropped` = `priorIds` not in `currentIds`. Uses a
|
||||
`Set` for membership; preserves input order in the output arrays. When `priorIds` is empty (first brief),
|
||||
`added === currentIds` and `dropped === []`.
|
||||
- **`export function parseSurfacedFrontmatter(md: string): string[]`** — pure; extracts the `surfaced:` value
|
||||
from a brief's full text via a single line-anchored regex (mirrors the hook's `extractYaml` idiom:
|
||||
`/^surfaced: *([^\n]*)/m`), splits on `,`, trims, drops empties. Returns `[]` when the line is **absent, blank,
|
||||
or malformed** (a pre-R3e brief, or a hand-edited file) — degrades to "empty prior," never throws. Real ids are
|
||||
comma-free hex (`store.ts:69-72`), so the CSV is unambiguous.
|
||||
- **`export function selectPriorBriefFile(filenames: string[], today: string): string | null`** — pure; from a
|
||||
directory listing, returns the **lexicographically greatest** filename matching `^\d{4}-\d{2}-\d{2}\.md$` whose
|
||||
date is **strictly less than** `today` (ISO dates sort lexicographically, so string compare = date compare),
|
||||
else `null`. Mirrors the hook's `latestMorningBrief` filter+sort (`session-start.mjs:63-66`) but **excludes
|
||||
today and any future-dated file** — so a same-day re-run diffs against the true previous day, not its own
|
||||
just-written file (the byte-determinism guarantee, SC8).
|
||||
- **`renderBrief` gains an optional `diff` param** (`brief.ts:259`): `renderBrief(ranking: BriefRanking, diff?:
|
||||
BriefDiff)`. Two additive emissions:
|
||||
- **Frontmatter `surfaced:` line** — `surfaced: ${surfacedIds(ranking).join(",")}` inserted **before**
|
||||
`schemaVersion:` (always emitted, even for an empty store → `surfaced: ` blank; this is the record the *next*
|
||||
day's diff reads, independent of whether *today* had a prior). `schemaVersion:` now renders **2**.
|
||||
- **A `## 🆕 Nytt siden sist` section** (the header gains ` (<prior-date>)` when a prior brief exists, per SC9),
|
||||
placed **after the intro line and before `## 🎯 Topp-treff`** (the
|
||||
delta leads, then the full ranked list). Branches (all deterministic):
|
||||
- **no diff arg / `priorDate === null` with added** → `_Første brief — alt nedenfor er nytt._`
|
||||
- **`priorDate === null` with no added** (empty first brief) → `_Første brief._`
|
||||
- **`priorDate !== null`, `added` non-empty** → one bullet per added id, its entry resolved from the ranking
|
||||
(title + matched pillars + date + link + id, reusing the bullet idiom), then a tally line
|
||||
`_${carried.length} båret over, ${dropped.length} ikke vist i dag._`
|
||||
- **`priorDate !== null`, `added` empty** → `_Ingenting nytt siden ${priorDate}._` (+ the same tally line)
|
||||
- When `diff` is omitted (a bare `renderBrief(ranking)` call, e.g. a unit test that does not exercise the diff),
|
||||
it defaults to the empty diff (`{priorDate:null,added:[],carried:[],dropped:[]}`) → the **`_Første brief._`**
|
||||
section branch (`priorDate===null`, `added` empty). The `surfaced:` frontmatter line is **independent of the
|
||||
diff** — always `surfacedIds(ranking).join(",")` (blank **only** for an empty store), so a non-empty ranking
|
||||
still emits its real surfaced ids. (This keeps existing single-arg call sites compiling and semantically valid.)
|
||||
- **`briefSummary` gains an optional `diff` param** (`brief.ts:204`): `briefSummary(ranking, diff?)`. When `diff`
|
||||
is present, `priorDate !== null`, and `added.length > 0`, it appends ` ${added.length} nye siden sist.` to the
|
||||
one-line headline — so the **SessionStart hook surfaces the delta for free** (it already shows the `summary:`
|
||||
line verbatim; no hook edit). The marker is suppressed on the first brief (`priorDate === null`) and when
|
||||
nothing is new (no noise). It carries **no double-quote and no newline** (the `^summary: *"?([^"\n]*)"?`
|
||||
hook-regex invariant, `brief.ts:200-203`). `renderBrief` passes its `diff` through to `briefSummary` so the
|
||||
frontmatter `summary:` and the `--json summary` agree.
|
||||
|
||||
### S-cli — `scripts/trends/src/cli.ts` (EDIT) — prior-brief discovery + the diff in `--json`
|
||||
|
||||
- **The `brief` handler discovers the prior brief and computes the diff** (between the ranking at `cli.ts:339`
|
||||
and the render at `:340`): `readdirSync(outDir)` (guarded by `existsSync` — a first run has no dir) →
|
||||
`selectPriorBriefFile(files, day)` → if found, `readFileSync` it and `parseSurfacedFrontmatter` → build
|
||||
`diffSurfaced(surfacedIds(ranking), priorIds, priorDate)`; on any fs error, degrade to the empty-prior diff
|
||||
(`priorDate: null`). Pass the diff into `renderBrief(ranking, diff)`. Adds `readdirSync` to the existing
|
||||
`node:fs` import (`cli.ts:51`).
|
||||
- **`--json` gains a `diff` object** (`cli.ts:352`): `diff: { priorDate, added: added.length, carried:
|
||||
carried.length, dropped: dropped.length }` — counts, not id lists (the headless `run-daily.sh` collapses
|
||||
`--json` to one cron-log line). The non-JSON console line (`cli.ts:355`) appends `, N nye siden sist` when
|
||||
`added > 0 && priorDate !== null`.
|
||||
- **`--no-mark` is unchanged in meaning** — it still governs only the **store** seen-log write (`cli.ts:347-349`).
|
||||
The artifact's `surfaced:` frontmatter records what the brief showed **regardless** of `--no-mark` (it is a
|
||||
property of the rendered brief, not of the store mutation). No new flag.
|
||||
|
||||
### Wiring (D-default — WIRE, mirrors R3a/R3b/R3c/R3d)
|
||||
|
||||
- `agents/trend-spotter.md` (EDIT, **prose-only, minimal**): one line — the morning brief now records which trends
|
||||
it showed (frontmatter `surfaced:`) and renders a **day-over-day diff** ("Nytt siden sist") against the most
|
||||
recent prior brief — no new capture step; the agent's polling/capture path is unchanged. Domain-general (no
|
||||
vendor/sector token).
|
||||
- `scripts/trends/README.md` (EDIT): add a `## Brief history + diff (RE-R3e)` section between the R3d temporal-
|
||||
overlay section and `## Tests`: the `surfaced:` frontmatter record, the `selectPriorBriefFile` prior-discovery
|
||||
(strict `< today`, same-day re-run determinism), the `diffSurfaced` partitions, the section + the summary
|
||||
marker, and the `BRIEF_SCHEMA_VERSION 1→2` boundary (artifact-only; store `SCHEMA_VERSION` stays 4).
|
||||
- `scripts/test-runner.sh` (EDIT): bump `TRENDS_TESTS_FLOOR` (**live `:716`**, currently **216**) to the `tests N`
|
||||
line reported after the suite runs — **recounted live**, **append** `+ RE-R3e: brief +N, cli +N (brief history
|
||||
+ diff)` to the inline breakdown comment. Add **Section 16n** ("Trends Brief History / Diff", RE-R3e) **between
|
||||
Section 16m's closing `echo ""` and the Section 18 header** (anti-erosion must stay last). Mirror 16m's shape:
|
||||
**unconditional**, deps-absent-safe (`grep -qF` + a non-vacuity self-test emitting **one** pass/fail). **6
|
||||
emitters** (all on tracked source, no `tsx`, all literals **ASCII** — the section header emoji `🆕` is **never**
|
||||
grepped; the shell stays ASCII-clean for bash 3.2 `set -u`): (1) self-test; (2) `export function diffSurfaced`
|
||||
in `brief.ts`; (3) `parseSurfacedFrontmatter` in `brief.ts`; (4) the section header literal `Nytt siden sist`
|
||||
in `brief.ts`; (5) `selectPriorBriefFile` in `cli.ts` (the diff wiring); (6) the frontmatter emit `surfaced: `
|
||||
in `brief.ts`. **6 unconditional emitters → bump `ASSERT_BASELINE_FLOOR` 117 → exactly 123** (**live `:1473`**;
|
||||
"live recount" is the safety net; the expected value is the pinned 117 + 6). Insert the 16n clause into the
|
||||
**header-enumeration prose chain (`:53-64`)** before "…the assertion-count anti-erosion floor (SC6) in Section
|
||||
18," and **append the R3e (→123) narration** to the Section-18 floor-history comment (which ends "= 117").
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **A sidecar `.json` membership manifest** — OUT (SD1). Membership lives in the brief's own frontmatter
|
||||
(`surfaced:`), keeping one self-describing artifact (B3). No second file per day.
|
||||
- **`dropped` resolved to an acted/skipped/aged *reason*** — OUT (SD2), and the **one explicit follow-up**.
|
||||
`dropped` renders as a **count** ("N ikke vist i dag"). Labelling *why* each dropped id left (acted/skipped via
|
||||
`status`, or aged past `freshDays`) would require injecting the store records into the render — `brief.ts` would
|
||||
no longer be store-free. Honest framing for R3e: "ikke vist i dag," never "resolved." A small, clearly-scoped
|
||||
follow-up (CLI resolves dropped ids → `{title, status}` and passes them to a richer render) if the loop-closing
|
||||
signal proves worth the coupling.
|
||||
- **A browsable history INDEX file** (e.g. a rolling `history.md` of all past briefs) — OUT. The dated files +
|
||||
the `surfaced:` frontmatter **are** the history (grep-bar, lenkbar — B3); an index is a presentation nicety, not
|
||||
a capability gap.
|
||||
- **A SessionStart hook change to render the diff** — OUT. The "N nye siden sist" marker rides the **existing**
|
||||
`summary:` surfacing (`session-start.mjs:534-536`) — no hook edit, no hook test, no new frontmatter field the
|
||||
hook must learn. (The hook still reads only `date`+`summary`.)
|
||||
- **`schedule` / `run-daily.sh` changes** — OUT. The nightly run calls `brief` (`run-daily.sh:33`), which now
|
||||
computes the diff internally → the scheduled brief gets "Nytt siden sist" **automatically**, with **no**
|
||||
scheduler edit (no R3c regression surface).
|
||||
- **A new store field / schema bump / store mutation for the diff** — OUT. `SCHEMA_VERSION` stays **4**;
|
||||
`types.ts`/`store.ts` are **untouched**. The membership is an artifact property; the diff is derived at the CLI
|
||||
edge.
|
||||
- **"New" defined as first-ever-sighting (`surfacedCount === 0`)** — OUT (rejected as less correct). R3e's "new"
|
||||
is **relative to the immediately-prior brief** (artifact diff), which also flags a trend *re-emerging* after a
|
||||
gap — the honest meaning of "siden sist." (`surfacedCount` stays the R3d saturation input, a different
|
||||
question.)
|
||||
- **Diffing against an arbitrary historical brief (`--since <date>`)** — OUT. R3e diffs against the **most recent
|
||||
prior** brief only (the "since last" a daily feed needs). An arbitrary baseline is a later nicety.
|
||||
- **A new module / new test file** — none. The diff lives in `brief.ts`; its tests in `brief.test.ts`. (No
|
||||
pathguard surface — all EDITs.)
|
||||
- **New agent / new command / new reference doc** — none. R3e EDITs `brief.ts` + `cli.ts` + their tests + one
|
||||
agent (prose) + README + gate. Counts stay **29/19/27**. (The SSOT `trend-scoring-modes.md` is **not** touched —
|
||||
the diff is not a scoring concern; scope fence.)
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **TDD iron law (two-phase RED):** failing tests land **BEFORE** implementation. Phase A — the new named exports
|
||||
(`diffSurfaced`/`parseSurfacedFrontmatter`/`selectPriorBriefFile`/`BriefDiff`) need non-throwing **stubs** first
|
||||
(Node16 ESM throws a missing named import at module-load), THEN value-assertion RED against the constant stubs.
|
||||
Phase B — the CLI two-day diff test is value-RED against the existing handler (no `surfaced:` write, no prior
|
||||
read, no `diff` in `--json` today). The plan does **not** claim a single "everything fails before any code" run.
|
||||
- **`brief.ts` stays pure** (no clock, no fs, no env, no AI): `diffSurfaced`/`parseSurfacedFrontmatter`/
|
||||
`selectPriorBriefFile` all take strings/arrays and return values — the directory read + file read live in
|
||||
`cli.ts` (the edge), exactly like `today`/`pillars` are injected. The module's "No fs, no clock, no AI, no
|
||||
network" header claim is preserved.
|
||||
- **Determinism of the brief:** given `(store, pillars, today, freshDays, firstMoverDays, saturationAt, diff)` the
|
||||
rendered `.md` is **byte-identical** (the diff is now an injected input, like `today`). Critically, a **same-day
|
||||
re-run is byte-identical**: `selectPriorBriefFile` excludes `${today}.md` (strict `<`), so the re-run diffs
|
||||
against the same previous day's brief and re-writes the same `surfaced:` line (R3c SC7 preserved).
|
||||
- **`surfaced:` records the shown set, `--no-mark`-independent:** the frontmatter line is `surfacedIds(ranking)`
|
||||
joined — what the brief *showed* — regardless of whether the store seen-log was written (`--no-mark` governs the
|
||||
store mutation only). The artifact is always self-consistent.
|
||||
- **Frozen composite + temporal overlay untouched (R3a + R3d):** R3e adds **no** `cmp` key and changes **no**
|
||||
ranking — `rankForBrief` is unchanged. The diff is a **post-ranking, render-time** layer over the same surfaced
|
||||
set. `score.composite` is never read for mutation; the R3d `temporal` overlay is orthogonal (it orders within
|
||||
the brief; the diff compares across briefs).
|
||||
- **Schema boundary:** `BRIEF_SCHEMA_VERSION` bumps **1 → 2** (the artifact's frontmatter gained `surfaced:`);
|
||||
the store's `SCHEMA_VERSION` stays **4**; `types.ts`/`store.ts`/`score.ts`/`item.ts`/`schedule.ts`/
|
||||
`run-daily.sh` are **untouched**.
|
||||
- **Hook unaffected:** the SessionStart surfacing reads `date`+`summary` only (`session-start.mjs:60-77`). The new
|
||||
`surfaced:` frontmatter line is `^surfaced:`-keyed (the `^summary:`-anchored, line-scoped `extractYaml` cannot
|
||||
match it), and the `summary:` marker carries no `"`/`\n` — so the regex still captures the summary whole. R3e
|
||||
touches neither the hook nor the field set the hook reads; the hook suite must still pass untouched (regression
|
||||
sanity; R3e adds no hook test).
|
||||
- **ASCII-only gate literals:** `scripts/test-runner.sh` must stay ASCII (a multibyte char crashes bash 3.2 under
|
||||
`set -u`). The Section-16n sentinels grep the **ASCII** literals (`export function diffSurfaced`,
|
||||
`parseSurfacedFrontmatter`, `Nytt siden sist`, `selectPriorBriefFile`, `surfaced: `) — **never** the `🆕` emoji
|
||||
(which lives only in `brief.ts` source + rendered output, asserted by the TS tests, not by the shell gate).
|
||||
- **Domain-general:** no hard-coded user/repo path, no vendor/sector token in any edit. The section header +
|
||||
Norwegian copy (`Nytt siden sist`, `båret over`, `ikke vist i dag`, `nye siden sist`, `Første brief`) are
|
||||
domain-general UI copy (the brief's existing language); pillars/topics remain config. Section 17 de-niche stays
|
||||
green.
|
||||
- **Pathguard:** R3e adds **no new files** — every change is an EDIT of an existing file (write-allowed). (No
|
||||
`.mjs`-under-`hooks/scripts/` surface, no new `scripts/` file.)
|
||||
- **Counts** (refs/agents/commands 27/19/29) unchanged — **recounted live at land**, never pinned/guessed.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1 (diffSurfaced — partitions + order + empty prior)** — `diffSurfaced(["a","b","c"], ["b","c","d"], "2026-
|
||||
06-25")` → `{priorDate:"2026-06-25", added:["a"], carried:["b","c"], dropped:["d"]}` (added/carried in current
|
||||
order, dropped in prior order). `diffSurfaced(["a","b"], [], null)` → `{priorDate:null, added:["a","b"],
|
||||
carried:[], dropped:[]}` (empty prior ⇒ everything added). Pure: same inputs → same output. The three
|
||||
partitions are **mutually disjoint** — `Set` membership is binary (an id is in `priorIds` or not), so each id
|
||||
lands in exactly one of added/carried and dropped is disjoint from both; within each list, order and any
|
||||
duplicates **mirror the input** (`surfacedIds` yields **distinct** ids in production, so within-list dups never
|
||||
arise — the cross-partition exclusivity is the real invariant, not within-list dedup).
|
||||
- **SC2 (parseSurfacedFrontmatter — read + degrade)** — parses `surfaced: 1a2b,3c4d,5e6f` (in a full frontmatter
|
||||
block) → `["1a2b","3c4d","5e6f"]`; a **blank** `surfaced: ` → `[]`; an **absent** `surfaced:` line (a pre-R3e
|
||||
brief) → `[]`; whitespace around ids is trimmed; the `summary:`/`store:`/`date:` lines are **not** mismatched
|
||||
(line-anchored). Never throws on malformed input.
|
||||
- **SC3 (selectPriorBriefFile — strict-prior selection)** — from `["2026-06-24.md","2026-06-25.md","2026-06-26.md",
|
||||
"README.md","2026-06-30.md"]` with `today="2026-06-26"` → `"2026-06-25.md"` (greatest `< today`; **excludes**
|
||||
today `2026-06-26.md` and the future `2026-06-30.md`; ignores the non-dated `README.md`). Empty list, or no file
|
||||
`< today`, → `null`.
|
||||
- **SC4 (frontmatter `surfaced:` line + round-trip)** — `renderBrief(ranking, diff)` emits exactly one
|
||||
`^surfaced: <csv>$` line, equal to `surfacedIds(ranking).join(",")`, positioned before `schemaVersion: 2`; an
|
||||
empty-store brief emits `surfaced: ` (blank); `parseSurfacedFrontmatter(renderBrief(r, d))` round-trips to
|
||||
`surfacedIds(r)`. `schemaVersion:` renders `2`.
|
||||
- **SC5 (Nytt siden sist section — all four branches)** — the rendered body contains `## 🆕 Nytt siden sist`;
|
||||
with `priorDate:null` + added → `Første brief — alt nedenfor er nytt`; with `priorDate:null` + no added (empty
|
||||
store) → `Første brief.`; with a prior + `added` → one bullet per added entry (its **title** present, resolved
|
||||
from the ranking) + `N båret over, M ikke vist i dag`; with a prior + no added → `Ingenting nytt siden <date>`
|
||||
+ the tally. The section precedes `## 🎯 Topp-treff`.
|
||||
- **SC6 (summary delta marker)** — `briefSummary(ranking, diff)` with `priorDate !== null` and `added.length > 0`
|
||||
ends with ` ${added.length} nye siden sist.`; with `priorDate:null` (first brief) or `added.length === 0`, the
|
||||
marker is **absent** (and `briefSummary(ranking)` with no diff === `briefSummary(ranking, emptyDiff)` — no
|
||||
marker, so the existing `frontmatter summary === briefSummary(r)` test stays green). The summary contains **no**
|
||||
`"` and **no** `\n`.
|
||||
- **SC7 (schema boundary)** — `BRIEF_SCHEMA_VERSION === 2`; `SCHEMA_VERSION === 4`; `types.ts` untouched; a
|
||||
`brief` run does **not** change any record's `score.composite` (only `surfacedCount`/`lastSurfacedAt` move — the
|
||||
existing R3b behaviour, since `rankForBrief`/`markSurfaced` are unchanged).
|
||||
- **SC8 (determinism, incl. same-day re-run)** — two `brief` renders with the same `(store, pillars, today,
|
||||
opts, diff)` → byte-identical `.md`. End-to-end via the CLI: running `brief` **twice on the same day** (the
|
||||
second after the first wrote `${day}.md`) → byte-identical files, because `selectPriorBriefFile` excludes the
|
||||
same-day file and picks the same prior day.
|
||||
- **SC9 (CLI diff wiring — two-day sequence)** — write a day-1 brief (records `surfaced:` for its cohort), then a
|
||||
day-2 brief over a store with one **new** trend: the day-2 `.md` `## 🆕 Nytt siden sist (<day-1>)` section lists
|
||||
the new trend, the day-2 `--json` carries `diff: { priorDate:<day-1>, added:≥1, carried:…, dropped:… }`, and the
|
||||
console line appends `N nye siden sist`. A **first** run (empty dir) → `diff.priorDate === null`. A **custom
|
||||
`--out`** isolates discovery to that dir (the diff reads prior briefs only from `outDir`).
|
||||
- **SC10 (gate + wiring + de-niche)** — `bash scripts/test-runner.sh` → `FAIL=0`: trends suite green at the
|
||||
bumped `TRENDS_TESTS_FLOOR`; new **Section 16n** green (the six ASCII sentinels + non-vacuity self-test);
|
||||
`ASSERT_BASELINE_FLOOR` = **123** (117 + 6); Section 17 de-niche green; counts 29/19/27; the hook suite still
|
||||
green untouched (`node --test hooks/scripts/__tests__/*.test.mjs`).
|
||||
|
||||
## 7. Verification
|
||||
|
||||
**Deterministic:** `bash scripts/test-runner.sh` → `FAIL=0`; trends suite ≥ new floor; Section 16n self-test +
|
||||
greps pass; `ASSERT_BASELINE_FLOOR` = 123; Section 17 de-niche green; ref/agent/command counts unchanged.
|
||||
**Regression sanity:** `node --test hooks/scripts/__tests__/*.test.mjs` → still green untouched (R3e touches no
|
||||
hook). The R3c suite (`schedule.test.ts`/`run-daily.test.ts`) still green untouched (`schedule.ts`/`run-daily.sh`
|
||||
not edited).
|
||||
**Behavioural (manual):**
|
||||
1. `D=/tmp/r3e-mb-$$; S=/tmp/r3e-$$.json` (unique dir, no `rm`). Seed a store with two on-pillar trends and run a
|
||||
day-1 brief: `node --import tsx src/cli.ts brief --pillars ai,gov --out "$D" --store "$S"` → inspect the `.md`:
|
||||
the frontmatter carries `surfaced: <ids>` and `schemaVersion: 2`; the `## 🆕 Nytt siden sist` section says
|
||||
`Første brief — alt nedenfor er nytt`.
|
||||
2. `capture` a third on-pillar trend into the same store, then run a day-2 brief **with a later `today`** (seed via
|
||||
a second dated file is not possible — use `--out "$D"` so day-1's `${day}.md` is the prior; on a real next-day
|
||||
run the date advances): inspect the new `.md` → `## 🆕 Nytt siden sist (<prior-date>)` lists the new trend with
|
||||
its title, then `N båret over, M ikke vist i dag`; the `--json` shows `diff.added ≥ 1`.
|
||||
3. Re-run the **same-day** brief → the written `.md` is **byte-identical** (`diff` against the same prior file;
|
||||
`surfaced:` re-written identically) — `cmp` the two files.
|
||||
4. Confirm the seeded records' `score.composite` values are unchanged after the briefs (only `surfacedCount`/
|
||||
`lastSurfacedAt` advance) — the diff never mutates the store ranking.
|
||||
5. Confirm a pre-R3e brief (no `surfaced:` line) as the prior → `parseSurfacedFrontmatter` returns `[]` → every
|
||||
trend reads as `added` (graceful degrade, no crash).
|
||||
|
||||
## 8. Open questions for the go-gate
|
||||
|
||||
Two architectural decisions are **CONFIRMED** (operator, AskUserQuestion 2026-06-26): **SD1** persist membership
|
||||
in the brief's frontmatter (no sidecar); **SD2** `added` with titles + `dropped` as a count (`brief.ts`
|
||||
store-free). Residual decisions, all baked to the recommended default — confirm or redirect with "Go":
|
||||
- **D1 — "new" = relative to the immediately-prior brief (artifact diff), not first-ever (`surfacedCount 0`)?**
|
||||
YES (rec). It is the honest meaning of "siden sist" and catches re-emergence; it is also what unlocks `dropped`.
|
||||
Re-open only to redefine "new" as first-ever.
|
||||
- **D2 — diff against the most recent prior brief only (no `--since` baseline)?** YES (rec). The "since last" a
|
||||
daily feed needs. Re-open only to add an arbitrary historical baseline.
|
||||
- **D3 — `## 🆕 Nytt siden sist` placed before `## 🎯 Topp-treff` (delta leads)?** YES (rec). The one thing a feed
|
||||
leads with. Re-open only to place it after the ranked list (appendix) or omit the header on a first brief.
|
||||
- **D4 — summary marker ` N nye siden sist.` (suppressed on first brief / when nothing new)?** YES (rec). The
|
||||
delta the SessionStart hook surfaces for free, with no hook edit. Drop only to keep the summary minimal.
|
||||
- **D5 — `surfaced:` frontmatter always emitted (incl. `--no-mark`, incl. empty store → blank)?** YES (rec). It is
|
||||
the record the *next* diff reads; gating it on `--no-mark` or non-empty would silently break tomorrow's diff.
|
||||
Re-open only to gate it.
|
||||
- **D6 — `dropped`/`carried` render as a one-line count (no titles); `dropped` framed "ikke vist i dag"?** YES
|
||||
(rec, = SD2). Keeps `brief.ts` store-free; honest (a count cannot prove "resolved"). Re-open only to pull the
|
||||
reason-labeled follow-up into R3e now.
|
||||
- **D7 — `BRIEF_SCHEMA_VERSION` 1 → 2; store `SCHEMA_VERSION` stays 4?** YES (rec). The frontmatter gained a
|
||||
field (the first artifact-schema change since R2b); no store field. Re-open only to add a store field instead.
|
||||
- **D8 — README gets the R3e section; `trend-spotter.md` gets one prose line; the SSOT is NOT touched?** YES
|
||||
(rec). The diff is not a scoring concern — touching `trend-scoring-modes.md` would be scope creep. Re-open only
|
||||
to add an SSOT note.
|
||||
- **D9 — commit split?** Docs commit first, then **one** code commit (rec) — the diff (helpers + render + CLI
|
||||
wiring + gate) is one coherent feature. Re-open only for a helpers-then-wiring split.
|
||||
|
||||
## 9. Light-Voyage review — folded
|
||||
|
||||
Three Opus reviewers ran COLD on this brief + the plan against live `scripts/trends/` code (scope-guardian,
|
||||
brief-reviewer, plan-critic — the R3c/R3d discipline). **Verdicts:** scope-guardian **MIXED** · brief-reviewer
|
||||
**PROCEED_WITH_RISKS** · plan-critic **REWORK (0.88)**. They **converged on 2 MAJOR** (both re-verified against
|
||||
live code before folding) + 4 MINOR. Every line-cite, the floors, the regex/lex/hook safety, and the §3 scope
|
||||
fence were **confirmed correct** by all three and left untouched.
|
||||
|
||||
**MAJOR-1 — a hard schema literal breaks at the 1→2 bump (the §6/Step-1 "no existing assertion breaks" scoping
|
||||
missed it).** `tests/brief.test.ts:574` is `assert.equal(BRIEF_SCHEMA_VERSION, 1)` — a **hard literal**, not the
|
||||
constant-tracking RegExp at `:163` (`new RegExp("\\nschemaVersion: " + BRIEF_SCHEMA_VERSION + "\\n")` auto-tracks).
|
||||
It lives in the `rankForBrief — no schema/score mutation` block (`:568-577`), **outside** the frontmatter tests
|
||||
§6/plan-Step-1 enumerated, so the "every frontmatter assertion auto-tracks" claim did not cover it. **Resolution:**
|
||||
the GREEN schema bump (`BRIEF_SCHEMA_VERSION = 2`) must **also flip `:574` → `, 2)`** in the same step (plan
|
||||
Step 3). `:575` (`assert.equal(SCHEMA_VERSION, 4)`, the store schema) is untouched. Folded into plan Step 3 + R1 +
|
||||
Step 1's enumeration. (Swept live: `:574` is the *only* hard `BRIEF_SCHEMA_VERSION` literal in the suite;
|
||||
`cli.test.ts:102` asserts the **store** `persisted.schemaVersion === SCHEMA_VERSION` (4) — unrelated, stays green.)
|
||||
|
||||
**MAJOR-2 — the `--json summary` would diverge from the file frontmatter on day-2 (the "one source" invariant).**
|
||||
The file's frontmatter `summary:` is built inside `renderBrief` (`brief.ts:265`), which Step 3 routes through
|
||||
`briefSummary(ranking, diff)` → on day-2 it carries the ` N nye siden sist.` marker. But the CLI's `--json`
|
||||
`summary` field reads a **separate** `const summary = briefSummary(ranking)` (`cli.ts:350`, comment `// SAME source
|
||||
the frontmatter carries`) that Step 4 left unthreaded → no marker. `cli.test.ts:268` asserts
|
||||
`fileFrontmatter.summary === json.summary` ("one source") → would **break** on day-2. **Resolution:** Step 4
|
||||
changes `cli.ts:350` to `briefSummary(ranking, diff)` (the `diff` is in scope — Step 4 computes it between the
|
||||
ranking at `:339` and the render at `:340`). **Safe on day-1:** `priorDate === null` ⇒ the marker is suppressed ⇒
|
||||
byte-identical to today's string. Folded into plan Step 4 + the files-touched table.
|
||||
|
||||
**MINOR (folded):**
|
||||
- **M1 — `BriefDiff` is an interface (type-only export).** Anywhere it is referenced as a type (tests or `cli.ts`),
|
||||
import it with **`import type { BriefDiff }`**, never a value import — under Node16 ESM + tsx a type-only export
|
||||
is stripped from the emitted JS, so a value-import named binding fails to resolve at **module-load** (the same
|
||||
Phase-A hazard as a missing named import). The plan's Step-4 code does **not** annotate `: BriefDiff` (it infers
|
||||
from `diffSurfaced`'s return) and the SC tests pass object literals — so in practice no `BriefDiff` import is
|
||||
needed; the rule is the guardrail if one is added. Folded into plan Step 2/4.
|
||||
- **M2 — SC9's prior brief is a real renamed brief, not a hand-fixture (rename-real-write).** Replace the "pre-write
|
||||
a `<prior-date>.md` fixture carrying a `surfaced:` line" mechanism (plan Step 4 note / Phase B / Step 7) with:
|
||||
run `brief` once (writes `${today}.md` with a genuine `surfaced:` line), `mv ${today}.md → 2026-06-20.md` (a fixed
|
||||
past date) **in the same `--out`**, then run `brief` again. This (a) closes the write→read loop **clock-free** (no
|
||||
`today()` manipulation), and (b) **guarantees the prior `surfaced:` ids are real store ids** (they came from a
|
||||
real run), so `carried` is non-trivial and `added` is *exactly* the newly-captured trend — a hand-fixture risks an
|
||||
id mismatch that makes everything read as added/dropped (a weaker, possibly-vacuous test). Folded into plan Step 4
|
||||
note, Phase B, Step 7, SC9.
|
||||
- **M3 — SC1 "repeated id" wording.** `diffSurfaced` uses `currentIds.filter(...)`/`priorIds.filter(...)`, which
|
||||
**preserve** within-list duplicates — so "not double-counted" is wrong as within-list dedup. What the `Set`
|
||||
membership actually guarantees is **cross-partition disjointness** (added/carried/dropped are mutually exclusive).
|
||||
Reworded in §6 SC1 + plan Step 1/verification. (Production `surfacedIds` yields distinct ids, so within-list dups
|
||||
never arise.)
|
||||
- **M4 — §3 "empty `surfaced:`" self-contradiction.** The bare `renderBrief(ranking)` default-diff prose said it
|
||||
yields "an empty `surfaced:` reflecting the ranking" — contradictory: `surfaced:` is always
|
||||
`surfacedIds(ranking).join(",")` (**non-empty** for a non-empty ranking; blank only for an empty store),
|
||||
independent of the diff; the default empty diff only drives the **`_Første brief._`** section branch. Reworded in
|
||||
§3 (the `renderBrief` default-diff bullet).
|
||||
|
||||
**Confirmed correct by all three — left untouched:** every line-cite (`brief.ts:23/204/265/305`,
|
||||
`cli.ts:339/340/350/352`, `session-start.mjs:60-77/534-536`); the floors (`TRENDS_TESTS_FLOOR` 216 @ live `:716`,
|
||||
`ASSERT_BASELINE_FLOOR` 117 @ live `:1473` → 123 = 117 + 6 unconditional 16n emitters); Section 16m is the last
|
||||
trends section (16n sits between its `echo ""` and Section 18); the 6 ASCII sentinels are non-vacuous; the
|
||||
`/^surfaced: *([^\n]*)/m` regex, the ISO-lex compare, the `^surfaced:` ≠ `^summary:` hook-safety, the `--json`
|
||||
shape, and the same-day-determinism strict `<`.
|
||||
110
docs/research-engine/plan-re-r1.md
Normal file
110
docs/research-engine/plan-re-r1.md
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
# Plan — RE-R1: item-schema (B1) + triage-scorer (B2) as tested code
|
||||
|
||||
> **Brief:** `docs/research-engine/brief-re-r1.md`. **Slice:** RE-R1 (research-engine rung-2, slice 1).
|
||||
> **TDD-order:** RED (item + score tests as logic-RED) → GREEN (item.ts, score.ts) → GREEN (CLI + cli tests) → wire trend-spotter prose → gate floors → behavioural → land. **Counts recounted live at land, never pinned/guessed.**
|
||||
> **Light-Voyage hardened:** scope-guardian ALIGNED; brief-reviewer + plan-critic findings folded (see §Plan-critic — folded).
|
||||
|
||||
## Goal
|
||||
|
||||
Move the research engine's deterministic core — the canonical item envelope (B1) and the composite/band/threshold arithmetic (B2) — out of `agents/trend-spotter.md` prose into pure, tested TypeScript under `scripts/trends/`, behind a CLI seam the agent and a future orchestrator call. No store-schema change; the five judgment scores stay with the model; wiring `normalizeItem` into the store is R2.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | SC |
|
||||
|---|---|---|
|
||||
| `scripts/trends/src/item.ts` | **NEW** — `TrendItem` ingress envelope + `normalizeItem` / `normalizeItems` (pure, validating; no `id`) | SC1 |
|
||||
| `scripts/trends/src/score.ts` | **NEW** — `ScoreMode`, per-mode weight consts (mirror SSOT), `composite`, `band`, `triage` | SC2, SC3, SC4 |
|
||||
| `scripts/trends/src/cli.ts` | **EDIT** — add `normalize` + `score` subcommands (stdin JSON in, JSON out, exit 2 on bad invocation) | SC5 |
|
||||
| `scripts/trends/tests/item.test.ts` | **NEW** — normalize: required-field errors, whitespace/topic dedupe, publishedAt validate, batch partition | SC1 |
|
||||
| `scripts/trends/tests/score.test.ts` | **NEW** — golden composite (both modes: all-tens=10.0 + `{10,8,6,4,2}`=7.0 + weights-sum-1.0 + pinned weights), range guard, band boundaries + pinned action strings, triage gate/rank | SC2, SC3, SC4 |
|
||||
| `scripts/trends/tests/cli.test.ts` | **NEW** — subprocess: `normalize`/`score` happy path (stdin→JSON) + exit-2 bad invocation | SC5 |
|
||||
| `agents/trend-spotter.md` | **EDIT** — replace L124-137 "compute composite/bands yourself" with a pointer naming `src/cli.ts score` as the deterministic-step owner; domain-general | SC7 |
|
||||
| `scripts/test-runner.sh` | **EDIT** — `TRENDS_TESTS_FLOOR` 24→recount (stays inside deps guard); NEW unconditional Section **16g** (before Section 18); `ASSERT_BASELINE_FLOOR` 84→recount; header enumeration | SC6, SC7 |
|
||||
| `docs/research-engine/{brief,plan}-re-r1.md` | **NEW** — slice docs (TRACKED, like `docs/second-brain/*`) | — |
|
||||
| `STATE.md` | **EDIT at land** — Telling-block reconcile (floors, gate total). *Land bookkeeping, not slice scope; LOCAL-ONLY.* | — |
|
||||
|
||||
**Not touched (scope fence):** `scripts/trends/src/{store.ts,types.ts}` (no schema change, `SCHEMA_VERSION` stays 1) · `references/trend-scoring-modes.md` (SSOT unchanged) · `references/*` (no new ref doc) · `config/trends-sources.template.md` (source list — not wired in R1) · `agents/*` count (19) · `commands/*` (29) · `hooks/**` · `.gitignore` (trends lines already present).
|
||||
|
||||
## Step 1 — (RED) failing tests for B1 + B2
|
||||
|
||||
Write `tests/item.test.ts` and `tests/score.test.ts` against not-yet-existing modules. Make them **logic-RED**, not import-RED, with a stub strategy split by assertion type:
|
||||
- **arithmetic / "returns X" tests** → stub returns a *wrong constant* (so the equality assertion fails on value, not on a throw);
|
||||
- **"should throw" validation tests** → stub returns a valid-looking value (so the `assert.throws` fails because nothing threw).
|
||||
|
||||
`item.test.ts`: well-formed raw → canonical item (fields verbatim; topics normalized+deduped); missing/empty `source|title|url` → `{ok:false}` naming the field; whitespace collapse via the same normalization as `store.normalizeField`; `publishedAt` present-and-ISO → kept, absent → undefined, present-and-invalid → `{ok:false}`; `normalizeItems` partitions a batch into `{items, errors}`; the canonical item carries **no `id`**.
|
||||
|
||||
`score.test.ts`: `composite` for both modes on (a) all-tens → exactly **10.0**, (b) the asymmetric vector `{10,8,6,4,2}` in dimension order → **7.0**; a dimension = 0 or 11 → throws; a **pinned-weights** assertion (each mode's five constants equal the SSOT values, Σ=1.0); `band` at 8.0 / 6.0 / 4.0 / 2.0 / below → correct priority + **the exact SSOT action string** (kortform + long-form); `triage` with threshold 4.0 → kept (≥4.0, sorted composite-desc, annotated composite+band) and dropped (<4.0).
|
||||
|
||||
**RED proof (record in commit):** `(cd scripts/trends && npm test)` → new cases fail with assertion errors (not module-not-found).
|
||||
|
||||
## Step 2 — (GREEN) implement `item.ts`
|
||||
|
||||
Implement the `TrendItem` ingress envelope `{source,title,url,publishedAt?,topics[],summary?}` + `normalizeItem`/`normalizeItems` to pass Step 1's item cases. Reuse `normalizeField` (import from `./store.js`) for whitespace; topic normalize + dedupe. `publishedAt` validated with a strict ISO-date check; **carried, not persisted** (comment: the *source's* publish date, forward-compat for B4 freshness — distinct from the store's `capturedAt`). **Do not derive or carry `id`** — the store owns it via `addTrend`→`trendId`; the envelope has no id field. **Do not wire to the store** — the item→`TrendInput` bridge (`capturedAt` injection) is R2.
|
||||
|
||||
## Step 3 — (GREEN) implement `score.ts`
|
||||
|
||||
Encode the two weight-sets as `const` records mirroring `trend-scoring-modes.md` (header comment: the SSOT is the human source, "ordering is the signal, not a measured coefficient", + the SSOT path). Implement `composite` (validate each dimension ∈[1,10], weighted sum), `band` (the five-range map → `{priority, kortformAction, longformAction}` using the **exact SSOT action strings**), and `triage` (`kept`/`dropped` + composite-desc sort + per-item composite/band annotation). Make Step 1's score cases green (incl. the pinned-weights + pinned-action-string assertions).
|
||||
|
||||
## Step 4 — (GREEN) CLI subcommands + `cli.test.ts`
|
||||
|
||||
Add `normalize` and `score` to `cli.ts`'s `main` dispatch. Both **read the JSON payload from stdin** (not a flag — the existing `--json` is an *output* toggle and must keep that meaning) and **print JSON to stdout**. `normalize` → canonical items / `{ok:false}` error entries. `score` → `triage` with `--mode`/`--threshold`, prints `{kept, dropped}`. Exit **2** on a malformed invocation (unparseable stdin, missing required flag) via the existing `usage()` path; **0** otherwise. Write `tests/cli.test.ts` (subprocess: spawn `node --import tsx src/cli.ts <sub>` with a piped stdin payload) covering a happy path + an exit-2 bad-invocation for each new subcommand.
|
||||
|
||||
## Step 5 — wire `trend-spotter.md` (prose pointer)
|
||||
|
||||
Replace the L124-137 "score 5 dims, take the weighted composite, apply the bands yourself" instruction with: the agent supplies the five 1–10 judgment scores, then pipes them to **`scripts/trends/src/cli.ts score`** (the deterministic owner of composite + bands + threshold). The replacement prose **must contain the literal `src/cli.ts score`** (the exact string Step 6's `grep -qF` matches). Keep it domain-general — no vendor/sector tokens (Section 17). The agent still owns mode selection and the qualitative scoring.
|
||||
|
||||
## Step 6 — gate: floors + new unconditional section
|
||||
|
||||
In `scripts/test-runner.sh`:
|
||||
- Bump `TRENDS_TESTS_FLOOR` 24 → **live recount** after Steps 1–4 (24 + new item/score/cli cases). It **stays inside** the `if [ -x scripts/trends/node_modules/.bin/tsx ]` deps guard (conditional — do not hoist it out; that would break fresh-clone safety).
|
||||
- Add **Section 16g** (label it 16g; place it **after Section 17 / before Section 18**, since Section 18 anti-erosion must run last). Three **unconditional**, deps-absent-safe checks (pure `grep`, no `tsx`): (1) `score.ts` encodes both `kortform` and `long-form` weight-sets; (2) `grep -qF "src/cli.ts score" agents/trend-spotter.md`; (3) a non-vacuity self-test for those greps (house pattern, per Sections 16c–17).
|
||||
- Bump `ASSERT_BASELINE_FLOOR` 84 → **live recount** (= 84 + the count of new unconditional `pass`/`fail` emitters in 16g; expected ~87 with the self-test, **recounted at land, not pinned**). Update the section-header enumeration comment.
|
||||
|
||||
## Step 7 — behavioural verification
|
||||
|
||||
`(cd scripts/trends && npm install)` if needed, then run (verified invocation form, not `npm run start`):
|
||||
`echo '<3-item sample>' | node --import tsx src/cli.ts normalize` and
|
||||
`echo '<scored sample>' | node --import tsx src/cli.ts score --mode kortform --threshold 4.0`;
|
||||
confirm by hand that one ≥4.0 item is kept (correct band/action) and one <4.0 is dropped. Run full `bash scripts/test-runner.sh` → `FAIL=0`.
|
||||
|
||||
## Step 8 — land
|
||||
|
||||
Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor, gate total). Commit order (house style: `feat … [skip-docs]` for code; plain for docs): **(1)** docs commit `docs/research-engine/{brief,plan}-re-r1.md` (no suffix, tracked); **(2)** code commit `scripts/trends/*` + `agents/trend-spotter.md` + `scripts/test-runner.sh` with `[skip-docs]`. Check the push window (`date '+%u %H:%M'`); `origin` is the PUBLIC `open/` remote → **confirm with operator before push**. No version bump (additive; `v0.5.2` dev).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|---|---|---|---|
|
||||
| — | RED proof | `(cd scripts/trends && npm test)` after Step 1 | new cases fail on assertion (logic-RED), not module-not-found |
|
||||
| SC1 | normalize | `npm test` (item.test.ts) | required-field `{ok:false}` + dedupe + publishedAt + batch-partition cases green; no `id` on item |
|
||||
| SC2 | composite | `npm test` (score.test.ts) | all-tens=10.0 both modes; `{10,8,6,4,2}`=7.0 both modes; pinned weights; range guard throws |
|
||||
| SC3 | bands | `npm test` (score.test.ts) | 8.0/6.0/4.0/2.0 boundaries → correct priority + exact SSOT action string |
|
||||
| SC4 | triage | `npm test` (score.test.ts) | kept ≥ threshold ranked desc + annotated; dropped below |
|
||||
| SC5 | CLI | `npm test` (cli.test.ts) + manual stdin run | JSON out; exit 2 malformed invocation / 0 well-formed |
|
||||
| SC6 | gate | `bash scripts/test-runner.sh` | `FAIL=0`; trends ≥ new floor; ASSERT floor = live recount |
|
||||
| SC7 | wiring + de-niche | Section 16g greps + Section 17 | `src/cli.ts score` present in trend-spotter.md; no vendor/sector tokens; counts 19/29/27 |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — SSOT/code drift.** Weights, band thresholds, AND the ten action strings now live in both `trend-scoring-modes.md` and `score.ts`. *Mitigated:* `score.test.ts` pins all three (weights + Σ=1.0 + thresholds + exact action strings) against the SSOT values, with an SSOT-path comment naming the markdown as the human source. (A markdown-table-parsing cross-check is deferred — out of scope.)
|
||||
- **R2 — `publishedAt` carried but not persisted; `capturedAt` not on the item.** Could read as a dangling field. *Mitigated:* explicit comment (source publish-date, forward-compat for B4) + the brief's non-goal; the store bridge (capturedAt injection) is explicitly R2; the scorer does not depend on either.
|
||||
- **R3 — editing `trend-spotter.md` could trip the de-niche guard.** *Mitigated:* Section 17 runs in the gate; replacement prose is pillar-driven and vendor/sector-free.
|
||||
- **R4 — new gate checks must survive a deps-absent fresh clone.** *Mitigated:* the three new checks are pure `grep`/self-test on tracked source files (no `tsx`), so they are unconditional and safe; the arithmetic proof stays inside the deps-gated suite; `TRENDS_TESTS_FLOOR` stays inside the deps guard.
|
||||
- **R5 — default threshold (4.0) may not match operator intent.** *Mitigated:* single config param; brief open question #1 settles it at the go-gate.
|
||||
- **R6 — CLI `--json` semantic collision.** *Mitigated:* new subcommands take payload via **stdin**, leaving `--json` as the existing output toggle; cli.test.ts encodes the stdin contract.
|
||||
|
||||
## Plan-critic — folded
|
||||
|
||||
plan-critic returned **REVISE** (1 blocker, 6 majors, 4 minors); brief-reviewer **PROCEED_WITH_RISKS**; scope-guardian **ALIGNED**. Resolution, each verified against live code:
|
||||
|
||||
- **[BLOCKER] `TrendInput` shape mismatch** (`store.ts:26-33` requires `capturedAt`, no `publishedAt`). ✅ Step 2 no longer claims a direct map; the item→store bridge is deferred to R2; envelope carries no `id`.
|
||||
- **[MAJOR] gate-section placement / Section-18-last** ✅ Step 6 pins **16g, before Section 18**.
|
||||
- **[MAJOR] `ASSERT_BASELINE_FLOOR` hard-pinned 86** ✅ Step 6 now **live recount** (~+3 with the house self-test), not pinned.
|
||||
- **[MAJOR] `TRENDS_TESTS_FLOOR` could be hoisted out of the deps guard** ✅ Step 6 states it stays conditional.
|
||||
- **[MAJOR] CLI `--json` input/output overload** (`cli.ts:79` output toggle) ✅ Step 4 reads payload from **stdin**.
|
||||
- **[MAJOR] band action-string drift unguarded** ✅ Step 1/3 + R1 pin the thresholds + action strings.
|
||||
- **[MAJOR] Step 5/6 grep literal unpinned** ✅ pinned to `src/cli.ts score` in both steps.
|
||||
- **[MINOR] RED-stub strategy** ✅ Step 1 splits stub by assertion type. **[MINOR] `npm run start` unverified** ✅ Step 7 uses `node --import tsx src/cli.ts`. **[MINOR] commit grouping** ✅ Step 8 fixes order. **[MINOR] empty folded sections** ✅ filled.
|
||||
- **[scope-guardian MINOR] STATE.md + `config/trends-sources.template.md`** ✅ STATE.md added as a land-bookkeeping row; template added to the scope fence.
|
||||
|
||||
**scope-guardian — ALIGNED:** every SC1–SC7 traces to a step; zero scope creep; every NON-goal respected.
|
||||
192
docs/research-engine/plan-re-r2a.md
Normal file
192
docs/research-engine/plan-re-r2a.md
Normal file
|
|
@ -0,0 +1,192 @@
|
|||
# Plan — RE-R2a: capture bridge (item→store) + publishedAt persistence
|
||||
|
||||
> **Brief:** `docs/research-engine/brief-re-r2a.md`. **Slice:** RE-R2a (research-engine rung-2, slice 2 — R2 data layer).
|
||||
> **TDD-order:** RED (migration + bridge + capture tests as logic-RED) → GREEN (`types.ts`/`store.ts` migration + persist) →
|
||||
> GREEN (`item.ts` bridge) → GREEN (`cli.ts capture` + tests) → wire `trend-spotter.md` → gate floors → behavioural → land.
|
||||
> **Counts recounted live at land, never pinned/guessed.**
|
||||
> **Light-Voyage hardened:** scope-guardian / brief-reviewer / plan-critic findings folded (see §Plan-critic — folded).
|
||||
|
||||
## Goal
|
||||
|
||||
Build the item→store bridge RE-R1 deferred: a pure `itemToInput` mapping the validated `TrendItem` envelope to a
|
||||
store input (injecting `capturedAt`, carrying `publishedAt`), a `publishedAt` schema bump (v1→v2) with a lossless
|
||||
migrate-on-load, and a `capture` CLI that closes the poll→normalize→store loop. No scoring change; no hook touch; the
|
||||
dated brief + surfacing are R2b.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | SC |
|
||||
|---|---|---|
|
||||
| `scripts/trends/src/types.ts` | **EDIT** — `SCHEMA_VERSION 1→2`; `publishedAt?: string` on `TrendRecord` (after `capturedAt`, with the capturedAt-distinction comment) | SC3 |
|
||||
| `scripts/trends/src/store.ts` | **EDIT** — `TrendInput` +`publishedAt?`; `addTrend` persists it (conditional-spread, first-sight kept on merge); `loadStore` forward migrate-on-load (`max(onDisk, SCHEMA_VERSION)`) | SC2, SC3 |
|
||||
| `scripts/trends/src/item.ts` | **EDIT** — pure `itemToInput(item, capturedAt): TrendInput` (injects capturedAt; no id; no re-validate) | SC1 |
|
||||
| `scripts/trends/src/cli.ts` | **EDIT** — `capture` subcommand only (stdin→normalize→bridge→addTrend→saveStore→summary; exit 2 bad invocation; `--json`). *(`add --published-at` deferred at light-Voyage — see brief §4.)* | SC4 |
|
||||
| `scripts/trends/tests/store.test.ts` | **EDIT** — migration cases (v1→v2 lossless, idempotent, round-trip) + `publishedAt` persist/first-sight-merge cases | SC2, SC3 |
|
||||
| `scripts/trends/tests/item.test.ts` | **EDIT** — `itemToInput` mapping cases (capturedAt inject, carry-through, no id, absent publishedAt omitted) | SC1 |
|
||||
| `scripts/trends/tests/cli.test.ts` | **EDIT** — `capture` happy path (stdin→store), duplicate/merge, content-invalid in errors[], exit-2 bad invocation, `--json` summary | SC4 |
|
||||
| `scripts/trends/README.md` | **EDIT** — add `publishedAt?` to the record-shape block + a `capture` example (honest schema/CLI doc) | — |
|
||||
| `agents/trend-spotter.md` | **EDIT (Open Q#1, default)** — Step 4.5 `add`→`capture`; carries literal `src/cli.ts capture`; domain-general | SC6 |
|
||||
| `scripts/test-runner.sh` | **EDIT** — `TRENDS_TESTS_FLOOR` 62→recount (stays inside deps guard); NEW unconditional **Section 16h** (before Section 18); `ASSERT_BASELINE_FLOOR` 87→recount; anti-erosion header enumeration | SC5, SC6 |
|
||||
| `docs/research-engine/{brief,plan}-re-r2a.md` | **NEW** — slice docs (TRACKED, like `docs/second-brain/*`) | — |
|
||||
| `STATE.md` | **EDIT at land** — Telling-block reconcile (trends floor, ASSERT floor, gate total). *Land bookkeeping, not slice scope; LOCAL-ONLY.* | — |
|
||||
|
||||
**Not touched (scope fence):** `scripts/trends/src/score.ts` (no scoring change) · `references/trend-scoring-modes.md` + `references/*` (SSOT unchanged, no new ref) · `hooks/**` (no surfacing — R2b) · `config/trends-sources.template.md` · no new `.ts` source file (bridge in `item.ts`) · `agents/*` count (19) · `commands/*` (29) · `.gitignore` (trends lines present).
|
||||
|
||||
## Step 1 — (RED) failing tests for migration + bridge + capture
|
||||
|
||||
Extend the three existing test files against the not-yet-changed code. Logic-RED (not import-RED), stub-by-assertion-type.
|
||||
**Critical RED-vs-GREEN-guard split** (plan-critic blocker): against old code (`SCHEMA_VERSION=1`, `loadStore` returns
|
||||
`parsed.schemaVersion ?? SCHEMA_VERSION`), only some assertions actually fail:
|
||||
- **Genuinely RED** (old code fails): v1-fixture load → `schemaVersion===2` (old returns 1); v1 round-trip
|
||||
`loadStore`→`saveStore` writes `schemaVersion:2` (old writes 1); `addTrend({…, publishedAt})` persists it (old
|
||||
drops it); `itemToInput` carries/injects correctly (add a thin **throwing** stub for the not-yet-existing export so
|
||||
the case fails on assertion, not on `undefined is not a function`).
|
||||
- **GREEN-only regression guards** (pass against old code — NOT labelled RED): a v2 fixture load → `schemaVersion===2`
|
||||
(old already returns 2 via `?? `); **missing** `schemaVersion` → 2 (old `??` already yields current); **non-numeric**
|
||||
`schemaVersion` → 2; empty/absent store → `{schemaVersion:2,trends:[]}`. These are written in Step 1 but documented
|
||||
as regression guards, so the RED proof is not falsely claimed for them.
|
||||
|
||||
`store.test.ts`: the RED migration cases above + the regression-guard cases; `addTrend` without `publishedAt` omits the
|
||||
key; re-capture (same title+url) leaves existing `publishedAt` unchanged and only unions topics; **absent→present**
|
||||
re-capture (first sighting lacked `publishedAt`, re-capture carries one) does **NOT** add it (no back-fill, Open Q#2),
|
||||
`merged` reflects topic change alone.
|
||||
`item.test.ts`: `itemToInput(item,"2026-06-24")` → `capturedAt` injected (`==="2026-06-24"`), all fields carried verbatim,
|
||||
**no `id`**, item without `publishedAt` → input without the key; **field-confusion guard:** an item whose `publishedAt`
|
||||
differs from the injected date → `result.capturedAt !== result.publishedAt`.
|
||||
`cli.test.ts`: `capture` happy (one valid item piped → store gains it, `added:1`); a batch with one invalid item →
|
||||
`errors[]` carries it, valid one added, exit 0; the summary tally **sums to the input size**; the captured record's
|
||||
`capturedAt` matches `/^\d{4}-\d{2}-\d{2}$/` and `!==` the item's `publishedAt`; empty/unparseable stdin → exit 2;
|
||||
`--json` emits the summary object.
|
||||
|
||||
**RED proof (record in commit):** `(cd scripts/trends && npm test)` → the **genuinely-RED** cases fail on assertion
|
||||
(logic-RED), not module-not-found; the regression-guard cases may pass pre-change (documented, not claimed RED).
|
||||
|
||||
## Step 2 — (GREEN) schema migration: `types.ts` + `store.ts`
|
||||
|
||||
`types.ts`: `SCHEMA_VERSION = 2`; add `publishedAt?: string` to `TrendRecord` after `capturedAt` with the
|
||||
distinction comment (source publish-date; forward-compat for B4; distinct from capturedAt).
|
||||
`store.ts`: `TrendInput` gains `publishedAt?`; `addTrend` adds `...(input.publishedAt !== undefined ? { publishedAt: input.publishedAt } : {})` to the new-record literal (after `capturedAt`); the merge branch is **unchanged** (topics union only — first-sight `publishedAt` kept, no back-fill). `loadStore` returns `schemaVersion: Math.max(onDisk, SCHEMA_VERSION)` where `onDisk = typeof parsed.schemaVersion === "number" ? parsed.schemaVersion : SCHEMA_VERSION` (forward-only stamp handling string/`NaN`/absent → current; never crashes); **the existing `Array.isArray(parsed.trends) ? … : []` coercion (`store.ts:79`) is preserved verbatim — a corrupt `trends` field stays out of the losslessness claim**. Make Step 1's RED migration + persist cases (and the regression guards) green.
|
||||
|
||||
## Step 3 — (GREEN) bridge: `itemToInput` in `item.ts`
|
||||
|
||||
Add `import type { TrendInput } from "./store.js";` (item.ts already imports the `normalizeField` *value* from there;
|
||||
this adds the *type* — dependency direction `item.ts → store.ts` stays acyclic). Then **replace the Step-1 throwing
|
||||
stub** with `export function itemToInput(item: TrendItem, capturedAt: string): TrendInput` returning
|
||||
`{ source, title, url, topics: [...item.topics], capturedAt, ...(item.publishedAt !== undefined ? { publishedAt: item.publishedAt } : {}), ...(item.summary !== undefined ? { summary: item.summary } : {}) }`. No `id`; no re-validation (the envelope is already validated). Confirm no throwing stub survives into GREEN. Make Step 1's bridge cases green.
|
||||
|
||||
## Step 4 — (GREEN) CLI `capture` + `cli.test.ts`
|
||||
|
||||
Add `capture` to `cli.ts`'s `main` dispatch: read stdin via the existing `readStdinJson()` (its empty/unparseable path
|
||||
already does `usage()`→exit 2); `Array.isArray(payload) ? normalizeItems(payload) : normalizeItem(payload)`; for each
|
||||
valid item, `itemToInput(item, today())` → `addTrend(store, res.store…)`; `saveStore` once. **Tally derived from
|
||||
`AddResult {added, merged}` (no `duplicates` field, `store.ts:35-41`):** `added += res.added ? 1 : 0`;
|
||||
`merged += (!res.added && res.merged) ? 1 : 0`; `duplicates += (!res.added && !res.merged) ? 1 : 0`. Human summary by
|
||||
default; `--json` prints `{added, duplicates, merged, errors}`. **`add --published-at` is NOT added (deferred).** Write
|
||||
the `cli.test.ts` cases (subprocess: `node --import tsx src/cli.ts capture` with piped stdin + a `--store` temp path),
|
||||
including an explicit `added + merged + duplicates + errors.length === payload.length` assertion and a
|
||||
`capturedAt` shape (`/^\d{4}-\d{2}-\d{2}$/`) + `!== publishedAt` check. **`capturedAt` *exact-value* assertions live in
|
||||
`item.test.ts` (injected fixed date), never in `cli.test.ts` (which reads the wall clock — would be flaky).**
|
||||
|
||||
## Step 5 — wire `trend-spotter.md` (Open Q#1, default) + README
|
||||
|
||||
Replace Step 4.5's N× `add` block (`trend-spotter.md:282-301`) with: build a raw-item JSON batch (the same items
|
||||
already scored), pipe it to **`scripts/trends/src/cli.ts capture`** in one call — it normalizes + folds + persists
|
||||
`publishedAt`. Replacement prose **must contain the literal `src/cli.ts capture`** (Section 16h `grep -qF`). Keep the
|
||||
"skip silently if no deps" escape hatch + domain-general phrasing (no vendor/sector tokens — Section 17). Update
|
||||
`scripts/trends/README.md`: add `publishedAt?` to the record-shape block, add a `capture` CLI example, **and correct
|
||||
the `add` framing** — `README.md:47-64` currently calls `add` "the capture path"; after the re-point `add` is the
|
||||
**manual single-trend** path and `capture` is the **normalizing batch** path. Fix the framing, don't just append (else
|
||||
the README contradicts the new agent wiring).
|
||||
|
||||
## Step 6 — gate: floors + new unconditional Section 16h
|
||||
|
||||
In `scripts/test-runner.sh`:
|
||||
- Bump `TRENDS_TESTS_FLOOR` 62 → **live recount** after Steps 1–4. Stays **inside** the `if [ -x …/tsx ]` deps guard
|
||||
(conditional — do not hoist). **Also update the inline breakdown comment** at `test-runner.sh:695`
|
||||
(`# store 24 + RE-R1: item 18 + score 16 + cli 4`) — extend it with the R2a additions (e.g.
|
||||
`+ RE-R2a: store +N + item +M + cli +K`), so the comment doesn't drift from the number (the exact erosion the
|
||||
anti-erosion section guards against).
|
||||
- Add **Section 16h** ("Trends Capture Wiring"), mirroring Section 16g's three-check shape, **after Section 17 /
|
||||
before Section 18**. Three **unconditional**, deps-absent-safe checks (pure `grep`/self-test, no `tsx`):
|
||||
(1) a non-vacuity self-test (accept a probe carrying `src/cli.ts capture`, reject one without it);
|
||||
(2) `grep -qF 'command === "capture"' scripts/trends/src/cli.ts` — deps-absent proof the handler exists (`grep -qF`
|
||||
exact, mirroring 16g lines 1058/1064 — not `grep -E`);
|
||||
(3) `grep -qF "src/cli.ts capture" agents/trend-spotter.md`.
|
||||
- Bump `ASSERT_BASELINE_FLOOR` 87 → **live recount** (= 87 + the new unconditional emitters in 16h; expected ~90,
|
||||
**recounted at land, not pinned**). Update the Section-18 anti-erosion header enumeration comment.
|
||||
|
||||
## Step 7 — behavioural verification
|
||||
|
||||
`(cd scripts/trends && npm install)` if needed, then:
|
||||
`echo '[{"source":"tavily","title":"X","url":"https://e/x","topics":["a"],"publishedAt":"2026-06-20"},{"title":"bad"}]' | node --import tsx src/cli.ts capture --store /tmp/r2a-trends.json --json` →
|
||||
confirm `added:1`, the invalid entry in `errors[]`, and `node … list --store /tmp/r2a-trends.json --json` shows the
|
||||
persisted `publishedAt`. Author a v1 fixture (`{"schemaVersion":1,"trends":[{…no publishedAt}]}`), run `status --json`
|
||||
→ confirm count intact (migration lossless). Run full `bash scripts/test-runner.sh` → `FAIL=0`.
|
||||
|
||||
## Step 8 — land
|
||||
|
||||
Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor, gate total). Commit
|
||||
order (house style): **(1)** docs commit `docs/research-engine/{brief,plan}-re-r2a.md` (no suffix, tracked); **(2)**
|
||||
code commit `scripts/trends/*` + `agents/trend-spotter.md` + `scripts/trends/README.md` + `scripts/test-runner.sh`
|
||||
with `[skip-docs]`. Push freely (window restriction lifted 2026-06-24; gitleaks runs at commit; `origin` = PUBLIC
|
||||
`open/` — STATE/`*.local.*` never pushed). No version bump (additive; `v0.5.2` dev).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|---|---|---|---|
|
||||
| — | RED proof | `(cd scripts/trends && npm test)` after Step 1 | new cases fail on assertion (logic-RED), not module-not-found |
|
||||
| SC1 | bridge | `npm test` (item.test.ts) | capturedAt injected (`==="2026-06-24"`); fields carried; no `id`; absent publishedAt omitted; `capturedAt !== publishedAt` (field-confusion guard) |
|
||||
| SC2 | persist | `npm test` (store.test.ts) | publishedAt persisted when present / omitted when absent; first-sight kept on merge; absent→present re-capture does NOT back-fill |
|
||||
| SC3 | migrate | `npm test` (store.test.ts) | v1→v2 lossless (records intact, no publishedAt invented); missing/non-numeric schemaVersion → 2; empty store → v2/[]; idempotent; round-trip writes v2 |
|
||||
| SC4 | capture | `npm test` (cli.test.ts) + manual | normalize+fold; errors[] for invalid; tally sums to input size; capturedAt today-shaped & ≠ publishedAt; exit 2 bad stdin / 0 well-formed; `--json` summary |
|
||||
| SC5 | gate | `bash scripts/test-runner.sh` | `FAIL=0`; trends ≥ new floor; Section 16h green; ASSERT floor = live recount |
|
||||
| SC6 | wiring + de-niche | Section 16h greps + Section 17 | `src/cli.ts capture` in trend-spotter.md; no vendor/sector tokens; counts 19/29/27 |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — migration eats or rewrites records.** *Mitigated:* migrate-on-load is a version *stamp* only (`max(onDisk,
|
||||
current)`); records pass through untouched; SC3 pins lossless + idempotent + round-trip on a real v1 fixture.
|
||||
- **R2 — `publishedAt` first-sight vs back-fill ambiguity.** *Mitigated:* brief Open Q#2 settles it at first-sight
|
||||
(no back-fill); SC2 pins "existing publishedAt unchanged on re-capture"; back-fill explicitly deferred.
|
||||
- **R3 — `capture` exit-code semantics drift** (content-invalid item leaking into a non-zero exit). *Mitigated:* SC4
|
||||
pins exit 2 = malformed invocation only; content-invalid → `errors[]` at exit 0; mirrors `normalize`/`score`.
|
||||
- **R4 — editing `trend-spotter.md` trips the de-niche guard.** *Mitigated:* Section 17 runs in the gate; replacement
|
||||
prose is pillar/source-driven and vendor/sector-free; only the store-fold mechanism changes (`add`→`capture`).
|
||||
- **R5 — new gate checks must survive a deps-absent fresh clone.** *Mitigated:* Section 16h is pure `grep`/self-test on
|
||||
tracked source (no `tsx`) → unconditional; `TRENDS_TESTS_FLOOR` stays inside the deps guard.
|
||||
- **R6 — future schema downgrade (on-disk > current) silently drops unknown fields on save.** *Mitigated/deferred:*
|
||||
cannot happen pre-v3; `max()` already refuses to downgrade the stamp; field-preservation-on-save is an R-future
|
||||
concern, noted not handled (no impossible-scenario code).
|
||||
|
||||
## Plan-critic — folded
|
||||
|
||||
plan-critic returned **REVISE** (1 blocker, 5 majors, 5 minors); brief-reviewer **PROCEED_WITH_RISKS**;
|
||||
scope-guardian **ALIGNED**. Resolution, each verified against live code:
|
||||
|
||||
- **[BLOCKER] "v2 load idempotent" is not RED** (`store.ts:78` `?? ` already returns 2 for a v2 fixture under old
|
||||
`SCHEMA_VERSION=1`). ✅ Step 1 now splits **genuinely-RED** (v1→2 load + round-trip + addTrend-persist + itemToInput)
|
||||
from **GREEN-only regression guards** (v2-idempotence, missing/non-numeric/empty); the RED proof is claimed only for
|
||||
the former.
|
||||
- **[MAJOR] capture tally mis-maps onto `AddResult`** (no `duplicates` field, `store.ts:35-41`). ✅ Step 4 pins the
|
||||
exact derivation (`added`/`merged`/`duplicates` from `res.added`+`res.merged`) + a `sum === payload.length` test.
|
||||
- **[MAJOR] `TRENDS_TESTS_FLOOR` breakdown comment (`:695`) left stale.** ✅ Step 6 now extends the inline breakdown
|
||||
comment alongside the number.
|
||||
- **[MAJOR] `import isValidIso` impossible** (`item.ts:51` private). ✅ `add --published-at` **deferred entirely**
|
||||
(brief §4) — no export-vs-inline decision, `item.ts` edit stays `itemToInput`-only.
|
||||
- **[MAJOR] README `add`-as-capture framing would contradict the re-point.** ✅ Step 5 corrects the framing
|
||||
(`add` = manual single-trend; `capture` = normalizing batch), not just appends.
|
||||
- **[MAJOR] empty "folded" placeholders.** ✅ this section + brief §9 filled.
|
||||
- **[MINOR] stub cleanup unstated** ✅ Step 3 states the throwing stub is **replaced**, none survives into GREEN.
|
||||
**[MINOR] `today()` capturedAt untestable in cli.test** ✅ Step 4 routes exact-value assertions to `item.test.ts`,
|
||||
cli.test asserts shape + `≠ publishedAt`. **[MINOR] 16h grep flag** ✅ Step 6 specifies `grep -qF`.
|
||||
**[MINOR] non-numeric schemaVersion untested** ✅ added to Step 1 regression guards + SC3. **[MINOR] README under
|
||||
`[skip-docs]`** ✅ kept in the code commit (it documents the shipped code, like R1's `trend-spotter.md`); noted.
|
||||
- **[brief-reviewer MAJOR] capturedAt injection unverified at the new ingress** ✅ SC1 + SC4 field-confusion guards.
|
||||
**[brief-reviewer MAJOR] "lossless" over-claim** ✅ §5/§3 scope it to well-formed stores; malformed-`trends`
|
||||
coercion unchanged + out of scope. **[brief-reviewer MINOR] absent→present back-fill** ✅ pinned by an SC2 test.
|
||||
- **[plan-critic headless-readiness 60]** N/A — R2a is executed **in-session, operator-driven** (driftsmodell), not
|
||||
as a headless autonomous run, so per-step revert/halt clauses aren't needed (R1's plan had none either).
|
||||
|
||||
**scope-guardian — ALIGNED:** every SC1–SC6 traces to a step; zero creep (README is in-change documentation); every
|
||||
§4 non-goal held (no hook touch, no brief artifact, no relevance/saturation/status, no back-fill, no scoring change).
|
||||
282
docs/research-engine/plan-re-r2b.md
Normal file
282
docs/research-engine/plan-re-r2b.md
Normal file
|
|
@ -0,0 +1,282 @@
|
|||
# Plan — RE-R2b: dated morning-brief artifact + session-start surfacing
|
||||
|
||||
> **Brief:** `docs/research-engine/brief-re-r2b.md`. **Slice:** RE-R2b (research-engine rung-2 — R2 visible layer).
|
||||
> **TDD-order:** RED (`brief.ts` rank/render/path + `cli.ts brief` tests as logic-RED) → GREEN (`brief.ts` pure
|
||||
> functions) → GREEN (`cli.ts brief` subcommand) → GREEN (`session-start.mjs` surfacing + hook test) → wire
|
||||
> `trend-spotter.md` + README → gate floors + Section 16i → behavioural → land.
|
||||
> **Counts recounted live at land, never pinned/guessed.**
|
||||
> **Light-Voyage hardened:** scope-guardian / brief-reviewer / plan-critic findings folded (see §Plan-critic — folded).
|
||||
|
||||
## Goal
|
||||
|
||||
Make the accumulated, publish-dated store **visible**: a pure `rankForBrief` (pillar-overlap → recency over the
|
||||
store, with a `publishedAt ?? capturedAt` freshness window) + a pure `renderBrief` (a dated Markdown artifact
|
||||
with a hook-surfaceable `summary` frontmatter), a `brief` CLI that writes the dated file, and a zero-tsx
|
||||
session-start surfacing of the latest brief. No store schema change; no scoring change; no scheduler — the
|
||||
autonomous trigger + freshness-as-seen-log + relevance scoring stay R3.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | SC |
|
||||
|---|---|---|
|
||||
| `scripts/trends/src/brief.ts` | **NEW** — pure `rankForBrief` + `renderBrief` + `briefSummary` (single summary source) + `defaultBriefDir` (derived from `defaultStorePath`) + `BriefRanking`/`BriefEntry` types + `BRIEF_SCHEMA_VERSION` | SC1, SC2, SC3 |
|
||||
| `scripts/trends/src/cli.ts` | **EDIT** — `brief` subcommand (flag-driven: store→rank→render→write `<outDir>/<today>.md`; `--pillars`/`--fresh-days`/`--out`/`--store`/`--json`; exit 2 bad invocation) + `usage()`/header doc line + import from `brief.js` | SC4 |
|
||||
| `scripts/trends/tests/brief.test.ts` | **NEW** — `rankForBrief` grouping/freshness/sort + `renderBrief` frontmatter/`summary`/empty-case/determinism + `defaultBriefDir` | SC1, SC2, SC3 |
|
||||
| `scripts/trends/tests/cli.test.ts` | **EDIT** — `brief` happy path (writes dated file, prints path), `--json` summary, `--fresh-days` bad → exit 2, empty `--pillars` → exit 0 no-match brief, `--out` override | SC4 |
|
||||
| `hooks/scripts/session-start.mjs` | **EDIT** — module-private `latestMorningBrief(briefDir)` (zero-tsx readdir+read+`extractYaml`) + an unconditional `## Morning Brief` block after the brain nudge (`:500-504`), brief dir = `join(getDataRoot('trends'),'morning-brief')` | SC5 |
|
||||
| `hooks/scripts/__tests__/session-start-morning-brief.test.mjs` | **NEW** — subprocess + `LINKEDIN_STUDIO_DATA` fixture (mirrors `session-start-trends-staleness.test.mjs`): brief present → block surfaces; absent → no block/no crash; zero-tsx | SC5 |
|
||||
| `agents/trend-spotter.md` | **EDIT (Open Q#1, default)** — after Step 4.5 `capture`, run `brief --pillars <scanned pillars>`; carries literal `src/cli.ts brief`; domain-general | SC6 |
|
||||
| `scripts/trends/README.md` | **EDIT** — document `brief` subcommand + `trends/morning-brief/YYYY-MM-DD.md` artifact + frontmatter shape | — |
|
||||
| `scripts/test-runner.sh` | **EDIT** — `TRENDS_TESTS_FLOOR` 79→recount (inside deps guard) + inline breakdown comment (`:697`); NEW unconditional **Section 16i** (after 16h / before 18); `ASSERT_BASELINE_FLOOR` 90→recount; header-enumeration comment (`:33-46`) | SC6 |
|
||||
| `docs/research-engine/{brief,plan}-re-r2b.md` | **NEW** — slice docs (TRACKED, like `docs/second-brain/*`) | — |
|
||||
| `STATE.md` | **EDIT at land** — Telling-block reconcile (trends floor, ASSERT floor, hook-suite, gate total). *Land bookkeeping, LOCAL-ONLY.* | — |
|
||||
|
||||
**Not touched (scope fence):** `scripts/trends/src/{types,store,item,score}.ts` (no schema change, no scoring
|
||||
change, `queryByTopic` NOT refactored) · `references/*` (SSOT unchanged, no new ref) · `config/*` · no new
|
||||
`.mjs` under `hooks/scripts/` (surfacing edits existing `session-start.mjs`) · `agents/*` count (19) ·
|
||||
`commands/*` (29) · `references/*` (27) · `.gitignore` (trends lines present).
|
||||
|
||||
## Step 1 — (RED) failing tests for brief ranking, rendering, and the `brief` CLI
|
||||
|
||||
Author `brief.test.ts` (new) + extend `cli.test.ts`. To make the brief cases fail **on assertion** (not on
|
||||
module-absent — the R2a discipline, `plan-re-r2a.md:38-47`), Step 1 creates `brief.ts` with **wrong-but-non-
|
||||
throwing stubs**: `rankForBrief` → `{ today, freshDays: opts?.freshDays ?? 7, totals:{trends:0,matched:0,
|
||||
fresh:0}, topMatches:[], singleMatches:[], olderMatched:[] }`; `renderBrief` → `""`; `briefSummary` → `""`;
|
||||
`defaultBriefDir` → `""`. Cases then fail on value assertions (empty buckets ≠ expected, `""` lacks
|
||||
frontmatter), not `module-not-found`. The stubs are **replaced** in Step 2 — none survives into GREEN.
|
||||
|
||||
`brief.test.ts`:
|
||||
- **rank/group (SC1):** a fixture store with trends at known `overlap` (0/1/2+) × freshness → `topMatches` only
|
||||
`overlap≥2 & fresh`, `singleMatches` only `overlap===1 & fresh`, `olderMatched` only `overlap≥1 & stale`,
|
||||
`overlap===0` absent everywhere; within-group order `overlap desc → effectiveDate desc → title asc → url asc`;
|
||||
`matchedPillars` = actual matched names (case-insensitive match, pillar original case kept); `totals` correct.
|
||||
**Total-order fixture:** two records with the **same title + effectiveDate + overlap but different url** →
|
||||
assert a fixed order by `url asc` (proves order is independent of store insertion / V8 sort stability).
|
||||
- **freshness (SC2):** `effectiveDate = publishedAt ?? capturedAt`; fresh-by-publishedAt-despite-old-capturedAt
|
||||
and fresh-by-capturedAt-fallback-when-publishedAt-absent; boundary `ageDays === freshDays` is fresh;
|
||||
`freshDays: 14` re-buckets a 10-day item from older→fresh.
|
||||
- **render + summary (SC3):** `renderBrief` output starts with frontmatter carrying `date` / a column-0
|
||||
single-line `summary` (assert **no `"` and no `\n`** in the summary line) / `store: { trends, matched, fresh }`
|
||||
/ `schemaVersion: 1`; the frontmatter `summary` **equals `briefSummary(ranking)`** (one source); body has the
|
||||
three section headings + the pinned entry-line shape; the empty ranking → a valid brief whose `summary` is
|
||||
`briefSummary(emptyRanking)` = the "no fresh" line; **determinism:** two `renderBrief` calls on the same
|
||||
ranking are byte-identical.
|
||||
- **path:** `defaultBriefDir()` ends with `trends/morning-brief` and honors `LINKEDIN_STUDIO_DATA` (set the env
|
||||
in-test, assert prefix == that root; restore) — proving it rides on `defaultStorePath`'s root.
|
||||
|
||||
`cli.test.ts` (subprocess `node --import tsx src/cli.ts brief` with a `--store` temp + `--out` temp):
|
||||
- happy: `--pillars a,b --store <tmp-with-matches> --out <tmp>` writes `<out>/<today>.md` (file exists;
|
||||
today-shaped name) and prints the path; `--json` → object with `path/date/totals/summary`, `summary` equal to
|
||||
what the file's frontmatter carries.
|
||||
- `--fresh-days xyz` → exit 2; empty `--pillars` (omit the flag) → exit 0 + a file written (no-match brief);
|
||||
**bare `--out` (no value)** → does NOT write to `./true/` (falls back to `defaultBriefDir()` — the
|
||||
`flags.out !== "true"` guard).
|
||||
|
||||
**RED proof (record in commit):** `(cd scripts/trends && npm test)` → `brief.test.ts` cases fail on **assertion**
|
||||
(wrong-value stubs), and `cli.test.ts` brief cases fail **logic-RED** against the existing dispatch (an unknown
|
||||
`brief` command hits `usage()` → exit 2, no file written) — neither is module-not-found.
|
||||
|
||||
## Step 2 — (GREEN) pure brief layer: `brief.ts`
|
||||
|
||||
Create `scripts/trends/src/brief.ts`:
|
||||
- `import type { TrendStore, TrendRecord } from "./types.js";`
|
||||
- Types: `BriefEntry { trend: TrendRecord; overlap: number; matchedPillars: string[]; effectiveDate: string;
|
||||
ageDays: number }`; `BriefRanking { today: string; freshDays: number; totals: { trends: number; matched:
|
||||
number; fresh: number }; topMatches: BriefEntry[]; singleMatches: BriefEntry[]; olderMatched: BriefEntry[] }`.
|
||||
- `export const BRIEF_SCHEMA_VERSION = 1;`
|
||||
- `import { defaultStorePath } from "./store.js";` + `import { join, dirname } from "node:path";` (NO `homedir`
|
||||
— the root comes via `defaultStorePath`, not re-resolved).
|
||||
- `rankForBrief(store, pillars, today, opts)`: `freshDays = opts?.freshDays ?? 7`; lowercase the pillar set
|
||||
once; for each trend compute `overlap`/`matchedPillars` (case-insensitive membership, original-case pillar
|
||||
preserved — same idiom as `store.ts:151-152`, recomputed here, `queryByTopic` untouched), `effectiveDate =
|
||||
trend.publishedAt ?? trend.capturedAt`, `ageDays = Math.floor((Date.parse(today) -
|
||||
Date.parse(effectiveDate)) / 86400000)` (a **local** day-delta — NOT imported from `cli.ts`'s `daysBetween`
|
||||
`cli.ts:107-109`; importing it would invert the dependency direction, as `cli.ts` imports `brief.ts`).
|
||||
Bucket; sort each bucket `overlap desc → effectiveDate desc → title asc → url asc` (total order); `totals`.
|
||||
- `briefSummary(ranking)`: the single summary source — fresh>0 ? `<fresh> ferske tema-signaler matcher
|
||||
pillarene dine. Topp: «<top title>» (<top pillar> · <age>d).` : `Ingen ferske tema-signaler på pillarene dine
|
||||
(av <trends> i lager).` One line, no `"`, no `\n`.
|
||||
- `renderBrief(ranking)`: build the YAML frontmatter with `summary: ${briefSummary(ranking)}` (the **shared**
|
||||
source, never re-derived), `date`/`store`/`ranking`/`schemaVersion`, then the three sections with the pinned
|
||||
entry-line shape (titles in «»). Column-0 `summary`.
|
||||
- `defaultBriefDir()`: `join(dirname(defaultStorePath()), "morning-brief")` — `defaultStorePath()` =
|
||||
`<root>/trends/trends.json`, so `dirname` = `<root>/trends`, + `morning-brief` = `<root>/trends/morning-brief`.
|
||||
**One** root resolution (reuses `defaultStorePath`); no duplicate `LINKEDIN_STUDIO_DATA ?? join(homedir()…)`.
|
||||
- **Replace the Step-1 wrong-value stubs.** Make Step 1's `brief.test.ts` cases green.
|
||||
|
||||
## Step 3 — (GREEN) CLI `brief` + `cli.test.ts`
|
||||
|
||||
In `cli.ts`: add `import { rankForBrief, renderBrief, briefSummary, defaultBriefDir } from "./brief.js";`, add
|
||||
`mkdirSync`/`writeFileSync` to the `node:fs` import (currently only `readFileSync` `cli.ts:30`), and add
|
||||
`import { join } from "node:path";`. Add the `brief` branch to `main`'s dispatch (after `capture`): `const
|
||||
pillars = splitTopics(flags.pillars)`; parse `--fresh-days` (default 7; `Number.isNaN` → `usage("--fresh-days
|
||||
must be a number")`); `const outDir = flags.out && flags.out !== "true" ? flags.out : defaultBriefDir()` (the
|
||||
`!== "true"` guard is required — `parseFlags` `cli.ts:52-53` yields `"true"` for a bare `--out`); `const ranking
|
||||
= rankForBrief(loadStore(storePath), pillars, today(), { freshDays })`; `const md = renderBrief(ranking)`;
|
||||
`const path = join(outDir, today() + ".md")`; `mkdirSync(outDir, { recursive: true }); writeFileSync(path, md,
|
||||
"utf8")`; human → `Wrote brief: <path> (M matched, K fresh)`; `--json` → `{ path, date: ranking.today, totals:
|
||||
ranking.totals, summary: briefSummary(ranking) }` — `summary` is `briefSummary(ranking)`, the **same** source
|
||||
`renderBrief` puts in the frontmatter (no re-derivation). Add the `brief …` line to `usage()` (`cli.ts:71-84`)
|
||||
and the header doc-comment (`cli.ts:1-28`). Make Step 1's `cli.test.ts` brief cases green. **`today()`
|
||||
exact-value assertions are NOT in `cli.test.ts`** (it reads the wall clock) — the date-shape
|
||||
(`/^\d{4}-\d{2}-\d{2}$/`) is asserted on the written filename; exact-date logic is covered in `brief.test.ts`
|
||||
via injected `today`.
|
||||
|
||||
## Step 4 — (GREEN) session-start surfacing + hook test
|
||||
|
||||
`session-start.mjs`: add `latestMorningBrief(briefDir)` near `trendsNewestCapture` (`:38`): `existsSync` guard →
|
||||
`readdirSync(briefDir).filter(f => /^\d{4}-\d{2}-\d{2}\.md$/.test(f)).sort().reverse()[0]` → read it →
|
||||
`{ date: extractYaml(c,'date'), summary: extractYaml(c,'summary'), file: join(briefDir, name) }`; any
|
||||
throw/absence → `null`. `extractYaml` (`:19-23`) captures `[^"\n]*` + `.trim()`, so `date`/`summary` are
|
||||
**newline-free** → the surfaced block needs **no** `.replace(/\n/g,'\\n')` (contrast the multi-line state
|
||||
sections `:320`); only the static `\\n` literals are used. Inject **after the brain-missing nudge
|
||||
(`:500-504`)**, unconditional:
|
||||
```
|
||||
const latestBrief = latestMorningBrief(join(getDataRoot('trends'), 'morning-brief'));
|
||||
if (latestBrief && latestBrief.summary) {
|
||||
context += `\\n## Morning Brief (${latestBrief.date})\\n${latestBrief.summary}\\n→ Full brief: ${latestBrief.file}\\n`;
|
||||
}
|
||||
```
|
||||
(`getDataRoot` `:11` + `join` `:6` are already imported; `readdirSync` already imported `:5`.) **No tsx.** Then
|
||||
`session-start-morning-brief.test.mjs` (new): **read `session-start-trends-staleness.test.mjs` first** and copy
|
||||
its exact structure (the env-var name it sets, how it spawns `session-start.mjs`, how it parses the JSON
|
||||
`hookSpecificOutput.additionalContext`, how it writes the fixture under the temp root). Cases: (a) with a
|
||||
fixture `trends/morning-brief/<date>.md` under `env.LINKEDIN_STUDIO_DATA` → `additionalContext` includes `##
|
||||
Morning Brief`, the `summary`, the file path, and **no raw `\n` inside that block** (the `\\n` idiom held);
|
||||
(b) no brief dir → no `Morning Brief` block + `continue: true` (no crash); (c) **path cross-check** — write the
|
||||
fixture at the path `defaultBriefDir()` resolves to under that same `LINKEDIN_STUDIO_DATA` and confirm the hook
|
||||
finds it (CLI-write path == hook-read path). The temp root has **no** `node_modules/tsx` (zero-dep proof).
|
||||
**This test runs under `node --test hooks/scripts/__tests__/`, the separate hook-suite gate — NOT
|
||||
`test-runner.sh`.**
|
||||
|
||||
## Step 5 — wire `trend-spotter.md` (Open Q#1, default) + README
|
||||
|
||||
In `agents/trend-spotter.md`, after the Step 4.5 `capture` block, add a step: build the pillar list the agent
|
||||
already scans and run **`node --import tsx scripts/trends/src/cli.ts brief --pillars <pillars>`** to write
|
||||
today's dated brief. Replacement prose **must contain the literal `src/cli.ts brief`** (Section 16i `grep -qF`).
|
||||
Keep the "skip silently if no deps" escape hatch + domain-general phrasing (no vendor/sector tokens — Section
|
||||
17). Update `scripts/trends/README.md`: a `brief` CLI example + the `trends/morning-brief/YYYY-MM-DD.md`
|
||||
artifact + its frontmatter (`date`/`summary`/`store`/`schemaVersion`), framed as "the dated, surfaced read over
|
||||
the store" (distinct from `query`/`list` interactive dumps).
|
||||
|
||||
## Step 6 — gate: floors + new unconditional Section 16i
|
||||
|
||||
In `scripts/test-runner.sh`:
|
||||
- Set `TRENDS_TESTS_FLOOR` (`:697`, currently 79) to the **`tests N` line** reported by `(cd scripts/trends &&
|
||||
npm test)` after Steps 1–3 — recounted live, NOT an additive guess. Stays **inside** the `if [ -x …/tsx ]`
|
||||
deps guard. **Append** `+ RE-R2b: brief +N + cli +M (morning-brief)` to the inline per-slice breakdown comment
|
||||
on `:697` so the comment can't drift from the number.
|
||||
- Add **Section 16i** ("Trends Brief Wiring", RE-R2b), mirroring Section 16h's shape (`:1074-1116`). **Placement
|
||||
(verified):** file order is 17→16g→16h→18 (`:943/:1010/:1074/:1118`), so **16h is the last section before
|
||||
Section 18** — insert 16i **after 16h's closing `fi`/`echo ""` (~`:1116`), before the Section 18 block
|
||||
(`:1118`)** (anti-erosion Section 18 must stay last so it counts every prior check). Four **unconditional**,
|
||||
deps-absent-safe checks (pure `grep -qF`/self-test, no `tsx`):
|
||||
(1) a non-vacuity self-test (probe carrying `src/cli.ts brief` accepted, one without rejected);
|
||||
(2) `grep -qF 'command === "brief"' scripts/trends/src/cli.ts`;
|
||||
(3) `grep -qF 'src/cli.ts brief' agents/trend-spotter.md`;
|
||||
(4) `grep -qF 'latestMorningBrief' hooks/scripts/session-start.mjs` (surfacing is wired, not just doc'd).
|
||||
- Bump `ASSERT_BASELINE_FLOOR` (`:1136`, currently 90) → **live recount** (= 90 + the new unconditional 16i
|
||||
emitters; expected ~94, **recounted at land**). Update the **header-enumeration prose chain** (`:33-46`) by
|
||||
inserting the 16i clause **between** the 16h clause (`:43-45`) and the Section-18 clause (`:46`), preserving
|
||||
sentence flow (it's prose, not an append).
|
||||
- **NOT touched here:** the hook suite has no floor in `test-runner.sh` (no `HOOK_TESTS_FLOOR`, no root
|
||||
`package.json`); the new hook test is gated by `node --test hooks/scripts/__tests__/` (Step 4), separately.
|
||||
|
||||
## Step 7 — behavioural verification
|
||||
|
||||
`(cd scripts/trends && npm install)` if needed, then run brief §7's three behavioural steps (seed via
|
||||
`capture`, generate via `brief --json`, surface via `session-start.mjs` with `LINKEDIN_STUDIO_DATA`). Confirm
|
||||
the top-match grouping, the `summary`, and the surfaced block (tsx absent for the surfacing). Run full
|
||||
`bash scripts/test-runner.sh` → `FAIL=0`.
|
||||
|
||||
## Step 8 — land
|
||||
|
||||
Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor, hook-suite, gate
|
||||
total). Commit order (house style): **(1)** docs commit `docs/research-engine/{brief,plan}-re-r2b.md` (no
|
||||
suffix, tracked); **(2)** code commit `scripts/trends/src/brief.ts` + `cli.ts` + tests +
|
||||
`hooks/scripts/session-start.mjs` + the new hook test + `agents/trend-spotter.md` + `scripts/trends/README.md`
|
||||
+ `scripts/test-runner.sh` with `[skip-docs]`. Push freely (window lifted; gitleaks at commit; `origin` =
|
||||
PUBLIC `open/` — STATE/`*.local.*` never pushed). No version bump (additive; `v0.5.2` dev).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|---|---|---|---|
|
||||
| — | RED proof | `(cd scripts/trends && npm test)` after Step 1 | brief cases fail on assertion (logic-RED), not module-not-found |
|
||||
| SC1 | rank/group | `npm test` (brief.test.ts) | groups by overlap×freshness; overlap-0 excluded; sort overlap→effectiveDate→title; matchedPillars + totals correct |
|
||||
| SC2 | freshness | `npm test` (brief.test.ts) | effectiveDate = publishedAt ?? capturedAt; boundary `=freshDays` fresh; freshDays configurable |
|
||||
| SC3 | render | `npm test` (brief.test.ts) | frontmatter date/summary(no `"`)/store/schemaVersion:1; 3 sections; empty→valid no-match brief; deterministic bytes |
|
||||
| SC4 | CLI brief | `npm test` (cli.test.ts) + manual | writes `<out>/<today>.md`; `--json` {path,date,totals,summary}; `--fresh-days` bad → exit 2; empty `--pillars` → exit 0 no-match |
|
||||
| SC5 | surfacing | `node --test hooks/scripts/__tests__/` + manual | brief present → `## Morning Brief` + summary + path, no raw `\n`; absent → no block/no crash; zero-tsx; CLI-write path == hook-read path |
|
||||
| SC6(a) | gate + wiring + de-niche | `bash scripts/test-runner.sh` | `FAIL=0`; trends ≥ floor; Section 16i green; ASSERT floor recount; Section 17; counts 27/19/29 |
|
||||
| SC6(b) | hook gate | `node --test hooks/scripts/__tests__/` | new `session-start-morning-brief.test.mjs` green (hook suite is NOT part of test-runner.sh) |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — non-determinism leaks into the brief** (clock/env inside the pure functions → flaky bytes). *Mitigated:*
|
||||
`today`/`pillars`/`freshDays` injected; SC3 asserts byte-identical output; `ageDays` uses the injected `today`.
|
||||
- **R2 — `extractYaml` mis-reads the `summary`** (a `"` or newline in the value truncates it). *Mitigated:*
|
||||
`renderBrief` emits a single-line `summary` with no embedded `"` (titles in «»); SC3 asserts it; the hook
|
||||
guards `latestBrief.summary` truthy before surfacing.
|
||||
- **R3 — the hook accidentally needs tsx** (e.g. someone "reuses" the TS ranker in the hook). *Mitigated:*
|
||||
surfacing is `readdir`+`readFile`+`extractYaml` only; SC5 runs with tsx absent; the B-S3 precedent
|
||||
(`:32-37`) is the explicit pattern.
|
||||
- **R4 — editing `trend-spotter.md` trips the de-niche guard.** *Mitigated:* Section 17 runs in the gate;
|
||||
the added step is pillar-driven (user config), vendor/sector-free; only a `brief` call is added.
|
||||
- **R5 — new gate checks must survive a deps-absent fresh clone.** *Mitigated:* Section 16i is pure
|
||||
`grep`/self-test on tracked source (no `tsx`) → unconditional; `TRENDS_TESTS_FLOOR` stays inside the deps guard.
|
||||
- **R6 — brief dir vs store path drift** (hook `join(getDataRoot('trends'),'morning-brief')` vs the CLI's
|
||||
`defaultBriefDir()`). *Mitigated:* `defaultBriefDir()` now **derives from `defaultStorePath()`** (one root
|
||||
resolution, not a re-implementation) — so `brief.ts` and `store.ts` cannot disagree on the root. The remaining
|
||||
hook-vs-CLI pair (`getDataRoot('trends')` vs `defaultStorePath`'s root) is the **same store-path twin already
|
||||
in production** (the B-S3 staleness nudge reads `join(getDataRoot('trends'),'trends.json')` `:376` and must
|
||||
already equal the CLI's `defaultStorePath()` for that nudge to work) and is behaviorally guarded by
|
||||
`__tests__/data-root.test.mjs`. SC5(c) adds a direct write-then-read cross-check; `brief.test` pins
|
||||
`defaultBriefDir` against `LINKEDIN_STUDIO_DATA`.
|
||||
- **R7 — `brief.ts` is a new source file** (R2a forbade one for the bridge). *Mitigated:* justified by cohesion
|
||||
(a ranking+rendering module with its own types/version, not a single-use function); §5 states it; counts
|
||||
unaffected (no new ref/agent/command).
|
||||
|
||||
## Plan-critic — folded
|
||||
|
||||
plan-critic returned **REVISE** (2 blockers, 5 majors, 4 minors); brief-reviewer **PROCEED_WITH_RISKS** (1
|
||||
major + minors); scope-guardian **ALIGNED** (0 findings; counts 27/19/29 verified live). Resolution, each
|
||||
verified against live code:
|
||||
|
||||
- **[BLOCKER] `--json` summary source left as "re-derive OR expose"** — RED can't target an undecided API.
|
||||
✅ committed to `export function briefSummary(ranking): string` (Step 2); `renderBrief` frontmatter + CLI
|
||||
`--json` both call it (Step 3); Step 1 asserts `renderBrief`'s frontmatter `summary === briefSummary(ranking)`.
|
||||
- **[BLOCKER] Section 16i placement ambiguous** ("after 16h / before 18" vs brief's wording). ✅ verified file
|
||||
order 17→16g→16h→18 (`:943/:1010/:1074/:1118`); Step 6 pins 16i after 16h's closing `echo ""` (~`:1116`),
|
||||
before Section 18 (`:1118`); states 16h is the last section before 18.
|
||||
- **[MAJOR] throwing-stub RED contradicts "fail on assertion, not module-not-found"** (R2a's blocker class).
|
||||
✅ Step 1 now uses **wrong-but-non-throwing** stubs (empty buckets / `""`) so `brief.test` fails on assertion;
|
||||
`cli.test` brief cases are logic-RED against the existing dispatch (unknown command → `usage()` exit 2).
|
||||
- **[MAJOR] brief §3 `flags.out ?? defaultBriefDir()` writes `./true/`** for a bare `--out` (`parseFlags`
|
||||
`cli.ts:52-53`). ✅ brief §3 + Step 3 use the `flags.out !== "true"` guard; Step 1 adds a bare-`--out`
|
||||
cli.test case.
|
||||
- **[MAJOR] `TRENDS_TESTS_FLOOR` recount ambiguous** (full `tests N` vs additive). ✅ Step 6 pins it to the
|
||||
`tests N` line after Steps 1–3, comment appends `+ RE-R2b: brief +N + cli +M`.
|
||||
- **[MAJOR] `defaultBriefDir` triple-twin root re-resolution** (drift risk). ✅ it now derives from
|
||||
`defaultStorePath()` (one root resolution, Step 2); R6 + SC5(c) cite the existing `data-root.test.mjs` twin
|
||||
guard and add a write-then-read cross-check.
|
||||
- **[MAJOR] surfacing newline-escape unproven.** ✅ Step 4 + SC5 state `extractYaml`'s `[^"\n]*` + `.trim()`
|
||||
makes `date`/`summary` newline-free → no `.replace` needed; SC5 asserts no raw `\n` in the block.
|
||||
- **[brief-reviewer MAJOR] SC6 "hook-suite at recount" attributed to `test-runner.sh`** (which neither runs nor
|
||||
counts it — no `HOOK_TESTS_FLOOR`, no root `package.json`). ✅ SC6 + the verification table split into two
|
||||
gate commands: `test-runner.sh` (trends/16i/ASSERT/de-niche/counts) and `node --test hooks/scripts/__tests__/`
|
||||
(the new hook test).
|
||||
- **[MINOR] non-total sort order** (same title, different url ties fully). ✅ added `url asc` final tie-break
|
||||
(Step 2/SC1); Step 1 adds a same-title/diff-url fixture. **[MINOR] local `ageDays` duplication unstated** ✅
|
||||
Step 2 states the deliberate non-import (dependency direction `cli.ts → brief.ts`). **[MINOR] `.md$` anchor**
|
||||
✅ confirmed in the filter regex (Step 4). **[MINOR] header-enumeration prose flow** ✅ Step 6 inserts the 16i
|
||||
clause in-prose between the 16h and Section-18 clauses. **[brief-reviewer MINOR] body entry-line age format**
|
||||
✅ pinned in brief §3 + SC3. **[brief-reviewer MINOR] Open Q#2/#4 are load-bearing** ✅ reframed as
|
||||
confirmations (brief §8).
|
||||
- **[plan-critic headless-readiness] N/A** — R2b executes **in-session, operator-driven** (driftsmodell), not
|
||||
as a headless autonomous run, so per-step revert/halt clauses aren't needed (R1/R2a had none either).
|
||||
|
||||
**scope-guardian — ALIGNED:** every SC1–SC6 traces to a step; zero creep; all nine §4 non-goals held (no
|
||||
scheduler, no scoring change, no store-schema change, no relevance/saturation/status fields, no pillar-from-state
|
||||
resolution, no delivery channel, no brief-diffing, no new agent/command); counts 27/19/29 verified live;
|
||||
`brief.ts` the only new source file.
|
||||
321
docs/research-engine/plan-re-r3a.md
Normal file
321
docs/research-engine/plan-re-r3a.md
Normal file
|
|
@ -0,0 +1,321 @@
|
|||
# Plan — RE-R3a: persist the relevance score + rank the morning brief on it
|
||||
|
||||
> **Brief:** `docs/research-engine/brief-re-r3a.md`. **Slice:** RE-R3a (research-engine rung-2 — R3 slice 1,
|
||||
> research-deepening: the relevance half of hull 5 + the hull-3 schema remainder).
|
||||
> **TDD-order (two-phase RED — light-Voyage BLOCKER fold):** Step 1 records RED in two phases —
|
||||
> **(A)** true logic-RED for `store`/`brief`/`cli` against the pre-edit code (inline fixtures, no new import);
|
||||
> **(B)** for `score`/`item`, land non-throwing stubs for the new `score.ts` exports first (Node16 ESM throws a
|
||||
> missing named import at module-load, not on assertion), then record the value-assertion RED against the stubs.
|
||||
> Then GREEN: S-score envelope → S-types + S-store (first-sight persist + v2→v3 migrate) → S-item (validate +
|
||||
> bridge) → S-brief (composite-sort + render band+mode) → S-cli (doc-only) → wire `trend-spotter.md` + README →
|
||||
> gate floors + Section 16j → behavioural → land.
|
||||
> **Counts recounted live at land, never pinned/guessed.**
|
||||
> **Go-gate decisions (confirmed "Go" 2026-06-24):** D1 4-field `TrendScore` · D2 composite primary within
|
||||
> bucket · D3 first-sight · D4 one slice (data-then-visible commit order) · D6 mode shown in per-entry render.
|
||||
> **Light-Voyage hardened:** scope-guardian ALIGNED (0) / brief-reviewer PROCEED_WITH_RISKS (6 MINOR) /
|
||||
> plan-critic REVISE (1 BLOCKER, 4 MAJOR, 4 MINOR) — all folded (see §Plan-critic — folded).
|
||||
|
||||
## Goal
|
||||
|
||||
Stop discarding the relevance judgment the `trend-spotter` agent already computes. Persist a 4-field
|
||||
`TrendScore { mode, dimensions, composite, priority }` on the store record (schema v2→v3, additive lossless
|
||||
migrate — the R2a pattern), computed deterministically from the agent's five judgment scores by the already-built
|
||||
`score.ts` (`composite`+`band`, one owner). Then make `rankForBrief` order each bucket on composite first, and
|
||||
`renderBrief` surface the band + mode. No re-score-on-recapture, no saturation/status, no scheduler, no new
|
||||
source file — those stay later R3 slices.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | SC |
|
||||
|---|---|---|
|
||||
| `scripts/trends/src/score.ts` | **EDIT** — `TrendScore` interface + `requiredDimensions(mode)` (ordered) + `scoreEnvelope(mode, dimensions)` (composes the existing `composite`+`band`, no new arithmetic; throws on bad dim by contract) | SC1 |
|
||||
| `scripts/trends/src/types.ts` | **EDIT** — `import type { TrendScore }`; `TrendRecord.score?: TrendScore`; `SCHEMA_VERSION` 2→3; doc-comment | SC4 |
|
||||
| `scripts/trends/src/store.ts` | **EDIT** — `TrendInput.score?: TrendScore`; `addTrend` persists `score` first-sight on add (conditional spread), duplicate does NOT update; migrate comment v1→v2→v3 (logic unchanged) | SC3, SC4 |
|
||||
| `scripts/trends/src/item.ts` | **EDIT** — `TrendItem.score?: {mode,dimensions}`; `normalizeItem` validates (non-array `score`/`dimensions`; mode; the mode's five dims in [1,10]) → structured error, carries validated dims; `itemToInput` carries `scoreEnvelope(...)` (throws by contract on direct bad dims) | SC2 |
|
||||
| `scripts/trends/src/brief.ts` | **EDIT** — `rankForBrief` comparator: composite primary (`?? -1`), buckets unchanged; `renderBrief`/`renderBulletEntry` surface `· <priority> (<mode>)` (full pinned shapes); `briefSummary` band only; exact `ranking:` descriptor | SC5, SC6 |
|
||||
| `scripts/trends/src/cli.ts` | **EDIT (doc-only behavior)** — capture persists score automatically via `itemToInput` (no logic change); header doc-comment note | SC7 |
|
||||
| `scripts/trends/tests/score.test.ts` | **EDIT** — `requiredDimensions` (both modes, ordered + order pinned) + `scoreEnvelope` (composite/priority = existing funcs; bad dim throws) | SC1 |
|
||||
| `scripts/trends/tests/item.test.ts` | **EDIT** — score validation (valid carried/validated dims; bad mode/missing/out-of-range/non-array/array-dims → structured error, no throw) + `itemToInput` envelope + `itemToInput` direct bad-dim throws | SC2 |
|
||||
| `scripts/trends/tests/store.test.ts` | **EDIT** — first-sight persist (new persists; duplicate keeps first score, topics union; score-free add) + v2→v3 migration (lossless/idempotent + score-survives-round-trip, mirrors `:403-476` with `2`→`3`) | SC3, SC4 |
|
||||
| `scripts/trends/tests/brief.test.ts` | **EDIT** — composite-primary within bucket; unscored last (`-1`); total order; full render lines (`· <priority> (<mode>)` scored / unchanged unscored); summary band; unscored single-match-top summary; quote-safe summary; exact `ranking:` string; determinism | SC5, SC6 |
|
||||
| `scripts/trends/tests/cli.test.ts` | **EDIT** — capture batch with score → record carries computed composite/priority (read back via `list --json`); bad score → `errors[]`, valid added, exit 0 | SC7 |
|
||||
| `agents/trend-spotter.md` | **EDIT** — Step 4.5 capture batch carries per-item `"score":{"mode":"kortform","dimensions":{…}}`; prose ("carry the Step-2 scores"); domain-general; contains literal `"dimensions"` (currently absent → grep non-vacuous) | SC8 |
|
||||
| `scripts/trends/README.md` | **EDIT** — item `score` field (judgment in) + persisted `TrendScore` (out) + brief ranks on composite | — |
|
||||
| `scripts/test-runner.sh` | **EDIT** — `TRENDS_TESTS_FLOOR` 104→recount + breakdown comment (`:701`); NEW unconditional **Section 16j** (after 16i `~:1171` / before 18 `:1173`); `ASSERT_BASELINE_FLOOR` 94→**99**; header-enumeration chain (16i clause `:46-49`, Section-18 clause `:49`) | SC8 |
|
||||
| `docs/research-engine/{brief,plan}-re-r3a.md` | **NEW** — slice docs (TRACKED, like `docs/second-brain/*`) | — |
|
||||
| `STATE.md` | **EDIT at land** — Telling-block reconcile (trends floor, ASSERT floor 99, schema v3, gate total). *Land bookkeeping, LOCAL-ONLY.* | — |
|
||||
|
||||
**Not touched (scope fence):** `references/trend-scoring-modes.md` (SSOT weights/bands unchanged) · the
|
||||
SessionStart hook + its tests (R3a touches no hook; no frontmatter-schema change; no new hook test) ·
|
||||
`queryByTopic`/`history`/`newestCaptureDate` (store query unchanged) · the `score` CLI digest path + the `add`
|
||||
manual path (`cli.ts`) · `config/*` · no new `.ts`/`.mjs` file · `agents/*` count (19) · `commands/*` (29) ·
|
||||
`references/*` (27) · `.gitignore` (trends lines present).
|
||||
|
||||
## Step 1 — (RED, two phases) failing tests across score/item/store/brief/cli
|
||||
|
||||
RED discipline (R2a/R2b) + the light-Voyage BLOCKER fold: a missing **named** import throws at module-load under
|
||||
Node16 ESM (`package.json:8` `node --import tsx --test`; `tsconfig.json` `module: Node16`), so `score`/`item`
|
||||
(which reference new `score.ts` exports) cannot be assertion-RED before those exports exist. Split the RED proof:
|
||||
|
||||
**Phase A — true logic-RED against the pre-edit code** (`store`/`brief`/`cli` build fixtures inline; `TrendStore`/
|
||||
`TrendRecord` are `import type`, erased by tsx; they import no new runtime symbol):
|
||||
- `store.test.ts`: `addTrend(emptyStore, inputWithScore)` → record has `score`; a second `addTrend` (same
|
||||
title+url, **different** score) → stored score **unchanged**, `added:false`, topics unioned; score-free add →
|
||||
score-free record. **Migration:** a `schemaVersion:2` store with score-less records → `loadStore` gives
|
||||
`schemaVersion:3`, records intact, `"score" in record === false` (not invented); round-trip writes
|
||||
`schemaVersion:3`; a v3 store with `score` is idempotent; **a v3 store's `score` survives load+resave**
|
||||
(read back, re-save, re-read — `score` byte-identical). *(Mirror the RE-R2a block `store.test.ts:403-476`,
|
||||
retitled `(RE-R3a / score v2→v3)`, every `schemaVersion` assertion literal flipped `2`→`3`.)*
|
||||
- `brief.test.ts`: two **same-overlap, same-freshness** records, composites 9 vs 6 → the 9 sorts first in its
|
||||
bucket; a **scored** vs **unscored** same-bucket pair → scored first (the `-1` sentinel); a same-title/diff-url
|
||||
**both-unscored** pair → fixed by `url asc` (total order intact); `renderBrief` for a scored top entry contains
|
||||
the **full line** `… (<age>d) · <priority> (<mode>) · Pillarer: …`, for an unscored one the **unchanged** line
|
||||
(no token) — assert full lines, not substrings; a scored bullet contains `… (<age>d) · <priority> (<mode>) ·
|
||||
🔗 …`; `briefSummary` names the band on a scored top, omits it on an unscored top, and is one line with no `"`
|
||||
**even when the top title contains a guillemet/quote**; a store whose only fresh match is a **single-pillar
|
||||
unscored** record → summary with no `· <priority>` token; the `ranking:` line equals the pinned string
|
||||
verbatim; two `renderBrief` calls byte-identical.
|
||||
- `cli.test.ts`: a `capture` batch (subprocess, `--store` temp) with a valid per-item `score` → a following
|
||||
`list --store <tmp> --json` shows the record's `score` with the computed composite/priority; a batch with one
|
||||
bad score (`timing:99`) → JSON `errors[]` non-empty, the valid items added, **exit 0**.
|
||||
|
||||
**Phase B — stub-first, then value-assertion RED** (`score`/`item` reference new exports):
|
||||
- Land **non-throwing stubs** in `score.ts` so the imports resolve: `requiredDimensions → []`; `scoreEnvelope →
|
||||
{ mode, dimensions, composite: 0, priority: "Skip" }`. (These are the wrong-value stubs Step 2 replaces.)
|
||||
- `score.test.ts`: `requiredDimensions("kortform")`/`("long-form")` deep-equal the two **ordered** five-key
|
||||
lists (and a guard that the order matches `Object.keys(KORTFORM_WEIGHTS)` so a SSOT reorder fails);
|
||||
`scoreEnvelope(mode, dims)` returns `{mode, dimensions, composite, priority}` with composite/priority **equal
|
||||
to `composite(dims,mode)` / `band(...).priority`** (assert against the existing functions — not hard-coded —
|
||||
so they share one owner); a bad dim makes `scoreEnvelope` throw (it calls `composite`). Fails on assertion
|
||||
against the stubs (`[] ≠ expected`, `composite 0 ≠ real`).
|
||||
- `item.test.ts`: a valid `score` → `normalizeItem` `ok:true` carrying the **validated** dims; bad `mode` / a
|
||||
missing dimension / a dim `0` or `11` / a non-object `score` / an **array** `dimensions` → `ok:false` with an
|
||||
`invalid score` error (**does not throw**); absent `score` → key omitted. `itemToInput(validItemWithScore,
|
||||
"2026-06-24")` → `score` equals `scoreEnvelope(mode,dims)`; without score → no `score`; **`itemToInput` with an
|
||||
out-of-range dim throws** (the defense-in-depth contract). Fails on assertion against the stubs.
|
||||
|
||||
**RED proof (record in commit, two phases):** Phase A — `(cd scripts/trends && npm test)` before any src edit →
|
||||
the `store`/`brief`/`cli` new cases fail on **assertion** (logic-RED), not module-not-found. Phase B — after the
|
||||
non-throwing stubs land, the `score`/`item` cases fail on **value assertion** against the stubs. The plan does
|
||||
**not** claim a single "all five fail before any code" run.
|
||||
|
||||
## Step 2 — (GREEN) `score.ts` envelope
|
||||
|
||||
In `scripts/trends/src/score.ts`:
|
||||
- `export interface TrendScore { mode: ScoreMode; dimensions: DimensionScores; composite: number; priority:
|
||||
Priority }`.
|
||||
- `export function requiredDimensions(mode: ScoreMode): string[] { return Object.keys(WEIGHTS[mode]); }` —
|
||||
**ordered** (insertion order of the SSOT weight literal); `score.test` pins the order.
|
||||
- `export function scoreEnvelope(mode: ScoreMode, dimensions: DimensionScores): TrendScore { const c =
|
||||
composite(dimensions, mode); return { mode, dimensions, composite: c, priority: band(c).priority }; }` —
|
||||
composes the existing pure functions; **no new arithmetic**; throws via `composite` on a bad dim (its
|
||||
contract). Replace the Phase-B stubs. Make `score.test` green.
|
||||
|
||||
## Step 3 — (GREEN) `types.ts` + `store.ts` (schema v3 + first-sight persist)
|
||||
|
||||
- `types.ts`: `import type { TrendScore } from "./score.js";`; add `score?: TrendScore;` to `TrendRecord`
|
||||
(doc-comment marks it as the realized `:21-23` field); `SCHEMA_VERSION = 3`.
|
||||
- `store.ts`: add `score?: TrendScore;` to `TrendInput` (`import type { TrendScore } from "./score.js"`);
|
||||
in `addTrend`'s new-record branch add `...(input.score !== undefined ? { score: input.score } : {})` (after the
|
||||
`summary` spread `:136`); the **duplicate** branch is unchanged (topics union only — score is first-sight,
|
||||
D3). Extend the `loadStore` migrate comment to *"v1→v2→v3 all additive-optional … the migration is the version
|
||||
stamp alone"* — **no logic change** (`Math.max(onDisk, SCHEMA_VERSION)` `:87` already handles v3; `saveStore`
|
||||
`JSON.stringify` `:95` preserves `score`). Make `store.test` green (first-sight + v2→v3 + survives-round-trip).
|
||||
|
||||
## Step 4 — (GREEN) `item.ts` (validate + bridge)
|
||||
|
||||
- `TrendItem` gains `score?: { mode: ScoreMode; dimensions: DimensionScores };` (`import type { ScoreMode,
|
||||
DimensionScores } from "./score.js"` + `import { requiredDimensions } from "./score.js"`).
|
||||
- `normalizeItem`: after the `publishedAt` validation (`:99-106`), if `r.score !== undefined && r.score !==
|
||||
null`, validate: `score` is a **non-array** object; `mode` ∈ `{kortform, long-form}`; `dimensions` is a
|
||||
**non-array** object; **for each key in `requiredDimensions(mode)`**, the value is a number in [1,10]. On any
|
||||
failure `errors.push("invalid score: <reason>")`; on success build `score = { mode, dimensions }` from the
|
||||
**validated** values (not raw `r.score.dimensions`). Carried into the returned `TrendItem` via conditional
|
||||
spread (key omitted when absent/invalid). **Never throws** — structured errors only (the `publishedAt`
|
||||
discipline).
|
||||
- `itemToInput`: add `...(item.score !== undefined ? { score: scoreEnvelope(item.score.mode,
|
||||
item.score.dimensions) } : {})` (`import { scoreEnvelope } from "./score.js"`). On the **capture path** the
|
||||
dims are pre-validated by `normalizeItem`, so `scoreEnvelope`→`composite` cannot throw there; called
|
||||
**directly** with bad dims it throws by contract (defense-in-depth — SC2 asserts it). Make `item.test` green.
|
||||
|
||||
## Step 5 — (GREEN) `brief.ts` (composite sort + render band+mode)
|
||||
|
||||
- `rankForBrief` comparator (`:94-98`): prepend `(b.trend.score?.composite ?? -1) - (a.trend.score?.composite
|
||||
?? -1) ||` before the existing `b.overlap - a.overlap || …`. **Sentinel `-1`** (composite ≥ 1.0 always; `-1`
|
||||
sorts unscored last and subtracts cleanly). Buckets (`:100-104`) and totals unchanged.
|
||||
- `renderTopEntry` (`:132-141`): when `e.trend.score` is present, insert `· ${e.trend.score.priority}
|
||||
(${e.trend.score.mode})` into the meta line **between `(${e.ageDays}d)` and `· Pillarer`** (unscored:
|
||||
unchanged). `renderBulletEntry` (`:143-145`): when scored, insert `· ${score.priority} (${score.mode})`
|
||||
**before `· 🔗`** (unscored: unchanged). Both shapes asserted as **full lines** in `brief.test`.
|
||||
- `briefSummary` (`:122-130`): in the fresh>0 branch, when `top.trend.score` is present include `·
|
||||
${top.trend.score.priority}` in the top mention (**band only — no mode** to keep the one-line headline clean):
|
||||
`… Topp: «<title>» (<pillar> · <priority> · <age>d).`; when the top is unscored omit the token:
|
||||
`… Topp: «<title>» (<pillar> · <age>d).` Keep it one line, no `"`/`\n`.
|
||||
- `renderBrief`'s `ranking:` line (`:160`) → the **exact pinned** string
|
||||
`composite desc, then pillar-overlap desc, then publishedAt desc (capturedAt fallback); freshDays
|
||||
${ranking.freshDays}` (asserted byte-for-byte). `BRIEF_SCHEMA_VERSION` stays 1. Make `brief.test` green.
|
||||
|
||||
## Step 6 — (GREEN) `cli.ts` (doc-only) + `cli.test`
|
||||
|
||||
`capture` (`:243-269`) already folds through `itemToInput` (`:254`), which now carries `score` → capture
|
||||
persists it with **no logic change**. Update only the header doc-comment (`:15-21`): note `capture` persists an
|
||||
optional relevance score computed from the item's judgment scores. Make `cli.test`'s capture-persists-score +
|
||||
bad-score-in-errors cases green. *(No `today()` exact-value assertions in `cli.test` — the wall clock is read at
|
||||
the edge; composite/priority are deterministic and asserted on the read-back record.)*
|
||||
|
||||
## Step 7 — wire `trend-spotter.md` (Open Q#1 default = WIRE) + README
|
||||
|
||||
In `agents/trend-spotter.md` Step 4.5 (`:291-298`), extend each capture-batch item with **`"score": {"mode":
|
||||
"kortform", "dimensions": {"pillar": N, "audience": N, "timing": N, "angle": N, "authority": N}}`** — the same
|
||||
five judgment scores computed in Step 2 (`:134`). Add prose: don't discard the Step-2 scores; fold them into the
|
||||
capture batch so the store persists the relevance assessment and the morning brief ranks on it. Mode defaults
|
||||
`kortform`; `long-form` when invoked from `/linkedin:newsletter` (the long-form dims are
|
||||
`pillar/depth/angle/authority/currency`, `trend-scoring-modes.md:59-65`). The replacement prose **must contain
|
||||
the literal `"dimensions"`** (Section 16j `grep -qF`; verified absent today → non-vacuous). Keep the "skip
|
||||
silently if no deps" escape hatch + domain-general phrasing (Section 17). Update `scripts/trends/README.md`: the
|
||||
item `score` field (judgment in), the persisted `TrendScore` (composite/priority out), and that the brief now
|
||||
ranks on composite.
|
||||
|
||||
## Step 8 — gate: floors + new unconditional Section 16j
|
||||
|
||||
In `scripts/test-runner.sh`:
|
||||
- Set `TRENDS_TESTS_FLOOR` (`:701`, currently 104) to the **`tests N` line** reported by `(cd scripts/trends &&
|
||||
npm test)` after Steps 1–6 — recounted live, NOT additive-guessed. Stays **inside** the `if [ -x …/tsx ]`
|
||||
deps guard. **Append** `+ RE-R3a: score +N` to the inline breakdown comment (`:701`) so it can't drift.
|
||||
- Add **Section 16j** ("Trends Score Wiring", RE-R3a), mirroring Section 16i's shape (`:1122-1171`). **Placement
|
||||
(verified):** file order is 17→16g→16h→16i→18 (`:947/:1014/:1078/:1122/:1173`), so **16i is the last section
|
||||
before Section 18** — insert 16j **after 16i's closing `fi`/`echo ""` (~`:1171`), before the Section 18 block
|
||||
(`:1173`)** (anti-erosion Section 18 must stay last so it counts every prior check). Five **unconditional**,
|
||||
deps-absent-safe checks (pure `grep -qF`/self-test, no `tsx`), the self-test emitting **one** pass/fail like
|
||||
16i:
|
||||
(1) a non-vacuity self-test (a probe carrying `score?.composite` accepted, one without rejected);
|
||||
(2) `grep -qF 'export interface TrendScore' scripts/trends/src/score.ts`;
|
||||
(3) `grep -qF 'score?: TrendScore' scripts/trends/src/types.ts`;
|
||||
(4) `grep -qF '"dimensions"' agents/trend-spotter.md` (the capture batch carries the judgment);
|
||||
(5) `grep -qF 'score?.composite' scripts/trends/src/brief.ts` (the brief ranks on it — payoff wired, not
|
||||
merely doc'd).
|
||||
- Bump `ASSERT_BASELINE_FLOOR` (`:1193`, currently 94) → **exactly 99** (94 + the 5 new unconditional 16j
|
||||
emitters; the self-test emits one pass/fail like 16i, so 99 is deterministic — "live recount" is the safety
|
||||
net, not a guess). Update the **header-enumeration prose chain** by inserting the 16j clause **between** the
|
||||
16i clause (`:46-49`) and the Section-18 clause (`:49`), preserving sentence flow (it's prose, not an append).
|
||||
- **NOT touched here:** the hook suite (no `HOOK_TESTS_FLOOR` in `test-runner.sh`; R3a adds no hook test). It
|
||||
must still pass untouched (`node --test hooks/scripts/__tests__/`) as a regression sanity at land.
|
||||
|
||||
## Step 9 — behavioural verification
|
||||
|
||||
`(cd scripts/trends && npm install)` if needed, then run brief §7's four behavioural steps (capture A>B with
|
||||
scores, `list --json` to confirm persisted composite/priority, `brief --json` to confirm A precedes B + the band
|
||||
in the summary + `· <priority> (kortform)` in the entry line, a bad-score item lands in `errors[]` with exit 0).
|
||||
Run full `bash scripts/test-runner.sh` → `FAIL=0` (`ASSERT_BASELINE_FLOOR` 99); run `node --test
|
||||
hooks/scripts/__tests__/` → still green (untouched regression).
|
||||
|
||||
## Step 10 — land
|
||||
|
||||
Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor 99, schema v3,
|
||||
gate total). Commit order (house style): **(1)** docs commit `docs/research-engine/{brief,plan}-re-r3a.md` (no
|
||||
suffix, tracked); **(2)** code commit — the six `src/*.ts` + five test files + `agents/trend-spotter.md` +
|
||||
`scripts/trends/README.md` + `scripts/test-runner.sh` with `[skip-docs]`. (Per D4, the code commit MAY be split
|
||||
into a data commit [score/types/store/item/cli + their tests] and a visible commit [brief + its tests + agent
|
||||
wire] if the R2a/R2b two-commit rhythm is preferred.) Push freely (window lifted; gitleaks at commit; `origin` =
|
||||
PUBLIC `open/` — STATE/`*.local.*` never pushed). No version bump (additive; `v0.5.2` dev).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|---|---|---|---|
|
||||
| — | RED Phase A | `(cd scripts/trends && npm test)` before src edits | store/brief/cli new cases fail on assertion (logic-RED), not module-not-found |
|
||||
| — | RED Phase B | `npm test` after non-throwing stubs | score/item cases fail on value assertion against stubs (`[] ≠ expected`, `composite 0 ≠ real`) |
|
||||
| SC1 | score envelope | `npm test` (score.test) | `requiredDimensions` both modes ordered (order pinned); `scoreEnvelope` composite/priority = existing funcs; bad dim throws |
|
||||
| SC2 | item validate + bridge + contract | `npm test` (item.test) | valid score carried (validated dims); bad mode/missing/out-of-range/non-array/array-dims → structured error (no throw); `itemToInput` envelope; direct bad-dim throws |
|
||||
| SC3 | first-sight persist | `npm test` (store.test) | new persists score; duplicate keeps first score (topics union); score-free add works |
|
||||
| SC4 | migration v2→v3 | `npm test` (store.test) | v2 loads as v3, records intact, no score invented; round-trip writes v3; v3 idempotent; score survives load+resave |
|
||||
| SC5 | brief ranks on composite | `npm test` (brief.test) + manual | composite primary within bucket; unscored last (`-1`); total order; deterministic |
|
||||
| SC6 | render band+mode | `npm test` (brief.test) | full lines `· <priority> (<mode>)` scored / unchanged unscored; summary band (no mode); unscored single-match summary; quote-safe; exact `ranking:` string |
|
||||
| SC7 | CLI persists score | `npm test` (cli.test) + manual | capture batch w/ score → record carries composite/priority; bad score → `errors[]`, valid added, exit 0 |
|
||||
| SC8 | gate + wiring + de-niche | `bash scripts/test-runner.sh` | `FAIL=0`; trends ≥ floor; Section 16j green; `ASSERT_BASELINE_FLOOR`=99; Section 17; counts 27/19/29 |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — composite/band drift from the SSOT** (someone re-implements the math in `scoreEnvelope`). *Mitigated:*
|
||||
`scoreEnvelope` *composes* `composite()`+`band()` (one owner); SC1 asserts equality against those functions;
|
||||
`score.test.ts:12-30` (weights) + the band-string assertions already pin them to the SSOT.
|
||||
- **R2 — a bad score crashes the capture loop.** *Mitigated:* `normalizeItem` fully validates the score (mode +
|
||||
the mode's five dims in [1,10], non-array objects) → structured error into `errors[]`; on the capture path
|
||||
`itemToInput`→`composite` is unreachable for bad dims; SC2 + SC7 assert no-throw + `errors[]` routing.
|
||||
`itemToInput` called **directly** with bad dims throws by contract (SC2) — defense-in-depth, not a leak.
|
||||
- **R3 — comparator NaN from the unscored sentinel** (`-Infinity - -Infinity`). *Mitigated:* sentinel is `-1`
|
||||
(composite ≥ 1.0 = min 1×Σweights); subtracts cleanly; SC5 asserts the both-unscored total order holds.
|
||||
- **R4 — losing the brief's determinism** (composite ties not fully broken). *Mitigated:* the new term is a
|
||||
leading tie-break; the existing `overlap → effectiveDate → title → url` chain still gives a total order (the
|
||||
`(title,url)` pair is the unique dedupe id, `store.ts:66-68`); SC5 asserts byte-identical output + the
|
||||
same-title/diff-url case.
|
||||
- **R5 — `extractYaml` mis-reads the `summary`** if the band token introduces a `"`/newline. *Mitigated:* the
|
||||
band strings (`Immediate`/`High`/…) are bare words; the summary stays one line, no `"` — asserted in SC6 even
|
||||
with a guillemet/quote in the top title; the surfacing hook is untouched.
|
||||
- **R6 — migration not actually lossless** (a v2 record mutated on load, or a new field stripped on resave).
|
||||
*Mitigated:* the migration is the version stamp alone (logic unchanged); `saveStore` `JSON.stringify` `:95`
|
||||
strips nothing; SC4 mirrors the proven R2a `store.test:403-476` and adds a score-survives-round-trip case.
|
||||
- **R7 — editing `trend-spotter.md` trips the de-niche guard.** *Mitigated:* Section 17 runs in the gate; the
|
||||
added `score`/`dimensions` are the rubric's generic dimension names + the user's pillars, vendor/sector-free.
|
||||
- **R8 — new gate checks must survive a deps-absent fresh clone.** *Mitigated:* Section 16j is pure
|
||||
`grep`/self-test on tracked source (no `tsx`) → unconditional; `TRENDS_TESTS_FLOOR` stays inside the deps guard.
|
||||
- **R9 — import cycle.** *Mitigated:* `score.ts` imports nothing internal today (`:1-17`, verified leaf). R3a
|
||||
adds three new **one-way** inbound edges to it — `score.ts ← types.ts` (`TrendScore` type), `score.ts ←
|
||||
store.ts` (`TrendScore` type on `TrendInput`), `score.ts ← item.ts` (`ScoreMode`/`DimensionScores` types +
|
||||
`requiredDimensions`/`scoreEnvelope` values). The DAG stays acyclic: `score.ts (leaf) ← {types, store, item,
|
||||
brief, cli}`, since `score.ts` imports none of them back.
|
||||
- **R10 — mode-mixing makes the ranking apples-to-oranges** (kortform vs long-form composites ranked together).
|
||||
*Accepted for R3a, mitigated visibly:* almost all records are `kortform` (the default); the body entry line
|
||||
**shows the mode** (`<priority> (<mode>)`, D6/SC6) so the operator can see when adjacent entries used different
|
||||
instruments; a mode-segmented brief is a documented R3-later non-goal (brief §4).
|
||||
|
||||
## Plan-critic — folded
|
||||
|
||||
Three Opus reviewers ran, each verifying claims against live code: **scope-guardian ALIGNED** (0 findings; counts
|
||||
27/19/29 + "no new source file" verified live); **brief-reviewer PROCEED_WITH_RISKS** (all four load-bearing
|
||||
claims verified TRUE; 6 MINOR); **plan-critic REVISE** (1 BLOCKER, 4 MAJOR, 4 MINOR; score 72/C). Resolution,
|
||||
each verified against live code:
|
||||
|
||||
- **[BLOCKER — plan-critic] Step 1 RED-proof self-contradictory for score/item under ESM** (a missing named
|
||||
import throws at module-load, not on assertion; the stub-first fix inverted the stated ordering). ✅ Step 1 is
|
||||
now **explicitly two-phase**: Phase A true logic-RED for `store`/`brief`/`cli` against pre-edit code; Phase B
|
||||
stub-first then value-assertion RED for `score`/`item`. The header blockquote + brief §5 + §TDD-order state it;
|
||||
the "all five fail before any code" claim is removed.
|
||||
- **[MAJOR — plan-critic] no-throw guarantee overstated** (`itemToInput` is public; direct bad-dim calls throw).
|
||||
✅ Step 4 + R2 + brief §5 reword it **path-specific** (no throw on the capture path because `normalizeItem`
|
||||
gates it; direct calls throw by contract); SC2 adds a direct-throw assertion + a carries-validated-dims
|
||||
assertion.
|
||||
- **[MAJOR — plan-critic] mode-mixing waved away + "mode shown per entry" contradicted the render spec.** ✅ D6:
|
||||
the render now shows `· <priority> (<mode>)` per body entry (Step 5); brief §4 adds a mode-blind-ranking
|
||||
non-goal with the rationale (mode visible, mostly kortform); SC6 asserts the full line incl. mode; R10 added.
|
||||
- **[MAJOR — plan-critic] `requiredDimensions` order contract ambiguous** (SC1 hard-coded arrays vs membership
|
||||
use). ✅ pinned **ordered** (Step 2 + SC1 deep-equal the SSOT-order array; `score.test` pins the order;
|
||||
`normalizeItem` consumes as a set).
|
||||
- **[MAJOR — plan-critic] `ASSERT_BASELINE_FLOOR` "~99" not pinned.** ✅ pinned **99** (94 + 5 unconditional 16j
|
||||
emitters; the self-test emits one pass/fail like 16i) — Step 8 + SC8 + brief §3.
|
||||
- **[MINOR — brief-reviewer] SC4 ref `:403-471` stale + v2 assertions** → ✅ `:403-476` + "flip every
|
||||
`schemaVersion` literal 2→3" note (SC4, Step 1).
|
||||
- **[MINOR — brief-reviewer] R1 SSOT-pin cite was the doc-comment** (`score.ts:9-13`) → ✅ now `score.test.ts:12-30`
|
||||
(R1, brief §2/§5).
|
||||
- **[MINOR — brief-reviewer] bullet `· <priority>` placement substring-only** → ✅ full pinned line shape
|
||||
(priority+mode before `🔗`), asserted as a full line (Step 5, SC6).
|
||||
- **[MINOR — brief-reviewer] three diverging `ranking:` descriptor strings** → ✅ one verbatim target string,
|
||||
asserted byte-for-byte (Step 5, SC6).
|
||||
- **[MINOR — brief-reviewer] unscored single-match-top summary path untested** → ✅ added as a Phase-A
|
||||
brief.test case + SC6.
|
||||
- **[MINOR — brief-reviewer] `normalizeItem` non-array object case understated in the brief** → ✅ "non-array"
|
||||
added to both `score` and `dimensions` object checks (Step 4, brief §3, SC2 array-dims case).
|
||||
- **[MINOR — plan-critic] header-chain line-ref `:33-49` loose** → ✅ tightened to the 16i clause `:46-49` /
|
||||
Section-18 clause `:49` (Step 8).
|
||||
- **[MINOR — plan-critic] R9 DAG omitted the new `score.ts ←` edges** → ✅ R9 now lists all three one-way edges.
|
||||
- **[MINOR — plan-critic] SC6 quote-safety regression with the new token** → ✅ SC6 asserts the summary stays
|
||||
one-line/no-`"` with a scored top title containing a guillemet/quote.
|
||||
- **[MINOR — plan-critic] SC4 forward-compat / score-survives-round-trip untested** → ✅ added to SC4 + Step 1
|
||||
store.test.
|
||||
- **[plan-critic headless-readiness] N/A** — R3a executes **in-session, operator-driven** (driftsmodell), not as
|
||||
a headless autonomous run, so per-step revert/halt clauses aren't needed (R1/R2a/R2b had none either).
|
||||
|
||||
**scope-guardian — ALIGNED:** every SC1–SC8 traces to a step; zero creep; all §4 non-goals held (no
|
||||
re-score-on-recapture, no saturation/status/first-mover field, no scheduler, no seen-log, no brief-diff, no
|
||||
A1–A4, no mode-filter, no `score` in the `add` path, no new source file/agent/command); counts 27/19/29 verified
|
||||
live; exactly 6 `src/*.ts` + 5 `tests/*.test.ts`, all edited, none added.
|
||||
384
docs/research-engine/plan-re-r3b.md
Normal file
384
docs/research-engine/plan-re-r3b.md
Normal file
|
|
@ -0,0 +1,384 @@
|
|||
# Plan — RE-R3b: trend lifecycle — re-score on re-capture · status (acted/skipped) · seen-log
|
||||
|
||||
> **Brief:** `docs/research-engine/brief-re-r3b.md`. **Slice:** RE-R3b (research-engine rung-2 — R3 slice 2, the
|
||||
> **lifecycle** slice: re-score-on-recapture + status `new`/`acted`/`skipped` + the `surfacedCount`/`lastSurfacedAt`
|
||||
> seen-log).
|
||||
> **TDD-order (two-phase RED — light-Voyage BLOCKER fold, inherited from R3a):** Step 1 records RED in two phases —
|
||||
> **(A)** true logic-RED for the re-score + migration parts of `store.test` (existing `addTrend`/`loadStore`,
|
||||
> inline fixtures), all of `brief.test` (behaviour change to existing `rankForBrief`/`renderBrief`), and `cli.test`
|
||||
> (subprocess — `act`/`skip` are unknown commands today → assertion-RED); **(B)** for `setStatus`/`markSurfaced`/
|
||||
> `effectiveStatus` (new `store.ts` exports), land non-throwing stubs first (Node16 ESM throws a missing named
|
||||
> import at module-load, not on assertion), then record value-assertion RED against the stubs. Then GREEN:
|
||||
> S-store stubs→real (effectiveStatus + setStatus + markSurfaced + re-score) → S-types (3 fields + v3→v4) **+ the
|
||||
> R3a-migration-block reconcile** → S-brief (status filter + id/marker render + surfacedIds + descriptor) → S-cli
|
||||
> (act/skip/reset + brief-marks-surfaced + --no-mark) → wire `trend-spotter.md` (prose) + README → gate floors +
|
||||
> Section 16k → behavioural → land.
|
||||
> **Counts recounted live at land, never pinned/guessed.**
|
||||
> **Architectural decisions (CONFIRMED, AskUserQuestion 2026-06-25):** A1 on-record seen-log, brief records
|
||||
> surfacing (rankForBrief pure, `--no-mark`) · A2 re-score last-wins (score the one mutable field; status not
|
||||
> reset) · A3 acted/skipped EXCLUDED from the brief. Go-gate D1–D9 baked to recommended defaults (brief §8).
|
||||
> **Light-Voyage hardened:** scope-guardian ALIGNED (0 creep / 0 gaps; 2 MINOR line-cites) · brief-reviewer
|
||||
> PROCEED_WITH_RISKS (all 9 load-bearing claims TRUE; 1 MEDIUM + 3 LOW) · plan-critic PROCEED_WITH_RISKS (78/B;
|
||||
> 1 MAJOR + 5 MINOR) — all folded (see §Plan-critic — folded). The MAJOR (the `brief` CLI `store` binding) is
|
||||
> fixed by the Step-5 hoist.
|
||||
|
||||
## Goal
|
||||
|
||||
Give a trend a **life after first capture**. (1) **Re-score on re-capture** — `addTrend`'s duplicate branch now
|
||||
refreshes `score` (last-wins; timing decays), reusing the already-built capture path (no new arithmetic, `score.ts`
|
||||
untouched). (2) **Status lifecycle** — `new`/`acted`/`skipped` on the record, set by new `act`/`skip`/`reset` CLI
|
||||
verbs; the brief **excludes** handled trends. (3) **Seen-log** — `surfacedCount`/`lastSurfacedAt` accumulated
|
||||
(per-day-idempotent) by the `brief` CLI after the pure ranking, the temporal foundation slices (c)+(b) build on.
|
||||
Schema v3→v4 (additive lossless migrate — the R3a pattern, **plus** reconciling the R3a migration block's
|
||||
hard-coded `3` literals that the bump would otherwise regress). No saturation scoring, no scheduler, no brief-diff,
|
||||
no new source file — those stay later R3 slices.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | SC |
|
||||
|---|---|---|
|
||||
| `scripts/trends/src/types.ts` | **EDIT** — `export type TrendStatus`; `TrendRecord` gains `status?`/`surfacedCount?`/`lastSurfacedAt?` (all optional); `SCHEMA_VERSION` 3→4; doc-comment | SC1, SC4 |
|
||||
| `scripts/trends/src/store.ts` | **EDIT** — `effectiveStatus(t)` (absent⇒new); `setStatus(store,id,status)`; `markSurfaced(store,ids,today)` (per-day idempotent); `addTrend` duplicate branch re-scores (last-wins, `merged` broadened, no false-merge); migrate comment v1→…→v4 (logic unchanged) | SC1, SC2, SC3, SC4 |
|
||||
| `scripts/trends/src/brief.ts` | **EDIT** — `rankForBrief` excludes `effectiveStatus !== "new"`; `renderTopEntry`/`renderBulletEntry` append `· \`<id>\`` + `surfacedToken` (`· sett Nx` when count≥2); `surfacedIds(ranking)`; `ranking:` descriptor gains `; excludes acted/skipped` | SC5, SC6 |
|
||||
| `scripts/trends/src/cli.ts` | **EDIT** — `act`/`skip`/`reset --id` (setStatus, not-found→exit 2); `brief` **hoists `const store = loadStore(storePath)`** + records surfacing via `markSurfaced`+`saveStore` unless `--no-mark`, `--json` gains `marked`; exit-code doc-comment (`:33`) + capture tally comment (`:251-252`) broadened; usage + header doc | SC7, SC8 |
|
||||
| `scripts/trends/tests/store.test.ts` | **EDIT** — effectiveStatus; setStatus (found/absent/reset); markSurfaced (increment/idempotent/skip); re-score (last-wins/no-false-merge/status+provenance untouched/no-score-noop); **NEW `(RE-R3b / lifecycle v3→v4)` migration block** + **reconcile the existing `(RE-R3a / score v2→v3)` block's `3` literals** | SC1–SC4 |
|
||||
| `scripts/trends/tests/brief.test.ts` | **EDIT** — exclude acted/skipped (all buckets; totals.trends full; only-handled→empty); render full lines (`· \`<id>\``; `· sett Nx` at count≥2 / none below); `surfacedIds`; exact `ranking:` string; determinism | SC5, SC6 |
|
||||
| `scripts/trends/tests/cli.test.ts` | **EDIT** — act/skip/reset (read back via `list --json`; unknown id→exit 2; missing --id→exit 2); brief marks surfaced (`surfacedCount:1`+`lastSurfacedAt`, `marked` in json); second-same-day idempotent (`marked:0`); `--no-mark` (no write); brief md omits acted/skipped | SC7, SC8 |
|
||||
| `scripts/trends/src/item.ts` | **UNTOUCHED** — re-score reuses the existing `itemToInput`→`scoreEnvelope` bridge (R3a). Listed to assert it is *not* in scope. | — |
|
||||
| `scripts/trends/src/score.ts` | **UNTOUCHED** — no scoring-math change. Listed to assert it is *not* in scope. | — |
|
||||
| `agents/trend-spotter.md` | **EDIT (prose-only, minimal)** — one line: re-capture refreshes the score (timing decays); operator marks `acted`/`skipped` via the CLI so the brief stops re-surfacing handled work. No batch-shape change (score already emitted, R3a). Domain-general. | — |
|
||||
| `scripts/trends/README.md` | **EDIT** — status lifecycle + verbs, seen-log (per-day idempotent, brief-recorded), re-score last-wins, brief exclude-handled + `--no-mark` | — |
|
||||
| `scripts/test-runner.sh` | **EDIT** — `TRENDS_TESTS_FLOOR` 146→recount + breakdown comment (`:705`); NEW unconditional **Section 16k** (after 16j's block `~:1235` / before Section 18 header `:1237`); `ASSERT_BASELINE_FLOOR` 99→**105**; header-enumeration chain | SC9 |
|
||||
| `docs/research-engine/{brief,plan}-re-r3b.md` | **NEW** — slice docs (TRACKED, like `docs/second-brain/*`) | — |
|
||||
| `STATE.md` | **EDIT at land** — Telling-block reconcile (trends floor, ASSERT floor 105, schema v4, gate total). *Land bookkeeping, LOCAL-ONLY.* | — |
|
||||
|
||||
**Not touched (scope fence):** `references/trend-scoring-modes.md` (no scoring-math change) · `score.ts` + `item.ts`
|
||||
(re-score reuses the R3a capture path) · the SessionStart hook + its tests (no hook change; no frontmatter-schema
|
||||
change; `BRIEF_SCHEMA_VERSION` stays 1; no new hook test) · `queryByTopic`/`history`/`newestCaptureDate` (store
|
||||
query unchanged) · the CLI `status` staleness reader (unrelated to the new `TrendStatus` type — the name collision
|
||||
is pre-existing, not reconciled here) · `config/*` · no new `.ts`/`.mjs` file · `agents/*` count (19) ·
|
||||
`commands/*` (29) · `references/*` (27) · `.gitignore` (trends lines present) · the two inert `schemaVersion: 2`
|
||||
fixture literals — `brief.test.ts:53` (`rankForBrief` ignores `schemaVersion`) AND `cli.test.ts:247` (the seed
|
||||
store is re-stamped to current on capture/brief load; no test reads its on-disk version — `cli.test.ts:102` uses
|
||||
the `SCHEMA_VERSION` constant) — both **left as-is** (folded — brief-reviewer #3: enumerated so "miss none" is
|
||||
literally true).
|
||||
|
||||
## Step 1 — (RED, two phases) failing tests across store/brief/cli
|
||||
|
||||
Same RED discipline as R3a (two-phase, light-Voyage BLOCKER fold): a missing **named** import throws at
|
||||
module-load under Node16 ESM, so the `store.test` cases that reference the new `setStatus`/`markSurfaced`/
|
||||
`effectiveStatus` exports cannot be assertion-RED before stubs exist. Split:
|
||||
|
||||
**Phase A — true logic-RED against the pre-edit code** (uses existing `addTrend`/`loadStore`/`saveStore`/
|
||||
`rankForBrief`/`renderBrief`; `TrendStore`/`TrendRecord` are `import type`, erased by tsx; subprocess for cli):
|
||||
- `store.test.ts` — **re-score** (uses existing `addTrend`): `addTrend(store, dupSameTitleUrlDifferentScore)` →
|
||||
stored `score` **replaced**, `merged:true`, `added:false`, topics unioned, `source`/`capturedAt`/`publishedAt`
|
||||
unchanged; a **byte-identical** re-score (no new topics) → `merged:false`; a duplicate with **no** `score` →
|
||||
stored score unchanged; re-capture of a record with `status:"acted"` (constructed inline) → score updated,
|
||||
`status` still `"acted"`, `surfacedCount` untouched. *(These fail against pre-edit `addTrend`, which never
|
||||
touches `score` on a duplicate.)*
|
||||
- `store.test.ts` — **NEW migration block `(RE-R3b / lifecycle v3→v4)`** (uses existing `loadStore`/`saveStore`,
|
||||
**hard-coded `4`** so it is RED while `SCHEMA_VERSION` is still 3 — the genuine-RED device; the target+idempotent
|
||||
assertions are switched to `SCHEMA_VERSION` at GREEN, Step 3, per brief-reviewer #4): a
|
||||
`schemaVersion:3` store with score-bearing, lifecycle-field-less records → `loadStore` gives `schemaVersion:4`,
|
||||
records intact, `"status" in record === false` (+ `surfacedCount`/`lastSurfacedAt` absent — none invented);
|
||||
round-trip writes `schemaVersion:4`; a `schemaVersion:4` store with lifecycle fields loads **idempotent**
|
||||
(stays 4, fields deep-equal); the three new fields **survive load→save→load**.
|
||||
- `brief.test.ts` — **exclude** (uses existing `rankForBrief`): a store with `new` + `acted` + `skipped` matches
|
||||
all fresh+on-pillar → only the `new` records appear in any bucket; `totals.trends` equals the **full** count
|
||||
(incl. handled); a store whose only matches are `acted`/`skipped` → all buckets empty + the "no fresh signals"
|
||||
`briefSummary`. **render** (uses existing `renderBrief`): a top entry's meta line ends `· \`<id>\`` (full line);
|
||||
a bullet ends `· \`<id>\`` (full line); a record with `surfacedCount:3` shows `· sett 3x` (full line), one with
|
||||
`surfacedCount:1`/absent shows **no** surfaced token (full line); `surfacedIds(ranking)` (new export — see
|
||||
Phase B note) … **[moved to Phase B]**; the `ranking:` line equals the pinned string ending `; excludes
|
||||
acted/skipped`; two `renderBrief` calls byte-identical.
|
||||
- `cli.test.ts` — **act/skip/reset** (subprocess, `--store` temp): `act --id <id>` then `list --json` → record
|
||||
`status:"acted"`; `skip` → `"skipped"`; `reset` → `"new"`; an **unknown** id → **exit 2** + store unchanged; a
|
||||
missing `--id` → exit 2. **brief-marks-surfaced**: `brief --store <tmp>` on fresh matches → `list --json` shows
|
||||
the surfaced trends `surfacedCount:1` + today's `lastSurfacedAt`, the `brief --json` carries `marked:<n>`; a
|
||||
**second** same-day `brief` → `marked:0`, counts unchanged; `brief --no-mark` → `marked:0`, no `surfacedCount`
|
||||
written; the brief `.md` **omits** an `acted` record. **re-score tally** (folded — plan-critic #4): `capture` a
|
||||
scored item, then `capture` the same title+url with a **changed** score → the second `capture --json` reports
|
||||
`merged:1` and `list --json` shows the **updated** composite. *(Fail today: `act`/`skip`/`reset` are unknown
|
||||
commands → `usage` exit 2 but no status set; `brief` does not write `surfacedCount` / emit `marked`; a re-capture
|
||||
with a changed score is a plain `duplicate` (score discarded), not `merged`.)*
|
||||
|
||||
**Phase B — stub-first, then value-assertion RED** (`store.test`/`brief.test` reference new exports):
|
||||
- Land **non-throwing stubs** so the imports resolve: in `store.ts` — `effectiveStatus → "new"` (constant),
|
||||
`setStatus → { store, found:false }`, `markSurfaced → { store, marked:0 }`; in `brief.ts` — `surfacedIds → []`.
|
||||
(Wrong-value stubs Step 2/4 replace.)
|
||||
- `store.test.ts`: `effectiveStatus({…status:"acted"})` is `"acted"` (stub returns `"new"` → RED);
|
||||
`setStatus(store, presentId, "skipped")` → `{ found:true }` and the record's status set (stub `found:false` →
|
||||
RED); `markSurfaced(store,[idA],today)` increments + sets `lastSurfacedAt`, `marked:1`, and is per-day
|
||||
idempotent on a re-call (stub `marked:0`, no mutation → RED).
|
||||
- `brief.test.ts`: `surfacedIds(ranking)` deep-equals the ids of `topMatches ∪ singleMatches ∪
|
||||
olderMatched.slice(0,5)` (stub `[]` → RED).
|
||||
|
||||
**RED proof (record in commit, two phases):** Phase A — `(cd scripts/trends && npm test)` before any src edit →
|
||||
the re-score/migration/brief/cli new cases fail on **assertion** (logic-RED), not module-not-found. Phase B —
|
||||
after the non-throwing stubs land, the `effectiveStatus`/`setStatus`/`markSurfaced`/`surfacedIds` cases fail on
|
||||
**value assertion** against the stubs. The plan does **not** claim a single "everything fails before any code" run.
|
||||
|
||||
## Step 2 — (GREEN) `types.ts` lifecycle type+fields, then `store.ts` functions + re-score
|
||||
|
||||
**(Reordered — folded plan-critic #5: the type must exist before `store.ts` uses it.)** First, in
|
||||
`scripts/trends/src/types.ts` — add **`export type TrendStatus = "new" | "acted" | "skipped";`** and the three
|
||||
optional `TrendRecord` fields (`status?: TrendStatus`, `surfacedCount?: number`, `lastSurfacedAt?: string`) with
|
||||
doc-comments marking them the realized lifecycle fields the `:22` note anticipated. **Do NOT bump `SCHEMA_VERSION`
|
||||
here** — the bump lands in Step 3, atomic with the R3a-block reconcile (so the suite is never bumped-but-unreconciled).
|
||||
|
||||
Then in `scripts/trends/src/store.ts` (replace the Phase-B stubs):
|
||||
- `import type { TrendStatus } from "./types.js";` (type-only — no cycle; the DAG stays `score ← types ← store`,
|
||||
acyclic).
|
||||
- `export function effectiveStatus(t: TrendRecord): TrendStatus { return t.status ?? "new"; }`.
|
||||
- `export function setStatus(store, id, status): { store: TrendStore; found: boolean }` — `const t =
|
||||
store.trends.find((x) => x.id === id); if (!t) return { store, found: false }; t.status = status; return {
|
||||
store, found: true };`. Sets `"new"` explicitly on a `reset`.
|
||||
- `export function markSurfaced(store, ids: string[], today: string): { store: TrendStore; marked: number }` —
|
||||
`const wanted = new Set(ids); let marked = 0; for (const t of store.trends) { if (!wanted.has(t.id)) continue;
|
||||
if (t.lastSurfacedAt === today) continue; t.surfacedCount = (t.surfacedCount ?? 0) + 1; t.lastSurfacedAt =
|
||||
today; marked++; } return { store, marked };`. Per-day idempotent.
|
||||
- **Re-score in `addTrend`'s duplicate branch** (`:127-131`): after `existing.topics = topics;` compute
|
||||
`let changed = topicsChanged;` (rename the `unionTopics` result) and add:
|
||||
`if (input.score !== undefined && JSON.stringify(existing.score) !== JSON.stringify(input.score)) {
|
||||
existing.score = input.score; changed = true; }`; `return { store, added: false, merged: changed };`. Broaden
|
||||
the `AddResult.merged` doc-comment to *"the existing record was mutated — topics unioned and/or score
|
||||
refreshed."* The new-record branch is unchanged. Make the Phase-A re-score + Phase-B lifecycle `store.test`
|
||||
cases green.
|
||||
|
||||
## Step 3 — (GREEN) `SCHEMA_VERSION` 3→4 bump — ATOMIC with the R3a-block reconcile
|
||||
|
||||
The bump + every test it touches land in **one** step (verified-correct by plan-critic #2): the suite is never
|
||||
bumped-but-unreconciled. *(At RED, Step 1, only the new R3b block was failing — hard-coded `4` while SCHEMA_VERSION
|
||||
was still 3; the existing R3a block was green at `3===3`. This step flips SCHEMA_VERSION to 4, which would regress
|
||||
the R3a block's `3` literals UNLESS reconciled here — hence atomic.)*
|
||||
|
||||
- `types.ts`: **`SCHEMA_VERSION = 4`**. Extend the `store.ts` `loadStore` migrate comment to *"v1→v2→v3→v4 all
|
||||
additive-optional … the migration is the version stamp alone"* (no logic change — `Math.max(onDisk,
|
||||
SCHEMA_VERSION)` already stamps v4; `saveStore` preserves the new fields).
|
||||
- **Reconcile the existing `(RE-R3a / score v2→v3)` block** (`store.test.ts:558-650`) — the bump to 4 regresses
|
||||
its hard-coded `3` literals (a v3 store now migrates to **4**, so "v3 idempotent" is no longer true). The
|
||||
brief-reviewer's full enumeration confirmed **exactly three** breaking assertions (`:588`/`:602`/`:628`); apply
|
||||
the minimal, intent-preserving fix (align to the R2a block's `SCHEMA_VERSION` discipline, `:493`):
|
||||
- `:588` `assert.equal(s.schemaVersion, 3, "v2 store must migrate to v3")` → `SCHEMA_VERSION` + message "must
|
||||
migrate to the current version"; **flip the test title at `:571`** ("…loads stamped as **v3**" → "…to the
|
||||
current version") and the **stale `:570` comment** ("…≠ 3").
|
||||
- `:602` `assert.equal(onDisk.schemaVersion, 3)` → `SCHEMA_VERSION`; **flip the test title at `:598`** ("…writes
|
||||
**schemaVersion:3** to disk" → "…writes the current schemaVersion", mirroring the reconciled R2a twin `:502`).
|
||||
- `:606` test **"a v3 store … loads idempotent"** → **retitle** "a v3 store migrates to the current version,
|
||||
score preserved" and change `:628` `assert.equal(s.schemaVersion, 3)` → `SCHEMA_VERSION` (the
|
||||
`assert.deepEqual(score, …)` stays — score survives the migration). *(The v4-**idempotent** guarantee now
|
||||
lives in the new R3b block.)*
|
||||
- `:638` test "a v3 store's score survives load→save→load" → **verified** it asserts only the `score` field
|
||||
(`:663`, no version literal) — **no change**.
|
||||
- **The new `(RE-R3b / lifecycle v3→v4)` block** (landed RED in Step 1 with hard-coded `4`): switch its **target +
|
||||
idempotent** version assertions from hard-`4` to **`SCHEMA_VERSION`** (folded — brief-reviewer #4: now == 4 and
|
||||
**future-proof**, so R3c's v4→v5 bump won't have to reconcile this block — breaking the cycle R3b pays for R3a).
|
||||
The v3 **input** fixtures stay literal `3` (they are old-version inputs). *(The hard-`4` was only the genuine-RED
|
||||
device for the Step-1 run; record that RED proof in the commit log.)*
|
||||
- Make the new `(RE-R3b / lifecycle v3→v4)` block + the reconciled R3a block both green; the full suite green after
|
||||
the bump (SC4).
|
||||
|
||||
## Step 4 — (GREEN) `brief.ts` (exclude handled + id/marker render + surfacedIds)
|
||||
|
||||
- `import { effectiveStatus } from "./store.js";` (brief.ts already imports `defaultStorePath` from there — one-way).
|
||||
- `rankForBrief` entry loop (`:82-92`): add `if (effectiveStatus(trend) !== "new") continue;` immediately before
|
||||
the `if (overlap === 0) continue;` (`:89`). `totals.trends` stays `store.trends.length` (`:116`). Buckets +
|
||||
the composite total order (R3a) unchanged.
|
||||
- `surfacedToken(e)` helper (after `scoreToken`, `:141-145`): `const c = e.trend.surfacedCount; return c &&
|
||||
c >= 2 ? ` · sett ${c}x` : "";`. **Prior-day semantic (folded — plan-critic #3):** the CLI records today's
|
||||
surfacing **after** `renderBrief`, so `surfacedCount` here is the **prior-day** count — `· sett Nx` = "shown on
|
||||
N prior distinct days" (today's appearance is recorded but not yet in this render). Document it in the README;
|
||||
`brief.test` asserts the token by setting `surfacedCount` directly, the cross-day behaviour by §8.
|
||||
- `renderTopEntry` (`:147-156`) meta line (`:150`): append `${surfacedToken(e)}` after `${scoreToken(e)}` and
|
||||
` · \`${e.trend.id}\`` at the very end (after `Pillarer: …`).
|
||||
- `renderBulletEntry` (`:158-160`): append `${surfacedToken(e)}` after `${scoreToken(e)}` and ` · \`${e.trend.id}\``
|
||||
at the very end (after `🔗 ${e.trend.url}`).
|
||||
- `export function surfacedIds(ranking: BriefRanking): string[]` — `return [...ranking.topMatches,
|
||||
...ranking.singleMatches, ...ranking.olderMatched.slice(0, 5)].map((e) => e.trend.id);` (mirrors the render's
|
||||
`:199` `.slice(0, 5)`). Replace the Phase-B stub.
|
||||
- `renderBrief`'s `ranking:` line (`:175`) → append `; excludes acted/skipped` to the pinned string. `briefSummary`
|
||||
+ `BRIEF_SCHEMA_VERSION` unchanged. Make `brief.test` green.
|
||||
|
||||
## Step 5 — (GREEN) `cli.ts` (act/skip/reset + brief marks surfaced + --no-mark)
|
||||
|
||||
- Import `setStatus`, `markSurfaced` from `./store.js`; `surfacedIds` from `./brief.js`.
|
||||
- A shared `setStatusCmd(status: TrendStatus)` inline helper (or three branches): read `flags.id`; if missing/`"true"`
|
||||
→ `usage('<cmd> needs --id <id>')`; `const store = loadStore(storePath); const res = setStatus(store, flags.id,
|
||||
status); if (!res.found) { console.error(`error: no trend with id: ${flags.id}`); process.exit(2); }
|
||||
saveStore(storePath, store); console.log(`Marked ${flags.id} ${status}`);`. Wire `command === "act"` →
|
||||
`"acted"`, `"skip"` → `"skipped"`, `"reset"` → `"new"`.
|
||||
- **Broaden the exit-code doc-comment** (folded — plan-critic #2): `cli.ts:33` *"2 on usage error (incl.
|
||||
unparseable stdin / bad flag)"* → *"2 on usage error or a not-found id (act/skip/reset)"*. A wrong `--id` value
|
||||
is an argument-class error (exit 2), distinct from `capture`'s data items (`cli.ts:31` — `errors[]`, never the
|
||||
exit code). No third code introduced.
|
||||
- `brief` branch (`:274-298`): **hoist the load** (folded — plan-critic #1 / brief-reviewer #1): `cli.ts:286` is
|
||||
`const ranking = rankForBrief(loadStore(storePath), …)` — there is **no `store` variable** (verified). Replace
|
||||
with `const store = loadStore(storePath); const ranking = rankForBrief(store, pillars, day, { freshDays });`.
|
||||
Then after `writeFileSync(path, md, "utf8")` (`:290`), add: `const mark = flags["no-mark"] !== "true"; const
|
||||
marked = mark ? markSurfaced(store, surfacedIds(ranking), day).marked : 0; if (mark) saveStore(storePath,
|
||||
store);` — the **hoisted `store`** holds the full inventory, so acted/skipped records (filtered from the
|
||||
ranking, still in the store) are preserved on resave; the `.md` is rendered from the pure `ranking` **before**
|
||||
the mutation. Add `marked` to the `--json` object (`:293`) and the human summary line.
|
||||
- **Update the `capture` tally comment** (folded — plan-critic #4): `cli.ts:251-252` *"`merged` (existing gained
|
||||
topics)"* → *"`merged` (existing gained topics and/or a refreshed score)"* (no tally-logic change).
|
||||
- `usage()` (`:82-91`) + header synopsis (`:5-13`): add `act`/`skip`/`reset --id <id>` and `[--no-mark]`; a
|
||||
one-line header note (lifecycle verbs set status; the brief excludes handled trends + records surfacing;
|
||||
re-capture refreshes the score). Make `cli.test` green (incl. the re-captured-changed-score → `merged:1` tally
|
||||
assertion, plan-critic #4).
|
||||
|
||||
## Step 6 — wire `trend-spotter.md` (prose) + README
|
||||
|
||||
In `agents/trend-spotter.md`: add **one prose line** (no batch-shape change — the per-item `score` is already
|
||||
emitted, R3a): re-capturing a known trend now **refreshes** its relevance score (timing decays), and the operator
|
||||
marks trends `acted`/`skipped` via the CLI (`act`/`skip --id`) so the morning brief stops re-surfacing handled
|
||||
work. Domain-general (Section 17). Update `scripts/trends/README.md`: the status lifecycle (`new`/`acted`/`skipped`
|
||||
+ `act`/`skip`/`reset`), the seen-log (`surfacedCount`/`lastSurfacedAt`, per-day idempotent, recorded by `brief`),
|
||||
re-score-on-recapture (last-wins), and the brief's exclude-handled + `--no-mark` behaviour.
|
||||
|
||||
## Step 7 — gate: floors + new unconditional Section 16k
|
||||
|
||||
In `scripts/test-runner.sh`:
|
||||
- Set `TRENDS_TESTS_FLOOR` (`:705`, currently 146) to the **`tests N` line** reported by `(cd scripts/trends &&
|
||||
npm test)` after Steps 1–6 — recounted live, NOT additive-guessed. Stays **inside** the deps guard. **Append**
|
||||
`+ RE-R3b: lifecycle +N` to the inline breakdown comment.
|
||||
- Add **Section 16k** ("Trends Lifecycle Wiring", RE-R3b), mirroring Section 16j (header `:1177`, block runs
|
||||
through `~:1235`). **Placement (verified live — scope-guardian / brief-reviewer):** 16j is the last section
|
||||
before Section 18 (anti-erosion, header `:1237`) — insert 16k **after 16j's block (`~:1235`), before Section 18
|
||||
(`:1237`)** (anti-erosion must stay last so it counts every prior check). Six
|
||||
**unconditional**, deps-absent-safe checks (pure `grep -qF`/self-test, no `tsx`), the self-test emitting **one**
|
||||
pass/fail like 16j:
|
||||
(1) a non-vacuity self-test (a probe carrying `effectiveStatus` accepted, one without rejected);
|
||||
(2) `grep -qF 'export type TrendStatus' scripts/trends/src/types.ts`;
|
||||
(3) `grep -qF 'surfacedCount' scripts/trends/src/types.ts` (the seen-log field);
|
||||
(4) `grep -qF 'export function markSurfaced' scripts/trends/src/store.ts` (the seen-log writer);
|
||||
(5) `grep -qF 'effectiveStatus' scripts/trends/src/brief.ts` (the brief excludes handled);
|
||||
(6) `grep -qF 'command === "act"' scripts/trends/src/cli.ts` (the lifecycle verb).
|
||||
- Bump `ASSERT_BASELINE_FLOOR` (`:1259`, currently 99) → **exactly 105** (99 + the 6 new unconditional 16k
|
||||
emitters; the self-test emits one pass/fail like 16j, so 105 is deterministic — "live recount" is the safety
|
||||
net, not a guess). Update the **header-enumeration prose chain** (`:49-53`) by inserting the 16k clause
|
||||
**between** the 16j clause and the Section-18 clause, preserving sentence flow.
|
||||
- **NOT touched here:** the hook suite (no `HOOK_TESTS_FLOOR` in `test-runner.sh`; R3b adds no hook test). It must
|
||||
still pass untouched (`node --test hooks/scripts/__tests__/*.test.mjs`) as a regression sanity at land.
|
||||
|
||||
## Step 8 — behavioural verification
|
||||
|
||||
`(cd scripts/trends && npm install)` if needed, then run brief §7's seven behavioural steps (capture A; re-capture
|
||||
A with a lower timing → `list --json` shows the composite dropped + `merged:1`; `brief --json` → `marked:1`,
|
||||
`list` shows `surfacedCount:1`+`lastSurfacedAt`, entry line shows `· \`<id>\``; re-run brief → `marked:0`;
|
||||
`act --id <A>` → A absent from the brief md; `reset --id <A>` → A reappears; `--no-mark` → no surfacedCount
|
||||
written). Run full `bash scripts/test-runner.sh` → `FAIL=0` (`ASSERT_BASELINE_FLOOR` 105); run `node --test
|
||||
hooks/scripts/__tests__/*.test.mjs` → still green (untouched regression).
|
||||
|
||||
## Step 9 — land
|
||||
|
||||
Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor 105, schema v4,
|
||||
gate total). Commit order (house style): **(1)** docs commit `docs/research-engine/{brief,plan}-re-r3b.md` (no
|
||||
suffix, tracked); **(2)** code commit — the three `src/*.ts` (`types`/`store`/`brief`/`cli` — four) + three test
|
||||
files + `agents/trend-spotter.md` + `scripts/trends/README.md` + `scripts/test-runner.sh` with `[skip-docs]`
|
||||
(D9: single code commit — the lifecycle is tightly coupled). Push freely (window lifted; gitleaks at commit;
|
||||
`origin` = PUBLIC `open/` — STATE/`*.local.*` never pushed). No version bump (additive; `v0.5.2` dev).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|---|---|---|---|
|
||||
| — | RED Phase A | `(cd scripts/trends && npm test)` before src edits | re-score/migration/brief/cli new cases fail on assertion (logic-RED), not module-not-found |
|
||||
| — | RED Phase B | `npm test` after non-throwing stubs | effectiveStatus/setStatus/markSurfaced/surfacedIds fail on value assertion against stubs |
|
||||
| SC1 | status + effectiveStatus + setStatus | `npm test` (store.test) | effectiveStatus absent⇒new; setStatus found/absent; reset⇒new |
|
||||
| SC2 | re-score last-wins | `npm test` (store.test) | dup w/ diff score → replaced, merged:true, provenance+status+surfaced untouched; identical→merged:false; no-score→unchanged; acted stays acted |
|
||||
| SC3 | markSurfaced + idempotency | `npm test` (store.test) | increments + sets lastSurfacedAt; same-day re-call marked:0; later day increments; unknown id skipped |
|
||||
| SC4 | migration v3→v4 | `npm test` (store.test) | v3 loads as v4, intact, no field invented; round-trip writes v4; v4 idempotent; new fields survive; **R3a block reconciled (no regression)** |
|
||||
| SC5 | brief excludes handled | `npm test` (brief.test) | only `new` in buckets; totals.trends full; only-handled→empty + no-fresh summary; surviving order = R3a total order |
|
||||
| SC6 | render id + marker + descriptor | `npm test` (brief.test) | full lines `· \`<id>\``; `· sett Nx` at count≥2 / none below; surfacedIds set; exact `ranking:` ending `; excludes acted/skipped`; deterministic |
|
||||
| SC7 | CLI act/skip/reset | `npm test` (cli.test) + manual | act→acted, skip→skipped, reset→new (via list --json); unknown id→exit 2; missing --id→exit 2 |
|
||||
| SC8 | CLI brief marks + --no-mark + exclude | `npm test` (cli.test) + manual | surfacedCount:1 + lastSurfacedAt + marked in json; second same-day marked:0; --no-mark no write; md omits acted/skipped |
|
||||
| SC9 | gate + wiring + de-niche | `bash scripts/test-runner.sh` | FAIL=0; trends ≥ floor; Section 16k green; ASSERT_BASELINE_FLOOR=105; Section 17; counts 27/19/29; hook suite green |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — the v3→v4 bump silently regresses the R3a migration block** (its hard-coded `3` literals; "v3
|
||||
idempotent" is false after the bump). *Mitigated:* Step 3 **explicitly reconciles** the `(RE-R3a / score v2→v3)`
|
||||
block (flip `3`→`SCHEMA_VERSION`, retitle the idempotent test to a forward-migration test); SC4 asserts the
|
||||
full suite green after the bump. This is the load-bearing migration subtlety — caught by premise verification
|
||||
before drafting, not after.
|
||||
- **R2 — re-score corrupts provenance / status / the seen-log** (over-broad mutation in `addTrend`). *Mitigated:*
|
||||
the duplicate branch touches **only** `topics` + `score`; `source`/`capturedAt`/`publishedAt`/`status`/
|
||||
`surfacedCount`/`lastSurfacedAt` are untouched; SC2 asserts each is preserved (incl. an acted-trend re-capture
|
||||
keeping `status:"acted"`).
|
||||
- **R3 — false-merge inflation** (a re-capture with an identical score reported as `merged`). *Mitigated:* the
|
||||
`JSON.stringify(existing.score) !== JSON.stringify(input.score)` guard (the envelope is built in a fixed key
|
||||
order by `scoreEnvelope`, so the compare is stable); SC2 asserts an identical re-score → `merged:false`.
|
||||
- **R4 — non-idempotent surfacing** (re-running today's brief double-counts; the autonomous loop (c) would inflate
|
||||
`surfacedCount`). *Mitigated:* `markSurfaced` skips records whose `lastSurfacedAt === today`; SC3 + SC8 assert a
|
||||
same-day re-call → `marked:0`, counts unchanged.
|
||||
- **R5 — the brief CLI's new store write loses data** (acted/skipped records dropped on resave, or the brief
|
||||
written from a mutated ranking). *Mitigated:* `markSurfaced` mutates only the `surfacedCount`/`lastSurfacedAt`
|
||||
of the **surfaced** ids on the **already-loaded full store**; `saveStore` writes the whole store (handled
|
||||
records preserved); the `.md` is rendered from the pure `ranking` **before** the mutation; SC8 asserts the md
|
||||
omits acted/skipped AND the store still contains them with surfacing recorded.
|
||||
- **R6 — `rankForBrief` loses purity** (the status filter or surfacing leaking fs/mutation into the pure render).
|
||||
*Mitigated:* the filter is a pure read of `effectiveStatus`; the seen-log **write** is only in the `brief` CLI
|
||||
edge, guarded by `--no-mark`; SC5/SC6 assert deterministic, byte-identical render; `markSurfaced`/`setStatus`/
|
||||
`surfacedIds`/`effectiveStatus` are all pure (no fs).
|
||||
- **R7 — losing the brief's determinism** (the id/marker tokens or the status filter perturbing the total order).
|
||||
*Mitigated:* the id + `surfacedToken` are deterministic reads; the filter only removes records, preserving the
|
||||
R3a composite total order on the survivors; SC5/SC6 assert byte-identical output.
|
||||
- **R8 — editing `trend-spotter.md` trips the de-niche guard.** *Mitigated:* Section 17 runs in the gate; the
|
||||
added prose is generic lifecycle wording (`acted`/`skipped`/"refresh the score"), vendor/sector-free.
|
||||
- **R9 — new gate checks must survive a deps-absent fresh clone.** *Mitigated:* Section 16k is pure
|
||||
`grep`/self-test on tracked source (no `tsx`) → unconditional; `TRENDS_TESTS_FLOOR` stays inside the deps guard.
|
||||
- **R10 — import cycle.** *Mitigated:* the new edges are one-way: `store.ts ← types.ts` (`TrendStatus` type into
|
||||
`store`, via `types`), `brief.ts → store.ts` (`effectiveStatus` value — brief already imports store), `cli.ts →
|
||||
{store,brief}` (already). `score.ts`/`item.ts` are untouched. The DAG stays acyclic: `score (leaf) ← types ←
|
||||
store ← brief ← cli`.
|
||||
- **R11 — the `status` name collision** (the CLI `status` staleness subcommand vs the new `TrendStatus` lifecycle).
|
||||
*Accepted:* the subcommand reads store staleness (newest capture), the type is the per-record lifecycle — no
|
||||
shared code; the collision is pre-existing and cosmetic; not reconciled here (documented non-goal).
|
||||
|
||||
## Plan-critic — folded
|
||||
|
||||
Three Opus reviewers ran, each verifying claims against live code: **scope-guardian ALIGNED** (0 creep, 0 gaps;
|
||||
counts 27/19/29 + `score.ts`/`item.ts`-untouched + A1/A2/A3-consistency verified live; 2 MINOR plan line-cites);
|
||||
**brief-reviewer PROCEED_WITH_RISKS** (all 9 load-bearing claims TRUE; the v3→v4 reconcile complete for every
|
||||
breaking literal, enumerated; 1 MEDIUM + 3 LOW); **plan-critic PROCEED_WITH_RISKS (78/B)** (the two-phase RED,
|
||||
the atomic bump+reconcile, the `merged` broadening's non-regression, the `surfacedIds` formula, and the gate
|
||||
arithmetic all verified correct; 1 MAJOR + 5 MINOR). Resolution, each verified against live code:
|
||||
|
||||
- **[MAJOR — plan-critic #1 / MEDIUM — brief-reviewer #1] the `brief` CLI's `store` binding does not exist.**
|
||||
`cli.ts:286` inlines `rankForBrief(loadStore(storePath), …)` — no `const store`, so the `markSurfaced(store,
|
||||
…)` / `saveStore(storePath, store)` edit referenced an undefined identifier (would not compile). ✅ Step 5 now
|
||||
**hoists** `const store = loadStore(storePath); const ranking = rankForBrief(store, …);`; the brief §3 S-cli +
|
||||
A1 wording + R5 corrected to the hoisted binding.
|
||||
- **[MINOR — plan-critic #2] not-found id → exit 2 contradicted the documented exit-code contract** (`cli.ts:33`
|
||||
"2 on usage error"; `cli.ts:31` data-conditions never via exit code). ✅ Step 5 **broadens the `:33`
|
||||
doc-comment** to "2 on usage error or a not-found id (act/skip/reset)" (a wrong `--id` is an argument-class
|
||||
error); no third code introduced.
|
||||
- **[MINOR — plan-critic #3] `· sett Nx` off-by-one** (render precedes the surfacing mutation → the token reflects
|
||||
the prior-day count). ✅ the **prior-day semantic** is now explicit (Step 4 + brief §3 + README): `· sett Nx` =
|
||||
"shown on N prior distinct days"; the `brief.test` sets `surfacedCount` directly, §8 exercises the cross-day path.
|
||||
- **[MINOR — plan-critic #4] `capture` tally comment stale + the re-score CLI tally untested.** ✅ Step 5 updates
|
||||
the `cli.ts:251-252` comment (`merged` = topics ∪ score-refresh); Step 1 + SC2 add a subprocess assertion that a
|
||||
re-captured changed-score item reports `merged:1` with the updated composite.
|
||||
- **[MINOR — plan-critic #5] Step 2 used `TrendStatus` before Step 3 defined it.** ✅ **reordered**: Step 2 adds
|
||||
the `TrendStatus` type + the three fields to `types.ts` first (then the `store.ts` functions); Step 3 isolates
|
||||
the atomic `SCHEMA_VERSION` bump + the R3a reconcile.
|
||||
- **[MINOR — plan-critic #6 / brief-reviewer LOW] stale R3a test titles + comment** (`store.test.ts:571`/`:598`
|
||||
titles, `:570` comment still say "v3" after the bump). ✅ Step 3 flips both titles to "the current version" +
|
||||
refreshes the comment, alongside the `:588`/`:602`/`:628` assertion flips.
|
||||
- **[LOW — brief-reviewer #4] forward-debt: the new R3b migration block hard-coded `4`** (perpetuating the
|
||||
reconcile-cycle). ✅ Step 3 commits the new block's **target + idempotent** assertions against `SCHEMA_VERSION`
|
||||
(future-proof; the hard-`4` is the Step-1 RED device only); the v3 **input** fixtures stay literal `3`.
|
||||
- **[LOW — brief-reviewer #3] `cli.test.ts:247` inert fixture not enumerated.** ✅ added to the scope-fence
|
||||
enumeration alongside `brief.test.ts:53` ("miss none" now literally true).
|
||||
- **[MINOR — scope-guardian] two plan line-cites** for the gate placement (`~:1191`/`:1262`). ✅ corrected to
|
||||
`~:1235` (after 16j's block) / `:1237` (Section 18 header) in Step 7 + the Files-touched table.
|
||||
|
||||
**scope-guardian — ALIGNED:** every SC1–SC9 traces to a step; zero creep, zero gaps; all §4 non-goals held (no
|
||||
saturation scoring, no scheduler, no brief-diff, no A1–A4, no status/surfaced input on capture, no act/skip-by-title,
|
||||
no auto-act-on-publish, no new source/agent/command, `BRIEF_SCHEMA_VERSION` unchanged); counts 27/19/29 verified
|
||||
live; `score.ts`/`item.ts` untouched verified (`itemToInput` already builds the envelope on every capture incl.
|
||||
re-capture); the R3a-block reconcile is a necessary prerequisite the bump forces, not creep.
|
||||
|
||||
**[plan-critic headless-readiness] N/A** — R3b executes **in-session, operator-driven** (driftsmodell), not as a
|
||||
headless autonomous run, so per-step revert/halt clauses aren't needed (R1/R2a/R2b/R3a had none either).
|
||||
376
docs/research-engine/plan-re-r3c.md
Normal file
376
docs/research-engine/plan-re-r3c.md
Normal file
|
|
@ -0,0 +1,376 @@
|
|||
# Plan — RE-R3c: autonomous trigger — scheduler + headless entry point
|
||||
|
||||
> **Brief:** `docs/research-engine/brief-re-r3c.md`. **Slice:** RE-R3c (research-engine rung-2 — R3 slice 3, the
|
||||
> **autonomy** slice: the scheduler trigger + the headless entry that runs the deterministic morning brief with no
|
||||
> interactive session). Closes hulls (1) no autonomous trigger + (6) no headless entry point.
|
||||
> **TDD-order (two-phase RED — light-Voyage discipline, inherited):** Step 1 records RED in two phases — **(A)**
|
||||
> subprocess assertion-RED against the **existing** CLI (`schedule` is an unknown command → `usage` exit 2; the
|
||||
> `run-daily.sh` wrapper file is absent → exit 127) on the exit-0/stdout assertions; **(B)** for the NEW
|
||||
> `schedule.ts` exports the tests import, land **non-throwing stubs** first (Node16 ESM throws a missing named
|
||||
> import at module-load, not on assertion), then record value-assertion RED against them. Then GREEN: `schedule.ts`
|
||||
> stubs→real → `run-daily.sh` wrapper → `cli.ts` `schedule` verb → wire `trend-spotter.md` (prose) + README →
|
||||
> gate floors + Section 16l → behavioural → land.
|
||||
> **Counts recounted live at land, never pinned/guessed.**
|
||||
> **Architectural decisions (CONFIRMED, AskUserQuestion 2026-06-26):** C1 deterministic brief-only (no AI capture —
|
||||
> that is slice e) · C2 print-first installer (emit + the operator runs the system mutation; `--install` writes only
|
||||
> the inert launchd plist file; the tool never runs `launchctl`/`crontab`). Go-gate D1–D9 baked to recommended
|
||||
> defaults (brief §8).
|
||||
> **Light-Voyage:** scope-guardian / brief-reviewer / plan-critic to run on these drafts; findings folded in
|
||||
> §Plan-critic — folded before the code commit.
|
||||
|
||||
## Goal
|
||||
|
||||
Make the daily research loop **closed** and **headless**. (1) **Autonomous trigger** — a `schedule` CLI verb that
|
||||
emits (print-first) a launchd plist (macOS) / crontab line (Linux) firing a daily brief, and `--install` that
|
||||
writes only the inert launchd plist file (the operator runs the one activation command). (2) **Headless entry
|
||||
point** — `run-daily.sh`, a bash-3.2 wrapper that runs the **deterministic** `brief` from a scheduler's
|
||||
profile-less env (resolves node + the data-dir log seam + working dir), invoked identically by launchd and cron.
|
||||
**No AI capture** (C1 — that is slice e, which plugs into the documented pre-brief seam); **no schema change**
|
||||
(`SCHEMA_VERSION` 4 / `BRIEF_SCHEMA_VERSION` 1 untouched); **no new agent/command/reference**. Two new source
|
||||
files under `scripts/trends/` (`schedule.ts` pure module + `run-daily.sh` wrapper) + their tests; EDIT `cli.ts` +
|
||||
`trend-spotter.md` (prose) + README + the gate.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | SC |
|
||||
|---|---|---|
|
||||
| `scripts/trends/src/schedule.ts` | **NEW (pure module)** — `ScheduleSpec` (incl. `env: Record<string,string>`); `launchdPlist`/`crontabLine`/`installInstructions`/`uninstallInstructions`/`defaultLabel`. No clock/fs/env/AI (renders `spec.env`; reads no env). | SC1, SC2 |
|
||||
| `scripts/trends/run-daily.sh` | **NEW (wrapper)** — bash 3.2; resolves own dir + **`cd "$DIR"`** + node (`NODE_BIN`/`command -v`) + log seam (`${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/trends/cron.log`); runs `cli.ts" brief "$@" --json`, **compacts** the pretty-printed json to one line; logs `<ts> exit=<code> <compact-json>`; (e)-seam comment. | SC7, SC8 |
|
||||
| `scripts/trends/src/cli.ts` | **EDIT** — `schedule --pillars <a,b> [--at HH:MM] [--fresh-days N] [--platform auto\|launchd\|cron] [--install] [--uninstall] [--store <p>]`; adds `dirname` (node:path) + `homedir` (node:os) + `fileURLToPath` (node:url) imports; resolves paths from runtime (`process.execPath`/`import.meta.url`/`defaultStorePath`); `logPath = join(dirname(defaultStorePath()), "cron.log")`; `env` always `{NODE_BIN, LINKEDIN_STUDIO_DATA(resolved)}`; `args` without leading `"brief"`; print-first dispatch; `--install` writes the inert plist only (never `launchctl`/`crontab`); usage + header doc; **no new exit code** | SC1–SC6, SC9 |
|
||||
| `scripts/trends/tests/schedule.test.ts` | **NEW** — pure-emitter assertions: plist keys (Label/ProgramArguments/StartCalendarInterval/Std*Path/EnvironmentVariables), crontab line shape, install/uninstall instruction strings, `defaultLabel`, determinism | SC1, SC2 |
|
||||
| `scripts/trends/tests/cli.test.ts` | **EDIT** — subprocess: `schedule --print` (launchd + cron) emits the artifact + instruction, exit 0; `--platform auto` branches on `process.platform`; `--print` writes nothing (temp HOME LaunchAgents empty); `--install` launchd writes the plist file + prints `launchctl` (never runs it); `--install` cron prints only; no `--pillars`/bad `--at`/bad `--platform` → exit 2 | SC3–SC6, SC9 |
|
||||
| `scripts/trends/tests/run-daily.test.ts` | **NEW** — subprocess: wrapper on a seeded store writes the dated brief + appends to `cron.log` + exit 0; second same-day run → byte-identical `.md`, `surfacedCount` not double-counted, second log line; data-path twin consistency (wrapper log dir == `defaultStorePath` dir == `getDataRoot('trends')`) | SC7, SC8 |
|
||||
| `scripts/trends/src/brief.ts` | **UNTOUCHED** — the nightly run reuses the existing deterministic `brief`/`rankForBrief`/`renderBrief`. Listed to assert it is *not* in scope. | — |
|
||||
| `scripts/trends/src/store.ts` · `types.ts` · `item.ts` · `score.ts` | **UNTOUCHED** — no data-shape / scoring / render change. Listed to assert they are *not* in scope. | — |
|
||||
| `agents/trend-spotter.md` | **EDIT (prose-only, minimal)** — one line: the brief can now be scheduled to regenerate autonomously (deterministic, from the store) via `schedule`; polling stays the capture path (autonomous AI polling is a later slice). Domain-general. | — |
|
||||
| `scripts/trends/README.md` | **EDIT** — the headless wrapper + `schedule` (print-first, launchd/cron, `--install`/`--uninstall`), the deterministic-brief-only boundary (C1) + the (e) seam, `cron.log`, per-day idempotency | — |
|
||||
| `scripts/test-runner.sh` | **EDIT** — `TRENDS_TESTS_FLOOR` 171→recount + breakdown comment (`:709`); NEW unconditional **Section 16l** (after 16k's block `~:1305`, before Section 18 header `:1307`); `ASSERT_BASELINE_FLOOR` 105→**111** (`:1329`); header-enumeration chain (`:57`) + Section-18 floor-history narration (`~:1310-1324`) | SC10 |
|
||||
| `docs/research-engine/{brief,plan}-re-r3c.md` | **NEW** — slice docs (TRACKED, like `docs/second-brain/*`) | — |
|
||||
| `STATE.md` | **EDIT at land** — Telling-block reconcile (trends floor, ASSERT floor 111, gate total; schema unchanged v4). *Land bookkeeping, LOCAL-ONLY.* | — |
|
||||
|
||||
**Not touched (scope fence):** `brief.ts`/`store.ts`/`types.ts`/`item.ts`/`score.ts` (the nightly run reuses the
|
||||
existing brief path; no data/scoring/render change) · `references/trend-scoring-modes.md` + `algorithm-signals-
|
||||
reference.md` (no scoring change) · the SessionStart hook + its tests (R3c generates the brief the hook already
|
||||
surfaces; no hook change, no frontmatter-schema change, no new hook test) · `config/*` · `commands/*` (29 — no
|
||||
new command) · `agents/*` count (19 — `trend-spotter.md` is a prose EDIT) · `references/*` (27) · `.gitignore`
|
||||
(`scripts/trends/{node_modules,build}` already covered; `cron.log` lives under the external data dir, never in the
|
||||
repo) · `BRIEF_SCHEMA_VERSION` (1) · `SCHEMA_VERSION` (4).
|
||||
|
||||
## Step 1 — (RED, two phases) failing tests across schedule/cli/wrapper
|
||||
|
||||
**Phase A — subprocess assertion-RED against the pre-edit code** (no new import needed; the CLI is invoked as a
|
||||
subprocess and the wrapper file is simply absent):
|
||||
- `cli.test.ts` — **`schedule`**: `schedule --pillars ai,gov --platform launchd --print` today → `usage` (unknown
|
||||
command) **exit 2**, no plist on stdout → RED against the assertion (expects exit 0 + a plist with `Label
|
||||
com.linkedin-studio.trends.daily`); `--platform cron --print` → RED (expects the crontab line + the `crontab -`
|
||||
instruction string); `--print` with a temp `HOME` → RED (expects `<HOME>/Library/LaunchAgents` empty *and* exit
|
||||
0); no `--pillars` / `--at 25:00` / `--platform bogus` → these already exit 2 today (unknown command), so assert
|
||||
the **post-implementation** behaviour (still exit 2, but for the validation reason) — recorded as RED only where
|
||||
the message/route differs (kept minimal; the load-bearing RED is the `--print` emit). *(Fail today: `schedule`
|
||||
is an unknown command.)*
|
||||
- `run-daily.test.ts` — **wrapper**: invoke **via `bash scripts/trends/run-daily.sh --pillars ai --store <tmp>
|
||||
--out <tmp>/mb`** (through `bash`, NOT a direct executable spawn — folded — plan-critic #7: a direct exec of a
|
||||
missing file throws ENOENT/`status:null`, a module-not-found-class failure; `bash <missing>` exits **127**, a
|
||||
clean assertion-RED) with `LINKEDIN_STUDIO_DATA=<tmp>` → the file is **absent** → exit **127** → RED against the
|
||||
assertion (expects the dated `.md` written + exactly one `cron.log` line + exit 0). *(Fail today: the wrapper does
|
||||
not exist.)*
|
||||
|
||||
**Phase B — stub-first, then value-assertion RED** (`schedule.test` imports the new `schedule.ts` exports):
|
||||
- Land **non-throwing stubs** so the imports resolve: in `schedule.ts` — `launchdPlist → ""`, `crontabLine → ""`,
|
||||
`installInstructions → ""`, `uninstallInstructions → ""`, `defaultLabel → ""` (+ the `ScheduleSpec` interface,
|
||||
erased by tsx). (Wrong-value stubs Step 2 replaces.)
|
||||
- `schedule.test.ts`: `launchdPlist(spec)` contains `<key>Label</key>` + `spec.label` + `StartCalendarInterval` +
|
||||
`spec.hour`/`spec.minute` + the `cron.log` path (stub `""` → RED); `crontabLine(spec)` matches the
|
||||
`<min> <hour> * * * … run-daily.sh … >> <log> 2>&1 # <label>` shape (stub `""` → RED); `installInstructions` /
|
||||
`uninstallInstructions` carry the `launchctl bootstrap` / `crontab -` recipes (stub `""` → RED); `defaultLabel()`
|
||||
is `com.linkedin-studio.trends.daily` (stub `""` → RED); two `launchdPlist` calls byte-identical.
|
||||
|
||||
**RED proof (record in commit, two phases):** Phase A — `(cd scripts/trends && npm test)` before any src edit →
|
||||
the `schedule` cli cases fail on the exit-0/stdout assertion (logic-RED) and the wrapper case fails on exit 127,
|
||||
not on a missing module. Phase B — after the non-throwing `schedule.ts` stubs land, the `schedule.test` cases fail
|
||||
on **value assertion** against the `""` stubs. The plan does **not** claim a single "everything fails before any
|
||||
code" run.
|
||||
|
||||
## Step 2 — (GREEN) `scripts/trends/src/schedule.ts` — the pure emitters
|
||||
|
||||
Replace the Phase-B stubs with the real, pure implementations (no clock/fs/env/AI; every value via `ScheduleSpec` —
|
||||
the emitters **render** `spec.env`, they never read `process.env`):
|
||||
- `export interface ScheduleSpec { platform: "launchd" | "cron"; label: string; nodeBin: string; wrapperPath:
|
||||
string; args: string[]; hour: number; minute: number; logPath: string; workingDir: string; env:
|
||||
Record<string, string>; }`. **`env` is canonical** (folded — brief-reviewer #4 / plan-critic #3: declared once,
|
||||
here + brief §3 + the Files table + the Step-1 fixtures — not mutated mid-step). The CLI always builds it as
|
||||
`{ NODE_BIN, LINKEDIN_STUDIO_DATA }`; the emitters only stringify it.
|
||||
- `launchdPlist(spec)` — a pinned, well-formed `<?xml … !DOCTYPE plist …>` template: `Label`=`spec.label`;
|
||||
`ProgramArguments` = `["/bin/bash", spec.wrapperPath, ...spec.args]` (each as a `<string>`);
|
||||
`StartCalendarInterval` `<dict>` with `<key>Hour</key><integer>${spec.hour}</integer>` + `Minute`;
|
||||
`EnvironmentVariables` `<dict>` rendered from **`spec.env`** (each `K`→`<key>K</key><string>V</string>`);
|
||||
`WorkingDirectory`=`spec.workingDir`; `StandardOutPath`/`StandardErrorPath`=`spec.logPath`; `RunAtLoad`=`<false/>`.
|
||||
XML-escape any value that could contain `&`/`<`/`>` (paths are safe, but escape defensively for well-formedness).
|
||||
- `crontabLine(spec)` — `${spec.minute} ${spec.hour} * * * ${envPrefix} /bin/bash ${spec.wrapperPath}
|
||||
${spec.args.join(" ")} >> ${spec.logPath} 2>&1 # ${spec.label}` where `envPrefix` = the `spec.env` map as
|
||||
`K=V K=V` (cron's inline env form). **Returns a string; never executes `crontab`.**
|
||||
- `installInstructions(spec, plistTargetPath?)` — launchd: `"Wrote ${plistTargetPath}. Activate:\n launchctl
|
||||
bootstrap gui/$(id -u) ${plistTargetPath}"`; cron: `"Add the line above:\n (crontab -l 2>/dev/null; echo
|
||||
'${crontabLine(spec)}') | crontab -"`.
|
||||
- `uninstallInstructions(spec, plistTargetPath?)` — launchd: `"launchctl bootout gui/$(id -u)/${spec.label} && rm
|
||||
${plistTargetPath}"`; cron: `"crontab -l | grep -vF '# ${spec.label}' | crontab -"`.
|
||||
- `defaultLabel()` — `"com.linkedin-studio.trends.daily"`.
|
||||
Make the Phase-B `schedule.test` cases green.
|
||||
|
||||
## Step 3 — (GREEN) `scripts/trends/run-daily.sh` — the headless wrapper
|
||||
|
||||
Create the bash-3.2 wrapper (Write a new file under `scripts/trends/` — allowed; if Write is blocked, the operator
|
||||
authorizes via the R2b `!cp` fallback, Risk R4). Shape:
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
# RE-R3c headless entry: runs the DETERMINISTIC morning brief from a scheduler's
|
||||
# profile-less env. No AI. The (e) slice will insert a pre-brief capture step here.
|
||||
# Data-path: the FOURTH sanctioned twin of store.ts:defaultStorePath / data-root.mjs:getDataRoot
|
||||
# / analytics/storage.ts:getDataRoot (shell form of the references/data-path-convention.md
|
||||
# inline seam). Keep in sync.
|
||||
set -eu
|
||||
DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
cd "$DIR" # so `--import tsx` resolves node_modules even under cron's $HOME CWD
|
||||
NODE_BIN="${NODE_BIN:-$(command -v node 2>/dev/null || true)}"
|
||||
if [ -z "$NODE_BIN" ]; then for c in /usr/local/bin/node /opt/homebrew/bin/node /usr/bin/node; do
|
||||
[ -x "$c" ] && NODE_BIN="$c" && break; done; fi
|
||||
LOG="${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/trends/cron.log"
|
||||
mkdir -p "$(dirname "$LOG")"
|
||||
TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
if [ -z "$NODE_BIN" ]; then printf '%s exit=127 node not found\n' "$TS" >> "$LOG"; exit 127; fi
|
||||
set +e
|
||||
OUT="$("$NODE_BIN" --import tsx "$DIR/src/cli.ts" brief "$@" --json 2>&1)"; CODE=$?
|
||||
set -e
|
||||
OUT="$(printf '%s' "$OUT" | tr '\n' ' ' | tr -s ' ')" # brief --json is pretty-printed -> one line
|
||||
printf '%s exit=%s %s\n' "$TS" "$CODE" "$OUT" >> "$LOG"
|
||||
exit "$CODE"
|
||||
```
|
||||
- `chmod +x` (or the CLI's `--install` does it). All expansions quoted; ASCII-only; no bash-4 features. `cd "$DIR"`
|
||||
makes the one wrapper scheduler-agnostic (folded — brief-reviewer #1). `CODE=$?` is captured **before** the
|
||||
compaction pipe (so it is node's code, not `tr`'s — folded — plan-critic #1).
|
||||
- The scheduler bakes `NODE_BIN=<process.execPath>` + a resolved-absolute `LINKEDIN_STUDIO_DATA` (so a scheduled
|
||||
run never evaluates `$HOME` under `set -u` — folded — plan-critic #9) + `--pillars … --fresh-days N` into `"$@"`;
|
||||
the wrapper hard-codes the `brief` subcommand (the baked `args` carry **no** leading `brief` — folded —
|
||||
plan-critic #8) and appends `--json`.
|
||||
Make `run-daily.test.ts` green (SC7, SC8): the seeded-store run writes the dated `.md` + one compact log line +
|
||||
exit 0; a `cwd=<unrelated>` run still resolves `tsx` (CWD-independence); a same-day re-run is byte-identical with
|
||||
`surfacedCount` not double-counted.
|
||||
|
||||
## Step 4 — (GREEN) `cli.ts` — the `schedule` subcommand
|
||||
|
||||
- Import `{ launchdPlist, crontabLine, installInstructions, uninstallInstructions, defaultLabel }` from
|
||||
`./schedule.js`; **add `dirname` to the `node:path` import** (`cli.ts:41` is `import { join } from "node:path"`
|
||||
— `dirname` is NOT there today, folded — plan-critic #5; `join` stays); import `homedir` from `node:os` (the
|
||||
LaunchAgents path) + `fileURLToPath` from `node:url`. `defaultStorePath` is already imported (`cli.ts:45`).
|
||||
- A `schedule` branch (after the `brief` branch, before the trailing `usage`):
|
||||
- `const pillars = splitTopics(flags.pillars); if (pillars.length === 0) usage("schedule needs --pillars <a,b>");`
|
||||
- parse `--at` (default `"07:00"`): split on `:`, `Number.parseInt` both; validate `0≤H≤23`, `0≤M≤59` → else
|
||||
`usage("--at must be HH:MM (00:00–23:59)")`.
|
||||
- parse `--fresh-days` (default 7, reuse the `brief` validation idiom).
|
||||
- `const platform = flags.platform && flags.platform !== "true" ? flags.platform : (process.platform ===
|
||||
"darwin" ? "launchd" : "cron");` validate `platform ∈ {launchd, cron, auto}` (auto already resolved) → else
|
||||
`usage`.
|
||||
- resolve runtime paths (no hard-coding): `const here = dirname(fileURLToPath(import.meta.url)); // …/src`;
|
||||
`const wrapperPath = join(here, "..", "run-daily.sh"); const workingDir = join(here, ".."); const nodeBin =
|
||||
process.execPath;`. **logPath (pinned — folded, all three reviewers):** `const logPath = join(dirname
|
||||
(defaultStorePath()), "cron.log");` — derived from `defaultStorePath()` (= `<root>/trends/trends.json` →
|
||||
`dirname` = `<root>/trends` → `<root>/trends/cron.log`), the data-root anchor the wrapper also uses; **NOT**
|
||||
`dirname(storePath)` (the `--store` override) and **NOT** with a spurious `".."`. Matches `defaultBriefDir`'s
|
||||
`dirname(defaultStorePath())` idiom (`brief.ts:236`), so `cron.log` is a sibling of `morning-brief/`.
|
||||
- `const root = process.env.LINKEDIN_STUDIO_DATA ?? join(homedir(), ".claude", "linkedin-studio");
|
||||
const env: Record<string,string> = { NODE_BIN: nodeBin, LINKEDIN_STUDIO_DATA: root };` — **always** bake the
|
||||
resolved-absolute root (pins the scheduled run to the install-time data dir **and** removes the wrapper's
|
||||
`$HOME`-unset `set -u` edge — folded — plan-critic #9 / brief-reviewer #9).
|
||||
- `const args = ["--pillars", pillars.join(","), "--fresh-days", String(freshDays)];` **(no leading `"brief"`** —
|
||||
the wrapper owns the subcommand; folded — plan-critic #8) and append `["--store", storePath]` **iff** the
|
||||
operator passed an explicit `--store` (so the scheduled run targets the same store; otherwise the wrapper's
|
||||
default resolves it).
|
||||
- `const label = defaultLabel(); const spec: ScheduleSpec = { platform, label, nodeBin, wrapperPath, args, hour,
|
||||
minute, logPath, workingDir, env };`
|
||||
- dispatch:
|
||||
- **`--uninstall`** → print `uninstallInstructions(spec, plistTarget)`; if the launchd plist file exists,
|
||||
`rmSync` it (reversible); exit 0.
|
||||
- **`--install`** → launchd: `const plistTarget = join(homedir(), "Library", "LaunchAgents", `${label}.plist`);
|
||||
mkdirSync(dirname, {recursive:true}); writeFileSync(plistTarget, launchdPlist(spec))` + `console.log` the
|
||||
plist path + `installInstructions(spec, plistTarget)` — **never run `launchctl`**. cron: `console.log(crontabLine
|
||||
(spec)); console.log(installInstructions(spec))` — **never run `crontab`**. exit 0.
|
||||
- **default / `--print`** → `console.log(platform === "launchd" ? launchdPlist(spec) : crontabLine(spec));
|
||||
console.log(installInstructions(spec, platform === "launchd" ? join(homedir(),"Library","LaunchAgents",
|
||||
`${label}.plist`) : undefined));` — **no fs**. exit 0.
|
||||
- **Header doc-comment** (`cli.ts:5-14`, `:36-37`): add the `schedule …` synopsis line + a note that `schedule` is
|
||||
**print-first** (emits the plist/crontab + the install command; `--install` writes only the inert launchd plist
|
||||
file; the tool never runs `launchctl`/`crontab`) and runs the **deterministic** brief (no AI capture — slice e).
|
||||
**No new exit code** (0 success / 2 usage): an autonomy install never *runs* the system mutation, so there is no
|
||||
install-failure path to encode.
|
||||
- `usage()` (`:86-100`) — add the `schedule --pillars <a,b> [--at HH:MM] [--fresh-days N] [--platform
|
||||
auto|launchd|cron] [--install|--uninstall] [--store <path>]` line.
|
||||
Make the Phase-A `cli.test` `schedule` cases green (print emit, auto-platform, print-writes-nothing,
|
||||
install-launchd-writes-plist, install-cron-prints-only, validation exits).
|
||||
|
||||
## Step 5 — wire `trend-spotter.md` (prose) + README
|
||||
|
||||
In `agents/trend-spotter.md`: add **one prose line** (no batch-shape change): the morning brief can now be
|
||||
**scheduled** to regenerate autonomously (deterministic, from the store) via the `schedule` CLI verb; the agent's
|
||||
polling stays the capture path (autonomous AI polling is a later slice). Domain-general (Section 17). Update
|
||||
`scripts/trends/README.md`: the headless wrapper (`run-daily.sh`) + the `schedule` subcommand (print-first,
|
||||
launchd/cron, `--install`/`--uninstall`, `--at`/`--platform`), the **deterministic-brief-only boundary (C1)** + the
|
||||
documented (e) AI-capture seam, the `cron.log` location, and the R3b per-day idempotency that makes a double-fire
|
||||
safe.
|
||||
|
||||
## Step 6 — gate: floors + new unconditional Section 16l
|
||||
|
||||
In `scripts/test-runner.sh`:
|
||||
- Set `TRENDS_TESTS_FLOOR` (`:709`, currently 171) to the **`tests N` line** reported by `(cd scripts/trends &&
|
||||
npm test)` after Steps 1–5 — recounted live, NOT additive-guessed. Stays **inside** the deps guard. **Append**
|
||||
`+ RE-R3c: scheduler +N` to the inline breakdown comment.
|
||||
- Add **Section 16l** ("Trends Scheduler / Headless Wiring", RE-R3c), mirroring Section 16k (unconditional,
|
||||
deps-absent-safe, pure `grep -qF`/self-test, no `tsx`). **Placement (verified live):** Section 16k ends `~:1305`,
|
||||
Section 18 begins `:1307` — insert 16l **after 16k's block (`~:1305`), before Section 18 (`:1307`)** (anti-erosion
|
||||
must stay last so it counts every prior check). Six **unconditional** checks, the self-test emitting **one**
|
||||
pass/fail like 16k:
|
||||
(1) a non-vacuity self-test (a probe carrying `launchdPlist` accepted, one without rejected);
|
||||
(2) `grep -qF 'export function launchdPlist' scripts/trends/src/schedule.ts`;
|
||||
(3) `grep -qF 'export function crontabLine' scripts/trends/src/schedule.ts`;
|
||||
(4) `grep -qF 'command === "schedule"' scripts/trends/src/cli.ts` (the verb);
|
||||
(5) `grep -qF 'cli.ts" brief' scripts/trends/run-daily.sh` (the wrapper runs the deterministic brief — the
|
||||
sentinel matches the literal `…cli.ts" brief`, folded — plan-critic #8);
|
||||
(6) `grep -qF 'LINKEDIN_STUDIO_DATA:-' scripts/trends/run-daily.sh` (the data-path twin seam).
|
||||
- Bump `ASSERT_BASELINE_FLOOR` (**`:1329`**, currently 105 — folded, all three reviewers: the earlier `:1259` cite
|
||||
was wrong, that line is `LIFECYCLE_FILTER_LIT`) → **exactly 111** (105 + the 6 new unconditional 16l emitters;
|
||||
the self-test emits one pass/fail like 16k, so 111 is deterministic — "live recount" is the safety net, not a
|
||||
guess). Insert the 16l clause into the **header-enumeration prose chain at `:57`** (before "…the assertion-count
|
||||
anti-erosion floor (SC6) in Section 18"), preserving sentence flow. **Also append** the RE-R3b (→105) + RE-R3c
|
||||
16l (→111) narration to the **Section-18 floor-history comment** (`~:1310-1324`, which still stops at "= 99"
|
||||
because R3b's +6 was never narrated — folded — scope-guardian #7), so the comment matches the live floor.
|
||||
- **NOT touched here:** the hook suite (no `HOOK_TESTS_FLOOR` in `test-runner.sh`; R3c adds no hook test). It must
|
||||
still pass untouched (`node --test hooks/scripts/__tests__/*.test.mjs`) as a regression sanity at land.
|
||||
|
||||
## Step 7 — behavioural verification
|
||||
|
||||
`(cd scripts/trends && npm install)` if needed, then run brief §7's six behavioural steps: seed a store; `schedule
|
||||
… --platform launchd --print` + `… --platform cron --print` (inspect); `plutil -lint -` the plist → OK;
|
||||
`LINKEDIN_STUDIO_DATA=/tmp/r3c-data ./run-daily.sh brief --pillars ai,gov --store /tmp/r3c.json --out
|
||||
/tmp/r3c-data/trends/morning-brief` → dated `.md` written + `cron.log` line + exit 0; re-run same day → `.md`
|
||||
byte-identical, `surfacedCount:1` (per-day idempotent), second log line; `schedule … --install` with a throwaway
|
||||
`HOME` → the plist file written under `<HOME>/Library/LaunchAgents/` + the `launchctl bootstrap` command printed
|
||||
(do **not** activate against the real system unless intentional). Run full `bash scripts/test-runner.sh` → `FAIL=0`
|
||||
(`ASSERT_BASELINE_FLOOR` 111); run `node --test hooks/scripts/__tests__/*.test.mjs` → still green (untouched
|
||||
regression).
|
||||
|
||||
## Step 8 — land
|
||||
|
||||
Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor 111, gate total;
|
||||
schema unchanged v4). Commit order (house style): **(1)** docs commit `docs/research-engine/{brief,plan}-re-r3c.md`
|
||||
(no suffix, tracked); **(2)** code commit — `schedule.ts` + `run-daily.sh` + `cli.ts` + three test files
|
||||
(`schedule.test`/`cli.test`/`run-daily.test`) + `agents/trend-spotter.md` + `scripts/trends/README.md` +
|
||||
`scripts/test-runner.sh` with `[skip-docs]` (D9: single code commit — the scheduler is one coherent feature).
|
||||
Push freely (window lifted; gitleaks at commit; `origin` = PUBLIC `open/` — STATE/`*.local.*` never pushed). No
|
||||
version bump (additive; `v0.5.2` dev).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|---|---|---|---|
|
||||
| — | RED Phase A | `(cd scripts/trends && npm test)` before src edits | `schedule` cli cases fail on the exit-0/stdout assertion (logic-RED); wrapper case fails on exit 127, not module-not-found |
|
||||
| — | RED Phase B | `npm test` after non-throwing `schedule.ts` stubs | launchdPlist/crontabLine/install/uninstall/defaultLabel fail on value assertion against the `""` stubs |
|
||||
| SC1 | launchd plist emit | `npm test` (schedule.test, cli.test) | plist has Label/ProgramArguments/StartCalendarInterval(H/M)/Std*Path/EnvironmentVariables; deterministic |
|
||||
| SC2 | crontab line emit (string only) | `npm test` (schedule.test, cli.test) | `<m> <h> * * * … run-daily.sh brief --pillars … >> <log> 2>&1 # <label>` + `crontab -` instruction; never executes crontab |
|
||||
| SC3 | platform auto | `npm test` (cli.test) | darwin→launchd, else→cron; branches on `process.platform` |
|
||||
| SC4 | print-first writes nothing | `npm test` (cli.test) | `--print` → exit 0, stdout plist, temp-HOME LaunchAgents empty |
|
||||
| SC5 | `--install` launchd inert plist | `npm test` (cli.test) | plist FILE written under temp HOME LaunchAgents; `launchctl bootstrap` printed; launchctl never run; exit 0 |
|
||||
| SC6 | `--install` cron prints only | `npm test` (cli.test) | line + `crontab -` instruction printed; crontab never run; exit 0 |
|
||||
| SC7 | headless wrapper runs brief + logs | `npm test` (run-daily.test, via `bash`) + manual | dated `.md` written; **one** compact `cron.log` line; exit 0; second same-day run byte-identical + surfacedCount not double-counted; CWD-independent (`cd "$DIR"`); no AI |
|
||||
| SC8 | data-path twin consistency | `npm test` (run-daily.test) | wrapper log dir == `dirname(defaultStorePath())` (not `--store`) == `getDataRoot('trends')`, default + override |
|
||||
| SC9 | usage / validation | `npm test` (cli.test) | no `--pillars`/bad `--at`/bad `--platform` → exit 2; fs untouched |
|
||||
| SC10 | gate + wiring + de-niche | `bash scripts/test-runner.sh` | FAIL=0; trends ≥ floor; Section 16l green; ASSERT_BASELINE_FLOOR=111; Section 17; counts 29/19/27; hook suite green |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — the wrapper's data-path twin drifts from `store.ts`/`data-root.mjs`.** *Mitigated:* SC8 behavioral
|
||||
twin-consistency test (resolve all three for a temp `LINKEDIN_STUDIO_DATA` override; assert equal); the wrapper
|
||||
uses the exact `references/data-path-convention.md` rule-1 inline form; documented as the third twin (mirroring
|
||||
`data-root.mjs`'s comment).
|
||||
- **R2 — a test runs `crontab`/`launchctl` and trips the global guard or mutates the real system.** *Mitigated:*
|
||||
every test asserts the **emitted string / written file on stdout/temp-HOME** — none executes `crontab` or
|
||||
`launchctl`; `--install` tests use a throwaway `HOME` and assert only the inert plist file + printed command.
|
||||
(A test that ran `crontab` would be a BLOCKER — invariant §5.)
|
||||
- **R3 — malformed launchd plist (won't load).** *Mitigated:* SC1 asserts every required key; the plist is a
|
||||
pinned `<!DOCTYPE plist …>` template; behavioural Step 7 runs `plutil -lint -` (macOS, deps-present manual check
|
||||
— not a gate, since a deps-absent fresh clone has no `plutil`).
|
||||
- **R4 — the two NEW files (`schedule.ts`, `run-daily.sh`) blocked at Write.** *Mitigated:* they are under
|
||||
`scripts/trends/` (allowed — `cli.ts`/`test-runner.sh` were added there; the pathguard blocks NEW `.mjs` under
|
||||
`hooks/scripts/` only). If Write is nonetheless blocked, the operator authorizes the R2b `!cp` fallback
|
||||
(write to scratch, `cp` in). This is an implementation-step risk, not a docs-step blocker.
|
||||
- **R5 — the nightly deterministic brief is a near-no-op without new captures (thin visible value).**
|
||||
*Accepted/honest (C1):* R3c is the **mechanism**; freshness-aging + per-day surfacing accumulation are real (feed
|
||||
slice b), but the visible autonomous-research payoff lands with slice (e), which plugs AI capture into the
|
||||
documented pre-brief seam. Stated plainly in the README + the brief §1 value framing — no salesmanship.
|
||||
- **R6 — `--at HH:MM` parse / validation gaps.** *Mitigated:* explicit `0≤H≤23`/`0≤M≤59` validation → `usage`
|
||||
exit 2; SC9 asserts `25:00`/`7:99`/`noon` all exit 2.
|
||||
- **R7 — bash 3.2 incompatibility / `set -u` crash in the wrapper.** *Mitigated:* ASCII-only, all expansions
|
||||
quoted, no `declare -A`/`mapfile`/`|&`; the `set +e`/`set -e` fence around the node call captures the exit code
|
||||
cleanly; the operator's macOS (bash 3.2) is the test bed; `run-daily.test` runs it as a subprocess in CI-shape.
|
||||
- **R8 — node unresolvable in launchd/cron's profile-less env.** *Mitigated:* the scheduler bakes
|
||||
`NODE_BIN=<process.execPath>` (absolute) into the artifact env; the wrapper falls back to `command -v node` then
|
||||
common locations, logging `exit=127 node not found` if truly absent; `WorkingDirectory` is set so tsx resolves.
|
||||
- **R9 — editing `trend-spotter.md` / the new source trips the de-niche guard.** *Mitigated:* Section 17 runs in
|
||||
the gate; the added prose + the source carry only generic scheduling wording; the launchd Label is the plugin
|
||||
namespace; pillars are args (no vendor/sector token).
|
||||
- **R10 — new gate checks must survive a deps-absent fresh clone.** *Mitigated:* Section 16l is pure
|
||||
`grep`/self-test on tracked source (`schedule.ts` + `cli.ts` + `run-daily.sh`; no `tsx`) → unconditional;
|
||||
`TRENDS_TESTS_FLOOR` stays inside the deps guard.
|
||||
- **R11 — import cycle.** *Mitigated:* `schedule.ts` is a **leaf** (imports nothing from the package); `cli.ts`
|
||||
(root) imports it. The DAG stays acyclic: `score (leaf) ← types ← store ← brief ← cli`, with `schedule (leaf) ←
|
||||
cli` added. `brief.ts`/`store.ts`/`item.ts`/`score.ts`/`types.ts` untouched.
|
||||
|
||||
## Plan-critic — folded
|
||||
|
||||
Three Opus reviewers ran, each verifying claims against live code: **scope-guardian ALIGNED** (0 creep, 0 gaps;
|
||||
every SC1–SC10 traces to a step; no AI/capture, no schema bump, counts 29/19/27 + the untouched-files claim
|
||||
verified live; 1 MAJOR line-cite + 6 MINOR); **brief-reviewer PROCEED_WITH_RISKS** (RED premises TRUE, the
|
||||
data-path trio genuinely consistent, the C2 no-execution path confirmed; 2 MAJOR + 5 MEDIUM/LOW); **plan-critic
|
||||
REVISE 73/C** (the two-phase RED, de-niche safety, the bash-3.2 `set -eu` fence, the `import.meta.url`/`process.
|
||||
execPath` path resolution, and the **+6→111 gate arithmetic** all verified correct; the C grade is largely the
|
||||
legacy-manifest-format penalty — hand-authored slice docs, not trekexecute manifests). Resolution, each verified
|
||||
against live code:
|
||||
|
||||
- **[MAJOR — plan-critic #1] `brief --json` is pretty-printed** (`cli.ts:323` `JSON.stringify(…, null, 2)`), so the
|
||||
wrapper's "one log line" / SC7 contract was false. ✅ Step 3 captures `CODE=$?` first, then **compacts** `OUT`
|
||||
(`tr '\n' ' ' | tr -s ' '`); SC7 asserts exactly one line.
|
||||
- **[MAJOR — brief-reviewer #1] the wrapper never `cd`s to its package dir** → cron (CWD `$HOME`) would fail to
|
||||
resolve `tsx` (the plist sets `WorkingDirectory`, cron does not). ✅ Step 3 adds **`cd "$DIR"`**; SC7 gains a
|
||||
CWD-independence case.
|
||||
- **[MAJOR/MAJOR/MINOR — all three] the `logPath` expression** self-contradicted (`".."` escaping `trends/`) and
|
||||
used the `--store` override base. ✅ Step 4 pins **`join(dirname(defaultStorePath()), "cron.log")`** (data-root
|
||||
anchor, matches the wrapper); the `".."` and `storePath` variants removed; SC8 retargeted.
|
||||
- **[MAJOR — brief-reviewer #4 / plan-critic #3] `ScheduleSpec.env` stated three ways.** ✅ `env:
|
||||
Record<string,string>` is **canonical** (Step 2 interface + brief §3 + Files table + Step-1 fixtures), always
|
||||
`{ NODE_BIN, LINKEDIN_STUDIO_DATA(resolved) }`; the emitters render it only (purity holds).
|
||||
- **[MAJOR — all three] `ASSERT_BASELINE_FLOOR` line-cite `:1259` → `:1329`** (verified live; `:1259` is
|
||||
`LIFECYCLE_FILTER_LIT`; value 105→111 correct). ✅ Step 6 + Files table corrected; `TRENDS_TESTS_FLOOR` `:705` →
|
||||
`:709`; header-enum insertion `:49-53` → `:57`; the stale Section-18 "= 99" narration gets the R3b(→105)+R3c(→111)
|
||||
append.
|
||||
- **[MINOR — plan-critic #5] `dirname` not imported in `cli.ts`** (`:41` = `join` only). ✅ Step 4 adds `dirname`
|
||||
to the `node:path` import (+ `homedir`/`fileURLToPath`).
|
||||
- **[MINOR — plan-critic #8 / brief-reviewer #5] stray double `brief`** (`cli.ts brief brief …`). ✅ the baked
|
||||
`args` drop the leading `"brief"` (Step 4); the wrapper owns the subcommand; the 16l sentinel matches `cli.ts"
|
||||
brief` (Step 6 (5)).
|
||||
- **[MINOR — plan-critic #7] wrapper-absent RED is invocation-dependent.** ✅ Step 1 pins the test through
|
||||
**`bash run-daily.sh`** so the absent file is exit 127 (clean assertion-RED), not a thrown ENOENT.
|
||||
- **[MINOR — plan-critic #9 / brief-reviewer #9] `$HOME`-unset `set -u` edge.** ✅ the CLI **always** bakes a
|
||||
resolved-absolute `LINKEDIN_STUDIO_DATA` (Step 4 env), so a scheduled run never evaluates `$HOME`.
|
||||
- **[MINOR — scope-guardian #4] pathguard justification fabricated.** ✅ restated (brief §5): `~/repos/*` is
|
||||
allowlisted; the conclusion (new files write-allowed) holds.
|
||||
- **[LOW — brief-reviewer #7] SC1 "lint-valid" only grep-checked.** ✅ SC1 asserts **well-formed** (balanced-tag/
|
||||
parse) + key-complete; `plutil -lint` stays the deps-present manual check (Step 7).
|
||||
- **[LOW — brief-reviewer #8] twin census undercounted (3 → 4).** ✅ `analytics/storage.ts` named as the existing
|
||||
third; `run-daily.sh` the fourth (brief §5 + the wrapper comment).
|
||||
- **[MINOR — scope-guardian/plan-critic] de-niche safety (Section 17) confirmed:** `NICHE_TOKENS=Microsoft|Azure|
|
||||
Copilot|public sector|offentlig sektor`, scoped to an allowlist (`trend-spotter.md`/`content-planner.md`/
|
||||
`content-framework.md`); `schedule.ts`/`run-daily.sh` are not scanned, the agent prose carries no token, the label
|
||||
`com.linkedin-studio.trends.daily` is clean. R9 holds (no change needed).
|
||||
|
||||
**[plan-critic headless-readiness]** — R3c **builds** a headless entry, but the slice itself is authored and landed
|
||||
**in-session, operator-driven** (driftsmodell), not as a headless autonomous run, so per-step revert/halt clauses
|
||||
aren't needed (R1/R2a/R2b/R3a/R3b had none either). The *artifact* it ships (`run-daily.sh`) is the headless entry;
|
||||
the *development* is in-session.
|
||||
335
docs/research-engine/plan-re-r3d.md
Normal file
335
docs/research-engine/plan-re-r3d.md
Normal file
|
|
@ -0,0 +1,335 @@
|
|||
# Plan — RE-R3d: temporal overlay — first-mover + saturation (R3 slice b)
|
||||
|
||||
> **Brief:** `docs/research-engine/brief-re-r3d.md`. **Slice:** RE-R3d (research-engine rung-2 — R3 slice **(b)**:
|
||||
> the live temporal overlay). Closes the rest of hull #3 — **first-mover** + **saturation** (relevance ✅ R3a;
|
||||
> status ✅ R3b; `angle` already a scored dimension). **Zero new files** — pure EDITs.
|
||||
> **TDD-order (two-phase RED — light-Voyage discipline, inherited):** Step 1 records RED in two phases — **(A)**
|
||||
> `temporalSignal` is a NEW named export of the EXISTING `brief.ts`; Node16 ESM throws a missing named import at
|
||||
> module-load (every `brief.test.ts` test would error, not assert), so land a **non-throwing stub** first
|
||||
> (`temporalSignal → {tier:"fresh",…}` + the type exports + `BriefEntry.temporal` populated by the stub in
|
||||
> `rankForBrief`), then record value-assertion RED against it (the constant "fresh" stub fails the
|
||||
> first-mover/saturated/ordering/token assertions); **(B)** the CLI flag tests are value-RED against the existing
|
||||
> `brief` handler (the new flags are silently ignored by `parseFlags` today → tuned-threshold behaviour unchanged).
|
||||
> Then GREEN: real `temporalSignal` + `RankOptions` knobs + `cmp` key → render tokens + summary marker + descriptor
|
||||
> → `cli.ts` flags → wire `trend-spotter.md` (prose) + SSOT note + README → gate floors + Section 16m →
|
||||
> behavioural → land.
|
||||
> **Counts recounted live at land, never pinned/guessed.**
|
||||
> **Architectural decisions (CONFIRMED, AskUserQuestion 2026-06-26):** SB1 derived-at-brief (no schema bump) · SB2
|
||||
> refine recency within the composite tier (composite stays PRIMARY). Go-gate D1–D10 baked to recommended defaults
|
||||
> (brief §8).
|
||||
> **Light-Voyage:** scope-guardian / brief-reviewer / plan-critic to run on these drafts; findings folded in
|
||||
> §Plan-critic — folded before the code commit.
|
||||
|
||||
## Goal
|
||||
|
||||
Make the morning brief read the **temporal axis** R3b logs but the ranking ignores. Two derived signals, computed
|
||||
**at brief time** from already-persisted fields (no new store field, no schema bump): **first-mover** (recent AND
|
||||
never surfaced on a prior day — "you'd be early") ranks a trend up; **saturation** (surfaced on `>= saturationAt`
|
||||
prior days — "you keep seeing this and not acting") ranks it down. The R3a relevance composite stays the PRIMARY
|
||||
sort key (SB2); the overlay is a new `cmp` key **after** pillar-overlap, **before** `effectiveDate`, re-ordering
|
||||
only WITHIN a (composite, overlap) tier. It also promotes the R3b `· sett Nx` display hint into a graded badge set
|
||||
and surfaces a first-mover marker on the one-line summary. **No schema change** (`SCHEMA_VERSION` 4 /
|
||||
`BRIEF_SCHEMA_VERSION` 1 untouched); **no new agent/command/reference/module/file**; `score.ts`/`store.ts`/
|
||||
`types.ts`/`item.ts`/`schedule.ts`/`run-daily.sh` untouched.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | SC |
|
||||
|---|---|---|
|
||||
| `scripts/trends/src/brief.ts` | **EDIT** — add `TemporalTier`/`TemporalSignal` exports + pure `temporalSignal(ageDays, surfacedCount, {firstMoverDays, saturationAt})`; `RankOptions` gains `firstMoverDays?`(def 2)/`saturationAt?`(def 3); `BriefEntry` gains `temporal`; `rankForBrief` threads the opts + populates `temporal`; `cmp` gains `b.temporal.rank - a.temporal.rank` after overlap, before effectiveDate; `surfacedToken`→`temporalToken` (badges); `briefSummary` first-mover marker; `ranking:` descriptor names the new key | SC1–SC5, SC7, SC9 |
|
||||
| `scripts/trends/src/cli.ts` | **EDIT** — `brief … [--first-mover-days N] [--saturation-at N]` (mirror `--fresh-days` at `:312-317`, into `rankForBrief` opts at `:323`); usage line (`:110`) + header synopsis (`:14`) note the overlay; **`schedule` untouched** (nightly uses defaults) | SC6 |
|
||||
| `scripts/trends/tests/brief.test.ts` | **EDIT** — `temporalSignal` unit incl. clamp + future-date + neutral cases (SC1/SC2/SC9); **disagreement** ordering fixture (SC3); render badges (SC4) + **4 touched assertions**: `:407` `sett 3x`→`mettet (3x)`, `:408` unchanged (≥2 gate), `:325-331`+`:410-416` descriptor segment, `:96-102` re-based to a shared tier; summary marker (SC5); determinism (SC7); no-score-mutation (SC8) | SC1–SC5, SC7–SC9 |
|
||||
| `scripts/trends/tests/cli.test.ts` | **EDIT** — subprocess: `brief --first-mover-days N --saturation-at N` changes tiers vs defaults; absent → defaults 2/3; `--first-mover-days -1`/`x`, `--saturation-at 0`/`x` → exit 2 | SC6 |
|
||||
| `scripts/trends/src/types.ts` · `store.ts` · `score.ts` · `item.ts` · `schedule.ts` · `run-daily.sh` | **UNTOUCHED** — no data-shape/scoring/store/scheduler change. Listed to assert they are *not* in scope (`SCHEMA_VERSION` 4 held in `types.ts`; the R3c scheduler suite stays green untouched). | — |
|
||||
| `agents/trend-spotter.md` | **EDIT (prose-only, minimal)** — one line: the brief now applies a live temporal overlay (first-mover↑/saturated↓) at rank time, derived from dates + the seen-log; capture path unchanged. Domain-general. | — |
|
||||
| `references/trend-scoring-modes.md` | **EDIT (one-line note, "Consumers")** — the brief applies a within-composite-tier temporal overlay (RE-R3d); **no** weight/band/formula change. | — |
|
||||
| `scripts/trends/README.md` | **EDIT** — new `## Temporal overlay (RE-R3d)` between the R3c scheduler section (`:135-154`) and `## Tests` (`:155`): first-mover/saturation, derived-not-stored, the flags + defaults, the `cmp` integration, the badges | — |
|
||||
| `scripts/test-runner.sh` | **EDIT** — `TRENDS_TESTS_FLOOR` (`:713`, 192)→recount + breakdown comment; NEW unconditional **Section 16m** between 16l's `echo ""` (`:1374`) and Section 18 (`:1376`); `ASSERT_BASELINE_FLOOR` (`:1403`) 111→**117**; header-enum chain (`:57-62`) + Section-18 floor-history narration (`:1376-1402`, ends "= 111") | SC10 |
|
||||
| `docs/research-engine/{brief,plan}-re-r3d.md` | **NEW** — slice docs (TRACKED, like `docs/second-brain/*`) | — |
|
||||
| `STATE.md` | **EDIT at land** — Telling-block reconcile (trends floor, ASSERT floor 117, gate total; schema unchanged v4; **correct the stale :709/:1329 line-cites to live :713/:1403**). *Land bookkeeping, LOCAL-ONLY.* | — |
|
||||
|
||||
**Not touched (scope fence):** `types.ts`/`store.ts`/`score.ts`/`item.ts` (no data-shape/scoring change;
|
||||
`SCHEMA_VERSION` 4) · `schedule.ts`/`run-daily.sh` + their tests (R3c untouched — the nightly run uses default
|
||||
thresholds; no regression surface) · the SessionStart hook + its tests (R3d changes no frontmatter *field*; the
|
||||
summary's new marker is regex-safe; no hook change/test) · `config/*` · `commands/*` (29) · `agents/*` count (19
|
||||
— `trend-spotter.md` is a prose EDIT) · `references/*` count (27 — `trend-scoring-modes.md` is an EDIT) ·
|
||||
`algorithm-signals-reference.md` (cited for grounding, not edited) · `.gitignore` (no new artifact) ·
|
||||
`BRIEF_SCHEMA_VERSION` (1) · `SCHEMA_VERSION` (4).
|
||||
|
||||
## Step 1 — (RED, two phases) failing tests across brief/cli
|
||||
|
||||
**Phase A — stub-first, then value-assertion RED** (`brief.test.ts` imports the new `brief.ts` exports):
|
||||
- Land **non-throwing stubs** so the static imports resolve (Node16 ESM links named imports before any test runs):
|
||||
in `brief.ts` — add `export type TemporalTier`, `export interface TemporalSignal`, and `export function
|
||||
temporalSignal(): TemporalSignal { return { tier: "neutral", firstMover: false, surfacings: 0, rank: 2 }; }` (a
|
||||
constant stub ignoring its args); add `temporal: TemporalSignal` to `BriefEntry` and populate it in `rankForBrief`
|
||||
via the stub. The `cmp` key + `temporalToken` + the summary marker are **NOT** added yet (Steps 2–3).
|
||||
- `brief.test.ts` (value-RED against the stub):
|
||||
- **unit (SC1/SC2/SC9)** — `temporalSignal(1,0,{firstMoverDays:2,saturationAt:3})` expects
|
||||
`{tier:"first-mover",rank:3}` (stub returns "neutral" → RED); `(5,3,…)`→`saturated`; `(5,2,…)`→`warming`;
|
||||
`(5,0,…)`→`neutral`; `(3,0,…)`→`neutral` (past window); `(-1,0,…)`→`neutral` (future-date `>=0` guard); the
|
||||
clamp case `(5,5,{…,saturationAt:0})`→`saturated` via `at=1`.
|
||||
- **ordering (SC3) — DISAGREEMENT fixture** (folded — plan-critic M1: `surfacedCount` correlates with age, so a
|
||||
naive first-mover-vs-saturated fixture is *already* ordered by the existing `effectiveDate`-desc key → GREEN in
|
||||
Phase A, vacuous in GREEN). Three same-overlap entries: **A** `neutral` (surfaced 0, **older** date ≈5d,
|
||||
composite 7.0); **B** `warming` (surfaced 2, **newer** date ≈1d, composite 7.0); **Z** `saturated` (surfaced 4,
|
||||
composite **8.5**). Expect `topMatches.map(e=>e.trend.title) === [Z, A, B]`. Phase A (stub: constant rank, no
|
||||
temporal key) yields `[Z, B, A]` (`effectiveDate`-desc decides A vs B) → RED; GREEN (Step 2 inserts
|
||||
`temporal.rank`) flips A above the newer B, proving the key.
|
||||
- **render (SC4)** — `renderBrief(...)` expects `· 🥇 først ute` on the first-mover entry, `· 🔁 mettet (3x)` on a
|
||||
`surfacedCount 3` entry, `· sett 2x` on `surfacedCount 2`, **no badge** on `surfacedCount 1`, none on a
|
||||
`neutral` entry (stub render still emits the old `surfacedToken` → RED). **Four existing assertions touched**
|
||||
(folded — brief-reviewer MEDIUM-2/3): (1) `:407` `· sett 3x` → `· 🔁 mettet (3x)` (its surfacedCount-3 trend is
|
||||
now saturated); (2) `:408` `!md.includes("sett 1x")` **stays unchanged** (the ≥2 badge gate keeps
|
||||
surfacedCount-1 markerless); (3) `:325-331` + (4) the regex `:410-416` (the `ranking:` descriptor) gain `then
|
||||
temporal (first-mover↑/saturated↓), `; and the `effectiveDate`-isolation test `:96-102` is **re-based** so both
|
||||
entries share a `neutral` tier (else the new key, not `effectiveDate`, silently decides it — coverage erosion).
|
||||
- **summary (SC5)** — `briefSummary(...)` with a first-mover top expects `· 🥇 først ute` in the headline (stub →
|
||||
no marker → RED); assert the summary has no `"` and no `\n`.
|
||||
- **no-score-mutation (SC8)** — capture a store's `score.composite` values, run `rankForBrief` (pure — no
|
||||
mutation) and assert unchanged; assert `BRIEF_SCHEMA_VERSION === 1`, `SCHEMA_VERSION === 4`.
|
||||
|
||||
**Phase B — subprocess value-RED against the existing CLI handler** (no new import; the flags are parsed-but-unused
|
||||
today):
|
||||
- `cli.test.ts` — `brief --pillars ai --first-mover-days 1 --saturation-at 2 --store <seeded> --out <tmp>`:
|
||||
today `parseFlags` stores the flags but the `brief` handler ignores them → the rendered tiers match the defaults
|
||||
→ RED. **The tier badges live in the written `.md` body, not in `--json`** (which returns only
|
||||
`{path,date,totals,summary,marked}`, `cli.ts:336`), so the assertion **`readFileSync(path)`** and checks the
|
||||
badge (a `surfacedCount 2` trend renders `· 🔁 mettet (2x)` at `--saturation-at 2`, but `· sett 2x` at the
|
||||
default) — folded — brief-reviewer LOW-5; the first-mover marker is also observable via the `--json` `summary`.
|
||||
`--first-mover-days x` / `--saturation-at 0` today are ignored (exit 0) → RED against the expected `usage` exit 2.
|
||||
|
||||
**RED proof (record in commit, two phases):** Phase A — after the non-throwing `temporalSignal` stub lands,
|
||||
`(cd scripts/trends && npm test)` fails the unit/ordering/render/summary cases on **value** assertions against the
|
||||
constant-"fresh" stub (not module-not-found). Phase B — the `cli.test` flag cases fail on value/exit assertions
|
||||
against the flag-ignoring handler. The plan does **not** claim a single "everything fails before any code" run.
|
||||
|
||||
## Step 2 — (GREEN) `brief.ts` — the real signal + ranking integration
|
||||
|
||||
Replace the Phase-A stub with the real, pure implementation:
|
||||
- `export type TemporalTier = "first-mover" | "fresh" | "warming" | "saturated";`
|
||||
- `export interface TemporalSignal { tier: TemporalTier; firstMover: boolean; surfacings: number; rank: number; }`
|
||||
- `export function temporalSignal(ageDays, surfacedCount, opts): TemporalSignal`:
|
||||
```ts
|
||||
const surfacings = surfacedCount ?? 0;
|
||||
const at = Math.max(1, opts.saturationAt); // defensive clamp (plan-critic m3)
|
||||
const firstMover = ageDays >= 0 && ageDays <= opts.firstMoverDays && surfacings === 0; // >=0 guard (m5)
|
||||
const tier: TemporalTier = firstMover ? "first-mover"
|
||||
: surfacings >= at ? "saturated"
|
||||
: surfacings >= 1 ? "warming" : "neutral";
|
||||
const rank = tier === "first-mover" ? 3 : tier === "neutral" ? 2 : tier === "warming" ? 1 : 0;
|
||||
return { tier, firstMover, surfacings, rank };
|
||||
```
|
||||
Pure (no clock/fs/env). Branches disjoint + total (`firstMover` requires `surfacings === 0`; `saturated`/
|
||||
`warming` both require `surfacings >= 1`).
|
||||
- `RankOptions` (`:51-54`) gains `firstMoverDays?: number` + `saturationAt?: number`. `rankForBrief` (`:78`) reads
|
||||
`const firstMoverDays = opts.firstMoverDays ?? 2; const saturationAt = opts.saturationAt ?? 3;` and populates
|
||||
`BriefEntry.temporal = temporalSignal(ageDays, trend.surfacedCount, { firstMoverDays, saturationAt })` at the
|
||||
push site (`:93`).
|
||||
- `cmp` (`:102-107`): insert `b.temporal.rank - a.temporal.rank ||` **after** `b.overlap - a.overlap ||` and
|
||||
**before** `b.effectiveDate.localeCompare(a.effectiveDate)`. Composite stays the first key (SB2). The chain stays
|
||||
a total order. The bucketing (`isFresh`/`freshDays`, `:109-113`) is **unchanged**.
|
||||
Make the Phase-A unit + ordering cases green.
|
||||
|
||||
## Step 3 — (GREEN) `brief.ts` — render badges + summary marker + descriptor
|
||||
|
||||
- Replace `surfacedToken` (`:154-157`) with `temporalToken(e: BriefEntry): string`:
|
||||
```ts
|
||||
const t = e.temporal;
|
||||
if (t.tier === "first-mover") return " · 🥇 først ute";
|
||||
if (t.tier === "saturated") return ` · 🔁 mettet (${t.surfacings}x)`;
|
||||
if (t.tier === "warming" && t.surfacings >= 2) return ` · sett ${t.surfacings}x`; // preserves the R3b ≥2 badge contract
|
||||
return ""; // neutral, or warming with surfacings 1
|
||||
```
|
||||
Replace the `surfacedToken(e)` call in `renderTopEntry` (`:162`) and `renderBulletEntry` (`:171`) with
|
||||
`temporalToken(e)`. (`scoreToken` untouched.)
|
||||
- `briefSummary` (`:131-141`): compute `const fm = top.temporal.firstMover ? " · 🥇 først ute" : "";` and emit
|
||||
`(${pillar}${band}${fm} · ${top.ageDays}d)`. The marker carries no `"`/`\n` (hook-regex invariant, `:128-130`).
|
||||
- `renderBrief`'s `ranking:` descriptor (`:187`): insert `then temporal (first-mover↑/saturated↓), ` between
|
||||
`pillar-overlap desc, ` and `then publishedAt desc`.
|
||||
Make the Phase-A render + summary cases green.
|
||||
|
||||
## Step 4 — (GREEN) `cli.ts` — the two `brief` threshold flags
|
||||
|
||||
After the `--fresh-days` block (`:313-317`), add (mirroring its idiom):
|
||||
```ts
|
||||
let firstMoverDays = 2;
|
||||
if (flags["first-mover-days"] && flags["first-mover-days"] !== "true") {
|
||||
const n = Number.parseInt(flags["first-mover-days"], 10);
|
||||
if (Number.isNaN(n) || n < 0) usage("--first-mover-days must be a non-negative integer");
|
||||
firstMoverDays = n;
|
||||
}
|
||||
let saturationAt = 3;
|
||||
if (flags["saturation-at"] && flags["saturation-at"] !== "true") {
|
||||
const n = Number.parseInt(flags["saturation-at"], 10);
|
||||
if (Number.isNaN(n) || n < 1) usage("--saturation-at must be a positive integer");
|
||||
saturationAt = n;
|
||||
}
|
||||
```
|
||||
Pass into the rank call (`:323`): `rankForBrief(store, pillars, day, { freshDays, firstMoverDays, saturationAt })`.
|
||||
Update the `brief` usage line (`:110`) + header synopsis (`:14`) to list the two flags + a one-line note that the
|
||||
brief applies a derived temporal overlay (first-mover↑/saturated↓) at rank time. **No new exit code** (0/2). The
|
||||
`schedule` branch (`:343-417`) is **untouched** (nightly run uses defaults). Make the Phase-B `cli.test` cases green.
|
||||
|
||||
## Step 5 — wire `trend-spotter.md` (prose) + SSOT note + README
|
||||
|
||||
- `agents/trend-spotter.md` — one prose line (no batch-shape change): the morning brief now applies a **live
|
||||
temporal overlay** at rank time (first-mover ranked up, repeatedly-surfaced/saturated ranked down), **derived**
|
||||
from the publish/capture dates + the seen-log — no new capture step; the polling/capture path is unchanged.
|
||||
Domain-general (Section 17).
|
||||
- `references/trend-scoring-modes.md` — under "Consumers", add ONE line: the morning brief applies a brief-time
|
||||
temporal overlay (first-mover/saturation, RE-R3d) as a within-composite-tier ranking refinement; this does **not**
|
||||
change the capture-time weights above. (No weight/band/formula edit.)
|
||||
- `scripts/trends/README.md` — add `## Temporal overlay (RE-R3d)` between `:154` and `## Tests` (`:155`): the
|
||||
first-mover/saturation definitions (self-surfacing, not market-coverage), the **derived-not-stored** boundary
|
||||
(no schema bump), the `--first-mover-days`(2)/`--saturation-at`(3) flags, the `cmp` integration (composite stays
|
||||
primary), the badge set (`🥇 først ute` / `🔁 mettet (Nx)` / `sett Nx` / none).
|
||||
|
||||
## Step 6 — gate: floors + new unconditional Section 16m
|
||||
|
||||
In `scripts/test-runner.sh`:
|
||||
- Set `TRENDS_TESTS_FLOOR` (`:713`, currently **192**) to the **`tests N`** line reported by `(cd scripts/trends &&
|
||||
npm test)` after Steps 1–5 — recounted live, NOT additive-guessed. Stays inside the deps guard. **Append**
|
||||
`+ RE-R3d: brief +N, cli +N (temporal overlay)` to the inline breakdown comment.
|
||||
- Add **Section 16m** ("Trends Temporal Overlay", RE-R3d), mirroring Section 16l (unconditional, deps-absent-safe,
|
||||
pure `grep -qF`/self-test, no `tsx`, **all literals ASCII** — the badge emoji are NEVER grepped; the shell must
|
||||
stay ASCII-clean for bash 3.2 `set -u`). **Placement (verified live):** between Section 16l's trailing `echo ""`
|
||||
(`:1374`) and the Section 18 header (`:1376`) — anti-erosion stays last. Six **unconditional** checks, the
|
||||
self-test emitting **one** pass/fail like 16l:
|
||||
(1) a non-vacuity self-test (a probe carrying `temporalSignal` accepted, one without rejected);
|
||||
(2) `grep -qF 'export function temporalSignal' scripts/trends/src/brief.ts`;
|
||||
(3) `grep -qF 'b.temporal.rank' scripts/trends/src/brief.ts` (the cmp integration);
|
||||
(4) `grep -qF '"first-mover"' scripts/trends/src/brief.ts` (the tier literal);
|
||||
(5) `grep -qF 'first-mover-days' scripts/trends/src/cli.ts` (the flag);
|
||||
(6) `grep -qF 'saturation-at' scripts/trends/src/cli.ts` (the flag).
|
||||
- Bump `ASSERT_BASELINE_FLOOR` (**`:1403`**, currently **111**) → **exactly 117** (111 + the 6 new unconditional
|
||||
16m emitters; the self-test emits one pass/fail like 16l, so 117 is deterministic — "live recount" is the safety
|
||||
net, not a guess). Insert the 16m clause into the **header-enumeration prose chain (`:57-62`)** before "…the
|
||||
assertion-count anti-erosion floor (SC6) in Section 18", preserving sentence flow. **Append** the RE-R3d (→117)
|
||||
narration to the **Section-18 floor-history comment** (`:1376-1402`, which ends "= 111").
|
||||
- **NOT touched here:** the hook suite (no `HOOK_TESTS_FLOOR` in `test-runner.sh`; R3d adds no hook test). It must
|
||||
still pass untouched (`node --test hooks/scripts/__tests__/*.test.mjs`) as a regression sanity at land.
|
||||
|
||||
## Step 7 — behavioural verification
|
||||
|
||||
`(cd scripts/trends && npm install)` if needed, then run brief §7's five behavioural steps: seed a store with a
|
||||
fresh+unsurfaced, a warming (`surfacedCount` 2), and a saturated (`surfacedCount` 4) trend at the **same**
|
||||
composite; `brief --pillars … --out /tmp/r3d-mb --store /tmp/r3d.json` → the fresh+unsurfaced sorts first with
|
||||
`· 🥇 først ute`, the saturated last with `· 🔁 mettet (4x)`, the `ranking:` descriptor names the temporal key;
|
||||
re-run with `--first-mover-days 0 --saturation-at 2` → first-mover badge gone, `surfacedCount 2` escalates to
|
||||
`mettet (2x)`; `--first-mover-days x` → exit 2; confirm the seeded `score.composite` values are unchanged after the
|
||||
brief (only `surfacedCount`/`lastSurfacedAt` advance). Run full `bash scripts/test-runner.sh` → `FAIL=0`
|
||||
(`ASSERT_BASELINE_FLOOR` 117, trends ≥ new floor, Section 16m green, Section 17 de-niche green, counts 29/19/27);
|
||||
run `node --test hooks/scripts/__tests__/*.test.mjs` → still green (untouched regression); confirm `schedule.test`/
|
||||
`run-daily.test` still green (R3c untouched).
|
||||
|
||||
## Step 8 — land
|
||||
|
||||
Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor 117, gate total;
|
||||
schema unchanged v4; **correct the stale :709/:1329 cites to live :713/:1403**). Commit order (house style):
|
||||
**(1)** docs commit `docs/research-engine/{brief,plan}-re-r3d.md` (no suffix, tracked); **(2)** code commit —
|
||||
`brief.ts` + `cli.ts` + `brief.test.ts` + `cli.test.ts` + `agents/trend-spotter.md` +
|
||||
`references/trend-scoring-modes.md` + `scripts/trends/README.md` + `scripts/test-runner.sh` with `[skip-docs]`
|
||||
(D10: single code commit — the overlay is one coherent feature). Push freely (window lifted; gitleaks at commit;
|
||||
`origin` = PUBLIC `open/` — STATE/`*.local.*` never pushed). No version bump (additive; `v0.5.2` dev).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|---|---|---|---|
|
||||
| — | RED Phase A | `(cd scripts/trends && npm test)` after the `temporalSignal` stub | unit/ordering/render/summary cases fail on **value** assertions vs the constant-"fresh" stub (not module-not-found) |
|
||||
| — | RED Phase B | `npm test` (cli.test) before the flag impl | `--first-mover-days`/`--saturation-at` cases fail on value/exit vs the flag-ignoring handler |
|
||||
| SC1 | first-mover detection | `npm test` (brief.test) | `(≤firstMoverDays, surfaced 0)` → first-mover; `(>window,0)`/`(≤window,≥1)` → not |
|
||||
| SC2 | saturation grading | `npm test` (brief.test) | `surfacings ≥ saturationAt` → saturated; `1..at-1` → warming; `0` → fresh (inclusive `>=`) |
|
||||
| SC3 | ranking — within-tier re-order, composite dominates (DISAGREE fixture) | `npm test` (brief.test) | `[Z(8.5,sat), A(7.0,neutral,older), B(7.0,warming,newer)]`; temporal key flips A above newer B; Phase A `[Z,B,A]`→RED; total order |
|
||||
| SC4 | render badges + ≥2 boundary | `npm test` (brief.test) | `🥇 først ute` / `🔁 mettet (Nx)` / `sett Nx` (warming≥2) / none (warming 1, neutral); `:407` updated, `:408` stays green |
|
||||
| SC5 | summary first-mover marker | `npm test` (brief.test) | first-mover top → `· 🥇 først ute` in headline; else absent; no `"`/`\n` |
|
||||
| SC6 | CLI flags | `npm test` (cli.test) | flags change tiers vs defaults; defaults 2/3 when absent; bad values → exit 2 |
|
||||
| SC7 | determinism | `npm test` (brief.test) | same `(store,pillars,today,freshDays,firstMoverDays,saturationAt)` → byte-identical `.md` |
|
||||
| SC8 | no schema / no score mutation | `npm test` (brief.test) | `SCHEMA_VERSION` 4; `BRIEF_SCHEMA_VERSION` 1; `score.composite` unchanged after a brief |
|
||||
| SC9 | purity | `npm test` (brief.test) | `temporalSignal` stable over a grid; first-mover ⊆ recent∧unsurfaced; saturated ⇔ `surfacings≥saturationAt` |
|
||||
| SC10 | gate + wiring + de-niche | `bash scripts/test-runner.sh` | FAIL=0; trends ≥ floor; Section 16m green; `ASSERT_BASELINE_FLOOR`=117; Section 17; counts 29/19/27; hook suite green |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — changing `cmp` re-orders existing brief output → silently breaks downstream expectations.** *Mitigated:*
|
||||
composite stays the PRIMARY key (SB2) — the overlay only re-orders WITHIN a (composite, overlap) tier; the
|
||||
bucketing is unchanged; SC3 pins the exact order; SC7 pins byte-determinism; the hook reads only date+summary
|
||||
(unaffected).
|
||||
- **R2 — the `surfacedToken`→`temporalToken` promotion breaks pinned R3b/descriptor assertions.**
|
||||
*Mitigated/expected (folded — all three reviewers):* the warming badge is gated at `surfacings >= 2`, so
|
||||
`brief.test.ts:408` (`!md.includes("sett 1x")`) stays green and the R3b ≥2 contract is preserved exactly; Step 1
|
||||
enumerates the **four** touched assertions (`:407` `sett 3x`→`mettet (3x)`; `:325-331`+`:410-416` descriptor) and
|
||||
re-bases the coverage-eroded `:96-102`. `surfacedCount 2` still renders `· sett 2x`.
|
||||
- **R3 — emoji in the gate crashes bash 3.2 `set -u`.** *Mitigated:* Section 16m greps ONLY ASCII literals
|
||||
(`temporalSignal`, `b.temporal.rank`, `"first-mover"`, `first-mover-days`, `saturation-at`); the emoji live only
|
||||
in `brief.ts` source + rendered output, asserted by the TS tests, never by the shell gate.
|
||||
- **R4 — the summary marker breaks the SessionStart extractYaml regex.** *Mitigated:* `🥇 først ute` carries no
|
||||
`"` and no `\n`; SC5 asserts the invariant; the hook suite is a land-time regression check.
|
||||
- **R5 — saturation framing overclaims (reads as market-coverage).** *Accepted/honest:* R3d's saturation is
|
||||
**self-surfacing** (our seen-log), a proxy for a closing/ignored window — NOT external coverage (that is slice
|
||||
e, AI polling). The README + badge wording say "seen N×", not "covered online"; the brief §4 non-goal states the
|
||||
boundary. No salesmanship.
|
||||
- **R6 — first-mover default (2) too tight / saturationAt default (3) arbitrary.** *Mitigated:* both are
|
||||
CLI-tunable (`--first-mover-days`/`--saturation-at`), documented as deliberate defaults (like `freshDays` 7),
|
||||
grounded in the SSOT timing band (`<24-72h`) + the existing `sett Nx` `>=2` hint; D1/D2 are operator go-gate
|
||||
knobs.
|
||||
- **R7 — float/`-1` composite sentinel interaction with the new integer key.** *Mitigated:* the temporal key is a
|
||||
separate `||` term (a small integer diff); it never touches the `score?.composite ?? -1` term; the comparator
|
||||
stays a sum-free short-circuit chain (no NaN risk).
|
||||
- **R8 — editing `trend-spotter.md` / the SSOT trips the de-niche guard (Section 17).** *Mitigated:* the added
|
||||
prose + the SSOT note carry only generic overlay wording; pillars/topics stay config; Section 17 runs in the
|
||||
gate.
|
||||
- **R9 — gate checks must survive a deps-absent fresh clone.** *Mitigated:* Section 16m is pure `grep`/self-test on
|
||||
tracked source (`brief.ts` + `cli.ts`; no `tsx`) → unconditional; `TRENDS_TESTS_FLOOR` stays inside the deps
|
||||
guard.
|
||||
- **R10 — STATE's pinned floor line-cites (`:709`/`:1329`) are stale (live `:713`/`:1403`).** *Mitigated:* caught
|
||||
at brief time (the line numbers drifted when R3c added Section 16l + the floor-history narration); the plan cites
|
||||
live values; Step 8 corrects STATE.
|
||||
- **R11 — a temporal ordering test that passes WITHOUT the feature (false RED / vacuous GREEN).** *Mitigated
|
||||
(folded — plan-critic M1):* `surfacedCount` correlates with age, so a naive first-mover-vs-saturated fixture is
|
||||
already ordered by `effectiveDate`-desc. The SC3 fixture is built to force the temporal key and `effectiveDate`
|
||||
to **disagree** (older-`neutral` A vs newer-`warming` B at equal composite), so it is RED in Phase A and the new
|
||||
key is provably what decides in GREEN; the coverage-eroded `:96-102` is re-based to a shared tier.
|
||||
|
||||
## Plan-critic — folded
|
||||
|
||||
Three Opus reviewers ran on the brief + this plan, each verifying against live code; they **converged on the same
|
||||
two defects**. Verdicts: **scope-guardian MIXED** (0 hard creep; both confirmed decisions honored; every SC traces
|
||||
to a step; the floor/line cites verified live); **brief-reviewer PROCEED_WITH_RISKS** (all seven RED-premise/
|
||||
correctness claims HOLD; the gap was GREEN-completeness — 1 of 4 breaking assertions listed); **plan-critic
|
||||
APPROVE_WITH_NOTES 78/B** (floor arithmetic, line-cites, grep sentinels, cmp total-order, two-phase-RED structure
|
||||
all verified correct). Per-finding resolution (full headline list in `brief-re-r3d.md §9`):
|
||||
|
||||
- **[MAJOR — all three] warming badge fired at `>=1`, but live `surfacedToken` fires at `>=2` (`brief.ts:156`) +
|
||||
`brief.test.ts:408` pins `!sett 1x`.** ✅ Step 3 gates the warming badge at `surfacings >= 2` (R3b ≥2 contract
|
||||
preserved exactly; `:408` unchanged); the warming *tier* still demotes in `rank`. SC4 gains the surfacedCount-1
|
||||
no-badge boundary. The "preserves the hint" wording is corrected.
|
||||
- **[MAJOR — plan-critic M1 / brief-reviewer MEDIUM-3] the ordering test was not RED + vacuous** (`surfacedCount`
|
||||
correlates with age → `effectiveDate`-desc already orders first-mover-vs-saturated). ✅ Step 1 SC3 fixture forces
|
||||
temporal↔date **disagreement** (older-`neutral` A vs newer-`warming` B, equal composite; expect `[Z,A,B]`, Phase
|
||||
A `[Z,B,A]`→RED); `:96-102` re-based to a shared tier (R11).
|
||||
- **[MEDIUM — brief-reviewer MEDIUM-2] the `ranking:` descriptor change breaks `:325-331` + `:410-416`.** ✅ Step 1
|
||||
+ the Files table enumerate all four touched assertions, not one.
|
||||
- **[MINOR — plan-critic m3] `temporalSignal` undefensive vs `saturationAt < 1`.** ✅ Step 2 clamps `const at =
|
||||
Math.max(1, opts.saturationAt)` inside the pure function (the CLI guard alone is insufficient — the function is a
|
||||
public, gate-grepped export). SC2 clamp case added.
|
||||
- **[MINOR — plan-critic m4] the "fresh" tier was a misnomer** (collides with `freshDays`; a 30-day unsurfaced
|
||||
trend is not "fresh"). ✅ renamed **`neutral`** throughout (Steps 1–3, SCs); the gate sentinel greps `"first-mover"`
|
||||
(unaffected). The in-bucket effect (unsurfaced ranks above seen-and-skipped within `olderMatched`) documented in
|
||||
brief §3 as intended.
|
||||
- **[MINOR — plan-critic m5] future `publishedAt` (ageDays < 0) became a first-mover "act now" headline.** ✅ Step 2
|
||||
adds the `ageDays >= 0` guard (future → `neutral`). SC1 gains the `(-1,0)` case.
|
||||
- **[MINOR — plan-critic m6 / brief-reviewer] nightly run locked to default thresholds** (the primary saturation
|
||||
consumer). ✅ reframed in brief §4 as a known limitation, not a "later nicety."
|
||||
- **[LOW — brief-reviewer LOW-5] SC6 tier assertions can't read tiers from `--json`** (it omits the body badges).
|
||||
✅ Step 1 Phase B reads the written `.md` via `readFileSync(path)`; the first-mover marker is also in the `--json`
|
||||
`summary`.
|
||||
- **[LOW — all three] long-form `angle` cite `:34` → `:32`** (`:34` = `currency`; substance holds). ✅ brief §0/§2
|
||||
corrected.
|
||||
|
||||
**Verified correct (no change needed):** the floor arithmetic (16m = 1 self-test + 5 greps = 6 → `ASSERT_BASELINE_
|
||||
FLOOR` 111→117), all live cites (`:713`/`:1403`/`:1374`/`:1376`/`:57-62`), the six ASCII grep sentinels match the
|
||||
literals Steps 2–4 write, the `cmp` insertion preserves a total order with no NaN risk, and the two-phase RED
|
||||
structurally avoids a module-load ERROR. **This slice is authored + landed in-session (driftsmodell), not as a
|
||||
headless autonomous run, so per-step revert/halt clauses are not needed** (R1/R2/R3a/R3b had none either).
|
||||
401
docs/research-engine/plan-re-r3e.md
Normal file
401
docs/research-engine/plan-re-r3e.md
Normal file
|
|
@ -0,0 +1,401 @@
|
|||
# Plan — RE-R3e: brief history + day-over-day diff (R3 slice d)
|
||||
|
||||
> **Brief:** `docs/research-engine/brief-re-r3e.md`. **Slice:** RE-R3e (research-engine rung-2 — R3 slice **(d)**:
|
||||
> brief history + day-over-day diff). Closes hull **#7** (*"ingen brief-historikk"*) — each brief records the
|
||||
> trends it showed (`surfaced:` frontmatter) and renders **"Nytt siden sist"** against the most recent prior
|
||||
> brief. **Zero new files** — pure EDITs (the two tracked slice docs aside).
|
||||
> **TDD-order (two-phase RED — light-Voyage discipline, inherited):** Step 1 records RED in two phases — **(A)**
|
||||
> `diffSurfaced`/`parseSurfacedFrontmatter`/`selectPriorBriefFile` (+ `BriefDiff`) are NEW named exports of the
|
||||
> EXISTING `brief.ts`; Node16 ESM throws a missing named import at module-load (every `brief.test.ts` test would
|
||||
> error, not assert), so land **non-throwing stubs** first (constant returns; `renderBrief`/`briefSummary` gain an
|
||||
> optional `diff` param ignored by the stub render), then record value-assertion RED against them (the constant
|
||||
> stubs fail the diff/parse/select/section/marker assertions); **(B)** the CLI two-day diff test is value-RED
|
||||
> against the existing `brief` handler (no `surfaced:` write, no prior read, no `diff` in `--json` today).
|
||||
> Then GREEN: real `diffSurfaced`/`parseSurfacedFrontmatter`/`selectPriorBriefFile` → `surfaced:` frontmatter +
|
||||
> `BRIEF_SCHEMA_VERSION` 1→2 → `## 🆕 Nytt siden sist` section → summary marker → `cli.ts` prior-discovery +
|
||||
> `--json diff` → wire `trend-spotter.md` (prose) + README → gate floors + Section 16n → behavioural → land.
|
||||
> **Counts recounted live at land, never pinned/guessed.**
|
||||
> **Architectural decisions (CONFIRMED, AskUserQuestion 2026-06-26):** SD1 frontmatter `surfaced:` (no sidecar) ·
|
||||
> SD2 `added` w/ titles + `dropped` as a count (`brief.ts` store-free). Go-gate D1–D9 baked to recommended
|
||||
> defaults (brief §8).
|
||||
> **Light-Voyage:** scope-guardian / brief-reviewer / plan-critic to run on these drafts; findings folded in
|
||||
> §Plan-critic — folded before the code commit.
|
||||
|
||||
## Goal
|
||||
|
||||
Turn the dated morning brief from a **standalone daily snapshot** into a **history rung with a day-over-day diff**.
|
||||
Each brief persists the set of trend ids it showed into its YAML frontmatter (`surfaced: <id-csv>`), bumping
|
||||
`BRIEF_SCHEMA_VERSION` 1→2 (the store's `SCHEMA_VERSION` stays 4 — **no store field**). The next brief discovers
|
||||
the most recent **prior** dated file (strictly `< today`), parses its `surfaced:` line, and renders a
|
||||
**`## 🆕 Nytt siden sist`** section — `added` (in today, not prior — the headline, with titles resolved from the
|
||||
ranking), `carried`/`dropped` as a one-line count — plus a ` N nye siden sist.` marker on the one-line summary the
|
||||
SessionStart hook already surfaces. The diff is a **pure, render-time** layer: `rankForBrief` and the R3a
|
||||
composite / R3d temporal overlay are **unchanged**; `brief.ts` stays **store-free and fs-free** (the directory +
|
||||
file reads live at the `cli.ts` edge, injected like `today`/`pillars`). **No new agent/command/reference/module/
|
||||
file**; `types.ts`/`store.ts`/`score.ts`/`item.ts`/`schedule.ts`/`run-daily.sh` + the hook untouched.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | SC |
|
||||
|---|---|---|
|
||||
| `scripts/trends/src/brief.ts` | **EDIT** — `BRIEF_SCHEMA_VERSION` 1→2; add `BriefDiff` interface + pure `diffSurfaced(currentIds, priorIds, priorDate)` + `parseSurfacedFrontmatter(md)` + `selectPriorBriefFile(filenames, today)`; `renderBrief` gains optional `diff?` → emits `surfaced:` frontmatter line (before `schemaVersion:`) + the `## 🆕 Nytt siden sist` section (before Topp-treff); `briefSummary` gains optional `diff?` → ` N nye siden sist.` marker; `renderBrief` passes `diff` through to `briefSummary` | SC1–SC8 |
|
||||
| `scripts/trends/src/cli.ts` | **EDIT** — `brief` handler: `readdirSync(outDir)` (existsSync-guarded) → `selectPriorBriefFile` → `readFileSync` + `parseSurfacedFrontmatter` → `diffSurfaced(surfacedIds(ranking), priorIds, priorDate)` → `renderBrief(ranking, diff)`; thread `diff` into the shared summary at `:350` (`briefSummary(ranking, diff)` — one-source, MAJOR-2); `--json` gains `diff:{priorDate,added,carried,dropped}` (counts); console line appends `, N nye siden sist`; add `readdirSync` to the `node:fs` import | SC9 |
|
||||
| `scripts/trends/tests/brief.test.ts` | **EDIT** — `diffSurfaced` unit (SC1), `parseSurfacedFrontmatter` unit (SC2), `selectPriorBriefFile` unit (SC3), `surfaced:` frontmatter + round-trip (SC4), `## 🆕 Nytt siden sist` four branches (SC5), summary marker + the `briefSummary(r)===briefSummary(r,empty)` invariant (SC6), `BRIEF_SCHEMA_VERSION===2` (SC7), determinism-with-diff (SC8). **Stubs imported in Phase A.** No existing assertion breaks (verified §Step 1) | SC1–SC8 |
|
||||
| `scripts/trends/tests/cli.test.ts` | **EDIT** — subprocess two-day sequence: day-1 brief writes `surfaced:`; day-2 over a +1-trend store → `Nytt siden sist (<day1>)` lists the new trend, `--json diff.added≥1`, console `N nye siden sist`; first run → `diff.priorDate===null`; custom `--out` isolates discovery | SC9 |
|
||||
| `scripts/trends/src/types.ts` · `store.ts` · `score.ts` · `item.ts` · `schedule.ts` · `run-daily.sh` | **UNTOUCHED** — no data-shape/scoring/store/scheduler change (`SCHEMA_VERSION` 4 held in `types.ts`; the R3c scheduler suite + R3d ranking stay green untouched). Listed to assert they are *not* in scope. | — |
|
||||
| `hooks/scripts/session-start.mjs` + its tests | **UNTOUCHED** — the `surfaced:` frontmatter line is `^surfaced:`-keyed (the `^summary:`-anchored `extractYaml` cannot match it); the summary marker is `"`/`\n`-free. No hook edit/test; the hook suite is a land-time regression check. | — |
|
||||
| `agents/trend-spotter.md` | **EDIT (prose-only, minimal)** — one line: the brief now records its shown set + renders a day-over-day diff ("Nytt siden sist"); capture path unchanged. Domain-general. | — |
|
||||
| `scripts/trends/README.md` | **EDIT** — new `## Brief history + diff (RE-R3e)` between the R3d temporal-overlay section and `## Tests`: `surfaced:` record, `selectPriorBriefFile` strict-prior + same-day determinism, `diffSurfaced` partitions, section + summary marker, `BRIEF_SCHEMA_VERSION 1→2` (artifact-only) | — |
|
||||
| `scripts/test-runner.sh` | **EDIT** — `TRENDS_TESTS_FLOOR` (live `:716`, 216)→recount + breakdown comment; NEW unconditional **Section 16n** between 16m's `echo ""` and Section 18; `ASSERT_BASELINE_FLOOR` (live `:1473`) 117→**123**; header-enum chain (`:53-64`) + Section-18 floor-history narration (ends "= 117") | SC10 |
|
||||
| `docs/research-engine/{brief,plan}-re-r3e.md` | **NEW** — slice docs (TRACKED, like `docs/second-brain/*`) | — |
|
||||
| `STATE.md` | **EDIT at land** — Telling-block reconcile (trends floor, ASSERT floor 123, gate total; `BRIEF_SCHEMA_VERSION` 1→2; store schema unchanged v4; correct stale line-cites to live `:716`/`:1473`). *Land bookkeeping, LOCAL-ONLY.* | — |
|
||||
|
||||
**Not touched (scope fence):** `types.ts`/`store.ts`/`score.ts`/`item.ts` (no data-shape/scoring change;
|
||||
`SCHEMA_VERSION` 4) · `schedule.ts`/`run-daily.sh` + their tests (R3c untouched — the nightly run gets the diff
|
||||
internally; no scheduler edit) · the SessionStart hook + its tests (R3e adds no field the hook reads; `surfaced:`
|
||||
+ marker are regex-safe) · `references/trend-scoring-modes.md` (the diff is not a scoring concern) · `config/*` ·
|
||||
`commands/*` (29) · `agents/*` count (19 — `trend-spotter.md` is a prose EDIT) · `references/*` count (27) ·
|
||||
`.gitignore` (no new artifact — the brief files already live under the gitignored data dir). `SCHEMA_VERSION` (4).
|
||||
|
||||
## Step 1 — (RED, two phases) failing tests across brief/cli
|
||||
|
||||
**Phase A — stub-first, then value-assertion RED** (`brief.test.ts` imports the new `brief.ts` exports):
|
||||
- Land **non-throwing stubs** so the static imports resolve (Node16 ESM links named imports before any test runs):
|
||||
in `brief.ts` — add `export interface BriefDiff`, and
|
||||
- `export function diffSurfaced(): BriefDiff { return { priorDate: null, added: [], carried: [], dropped: [] }; }`
|
||||
- `export function parseSurfacedFrontmatter(): string[] { return []; }`
|
||||
- `export function selectPriorBriefFile(): string | null { return null; }`
|
||||
(each a constant stub ignoring its args). Add an **optional** `diff?: BriefDiff` param to `renderBrief` **and**
|
||||
`briefSummary`, **wired but inert** in the stub: `renderBrief` does NOT yet emit the `surfaced:` line or the
|
||||
section; `briefSummary` does NOT yet emit the marker. (Keeps the static signatures stable for the RED tests
|
||||
while the *behaviour* is still absent → value-RED, not type-RED.) `BRIEF_SCHEMA_VERSION` is still **1** in Phase
|
||||
A (so the SC7 `=== 2` assertion is RED).
|
||||
- `brief.test.ts` (value-RED against the stubs):
|
||||
- **diffSurfaced (SC1)** — `diffSurfaced(["a","b","c"],["b","c","d"],"2026-06-25")` expects
|
||||
`{priorDate:"2026-06-25",added:["a"],carried:["b","c"],dropped:["d"]}` (stub returns all-empty/null → RED);
|
||||
`diffSurfaced(["a","b"],[],null)` expects `{priorDate:null,added:["a","b"],carried:[],dropped:[]}`; a repeated
|
||||
id not double-counted.
|
||||
- **parseSurfacedFrontmatter (SC2)** — a full frontmatter string with `surfaced: 1a2b,3c4d,5e6f` → `["1a2b",
|
||||
"3c4d","5e6f"]` (stub `[]` → RED); blank `surfaced: ` → `[]`; absent line → `[]`; trims whitespace; does not
|
||||
match `summary:`/`store:`.
|
||||
- **selectPriorBriefFile (SC3)** — `(["2026-06-24.md","2026-06-25.md","2026-06-26.md","README.md","2026-06-30.md"],
|
||||
"2026-06-26")` → `"2026-06-25.md"` (stub `null` → RED); empty list / none `< today` → `null`.
|
||||
- **frontmatter `surfaced:` + round-trip (SC4)** — `renderBrief(r, diff)` includes `\nsurfaced: ` + the
|
||||
`surfacedIds(r).join(",")` value, before `\nschemaVersion: 2\n`; `parseSurfacedFrontmatter(renderBrief(r,d))
|
||||
=== surfacedIds(r)`; empty store → `surfaced: ` blank. Stub render emits neither the line nor `schemaVersion:
|
||||
2` → RED.
|
||||
- **section (SC5)** — `renderBrief(r, diff)` contains `## 🆕 Nytt siden sist`; the four branches (first-brief+
|
||||
added → `Første brief — alt nedenfor er nytt`; empty first → `Første brief.`; prior+added → the added title +
|
||||
`båret over` + `ikke vist i dag`; prior+no-added → `Ingenting nytt siden <date>`). Stub render omits the
|
||||
section → RED. **Assert the section precedes `## 🎯 Topp-treff`** (index check).
|
||||
- **summary marker (SC6)** — `briefSummary(r, {priorDate:"2026-06-25",added:["x"],carried:[],dropped:[]})` ends
|
||||
with ` 1 nye siden sist.` (stub omits → RED); `briefSummary(r, {priorDate:null,…})` and `briefSummary(r,
|
||||
{…,added:[]})` have no marker; **`briefSummary(r) === briefSummary(r, emptyDiff)`** (the invariant that keeps
|
||||
the existing `:166-171` test green); no `"`/`\n`.
|
||||
- **schema (SC7)** — `BRIEF_SCHEMA_VERSION === 2` (stub still 1 → RED); `SCHEMA_VERSION === 4`.
|
||||
- **determinism (SC8)** — `renderBrief(r, d) === renderBrief(r, d)`; with a fixed `d`, stable bytes.
|
||||
- **Existing assertions — one hard literal flips with the bump; the rest auto-track (verified live, MAJOR-1):**
|
||||
the frontmatter tests are `:158-160` (`startsWith "---\n"` — unaffected), `:161-164` (`schemaVersion: ` built
|
||||
from the **imported** `BRIEF_SCHEMA_VERSION` constant via RegExp `:163` — auto-tracks 1→2; `date:`/`store:`
|
||||
unaffected), `:166-173` (`summary: === briefSummary(r)` — preserved by the SC6 invariant: with no prior the
|
||||
marker is suppressed, so `briefSummary(r)===briefSummary(r,emptyDiff)`), the determinism pair `:182-184` (both
|
||||
sides default to the empty diff → still equal), and the `ranking:` descriptor tests (**descriptor unchanged by
|
||||
R3e** → unaffected). **The one break:** `:574` `assert.equal(BRIEF_SCHEMA_VERSION, 1)` — a **hard literal** in
|
||||
the `rankForBrief — no schema/score mutation` block (`:568-577`), **outside** the frontmatter set, which the
|
||||
GREEN bump must flip to `, 2)` in **Step 3** (`:575` `assert.equal(SCHEMA_VERSION, 4)` stays — store schema
|
||||
untouched). Swept: `:574` is the *only* hard `BRIEF_SCHEMA_VERSION` literal in the suite. The new section is
|
||||
**additive**, asserted only by new tests; **no test pins the intro→Topp-treff adjacency** (verified — all body
|
||||
assertions are substring/`match`).
|
||||
|
||||
**Phase B — subprocess value-RED against the existing CLI handler** (no new import; the handler ignores prior
|
||||
briefs today):
|
||||
- `cli.test.ts` — a **two-day** sequence sharing one `--out <tmp>` dir: (1) seed a store, run `brief --pillars
|
||||
ai,gov --out <tmp> --store <s> --json` → today the written `.md` has **no `surfaced:` line** and the `--json`
|
||||
has **no `diff` key** → RED against the day-1 assertions (`surfaced:` present, `--json.diff.priorDate === null`).
|
||||
(2) `capture` one new on-pillar trend, then **rename** the day-1 `.md` to a fixed past date (`mv` it to
|
||||
`2026-06-20.md` in the same `<tmp>` — the **rename-real-write** mechanism, M2) and run a second `brief` → against
|
||||
the *existing* handler it reads no prior, renders no `Nytt siden sist (<date>)` section, `--json` has no `diff` →
|
||||
RED. **Diff content lives in the written `.md` body**, so the assertion `readFileSync(path)` checks `## 🆕 Nytt
|
||||
siden sist`; the `diff` counts are read from `--json`.
|
||||
|
||||
**RED proof (record in commit, two phases):** Phase A — after the non-throwing stubs land, `(cd scripts/trends &&
|
||||
npm test)` fails the diff/parse/select/frontmatter/section/marker/schema cases on **value** assertions against the
|
||||
constant stubs (not module-not-found). Phase B — the `cli.test` two-day cases fail on the missing `surfaced:`/
|
||||
`diff`/section. The plan does **not** claim a single "everything fails before any code" run.
|
||||
|
||||
## Step 2 — (GREEN) `brief.ts` — the three pure helpers
|
||||
|
||||
Replace the Phase-A stubs with the real, pure implementations (all no clock/fs/env):
|
||||
- `export interface BriefDiff { priorDate: string | null; added: string[]; carried: string[]; dropped: string[]; }`
|
||||
- `diffSurfaced(currentIds, priorIds, priorDate)`:
|
||||
```ts
|
||||
const prior = new Set(priorIds);
|
||||
const cur = new Set(currentIds);
|
||||
return {
|
||||
priorDate,
|
||||
added: currentIds.filter((id) => !prior.has(id)),
|
||||
carried: currentIds.filter((id) => prior.has(id)),
|
||||
dropped: priorIds.filter((id) => !cur.has(id)),
|
||||
};
|
||||
```
|
||||
Order-stable (filters preserve input order); empty `priorIds` ⇒ `added===currentIds`, `dropped===[]`.
|
||||
- `parseSurfacedFrontmatter(md)`:
|
||||
```ts
|
||||
const m = md.match(/^surfaced: *([^\n]*)/m);
|
||||
if (!m) return [];
|
||||
return m[1].split(",").map((s) => s.trim()).filter((s) => s.length > 0);
|
||||
```
|
||||
Absent/blank/malformed → `[]` (mirrors the hook's `extractYaml` line-anchoring; never throws).
|
||||
- `selectPriorBriefFile(filenames, today)`:
|
||||
```ts
|
||||
const todayFile = `${today}.md`;
|
||||
return (
|
||||
filenames
|
||||
.filter((f) => /^\d{4}-\d{2}-\d{2}\.md$/.test(f) && f < todayFile)
|
||||
.sort()
|
||||
.pop() ?? null
|
||||
);
|
||||
```
|
||||
ISO dates sort lexicographically, so `f < todayFile` = date `< today` (strict — excludes today + future);
|
||||
greatest remaining = the most recent prior. Mirrors `session-start.mjs:63-66`, minus today.
|
||||
Make the Phase-A diff/parse/select cases green.
|
||||
|
||||
## Step 3 — (GREEN) `brief.ts` — frontmatter `surfaced:` + schema bump + the section + the marker
|
||||
|
||||
- **`BRIEF_SCHEMA_VERSION = 2`** (`brief.ts:23`) — **and flip the one hard test literal in the same step**
|
||||
(MAJOR-1): `tests/brief.test.ts:574` `assert.equal(BRIEF_SCHEMA_VERSION, 1)` → `assert.equal(BRIEF_SCHEMA_VERSION,
|
||||
2)`. (The RegExp at `:163` and the new SC7 already track the constant; `:575` `assert.equal(SCHEMA_VERSION, 4)`
|
||||
is untouched.)
|
||||
- **`renderBrief(ranking, diff: BriefDiff = { priorDate: null, added: [], carried: [], dropped: [] })`** — the
|
||||
default empty diff keeps single-arg call sites valid. Two additive emissions:
|
||||
- In the frontmatter block (`brief.ts:263-269`), insert **before** the `schemaVersion:` line:
|
||||
`lines.push(\`surfaced: ${surfacedIds(ranking).join(",")}\`);` (empty store ⇒ `surfaced: ` blank). The
|
||||
`schemaVersion:` line now renders `2` via the bumped constant.
|
||||
- After the intro line (`brief.ts:273-276`) and **before** `## 🎯 Topp-treff` (`:278`), emit the section:
|
||||
```ts
|
||||
lines.push(diff.priorDate !== null ? `## 🆕 Nytt siden sist (${diff.priorDate})` : "## 🆕 Nytt siden sist");
|
||||
if (diff.priorDate === null) {
|
||||
lines.push(diff.added.length > 0 ? "_Første brief — alt nedenfor er nytt._" : "_Første brief._", "");
|
||||
} else if (diff.added.length === 0) {
|
||||
lines.push(`_Ingenting nytt siden ${diff.priorDate}._`,
|
||||
`_${diff.carried.length} båret over, ${diff.dropped.length} ikke vist i dag._`, "");
|
||||
} else {
|
||||
const byId = new Map(
|
||||
[...ranking.topMatches, ...ranking.singleMatches, ...ranking.olderMatched].map((e) => [e.trend.id, e]),
|
||||
);
|
||||
for (const id of diff.added) {
|
||||
const e = byId.get(id);
|
||||
if (e) lines.push(renderBulletEntry(e));
|
||||
}
|
||||
lines.push(`_${diff.carried.length} båret over, ${diff.dropped.length} ikke vist i dag._`, "");
|
||||
}
|
||||
```
|
||||
(`renderBulletEntry` is the existing bullet renderer — reused, no new format.) An added id always resolves
|
||||
(added ⊆ surfacedIds ⊆ ranking entries); the `if (e)` guard keeps it total.
|
||||
- Pass the diff through: `briefSummary(ranking, diff)` at the frontmatter `summary:` line (`brief.ts:265`).
|
||||
- **`briefSummary(ranking, diff?: BriefDiff)`** (`brief.ts:204`): after building the headline, append the marker:
|
||||
```ts
|
||||
const delta = diff && diff.priorDate !== null && diff.added.length > 0
|
||||
? ` ${diff.added.length} nye siden sist.` : "";
|
||||
return `${...existing headline...}${delta}`;
|
||||
```
|
||||
Suppressed on the first brief / when nothing new; carries no `"`/`\n` (the existing summary already guarantees
|
||||
this — the marker adds only digits + ASCII words + a period). The no-diff call (`briefSummary(ranking)`) yields
|
||||
exactly the pre-R3e string (the SC6 invariant).
|
||||
Make the Phase-A frontmatter/section/marker/schema cases green.
|
||||
|
||||
## Step 4 — (GREEN) `cli.ts` — prior-brief discovery + the diff in `--json`
|
||||
|
||||
- Add `readdirSync` to the `node:fs` import (`cli.ts:51`).
|
||||
- In the `brief` handler, **between** the ranking (`cli.ts:339`) and the render (`:340`):
|
||||
```ts
|
||||
const todayIds = surfacedIds(ranking);
|
||||
let priorIds: string[] = [];
|
||||
let priorDate: string | null = null;
|
||||
try {
|
||||
if (existsSync(outDir)) {
|
||||
const priorFile = selectPriorBriefFile(readdirSync(outDir), day);
|
||||
if (priorFile) {
|
||||
priorIds = parseSurfacedFrontmatter(readFileSync(join(outDir, priorFile), "utf8"));
|
||||
priorDate = priorFile.slice(0, 10);
|
||||
}
|
||||
}
|
||||
} catch { priorIds = []; priorDate = null; } // unreadable prior ⇒ first-brief path
|
||||
const diff = diffSurfaced(todayIds, priorIds, priorDate);
|
||||
const md = renderBrief(ranking, diff);
|
||||
```
|
||||
- Import `diffSurfaced`, `parseSurfacedFrontmatter`, `selectPriorBriefFile` from `./brief.js` (`cli.ts:71`). (If
|
||||
any code annotates `: BriefDiff`, import it via **`import type`** — it is an interface, stripped from emitted JS,
|
||||
so a value-import fails at module-load (M1); the code above infers the type from `diffSurfaced`'s return, so no
|
||||
`BriefDiff` import is actually needed.)
|
||||
- **Thread the diff into the shared summary** (`cli.ts:350`, MAJOR-2): change `const summary = briefSummary(ranking)`
|
||||
→ `const summary = briefSummary(ranking, diff)`. The frontmatter `summary:` (built inside `renderBrief` →
|
||||
`briefSummary(ranking, diff)`) and the `--json` `summary` (read from this var) must stay **one source**
|
||||
(`cli.test.ts:268`); without this, day-2's file carries the ` N nye siden sist.` marker but `--json.summary`
|
||||
would not. **Safe on day-1:** `priorDate === null` ⇒ marker suppressed ⇒ byte-identical to the pre-R3e string.
|
||||
- `--json` (`cli.ts:352`): add `diff: { priorDate: diff.priorDate, added: diff.added.length, carried:
|
||||
diff.carried.length, dropped: diff.dropped.length }`.
|
||||
- The non-JSON console line (`cli.ts:355`): append `${diff.added.length > 0 && diff.priorDate !== null ? \`, ${diff.added.length} nye siden sist\` : ""}`.
|
||||
- **No new flag, no new exit code.** **Note (Phase-B test mechanism — rename-real-write, M2):** `today()` is
|
||||
wall-clock, so a same-process two-day sequence cannot advance the date. Rather than hand-author a `<prior>.md`
|
||||
fixture (which risks an id mismatch — its `surfaced:` ids would not be real store ids, so every trend reads as
|
||||
added/dropped, a weak/vacuous test), the cli.test **runs `brief` for real** (writing `${today}.md` with a genuine
|
||||
`surfaced:` line = `surfacedIds(ranking)`), **renames** it to a fixed past date (`mv ${out}/${today}.md
|
||||
${out}/2026-06-20.md`), then runs `brief` again in the same `--out`. The second run discovers `2026-06-20.md` as
|
||||
the strict-prior, parses its **real** ids, and diffs against today's cohort — proving discovery + parse + diff
|
||||
clock-free, with `carried`/`added` that are *exactly* right (id-matched). Capture one new on-pillar trend between
|
||||
the runs → that trend is the sole `added`.
|
||||
Make the Phase-B cli cases green.
|
||||
|
||||
## Step 5 — wire `trend-spotter.md` (prose) + README
|
||||
|
||||
- `agents/trend-spotter.md` — one prose line (no batch-shape change): the morning brief now **records the trends
|
||||
it showed** (frontmatter `surfaced:`) and renders a **day-over-day diff** ("Nytt siden sist") against the most
|
||||
recent prior brief — no new capture step; the polling/capture path is unchanged. Domain-general (Section 17).
|
||||
- `scripts/trends/README.md` — add `## Brief history + diff (RE-R3e)` between the R3d temporal-overlay section and
|
||||
`## Tests`: the `surfaced:` frontmatter record (one self-describing artifact, `BRIEF_SCHEMA_VERSION 1→2`,
|
||||
store `SCHEMA_VERSION` stays 4), `selectPriorBriefFile` strict-`< today` discovery (same-day re-run
|
||||
determinism), the `diffSurfaced` partitions (added/carried/dropped), the `## 🆕 Nytt siden sist` section
|
||||
(added with titles, carried/dropped as a count, "ikke vist i dag" framing) + the ` N nye siden sist.` summary
|
||||
marker the SessionStart hook surfaces for free.
|
||||
|
||||
## Step 6 — gate: floors + new unconditional Section 16n
|
||||
|
||||
In `scripts/test-runner.sh`:
|
||||
- Set `TRENDS_TESTS_FLOOR` (live **`:716`**, currently **216**) to the **`tests N`** line reported by `(cd
|
||||
scripts/trends && npm test)` after Steps 1–5 — recounted live, NOT additive-guessed. Stays inside the deps
|
||||
guard. **Append** `+ RE-R3e: brief +N, cli +N (brief history + diff)` to the inline breakdown comment.
|
||||
- Add **Section 16n** ("Trends Brief History / Diff", RE-R3e), mirroring Section 16m (unconditional,
|
||||
deps-absent-safe, pure `grep -qF`/self-test, no `tsx`, **all literals ASCII** — the `🆕` emoji is NEVER grepped;
|
||||
the shell stays ASCII-clean for bash 3.2 `set -u`). **Placement (verify live):** between Section 16m's trailing
|
||||
`echo ""` and the Section 18 header — anti-erosion stays last. Six **unconditional** checks, the self-test
|
||||
emitting **one** pass/fail like 16m:
|
||||
(1) a non-vacuity self-test (a probe carrying `diffSurfaced` accepted, one without rejected);
|
||||
(2) `grep -qF 'export function diffSurfaced' scripts/trends/src/brief.ts`;
|
||||
(3) `grep -qF 'parseSurfacedFrontmatter' scripts/trends/src/brief.ts`;
|
||||
(4) `grep -qF 'Nytt siden sist' scripts/trends/src/brief.ts` (the section header literal — ASCII portion only);
|
||||
(5) `grep -qF 'selectPriorBriefFile' scripts/trends/src/cli.ts` (the diff wiring in the CLI);
|
||||
(6) `grep -qF 'surfaced: ' scripts/trends/src/brief.ts` (the frontmatter emit).
|
||||
- Bump `ASSERT_BASELINE_FLOOR` (live **`:1473`**, currently **117**) → **exactly 123** (117 + the 6 new
|
||||
unconditional 16n emitters; the self-test emits one pass/fail like 16m, so 123 is deterministic — "live recount"
|
||||
is the safety net, not a guess). Insert the 16n clause into the **header-enumeration prose chain (`:53-64`)**
|
||||
before "…the assertion-count anti-erosion floor (SC6) in Section 18," preserving sentence flow. **Append** the
|
||||
RE-R3e (→123) narration to the **Section-18 floor-history comment** (which ends "= 117").
|
||||
- **NOT touched here:** the hook suite (no `HOOK_TESTS_FLOOR` in `test-runner.sh`; R3e adds no hook test). It must
|
||||
still pass untouched (`node --test hooks/scripts/__tests__/*.test.mjs`) as a regression sanity at land.
|
||||
|
||||
## Step 7 — behavioural verification
|
||||
|
||||
`(cd scripts/trends && npm install)` if needed, then run brief §7's five behavioural steps with a **unique tmp dir
|
||||
(no `rm`)**: `D=/tmp/r3e-mb-$$; S=/tmp/r3e-$$.json`. Seed an on-pillar store; `brief --pillars … --out "$D"
|
||||
--store "$S"` → frontmatter carries `surfaced: <ids>` + `schemaVersion: 2`, the section says `Første brief — alt
|
||||
nedenfor er nytt`; **rename that real brief to a fixed past date** (`mv "$D/$(ls "$D")" "$D/2026-06-20.md"`),
|
||||
`capture` a new on-pillar trend, and re-run `brief` → `## 🆕 Nytt siden sist (2026-06-20)` lists the added trend +
|
||||
`N båret over, M ikke vist i dag`, `--json diff.added ≥ 1` (the **rename-real-write** path, M2 — the prior's ids
|
||||
are real, so `carried`/`added` are id-exact); same-day re-run → `cmp` the two `${day}.md` byte-identical; confirm
|
||||
`score.composite` unchanged after the briefs (only `surfacedCount`/`lastSurfacedAt` advance); strip the `surfaced:`
|
||||
line from the renamed prior (a pre-R3e brief) and re-run → every trend reads as added (graceful degrade). Run
|
||||
full `bash scripts/test-runner.sh` → `FAIL=0` (`ASSERT_BASELINE_FLOOR` 123, trends ≥ new floor, Section 16n green,
|
||||
Section 17 de-niche green, counts 29/19/27); run `node --test hooks/scripts/__tests__/*.test.mjs` → still green
|
||||
(untouched regression); confirm `schedule.test`/`run-daily.test` still green (R3c untouched).
|
||||
|
||||
## Step 8 — land
|
||||
|
||||
Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor 123, gate total;
|
||||
`BRIEF_SCHEMA_VERSION` 1→2; store schema unchanged v4; **correct the stale `:713`/`:1403` cites to live
|
||||
`:716`/`:1473`**). Commit order (house style): **(1)** docs commit `docs/research-engine/{brief,plan}-re-r3e.md`
|
||||
(no suffix, tracked); **(2)** code commit — `brief.ts` + `cli.ts` + `brief.test.ts` + `cli.test.ts` +
|
||||
`agents/trend-spotter.md` + `scripts/trends/README.md` + `scripts/test-runner.sh` with `[skip-docs]` (D9: single
|
||||
code commit — the diff is one coherent feature). Push freely (window lifted; gitleaks at commit; `origin` =
|
||||
PUBLIC `open/` — STATE/`*.local.*` never pushed). No version bump (additive; `v0.5.2` dev) — note
|
||||
`BRIEF_SCHEMA_VERSION` 1→2 is the **artifact** schema, not the plugin version.
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|---|---|---|---|
|
||||
| — | RED Phase A | `(cd scripts/trends && npm test)` after the stubs | diff/parse/select/frontmatter/section/marker/schema cases fail on **value** assertions vs the constant stubs (not module-not-found) |
|
||||
| — | RED Phase B | `npm test` (cli.test) before the wiring | the two-day cases fail on the missing `surfaced:`/`diff`/section |
|
||||
| SC1 | diffSurfaced partitions | `npm test` (brief.test) | added/carried/dropped order-stable; empty prior ⇒ all added; repeated id once |
|
||||
| SC2 | parseSurfacedFrontmatter | `npm test` (brief.test) | csv → ids; blank/absent/malformed → `[]`; line-anchored (no `summary:` mismatch) |
|
||||
| SC3 | selectPriorBriefFile | `npm test` (brief.test) | greatest `< today`; excludes today + future; ignores non-dated; none → `null` |
|
||||
| SC4 | `surfaced:` + round-trip | `npm test` (brief.test) | one `surfaced: <csv>` line = `surfacedIds(r).join(",")`, before `schemaVersion: 2`; round-trips; empty store → blank |
|
||||
| SC5 | Nytt siden sist (4 branches) | `npm test` (brief.test) | `## 🆕 Nytt siden sist`; first-brief/empty/added/no-added branches; section precedes Topp-treff |
|
||||
| SC6 | summary delta marker | `npm test` (brief.test) | ` N nye siden sist.` when prior+added; absent on first/no-added; `briefSummary(r)===briefSummary(r,empty)`; no `"`/`\n` |
|
||||
| SC7 | schema boundary | `npm test` (brief.test) | `BRIEF_SCHEMA_VERSION` 2; `SCHEMA_VERSION` 4; no `score.composite` mutation after a brief |
|
||||
| SC8 | determinism + same-day | `npm test` (brief.test) | same `(store,pillars,today,opts,diff)` → byte-identical; same-day re-run picks the same prior (strict `<`) |
|
||||
| SC9 | CLI diff wiring | `npm test` (cli.test) | two-day (**rename-real-write**: real day-1 brief renamed to `2026-06-20.md`): `Nytt siden sist (<day1>)` lists the new trend; `--json diff` counts; first run `priorDate null`; `--out` isolates |
|
||||
| SC10 | gate + wiring + de-niche | `bash scripts/test-runner.sh` | FAIL=0; trends ≥ floor; Section 16n green; `ASSERT_BASELINE_FLOOR`=123; Section 17; counts 29/19/27; hook suite green |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — adding `surfaced:` / bumping the schema breaks pinned tests.** *Mitigated (verified live):* the
|
||||
frontmatter `schemaVersion:` test uses the **imported** `BRIEF_SCHEMA_VERSION` constant (auto-tracks 1→2); the
|
||||
`summary:`-equality test is preserved by the SC6 invariant (`briefSummary(r)===briefSummary(r,empty)`);
|
||||
`startsWith "---\n"` and the `date:`/`ranking:` tests are unaffected. **The one hard break (MAJOR-1):** `:574`
|
||||
`assert.equal(BRIEF_SCHEMA_VERSION, 1)` — a literal **outside** the frontmatter set — which Step 3 flips to
|
||||
`, 2)` with the bump (`:575` `SCHEMA_VERSION === 4` stays). Step 1 enumerates every surviving assertion + this
|
||||
one break.
|
||||
- **R2 — inserting a section before Topp-treff breaks an ordering assertion.** *Mitigated (verified):* no existing
|
||||
test pins the intro→Topp-treff adjacency (all body assertions are substring/`match`); the new section is
|
||||
asserted only by new tests. SC5 pins the section-before-Topp index in the new suite.
|
||||
- **R3 — the `surfaced:` CSV could collide with a comma in an id.** *Mitigated:* real ids are 12-hex
|
||||
(`store.ts:69-72`) — comma-free; the join/split is unambiguous in production. The brief.test fixtures use
|
||||
comma-free ids for the round-trip. (Test `mkTrend` ids are `title|url`; the round-trip unit uses clean ids.)
|
||||
- **R4 — same-day re-run picks its own just-written file → self-diff (empty) → non-deterministic vs the first
|
||||
run.** *Mitigated:* `selectPriorBriefFile` filters `f < ${today}.md` (strict), so the same-day file is excluded
|
||||
and the re-run picks the same true-prior → byte-identical (SC8). This is the exact R3c SC7 guarantee, preserved.
|
||||
- **R5 — a malformed / hand-edited / pre-R3e prior brief crashes the diff.** *Mitigated:* `parseSurfacedFrontmatter`
|
||||
returns `[]` on absent/blank/malformed (never throws); the cli `try/catch` degrades any fs error to the
|
||||
empty-prior (first-brief) path. SC2 + behavioural step 5 cover it.
|
||||
- **R6 — the summary marker breaks the SessionStart `extractYaml` regex.** *Mitigated:* ` N nye siden sist.` is
|
||||
digits + ASCII words + a period — no `"`, no `\n`; SC6 asserts the invariant; the hook suite is a land-time
|
||||
regression check. The hook reads `date`+`summary` only; `surfaced:` is `^surfaced:`-keyed (the `^summary:`
|
||||
regex cannot match it).
|
||||
- **R7 — the `🆕` emoji in the gate crashes bash 3.2 `set -u`.** *Mitigated:* Section 16n greps ONLY ASCII
|
||||
literals (`export function diffSurfaced`, `parseSurfacedFrontmatter`, `Nytt siden sist`, `selectPriorBriefFile`,
|
||||
`surfaced: `); the emoji lives only in `brief.ts` source + rendered output, asserted by the TS tests.
|
||||
- **R8 — `BRIEF_SCHEMA_VERSION` bump misread as a store-schema/plugin-version change.** *Mitigated:* it is the
|
||||
**artifact** frontmatter version (`brief.ts:23`, distinct from the store's `SCHEMA_VERSION` — the comment says
|
||||
so); store `SCHEMA_VERSION` stays 4; no plugin version bump (additive). README + §4 state the boundary.
|
||||
- **R9 — the diff couples `brief.ts` to fs (directory read) → breaks the purity claim.** *Mitigated (SD2):* the
|
||||
three new helpers are pure (string/array in, value out); the `readdirSync`/`readFileSync` live in `cli.ts` (the
|
||||
edge), injected exactly like `today`/`pillars`. `brief.ts`'s "No fs" header claim holds.
|
||||
- **R10 — STATE's pinned floor line-cites (`:713`/`:1403`) are stale (live `:716`/`:1473`).** *Mitigated:* caught
|
||||
at brief time (the lines drifted when R3d added Section 16m + the floor-history narration); the plan cites live
|
||||
values; Step 8 corrects STATE.
|
||||
- **R11 — a diff test that passes WITHOUT the feature (vacuous GREEN).** *Mitigated:* SC1/SC2/SC3 are unit tests
|
||||
of pure functions whose stubs return constants (true value-RED in Phase A); SC9's day-2 asserts a **specific**
|
||||
added trend appears in `Nytt siden sist (<day1>)` AND the `--json diff.added` count — both absent in the
|
||||
flag-ignoring handler (Phase B RED). No fixture is ordered-by-accident.
|
||||
- **R12 — `--no-mark` desync: the artifact records `surfaced:` but the store seen-log is not written.** *Accepted/
|
||||
intended:* `surfaced:` is a property of the rendered brief (what it showed), `--no-mark` governs only the store
|
||||
mutation. The next diff reads the **artifact**, so it is correct regardless of `--no-mark`. SC4 (frontmatter)
|
||||
and the existing `--no-mark` test (`cli.test:392-398`) both hold.
|
||||
|
||||
## Plan-critic — folded
|
||||
|
||||
Three Opus reviewers (scope-guardian, brief-reviewer, plan-critic) ran COLD on the brief + this plan against live
|
||||
`scripts/trends/`. **Verdicts:** scope-guardian **MIXED** · brief-reviewer **PROCEED_WITH_RISKS** · plan-critic
|
||||
**REWORK (0.88)** — **converged on 2 MAJOR + 4 MINOR** (all re-verified against live code before folding; full
|
||||
rationale in `brief-re-r3e.md §9`).
|
||||
|
||||
- **MAJOR-1** — `tests/brief.test.ts:574` `assert.equal(BRIEF_SCHEMA_VERSION, 1)` is a **hard literal** (outside
|
||||
the frontmatter set §Step-1 enumerated; the `:163` RegExp auto-tracks). **Folded:** Step 3 flips it to `, 2)`
|
||||
with the constant bump; Step 1's enumeration + R1 now name it; `:575` (`SCHEMA_VERSION === 4`) stays.
|
||||
- **MAJOR-2** — `cli.ts:350` `const summary = briefSummary(ranking)` was left unthreaded → day-2 `--json.summary`
|
||||
would lose the marker the file's frontmatter carries, breaking the `cli.test.ts:268` "one source" invariant.
|
||||
**Folded:** Step 4 changes it to `briefSummary(ranking, diff)` (safe day-1; `priorDate===null` suppresses the
|
||||
marker).
|
||||
- **M1 (MINOR)** — `BriefDiff` is type-only → `import type` if referenced (Step 2/4); the Step-4 code infers it, so
|
||||
no import is actually needed.
|
||||
- **M2 (MINOR)** — SC9 uses **rename-real-write** (run `brief`, `mv ${day}.md → 2026-06-20.md`, re-run), not a
|
||||
hand-fixture → clock-free + id-exact prior. **Folded:** Step 4 note, Phase B, Step 7, SC9 row.
|
||||
- **M3 (MINOR)** — SC1 "repeated id not double-counted" reworded → **cross-partition disjointness** (filters
|
||||
preserve within-list dups; production ids are distinct). Brief §6 SC1 + Step 1.
|
||||
- **M4 (MINOR)** — brief §3 "empty `surfaced:`" contradiction reworded (the `surfaced:` line is diff-independent =
|
||||
`surfacedIds(ranking)`; the default empty diff only drives the `_Første brief._` section). Brief §3.
|
||||
|
||||
**Confirmed correct (untouched):** all line-cites, the floors (216 @ `:716`; 117 → 123 @ `:1473`), Section 16m as
|
||||
the last trends section, the 6 ASCII sentinels' non-vacuity, the regex/ISO-lex/hook-safety/`--json`-shape/same-day
|
||||
strict-`<` — verified by all three.
|
||||
102
docs/second-brain/architecture.md
Normal file
102
docs/second-brain/architecture.md
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
# Second Brain — Architecture Design
|
||||
|
||||
> **Status:** architecture **approved by operator 2026-06-23**. **SB-S0 (Foundation) + SB-S1 (Ingest + gold signal) + SB-S2 (Evolution loop) landed 2026-06-23** (`scripts/brain/`, 82 tests, gate-wired; ingest CLI + published-only invariant + operator-gated consolidation loop + session-start nudge). **SB-S3a–e landed 2026-06-24** (profile.md reader-wiring, supersede arm, cross-silo id-threading, operations.md ops centre, content-history retirement + read-side reconcile); only S4 (connector) remains design-phase.
|
||||
> **Boundary (confirmed 2026-06-23):** the **engine** (store schema · evolution loop · ingest seam) → **the plugin** (domain-general, shareable); the **user's data** (posts · articles · newsletters · plans · ideas) → the **per-user data dir** (`${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/`, survives reinstall); the **personal cockpit** (the operator's day-to-day operations centre) → **Maskinrommet** (a thin layer that reads/writes *through* the plugin's store, never a fork of the engine).
|
||||
> **Research inputs (three parallel threads, 2026-06-23):** `research/connector-egress.md` · `research/secondbrain-sota.md` · `research/silo-inventory.md`.
|
||||
|
||||
## The problem
|
||||
|
||||
A "second brain" stores everything about one creator — posts, articles, newsletters, data, plans, ideas — and compounds it into an ever-improving, user-aligned profile. It is **memory AND an operations centre**.
|
||||
|
||||
The three research threads reframed the task in a decisive way:
|
||||
|
||||
1. **There is nothing to migrate.** M0 already routed all **12 existing per-user silos** through one tested seam (`hooks/scripts/data-root.mjs` `getDataRoot()` + its TS twin `scripts/analytics/src/utils/storage.ts`), default `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/`. They all survive reinstall today. The task is **unify**, not move.
|
||||
2. **Not a vector/graph DB.** For *our* case the portable, auditable, drift-resistant 2026 standard is **plain-text + JSON + git** — because **Claude *is* the semantic-retrieval engine** (it reads files in-context). A heavyweight store adds infra, breaks portability, and invites lock-in for no gain.
|
||||
3. **Auto-ingest exists — for an EU/EEA member.** The DMA Member Data Portability API is self-serve in EU/EEA + Switzerland and gives automated pull of **content** (post text + articles), but **not** received-analytics. Analytics stays the existing manual CSV path. Scraping is a ToS breach with active enforcement → never baked in. So ingest is **manual-first as the contract**; any connector is an additive tributary.
|
||||
|
||||
The real gap (from the silo inventory): the 12 silos are **per-user but siloed and heterogeneous** — three storage idioms, two roots, no cross-references, provenance reinvented under different field names in each. A published post lands in three non-referencing places with no shared id. The question *"which raw material actually performs?"* (specific → post → measured analytics) is **unanswerable today** — and it is exactly what a second brain should answer.
|
||||
|
||||
## The shape: a thin Markdown hub over typed tributaries
|
||||
|
||||
```
|
||||
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/
|
||||
brain/
|
||||
index.md # MOC / hub — one-screen pointer to every tributary + freshness flags
|
||||
# (the "memory AND ops centre" front page)
|
||||
profile.md # SEMANTIC, two-layer: §static (stable) + §dynamic (evolving).
|
||||
# Each fact: value · first_seen · last_seen · evidence_count ·
|
||||
# provenance(human|published|ai-draft) · status(active|superseded)
|
||||
operations.md # PLANS / IDEAS + the "who I am now" anchor (frozen-past-self guard)
|
||||
journal/ # EPISODIC, append-only: YYYY-MM-session.md — raw, never edited;
|
||||
# the source the consolidation loop distils FROM
|
||||
ingest/
|
||||
inbox/ # NEW drop-zone: manual import OR (EU/EEA) connector output
|
||||
published/ # processed, provenance=published — the GOLD signal
|
||||
voice-samples/ # tributary (style) — stays its own store
|
||||
specifics-bank/ # tributary (raw material) — stays its own store
|
||||
trends/ # tributary (external signal) — stays its own store
|
||||
analytics/ # tributary (performance) — stays its own store
|
||||
```
|
||||
|
||||
### Tributaries vs hub — the explicit call
|
||||
|
||||
- **Stay separate (tributaries), feed the hub via distilled pointer-summaries:** voice-samples, specifics-bank, trends, analytics, ingest. Each has a *different dedupe key + lifecycle* (trends: title+url, fast decay · specifics-bank: human-only, near-permanent · analytics: numeric, immutable). Forcing one schema would destroy those distinctions — the silo inventory warns of this explicitly. The hub holds a distilled summary of each + a pointer.
|
||||
- **Fold INTO the hub:** the flat `profile/user-profile.md` → two-layer `brain/profile.md`. Scattered `audience-insights/` / `examples/` become dynamic-layer sections or tributary-summaries referenced from `index.md`. The **operations/plans centre is genuinely new**.
|
||||
|
||||
### Alternatives considered (and rejected)
|
||||
|
||||
- **Vector/graph DB (Pinecone/Neo4j/Zep-style):** rejected — adds infra, breaks plain-text portability + git audit/rollback, invites lock-in, and duplicates retrieval that Claude already does in-context. The four SOTA ideas that port cleanly to files (episodic/semantic split, provenance-weighting, evidence-threshold promotion, temporal-validity frontmatter) give most of the value; the vector/graph machinery does not port and is not needed.
|
||||
- **One unified schema across all silos:** rejected — destroys each silo's distinct dedupe key + lifecycle. Keep tributaries; add a thin hub + a cross-silo id.
|
||||
|
||||
## Invariants (the spine)
|
||||
|
||||
- **Provenance-weighted learning (the critical guard):** profile/voice learn from `provenance=published` (human-edited) **only**, never `ai-draft`. This is the model-collapse guard for our exact risk — a content engine that learns voice from what it itself drafted collapses toward its own priors. Generalises `voice-scrubber`'s existing rule ("gold standard = approved editions, not the English post corpus") to a system-wide invariant.
|
||||
- **One canonical entity id + one provenance shape**, threaded through the silos → makes the post ↔ specific ↔ trend ↔ analytics graph assemblable. Fixes the inventory's hardest problem.
|
||||
- **Episodic / semantic physically separated** (raw events in `journal/`, distilled facts in `profile.md`).
|
||||
- **Evidence-threshold promotion (anti-overfit):** the dynamic layer decays; a fact enters the **static** layer only on explicit confirmation OR ≥N independent observations. Single weird sessions never reshape identity.
|
||||
- **Temporal-validity + freshness nudge:** every fact carries `last_seen`; retrieval weights recent over old; the session-start nudge flags facts not refreshed in T days (extend the existing trend-staleness nudge brain-wide).
|
||||
- **Anti-sycophancy as a built-in default, not a toggle:** consolidation + content agents treat the profile as evidence to *test*, not flatter. (Personalization context measurably *increases* sycophancy, so counter-pressure it every time the profile is used.)
|
||||
- **Frozen-past-self guard:** `operations.md` holds a periodic user-authored "where I'm headed now" anchor that deprecates older inferences.
|
||||
- **Zero required curation (graveyard guard):** the loop compounds value with ~zero mandatory upkeep; curation is available but never required. git = free rollback.
|
||||
|
||||
## The evolution loop (the compounding mechanism)
|
||||
|
||||
Run on a cadence (reuse the existing session-start staleness-nudge surface) as a **sleep-time consolidation pass**, not inline cost:
|
||||
|
||||
1. **Capture (episodic, auto):** real-signal sessions append to `brain/journal/`. The ingest seam (`ingest/inbox/`) takes the user's actual published posts — manual import first; published output tagged `provenance=published`.
|
||||
2. **Consolidate (reflection):** an Opus agent reads new journal + published + tributary deltas, proposes profile updates as a **diff**, never a silent overwrite. Each candidate carries `evidence_count`, `provenance`, timestamps.
|
||||
3. **Promote with thresholds:** static-layer entry needs confirmation or ≥N observations; one-offs stay in the decaying dynamic layer.
|
||||
4. **Reconcile contradictions:** classify temporal-update (supersede, keep old `status: superseded` + date) · contradictory (keep both, timestamped) · condition-dependent (scope) · distractor (drop). Bias toward "keep both + surface the conflict" over silent overwrite.
|
||||
5. **Decay + freshness:** weight recent over old; flag stale facts at session-start.
|
||||
6. **Curate (optional, never required):** the diff is presentable; confirm/edit/delete/roll-back via git.
|
||||
|
||||
## Build sequence (full ambition, incremental — one slice per session)
|
||||
|
||||
| Slice | Content | Why this order |
|
||||
|-------|---------|----------------|
|
||||
| **SB-S0 — Foundation** ✅ *landed 2026-06-23* | `brain/` scaffold + two-layer `profile.md` (fold in `user-profile.md`) + `index.md` MOC + `operations.md` + `journal/`; entity-id + provenance shape as a small typed, tested module; `ingest/` dirs + manual-import contract. **No loop yet.** Shipped as `scripts/brain/` (TS, 34 tests, gate-wired BRAIN floor); fold = P1 labeled-scalars + P2 expertise (checkbox-prefs deferred, §8 of `plan-sb-s0.md`). | Smallest thing that stands up and is testable; locks the id/provenance spine everything hangs on |
|
||||
| **SB-S1 — Ingest + gold signal** ✅ *landed 2026-06-23* | Manual import → `ingest/published/` with `provenance=published` (CLI `brain ingest`); voice-trainer wired to learn from published-only, never `ai-draft`, gate-enforced. Shipped as `scripts/brain/src/ingest.ts` + `docs/second-brain/ingest-manual-import.md` (v0.5.1). **No profile.md mutation** (SB-S2). | The gold signal before the loop that consumes it |
|
||||
| **SB-S2 — Evolution loop** ✅ *landed 2026-06-23* | Operator-invoked, operator-gated consolidation: `brain consolidate` (`--gather`/`--propose`/`--apply --confirm`) → profile **diff** w/ evidence_count/provenance/timestamps; threshold-promotion (N=3); contradiction → keep-both with distinct ids (no supersede); decay-flag (90d); `consolidation-state.json` sidecar; zero-dep session-start consolidation-due nudge + scaffold-ensure. Shipped as `scripts/brain/src/consolidate.ts` + `docs/second-brain/consolidation-loop.md` (v0.5.2). **Operator decisions:** journal deferred · no new agent (session extracts) · motor-only (no reader until S3). | The compounding mechanism |
|
||||
| **SB-S3 — Cross-silo graph + ops centre** | Thread the id through tributaries (post↔specific↔trend↔analytics assemblable); flesh out `operations.md`; retire the dead `content-history.md` + triple-post reconciliation. **S3a ✅ first reader · S3b ✅ supersede · S3c ✅ cross-silo id-threading** (hub-side: the published record carries the `specifics`/`trends` ids it was built from + a pure analytics resolver — `scripts/brain/src/assemble.ts`, `brain assemble`; tributaries untouched). **S3d ✅ ops centre** (`operations.md` is a read tributary; `strategy-advisor` honours the dated "who I am now" anchor that *deprecates older inferences* — advisory/reader-side). **S3e ✅ hygiene + reconciliation** (dead `content-history.md` retired across its 8 plumbing surfaces; `brain reconcile` reconciles silo 1 `## Recent Posts` ↔ the silo 2↔3 graph read-side, surfacing the coverage gap — created posts never `brain ingest`-ed; auto-capture of specifics/trends = a flagged follow-up). **The second-brain arc is complete** (S4 EØS DMA-connector optional). | Finally answers "which raw material performs?" |
|
||||
| *(later / optional)* **SB-S4** | EU/EEA DMA portability API as an auto-tributary into `ingest/inbox/` | Additive; never a dependency |
|
||||
|
||||
## What's genuinely hard (honest flags)
|
||||
|
||||
1. **Voice fidelity to a private individual is limited even at SOTA** — the real win is grounded *content* (specifics-bank) over mimicked *style*. Manage expectations.
|
||||
2. **Detecting that a high-relevance fact went stale is UNSOLVED field-wide** (STALE benchmark: all tested models/frameworks fail). We timestamp + nudge; we cannot auto-detect "changed jobs." Keep a human in that loop.
|
||||
3. **Contradiction classification** (real change vs context-scoped vs noise) is emerging + error-prone → bias to keep-both-timestamped.
|
||||
4. **The connector** — no clean LinkedIn self-serve content API outside EU/EEA. Build the manual ingest seam as the contract; any connector is a tributary, never a dependency. Newsletter-edition coverage by the portability `ARTICLES` domain is **not fully verified** — check against a real export.
|
||||
5. **Sycophancy is structural** — app-level mitigations reduce, not eliminate.
|
||||
6. **Avoiding the graveyard is a product problem** — the loop must compound value with ~zero required curation or it dies in 6 months like 82% of second brains.
|
||||
|
||||
## Verification (SB-S0, when we build it)
|
||||
|
||||
- Set `LINKEDIN_STUDIO_DATA` to a temp dir, run init → assert `brain/{index,profile,operations}.md` + `journal/` + `ingest/{inbox,published}` exist via the `getDataRoot` seam.
|
||||
- `profile.md` parses two-layer (§static / §dynamic); `user-profile.md` fields fold in without loss (diff check).
|
||||
- entity-id module: deterministic id mint + provenance shape, unit-tested.
|
||||
- **No regression:** gate `scripts/test-runner.sh` 89/0/0 green; trends 24/24; specifics 28/28; contract 33/33.
|
||||
- **Key assumption to test early:** "Claude-as-retrieval-engine over plain files is sufficient (no vector DB)" — testable with a retrieval scenario once the brain holds content. Marked as assumption until proven.
|
||||
|
||||
## Bottom line
|
||||
|
||||
A thin two-layer Markdown hub (`brain/`: semantic `profile.md` + episodic `journal/` + ops `operations.md` + `index.md` MOC) over the existing typed tributaries, fed by a provenance-tagged ingest seam, maintained by a sleep-time consolidation loop with evidence-threshold promotion, temporal-validity reconciliation, and a built-in anti-sycophancy / anti-collapse stance. Every silo stays a tributary except the flat `user-profile.md`, which folds in.
|
||||
113
docs/second-brain/brief-sb-s1.md
Normal file
113
docs/second-brain/brief-sb-s1.md
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
# Task Brief — SB-S1 (Ingest + gold signal)
|
||||
|
||||
> **Status:** DRAFT for adversarial brief-review (light-Voyage step 2). Not yet operator-approved.
|
||||
> **Arc:** `architecture.md` (approved 2026-06-23). **Predecessor:** SB-S0 landed (`8c92719`) — `scripts/brain/` (types/id/profile/scaffold/cli, 34 tests, gate-wired). **This brief:** the second slice in detail; S2–S4 stay one-line in the arc.
|
||||
> **Inputs:** `architecture.md` §"evolution loop" steps 1–2, §"invariants" (provenance-weighted learning); SB-S0 code under `scripts/brain/`; the silo inventory (`docs/second-brain/research/silo-inventory.md`).
|
||||
> **Review delta (2026-06-23):** folded the brief-review (PROCEED_WITH_RISKS). Resolver standardised on `dataRoot` (M-resolver); D3 voice-samples↔ingest relationship pinned (M-D3); SC5 hardened to a gate-checkable lint (M-SC5); test-runner floor edits named as deliverables (m-floor); input path + D4 net-new note + Q3 sharpened (minors).
|
||||
|
||||
---
|
||||
|
||||
## 1. Objective
|
||||
|
||||
Capture the **gold signal** — the user's actual *published* posts — into `ingest/published/` tagged `provenance=published`, and **wire the voice/profile-learning surface to learn from that published signal only**, never from AI-drafted content. This is the one invariant the whole second-brain stands or falls on (the model-collapse guard): a content engine that learns its own voice from what it itself drafted collapses toward its own priors. SB-S1 lays the gold signal **before** the consolidation loop (SB-S2) that will consume it.
|
||||
|
||||
## 2. Background (why this slice, why now)
|
||||
|
||||
- **The invariant already has a single-agent precedent.** `voice-scrubber` enforces *"gold standard = approved Norwegian editions, never the English post corpus."* SB-S1 generalises that one agent's rule into a system-wide, data-backed invariant: learning consumes `provenance=published` only.
|
||||
- **The signal is genuinely new, not a re-count of existing silos.** A published post lands today in three non-referencing places — state `## Recent Posts` (hook + topic only), the dead `analytics/content-history.md`, and `analytics/posts/*.json` (metrics, from CSV). **None holds the full post text.** Voice learning needs the verbatim text; `ingest/published/` holding it is additive, not redundant. (Cross-silo *threading* of a shared id is SB-S3, explicitly out of scope here.)
|
||||
- **The provenance shape is already in place.** SB-S0 pinned `Provenance = human | published | ai-draft` and put a `provenance` field on every `ProfileFact`. SB-S1 is the first slice to actually *produce* `published` records and *gate* a learning consumer on them — it spends the spine S0 built.
|
||||
- **Manual-first is the contract.** Auto-ingest (EU/EEA DMA portability) is a later additive tributary (SB-S4); SB-S1's ingest is the manual path: the user supplies the published post text, the CLI lands it as a gold record.
|
||||
|
||||
## 3. Confirmed decisions (inherited from the approved arc — not for re-litigation)
|
||||
|
||||
1. **Boundary:** engine → plugin · user data → data dir · cockpit → Maskinrommet.
|
||||
2. **Plain-text/JSON + git, no vector/graph DB.** Claude is the retrieval engine.
|
||||
3. **Provenance-weighted learning** (`published` only, never `ai-draft`) is an invariant, not a toggle.
|
||||
4. **Tributaries keep their own schema/lifecycle** — SB-S1 does NOT reshape voice-samples, specifics, trends, or analytics stores.
|
||||
5. **No new seam function** — ingest paths resolve through the brain package's own inlined `dataRoot(sub)` resolver (`scripts/brain/src/dataRoot.ts`, the repo idiom — NOT the hooks-side `getDataRoot()` seam, a different function in a different package). So the twin-sync surface (`data-root.mjs ⇄ storage.ts`) stays untouched.
|
||||
|
||||
Review SHOULD challenge: the S1/S2 boundary (§5/§7), the store format (§5.1), feasibility, hidden coupling, testability, scope leak, missing risks. Review SHOULD NOT re-open the five decisions above unless internally contradictory or technically impossible.
|
||||
|
||||
## 4. Scope — where SB-S1 sits in the arc
|
||||
|
||||
| Slice | One-line scope | This brief |
|
||||
|-------|----------------|------------|
|
||||
| SB-S0 — Foundation | scaffold + profile fold + id/provenance spine | ✅ landed |
|
||||
| **SB-S1 — Ingest + gold signal** | manual import → `ingest/published/` `provenance=published`; wire the voice/profile-learning surface to published-only | **← this** |
|
||||
| SB-S2 — Evolution loop | sleep-time consolidation: journal+published+tributary deltas → profile **diff** (evidence/provenance/timestamps); threshold promotion; contradiction reconciliation; freshness nudge. **Owns session-start wiring of init.** | later |
|
||||
| SB-S3 — Cross-silo graph + ops centre | thread the id through tributaries (post↔specific↔trend↔analytics); flesh out `operations.md`; retire dead `content-history.md` | later |
|
||||
| SB-S4 *(optional)* | EU/EEA DMA portability as auto-tributary into `ingest/inbox/` | later |
|
||||
|
||||
## 5. SB-S1 in detail (the build target)
|
||||
|
||||
### 5.1 Deliverables
|
||||
|
||||
1. **A published-record data layer (TS, `scripts/brain/`).** A `PublishedRecord` shape and pure parse/serialize functions for a published post file. Reuse the SB-S0 idioms: `mintEntityId` for the canonical id, the constrained-header line-grammar idea (NO YAML dep), `normalizeProvenance` for the provenance token. Fields (proposed, open to the plan): `id` (sha256[:12] of normalized post text) · `provenance` (always `published`) · `published_date` (YYYY-MM-DD) · `captured_at` (YYYY-MM-DD) · `source` (`manual` | future connector) · the **verbatim post body**. Pure functions, deterministic, unit-tested.
|
||||
|
||||
2. **A `brain ingest` CLI** extending `scripts/brain/src/cli.ts` (same dispatch idiom). Proposed subcommands (final shape open to the plan):
|
||||
- `brain ingest <file>` — read a file containing one published post, mint id, write `ingest/published/<id>.md` with `provenance=published`. **Idempotent:** same text → same id → no duplicate (compare-then-skip, matching `initBrain`).
|
||||
- `brain ingest --scan-inbox` — process every file in `ingest/inbox/` into `ingest/published/`, then mark/clear the inbox source (drop-zone → processed, per the architecture's `inbox`→`published` model).
|
||||
- `brain published list` — inspect the gold corpus (id · `provenance` · date · first line). Surfacing `provenance` per record makes the invariant *observable* — the operator can eyeball that nothing `ai-draft` leaked into `published/`.
|
||||
|
||||
3. **Wire the voice/profile-learning surface to published-only (the invariant made enforceable).** Edit `agents/voice-trainer.md`'s "Gather" step. The pinned relationship to the existing corpus (resolves brief-review M-D3):
|
||||
- **ADD** `ingest/published/` (`provenance=published`) as the **primary gold source**, ranked above the legacy `voice-samples/` corpus.
|
||||
- **KEEP** `voice-samples/` as a human-curated tributary — **not reshaped, not redirected, not abandoned** (§7 honoured). By its existing contract its files are the user's own real writing (human-supplied via `/setup`/`/onboarding`); they are not `ai-draft`. voice-trainer keeps reading it but treats `ingest/published/` as the higher-trust signal.
|
||||
- **FORBID** learning voice/profile patterns from any content marked `provenance=ai-draft` — the explicit negative rule that generalises `voice-scrubber`'s *"gold standard = approved editions, never the draft corpus"* precedent to the system invariant.
|
||||
- **FENCE the auto-append trap:** the `voice-samples/` template hints at a (not-yet-built) Stop-hook that auto-appends posts after content sessions. S1 states explicitly that any such future auto-capture MUST tag provenance and admit `published` only — never blind-append generated drafts into a learning corpus. (Forward-looking guard, no S1 code; names the exact model-collapse trap so a later slice cannot reintroduce it.)
|
||||
- **voice-trainer stays Sonnet** (no model/frontmatter change → no structure-lint model-consistency break).
|
||||
|
||||
4. **Manual-import contract doc** — `docs/second-brain/ingest-manual-import.md`: the published-record file format, the CLI usage, and the published-only learning rule. (Design doc → `docs/`, NOT `references/`, so the reference-doc count guard does not fire.) **Net-new (verified):** SB-S0 shipped the `ingest/{inbox,published}` *dirs* only; no contract doc exists yet — the architecture's "SB-S0 … manual-import contract" wording referred to the dir shape, not a doc.
|
||||
|
||||
5. **Tests (TDD, failing-first)** — published-record parse/serialize round-trip (incl. a body with `]`/`|`/quotes/newlines), id determinism + dedup, CLI idempotency on re-ingest and `--scan-inbox`, source-absent / empty-inbox no-op. **Plus the SC5 published-only structure-lint** (a new gate check; see SC5). Gate stays green.
|
||||
|
||||
6. **Gate floor edits (named — Edit, not Write, on existing files → pathguard-safe):** bump `BRAIN_TESTS_FLOOR` in `scripts/test-runner.sh` to the new brain-suite count, and bump the assertion-count floor (`ASSERT_BASELINE_FLOOR` / the `TOTAL_CHECKS` anti-erosion floor) by the number of new `pass()` lines added (the BRAIN suite line is unchanged-count; the new SC5 structure-lint check adds one). Exact line numbers confirmed in the plan against the live `scripts/test-runner.sh`.
|
||||
|
||||
7. **Docs** — flip the `architecture.md` SB-S1 row to "landed"; this brief + the SB-S1 plan persisted under `docs/second-brain/`.
|
||||
|
||||
### 5.2 The S1/S2 boundary (the decision the review must pin)
|
||||
|
||||
- **SB-S1 ships:** the gold-signal **data layer** (records + CLI), and wires the **existing on-demand learning surface** (`voice-trainer`, invoked when the user runs a voice command) to the published-only rule. This is a data module + an agent-doc edit — no scheduler, no automatic profile mutation.
|
||||
- **SB-S2 owns:** the **automated** sleep-time consolidation loop (session-start cadence) that reads `published` + journal + tributary deltas and proposes `profile.md` **diffs** with evidence/provenance/timestamps + threshold promotion. **No profile.md is auto-mutated in S1.**
|
||||
|
||||
This boundary keeps S1 a clean, testable data+invariant slice and leaves all *automatic profile evolution* to S2. It is the #1 thing for the operator to confirm.
|
||||
|
||||
## 6. Success criteria (SB-S1 — testable)
|
||||
|
||||
- **SC1 (ingest, runtime):** with `LINKEDIN_STUDIO_DATA` set to a temp dir, `brain ingest <file>` creates `ingest/published/<id>.md` with `provenance=published` and the verbatim body, at the runtime data-path via the brain package's `dataRoot`. **`ingest` creates `ingest/published/` on demand** (mkdir-recursive) — it does NOT hard-depend on a prior `brain init`, so ingest-before-init works. Verified by a test asserting file existence + content with no prior init.
|
||||
- **SC2 (record round-trip):** a `PublishedRecord` serializes and parses back to an identical record, including a body containing `]`, `|`, quotes, and embedded newlines. Verified by a parse/serialize identity unit test.
|
||||
- **SC3 (id determinism + dedup):** the same post text mints the same id (and re-ingesting it is a no-op — no duplicate file); two different posts mint different ids. Verified by id + idempotency unit tests.
|
||||
- **SC4 (scan-inbox):** files dropped in `ingest/inbox/` are processed into `ingest/published/` by `--scan-inbox`; an empty inbox is a clean no-op; re-running does not re-create already-published records. Verified by a temp-dir test.
|
||||
- **SC5 (published-only invariant, gate-checkable):** a **new structure-lint check** in `scripts/test-runner.sh` (matching the Section 8–17 honesty-lint idiom: positive + negative assertion + a non-vacuity self-test) asserts that `agents/voice-trainer.md` (a) names `ingest/published/` in its gather/source instructions AND (b) carries the explicit negative string forbidding learning from `provenance=ai-draft`. The self-test proves the check is non-vacuous (it would FAIL on a fixture missing either string). The honest runtime limit (an agent instruction can still be ignored at runtime) is stated in the doc — but the *presence* of the wiring is gate-enforced, not checklist-trusted. (Resolves brief-review M-SC5: no "OR checklist" escape hatch.)
|
||||
- **SC6 (no regression / no new seam):** `scripts/test-runner.sh` green; floors hold (trends ≥24, specifics ≥28, contract ≥33, **brain ≥ its new floor**); hook tests pass; SB-S1 adds **no new seam function** so `data-root.test.mjs` twin-sync is untouched.
|
||||
|
||||
## 7. Non-goals / scope fence (SB-S1 does NOT)
|
||||
|
||||
- **No consolidation loop / no automatic profile mutation** (SB-S2). `brain ingest` writes only under `ingest/`, never to `brain/profile.md`.
|
||||
- **No session-start wiring** of init *or* ingest (SB-S2 owns scaffold wiring; ingest stays a manual CLI in S1).
|
||||
- **No cross-silo id threading** — ingest mints its own record id but does NOT weave it into trends/specifics/analytics/state (SB-S3).
|
||||
- **No reconciliation of the triple-recorded post** and **no retirement of `content-history.md`** (SB-S3).
|
||||
- **No connector / no LinkedIn API** (SB-S4). `source` allows a future connector value but no connector code ships.
|
||||
- **No tributary-schema edits** — voice-samples, specifics, trends, analytics stores keep their current shape. (`voice-trainer.md` is edited; the voice-samples *store format* is not.)
|
||||
- **No new seam function**, no state-file / two-roots reconciliation, no GUI/cockpit.
|
||||
|
||||
## 8. Constraints / risks
|
||||
|
||||
- **TDD iron law:** no production code without a failing test first.
|
||||
- **Code placement:** all new code is TS under `scripts/brain/` — **no new `.mjs` under `hooks/scripts/`**, so the `pre-write-pathguard.mjs` Category-6 guard does not fire (it matches `…/hooks/scripts/*.mjs` on Write only; SB-S1 writes none). `agents/voice-trainer.md` is an **existing** file → Edit, not Write → unaffected regardless.
|
||||
- **Public repo (`open/` Forgejo):** confirm before push; push only inside the window; STATE.md stays gitignored.
|
||||
- **Structure lint:** SB-S1 adds no `agents/`, `commands/`, `references/`, or `skills/` file (ingest is a CLI subcommand in an existing package; voice-trainer is edited; the contract doc lives in `docs/`). So no count guard fires absent a version bump. **Version bump is an open question** (§9): the agent behavioural change may warrant 0.5.0 → 0.5.1.
|
||||
- **Node v25 test quirk:** `node --test <dir>` fails — use `node --import tsx --test tests/*.test.ts`; run TS tools from their own dir; `npm install` in `scripts/brain` before the gate or the suite warn-skips.
|
||||
- **Risk — store-format ambiguity:** the published-record file format is the one data-shaping decision in S1 (mirrors SB-S0's fold risk). Mitigate by pinning the grammar + a round-trip test (SC2) before the CLI wires it.
|
||||
- **Risk — invariant enforceability:** "wire voice-trainer to published-only" is an agent-doc instruction, not executable code; an instruction can be ignored at runtime. Mitigate by making SC5 a *gate-checkable* lint where feasible (grep for the rule's presence), and by stating the limit honestly in the doc.
|
||||
- **Risk — S1/S2 scope creep:** the temptation is to start consuming `published/` into `profile.md` here. Fence held by §7 + SC (no test asserts profile mutation in S1).
|
||||
|
||||
## 9. Open questions for the operator (settle at "go")
|
||||
|
||||
1. **S1/S2 boundary (§5.2):** confirm S1 = data layer + voice-trainer invariant wiring, S2 = the automatic profile-diff loop. (Recommended.)
|
||||
2. **Profile vs voice in S1:** confirm that *profile* learning is invariant-only in S1 (provenance already on facts; no profile mutation) and that the only *active* consumer wired is `voice-trainer`. (Recommended.)
|
||||
3. **Store format (~decided, confirm):** published records as **file-per-post constrained-header markdown** (the `brain/profile.md` line-grammar idiom, no YAML) — matching the architecture's `inbox`→`published` per-file drop-zone. This deliberately does NOT reuse the specifics-bank single-JSON-index idiom (that store is a topic-tagged bank, not a per-file drop-zone). Confirm.
|
||||
4. **Version bump:** take 0.5.0 → 0.5.1, or no bump? SB-S0 set a no-bump precedent for count-neutral slices, but SB-S1 differs: it changes a *shipped agent's learning behaviour* (`voice-trainer`), a user-visible capability change — a stronger case for 0.5.1 than SB-S0 had. (Lean 0.5.1.)
|
||||
|
||||
## 10. Definition of done (SB-S1)
|
||||
|
||||
SC1–SC6 pass; deliverables 1–7 exist; non-goals untouched; the published-only invariant is wired into `voice-trainer` + documented + **gate-enforced (SC5 structure-lint)**; the BRAIN + assertion floors raised; committed (push per window, confirm first); `architecture.md` SB-S1 row → "landed"; version/CLAUDE.md/README touched only if a bump is taken.
|
||||
129
docs/second-brain/brief-sb-s2.md
Normal file
129
docs/second-brain/brief-sb-s2.md
Normal file
|
|
@ -0,0 +1,129 @@
|
|||
# Task Brief — SB-S2 (Evolution loop)
|
||||
|
||||
> **Status:** DRAFT for adversarial brief-review (light-Voyage step 2). Not yet operator-approved.
|
||||
> **Arc:** `architecture.md` (approved 2026-06-23). **Predecessors:** SB-S0 (scaffold + profile spine) + SB-S1 (ingest + published gold signal), both landed + released (v0.5.1).
|
||||
> **Operator scope decisions (2026-06-23, settled before this brief):** (1) **journal-capture DEFERRED** — S2 consumes `published` + tributary deltas only; (2) **deterministic CLI engine + the invoking session does the fuzzy extraction** (count-neutral, NO new agent); (3) **motor-only** — S2 evolves `brain/profile.md` but wiring a *reader* is S3 (accepted, flagged).
|
||||
> **Inputs:** `architecture.md` §"evolution loop" + §"invariants"; SB-S0/S1 code (`scripts/brain/`); the session-start surface (`hooks/scripts/session-start.mjs`).
|
||||
> **Review delta (2026-06-23):** folded the brief-review (REVISE). B1+B2 (the state-file write idiom is false AND the brain CLI can't reach the HOME-scoped state file) → resolved by storing consolidation state in a **brain data-root sidecar** (`brain/consolidation-state.json`), reachable by both `--apply` (dataRoot) and the session-start nudge (getDataRoot) — no state-file involvement. M3 `--propose` emits TWO artifacts (md + json). M4 **supersede dropped from S2** (keep-both only; supersede→S3). M5 SC1 split per-rule. Minors: SC3 split, candidate-file schema + gate-validation, freshness nudge = new-published-count only (no profile.md parse at session-start), consolidation matches `kind:'observed'` only (folded `profile-field` static seeds immutable in S2).
|
||||
|
||||
---
|
||||
|
||||
## 1. Objective
|
||||
|
||||
Build the **compounding mechanism**: an operator-invoked, sleep-time **consolidation pass** that reads the new published gold signal + tributary deltas, proposes a **diff** to the two-layer `brain/profile.md` (never a silent overwrite) — with evidence-threshold promotion, contradiction reconciliation (bias: keep-both-timestamped), provenance-gated learning (`published`/`human` only, never `ai-draft`), and temporal decay flagging (in the diff, not at session-start) — plus a **"consolidation-due" nudge** on the session-start surface (new-published count + days-since-last-run; per-fact stale nudging deferred), and the **session-start wiring of the scaffold init**. This is the loop that turns the gold signal (SB-S1) into an ever-improving, drift-resistant profile.
|
||||
|
||||
## 2. Background (why this shape, given the surfaces)
|
||||
|
||||
- **The loop cannot run AI at session-start.** `session-start.mjs` is zero-dep, must NOT spawn `tsx`, 10s timeout (the B-S3 rule). So the consolidation *pass* (which needs fuzzy fact-extraction) is **operator-invoked** ("sleep-time" = when the operator runs it), and session-start only emits a deterministic **nudge** ("consolidation due / N new published posts"). This is faithful to the architecture's "sleep-time consolidation, not inline cost."
|
||||
- **journal/ has no producer.** SB-S0 created the empty dir; nothing writes episodes. Per the operator decision, S2 consumes `published` (SB-S1, structured, provenance-tagged) + tributary deltas (specifics/trends/analytics — all have real timestamps); **journal-capture is its own later sub-slice.**
|
||||
- **The fuzzy extraction lives in the command/session layer, not a new agent** (the specifics-bank precedent: the deterministic store + CLI ship, the elicitation interview lives upstream). So S2 ships a deterministic **engine + CLI**; the invoking session reads the gathered deltas and produces candidate facts. Count-neutral (no `agents/` file → no reload, no count-lint bump).
|
||||
- **profile.md has no reader yet.** S2 evolves it into a (currently) unconsumed artifact; wiring content consumers is S3. Accepted: the value is that the profile compounds NOW so S3's reader inherits rich data. Flagged honestly (§9).
|
||||
- **The diff is operator-gated** — matches the longform `SendUserFile` + `[OPERATØR]` idiom (editorial-review): proposed, presentable, never silently applied; confirm/edit/roll-back via git. Anti-sycophancy: the profile is evidence to *test*, presented neutrally.
|
||||
|
||||
## 3. Confirmed decisions (inherited arc + operator scope — not for re-litigation)
|
||||
|
||||
1. **Boundary:** engine → plugin · user data → data dir · cockpit → Maskinrommet.
|
||||
2. **Invariants (architecture §"Invariants"):** provenance-weighted learning (`published`/`human` only, never `ai-draft`); episodic/semantic split; evidence-threshold promotion; contradiction → keep-both-timestamped; temporal-validity + freshness nudge; anti-sycophancy default; zero required curation.
|
||||
3. **Operator scope (§Status):** journal deferred · deterministic CLI + session extraction (no new agent) · motor-only (no reader until S3).
|
||||
4. **No new seam function** — brain code uses the package's own `dataRoot`; the session-start edit uses the existing `getDataRoot` (`data-root.mjs`).
|
||||
5. **No AI at session-start** (zero-dep nudge only); the consolidation pass is operator-invoked; **no auto-apply** of any diff.
|
||||
|
||||
Review SHOULD challenge: feasibility, the engine's classification rules (§5.1), the session-start edit's zero-dep safety, the diff/apply data shapes, hidden coupling, testability, scope leak, missing risks. Review SHOULD NOT re-open the §3 decisions unless internally contradictory or technically impossible.
|
||||
|
||||
## 4. Scope — where SB-S2 sits in the arc
|
||||
|
||||
| Slice | One-line scope | This brief |
|
||||
|-------|----------------|------------|
|
||||
| SB-S0 — Foundation | scaffold + profile fold + id/provenance spine | ✅ landed |
|
||||
| SB-S1 — Ingest + gold signal | manual import → `ingest/published/`; voice-trainer published-only | ✅ landed (v0.5.1) |
|
||||
| **SB-S2 — Evolution loop** | consolidation engine → profile **diff** (threshold/contradiction/decay, provenance-gated) + session-start freshness/consolidation-due nudge + scaffold-ensure wiring | **← this** |
|
||||
| SB-S3 — Cross-silo graph + ops centre | thread the id through tributaries; wire profile.md READERS; flesh out `operations.md`; retire dead `content-history.md` | later |
|
||||
| SB-S4 *(optional)* | EU/EEA DMA portability auto-tributary | later |
|
||||
|
||||
## 5. SB-S2 in detail (the build target)
|
||||
|
||||
### 5.1 Deliverables
|
||||
|
||||
1. **Deterministic consolidation engine** (`scripts/brain/src/consolidate.ts`) — PURE functions, no IO, heavily tested. Core: `proposeDiff({ current: ProfileDoc, candidates: Candidate[], today, opts }) → ProfileDiff`. A `Candidate` = `{ key, value, provenance, source, observed_date }`. **Matching:** every candidate is keyed `mintEntityId({kind:'observed', key})` and matched ONLY against existing `observed`-kind facts. The SB-S0 **folded `profile-field` static seeds are immutable in S2** — a candidate that semantically overlaps a folded field becomes its own `observed` fact (the keep-both philosophy; reconciling observed-vs-folded is operator-gated / S3). This avoids the `profile-field`↔`observed` id mismatch (fixes brief-review NICE). Classification rules:
|
||||
- **Reject** any `provenance: 'ai-draft'` candidate outright (model-collapse guard) — it never enters or bumps a fact.
|
||||
- **Add** (no matching `observed` fact, provenance ∈ {published, human}) → a new **dynamic** fact, `evidence_count: 1`, dates = `observed_date`/`today`.
|
||||
- **Evidence-bump** (matching fact, SAME value) → `evidence_count++`, `last_seen = today` (no other field touched).
|
||||
- **Promote** (a dynamic fact whose `evidence_count` reaches threshold `N`, default 3) → move dynamic→**static**.
|
||||
- **Conflict** (matching key, DIFFERENT value) → **keep-both-timestamped**: add the new value as its own fact, keep the old fact untouched (no evidence-bump of the old), and record the pair in `conflicts[]` — the architecture's explicit bias against silent overwrite. **No supersede in S2** (flipping `status:superseded` is deferred to S3; S2 never demotes/supersedes a fact).
|
||||
- **Decay-flag** — dynamic facts whose `last_seen` is older than `DECAY_DAYS` (default 90) are listed in `staleFlags[]` (information only; S2 never auto-demotes).
|
||||
`ProfileDiff` = `{ additions[], evidenceBumps[], promotions[], conflicts[], staleFlags[] }` (typed; NO `supersessions[]` — that arm is S3). The engine NEVER mutates `profile.md` — it returns proposals over a deep-frozen-safe input. `applyDiff(current, diff) → ProfileDoc` is a separate pure function (used only by the gated apply path); its output round-trips through `parseProfile`/`serializeProfile`.
|
||||
|
||||
2. **`brain consolidate` CLI** (extend `scripts/brain/src/cli.ts`) — the operator-invoked loop:
|
||||
- `brain consolidate --gather [--json]` — read-only: gather the deltas since the last run (new `published` records + each tributary's newest-timestamp summary + the current profile) and print them, for the **invoking session** to read and turn into candidate facts. (The fuzzy extraction lives here, in the session — no new agent.)
|
||||
- `brain consolidate --propose --candidates <file.json>` — **validate** the candidate file against the `Candidate` shape (malformed/missing field → non-zero exit, no profile touch — the only enforceable session↔engine contract), run `proposeDiff` over the current profile + the candidates, and write **TWO artifacts** to `brain/`: `pending-diff.md` (operator-readable, for the `SendUserFile` gate) AND `pending-diff.json` (the typed `ProfileDiff`, machine-applyable). Print both paths. Does NOT touch `profile.md`.
|
||||
- `brain consolidate --apply --diff brain/pending-diff.json --confirm` — apply the confirmed JSON diff to `profile.md` (the only path that writes the profile), then update the consolidation sidecar (D3). **Operator-gated** (requires `--confirm`); never auto-runs.
|
||||
|
||||
3. **Last-consolidation state — a brain data-root sidecar** `brain/consolidation-state.json` (`{ "last_run": "YYYY-MM-DD" }`), written by `--apply` and read by the session-start nudge. **This resolves brief-review B1+B2:** both the brain CLI (`dataRoot`) and the session-start hook (`getDataRoot`) resolve the data root, so the sidecar is reachable by both — whereas the plugin state file lives in HOME (`~/.claude/linkedin-studio.local.md`), a SIBLING of the data root the brain CLI cannot reach through its own resolver, and `state-updater.mjs` has no `last_import_date` writer to mimic (that scalar is written by the import *command* via LLM, not a function). Keeping consolidation state in the brain's own data-root territory is also conceptually right (brain state, not plugin state) and zero-dep JSON for the hook to read.
|
||||
|
||||
4. **Session-start wiring** (Edit `hooks/scripts/session-start.mjs` — existing file, zero-dep, pathguard-safe; all reads via the existing `getDataRoot`, the trend-nudge twin idiom):
|
||||
- **Scaffold-ensure (zero-dep):** mkdir the brain/ + ingest/ dirs if absent (idempotent, no tsx); if `brain/profile.md` is absent, emit a nudge to run `brain init` (seeding needs the tsx fold — kept manual, not inlined).
|
||||
- **Brain consolidation-due nudge (new-published-count only):** read `brain/consolidation-state.json` `last_run` (zero-dep JSON parse) + count `ingest/published/*.md` records (a `readdir` count — **no `profile.md` parse**, to respect the 10s budget + avoid a fragile zero-dep fact parser). Trend-nudge discipline: `daysSince(last_run)` + threshold + null-safe (a never-consolidated brain with no published records never nags). Emit e.g. "N published posts captured, last consolidation M days ago → run `brain consolidate`."
|
||||
- **Out of scope (honest):** **stale-FACT** nudging (parsing `profile.md` last_seen at session-start) is NOT shipped in S2 — it needs a zero-dep fact parser + cost, and bites harder once a reader exists. S2's brain-wide nudge = the new-published/consolidation-due signal only; per-fact decay is computed inside `proposeDiff` (`staleFlags[]`), surfaced in the diff, not at session-start.
|
||||
|
||||
5. **Operator-gated diff presentation** — the `--propose` artifact is a readable markdown diff (additions/bumps/promotions/conflicts/stale, sorted, neutral/anti-sycophantic framing); the invoking session `SendUserFile`s it and presents the `[OPERATØR]` gate before any `--apply` (the editorial-review idiom).
|
||||
|
||||
6. **Docs** — `docs/second-brain/consolidation-loop.md` (the loop contract: CLI usage, the engine rules, the operator-gate, the honest limits); this brief + the SB-S2 plan; flip the `architecture.md` SB-S2 row to landed.
|
||||
|
||||
7. **Tests + gate** — engine unit tests (every classification rule + idempotency + provenance-gating + threshold + keep-both + decay + `applyDiff` round-trip), CLI tests (gather/propose/apply, no-auto-apply, last-run recording), and a **session-start nudge test** in the hooks test suite (consolidation-due nudge fires/null-safe). Bump `BRAIN_TESTS_FLOOR` + the assertion floor (per the SB-S1-established empirical method) + the hook-test count if applicable.
|
||||
|
||||
### 5.2 What "owns the session-start wiring of the scaffold init" means here
|
||||
|
||||
S2 makes the brain **present + nudged** at session-start without running tsx: it zero-dep-ensures the dirs and nudges `brain init` (profile seeding stays the manual tsx CLI) + nudges `brain consolidate` when due. It does NOT run the consolidation pass automatically (that needs AI → operator-invoked).
|
||||
|
||||
## 6. Success criteria (SB-S2 — testable)
|
||||
|
||||
Engine rules each get their OWN criterion (fixes brief-review M5; N=3, DECAY_DAYS=90 pinned as the test constants — confirm §9 Q1 at "go" first):
|
||||
- **SC1a (add):** a candidate with no matching `observed` fact + provenance ∈ {published,human} populates `additions[]` as a dynamic fact, `evidence_count:1`.
|
||||
- **SC1b (reject ai-draft):** an `ai-draft` candidate yields an EMPTY diff — no add, no bump, no promote (the model-collapse guard, in code).
|
||||
- **SC1c (evidence-bump):** a candidate matching an existing fact with the SAME value populates `evidenceBumps[]` (`evidence_count++`, `last_seen=today`), no other field changed.
|
||||
- **SC1d (promote):** a dynamic fact reaching `evidence_count ≥ 3` populates `promotions[]` (dynamic→static).
|
||||
- **SC1e (conflict→keep-both):** a candidate matching a key with a DIFFERENT value populates `conflicts[]`, ADDS the new value as its own fact, and leaves the old fact untouched (no evidence-bump, no supersede) — both retained.
|
||||
- **SC1f (decay-flag):** a dynamic fact with `last_seen` older than 90 days is listed in `staleFlags[]`; no auto-demotion.
|
||||
- **SC1g (folded immutable):** a candidate overlapping a folded `profile-field` static seed creates a separate `observed` fact and never mutates the folded one.
|
||||
- **SC2 (immutability):** `proposeDiff` does not mutate its `current`/`candidates` inputs (asserted by structural equality of the inputs after the call).
|
||||
- **SC3 (apply round-trip):** `applyDiff(current, proposeDiff(...))` yields a `ProfileDoc` that round-trips exactly through `parseProfile`/`serializeProfile`.
|
||||
- **SC4 (idempotency):** re-running propose→apply with the same candidates over an already-applied profile yields no spurious additions (evidence-bumps/`last_seen` only; no duplicate facts).
|
||||
- **SC5 (CLI gating + validation):** `--gather` and `--propose` never write `profile.md` (asserted byte-unchanged after each); `--propose` writes `pending-diff.{md,json}` to `brain/` and REJECTS a malformed candidate file (non-zero exit, no profile touch); only `--apply --confirm` writes `profile.md` and updates `brain/consolidation-state.json`. Verified by temp-dir CLI tests.
|
||||
- **SC6 (session-start nudge, zero-dep):** the consolidation-due nudge fires when published records exist + `last_run` is stale/absent, and is silent when the brain is absent / nothing new; the edit adds no `tsx`/heavy dep and parses no `profile.md`. Verified by a hook-suite test (HOME + `LINKEDIN_STUDIO_DATA` set to separate temp dirs — the `session-start-trends-staleness.test.mjs` harness pattern — asserting the nudge string appears / is absent).
|
||||
- **SC7 (no regression / no new seam / count-neutral):** `scripts/test-runner.sh` green; floors hold (trends ≥24, specifics ≥28, contract ≥33, brain ≥ new floor); hook tests pass; **no new agent/command/ref/skill** (counts unchanged); no new seam function (twin-sync untouched); `compile-hooks.py --check` clean if `hooks.json` is regenerated.
|
||||
|
||||
## 7. Non-goals / scope fence (SB-S2 does NOT)
|
||||
|
||||
- **No journal-capture** (deferred sub-slice) — S2 reads `published` + tributary deltas only; nothing writes `brain/journal/`.
|
||||
- **No new agent** — the fuzzy extraction is done by the invoking session (no `agents/*.md`, no reload, no agent-count bump).
|
||||
- **No profile.md READER** — no content agent/command is wired to consume `brain/profile.md` (S3).
|
||||
- **No cross-silo id threading** into tributaries (S3); no `content-history.md` retirement / triple-post reconciliation (S3).
|
||||
- **No AI at session-start** — only a deterministic nudge; the consolidation pass is operator-invoked.
|
||||
- **No auto-apply** — no diff is ever applied without `--apply --confirm`; `profile.md` is mutated by exactly one gated path.
|
||||
- **No new hook `.mjs`** — only the existing `session-start.mjs` is edited (pathguard: Write of a new `hooks/scripts/*.mjs` is blocked).
|
||||
- **No new seam function**, no connector (S4), no GUI.
|
||||
|
||||
## 8. Constraints / risks
|
||||
|
||||
- **TDD iron law:** no production code without a failing test first.
|
||||
- **Code placement / pathguard:** engine + CLI are TS under `scripts/brain/` (safe). The session-start change is an **Edit** of the existing `hooks/scripts/session-start.mjs` (Edit is unaffected by the Category-6 Write guard) — **no new `.mjs` under `hooks/scripts/`**.
|
||||
- **Zero-dep hook discipline:** the session-start edit must NOT import tsx/the brain TS, must stay within the 10s timeout, and must use direct raw reads (the trend-nudge twin idiom). The brain consolidation is NOT run there.
|
||||
- **Hook compile:** if the SessionStart entry changes shape, regenerate `hooks.json` via `compile-hooks.py` (drift-guarded). Editing only the script body needs no recompile.
|
||||
- **Consolidation state is a brain sidecar, NOT the plugin state file** (fixes brief-review B1+B2): `brain/consolidation-state.json` under the data root, written by `--apply` (dataRoot) + read by the session-start hook (getDataRoot, zero-dep JSON). Do NOT route it through `~/.claude/linkedin-studio.local.md` — the brain CLI cannot reach HOME, and `state-updater.mjs` has no `last_import_date`-style writer to mimic (that scalar is an LLM/command write). Reading it at session-start is a `JSON.parse(readFileSync(...))`, the trend-`trends.json`-read twin.
|
||||
- **Public repo (`open/`):** confirm before push; window; STATE.md gitignored.
|
||||
- **Count-neutral:** no `agents/commands/references/skills` file added → no count guard fires (absent a version bump). **Version bump** likely 0.5.1 → 0.5.2 (new capability + session-start behavior) — confirm at "go".
|
||||
- **Risk — engine rule ambiguity:** the conflict-vs-supersede call is the fuzzy heart. Mitigation: bias to keep-both-timestamped (deterministic + safe), reserve supersede for an explicit confirm signal; pin each rule with a unit test before wiring the CLI.
|
||||
- **Risk — candidate extraction quality:** the *value* of the loop depends on the session extracting good candidates from the gathered deltas; the engine only guarantees the mechanics. Honest limit (the architecture's "voice fidelity is hard"): S2 ships the machinery, not a guarantee of insight.
|
||||
- **Risk — unconsumed output:** profile.md has no reader until S3; S2's value is deferred (compounding for later). Accepted by the operator; stated in `consolidation-loop.md`.
|
||||
- **Risk — session-start cost:** the nudge must stay cheap; counting `published/` files + one scalar read is O(files) — bound it (e.g. count via readdir, no parse) to respect the 10s budget.
|
||||
|
||||
## 9. Open questions for the operator (settle at "go")
|
||||
|
||||
1. **Promotion threshold N + decay window:** default `N = 3` independent observations to promote dynamic→static, `DECAY_DAYS = 90` for stale-flagging. Confirm or adjust. (Recommend N=3 / 90d — pinned as the test constants.)
|
||||
2. **Version bump:** 0.5.1 → 0.5.2 for the new consolidation capability + session-start nudge? (Lean 0.5.2, consistent with SB-S1.)
|
||||
|
||||
*Resolved by the brief-review (no longer open):* conflict = **keep-both only, no supersede in S2** (supersede→S3); diff artifacts = `brain/pending-diff.{md,json}` (git-trackable per "roll-back via git").
|
||||
|
||||
## 10. Definition of done (SB-S2)
|
||||
|
||||
SC1a–g + SC2–SC7 pass; deliverables 1–7 exist; non-goals untouched; the engine is provenance-gated + keep-both-only (no supersede) in CODE (not just docs); consolidation state is the brain sidecar (not the plugin state file); the consolidation pass is operator-invoked + the diff operator-gated (no auto-apply); session-start nudge zero-dep (no profile.md parse) + tested; committed (push per window, confirm first); `architecture.md` SB-S2 row → landed; version/CLAUDE.md/README touched only if a bump is taken.
|
||||
79
docs/second-brain/brief-sb-s3a.md
Normal file
79
docs/second-brain/brief-sb-s3a.md
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
# Brief — SB-S3a: the first profile.md READER
|
||||
|
||||
> **Slice:** SB-S3a (first sub-slice of SB-S3, the cross-silo + ops-centre arc step).
|
||||
> **Status:** LANDED `4fa411f` (2026-06-23) — gate 95/0/0, `ASSERT_BASELINE_FLOOR` 80. Light-Voyage-hardened (brief-review APPROVE · scope-guardian ALIGNED · plan-critic REVISE→folded). SC1/2/3/6 gate-proven; SC4 inspection-pass; SC5 (read-back) deferred to a reloaded session.
|
||||
> **Predecessors:** SB-S0 (id/profile substrate) · SB-S1 (published-gold ingest) · SB-S2 (consolidation motor — *motor-only, no reader*).
|
||||
|
||||
## 1. Operator decision (2026-06-23)
|
||||
|
||||
SB-S3 is the arc's largest slice and was decomposed into four sub-slices (value-first / risk-managed): **S3a reader · S3b supersede · S3c cross-silo id-threading · S3d hygiene+ops**. The operator picked **S3a — the profile.md reader — first**, and within S3a the first reader is **one agent: `strategy-advisor`**, wired **READ-only**.
|
||||
|
||||
## 2. Why a reader, and why first
|
||||
|
||||
The whole arc was built *"so S3's reader inherits rich data"* (`consolidation-loop.md:67`). S2 grows `brain/profile.md` but **no agent or command consumes it today** — verified: 0 references to `brain/profile.md` in `agents/`/`commands/`; every "profile" hit points at the *legacy* flat `profile/user-profile.md`, not the SB-S0 two-layer brain profile. The reader is the deferred payoff of S0→S2: it turns a motor-only system into one that **feeds content generation** — the first end-to-end proof of capture → consolidate → read-back-into-generation.
|
||||
|
||||
It is first because it has **no dependency on id-threading** (it reads the profile *facts*, not the cross-silo graph), it is **additive / non-breaking**, and it is the **lowest-risk** of the four (read-only consumption of an existing file).
|
||||
|
||||
## 3. Scope — what is IN (S3a)
|
||||
|
||||
1. **Wire `strategy-advisor` to read `brain/profile.md`.** Add the brain profile to the agent's existing **Step 0: Load Context** list:
|
||||
`${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/brain/profile.md`
|
||||
The agent reads the markdown in-context and reasons over it — no parser, consistent with the arc thesis (Claude *is* the retrieval engine; `architecture.md:14`).
|
||||
|
||||
2. **Consumption contract (how the agent uses it).** A short subsection in the agent prompt that tells it:
|
||||
- The profile has two layers — `## Static` (stable, high-confidence) and `## Dynamic` (emerging) — and each fact line carries `evidence_count` + `last_seen`. Higher `evidence_count` / more recent `last_seen` = stronger/fresher signal; weight accordingly.
|
||||
- **Anti-sycophancy (binding, `architecture.md:58`):** treat every profile fact as *evidence to TEST, not flatter*. Counter-pressure it against analytics/state each time it is used (e.g. "the profile says you lean X, but your last imports show Y — test that"). The profile informs, it never dictates or flatters.
|
||||
|
||||
3. **Graceful absence.** `brain/profile.md` does not exist until `brain init` runs (fresh installs have no brain). The agent must **degrade silently** when the file is missing or empty — no error, no "I couldn't find your profile" noise; it simply proceeds on its other context sources.
|
||||
|
||||
4. **A deterministic wiring test** in the structure-lint / `test-runner.sh` gate: two UNCONDITIONAL assertions — (a) `strategy-advisor.md`'s context-load declares `brain/profile.md`; (b) the agent carries the anti-sycophancy sentinel **literal `evidence to TEST`** (exact-literal grep + a non-vacuity self-test, per the repo lint idiom in `test-runner.sh` Sections 13/16c — loose patterns match vacuously). This is the TDD anchor (see §6) and protects the wiring against future agent-file edits.
|
||||
- **Assertion-floor lockstep (binding):** the two new checks are UNCONDITIONAL, so `ASSERT_BASELINE_FLOOR` in `test-runner.sh` (Section 18; currently **78** — verify at plan time) must be bumped by exactly +2 → **80** in the same change, per the assertion-erosion guard. Do NOT pin to a deps-present TOTAL.
|
||||
|
||||
5. **Doc/count reconciliation.** Update the binding counts/docs touched (CLAUDE.md agent table note if needed, STATE telling, `consolidation-loop.md` reader-status line: "S3a wires the first reader — `strategy-advisor`").
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred to later S3 sub-slices)
|
||||
|
||||
- **More than one reader.** `content-optimizer` is the obvious second reader and the pattern generalizes trivially, but S3a wires **exactly one** agent — the smallest valuable proof. Follow-on readers are separate work.
|
||||
- **A hook-level / `user-prompt-context.mjs` digest reader.** A broad "inject a profile digest into every prompt" reader is attractive but carries a twin `.mjs` parser (twin-sync with `profile.ts`) + per-prompt context cost — *not* a good first proof. Deferred.
|
||||
- **Cross-silo id-threading (S3c).** The reader consumes profile facts, not the post↔specific↔trend↔analytics graph. No `mintEntityId` threading here.
|
||||
- **Supersede arm (S3b)**, **content-history retirement / triple-post reconciliation + operations.md (S3d)** — separate sub-slices.
|
||||
- **Any WRITE to `brain/profile.md`.** The profile is mutated ONLY via `brain consolidate --apply --confirm` (operator-gated). S3a is strictly read-only; it adds no write path.
|
||||
- **A new parser or new `.mjs`.** S3a is markdown-agent edits + a lint assertion only.
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **READ-only** — S3a never writes the profile; the `--apply --confirm` gate stays the sole writer.
|
||||
- **Anti-sycophancy is a built-in default, not a toggle** (`architecture.md:58`).
|
||||
- **Provenance spine intact** — the profile already encodes provenance; the reader must not undermine the published-only learning guard.
|
||||
- **No tributary schema changes** (that's S3c).
|
||||
- **Fresh-clone safe** — missing brain → silent degrade, no crash, no nag from the agent.
|
||||
- **TDD iron law** — the failing wiring test lands before the agent edit.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1 — wired:** `strategy-advisor.md` Step 0 context-load includes `brain/profile.md`. *(lint assertion, deterministic)*
|
||||
- **SC2 — anti-sycophancy framed:** `strategy-advisor.md`'s profile-consumption subsection contains the exact durable sentinel literal **`evidence to TEST`**, asserted by an exact-literal grep + a non-vacuity self-test (repo idiom, `test-runner.sh` §§13/16c). *(lint assertion)*
|
||||
- **SC3 — gate green:** `scripts/test-runner.sh` (the structure lint) stays green with the new assertions; its `BRAIN_TESTS_FLOOR` (82) is unchanged — no brain TS is touched. The hook suite (~136) runs under the **separate** `node --test hooks/scripts/__tests__/*.test.mjs` runner (NOT part of `test-runner.sh`) and is untouched because no hook code path changes.
|
||||
- **SC6 — assertion floor honoured:** `ASSERT_BASELINE_FLOOR` bumped by exactly the number of new unconditional checks (+2 → 80); the gate's self-count check passes.
|
||||
- **SC4 — graceful absence (manual/behavioural):** running `strategy-advisor` with **no** `brain/profile.md` produces normal output, no error/noise about the missing file.
|
||||
- **SC5 — read-back works (manual/behavioural):** running `strategy-advisor` with a **populated** `brain/profile.md` surfaces ≥1 profile fact as *evidence-to-test* in its recommendation, counter-pressured against analytics/state — not parroted.
|
||||
|
||||
## 7. Verification
|
||||
|
||||
- **Deterministic (gate):** SC1–SC3 via the structure-lint assertion in `test-runner.sh`. `cd scripts/brain` not required — this is an agent-file + lint change; brain TS is untouched.
|
||||
- **Behavioural (manual, documented):** SC4 + SC5 — the honest limit. Agent-prompt behaviour is not unit-testable (the plugin's command-testing workstream is still open). The plan must include a documented manual run: (a) empty-brain run → clean output; (b) populated-brain run (seed via `brain init` + a `--apply --confirm` cycle, or a fixture profile) → confirm a profile fact appears as tested evidence. Record the result in STATE/changelog at land.
|
||||
|
||||
## 8. Open questions for brief-review / the operator
|
||||
|
||||
1. **Agent choice:** `strategy-advisor` recommended; operator may redirect to `content-optimizer` (post-level grounding) — confirm at the brief gate.
|
||||
2. **Lint location:** which existing lint file in the `test-runner.sh` gate hosts the SC1/SC2 assertion (plan resolves precisely).
|
||||
3. **Behavioural test honesty:** is the documented manual verification (SC4/SC5) acceptable for S3a's land, given agent-behaviour is not unit-testable — or does the operator want a fixture-driven harness scoped in (larger)?
|
||||
|
||||
## 9. Brief-review (light-Voyage) — folded
|
||||
|
||||
`voyage:brief-reviewer` verdict: **APPROVE** (scope crisp, non-goals correct, invariants inherited, deterministic-vs-behavioural split honest). Three [FIX]es folded above:
|
||||
- SC3 no longer conflates two runners (brain-82 floor in `test-runner.sh` vs the separate hook runner). ✅
|
||||
- `ASSERT_BASELINE_FLOOR` lockstep bump (+2 → 80) made binding in §3.4 + SC6. ✅
|
||||
- SC2 pinned to the exact literal `evidence to TEST` + non-vacuity self-test (anti-vacuity idiom). ✅
|
||||
|
||||
Reviewer confirmed: `strategy-advisor` is the better first reader than `content-optimizer`; SC4/SC5 manual split is correct; do NOT scope a behavioural harness into S3a (keeps "smallest valuable proof").
|
||||
111
docs/second-brain/brief-sb-s3b.md
Normal file
111
docs/second-brain/brief-sb-s3b.md
Normal file
|
|
@ -0,0 +1,111 @@
|
|||
# Brief — SB-S3b: the supersede arm
|
||||
|
||||
> **Slice:** SB-S3b (second sub-slice of SB-S3). **Status:** DRAFT — awaiting operator "go" before any code. Light-Voyage: brief-review APPROVE-WITH-FIXES (8 FIX folded; 3 RISK carried to the plan).
|
||||
> **Predecessors:** SB-S0 (id/profile substrate — reserved the `superseded` status) · SB-S1 (published gold) · SB-S2 (consolidation motor — keep-both, *no supersede*) · **SB-S3a (first reader `strategy-advisor` — LANDED `4fa411f`, gate 95/0/0, ASSERT floor 80).**
|
||||
> **Grounded in:** a read-only code map of `scripts/brain/` (file:line cited throughout) + a brief-review pass against the same code.
|
||||
|
||||
## 1. Operator decision (2026-06-23)
|
||||
|
||||
SB-S3 was decomposed into four sub-slices; after S3a (reader) landed, the recommended order is **S3b (supersede) → S3c (id-threading) → S3d (hygiene+ops, last)**. S3b is next because it has the **lowest blast radius** (bounded to the `scripts/brain/` TS package — pure functions, fully unit-testable, unlike S3a's behavioural SCs), it **completes the consolidation motor's one explicit TODO** (`consolidate.ts:11`: *"no supersede in S2 — that's S3"*), and it **strengthens the read path S3a just shipped** (a profile with supersede gives the new reader cleaner, non-contradictory facts).
|
||||
|
||||
## 2. Why supersede, and the central design decision
|
||||
|
||||
S2's engine handles a value-conflict by **keep-both**: the old fact is left untouched and a new alt fact is added with a distinct content-id (`consolidate.ts:120-129`). That is correct for a *genuine contradiction* (two views coexist), but wrong for a **temporal update** — when a new fact *replaces* an outdated one (the user changed roles, a number moved, a stance evolved). Today there is **no way to retire the stale fact**; it persists forever, and the S3a reader has to weigh a fact the user has already moved past.
|
||||
|
||||
The `superseded` status was reserved at SB-S0 (`types.ts:17`, `:23`; grammar `profile.ts:58`; round-trip proven `profile.test.ts:35`) precisely for this — **but nothing ever sets it.** S3b is the code that does.
|
||||
|
||||
**Central design decision (the "fuzzy heart", `brief-sb-s2.md:115`) — supersede is driven by an EXPLICIT signal, never engine auto-classification.** The S2 architecture splits cleanly: the **session** does the fuzzy extraction, the **engine** is deterministic mechanics, the **operator** gates the diff before apply. Classifying *temporal-update vs contradictory* is a fuzzy judgement — so it stays where the fuzziness already lives (the session's candidate extraction), not baked into a pure function. A candidate carries an explicit "this supersedes the prior fact for key X" intent; `proposeDiff` **deterministically** routes it; the operator **sees and confirms** the supersession in the rendered diff before `--apply --confirm` writes it. This honours the S2 promise (`brief-sb-s2.md`: *"reserve supersede for an explicit confirm signal"*) and keeps the engine deterministic and testable.
|
||||
|
||||
**Consequence (the engine has no classification safety net).** Because the engine never validates that a supersede *should* be a supersede, the **operator gate is the only place a mis-classification is caught** — and supersede is the first op that *retires* state (a mis-call silently retires a fact that should have been kept-both, and `status: superseded` makes it look intentional). Two obligations follow: SC6's rendering must make every supersession **visually unmistakable** (old value → new value, both shown, flagged as a retirement), and the doc reconciliation (§3.7) must add an honest-limit line mirroring `consolidation-loop.md:64` — *the engine guarantees supersede mechanics; whether a contradiction is really a temporal update is the session's + operator's judgement.*
|
||||
|
||||
## 3. Scope — what is IN (S3b)
|
||||
|
||||
1. **Candidate supersede signal.** Extend the transient `Candidate` (`consolidate.ts:33-39`) with an explicit, optional supersede intent — minimal shape: `supersedes?: string` (the **key** of the active prior fact this candidate replaces). Set only by the session when it judges a temporal update; absent for ordinary candidates (fully backward-compatible). `validateCandidates` (`cli.ts:147-157`) gains an optional single-line check for the new field.
|
||||
|
||||
2. **A sixth diff array.** `ProfileDiff` (`consolidate.ts:41-47`) gains `supersedes: { oldId, oldValue, newId, newValue }[]` (parallel to the existing `conflicts[]` — the five-array shape means a new op is a new array, not a union edit).
|
||||
|
||||
3. **`proposeDiff` routing (deterministic), with a precise matching rule.** "Matching active prior fact" = the fact returned by `byId.get(mintEntityId({kind:'observed', key}))` — i.e. the live fact at the canonical key-id (see §3.5: retired facts no longer hold the key-id, so this is unambiguously the *active* one). When a candidate carries `supersedes: <key>` AND that lookup finds a fact, route to `supersedes[]`. When the signal is present but **no fact is found at the key-id**, degrade gracefully to a plain **add** (defined, tested behaviour — never a crash). Candidates without the signal are untouched (keep-both/bump/promote/decay exactly as S2).
|
||||
|
||||
4. **`applyDiff` mutation (the first `superseded` writer), state-checked.** Add a branch (`consolidate.ts:144-162`) that, for each `supersedes[]` entry: (a) re-mints the retired fact to a distinct archival id and sets its `status: "superseded"`; (b) installs the new winner under the canonical key-id (see §3.5). The branch is **state-checked, not blind**: a `supersedes[]` entry whose `oldId` is absent from `current` or already `superseded` is **skipped** (this is what makes re-apply idempotent and guards stale diffs — §5). Superseded facts are **retained** in the doc (audit), per `architecture.md:69` (*"keep old `status: superseded` + date"*) — never deleted. Stays a **pure** function returning a new doc.
|
||||
|
||||
5. **Id discipline — winner takes the key-id; retired fact is re-minted off it.** Because `mintEntityId` is value-independent (`id.ts:33`), only one fact may hold the canonical key-id, and it must be the **live** one. So on supersede: the **new winner takes `mintEntityId({observed, key})`** (the canonical key-id), and the **retired fact is re-minted to a distinct archival id** (`mintContentId` over an archival seed, e.g. `superseded:${key}::${oldValue}::${oldFirstSeen}`, so it is stable and collision-free). This preserves the **no-duplicate-id** invariant AND keeps `mintEntityId(key)` always pointing at the live fact — so every downstream match (future same-key candidates, the S3a reader) resolves to the winner exactly as before supersede existed. *(This inverts the naïve "keep old id, mint winner fresh", which would leave the key-id on a retired fact and resurrect it on the next same-key candidate — the brief-review's [FIX 1].)*
|
||||
|
||||
6. **CLI rendering, non-vacuously checked.** `renderDiffMd` (`cli.ts:132-145`) gains a `## Supersessions (old → new)` section so the operator sees every retirement before confirming. **No new subcommand** — supersede rides the existing `--propose` / `--apply --confirm` flow; the sole `profile.md` writer stays `cli.ts:215`. Because the operator confirms via the **JSON** diff while reading the **MD**, a **brain-suite test** (`consolidate-cli.test.ts`, repo non-vacuity idiom) asserts **every `supersedes[]` JSON entry produces a corresponding rendered MD line** and a 0-entry diff renders no section — not merely that a section exists (RISK C). *(This is a brain-suite test, not a `test-runner.sh` Section — so it is counted by `BRAIN_TESTS_FLOOR`, not by a new unconditional structure-lint check; see §3.7.)*
|
||||
|
||||
7. **Gate + doc reconciliation.** **Verified baseline (S3a landed):** `BRAIN_TESTS_FLOOR` = 82 (`test-runner.sh:716`), `ASSERT_BASELINE_FLOOR` = 80 (`:913`). Bump `BRAIN_TESTS_FLOOR` by the new brain-test count; **`ASSERT_BASELINE_FLOOR` stays 80** — no new *unconditional `test-runner.sh` structure-lint* check is added (the render-fidelity check lives in the brain suite, §3.6). `BRAIN_TESTS_FLOOR` is the only floor that moves. Reconcile the now-stale "no supersede" claims (`consolidation-loop.md:34`, `:71`; `consolidate.ts:11` docstring) to the true state, and add the honest-limit line from §2.
|
||||
|
||||
8. **`--gather` active-only filter (plan-critic FIX 8b — completeness).** `runConsolidate --gather` (`cli.ts:174`) emits `profileFacts` to the extraction session with **no status filter** today. After supersede lands, that would re-present `status:superseded` archival facts as live context — re-introducing the exact retired values this slice exists to retire (§2). The gather output is filtered to `status === "active"` (one line). Without this the slice regresses its own goal.
|
||||
|
||||
9. **Decay excludes superseded (plan-critic FIX 8c — completeness).** A retained *dynamic* superseded fact is still in `current.dynamic`, so the decay scan (`consolidate.ts:132-134`) would throw `staleFlags` for it forever. The decay filter excludes `status:superseded` (static archival facts are already decay-exempt — dynamic-only scan). Supersede and decay stay orthogonal.
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **Engine auto-classification of temporal-update vs contradictory.** The signal is explicit (§2). The engine never guesses.
|
||||
- **The other two contradiction-typology branches** — *condition-dependent (scope)* and *distractor (drop)* (`architecture.md:69`, `research/secondbrain-sota.md:37`). S3b ships **temporal-update → supersede ONLY**; contradictory stays keep-both (S2). The other two are later, purely-additive arrays (deferring them leaves no broken half-state — confirmed by brief-review).
|
||||
- **Auto-demotion of stale facts.** Decay (`staleFlags`, `consolidate.ts:132-134`) stays **advisory** — never mutates status or removes a fact. Supersede and decay stay orthogonal (no shared field). A stale fact is *not* auto-superseded.
|
||||
- **Cross-silo id-threading (S3c)** · **content-history retirement / operations.md / triple-post reconciliation (S3d)** · **any new reader** (S3a's reader unchanged; `content-optimizer` is later).
|
||||
- **Chained / multi-fact supersede as a FEATURE.** One active prior → one new winner per candidate. Chains (A→B→C in one pass) and one-supersedes-many are out. **But the intra-batch SEMANTICS are defined, not undefined** (RISK A): the supersede fork carries a `touched`-style guard (analogous to `consolidate.ts:97`) so that within one candidate batch, two candidates targeting the same key, or a candidate superseding a fact another candidate just created, resolve deterministically (first-writer-wins + the later one degrades to add/bump) rather than corrupting. Deferring the *feature* is fine; the *batch rule* is in scope.
|
||||
- **Grammar changes.** None needed — `superseded` is already accepted end-to-end.
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **Operator-gated, single disk-writer.** No new write path to disk; `--apply --confirm` (`cli.ts:215`) stays the sole `profile.md` writer; `--apply` refuses without `--confirm`.
|
||||
- **Pure, idempotent engine.** `proposeDiff`/`applyDiff` stay pure (no I/O). Idempotency holds at BOTH layers: `proposeDiff` produces no new `supersedes[]` entry once a supersession has landed (winner matches by value → bump; old fact already superseded), and `applyDiff`'s supersede branch is **state-checked** so re-applying the same diff does not double-install the winner or double-flip status.
|
||||
- **Stale-diff safety (named assumption + guard).** A pending diff is meant to be applied against the profile it was proposed from; a profile mutated between propose and apply is operator error. S3b does not silently corrupt in that case: `applyDiff` **skips** any `supersedes[]` entry whose `oldId` is absent or already `superseded` in `current` (the §3.4 state-check), rather than retiring the wrong fact.
|
||||
- **Audit-preserving.** Superseded facts are retained with `status: superseded`, never deleted — history is inspectable.
|
||||
- **No-duplicate-id.** Every fact keeps a distinct id; the canonical key-id is held only by the live winner; the retired fact moves to a distinct archival id.
|
||||
- **Backward-compatible.** Candidates without the signal behave exactly as S2; a profile with zero supersessions serialises byte-identically to today (`serializeProfile` unchanged; grammar already round-trips `superseded`).
|
||||
- **Decay stays advisory** and orthogonal to supersede.
|
||||
- **No grammar / no cross-silo threading / no new reader** in S3b.
|
||||
- **TDD iron law** — each engine rule is pinned by a failing unit test before the CLI is wired (mirrors `brief-sb-s2.md:115`).
|
||||
|
||||
## 6. Success criteria (testable — deterministic, unit-level)
|
||||
|
||||
- **SC1 — routing:** a candidate with `supersedes: <key>` + a fact at `mintEntityId({observed, key})` → `proposeDiff` emits exactly one `supersedes[]` entry (oldId/oldValue + newId/newValue) and **no** `conflicts[]`/keep-both entry for it. *(unit)*
|
||||
- **SC2 — mutation + concrete ids:** `applyDiff` produces a doc where (a) the **winner** carries `id == mintEntityId({observed, key})`, `status: "active"`, `evidence_count: 1`, the new value; (b) the **retired** fact carries `status: "superseded"` and an id `!= mintEntityId({observed, key})` (the archival id); (c) the retired fact is **still present** (count of facts for the key = 2, one active + one superseded). *(unit)*
|
||||
- **SC3a — propose-level idempotency:** after a supersession has landed, a second `gather→propose` cycle with the *same* now-current value produces **no** new `supersedes[]` entry (the winner matches by value → bump). *(unit)*
|
||||
- **SC3b — apply-level idempotency:** `applyDiff` of a supersede diff onto a doc where that supersession is already applied does **not** double-install the winner, double-flip status, or create a duplicate id (the §3.4 state-check skips it). *(unit)*
|
||||
- **SC4 — graceful no-target:** a `supersedes: <key>` signal with **no fact at that key-id** (active or otherwise — superseded facts no longer hold the key-id) degrades to a plain add; no throw. *(unit)*
|
||||
- **SC5 — round-trip through the real path:** `parseProfile(serializeProfile(applyDiff(cur, supersedeDiff)))` deep-equals `applyDiff(cur, supersedeDiff)` — a superseded fact *produced by the engine* round-trips, not just the hand-built `profile.test.ts:35` fixture. *(unit)*
|
||||
- **SC6 — CLI gate + render fidelity:** `--propose` renders a `## Supersessions` section in `pending-diff.md` with one line per `supersedes[]` entry (old → new, both values shown), and a 0-entry diff renders no section (decoy); `--apply` writes the superseded status ONLY with `--confirm` (refuses otherwise). *(cli — brain suite)*
|
||||
- **SC7 — no regression:** contradictory candidates (no signal) still keep-both; bump/promote/decay/published-only all unchanged (regression pins). *(unit)*
|
||||
- **SC8 — intra-batch determinism:** a batch with two candidates superseding the same key (or one superseding a fact another candidate creates in the same batch) resolves deterministically per the §4 `touched`-guard rule — no duplicate id, no double-supersede. *(unit)*
|
||||
- **SC9 — gate green:** brain suite ≥ bumped `BRAIN_TESTS_FLOOR`; `scripts/test-runner.sh` green; **`ASSERT_BASELINE_FLOOR` unchanged at 80** (the render-fidelity check is a brain-suite test, so `BRAIN_TESTS_FLOOR` is the only floor that moves). *(plan-critic FIX 7)*
|
||||
- **SC10 — gather active-only:** `--gather` output excludes `status:superseded` facts (no retired value re-presented as live context). *(cli)*
|
||||
- **SC11 — decay excludes superseded:** a retained dynamic superseded fact throws no `staleFlags`. *(unit)*
|
||||
- **SC12 — supersede-wins ordering:** a diff that bumps/promotes AND supersedes the same id resolves to supersede-wins (the superseded fact is not bumped/promoted). *(unit)*
|
||||
|
||||
## 7. Verification
|
||||
|
||||
- **Deterministic (the whole slice):** SC1–SC9 are unit/CLI tests in `scripts/brain/tests/` + the structure-lint gate. **Unlike S3a, S3b has no behavioural-only SC** — supersede is pure engine mechanics, fully exercisable in the harness. A key reason it's a good second slice.
|
||||
- **Run:** `(cd scripts/brain && npm test)` then `bash scripts/test-runner.sh`.
|
||||
|
||||
## 8. Open questions — RESOLVED (operator 2026-06-23)
|
||||
|
||||
The id-lifecycle is resolved — winner-takes-key-id (§3.5), per brief-review [FIX 1]. The four forks are now operator-confirmed:
|
||||
|
||||
1. **Q-SCOPE → temporal-update → supersede ONLY** ✅ (condition-dependent + distractor deferred). Clean cut, no broken half-state.
|
||||
2. **Q-LAYER → winner enters DYNAMIC and re-earns promotion** ✅ (N=3). A superseded *static* fact stays static-but-superseded (decay-exempt — `consolidate.ts:132` scans dynamic only — acceptable as retained audit, not live). No static-inherit.
|
||||
3. **Q-SIGNAL → minimal `supersedes?: string` (target key)** ✅ (widen later only if condition-dependent/distractor are built).
|
||||
4. **Q-RETAIN → retain superseded facts as audit trail** ✅ (no pruning in S3b; any pruning is an S3d hygiene concern).
|
||||
|
||||
## 9. Brief-review (light-Voyage) — folded
|
||||
|
||||
`voyage:brief-reviewer` verdict: **APPROVE-WITH-FIXES** (PROCEED_WITH_RISKS). All 8 [FIX] folded above:
|
||||
- **[FIX 1]** post-supersede id lifecycle → §3.5 inverted to winner-takes-key-id + retired-fact-re-minted (the one latent-corruption fix). ✅
|
||||
- **[FIX 2]** SC3 idempotency layer → split into SC3a (propose) + SC3b (apply, state-checked). ✅
|
||||
- **[FIX 3]** SC2 concrete id/status/count assertions (non-vacuous). ✅
|
||||
- **[FIX 4]** stale-diff trust boundary → §5 named assumption + §3.4 skip-guard. ✅
|
||||
- **[FIX 5]** floor baseline → §3.7 states the **verified** post-S3a baseline (82 / 80; S3a landed `4fa411f`). ✅
|
||||
- **[FIX 6]** SC5 round-trips through `applyDiff` output, not the hand fixture. ✅
|
||||
- **[FIX 7]** Q-LAYER decided default → §3 IN-scope; only static-inherit stays open (§8.2). ✅
|
||||
- **[FIX 8]** SC4 names the scanned fact-set (the key-id lookup). ✅
|
||||
|
||||
The 3 [RISK] are carried as explicit plan obligations:
|
||||
- **[RISK A]** intra-batch supersede semantics → defined in §4 + pinned by SC8 (`touched`-guard).
|
||||
- **[RISK B]** operator gate is the only classification net → §2 consequence + §3.6 unmistakable render + §3.7 honest-limit doc line.
|
||||
- **[RISK C]** `renderDiffMd` JSON↔MD divergence → §3.6 render-fidelity check + SC6 lint.
|
||||
|
||||
Brief-review confirmed (no fix): byte-identical backward-compat, five→six-array idiom, no grammar change, the "temporal-update only" scope cut, and the optional-field backward-compat of `Candidate.supersedes?`.
|
||||
|
||||
**Plan-critic pass (folded back into this brief).** Reviewing the plan, `voyage:plan-critic` surfaced two latent-corruption BLOCKs (applyDiff must replace-in-place not copy; proposeDiff must value-guard the supersede fork) — both resolved in the plan — and two completeness gaps that belong in the brief's scope: the **`--gather` active-only filter (§3.8)** and **decay excludes superseded (§3.9)**, with SC10/SC11/SC12 added to §6. `voyage:scope-guardian`: **ALIGNED** (0 creep / 0 gap). The plan (`plan-sb-s3b.md`) carries the full BLOCK/FIX ledger.
|
||||
115
docs/second-brain/brief-sb-s3c.md
Normal file
115
docs/second-brain/brief-sb-s3c.md
Normal file
|
|
@ -0,0 +1,115 @@
|
|||
# Brief — SB-S3c: cross-silo id-threading (the "which raw material performs?" graph)
|
||||
|
||||
> **Slice:** SB-S3c (third sub-slice of SB-S3). **Status:** **LANDED** — operator go 2026-06-23; TDD complete, brain suite 113/113, gate 95/0/0 (`BRAIN_TESTS_FLOOR` 94→113, `ASSERT_BASELINE_FLOOR` unchanged at 80). Light-Voyage: brief-review **APPROVE-WITH-FIXES** (5 FIX folded §9), plan-critic **REVISE → 1 BLOCK + 4 MAJOR + 4 MINOR folded** (§10), scope-guardian **ALIGNED** (0 creep/0 gap).
|
||||
> **Predecessors:** SB-S0 (id/provenance substrate — `mintEntityId`/`mintContentId`, `id.ts:6-7` reserved "SB-S3 will thread this id through the tributaries") · SB-S1 (published-gold ingest — the post anchor `ingest/published/<contentId>.md`) · SB-S2 (consolidation motor) · SB-S3a (first reader `strategy-advisor` — LANDED `4fa411f`) · **SB-S3b (supersede arm — LANDED `585f972`, gate 95/0/0, BRAIN floor 94, ASSERT floor 80).**
|
||||
> **Grounded in:** a read-only code map of all four tributaries + `scripts/brain/` (file:line cited throughout) — Explore sweep + firsthand reads of `id.ts`, `consolidate.ts`, `types.ts`, `profile.ts`, `ingest.ts`.
|
||||
|
||||
## 1. Operator decision (2026-06-23)
|
||||
|
||||
SB-S3 was decomposed into four sub-slices; standing operator direction is **S3a (reader) → S3b (supersede) → S3c (id-threading) → S3d (hygiene+ops, last)**. S3a + S3b have landed. **S3c is next** — it is the slice the whole arc was built toward: the architecture's single hardest problem (`architecture.md:17`) — *"A published post lands in three non-referencing places with no shared id. The question 'which raw material actually performs?' (specific → post → measured analytics) is **unanswerable today**"* — and the explicit SB-S3 payoff (`architecture.md:54`, `:80`: *"thread the id through tributaries → post ↔ specific ↔ trend ↔ analytics assemblable"*).
|
||||
|
||||
The go-before-code gate still applies per slice: this brief + the plan are presented for operator "go" (with the genuine forks in §8) before any code.
|
||||
|
||||
## 2. The north-star query, and the central design decision
|
||||
|
||||
**What S3c must make answerable:** given a published post, **what raw material fed it** (the specific(s) that grounded it, the trend(s) that inspired it) and **how did it perform** (its measured analytics). The killer second-brain query is `specific → post → measured analytics`: *which raw material actually performs?*
|
||||
|
||||
**The grounding surfaced two hard constraints that reshape the slice** (this is why the design below is leaner — and more arc-aligned — than the STATE's anticipated "mutate every tributary schema"):
|
||||
|
||||
- **C-1 — the profile grammar is rigid.** `brain/profile.md` is a fixed **6-token line-grammar** (`profile.ts:27-37,57-58`; `FACT_RE`; `parse∘serialize===doc`, SC2). Persisting a fact→post provenance link **onto the fact** would add a 7th token = a **breaking `SCHEMA_VERSION` bump** (`types.ts:26`, "bumped only on a breaking grammar change") + a migration of every existing `profile.md`. That is its own slice, not S3c. **OUT (§4).**
|
||||
- **C-2 — analytics has no body and no URN.** The LinkedIn CSV export carries only `title` (the post-content column, taken **verbatim** by the parser — `csv-parser.ts:181`; the "first ~100 chars" in `models/types.ts:3` is a property of *LinkedIn's export*, not a code truncation) + `publishedDate` + metrics — **no full body, no permalink/URN anywhere in the codebase**. So analytics **physically cannot compute** the brain's `mintContentId(body)` (`id.ts:51`, body-byte-exact). Nor does its own `id` help: `PostAnalytics.id = hash(title + date)` (`csv-parser.ts:138`) is derived from the **very fields the resolver already joins on**, so it carries **zero independent linking signal**. The analytics↔post link therefore **cannot be a stored shared id** — it must be a **resolver** (join by title-prefix + date). This is an honest heuristic, not a guarantee, and the design names it as such.
|
||||
|
||||
**Central design decision — thread the cross-ids onto the HUB-side post record, not into each tributary; assemble analytics by resolver.** The architecture is explicit that **tributaries stay separate and the hub holds the distilled summary + pointers** (`architecture.md:43,49`: *"Keep tributaries; add a thin hub + a cross-silo id"*; *"Forcing one schema would destroy [each silo's] distinctions"*). The temporally-correct anchor is the **post**: trends + specifics exist *before* a post and feed it; analytics measures it *after*. The post's canonical object already exists — `ingest/published/<contentId>.md` — and it already carries the content-id (filename stem + `id:` header). Its **parse** side is tolerant of added header lines (`headerScalar` reads each key by regex, `ingest.ts:61-65`), so threading new fields is round-trip-safe; the **serializer** is a fixed 5-line array (`ingest.ts:51-57`) that S3c **edits** to conditionally append the two new lines (this is a deliberate edit, not an auto-handled field — corrected per brief-review FIX 5). Unlike the rigid profile grammar, this header grammar takes additive optional lines cleanly. So:
|
||||
|
||||
1. the **published record** carries the post's raw-material provenance as **additive, optional header lines** (`specifics:` + `trends:`, lists of tributary ids); and
|
||||
2. a **pure assembler** joins published-record ↔ analytics-row by title-prefix + date (with a confidence tier) to attach measured performance.
|
||||
|
||||
This keeps **all four tributary schemas (trends, specifics, analytics, post-tracking) untouched**, needs **no analytics base-36-id reconciliation**, and is strictly additive + backward-compatible — while delivering the exact graph the north-star query needs. *(That S3c's blast radius is bounded to the `scripts/brain/` package, not the tributaries, is a finding that corrects the STATE's "rører tributær-skjemaer" expectation — surfaced as the headline fork, §8 Fork-1.)*
|
||||
|
||||
## 3. Scope — what is IN (S3c)
|
||||
|
||||
1. **Published record carries raw-material provenance (additive, optional).** Extend `PublishedRecord` (`ingest.ts:27-40`) with `specifics: string[]` + `trends: string[]` (tributary ids the post was built from; default empty). `serializePublishedRecord` (`ingest.ts:50-58`) emits them as header lines **only when non-empty** (omit-empty → old records serialize byte-identically); `parsePublishedRecord` (`ingest.ts:75-99`) reads them (absent → `[]`). Ids are validated to the 12-hex shape (the existing `id` guard idiom, `ingest.ts:82`); a malformed id throws (never silently dropped). The `\n---\n` sentinel split is unaffected (header lines never contain it).
|
||||
|
||||
2. **Producer surface — tag raw material at ingest.** `ingestText` (`ingest.ts:165-180`) accepts optional `specifics?: string[]` / `trends?: string[]`, threaded onto the record. The `brain ingest` CLI gains **repeatable** `--specific <id>` / `--trend <id>` flags so the operator can tag a post's raw material at capture time. **This requires a parser change (FIX 1):** the shared `parseFlags` (`cli.ts:44-60`) returns `Record<string,string>` and **overwrites** on a repeated key (`out[key] = next`), so it cannot collect `[a,c]`. S3c extends flag-collection to gather repeated `--specific`/`--trend` into arrays — **without changing single-value flag behaviour** (`--file`/`--source`/`--date`/`--scan-inbox` parse exactly as today; regression-pinned, SC12). Minimal-disruption shape (a dedicated repeatable-flag pass for the two keys, or a `string|string[]` collect) is pinned in the plan. This makes the substrate **immediately usable end-to-end** (not fixture-only): ingest a post tagged `--specific X` → assemble → see the graph. Backward-compatible (flags absent → empty arrays = today's behaviour).
|
||||
|
||||
3. **The assembler (the payoff) — a PURE function.** `assemblePostGraph({ records, analytics })` over already-loaded inputs (no I/O in the core) returns, per published post: its `contentId`, its `specifics[]`/`trends[]` ids, and its **matched analytics** — `{ row, confidence }` carrying the **whole `PostAnalytics` row reference** (FIX 4: the assembler attaches the full row; which metric the *CLI renders* is Fork-4, decoupling the data shape from presentation), or `none`.
|
||||
**The join rule (named here, not deferred — FIX 3), absorbing the title↔body asymmetry (verbatim CSV title vs byte-exact body, C-2):** normalize **both** sides with the brain's content normalization (`normalizeContent` idiom: lowercase + collapse-whitespace + trim — the plan pins the exact fn / shared helper) and require **`norm(title)` is a prefix of `norm(body)`** with a **minimum prefix-length floor** (a too-short title must not false-match any body — floor value pinned in the plan, e.g. ≥ N chars). **Confidence tiers:** `high` = prefix-match (≥ floor) + same `published_date`; `low` = prefix-match (≥ floor) but date off by ≤ a small tolerance, OR a near-miss where LinkedIn truncated the title mid-word (the body's normalized text starts with `norm(title)` minus a trailing partial token); `none` = no qualifying prefix match. Deterministic, fully unit-testable (incl. a truncation near-miss case, SC7). The assembler reads tributary ids **as opaque 12-hex strings** and takes a **minimal analytics input shape** (`{ title, publishedDate, metrics, … }` — the raw `PostAnalytics` JSON shape, `models/types.ts:1-8`), so it does **not** import the trends/specifics/analytics packages (decoupled, §5).
|
||||
|
||||
4. **A thin read-only IO + CLI surface.** A `brain assemble` (or `brain graph`) subcommand: loads published records (`dataRoot('ingest/published')`) + the analytics batches by **inlining a raw-JSON read** of `dataRoot('analytics/posts')/*.json` (each file is an `AnalyticsBatch` with `.posts[]`; storage path `<root>/posts/<date>-<shortId>.json`, `storage.ts:146-152`) — **NOT** by importing the analytics package's `loadAllPosts` (`storage.ts:197`), per the §5 decoupling. It extracts the minimal shape from each `posts[]` entry (note the field is **`publishedDate`**, not `date` — RISK B), runs `assemblePostGraph`, and prints the post → raw-material → performance join (newest first). **Read-only** — it never writes the brain. A missing `analytics/posts/` (gitignored — absent on a fresh clone, RISK C) or missing `ingest/published/` degrades to an empty/partial graph (every post `match: none`), never a throw. *(Fork-3, §8: pure-function-only with CLI deferred is the lighter alternative.)*
|
||||
|
||||
5. **Gate + doc reconciliation.** Bump `BRAIN_TESTS_FLOOR` (currently **94**, `test-runner.sh:716`) by the new brain-test count; **`ASSERT_BASELINE_FLOOR` stays 80** — no new *unconditional structure-lint* check (the new tests are brain-suite tests, counted by `BRAIN_TESTS_FLOOR`, same idiom as S3b §3.7). Reconcile the now-true status lines (`consolidation-loop.md`, `architecture.md:80` SB-S3 row, CLAUDE.md/STATE counts, the `id.ts:6-7` "SB-S3 will thread" comment).
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **Persisting the fact→post link on `brain/profile.md` (the profile-grammar bump, C-1).** The post→specifics/trends graph + the analytics resolver answer the north-star query **without** touching the rigid 6-token grammar. A 7th `source`/provenance-id token is a breaking `SCHEMA_VERSION` change with a migration — its own slice. The fact already carries `provenance`; that is enough for S3c.
|
||||
- **Mutating the tributary schemas (trends/specifics/analytics/post-tracking).** Per `architecture.md:43,49` tributaries stay separate; the cross-ids live hub-side on the post record (§2). No `TrendRecord`/`Specific`/`PostAnalytics` field is added; no analytics base-36-id → sha256 reconciliation. *(Fork-1 is exactly this choice — operator may redirect to the broader mutate-tributaries shape.)*
|
||||
- **post-tracking (`state-updater.mjs`) content-id.** The loosest silo (regex-mutated markdown, prune-regex risk, `state-updater.mjs:116,145`) and **not on the `specific→post→analytics` critical path**. The post-tracking ↔ published ↔ analytics "triple-post reconciliation" is the named **S3d** concern (`architecture.md:80`). OUT here.
|
||||
- **A producer that auto-records which specifics/trends a draft used.** S3c gives the substrate + the **manual** `--specific/--trend` tag at ingest (§3.2). Auto-capture from a drafting command (e.g. `/linkedin:newsletter` recording its `livedSpecifics`) is follow-on wiring, like S3a wired one reader.
|
||||
- **A fuzzy/semantic analytics match beyond title-prefix + date.** No embedding/edit-distance match. If title-prefix+date doesn't resolve it, it's `none` (honest), not a guess. Tightening the matcher is later, additive.
|
||||
- **Backfilling existing untagged published records.** Old records parse with empty `specifics/trends` and round-trip byte-identically; no migration pass.
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **Tributaries untouched.** Zero schema/field change to trends, specifics, analytics, post-tracking. The assembler **reads** their ids/rows; it never writes them.
|
||||
- **Additive + backward-compatible.** A published record with no specifics/trends serializes **byte-identically** to today (omit-empty); `parse(serialize(rec))===rec` holds with the new fields; `serialize(parse(oldText))===oldText` for pre-S3c records. `mintContentId` is **body-only**, so adding headers never changes a record's id/filename (no dedupe disturbance).
|
||||
- **Pure core, single read-only IO seam.** `assemblePostGraph` is pure (inputs in, graph out — no clock/FS/network). The only IO is the read-only `brain assemble` loader; **no new disk writer** (the `--apply --confirm` consolidation path stays the sole `profile.md` writer; ingest stays the sole `published/` writer).
|
||||
- **Decoupled packages.** The brain assembler does not import the analytics/trends/specifics TS packages; it takes a minimal analytics input shape and treats tributary ids as opaque 12-hex strings.
|
||||
- **Honest matching.** The analytics↔post join is a **named heuristic** with explicit confidence tiers and a `none` outcome — never presented as a guaranteed key. The CLI shows the confidence so the operator can eyeball low-confidence matches.
|
||||
- **Anti-sycophancy / provenance spine intact** — S3c adds a read path; it does not weaken the published-only learning guard or the profile.
|
||||
- **Fresh-clone safe.** Missing `ingest/published/` or missing analytics → the assembler returns an empty/partial graph and the CLI degrades cleanly (no crash, no nag).
|
||||
- **TDD iron law** — each rule (round-trip with the new fields, the producer threading, the match-tier logic, the graceful-absence path) is pinned by a failing brain-suite test before the code.
|
||||
|
||||
## 6. Success criteria (testable — deterministic, unit-level)
|
||||
|
||||
- **SC1 — record round-trip with raw-material ids:** a `PublishedRecord` with non-empty `specifics`/`trends` satisfies `parsePublishedRecord(serializePublishedRecord(rec)) === rec` (deep-equal); ids preserved in order. *(unit)*
|
||||
- **SC2 — byte-identical backward-compat (byte contract pinned, FIX 2):** the new header lines, when present, are appended **after `source:`** and before the `\n---\n` sentinel; a record with **empty** `specifics`/`trends` emits the **unchanged 5-line header** (omit-empty), so against a fixture pre-S3c record string `oldText`, `serializePublishedRecord(parsePublishedRecord(oldText)) === oldText` (byte-equal). *(unit)*
|
||||
- **SC3 — id validation (parse + producer, FIX-add):** a `specifics`/`trends` entry that is not 12-hex throws on `parsePublishedRecord` (never silently dropped), mirroring the `id` guard (`ingest.ts:82`); and the symmetric producer-boundary case — `ingestText({ specifics:["XYZ"] })` with a non-12-hex id — is rejected (validated at the producer, or guaranteed to throw on the immediate round-trip; the plan picks the seam). *(unit)*
|
||||
- **SC4 — producer threading:** `ingestText({ ..., specifics:[a], trends:[b] })` writes a record whose round-tripped header carries exactly `[a]`/`[b]`; absent → `[]` (today's behaviour, regression pin). *(unit)*
|
||||
- **SC5 — CLI repeatable flags:** `brain ingest --specific a --specific c --trend b` produces a published record tagged `specifics:[a,c] trends:[b]`. *(cli — brain suite)*
|
||||
- **SC6 — assemble, high confidence (full row, FIX 4):** for a record whose normalized body begins with an analytics row's normalized title (≥ prefix-floor) and shares its `publishedDate`, `assemblePostGraph` returns that post with `match.confidence === "high"`, the **whole `PostAnalytics` row reference** attached, and its `specifics`/`trends` ids surfaced. *(unit)*
|
||||
- **SC7 — assemble, low / none / truncation near-miss (FIX 3):** (a) no qualifying prefix match → `match: none` (never a false `high`); (b) a too-short title (< prefix-floor) that is a literal prefix → **not** `high` (floor guards false positives); (c) a LinkedIn-truncated-mid-word title (body starts with `norm(title)` minus a trailing partial token) and/or a date off by ≤ tolerance → `"low"`, per the §3.3 rule. *(unit)*
|
||||
- **SC8 — assemble is pure + total:** empty `records` → empty graph; empty `analytics` → every post present with `match: none`; no throw on either. *(unit)*
|
||||
- **SC9 — read-only CLI:** `brain assemble` prints the join (post → specifics/trends → matched analytics + confidence), newest-first, and **writes nothing** (no `profile.md`/`published/` mutation — asserted). Missing analytics/published (RISK C, fresh-clone) → clean partial/empty output, no crash. *(cli — brain suite)*
|
||||
- **SC10 — gate green:** brain suite ≥ bumped `BRAIN_TESTS_FLOOR`; `scripts/test-runner.sh` green; `ASSERT_BASELINE_FLOOR` unchanged at 80. *(gate)*
|
||||
- **SC11 — no regression:** ingest dedup/collision-safety, the published-only invariant, consolidation, supersede — all unchanged (the new fields are additive). *(unit/regression pins)*
|
||||
- **SC12 — single-value flags unregressed (FIX 1):** after the repeatable-flag change, the existing single-value flags (`--file`/`--source`/`--date`) and boolean flags (`--scan-inbox`/`--confirm`) parse exactly as today across all subcommands. *(unit/cli)*
|
||||
|
||||
## 7. Verification
|
||||
|
||||
- **Deterministic (the whole slice):** SC1–SC11 are unit/CLI tests in `scripts/brain/tests/` + the structure-lint gate. **Like S3b (and unlike S3a), S3c has no behavioural-only SC** — the substrate, the producer, and the assembler are pure/CLI mechanics, fully exercisable in the harness with fixtures.
|
||||
- **Run:** `(cd scripts/brain && npm test)` then `bash scripts/test-runner.sh`.
|
||||
- **End-to-end demo (documented, optional):** with a real `LINKEDIN_STUDIO_DATA`: import an analytics CSV (existing path) → `brain ingest --specific <id>` a matching published post → `brain assemble` → confirm the post shows its specific + the matched analytics row. Records the north-star query working once, manually.
|
||||
|
||||
## 8. Open questions / genuine forks for brief-review + the operator (the go-gate)
|
||||
|
||||
1. **Fork-1 (headline — where the cross-ids live).** **RECOMMENDED: hub-side** — thread `specifics`/`trends` ids onto the brain's published record + resolve analytics (§2). Tributaries stay untouched; blast radius is bounded to `scripts/brain/`; most arc-aligned (`architecture.md:43`). **Alternative:** the STATE-anticipated *mutate-every-tributary* shape (add a cross-id field to `TrendRecord`/`Specific`/`PostAnalytics` + reconcile analytics' base-36 id) — higher blast, crosses 3 package boundaries + 3 gates, and (per C-2) still can't give analytics a real shared id without a resolver. The recommendation is to take the leaner hub-side cut; **operator confirms or redirects.**
|
||||
2. **Fork-2 (analytics↔post join).** **RECOMMENDED: pure resolver by title-prefix + date with confidence tiers** (§3.3) — the only mechanism available (C-2: no body/URN). Accept the heuristic + `none` outcome as honest. **Alternative:** require a manual `analyticsId` tag on the published record too (operator types it at ingest) — more precise but more friction and still operator-supplied. Recommend the resolver; a manual tag can be added later, additively.
|
||||
3. **Fork-3 (assembler surface).** **RECOMMENDED: pure `assemblePostGraph` + a thin read-only `brain assemble` CLI** (demonstrable end-to-end). **Alternative:** pure function + tests only, CLI deferred to when a reader/command consumes the graph (smaller, but no operator-visible payoff this slice). Recommend including the thin CLI.
|
||||
4. **Fork-4 (metric *rendered* — presentation only, FIX 4).** The assembler always attaches the **whole `PostAnalytics` row reference** (data shape fixed; SC6 depends on it). Fork-4 governs only what the **CLI renders** as the headline "performance" figure — `engagementRate` (compact) vs a small bundle (impressions + engagementRate + saves). RECOMMEND `engagementRate` as the headline with the row available for a `--verbose` expansion. Plan pins the rendered field; the data shape is not in question.
|
||||
|
||||
## 9. Brief-review (light-Voyage) — folded
|
||||
|
||||
`voyage:brief-reviewer` verdict: **APPROVE-WITH-FIXES**. The central design (hub-side id-threading + analytics resolver) was confirmed sound, honestly bounded against both C-1 and C-2, with a clean IN/OUT boundary (no broken half-state) and verified gate mechanics (`BRAIN_TESTS_FLOOR=94` `test-runner.sh:716`, `ASSERT_BASELINE_FLOOR=80` `:913`). Fork-1 was judged framed honestly (not a rigged fork — the recommendation concedes the alternative's only real advantage and notes C-2 defeats even that). All 5 [FIX] folded above:
|
||||
|
||||
- **[FIX 1]** repeatable-flag blocker — `parseFlags` (`cli.ts:44-60`) returns `Record<string,string>` and overwrites on repeat → §3.2 now scopes the collect-into-arrays change + SC12 regression pin (the most material fix; SC5 was unsatisfiable as written). ✅
|
||||
- **[FIX 2]** SC2 byte contract pinned — new lines appended **after `source:`**, omit-empty, fixture `oldText` byte-equality (`ingest.ts:51-57`). ✅
|
||||
- **[FIX 3]** §3.3 join rule named in-brief (not deferred) — both-side `normalizeContent` + minimum prefix-length floor + truncation near-miss tier; SC7 expanded to floor + near-miss cases. ✅
|
||||
- **[FIX 4]** assembler attaches the **full `PostAnalytics` row reference**; Fork-4 governs only CLI *rendering* — decouples SC6 (data) from the open fork. ✅
|
||||
- **[FIX 5]** §2 citation corrected — *parse* is forgiving (`headerScalar`, `ingest.ts:61-65`); *serializer* is a fixed 5-line array (`:51-57`) S3c edits; + the C-2 note that `PostAnalytics.id = hash(title+date)` carries no independent linking signal. ✅
|
||||
|
||||
The 3 [RISK] are carried as explicit plan obligations (see §3.3/§3.4/§5; the plan's risk ledger pins them):
|
||||
- **[RISK A]** all SCs use synthetic fixtures; the §7 end-to-end demo is the only real-data exercise of the resolver. A green gate does not prove a real LinkedIn CSV title matches a real captured body — the plan keeps the demo and treats a demo `none` as a *normalization-tightening signal* (honest-limit line, mirroring `consolidation-loop.md`/S3b §3.7), not a slice failure.
|
||||
- **[RISK B]** `brain assemble` reads `analytics/posts/*.json` by inlining a raw-JSON read (`dataRoot('analytics/posts')`), **not** `loadAllPosts` — and extracts from `publishedDate` (not `date`). Pinned in §3.4.
|
||||
- **[RISK C]** fresh-clone: `analytics/posts/` is gitignored/absent → the loader must degrade to every-post-`none` (SC8/SC9), never throw. Pinned in §3.4/§5.
|
||||
|
||||
## 10. Plan-critic + scope-guardian (light-Voyage) — folded
|
||||
|
||||
`voyage:plan-critic`: **REVISE → 1 BLOCK + 4 MAJOR + 4 MINOR**, all folded into `plan-sb-s3c.md` (the brief's design held; these were plan-precision defects):
|
||||
- **[BLOCK]** the plan misstated `runIngest`/`main` wiring — `runIngest(flags)` is dispatched without `rest` (`cli.ts:92,238`); Step 4.2 now explicitly changes the signature AND the call site so `collectRepeated(rest,…)` has its input.
|
||||
- **[MAJOR]** parse must NOT reuse `headerScalar` (`ingest.ts:61-65`) — it throws on absent keys, breaking every old record (SC2/SC11); Step 2.2 pins a non-throwing optional reader.
|
||||
- **[MAJOR]** analytics-root skew — `ANALYTICS_ROOT` override (`storage.ts:67-72`) honoured by the analytics package but not by brain `dataRoot`; Step 3.5 + R-B name it as a doc-comment caveat (degrades to `none`, not silent).
|
||||
- **[MAJOR]** `runAssemble` loader shape — needs full `PublishedRecord[]` (for `.body`), not `listPublished`'s summary; Step 4.3 pins full-record load, locally-derived `firstLine`, explicit desc sort.
|
||||
- **[MAJOR]** producer-boundary throw (SC3) under-specified — Step 2.3 pins the guard (regex/message/placement) + empty-array skip so `scanInbox`/SC11 stay green.
|
||||
- **[MINOR ×4]** tie-break determinism (Step 3.4 sorts rows), `PREFIX_FLOOR=24` rationale (Step 3.3), floor sub-count reconciliation (R-G), Step 1 compile-fixup of the two existing record literals.
|
||||
|
||||
`voyage:scope-guardian`: **ALIGNED** — 0 creep / 0 gap. All SC1–SC12 + all 5 FIXes + all 3 RISKs map to concrete plan steps; every brief §4 OUT item is genuinely excluded; the "Not touched" scope fence is accurate (four tributaries untouched, analytics READ-only via inlined JSON, hooks/agents/profile-grammar/`parseFlags` untouched).
|
||||
93
docs/second-brain/brief-sb-s3d.md
Normal file
93
docs/second-brain/brief-sb-s3d.md
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
# Brief — SB-S3d: the operations centre (`operations.md` becomes a READ tributary)
|
||||
|
||||
> **Slice:** SB-S3d (the ops-centre sub-slice of SB-S3). Splits the old "S3d hygiene+ops" charter: **S3d = ops centre only**; the hygiene + triple-post reconciliation (dead `content-history.md` retirement + post-tracking↔published↔analytics) moves to a new **SB-S3e** (the genuine LAST S3 slice).
|
||||
> **Status:** DRAFT (brief + plan written; light-Voyage hardening + go-before-code gate pending).
|
||||
> **Predecessors:** SB-S0 (scaffold seeds the bare `operations.md`) · SB-S3a (the profile.md reader pattern this mirrors) · SB-S3b/c (supersede + cross-silo graph — independent of this).
|
||||
|
||||
## 1. Operator decision (2026-06-23)
|
||||
|
||||
The S3d charter bundled three deliverables — (a) flesh out `operations.md`, (b) retire the dead `content-history.md`, (c) triple-post reconciliation. Grounding showed a clean fault line: **(a) is additive / new / self-contained** (touches only `scripts/brain/` + one reader agent, never the post silos), while **(b)+(c) are coupled** (the content-history back-fill question *is* the reconciliation question) and both touch the loose regex-mutated state silo S3c deliberately avoided. The operator chose **Split — ops centre first**: S3d ships (a) only; (b)+(c) become **SB-S3e**, the focused finale. This keeps S3d low-risk and isolates the silo surgery.
|
||||
|
||||
## 2. Why the ops centre, and why now
|
||||
|
||||
`architecture.md:9` frames the second brain as **"memory AND an operations centre."** S0→S3c delivered the *memory* half (profile, ingest, consolidation loop, cross-silo graph). The *operations centre* is the genuinely-new half (`architecture.md:44`: "the operations/plans centre is genuinely new"), and today it is **scaffold-only**: `operationsSeed()` (`scaffold.ts:52-64`) writes three bare headers — `## Who I am now (anchor)`, `## Plans`, `## Ideas` — that **no agent or command reads**. Verified: 0 references to `operations.md` in `consolidate.ts`/`assemble.ts`/`ingest.ts`/`cli.ts` and 0 in `agents/`.
|
||||
|
||||
The architecture's load-bearing spec is the **frozen-past-self guard** (`architecture.md:59`): *"`operations.md` holds a periodic user-authored 'where I'm headed now' anchor that deprecates older inferences."* This is the human override on the machine's anti-drift mechanisms (evidence-threshold / decay-flag / anti-sycophancy): where those keep the machine honest, the dated anchor lets the user **declare current direction that outranks stale inferences**. S3d makes that real — a reader honours the anchor.
|
||||
|
||||
It is the right next slice because it is **additive / non-breaking**, **read-only into generation** (no engine or silo write), and the natural successor to S3a (same "wire the first reader" shape), now extended with the genuinely-new *deprecation* semantics.
|
||||
|
||||
## 3. Scope — what is IN (S3d)
|
||||
|
||||
1. **Enrich the `operations.md` seed so the frozen-past-self guard is operational.** `operationsSeed()` (`scaffold.ts:52-64`) currently ships bare headers. The **genuinely-new seed content is the dated-anchor convention** `_As of YYYY-MM-DD:_` under the anchor (verified ABSENT from the current seed — `grep "As of" scaffold.ts` = 0) plus light item-format guidance for Plans/Ideas. Rationale: the guard "deprecates *older* inferences" — that is inherently temporal, so the anchor must carry a date the reader can compare against a profile fact's `last_seen`. **NB:** the phrase `deprecates older inferences` is NOT new to the seed — it already ships in the seed's guard comment (`scaffold.ts:58`) — so the seed's new-coverage test must key on the **date convention**, not that phrase (else it passes pre-enrichment; see SC3). Idempotency is **existence-skip**: `initBrain` writes a seed file only when it is absent (`scaffold.ts:105-113`), so an existing `operations.md` is never re-touched and a user edit is structurally safe regardless of seed content.
|
||||
- *Fork (open question for the go-gate):* the lighter alternative is **reader-only, no seed change** (Option A) — wire the reader against the existing bare seed. Recommended: **Option B (enrich)**, because an undated anchor makes the guard vague. See §8.
|
||||
|
||||
2. **Wire `strategy-advisor` to read `brain/operations.md`.** Add the file to the agent's existing Step 0 Load-Context block (`strategy-advisor.md:41-48`) and a short consumption subsection immediately after the profile subsection (`:63`). The agent reads the markdown in-context (no parser — Claude is the retrieval engine, `architecture.md:14`; consistent with S3a).
|
||||
|
||||
3. **Consumption contract — the frozen-past-self guard (advisory / reader-side).**
|
||||
- The **anchor** (`## Who I am now`) is the user's own **dated declaration of current direction** — it is **authoritative**, and it **deprecates older inferences**: when a `brain/profile.md` Dynamic fact predates the anchor's date or contradicts its stated direction, **prefer the anchor and flag the older fact as possibly-stale** — do not parrot it.
|
||||
- **Anti-sycophancy INVERSION (the genuinely-new nuance):** profile facts are *evidence to TEST* (S3a — challenge them); the **anchor is the opposite** — it is user-declared direction, not an inference to challenge, so it is honoured rather than counter-pressured. (It is direction, not praise — honouring it never licenses flattery.)
|
||||
- **Plans** = active commitments (current intent). **Ideas** = parking lot (suggestions, not commitments). Weigh accordingly.
|
||||
- Durable sentinel literal for the lint: **`deprecates older inferences`** (verbatim `architecture.md:59`).
|
||||
|
||||
4. **Graceful absence.** `brain/operations.md` (or an empty anchor) → the agent **degrades silently** — no error, no "missing ops centre" noise; it proceeds on its other context sources. (Fresh installs have no brain until `brain init`.)
|
||||
|
||||
5. **Deterministic wiring test** — a new **Section 16e** in the `test-runner.sh` gate, modelled byte-for-byte on Section 16d (SB-S3a): two UNCONDITIONAL checks — (a) a non-vacuity self-test (full-wiring probe detected; 3 under-wired probes — incl. a sibling-file `brain/profile.md` decoy — rejected); (b) a real-file grep asserting `strategy-advisor.md` carries BOTH literals `brain/operations.md` and `deprecates older inferences`. `ASSERT_BASELINE_FLOOR` 80→**82** (+2, lockstep). If Option B: a new `scaffold.test.ts` case asserts the dated-anchor seed convention → `BRAIN_TESTS_FLOOR` 113→**114**.
|
||||
|
||||
6. **Doc/count reconciliation.** Update `architecture.md:80` (the SB-S3 build-row) to reflect the split: S3d ✅ ops centre · **S3e remains** (content-history retirement / triple-post reconciliation). Update STATE telling (gate 95→97, ASSERT floor 80→82, brain 113→114). No CLAUDE.md count change (no new agent/command/reference doc; `strategy-advisor` model/frontmatter unchanged).
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **The hygiene + reconciliation half (b+c)** — dead `content-history.md` retirement and post-tracking↔published↔analytics reconciliation. This is **SB-S3e** (the new last slice). S3d touches **no** post silo, **no** `state-updater.mjs`, **no** `content-history` surface.
|
||||
- **Code-enforced deprecation.** S3d's guard is **advisory (reader-side)**: the agent prefers the dated anchor. It does NOT add a seam that makes `consolidate.ts` read the anchor and down-weight/flag older profile facts during `--propose`. That engine change is heavier (touches the sole writer of `profile.md`) and is an explicit **follow-up**, not S3d.
|
||||
- **A second reader.** `content-planner` is the obvious next consumer (plans/ideas feed planning); S3d wires **exactly one** agent (`strategy-advisor`, which already reads `profile.md`) — the smallest valuable proof. Follow-on.
|
||||
- **A parser / new `.mjs` for operations.md.** It is user-authored prose consumed in-context; it needs **no TS grammar** and no twin `.mjs` parser. (The only TS touched is the seed string in `scaffold.ts` + its test.)
|
||||
- **Any WRITE path to `operations.md` from the motor.** It stays user-authored; the scaffold's existence-skip is the only code that touches the file, and only to create it once (never re-touched once present).
|
||||
- **A "Maskinrommet cockpit" surface.** `architecture.md:4` reserves the day-to-day cockpit for a future thin layer that reads/writes *through* the store. S3d ships the store substrate (a read tributary), not an end-user cockpit.
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **READ-only into generation** — S3d adds no write path to any brain file; `brain consolidate --apply --confirm` stays the sole writer of `profile.md`, and `operations.md` stays user-authored.
|
||||
- **No post-silo contact** — zero edits to `state-updater.mjs`, `## Recent Posts`, `content-history`, analytics, or any tributary JSON. (That is S3e.)
|
||||
- **No tributary schema change** — operations.md is a hub file, not a tributary schema; no `ingest.ts`/`assemble.ts`/`consolidate.ts` change.
|
||||
- **Idempotent + no-clobber** — the enriched seed must preserve `initBrain`'s existence-skip (a seed is written only if the file is absent, `scaffold.ts:105-113`); a user-edited `operations.md` is never overwritten (`scaffold.test.ts:70-77` must stay green).
|
||||
- **Fresh-clone safe** — missing brain → the reader degrades silently; the seed change cannot crash a fresh init.
|
||||
- **Anti-sycophancy preserved** — the profile reader's `evidence to TEST` stance (S3a / Section 16d) is untouched; the anchor's *authoritative* stance is an addition for a different file, never a weakening of the profile stance.
|
||||
- **TDD iron law** — the failing Section-16e check (and, Option B, the failing scaffold seed assert) lands BEFORE the agent/seed edit.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1 — wired:** `strategy-advisor.md` Step 0 context-load includes `brain/operations.md`. *(lint, deterministic — Section 16e Check B)*
|
||||
- **SC2 — guard framed:** the operations-consumption subsection carries the exact durable literal **`deprecates older inferences`**, asserted by an exact-literal grep + a non-vacuity self-test with a sibling-file decoy (repo idiom, Sections 13/16c/16d). *(lint)*
|
||||
- **SC3 — seed operational (Option B):** `operationsSeed()` ships the **dated-anchor convention** `_As of YYYY-MM-DD:_`; a NEW `scaffold.test.ts` case asserts that exact convention (the RED-bearing literal — verified absent from the current seed, so it fails first; the `deprecates older inferences` phrase already ships and is a non-RED companion only). `BRAIN_TESTS_FLOOR` 113→114 (verified current reported `tests 113`, so new floor = recorded + 1). **RED proof recorded:** run `(cd scripts/brain && npm test)` against the un-enriched seed and confirm the new case fails BEFORE the seed edit. *(brain suite)*
|
||||
- **SC4 — gate green:** `scripts/test-runner.sh` stays green with Section 16e; TOTAL **95→97** (verified current 95; +2 unconditional checks; recount live at land rather than assume). The hook suite (separate runner) untouched (no hook path changes).
|
||||
- **SC5 — assertion floor honoured:** `ASSERT_BASELINE_FLOOR` bumped by exactly the new unconditional checks (+2 → 82); the self-count check passes.
|
||||
- **SC6 — graceful absence (manual/behavioural):** `strategy-advisor` with no `brain/operations.md` produces normal output, no missing-file noise.
|
||||
- **SC7 — guard works (manual/behavioural):** `strategy-advisor` with a populated dated anchor that contradicts an older profile fact **prefers the anchor and flags the older fact as deprecated/stale**, not parroted — the frozen-past-self override observably fires. **Honesty hedge (verifiseringsplikt):** if a per-call data-root cannot be exercised, this is recorded as **wiring-inspected-only, NOT a behavioural pass** — never record a guard-fires pass that was not actually run.
|
||||
|
||||
## 7. Verification
|
||||
|
||||
- **Deterministic (gate):** SC1/SC2/SC4/SC5 via Section 16e in `test-runner.sh`; SC3 via the brain suite (`cd scripts/brain && npm install` before the gate, else it warn-skips — STATE rule). No hook path changes → hook suite untouched.
|
||||
- **Behavioural (manual, documented — the honest limit):** SC6 + SC7 — agent-prompt behaviour is not unit-testable (command-testing workstream still open). The plan includes a documented manual run: (a) empty-brain → clean output; (b) seed a temp `operations.md` with a dated anchor that contradicts a hand-written profile fact → confirm the anchor wins and the older fact is flagged. Record the result in STATE/changelog at land; never overclaim a behavioural pass not run (verifiseringsplikt).
|
||||
|
||||
## 8. Open questions for brief-review / the go-gate
|
||||
|
||||
1. **Seed enrichment (Option B) vs reader-only (Option A).** Recommended **B** — a dated-anchor convention makes the temporal guard operational; A leaves the anchor undated and the guard vague. B costs one seed function + one brain test (`BRAIN_TESTS_FLOOR` +1). Confirm at the gate.
|
||||
2. **Advisory vs code-enforced deprecation.** Recommended **advisory (reader-side)** for S3d — keeps it additive, no engine change. Code-enforcement (consolidate.ts reads the anchor date to flag older profile facts) is a flagged follow-up. Confirm the deferral.
|
||||
3. **One reader (`strategy-advisor`) for S3d.** Confirm; `content-planner` as second reader is follow-on.
|
||||
4. **Doc home for the contract.** Recommended: update `architecture.md:80` (the build-row, now partly false post-split) + let the brief/plan be the contract record. Confirm we do NOT touch `consolidation-loop.md` (that doc is the profile *motor* contract; operations.md is not in that loop).
|
||||
|
||||
## 9. Light-Voyage review — folded
|
||||
|
||||
Three reviewers ran on the brief + plan (2026-06-23):
|
||||
|
||||
- **`voyage:scope-guardian`: ALIGNED** — 0 creep, 0 gaps, 0 dependency issues. Every IN item maps to a plan step; no OUT-list file is touched (the only mentions of post silos / `consolidate.ts` / `content-history` in the plan are the scope-fence list + the `architecture.md:80` doc string that assigns them to S3e). Doc reconciliation stays minimal. Proportional.
|
||||
- **`voyage:brief-reviewer`: REVISE** → 5 [FIX]es folded:
|
||||
- **FIX-1** (load-bearing): the sentinel `deprecates older inferences` **already ships in the seed** (`scaffold.ts:58`) and at `architecture.md:59` — so a seed test keyed on that phrase passes GREEN pre-enrichment, collapsing the TDD RED gate. Folded: SC3 + §3.1 now pin the new seed assertion to the **dated-anchor convention** `_As of YYYY-MM-DD:_` (verified absent: `grep "As of" scaffold.ts` = 0). The agent-file Check B still greps the phrase (correct there — the agent file does NOT yet carry it). ✅
|
||||
- **FIX-2:** SC3 now requires a **recorded RED proof** (run the brain suite against the un-enriched seed, confirm the new case fails first). ✅
|
||||
- **FIX-3:** "compare-then-skip" was a mischaracterization — `scaffold.ts:105-113` is **existence-skip** (writes only if absent; no content compare). Corrected in §3.1/§4/§5. ✅
|
||||
- **FIX-4:** SC7 now carries the honesty hedge (wiring-inspected-only unless a per-call data-root is exercised). ✅
|
||||
- **FIX-5:** gate TOTAL treated as "expected +2, recount live"; **verified live: 95** → 97. ✅
|
||||
- Floor arithmetic confirmed correct by the reviewer: `ASSERT` 80→82, `BRAIN` 113→114.
|
||||
- **`voyage:plan-critic`: 3 major + 3 minor** (folded into the plan, see `plan-sb-s3d.md` §"Plan-critic — folded"): the brain-test RED literal (= FIX-1), the **recorded** brain floor (verified `tests 113` → 114), `grep -qF` fixed-string pinning, header-enumeration range `:33-37`, `operationsSeed()` cite incl. braces, and the load-bearing retained `##` anchors for `scaffold.test.ts:48-58`.
|
||||
|
||||
**Net:** scope unchanged; the folds harden the TDD RED guarantee (the one real hole) + correct two prose mischaracterizations. Brief-reviewer's residual verdict after folds: structurally sound, well-scoped, honest.
|
||||
113
docs/second-brain/brief-sb-s3e.md
Normal file
113
docs/second-brain/brief-sb-s3e.md
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
# Brief — SB-S3e: hygiene + triple-post reconciliation (the LAST S3 slice)
|
||||
|
||||
> **Slice:** SB-S3e — the focused finale of SB-S3. Carries the two coupled deliverables the S3d split deferred: **(b) retire the dead `content-history.md`** + **(c) triple-post reconciliation** (silo 1 post-tracking ↔ silo 2 brain published-records ↔ silo 3 analytics). After this slice the second-brain arc is complete (S4 EØS DMA-connector is optional).
|
||||
> **Status:** DRAFT (brief + plan written; light-Voyage hardening + go-before-code gate pending).
|
||||
> **Predecessors:** SB-S0 (scaffold seeds the bare tree) · SB-S1 (`ingest/published/` gold-signal records = silo 2) · SB-S3c (`assemble.ts` built the silo 2↔3 join; deliberately did NOT touch silo 1) · SB-S3d (the ops-centre reader — independent).
|
||||
|
||||
## 1. Operator decision context (2026-06-23)
|
||||
|
||||
The original S3d charter bundled three deliverables; the S3d brief (§1) found a clean fault line and **split**: (a) the ops centre shipped as S3d (additive, no silo contact); **(b) + (c) are coupled and move here, to S3e**. They are coupled because **the content-history back-fill question *is* the reconciliation question**: `content-history.md` was a *second* auto-written record of created posts, so deciding its fate (retire vs resurrect) and deciding how the auto-tracked creation stream reconciles to the brain graph are the same decision. Both touch the loose, regex-mutated post-tracking silo that S3c deliberately avoided. S3e is the deliberate, isolated finale for that surgery.
|
||||
|
||||
## 2. The gap — grounded in code
|
||||
|
||||
A "post created via the plugin" leaves **three** separate records, and they do not agree:
|
||||
|
||||
| # | Silo | Path / writer | Carries | Join key? |
|
||||
|---|------|---------------|---------|-----------|
|
||||
| **1** | **post-tracking** | `## Recent Posts` in `$HOME/.claude/linkedin-studio.local.md` — auto-written by the Stop-hook → `updatePostTracking` (`state-updater.mjs:114-126`) | `[date] "hook (≤60c preview)" (charCount) - topic` | **none** |
|
||||
| **2** | **brain published-record** | `ingest/published/<contentId>.md` under the brain dataRoot — written **only** by manual `brain ingest --file …` (`cli.ts:122-138`) | `contentId`, `published_date`, `specifics[]`, `trends[]`, full `body` | `contentId` (internal) |
|
||||
| **3** | **analytics** | `analytics/posts/*.json` under the brain dataRoot — from the LinkedIn CSV import | `title`, `publishedDate`, `metrics` | none (no body, no URN) |
|
||||
|
||||
**S3c built the silo 2↔3 join** (`assemble.ts: assemblePostGraph`) by a confidence-tiered title-prefix + date heuristic, answering the arc's north-star query *"which raw material actually performs?"* (`specifics → post → measured analytics`).
|
||||
|
||||
**Two gaps remain, and they are the same coin:**
|
||||
|
||||
- **Silo 1 is joined to nothing.** The complete *auto-captured* stream of created posts (silo 1) never meets the graph. The graph (silo 2→3) sees **only** the subset the user *manually* ran `brain ingest` on — so the "which raw material performs" payoff is **blind to every post the user created but never hand-ingested.** Verified: silo 2 has **no auto-capture** — `assemble`'s empty-state message literally instructs the user to `brain ingest --file …` manually (`cli.ts:182`); STATE lists auto-capture as the explicit *optional* S3c-bonus, i.e. it does not exist.
|
||||
- **The dead `content-history.md`** is a *fourth*, genuinely-orphaned record (date/hook/topic/format/words/chars/source), written by flaky Stop-hook prose (`state-update-reminder.md:67-86`) and read by **nobody** (verified: the only `content-history` mentions in code are a `migrate-data.mjs` MOVE entry, its test, the `.gitignore`/SC2 data-class guards, and a `session-start.mjs` *comment* — zero readers). It is redundant with silo 1 (both auto-written, both unread by the graph) and resurrecting it would solve nothing the join needs. **Honesty nuance (the prose lies):** the writer prose being deleted *claims* the log "enables `/linkedin:report` and `analytics-interpreter` to track content production" (`state-update-reminder.md:85`) — but that was **never wired** (grep-verified: no command/agent reads the file; the `profile.md`/`analyze.md`/`audit.md` "content history" hits are the *conceptual* "have you posted on this topic" signal, not file reads). The claim is aspirational dead prose, which *strengthens* retirement — but it must be named so a mid-deletion reader doesn't chase a phantom reader.
|
||||
|
||||
## 3. Scope — what is IN (S3e)
|
||||
|
||||
### (b) Retire the dead `content-history.md` — clean deletion across its plumbing
|
||||
|
||||
Verified surfaces (grounded this session):
|
||||
|
||||
1. **Writer prose** — remove Section 5 "Content History Log" from `hooks/prompts/state-update-reminder.md:67-86` (the flaky generator). This is the only thing that ever wrote the file.
|
||||
2. **Template** — delete `config/content-history.template.md`.
|
||||
3. **Migration** — remove the B1 entry from `MOVE_FILES` in `hooks/scripts/migrate-data.mjs:33` (`['assets/analytics/content-history.md', 'analytics/content-history.md']`).
|
||||
4. **Migration test** — drop the content-history fixture write (`__tests__/migrate-data.test.mjs:37`), its two assertions (`:56`, `:61`), and trim the test-(a) description (`:50`). Test **count** unchanged (5 cases a–e stay) → hook-suite total unchanged.
|
||||
5. **`.gitignore:44`** — remove `assets/analytics/content-history.md`.
|
||||
6. **Gate guard** — remove the `assets/analytics/content-history\.md` alternative from `SC2_CLASSES` (`test-runner.sh:592`). The SC2 check stays (one pass/fail) → **gate total unchanged**.
|
||||
7. **Reference doc** — remove `content-history.md` from the data-tree diagram (`references/data-path-convention.md:26`).
|
||||
8. **Comment** — drop `content-history` from the `session-start.mjs:94` comment list.
|
||||
|
||||
**NOT in scope of (b):** `state-updater.mjs`'s `pruneContentHistory` (despite the name, it operates on the **live `## Recent Posts`** in the state file — `:138-172` — and is untouched), and any *conceptual* "content history" prose (e.g. `README:120` `/linkedin:profile` "content-history alignment" reads as the concept, not the file — verify-then-leave; never delete a conceptual mention).
|
||||
|
||||
### (c) Triple-post reconciliation — read-side joiner (recommended fork)
|
||||
|
||||
Extend the brain's read-only assembler so silo 1 is reconciled against the graph, **without writing the state file** (faithful to S3c's "read, never write").
|
||||
|
||||
1. **A new pure reconcile core** (`scripts/brain/src/reconcile.ts`): `reconcileRecentPosts({ recentPosts, records, graph })` — takes the parsed silo-1 entries + the **full `PublishedRecord[]`** (which carry `body`) + the existing `PostGraphNode[]`, and returns, per silo-1 entry, whether it is present in the brain (a matched published record) and/or analytics, with an explicit confidence tier — surfacing the **coverage gap**: created posts that never reached the brain graph. **Why `records` and not graph-only:** `PostGraphNode` carries `contentId` + analytics match but **no `body`** (`assemble.ts:48-54`), so the hook→post prefix-match needs the `PublishedRecord.body`; the core matches `hook → record.body`, then `record.id → graph node` for the analytics tier. No FS/clock/network in the core (the `assemble.ts` discipline).
|
||||
2. **A thin read-only state-file parser** (`parseRecentPosts(stateText)`): extract `[date] "hook" (chars) - topic` rows from the `## Recent Posts` section. The format source of truth is the **writer** `updatePostTracking` (`state-updater.mjs:116`: `- [date] "hook" (chars) - topic`) — **not** `pruneContentHistory`'s regex, which is **date-only** (`:145`, `/^- \[(\d{4}-\d{2}-\d{2})\] .+$/gm`) and captures no hook/chars/topic. `parseRecentPosts` is the *first* full-field reader of this section; it shares only the date-anchor shape with the pruner. A golden-string test pins a writer-produced entry round-tripping through `parseRecentPosts`.
|
||||
3. **A new brain IO seam for the state file.** Silo 1 lives at `STATE_FILE || $HOME/.claude/linkedin-studio.local.md` — **outside** the brain dataRoot (`LINKEDIN_STUDIO_DATA`). A new `loadRecentPosts()` IO resolves that path with the canonical precedence already exported as `data-root.mjs: getStateFile()` — `STATE_FILE` first, else `join(resolveHome(), '.claude', 'linkedin-studio.local.md')` where `resolveHome() = HOME || USERPROFILE || homedir()` (NOT bare `homedir()` — that would ignore a `HOME`/`USERPROFILE` override the writer honours). Reads read-only; absent → `[]` (fresh-clone safe). This is a **genuinely new root-skew caveat** (the state file resolves via `STATE_FILE`, a *different* root than the brain dataRoot — not a mirror of `assemble.ts`'s `ANALYTICS_ROOT` note, which is about the analytics *dir* under `LINKEDIN_STUDIO_DATA`). (A hook-side `.mjs` reconciler is **pathguard-blocked** — new `.mjs` under `hooks/scripts/` is forbidden — so the brain TS side is the only clean home; it copies `getStateFile()`'s chain since it cannot import the `.mjs`.)
|
||||
4. **A CLI subcommand** `brain reconcile` (`cli.ts`): loads silo 1 + the assembled graph, prints the per-post coverage report (in brain / in analytics / orphaned-in-state), and a one-line summary ("N created posts, M in the brain graph, K un-ingested → `brain ingest` them to feed the graph"). Read-only.
|
||||
5. **Deterministic gate section** (new Section, modelled on 16e/16d): unconditional checks that the reconcile wiring exists (CLI subcommand registered; core exported). `ASSERT_BASELINE_FLOOR` bumped by exactly the new unconditional checks.
|
||||
6. **Brain-suite tests** (RED first): `reconcile.test.ts` — coverage tiers (present/absent/orphaned), the truncated-hook prefix limit, empty-state degradation. `BRAIN_TESTS_FLOOR` bumped by the new passing count (recorded live, not assumed).
|
||||
|
||||
**Honest limits of read-side (documented, not hidden):** (i) silo 1 carries only a **≤60-char truncated hook preview**, a weaker prefix signal than silo 2's full body → lower match confidence, reported honestly as a tier, never as proof. (ii) Read-side **cannot retroactively assign** the `specifics`/`trends` ids a post was built from — those were never captured for un-ingested posts; the join can only say "this created post is missing from the graph," not reconstruct its raw material. (iii) The true gap-closer is **auto-capture at drafting** (silo-2 record minted with specifics/trends at creation) — explicitly a **follow-up, not S3e** (STATE's S3c-bonus). S3e ships the *honest read-side reconciliation + the coverage signal that motivates auto-capture next.*
|
||||
|
||||
## 4. Non-goals — what is OUT (deferred)
|
||||
|
||||
- **Write-side stamping** — embedding a `contentId` into the `## Recent Posts` entry. Rejected for S3e: it requires a `contentId` to *exist* at post-creation, i.e. **auto-capture** (out of scope), and it mutates the `$`-injection-defended regex silo (`updatePostTracking` already had to use replacement *functions* to stay `$`-safe — `state-updater.mjs:117-125`). Heavier + riskier + premature. See §8 fork 2.
|
||||
- **Auto-capture at drafting** (the real gap-closer) — minting a silo-2 published record with specifics/trends ids at post-creation. The motivated follow-up; S3e's coverage report is what makes its value legible.
|
||||
- **Resurrecting `content-history.md`** as a live read tributary (the back-fill arm of fork 1) — it would add a second weak auto-record the graph still can't join. Rejected; retirement chosen.
|
||||
- **Any change to `pruneContentHistory` / `## Recent Posts` writer / `state-updater.mjs`** — S3e reads silo 1, never writes it.
|
||||
- **A second reconcile consumer** (a command/agent that *acts* on the coverage report) — S3e ships the `brain reconcile` view; wiring an agent to nudge "ingest these" is follow-on.
|
||||
- **`assemble.ts` join-heuristic changes** — reconcile *consumes* the existing graph; it does not re-tune the silo 2↔3 matcher.
|
||||
|
||||
## 5. Boundaries / invariants (must hold)
|
||||
|
||||
- **READ-only into the state silo** — S3e adds no write path to `## Recent Posts` or any state-file field; `state-updater.mjs` stays the sole writer of silo 1.
|
||||
- **READ-only into generation** — `brain consolidate --apply --confirm` stays the sole writer of `profile.md`; reconcile writes nothing.
|
||||
- **Retirement is plumbing-only** — (b) removes a dead file + its writer/migration/guards; it must not touch the live `## Recent Posts`, `pruneContentHistory`, or any conceptual "content history" prose.
|
||||
- **No new hook `.mjs`** — pathguard forbids new `.mjs` under `hooks/scripts/`; reconcile lives in `scripts/brain/` (TS). Editing the existing `migrate-data.mjs` (a deletion) is an Edit of an existing file → allowed.
|
||||
- **State-file root precedence mirrored exactly** — `loadRecentPosts()` resolves `STATE_FILE || $HOME/.claude/linkedin-studio.local.md`, identical to the `.mjs` side; a divergence would silently read the wrong file.
|
||||
- **Fresh-clone / empty-state safe** — missing state file or empty `## Recent Posts` → reconcile degrades to a clean empty report, never a crash (mirrors `loadAnalyticsRows`).
|
||||
- **Migration idempotency intact** — removing the B1 MOVE entry must not break `migrate-data`'s `.migrated`-marker idempotency or the COPY/MOVE classes; the remaining test cases (b–e) stay green.
|
||||
- **TDD iron law** — the failing reconcile test (and the failing gate-section self-test) land BEFORE the reconcile code.
|
||||
|
||||
## 6. Success criteria (testable)
|
||||
|
||||
- **SC1 — content-history retired:** `grep -rn "content-history" hooks/ config/ scripts/test-runner.sh .gitignore references/data-path-convention.md` (NO `--include` filter — it must reach `.gitignore` and the `.md` ref-doc, which an extension-filter would silently skip) returns **zero** (file, writer, MOVE entry, gitignore line, SC2 alternative, ref-doc, comment all gone); the template file is deleted. The README `:120` *conceptual* mention is intentionally outside this grep set (concept, not file). *(grep + `ls` assertion — identical command in the plan, SC1)*
|
||||
- **SC2 — migration green after retirement:** `(cd … && node --test hooks/scripts/__tests__/migrate-data.test.mjs)` passes with cases a–e; no content-history assertion remains. *(hook suite)*
|
||||
- **SC3 — reconcile core correct (RED-first, LOGIC-RED not import-RED):** `reconcile.test.ts` asserts the three coverage tiers (in-graph / in-brain-only / orphaned-in-state), truncated-hook degradation, and empty-state → empty report. **RED proof recorded:** the artifact must show the **named tier assertions failing** (e.g. "expected `orphaned-in-state` for the un-ingested entry") against a **stub `reconcile.ts` that compiles but mis-classifies** (returns `[]` / wrong tiers) — a bare `MODULE_NOT_FOUND` import-RED proves nothing about the assertion logic and is rejected as vacuous. `BRAIN_TESTS_FLOOR` bumped to the recorded new count. *(brain suite)*
|
||||
- **SC4 — reconcile wired:** `brain reconcile` runs end-to-end on a seeded fixture (silo 1 + published + analytics) and prints the coverage report + summary; a new gate Section asserts the CLI subcommand + core export exist (unconditional self-test + real-file grep). *(lint + manual run)*
|
||||
- **SC5 — gate green + floor honoured:** `scripts/test-runner.sh` stays green; TOTAL recounted **live** at land (current 97; + the new unconditional checks). `ASSERT_BASELINE_FLOOR` bumped by exactly that many; SC2-dry-run still passes with the narrowed `SC2_CLASSES`.
|
||||
- **SC6 — root-skew (manual/behavioural — EXPECTED to land as a real pass):** with `STATE_FILE` pointing at a temp state file, `brain reconcile` reads silo 1 from it; with it unset it falls back to `$HOME/.claude/linkedin-studio.local.md`. This is a trivial `STATE_FILE` env toggle in a temp dir — exercisable in the same shell that runs the gate — so the **default expected landing state is a real behavioural pass**, not a hedge. **Fallback (verifiseringsplikt):** only if the env toggle genuinely cannot run, record wiring-inspected-only, NOT a behavioural pass — the hedge must never become a standing excuse to skip a runnable check.
|
||||
- **SC7 — counts reconciled:** `architecture.md` S3 build-row marked S3e ✅ (arc complete); STATE telling updated (gate total, ASSERT floor, brain floor); no CLAUDE.md command/agent/reference-doc count change (no new command/agent; `reconcile.ts` is a brain module + CLI subcommand, not a plugin surface — confirm at the gate whether `brain reconcile` warrants a one-line CLAUDE.md mention).
|
||||
|
||||
## 7. Verification
|
||||
|
||||
- **Deterministic (gate):** SC1/SC4/SC5 via grep + the new gate Section in `test-runner.sh`; SC2 via the migrate-data hook test; SC3 via the brain suite (`cd scripts/brain && npm install` before the gate, else warn-skip — STATE rule).
|
||||
- **Behavioural (manual, documented — the honest limit):** SC4 end-to-end run + SC6 root-skew — recorded in STATE/changelog at land; never overclaim a behavioural pass not run (verifiseringsplikt).
|
||||
|
||||
## 8. Open questions for brief-review / the go-gate
|
||||
|
||||
1. **Fork 1 — content-history: retire vs back-fill.** Recommended **retire** — zero readers, redundant with silo 1, and back-fill resurrects a second weak auto-record the graph still cannot join. Confirm.
|
||||
2. **Fork 2 — reconciliation: read-side joiner vs write-side stamp.** Recommended **read-side** — faithful to S3c's read-only invariant, avoids the `$`-injection regex silo, and write-side is blocked on auto-capture (no `contentId` exists at creation today) so it would be premature. Cost honestly stated: a new cross-seam read (brain TS → state file). Confirm; write-side + auto-capture become the flagged follow-up.
|
||||
3. **Scope of (c): build the read-side joiner now, or document-the-design + defer the build?** Recommended **build now** — the coverage report is the genuine deliverable that closes the arc and motivates auto-capture; a doc-only finale under-delivers the last slice. Confirm the build.
|
||||
4. **CLAUDE.md surface for `brain reconcile`.** The brain CLI is an internal tool, not a plugin command — recommended: a one-line architecture note, no command-count change. Confirm.
|
||||
|
||||
## 9. Light-Voyage review — folded
|
||||
|
||||
Three reviewers ran on the brief + plan (2026-06-23):
|
||||
|
||||
- **`voyage:scope-guardian`: ALIGNED** — 0 creep, 0 gaps, 0 dependency issues. All 14 IN items (b1–b8, c1–c6) map to a plan step; no NON-GOAL file is touched (the scope fence is consistent with the steps, and slightly broader/defensive); doc reconciliation is proportional (architecture build-row + STATE telling only). Every referenced file/line/symbol verified against the live codebase.
|
||||
- **`voyage:brief-reviewer`: REVISE** → 4 [FIX]es folded:
|
||||
- **FIX-1** (load-bearing): the parser must mirror the **writer** `updatePostTracking` (`state-updater.mjs:116`), NOT `pruneContentHistory` — whose `:145` regex is **date-only** and captures no hook/chars/topic. Folded into §3(c).2 (+ a golden-string round-trip test). The "they never disagree" claim was rebuilt on the writer-format source of truth. ✅
|
||||
- **FIX-2** (honesty): the writer prose being deleted *claims* `/linkedin:report` + `analytics-interpreter` as readers, but they were never wired — aspirational dead prose. Named in §2 so the "zero readers" verdict is auditable, not asserted. ✅
|
||||
- **FIX-3** (testability): SC3 RED tightened to a **logic-RED** (named tier assertions fail against a mis-classifying stub), rejecting a vacuous import-RED. ✅
|
||||
- **FIX-4** (testability + consistency): SC6 pre-committed as an **expected real behavioural pass** (trivial env toggle), wiring-only as documented fallback; and the root-skew caveat is **genuinely new** (state file via `STATE_FILE`), not a mirror of `assemble.ts`'s `ANALYTICS_ROOT` note — cite `data-root.mjs: getStateFile()` as the canonical precedence. Folded into §3(c).3 + SC6. ✅
|
||||
- Also folded: the **core signature** corrected to `{ recentPosts, records, graph }` (graph-only cannot prefix-match — `PostGraphNode` has no `body`), and **SC1's grep** unified with the plan's (no `--include` filter, which would skip `.gitignore`/`.md`).
|
||||
- **`voyage:plan-critic`: REVISE (3 blockers + 6 major + 4 minor)** — folded into `plan-sb-s3e.md` §"Plan-critic — folded". The three blockers were real factual errors against the code, each verified directly before folding: (B1) `listPublished()` is body-less → use `runAssemble`'s inline `parsePublishedRecord` loader (`cli.ts:162-176`, which already builds a `bodyById` Map); (B2) the core signature contradiction (= the brief's signature fold); (B3) the `:145`-regex misattribution (= FIX-1). Majors: the `getStateFile()` HOME-chain (not bare `homedir()`), single-literal gate decoys, Check-B literal-name constraint, recorded gate TOTAL (verified live **97**), demonstrated migrate idempotency, and the SC1 grep unification.
|
||||
|
||||
**Net:** scope unchanged (ALIGNED); the folds correct three load-bearing factual errors in the reconcile design (body-less loader, wrong regex, wrong HOME seam) + harden the RED guarantee and the gate's non-vacuity. The retirement arm (b) was already nearly executable; the reconcile arm (c) is now grounded in the actual `assemble.ts`/`ingest.ts`/`cli.ts` shapes.
|
||||
100
docs/second-brain/brief.md
Normal file
100
docs/second-brain/brief.md
Normal file
|
|
@ -0,0 +1,100 @@
|
|||
# Task Brief — Second Brain (compounding per-user knowledge engine)
|
||||
|
||||
> **Status:** revised after adversarial brief-review (light-Voyage step 2 → findings folded in). Architecture approved by operator 2026-06-23 (`architecture.md`).
|
||||
> **Inputs:** `architecture.md` (approved design) · `research/{connector-egress,secondbrain-sota,silo-inventory}.md`.
|
||||
> **Build target of this brief:** the arc objective + scope fence, and the **first slice SB-S0** in detail. SB-S1..S4 are scoped one-line; their detailed plans come later, one per slice.
|
||||
> **Review delta (2026-06-23):** B1 fold-source rewritten (template-as-field-source + source-absent no-op); B2 pathguard claim corrected (discrepancy flagged); M1 floors as `≥`; M2 scaffold trigger decided (idempotent CLI/TS subcommand, session-start wiring deferred to SB-S2); M3 profile grammar contract pinned; M4 `brain/` is runtime-only (no committed seed); minors m1–m4 folded.
|
||||
|
||||
---
|
||||
|
||||
## 1. Objective
|
||||
|
||||
Turn the plugin's existing siloed per-user stores into **one compounding "second brain"**: a per-user knowledge engine that accumulates everything about a creator (posts, articles, newsletters, plans, ideas), keeps a two-layer profile that improves over time, and stays aligned to who the user *is now* — drift-resistant and collapse-resistant. It is **memory AND an operations centre**.
|
||||
|
||||
The engine is a **plugin capability** (domain-general, shareable). The user's data lives in the per-user data dir. The user's personal cockpit lives in Maskinrommet (out of scope here).
|
||||
|
||||
## 2. Background (why this shape)
|
||||
|
||||
- **Nothing to migrate** — M0 already routed the **data-root silos** (trends, specifics, analytics, drafts/queue, plans, voice, profile, scaffolds) through one tested seam (`getDataRoot`), all surviving reinstall. The **state file** (silo A) stays on its own resolver (`getStateFile()`, `~/.claude/linkedin-studio.local.md`) and is NOT under the seam (see §7). The task is **unify**, not move. (`silo-inventory.md`)
|
||||
- **Plain-text + JSON + git, not a vector/graph DB** — Claude *is* the retrieval engine (reads files in-context); a heavyweight store adds infra + lock-in for no gain. (`secondbrain-sota.md` §B.1)
|
||||
- **Ingest is manual-first as the contract** — only EU/EEA DMA portability gives automated *content* pull (no analytics); scraping is a ToS breach. Connector is an additive tributary, never a dependency. (`connector-egress.md`)
|
||||
|
||||
## 3. Confirmed decisions (not open for re-litigation in review)
|
||||
|
||||
1. **Boundary:** engine → plugin · user data → data dir · cockpit → Maskinrommet.
|
||||
2. **Shape:** thin Markdown `brain/` hub (two-layer `profile.md` + episodic `journal/` + `operations.md` + `index.md` MOC) over the **existing typed tributaries** (voice-samples, specifics-bank, trends, analytics), fed by a provenance-tagged `ingest/` seam.
|
||||
3. **Tributaries keep their own schema/dedupe/lifecycle** — do NOT force one schema across silos. Only the user-profile field-set folds into `brain/profile.md`.
|
||||
4. **Invariants:** provenance-weighted learning (`published` only, never `ai-draft`); one canonical entity id + one provenance shape (`provenance ∈ {human, published, ai-draft}`); episodic/semantic split; evidence-threshold promotion; temporal-validity + freshness nudge; anti-sycophancy default; frozen-past-self anchor; zero required curation.
|
||||
|
||||
Review SHOULD challenge: feasibility, sequencing, hidden coupling, testability, scope leakage, missing risks. Review SHOULD NOT re-open the four decisions above (operator-approved) unless it finds they are internally contradictory or technically impossible.
|
||||
|
||||
## 4. Scope — the arc (slices)
|
||||
|
||||
| Slice | One-line scope |
|
||||
|-------|----------------|
|
||||
| **SB-S0 — Foundation** *(this brief)* | `brain/` scaffold + two-layer `profile.md` (fold in the user-profile field-set) + `index.md` + `operations.md` + `journal/`; entity-id + provenance shape as a small typed, tested module; `ingest/{inbox,published}` dirs + the manual-import **contract** (shape only). No loop, no ingest-processing logic, no connector. |
|
||||
| SB-S1 — Ingest + gold signal | Manual import populates `ingest/published/` with `provenance=published`; wire profile/voice to learn from published-only. |
|
||||
| SB-S2 — Evolution loop | Sleep-time consolidation: journal+published+tributary deltas → profile diff w/ evidence_count/provenance/timestamps; threshold promotion; contradiction reconciliation; brain-wide freshness nudge. **Owns the session-start wiring of the scaffold init.** |
|
||||
| SB-S3 — Cross-silo graph + ops centre | Thread the id through tributaries (post↔specific↔trend↔analytics); flesh out `operations.md`; retire dead `content-history.md` + triple-post reconciliation. |
|
||||
| SB-S4 *(optional, later)* | EU/EEA DMA portability API as auto-tributary into `ingest/inbox/`. |
|
||||
|
||||
## 5. SB-S0 in detail (the build target)
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
1. **`brain/` scaffold initialiser — an idempotent CLI/TS subcommand** (the trends-store idiom: runtime-created under the data-dir seam, NOT a tracked repo seed). It creates `brain/{index.md,profile.md,operations.md}` + `brain/journal/` + `ingest/{inbox,published}/` under `getDataRoot(subdir)` (which already accepts arbitrary subdirs — no new seam function needed). **Idempotent + migration-safe** (running twice is a no-op; never clobbers existing content). **No per-session cost** — session-start *wiring* of this init is explicitly deferred to SB-S2; SB-S0 ships only the invokable subcommand.
|
||||
|
||||
2. **`brain/profile.md` — two-layer schema** with `§ Static` and `§ Dynamic` sections. **Pinned contract (fixed):** one fact = one parseable record carrying all six fields `value · first_seen · last_seen · evidence_count · provenance(human|published|ai-draft) · status(active|superseded)`, and `serialize ∘ parse = identity`. **Open to the plan:** only the surface serialization syntax (frontmatter vs a defined line grammar).
|
||||
|
||||
3. **Fold the user-profile field-set into `brain/profile.md` — lossless + idempotent, with a defined source-absent path.** Source of the field *structure* is `config/user-profile.template.md` (the only file that ships; `profile/user-profile.md` is `expected-absent`, migrate-data.mjs:36). Behaviour:
|
||||
- **Source-absent (the common case):** scaffold an empty two-layer `profile.md` whose static layer contains every template field present-but-unfilled.
|
||||
- **Runtime instance present** (`${data}/profile/user-profile.md` exists): fold its filled values into the static layer.
|
||||
- Re-running never duplicates fields and never overwrites already-folded values.
|
||||
|
||||
4. **Entity-id + provenance module** — a small typed module (TS, matching the trends/specifics idiom) that mints one canonical entity id at creation and defines the single provenance shape (`{human|published|ai-draft}`). Pure functions, deterministic, unit-tested. **Not yet threaded through tributaries** (that's SB-S3) — SB-S0 only establishes the module + shape.
|
||||
|
||||
5. **`index.md` MOC + `operations.md` seeds (runtime-generated)** — minimal but real: `index.md` points to every tributary with a freshness placeholder; `operations.md` has the "who I am now" anchor section. Generated by the initialiser (D1), not committed to the repo tree.
|
||||
|
||||
6. **Tests** — unit tests for the module + the scaffold + the fold (incl. the source-absent path). No structure-lint count change expected (SB-S0 adds no `agents/`, `commands/`, `references/`, or `skills/` file); a version bump, if any, updates README/CHANGELOG badges only.
|
||||
|
||||
## 6. Success criteria (SB-S0 — testable)
|
||||
|
||||
- **SC1 (scaffold, runtime):** with `LINKEDIN_STUDIO_DATA` set to a temp dir, invoking the init subcommand creates `brain/{index,profile,operations}.md` + `brain/journal/` + `ingest/{inbox,published}/` **at the runtime data-path** (not the repo tree), through `getDataRoot`; invoking it twice is a no-op (idempotent, no clobber). Verified by a test that sets the env var, runs init twice, asserts existence + unchanged content.
|
||||
- **SC2 (profile parse):** `brain/profile.md` parses into `§Static`/`§Dynamic`; a fact round-trips through the pinned grammar with all six fields intact (`serialize ∘ parse = identity`). Verified by a parse/serialize unit test.
|
||||
- **SC3 (fold lossless + source-absent):** (a) **source-absent:** running the fold with no runtime user-profile yields a `profile.md` whose static layer lists every `config/user-profile.template.md` field; (b) **populated:** given a synthetic populated fixture, the fold carries every filled field into the static layer (diff shows none dropped); (c) re-running either case does not duplicate or overwrite. Verified by fixture-based diff tests.
|
||||
- **SC4 (id/provenance module):** the module mints a deterministic id (same input → same id) and **rejects** a malformed provenance value (throws; only `{human,published,ai-draft}` accepted); ≥1 unit test per public function.
|
||||
- **SC5 (no regression):** `scripts/test-runner.sh` is green (currently 89 passed / 0 failed / 0 warnings); each floored suite stays **≥ its floor** (trends ≥24, specifics ≥28, contract ≥33); all hook tests pass.
|
||||
- **SC6 (seam, no-op):** SB-S0 adds **no new seam function** (`brain/`/`ingest/` use the existing `getDataRoot(subdir)`), so the twin-sync surface is unchanged. Asserted as a no-op; *if* a future slice adds a seam function, the existing twin-sync test (`hooks/scripts/__tests__/data-root.test.mjs`) must be extended.
|
||||
|
||||
## 7. Non-goals / scope fence (SB-S0 does NOT)
|
||||
|
||||
- **No evolution loop / consolidation** (SB-S2).
|
||||
- **No ingest processing logic** — `ingest/` dirs + the documented manual-import contract only; no parser that reads inbox files yet (SB-S1).
|
||||
- **No connector** — no LinkedIn API code (SB-S4).
|
||||
- **No cross-silo threading** — the id module exists but is NOT yet woven into trends/specifics/analytics (SB-S3).
|
||||
- **No session-start wiring** of the scaffold init (SB-S2 owns it) — SB-S0 ships only the invokable subcommand.
|
||||
- **Do NOT touch tributary schemas** (voice/specifics/trends/analytics stores keep their current shape).
|
||||
- **Do NOT reconcile the two-roots question** (state file at `~/.claude/linkedin-studio.local.md` stays a sibling on `getStateFile()`; not folded in SB-S0).
|
||||
- **Do NOT retire `content-history.md` or reconcile the triple-recorded post** (SB-S3).
|
||||
- **No GUI / no cockpit** (Maskinrommet, out of repo scope).
|
||||
|
||||
## 8. Research plan / knowledge status
|
||||
|
||||
- **Done:** the three threads (connector reality, second-brain SOTA, silo inventory) are complete and persisted. No further research is required to start SB-S0. (Brief-review confirmed both §8 open assumptions are correctly non-blocking.)
|
||||
- **Open assumptions to test (flagged, not blocking SB-S0):**
|
||||
- "Claude-as-retrieval-engine over plain files is sufficient (no vector DB)" — testable once the brain holds content (post-SB-S1). Marked assumption.
|
||||
- Newsletter-edition coverage by the portability `ARTICLES` domain — only relevant to SB-S4; verify against a real export then.
|
||||
|
||||
## 9. Constraints / risks
|
||||
|
||||
- **TDD iron law:** no production code without a failing test first.
|
||||
- **New code placement:** SB-S0's new code is a **TS module under `scripts/`** (the trends/specifics idiom) + a runtime scaffold + edits to existing files. It creates **no new `.mjs` under `hooks/scripts/`**, so the pathguard does not affect SB-S0. **Verified 2026-06-23 (code-checked; STATE.md line 39 CONFIRMED correct, brief-review B2 was WRONG):** the llm-security plugin guard `pre-write-pathguard.mjs` IS enabled (`~/.claude/settings.json:148`) and registered as a PreToolUse hook with `matcher:"Write"` (Write-only); its Category 6 regex `/[\\/]hooks[\\/]scripts[\\/].*\.mjs$/` matches any `…/hooks/scripts/*.mjs`, **including this repo**. ⇒ a NEW `.mjs` under `hooks/scripts/` **cannot be written with the Write tool**; **editing an EXISTING `.mjs` is fine** (matcher is Write-only). Relevant to SB-S2 (which touches hook code → create new hook `.mjs` via Bash-heredoc, or edit an existing file). SB-S0 stays entirely in `scripts/`, so it is unaffected.
|
||||
- **Twin data-root implementations** must stay byte-compatible (`hooks/scripts/data-root.mjs` ⇄ `scripts/analytics/src/utils/storage.ts`); SB-S0 adds no new seam function so the twin contract is untouched (SC6).
|
||||
- **Public repo (`open/` Forgejo):** confirm before push; push only in the allowed window; STATE.md stays gitignored.
|
||||
- **Structure lint** counts `agents/`, `commands/`, `references/`, `skills/` + the version badge. SB-S0 adds none of these, so no count guard fires absent a version bump.
|
||||
- **Node v25 test quirk:** `node --test <dir>` doesn't work — use glob `__tests__/*.test.mjs`; run TS tools from their own dir.
|
||||
- **Risk — fold ambiguity:** the user-profile→profile.md fold is the one data-shaping step in SB-S0; if the grammar is under-specified it risks lossy or non-idempotent folding. Mitigate with fixture-based diff tests (SC3) covering both the source-absent and populated paths before wiring.
|
||||
|
||||
## 10. Definition of done (SB-S0)
|
||||
|
||||
SC1–SC6 all pass; SB-S0 deliverables 1–6 exist; non-goals untouched; committed (push per window); `architecture.md` slice status updated to "SB-S0 landed"; `CLAUDE.md`/README touched only if a version bump is taken.
|
||||
82
docs/second-brain/consolidation-loop.md
Normal file
82
docs/second-brain/consolidation-loop.md
Normal file
|
|
@ -0,0 +1,82 @@
|
|||
# Consolidation loop — the compounding mechanism (SB-S2)
|
||||
|
||||
> How the second brain turns the published gold signal into an ever-improving, drift-resistant
|
||||
> `brain/profile.md` — operator-invoked, operator-gated, deterministic. Part of the second-brain arc
|
||||
> (`architecture.md`); landed in SB-S2.
|
||||
|
||||
## The shape
|
||||
|
||||
The loop is **operator-invoked** ("sleep-time" = when you run it), not automatic — the session-start
|
||||
hook is zero-dep and cannot run AI, so it only **nudges** when consolidation is due. The pass itself is:
|
||||
|
||||
```
|
||||
brain consolidate --gather # 1. dump new published deltas + the current profile
|
||||
→ (the session reads them and extracts a Candidate[] JSON)
|
||||
brain consolidate --propose --candidates cand.json # 2. deterministic diff → brain/pending-diff.{md,json}
|
||||
→ (you review brain/pending-diff.md — the [OPERATØR] gate)
|
||||
brain consolidate --apply --diff brain/pending-diff.json --confirm # 3. the ONLY write to profile.md
|
||||
```
|
||||
|
||||
`--apply` records `brain/consolidation-state.json` `{last_run}`; the session-start nudge reads it +
|
||||
counts `ingest/published/*.md` to know when to nudge again. Roll back any apply via git.
|
||||
|
||||
## The deterministic engine (`scripts/brain/src/consolidate.ts`)
|
||||
|
||||
`proposeDiff({current, candidates, today, opts})` classifies each candidate (matched to existing facts
|
||||
by the candidate's `key`):
|
||||
|
||||
| Rule | Condition | Effect |
|
||||
|------|-----------|--------|
|
||||
| **reject** | `provenance: ai-draft` | dropped — never learns from the engine's own drafts (model-collapse guard) |
|
||||
| **add** | no matching fact, provenance `published`/`human` | new dynamic fact, `evidence_count: 1` |
|
||||
| **evidence-bump** | matching fact, same value | `evidence_count++`, `last_seen = today` |
|
||||
| **promote** | a dynamic fact reaches `N = 3` observations | dynamic → static |
|
||||
| **conflict** | matching key, different value, NO supersede signal | **keep both**, timestamped, with DISTINCT ids; the old fact is untouched (a genuine contradiction — both views coexist) |
|
||||
| **supersede** (SB-S3b) | matching key, different value, explicit `supersedes` signal on the candidate | retire the old fact (re-minted to an archival id, `status: superseded`, retained as audit) + install the new winner at the canonical key-id — a temporal update, not a contradiction |
|
||||
| **decay-flag** | a dynamic **active** fact's `last_seen` > `90` days | listed in `staleFlags` (informational; never auto-removed; superseded facts are decay-exempt) |
|
||||
|
||||
**Id model (no duplicate ids):** a concept's primary fact id is `mintEntityId({kind:'observed', key})`;
|
||||
a conflict alt fact id is `mintContentId('observed-alt:'+key+'::'+value+'::'+date)` — byte-distinct, so
|
||||
two facts never share an id. The SB-S0 folded `profile-field` static seeds use a different kind, so
|
||||
consolidation never mutates them (immutable in S2). `applyDiff` produces a `ProfileDoc` that round-trips
|
||||
exactly through the SB-S0 grammar; re-running is idempotent (bump, not duplicate).
|
||||
|
||||
Defaults: `promoteThreshold = 3`, `decayDays = 90` (operator-confirmed).
|
||||
|
||||
## The candidate file — the session↔engine contract
|
||||
|
||||
`--propose --candidates <file.json>` takes a JSON **array** of candidates; each is validated
|
||||
(malformed → non-zero exit, nothing written):
|
||||
|
||||
```json
|
||||
[
|
||||
{ "key": "primary-expertise", "value": "AI governance in the public sector",
|
||||
"provenance": "published", "source": "published:1a2b3c4d5e6f", "observed_date": "2026-05-26" }
|
||||
]
|
||||
```
|
||||
|
||||
- `key`, `value`, `source`, `observed_date` — non-empty strings; `provenance ∈ {human, published, ai-draft}`.
|
||||
- `key` and `value` must be **single-line** (no newline/CR — the profile grammar is one fact per line).
|
||||
- The session produces this from `--gather`'s output. The engine guarantees the *mechanics*; the *quality*
|
||||
of the candidates is the session's job (see limits).
|
||||
|
||||
## Honest limits
|
||||
|
||||
- **The loop's value depends on the session's extraction.** The engine only guarantees threshold/conflict/
|
||||
decay/provenance mechanics. Garbage candidates → a garbage diff. The operator gate + candidate-shape
|
||||
validation catch shape errors, not insight quality.
|
||||
- **`brain/profile.md` has one reader as of SB-S3a.** S2 evolved the profile motor-only; SB-S3a wired the
|
||||
first consumer — `strategy-advisor` reads it as *evidence-to-test* (anti-sycophancy: counter-pressured, never
|
||||
parroted). Broader consumption (more content agents, a hook-level prompt digest) remains later S3 work, and
|
||||
the profile is still mutated ONLY via `brain consolidate --apply --confirm`.
|
||||
- **Supersede is explicit + operator-gated; no AUTO-demotion (SB-S3b).** A temporal update retires the old
|
||||
fact ONLY when the candidate carries an explicit `supersedes` signal (the session's judgement), and only
|
||||
after the operator confirms the rendered `## Supersessions` in the diff. The engine guarantees the supersede
|
||||
*mechanics* (replace-in-place, archival re-mint, audit-retain); whether a contradiction is really a temporal
|
||||
update vs a genuine contradiction is the session's + operator's call — **the operator gate is the only
|
||||
classification net** (a mis-classified supersede silently retires a fact that should have been kept-both).
|
||||
Genuine contradictions (no signal) still **keep both**; stale facts are still only *flagged*, never auto-removed.
|
||||
- **No AI at session-start.** The nudge is a deterministic file-count + sidecar read; the consolidation
|
||||
pass is always operator-invoked.
|
||||
- **The session-start nudge is consolidation-due only** — it counts published records + days since last run;
|
||||
it does not parse `profile.md` for per-fact staleness (that cost/parser is deferred).
|
||||
87
docs/second-brain/ingest-manual-import.md
Normal file
87
docs/second-brain/ingest-manual-import.md
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
# Manual-import contract — ingest + the published gold signal (SB-S1)
|
||||
|
||||
> The contract for getting your **actual published posts** into the second brain as the
|
||||
> **gold signal** (`provenance=published`) that voice/profile learning is allowed to learn from.
|
||||
> Part of the second-brain arc (`architecture.md`); landed in SB-S1.
|
||||
|
||||
## Why this exists
|
||||
|
||||
Profile and voice must learn from **human-published content only** — never from what the engine
|
||||
itself drafted. A content engine that learns its own voice from its own drafts collapses toward its
|
||||
own priors (model collapse). So the brain has one provenance vocabulary — `human | published | ai-draft`
|
||||
— and the learning surfaces are bound to `published` and **forbidden from `provenance=ai-draft`**.
|
||||
|
||||
A published post already lands in three non-referencing places today (the state file's `## Recent Posts`,
|
||||
the dead `analytics/content-history.md`, and `analytics/posts/*.json` from the CSV) — but **none of them
|
||||
holds the full post text**. Voice learning needs the verbatim text, so `ingest/published/` storing it is
|
||||
additive, not a re-count. (Threading a shared id across those silos is a later slice, SB-S3.)
|
||||
|
||||
## The drop-zone model
|
||||
|
||||
```
|
||||
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/ingest/
|
||||
inbox/ # drop-zone: paste a published post here as a .md file (manual, or a future connector)
|
||||
published/ # processed gold records, provenance=published — one file per post, <id>.md
|
||||
```
|
||||
|
||||
- **`inbox/`** — you drop raw post files here (top-level `*.md`, one post per file).
|
||||
- **`published/`** — the `ingest` CLI writes processed, provenance-tagged records here. `<id>.md` where
|
||||
`<id> = sha256(verbatim body)[:12]` — byte-identity, so the same post never duplicates and two
|
||||
genuinely different posts never collide.
|
||||
|
||||
## CLI usage
|
||||
|
||||
Run from `scripts/brain` (`node --import tsx src/cli.ts <cmd>`), or via the package's `start` script.
|
||||
|
||||
```bash
|
||||
# one post from a file → ingest/published/<id>.md, tagged provenance=published
|
||||
brain ingest --file path/to/post.md [--source manual] [--date 2026-05-26]
|
||||
|
||||
# process everything dropped in ingest/inbox/ (top-level *.md, skips dotfiles), non-destructive
|
||||
brain ingest --scan-inbox
|
||||
|
||||
# inspect the gold corpus — id · provenance · date · first line (eyeball that nothing ai-draft leaked)
|
||||
brain published list [--json]
|
||||
```
|
||||
|
||||
- `--date` is the real publish date; it defaults to the ingest date when omitted.
|
||||
- Re-ingesting the same body is a **no-op** (idempotent). An id-collision with a *differing* body is
|
||||
disambiguated to `<id>-2.md` — a differing gold record is **never silently dropped**.
|
||||
- `ingest` **creates `ingest/published/` on demand** — it does not require a prior `brain init`.
|
||||
|
||||
## Record file grammar (no YAML)
|
||||
|
||||
One file per post: a fixed five-line header, a `---` sentinel, then the **verbatim body**.
|
||||
|
||||
```
|
||||
id: 1a2b3c4d5e6f
|
||||
provenance: published
|
||||
published_date: 2026-05-26
|
||||
captured_at: 2026-06-23
|
||||
source: manual
|
||||
---
|
||||
The verbatim post body goes here.
|
||||
|
||||
It may contain anything — blank lines, ] | characters, even a line that looks
|
||||
like --- — because the parser splits on the FIRST sentinel only.
|
||||
```
|
||||
|
||||
`parse ∘ serialize` is an exact identity over all six fields including the body. A record in
|
||||
`published/` whose `provenance` is anything other than `published` is treated as a **corruption signal**
|
||||
and rejected on parse.
|
||||
|
||||
## The published-only learning rule (and its honest limit)
|
||||
|
||||
- `agents/voice-trainer.md` is wired to read `ingest/published/` as its primary gold source and to
|
||||
**exclude `provenance=ai-draft` unconditionally**. This wiring is **gate-enforced** by a structure-lint
|
||||
(`scripts/test-runner.sh`, the Brain Published-Only Invariant section): the lint asserts the agent
|
||||
carries both the `ingest/published` source and the `provenance=ai-draft` exclusion.
|
||||
- **Honest limit:** the *wiring* is gate-enforced, but an agent instruction can still be ignored at
|
||||
runtime — the gate proves the rule is present, not that the model obeyed it on a given run. The
|
||||
strongest enforcement short of executing the agent.
|
||||
|
||||
## What SB-S1 does NOT do
|
||||
|
||||
No consolidation loop and **no automatic `brain/profile.md` mutation** (SB-S2 owns the sleep-time
|
||||
profile-diff loop) · no session-start wiring · no cross-silo id threading (SB-S3) · no connector
|
||||
(SB-S4). `source` carries a forward-compatible field for a future connector, but no connector code ships.
|
||||
162
docs/second-brain/plan-sb-s0.md
Normal file
162
docs/second-brain/plan-sb-s0.md
Normal file
|
|
@ -0,0 +1,162 @@
|
|||
# Implementation Plan — SB-S0 (Foundation)
|
||||
|
||||
> **Status:** revised after adversarial plan-review (`plan-critic` + `scope-guardian`, both code-verified) → all findings folded in. Ready for operator approval.
|
||||
> **Brief:** `brief.md` §5–§10. **Architecture:** `architecture.md`.
|
||||
> **Scope:** SB-S0 only — the `brain/` scaffold, the two-layer `profile.md` + lossless user-profile fold, the entity-id/provenance module, the `ingest/` seam dirs. **No loop, no ingest logic, no connector, no cross-silo threading, no session-start wiring** (brief §7).
|
||||
> **Review delta (2026-06-23):** B1 test convention → real TS idiom (`tests/*.test.ts` via tsx, `.js` specifiers); M2 profile format → **defined line-grammar (no YAML dep)**; M3 data-root → inline per-package resolver (repo idiom, not a new seam); M4 fold extraction rule pinned against the real template; m1 gate rationale corrected; m2 npm-install-before-gate; m3 provenance throws; m4 id keyed on stable slug; G1 push-confirm; C1 identity now whole-file.
|
||||
|
||||
---
|
||||
|
||||
## 1. What SB-S0 delivers (recap)
|
||||
|
||||
A new TS package `scripts/brain/` (an exact structural copy of `scripts/specifics-bank/`) exposing an idempotent `brain init` CLI that scaffolds the `brain/` + `ingest/` tree at the runtime data-path, a pinned two-layer `profile.md` **line-grammar** with parse/serialize round-trip, a lossless+idempotent user-profile fold (source-absent + populated), and a deterministic entity-id + provenance module. All unit-tested, TDD (failing test first). Gate stays green.
|
||||
|
||||
## 2. Package layout (`scripts/brain/`) — exact copy of `scripts/specifics-bank/` conventions
|
||||
|
||||
```
|
||||
scripts/brain/
|
||||
package.json # copy specifics-bank/package.json; "test": "node --import tsx --test tests/*.test.ts"; devDeps tsx + typescript ONLY (no yaml — see §3.3)
|
||||
tsconfig.json # copied VERBATIM from scripts/specifics-bank/tsconfig.json (rootDir ./src, include src/**/*)
|
||||
src/
|
||||
types.ts # Provenance, FactStatus, ProfileFact, ProfileDoc, constants
|
||||
id.ts # mintEntityId(), slugify(), normalizeProvenance()
|
||||
profile.ts # parseProfile(), serializeProfile(), foldUserProfile()
|
||||
dataRoot.ts # inline resolver (repo idiom — §3.5)
|
||||
cli.ts # `brain init`
|
||||
tests/ # NOTE: tests/*.test.ts, run via tsx; import source via .js specifiers (e.g. "../src/id.js")
|
||||
id.test.ts
|
||||
profile.test.ts
|
||||
scaffold.test.ts
|
||||
fold.test.ts
|
||||
fixtures/
|
||||
user-profile.populated.md # synthetic populated user-profile mirroring the template's extractable constructs (§3.4)
|
||||
```
|
||||
|
||||
**B1 fix (verified):** all three sibling TS packages use `tests/*.test.ts` via `node --import tsx --test tests/*.test.ts` and import source through `.js` specifiers (`specifics-bank/package.json:8`, `specifics-bank/tests/bank.test.ts:15` `from "../src/bank.js"`). The `__tests__/*.test.mjs` idiom is the HOOKS convention (no tsx loader) and cannot import `.ts` source — NOT used here.
|
||||
|
||||
## 3. Module designs
|
||||
|
||||
### 3.1 `types.ts`
|
||||
```ts
|
||||
export type Provenance = 'human' | 'published' | 'ai-draft';
|
||||
export type FactStatus = 'active' | 'superseded';
|
||||
export interface ProfileFact {
|
||||
id: string; // canonical entity id (id.ts), keyed on a stable slug
|
||||
value: string; // single-line (no embedded newline)
|
||||
first_seen: string; // YYYY-MM-DD
|
||||
last_seen: string; // YYYY-MM-DD
|
||||
evidence_count: number;
|
||||
provenance: Provenance;
|
||||
status: FactStatus;
|
||||
}
|
||||
export interface ProfileDoc { schemaVersion: 1; static: ProfileFact[]; dynamic: ProfileFact[]; }
|
||||
export const PROVENANCE_VALUES = ['human','published','ai-draft'] as const;
|
||||
```
|
||||
|
||||
### 3.2 `id.ts`
|
||||
- `slugify(label: string): string` → lowercase, trim, non-alphanumeric → `-`, collapse repeats. Stable across value edits.
|
||||
- `mintEntityId(seed: {kind: string; key: string}): string` → `sha256(`${kind}:${slugify(key)}`).slice(0,12)` (matches trends/specifics `sha256[:12]`; confirm the exact hash call against `scripts/specifics-bank/src/bank.ts` in step 1). Deterministic, pure.
|
||||
- `normalizeProvenance(raw: string): Provenance` → trim+lowercase; return the match or **throw** on anything outside `PROVENANCE_VALUES`. (m3: throws — SC4 asserts `assert.throws`; "normalises" dropped.)
|
||||
|
||||
### 3.3 `profile.ts` — pinned line-grammar (M2 decision: NO YAML dep)
|
||||
**Verified:** the repo has no YAML parser (`state-updater.mjs:43` `extractField` is per-field regex on single scalars; zero `yaml` dependency in-tree). Frontmatter-with-nested-arrays would force a new dep + an untested hand-rolled emitter. **Decision — a defined line-grammar that round-trips with the regex idiom and needs no dep:**
|
||||
|
||||
```
|
||||
# Profile
|
||||
|
||||
schemaVersion: 1
|
||||
|
||||
## Static
|
||||
|
||||
- [<provenance>|<status>|<first_seen>|<last_seen>|<evidence_count>|<id>] <value>
|
||||
|
||||
## Dynamic
|
||||
|
||||
- [<provenance>|<status>|<first_seen>|<last_seen>|<evidence_count>|<id>] <value>
|
||||
```
|
||||
|
||||
- The bracket holds six pipe-joined **constrained** tokens (two enums, two ISO dates, an int, a 12-hex id) — none can contain `]` or `|`. `<value>` is the rest of the line after `] ` (free single-line text; may contain `]`/`|`/quotes).
|
||||
- `parseProfile(text)`: split into `## Static` / `## Dynamic` sections; each `- [...]` line parsed by
|
||||
`^- \[(human|published|ai-draft)\|(active|superseded)\|(\d{4}-\d{2}-\d{2})\|(\d{4}-\d{2}-\d{2})\|(\d+)\|([0-9a-f]{12})\] (.*)$`.
|
||||
`schemaVersion` read by the existing scalar-regex idiom.
|
||||
- `serializeProfile(doc)`: deterministic token order → `- [...] value`, sections in fixed order.
|
||||
- **`parseProfile(serializeProfile(doc)) === doc`** over the WHOLE file (C1 resolved: there is no derived/ignored body — the lines ARE the authoritative representation). SC2 tests this identity.
|
||||
|
||||
### 3.4 `profile.ts` — `foldUserProfile()` (M4: extraction rule pinned against the real template)
|
||||
`config/user-profile.template.md` is heterogeneous (`config/user-profile.template.md:15-147`: bold-label scalars, a numbered expertise group, checkbox lists, non-labeled bullets, prose guidance). **Pinned extraction — exactly two productions:**
|
||||
- **P1 (labeled scalar):** any line matching `^\s*(?:- )?\*\*(.+?):\*\*\s*(.*)$` → `label`=g1, `value`=g2 (the `[placeholder]` text; treated as *unfilled/empty* when it is a bracketed placeholder, literal otherwise). One `ProfileFact` per match. Covers Name/Role/Organization/Industry, the `- **Primary:**` etc. sub-labels, Signature Elements, Writing Quirks, Current LinkedIn Status.
|
||||
- **P2 (expertise group):** under the `**Core Expertise Areas (...)**:` label, each `^\d+\.\s*(.*)$` line → one fact, label `expertise-area-N`.
|
||||
- **Excluded (deferred, noted §8):** checkbox lines (`- [ ] …` — Tone, LinkedIn Goals, Research MCPs, Asset Utilization), non-labeled bullet lists (Content Style Mix), the Voice-Profile-Summary numbered `1. **[Quality N]:**` items, prose guidance blocks (Universal anti-patterns 95-104, Research-Tooling explainer), and headers.
|
||||
|
||||
```
|
||||
foldUserProfile({ templateText, instanceText?, existing? }): ProfileDoc
|
||||
```
|
||||
1. Apply P1+P2 to `templateText` → the canonical field-set (id = `mintEntityId({kind:'profile-field', key: label})`, value empty, `provenance:'human'`, `evidence_count:0`, `status:'active'`, dates = run date). → `static[]`. (`dynamic[]` empty in S0.)
|
||||
2. **Source-absent (common):** no `instanceText` → return that field-set with empty values.
|
||||
3. **Populated:** `instanceText` present → re-run P1+P2 on it, copy filled values onto the matching id.
|
||||
4. **Idempotent:** if `existing` passed → merge by `id`; never duplicate, never overwrite a non-empty value, bump `last_seen` only.
|
||||
Pure; the CLI supplies the file texts. (m4: id keyed on `slugify(label)`, stable across value edits; label-text stability is a known S0 limit — §8.)
|
||||
|
||||
### 3.5 `dataRoot.ts` — inline resolver (M3: repo idiom, NOT a new seam)
|
||||
**Verified:** no shared util; cross-package import not configured (`specifics-bank/tsconfig.json` rootDir `./src`, no path map/workspace); siblings each inline it (`trends/src/store.ts:180`, `specifics-bank/src/bank.ts:150`, plus `analytics/src/utils/storage.ts:54`). Match the idiom:
|
||||
```ts
|
||||
import { homedir } from 'node:os'; import { join } from 'node:path';
|
||||
export const dataRoot = (sub: string) =>
|
||||
join(process.env.LINKEDIN_STUDIO_DATA ?? join(homedir(), '.claude', 'linkedin-studio'), sub);
|
||||
```
|
||||
This is a **private package resolver, not an exported seam function** → SC6 (no new seam, twin-sync untouched) holds. True de-duplication of the now-4 copies is an explicit out-of-scope refactor.
|
||||
|
||||
### 3.6 `scaffold.ts` — `initBrain(rootSub?)` (D1, D5) + `cli.ts`
|
||||
- Create idempotently (mkdir-recursive; **compare-then-skip**, never blind-write): `brain/journal/`, `ingest/inbox/`, `ingest/published/`; `brain/profile.md` (IF absent → `serializeProfile(foldUserProfile({templateText}))`), `brain/index.md` (IF absent → MOC seed: one line per tributary [voice-samples, specifics-bank, trends, analytics, ingest] with `freshness: —`), `brain/operations.md` (IF absent → `## Who I am now (anchor)` + empty `## Plans` / `## Ideas`).
|
||||
- Returns `{created[], skipped[]}`. **No session-start wiring** (SB-S2). `cli.ts`: `brain init` → `initBrain()`, print report. Dispatch shape copied from `specifics-bank/src/cli.ts`.
|
||||
|
||||
## 4. Files created / edited
|
||||
|
||||
**Created:** `scripts/brain/{package.json,tsconfig.json}` · `scripts/brain/src/{types,id,profile,dataRoot,cli}.ts` · `scripts/brain/tests/{id,profile,scaffold,fold}.test.ts` · `scripts/brain/fixtures/user-profile.populated.md`.
|
||||
|
||||
**Edited (minimal):**
|
||||
- `scripts/test-runner.sh` — add a `BRAIN_TESTS_FLOOR` block copied from the `TRENDS_TESTS_FLOOR` block (`scripts/test-runner.sh:681-694`): cd `scripts/brain`, run the suite, grep `ℹ tests N` (`grep -oE 'tests [0-9]+' | tail -1`), assert `≥ BRAIN_TESTS_FLOOR`, emit one `pass()` line. Bump Section 18's anti-erosion note (`:770`, `TOTAL_CHECKS >= 74` grows by 1).
|
||||
- `docs/second-brain/architecture.md` — flip SB-S0 row to "landed" (DoD).
|
||||
- (NO change to `data-root.mjs`/`storage.ts` — no seam function added, SC6.)
|
||||
|
||||
## 5. TDD test plan (failing-first) → success criteria
|
||||
|
||||
| Test (`tests/*.test.ts`) | Asserts | SC |
|
||||
|---|---|---|
|
||||
| `id.test.ts` | `mintEntityId` deterministic (same in→same id; diff→diff); `slugify` stable; `normalizeProvenance` returns the 3 values, `assert.throws` on others | SC4 |
|
||||
| `profile.test.ts` | `parseProfile(serializeProfile(doc)) === doc` over the whole file incl. a value containing `]`/`|`/quotes; sections `## Static`/`## Dynamic` present | SC2 |
|
||||
| `fold.test.ts` | (a) source-absent → field-set from template (P1+P2), values empty; (b) populated fixture → every filled field carried (diff none-dropped); (c) re-run → no dup, no overwrite | SC3 |
|
||||
| `scaffold.test.ts` | `LINKEDIN_STUDIO_DATA`=tmp → init creates all dirs+files at runtime path; second init = no-op (skipped[] covers them, files byte-identical) | SC1 |
|
||||
| (gate) `scripts/test-runner.sh` | green; trends ≥24, specifics ≥28, contract ≥33, **brain ≥ floor**; hooks pass | SC5 |
|
||||
| (n/a) | no new seam fn; unchanged `data-root.test.mjs` twin-sync still passes | SC6 |
|
||||
|
||||
Each test written **before** its module (iron law): red → implement → green.
|
||||
|
||||
## 6. Step sequence
|
||||
|
||||
1. **Confirm idioms in code (verify-first):** read `scripts/specifics-bank/{package.json,tsconfig.json,src/bank.ts,tests/bank.test.ts}` → copy the package/test scaffolding + the exact sha256 call. `npm install` in `scripts/brain` so tsx is present (m2: the gate `warn`-skips a suite if `node_modules/.bin/tsx` is absent — "green" must not mean green-because-skipped).
|
||||
2. `types.ts` (no logic).
|
||||
3. TDD `id.ts` (`id.test.ts`). → SC4.
|
||||
4. TDD `profile.ts` parse/serialize line-grammar (`profile.test.ts`). → SC2.
|
||||
5. Write `fixtures/user-profile.populated.md` mirroring the P1+P2 constructs; TDD `foldUserProfile` (`fold.test.ts`, 3 cases). → SC3.
|
||||
6. TDD `scaffold.ts` `initBrain` (`scaffold.test.ts`, temp-dir idempotency). → SC1.
|
||||
7. `cli.ts` `brain init`; smoke-run against a temp `LINKEDIN_STUDIO_DATA`.
|
||||
8. Wire `scripts/test-runner.sh` BRAIN floor (+ Section 18 note); run the FULL gate green. → SC5/SC6.
|
||||
9. Flip `architecture.md` SB-S0 → landed. **Commit; confirm before push (PUBLIC `open/` Forgejo) and only inside the push window; STATE.md stays gitignored** (G1).
|
||||
|
||||
## 7. Scope fence (echo — SB-S0 does NOT)
|
||||
|
||||
No consolidation loop · no ingest file parsing · no LinkedIn API · no id threading into tributaries · no session-start wiring · no tributary-schema edits · no state-file/two-roots reconciliation · no `content-history.md` retirement · no GUI.
|
||||
|
||||
## 8. Known limits / deferred (honest)
|
||||
|
||||
- **Fold covers only P1 labeled-scalars + P2 expertise group.** Checkbox preference lists (tone, goals, MCPs, assets), non-labeled bullets, and the Voice-Profile-Summary qualities are NOT folded in S0 — they are template *guidance/preferences*, foldable in a later slice when the profile schema grows. Stated so SC3a's field-set is finite + testable.
|
||||
- **Profile-field id keyed on `slugify(label)`** — stable across value edits (fixes the raw-label non-idempotency). A *renamed template label* would still mint a new id; acceptable in S0 (template is fixed) — re-confirm if SB-S1+ lets users edit labels.
|
||||
- **`parse∘serialize` identity holds for single-line values.** A value with an embedded newline is out of grammar; the fold never produces one (template/instance fields are single-line), but `addFact` paths in later slices must enforce it.
|
||||
- **Gate "green" is honest only with deps installed** (m2) — fresh clone must `npm install scripts/brain` or the brain suite `warn`-skips.
|
||||
|
||||
## 9. Risks
|
||||
|
||||
- **Fold under-specification** → mitigated: P1+P2 pinned + `fold.test.ts` covers source-absent + populated + re-run before the CLI wires it.
|
||||
- **Idempotency edge:** `initBrain` + fold merge must compare-then-skip (never blind-write); tested in `scaffold.test.ts` / `fold.test.ts` (c).
|
||||
- **Gate mechanics (corrected, m1):** 89 is the gate's own `PASS+FAIL` tally; each suite contributes exactly ONE `pass()` line (not its internal `tests N`); the only total guard is the growable `TOTAL_CHECKS >= 74` floor (`:770`). Adding BRAIN raises that floor by 1 — no `==89` guard exists, so no regression flag. The risk is the warn-skip (m2), handled in step 1/8.
|
||||
160
docs/second-brain/plan-sb-s1.md
Normal file
160
docs/second-brain/plan-sb-s1.md
Normal file
|
|
@ -0,0 +1,160 @@
|
|||
# Implementation Plan — SB-S1 (Ingest + gold signal)
|
||||
|
||||
> **Status:** review-hardened (`plan-critic` REVISE 63→folded; `scope-guardian` GAP→folded). Ready for operator approval / "go".
|
||||
> **Brief:** `brief-sb-s1.md` (review-hardened). **Arc:** `architecture.md`. **Predecessor:** SB-S0 (`scripts/brain/`).
|
||||
> **Review delta (2026-06-23):** B1 separator pinned to one exact string + edge battery; B2 id → verbatim-body hash (no `normalizeContent`) + collision-safe write; floors pinned EMPIRICALLY (not hand-arithmetic); SC5 grep → exact `grep -F` literals + `AI-generated` negative probe; CLI = net-new flag infra + `init` regression; per-step On-failure/Checkpoint + 2 circuit-breakers; D3 auto-append FENCE restored; voice-trainer anchors fixed (:136/:144); malformed-file try/catch + scanInbox `*.md`-only filter.
|
||||
> **Scope:** SB-S1 only — the published-record data layer + `brain ingest` CLI, and wiring `voice-trainer` to the published-only invariant (gate-enforced). **No consolidation loop, no automatic profile mutation, no session-start wiring, no cross-silo threading, no connector** (brief §7).
|
||||
|
||||
---
|
||||
|
||||
## 1. What SB-S1 delivers (recap)
|
||||
|
||||
A new TS module `scripts/brain/src/ingest.ts` (matching the SB-S0/specifics-bank idiom) defining a `PublishedRecord` shape with a pinned file-per-post **constrained-header grammar** (no YAML) that round-trips, a content-hash id over the **verbatim body** (`sha256(body)[:12]` — byte-identity dedup, no normalization), and idempotent, collision-safe IO (`writePublished`, `scanInbox`, `listPublished`) under the brain package's `dataRoot`. The `brain` CLI grows `ingest` + `published list` subcommands. `agents/voice-trainer.md` is wired to add `ingest/published/` as the primary gold source and forbid learning from `provenance=ai-draft`, and that wiring is **gate-enforced** by a new structure-lint (Section 16c). All unit-tested, TDD (failing test first). Gate stays green; BRAIN + assertion floors rise.
|
||||
|
||||
## 2. Package additions (`scripts/brain/`) — no new package, extend the existing one
|
||||
|
||||
```
|
||||
scripts/brain/
|
||||
src/
|
||||
ingest.ts # NEW — PublishedRecord type + parse/serialize (pure) + writePublished/scanInbox/listPublished (IO)
|
||||
id.ts # EDIT — add normalizeContent() + mintContentId() (the specifics-bank content-hash idiom)
|
||||
cli.ts # EDIT — add `ingest` + `published` dispatch (parseFlags idiom from specifics-bank/src/cli.ts)
|
||||
(types.ts, profile.ts, scaffold.ts, dataRoot.ts unchanged)
|
||||
tests/
|
||||
ingest.test.ts # NEW — record round-trip + id determinism/dedup (pure)
|
||||
publish.test.ts # NEW — temp-dir IO: write/idempotency/scan-inbox/create-on-demand
|
||||
fixtures/
|
||||
sample-inbox-post.md # NEW — a synthetic published post for scan-inbox + round-trip tests
|
||||
```
|
||||
|
||||
No `package.json`/`tsconfig.json` change (devDeps already tsx+typescript; no new dep — the grammar is regex, no YAML). The test command (`node --import tsx --test tests/*.test.ts`) already globs `tests/*.test.ts`, so the two new suites are picked up automatically.
|
||||
|
||||
## 3. Module designs
|
||||
|
||||
### 3.1 `id.ts` addition — `mintContentId` on the VERBATIM body (no normalization — fixes plan-critic B2)
|
||||
**Verify-first (step 1):** mirror the `specificId` hash idiom at `scripts/specifics-bank/src/bank.ts:57` (the function is named `specificId`, NOT `mintContentId` — that name is net-new here; `normalizeContent` lives at `bank.ts:52`). **Decision: do NOT reuse `normalizeContent` for a post body.**
|
||||
```ts
|
||||
// mintContentId: sha256 of the VERBATIM body, first 12 hex. The id IS byte-identity.
|
||||
export function mintContentId(text: string): string {
|
||||
return createHash("sha256").update(text).digest("hex").slice(0, 12);
|
||||
}
|
||||
```
|
||||
- **Why verbatim, not `normalizeContent`:** `normalizeContent` (bank.ts:52) does `trim().toLowerCase().replace(/\s+/g," ")` — correct for *short* specifics dedupe, but for a full post body it collapses ALL whitespace + case, so two **structurally different** published posts (same words, different line breaks/paragraphing/case) would mint the **same id** and the second would be silently skipped on write (data loss of the exact gold signal SB-S1 exists to capture, plan-critic B2). Hashing the verbatim body makes the id true byte-identity: only an identical re-ingest collides; any real difference mints a distinct id.
|
||||
- `mintContentId` is content-keyed — distinct from `mintEntityId({kind,key})` which slugifies a *label* (a body must never be slugified). Pure, deterministic. SC3.
|
||||
- **Defence-in-depth (write-side, §3.3):** even with byte-identity hashing, `writePublished` compares bodies on any id-collision and never silently drops a differing body — so no hash strategy can cause data loss.
|
||||
|
||||
### 3.2 `ingest.ts` — `PublishedRecord` + constrained-header grammar (no YAML)
|
||||
```ts
|
||||
export interface PublishedRecord {
|
||||
id: string; // mintContentId(body) — 12 hex, the dedupe key + filename stem
|
||||
provenance: 'published'; // S1 only ever writes `published`; the type pins it
|
||||
published_date: string; // YYYY-MM-DD (operator-supplied or defaults to captured_at)
|
||||
captured_at: string; // YYYY-MM-DD (ingest run date; supplied by caller — pure fn)
|
||||
source: string; // 'manual' (default) | future connector token
|
||||
body: string; // the VERBATIM post text (may contain ] | quotes newlines ---)
|
||||
}
|
||||
```
|
||||
**File grammar** (`ingest/published/<id>.md`) — the EXACT byte layout, used identically by serialize and parse (fixes plan-critic B1):
|
||||
```
|
||||
id: <12hex>
|
||||
provenance: published
|
||||
published_date: YYYY-MM-DD
|
||||
captured_at: YYYY-MM-DD
|
||||
source: manual
|
||||
---
|
||||
<verbatim body>
|
||||
```
|
||||
- **Pinned separator (one string, both directions):** the five header lines, then a line that is exactly `---`, then the verbatim body. Concretely: `serialize` = `headerLines.join("\n") + "\n---\n" + body`. There is **NO** blank-line padding around the `---` and **NO** `# Published Record` title (both removed — they were the B1 contradiction). The header block is fixed at exactly 5 lines.
|
||||
- **`parsePublishedRecord(text)`** → split on the FIRST occurrence of `\n---\n`. Left = header (parse the 5 scalars by the in-tree scalar-regex idiom: `^id:\s*([0-9a-f]{12})$/m`, `^published_date:\s*(\d{4}-\d{2}-\d{2})$/m`, etc.). Right = `body`, **verbatim, byte-exact** (a body that itself contains a `\n---\n` is preserved — we split on the FIRST sentinel only, via `indexOf`, never a global split). **`provenance`** is parsed and asserted to equal `published` (a `published/` record with any other provenance is a corruption signal → throw/skip, never silently accepted — fixes plan-critic minor 11; the type pins `'published'`).
|
||||
- **`parse(serialize(rec)) === rec`** over all six fields incl. the body (SC2). **Edge cases the round-trip test MUST cover** (plan-critic B1): empty body (`""`), body = `"\n"`, body with trailing newlines, body starting with `---`, body containing `\n---\n` mid-text, and a body whose first line is header-shaped (e.g. `id: 0123456789ab`). Because the header is a fixed 5-line block terminated by the first `\n---\n`, a header-shaped body line cannot leak into the header (it sits after the sentinel). The only constraint on the body: none needed — it is captured byte-exact after the first sentinel.
|
||||
|
||||
### 3.3 `ingest.ts` — IO (impure; caller supplies the run date, like `scaffold.ts`)
|
||||
- **`writePublished(rec): {written: boolean; path: string; collision?: boolean}`** — `dataRoot('ingest/published')`, mkdir-recursive (**create-on-demand — no hard dependency on `brain init`**, SC1), path = `<id>.md`. **Collision-safe compare-then-skip (fixes plan-critic B2 defence-in-depth):** if `<id>.md` exists, READ it, parse its body: (a) body byte-identical to `rec.body` → true duplicate → `{written:false}` (idempotent, no clobber — the `initBrain` discipline); (b) body DIFFERS (astronomically rare 48-bit collision, or a truncation artifact) → write to a disambiguated path `<id>-2.md` (next free suffix) and return `{written:true, collision:true}` — **never silently drop a differing body.**
|
||||
- **`ingestText({body, source?, published_date?, captured_at})`** — `mintContentId(body)`, build the record (`provenance:'published'`; `published_date` defaults to `captured_at` when absent), `writePublished`. Returns the record + written flag.
|
||||
- **`scanInbox({captured_at})`** — read **top-level `*.md` files only**, skipping dotfiles (`.DS_Store` etc. — macOS), no recursion (fixes plan-critic minor 9), in `dataRoot('ingest/inbox')` (empty/absent dir → clean no-op, SC4); for each, `ingestText({body: fileContent, source:'manual', captured_at})`. **Non-destructive in S1:** inbox files are LEFT in place (idempotent via the published dedup — a re-scan re-skips); auto-move/delete of processed inbox files is deferred (noted §8). Returns `{processed[], skipped[]}`.
|
||||
- **`listPublished()`** — read `ingest/published/*.md`; **parse each inside a try/catch — a malformed/hand-dropped file is skipped+counted, never crashes the command** (`parsePublishedRecord` can throw via the provenance assertion / `normalizeProvenance`, id.ts:45; fixes plan-critic minor 8). Return `{records: [{id, provenance, published_date, firstLine}], skipped: number}` sorted by date. Surfaces `provenance` so the operator can eyeball that nothing `ai-draft` leaked (brief §5.1 D2).
|
||||
|
||||
### 3.4 `cli.ts` — add NET-NEW flag-routing infra (not just "extend dispatch" — fixes plan-critic major 6)
|
||||
The brain `cli.ts` currently has **no** `parseFlags`, **no** `--json`, **no** `today()` — it dispatches purely on positional `argv[2]` (just `init`). So this step ADDS that infrastructure (copy `parseFlags` from `specifics-bank/src/cli.ts:41`; a local `today()` — `ingest.ts` itself stays pure with a caller-supplied date, so the CLI's `today()` is the only clock and does not shadow `scaffold.ts:28`'s private one). The **existing `init` positional branch is preserved verbatim**, coexisting with the new flag-driven branches:
|
||||
- **`brain ingest --file <path> [--source <s>] [--date <YYYY-MM-DD>]`** — read the file, `ingestText`, print `Wrote ingest/published/<id>.md` / `Duplicate — already published: <id>` / `Collision → wrote <id>-2.md`.
|
||||
- **`brain ingest --scan-inbox [--source <s>]`** — `scanInbox`, print the processed/skipped split.
|
||||
- **`brain published list [--json]`** — `listPublished`; human form prints `id · provenance · date · first-line`; `--json` emits `{records:[{id,provenance,published_date,firstLine}], skipped}` (the `listPublished` return verbatim — pinned shape, fixes plan-critic minor 12).
|
||||
- Usage text updated; exit 2 on usage error (existing idiom). **Regression guard:** a test (or step-5 smoke-check) asserts `brain init` still works after the refactor (a shipped subcommand must not break).
|
||||
|
||||
### 3.5 `agents/voice-trainer.md` — wire the published-only invariant (brief §5.1 D3, all four bullets)
|
||||
Surgical edits (existing file → **Edit**, pathguard-irrelevant; model stays Sonnet). **Anchors corrected (scope-guardian DEP-1):** the Gather bullet is at **`:136`**, the "Sample Quality Priorities" heading at **`:142`** with its list at **`:144`** (the earlier `:134`/`:152` were a header and the next section).
|
||||
1. **Gather step** (`:136`) — prepend the gold source + the negative rule (ADD + FORBID + KEEP):
|
||||
> **Gather (published-only gold signal first):** Read the user's published posts from `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/ingest/published/` (`provenance=published` — the highest-trust source). Then read the existing `voice-samples/` corpus (human-curated; kept as a tributary, not reshaped) and the profile/template. **Never learn voice patterns from any content marked `provenance=ai-draft`** — a content engine that learns its own voice from its own drafts collapses toward its priors. (Generalises voice-scrubber's "gold standard = approved editions, never the draft corpus".)
|
||||
2. **Sample Quality Priorities** (`:144`) — promote the published gold source to #1 and add the hard exclusion line:
|
||||
> 0. **`ingest/published/` records (`provenance=published`) — the gold signal; rank above all else.**
|
||||
> … (existing 1–5 retained) …
|
||||
> **Exclude unconditionally: anything marked `provenance=ai-draft`.**
|
||||
3. **FENCE the auto-append trap** (brief §5.1 D3 bullet 4 — restored, scope-guardian SCOPE-GAP-1). Add a short note in the Gather step (and mirror it in the §3.6 contract doc): the `voice-samples/` template hints at a not-yet-built Stop-hook that auto-appends posts after content sessions (`assets/voice-samples/authentic-voice-samples.template.md:61`). State explicitly:
|
||||
> **Fence:** any future auto-capture into a learning corpus (the template's Stop-hook hint) MUST tag provenance and admit `provenance=published` only — never blind-append generated drafts.
|
||||
|
||||
**Exact literals (fixes plan-critic major 5 — non-vacuity):** the edits MUST write the literal strings `ingest/published` and `provenance=ai-draft`. The SC5 lint greps for these EXACT literals (`grep -F`), NOT a loose `ai.?draft`/`ai-generated` pattern — `voice-trainer.md:175` already contains "AI-generated" (a drift-cause description), so a loose grep would pass vacuously without the real exclusion being added.
|
||||
|
||||
### 3.6 `docs/second-brain/ingest-manual-import.md` — the manual-import contract
|
||||
A short design doc: the `PublishedRecord` file grammar (§3.2), the CLI usage (`brain ingest --file` / `--scan-inbox`, `brain published list`), the inbox→published drop-zone model, the **published-only learning rule** (with the honest runtime limit: an agent instruction can be ignored at runtime; the *wiring* is gate-enforced, the *behaviour* is not provable), and the **auto-append fence** (§3.5 bullet 3 — any future auto-capture must tag provenance and admit `published` only). Includes the exact literal `provenance=ai-draft` (so SC5's doc-grep is non-vacuous). Lives in `docs/` (not `references/`) → no reference-count guard.
|
||||
|
||||
### 3.7 `scripts/test-runner.sh` — new SC5 lint (Section 16c) + floor bumps
|
||||
- **New Section 16c "Brain Published-Only Invariant (SB-S1)"** placed after Section 16b (Brain Foundation), mirroring Section 17's structure: a non-vacuity self-test (POSITIVE/NEGATIVE heredocs) + the file checks. Greps use the **exact literals** `ingest/published` and `provenance=ai-draft` via `grep -F` / `grep -qiE` with the `provenance=` qualifier — never a loose `ai.?draft`/`ai-generated` pattern (plan-critic major 5). Three `pass()` lines (none in a loop):
|
||||
1. **self-test** — POSITIVE probe contains BOTH literals → must pass; NEGATIVE probes: one missing `ingest/published`, one missing `provenance=ai-draft`, **and one containing `AI-generated` but NOT `provenance=ai-draft`** (proves the check doesn't pass on voice-trainer's pre-existing drift prose) → each must fail. Non-vacuous like Sections 8/13/17.
|
||||
2. **`voice-trainer.md`** contains `ingest/published` AND `provenance=ai-draft`.
|
||||
3. **`docs/second-brain/ingest-manual-import.md`** contains `provenance=ai-draft` (the rule is documented).
|
||||
- **`BRAIN_TESTS_FLOOR=34` (`:714`) → the actual post-implementation count.** Pinned at step 8 by reading the brain suite's real `tests N` AFTER the new suites land (TDD determines it; do NOT guess). **Step 8 must first assert the brain suite actually RAN (`BR_TESTS` non-empty — i.e. tsx installed) before pinning**, else a warn-skip would let the gate go green while SB-S1's own tests never ran (plan-critic major 4).
|
||||
- **`ASSERT_BASELINE_FLOOR=75` (`:798`) → the actual printed `TOTAL_CHECKS`.** The floor counts **runtime** `PASS+FAIL` increments, not static `pass()` call-sites (the file has 72 static sites but a 75 floor because some checks loop). The three new Section 16c checks are non-looping, so the delta is +3 → 78 — **but pin it empirically: run the gate at step 8, read the printed `TOTAL_CHECKS`, set the floor to that exact number** rather than hand-arithmetic (plan-critic major 3). Update the inline history comment at `:796`.
|
||||
|
||||
## 4. Files created / edited
|
||||
|
||||
**Created:** `scripts/brain/src/ingest.ts` · `scripts/brain/tests/{ingest,publish}.test.ts` · `scripts/brain/fixtures/sample-inbox-post.md` · `docs/second-brain/ingest-manual-import.md`.
|
||||
|
||||
**Edited:** `scripts/brain/src/id.ts` (+normalizeContent/+mintContentId) · `scripts/brain/src/cli.ts` (+ingest/+published) · `agents/voice-trainer.md` (gather + priorities) · `scripts/test-runner.sh` (Section 16c + two floor bumps) · `docs/second-brain/architecture.md` (flip SB-S1 row → landed).
|
||||
|
||||
**Conditional (only if Q4 = bump to 0.5.1):** `CLAUDE.md` header + README badge + CHANGELOG + the marketplace catalog `ref` (via `catalog/scripts/release-plugin.mjs` — the polyrepo release path; tag + catalog bump atomic). Decided at "go".
|
||||
|
||||
## 5. TDD test plan (failing-first) → success criteria
|
||||
|
||||
| Test (`tests/*.test.ts`) | Asserts | SC |
|
||||
|---|---|---|
|
||||
| `ingest.test.ts` | `parsePublishedRecord(serializePublishedRecord(rec)) === rec` over the **B1 edge-case battery**: empty body, body=`"\n"`, trailing newlines, body starting with `---`, body containing `\n---\n` mid-text, header-shaped first body line (`id: 0123456789ab`), and a body with `]`/`|`/quotes; `mintContentId` deterministic (same body→same id; whitespace/case difference→DIFFERENT id, proving no normalize-collision); a `published/` record with non-`published` provenance throws (corruption assertion) | SC2, SC3 |
|
||||
| `publish.test.ts` | `LINKEDIN_STUDIO_DATA`=tmp, NO prior init → `writePublished` creates `ingest/published/<id>.md` with `provenance=published` + verbatim body (create-on-demand); re-ingest identical body = `{written:false}`, no dup; **id-collision with a DIFFERING body → disambiguated `<id>-2.md`, no data loss**; `scanInbox` processes top-level `*.md` only (skips a `.DS_Store` fixture), → published; empty/absent inbox = no-op; re-scan re-skips; `listPublished` skips a malformed dropped file without throwing; **`brain init` still works after the CLI refactor** (regression) | SC1, SC3, SC4 |
|
||||
| (gate) `scripts/test-runner.sh` Section 16c | self-test non-vacuous (incl. the `AI-generated`-but-not-`ai-draft` negative probe); `voice-trainer.md` + contract doc carry the exact literals | SC5 |
|
||||
| (gate) `scripts/test-runner.sh` | green; trends ≥24, specifics ≥28, contract ≥33, **brain ≥ new floor (brain suite confirmed RAN, not skipped)**; hooks pass; assertion floor = printed `TOTAL_CHECKS` | SC6 |
|
||||
| (n/a) | no new seam fn; `data-root.test.mjs` twin-sync untouched | SC6 |
|
||||
|
||||
Each test written **before** its module (iron law): red → implement → green.
|
||||
|
||||
## 6. Step sequence
|
||||
|
||||
**Discipline (plan-critic major 7):** each TDD step is a **Checkpoint** — commit after its cycle goes green (small, revertible commits). Each step carries an **On-failure** clause; two hard **circuit-breakers**: if step 3 (round-trip) cannot be made green after pinning the separator, HALT (the grammar is wrong — do not proceed); if step 8 (full gate) is not green, HALT (do not commit a red gate).
|
||||
|
||||
1. **Verify-first:** read `scripts/specifics-bank/src/bank.ts:52,57` → confirm the `specificId` hash idiom (mirror it as `mintContentId` on the VERBATIM body; do NOT reuse `normalizeContent`). Confirm `scripts/brain/node_modules/.bin/tsx` exists (else `npm install` in `scripts/brain` — the gate warn-skips a depless suite; "green" must not mean green-because-skipped). *On-failure: if tsx won't install, STOP and report — the slice cannot be gate-verified.*
|
||||
2. TDD `id.ts` `mintContentId` (`ingest.test.ts` id cases: determinism + whitespace/case → different id). → SC3. *Checkpoint.*
|
||||
3. TDD `ingest.ts` grammar parse/serialize over the B1 edge-case battery (`ingest.test.ts` round-trip). → SC2. *On-failure: CIRCUIT-BREAKER — if identity can't hold, the separator is wrong; halt and re-pin §3.2 before any further step.* *Checkpoint.*
|
||||
4. Write `fixtures/sample-inbox-post.md` (+ a `.DS_Store` + a malformed file in test temp dirs); TDD `ingest.ts` IO — `writePublished` (incl. collision→disambiguate), `ingestText`, `scanInbox` (filter), `listPublished` (try/catch) (`publish.test.ts`). → SC1, SC4. *Checkpoint.*
|
||||
5. Add the CLI flag-routing infra + `ingest`/`published` branches (preserve `init`); regression-test `brain init`; smoke-run against a temp `LINKEDIN_STUDIO_DATA` (ingest a file, scan an inbox, list). *On-failure: if `brain init` regresses, revert the CLI refactor and isolate.* *Checkpoint.*
|
||||
6. Edit `agents/voice-trainer.md` (`:136` gather + `:144` priorities + the `provenance=ai-draft` exclusion + the auto-append fence — exact literals). *Checkpoint.*
|
||||
7. Write `docs/second-brain/ingest-manual-import.md` (incl. the `provenance=ai-draft` literal). *Checkpoint.*
|
||||
8. Add `scripts/test-runner.sh` Section 16c (self-test + voice-trainer + doc checks, exact-literal greps); run the gate, **confirm the brain suite RAN (`BR_TESTS` non-empty)**, then pin `BRAIN_TESTS_FLOOR` to the real `tests N` and `ASSERT_BASELINE_FLOOR` to the printed `TOTAL_CHECKS`; re-run the FULL gate green. → SC5, SC6. *On-failure: CIRCUIT-BREAKER — a red gate is never committed.* *Checkpoint.*
|
||||
9. Flip `architecture.md` SB-S1 row → landed. **Exact edit (scope-guardian/plan-critic minor 10):** the row at `architecture.md:78` begins `| **SB-S1 — Ingest + gold signal** |` — change its trailing status cell to the "landed" marker matching SB-S0's row style (verify the exact current cell text at edit time). Persist brief + plan. **Commit; confirm before push (PUBLIC `open/` Forgejo), only inside the window; STATE.md stays gitignored.** Conditional version bump per Q4.
|
||||
|
||||
## 7. Scope fence (echo — SB-S1 does NOT)
|
||||
|
||||
No consolidation loop · **no write to `brain/profile.md`** (ingest writes only under `ingest/`) · no session-start wiring · no id threading into tributaries · no triple-post reconciliation · no `content-history.md` retirement · no tributary-schema edits (voice-samples store untouched; only `voice-trainer.md` instructions edited) · no LinkedIn API/connector · no new seam function · no state-file/two-roots reconciliation · no GUI.
|
||||
|
||||
## 8. Known limits / deferred (honest)
|
||||
|
||||
- **Invariant is wired, not behaviourally proven.** SC5 gate-enforces that `voice-trainer` *carries* the published-only wiring + exclusion; it cannot prove the agent *obeys* it at runtime (an instruction can be ignored). Stated in the contract doc. The strongest feasible enforcement short of running the agent.
|
||||
- **scan-inbox is non-destructive in S1** — processed inbox files are left in place (idempotent via published dedup). Auto-move/delete of processed inbox files (the full drop-zone lifecycle) is deferred; the operator clears inbox manually for now.
|
||||
- **`published_date` defaults to `captured_at`** when the operator doesn't pass `--date` (and for scan-inbox, which has no per-file date). A real published date is operator-supplied; SB-S2's loop can refine from analytics.
|
||||
- **`voice-samples/` stays un-provenanced.** S1 keeps it as a human-curated tributary and does not retro-tag it (that would be a tributary-schema change, §7). The published-only invariant bites on the NEW gold source + the ai-draft exclusion; voice-samples is trusted as human by its existing contract.
|
||||
- **No cross-silo id.** A published post ingested here mints a content-hash id but is NOT yet linked to its analytics/state/specifics records (SB-S3).
|
||||
|
||||
## 9. Risks
|
||||
|
||||
- **Grammar identity edge (B1):** a body containing `\n---\n` could confuse the header/body split — mitigated by splitting on the **first** sentinel only (`indexOf`, never global) + the fixed 5-line header; the B1 edge-case battery (§5) is the proof. Circuit-breaker at step 3.
|
||||
- **Silent data loss (B2) — CLOSED:** id is now `sha256(verbatim body)[:12]` (no `normalizeContent`), so structurally-different posts never collide; and `writePublished` compares bodies on any id-collision and disambiguates rather than skipping — so no hash strategy can drop a differing gold record. Tested in `publish.test.ts`.
|
||||
- **Gate "green" honest only with deps installed** — `npm install scripts/brain` (step 1) AND step 8 asserts `BR_TESTS` non-empty before pinning the floor; otherwise a warn-skip lets the gate go green while SB-S1's own tests never ran.
|
||||
- **Floor-bump arithmetic (corrected):** `ASSERT_BASELINE_FLOOR` counts **runtime** `PASS+FAIL`, not static `pass()` sites — so do NOT hand-compute 75+3; read the printed `TOTAL_CHECKS` at step 8 and set the floor to it. `BRAIN_TESTS_FLOOR` = the actual brain `tests N` after the new suites land. Both pinned empirically at step 8, never guessed.
|
||||
- **Scope creep into S2:** the temptation is to consume `published/` into `profile.md`. Fenced by §7 + the test plan (no test asserts profile mutation; `writePublished` targets only `ingest/`).
|
||||
160
docs/second-brain/plan-sb-s2.md
Normal file
160
docs/second-brain/plan-sb-s2.md
Normal file
|
|
@ -0,0 +1,160 @@
|
|||
# Implementation Plan — SB-S2 (Evolution loop)
|
||||
|
||||
> **Status:** review-hardened (`plan-critic` REPLAN 58→folded; `scope-guardian` ALIGNED). Ready for operator approval / "go".
|
||||
> **Brief:** `brief-sb-s2.md` (review-hardened). **Arc:** `architecture.md`. **Predecessors:** SB-S0/S1 (`scripts/brain/`, v0.5.1).
|
||||
> **Review delta (2026-06-23):** B1 conflict id → distinct (primary `mintEntityId(key)` + alt `mintContentId('observed-alt:'+key+'::'+value+'::'+date)`, no duplicate ids); B2 `--gather` reads published files directly for bodies (not `listPublished`), tributary reads CUT; B3 hook tests run SEPARATELY (`node --test hooks/scripts/__tests__/`) as an explicit step — NOT via test-runner.sh (which runs no hook test) → SC7 reworded, assertion floor +0; M4 new hook-test `.mjs` authored via Bash-heredoc (pathguard-safe under either guard); M5 scaffold-ensure runs UNCONDITIONALLY (fresh-install path); M6 `\\n` escaped-newline idiom; M8 per-step commit msg + revert target; minors: single-line value validation, static-decay-exempt, tsx fail-loud.
|
||||
> **Scope:** the deterministic consolidation engine + `brain consolidate` CLI (operator-invoked, operator-gated) + the session-start consolidation-due nudge + scaffold-ensure. **No journal-capture, no new agent, no profile.md reader, no supersede, no cross-silo threading, no auto-apply, no AI at session-start** (brief §7).
|
||||
|
||||
---
|
||||
|
||||
## 1. What SB-S2 delivers (recap)
|
||||
|
||||
A new PURE TS engine `scripts/brain/src/consolidate.ts` (`proposeDiff` + `applyDiff` over the SB-S0 two-layer `ProfileDoc`), a `brain consolidate` CLI (`--gather` / `--propose` / `--apply --confirm`) that closes the loop operator-gated, a brain data-root sidecar `brain/consolidation-state.json` for the last-run timestamp, and a zero-dep **consolidation-due nudge** + **scaffold-ensure** edit to `hooks/scripts/session-start.mjs`. Count-neutral (no agent/command/ref/skill). All TDD; gate stays green; floors rise.
|
||||
|
||||
## 2. Package additions (`scripts/brain/`) — extend the existing package
|
||||
|
||||
```
|
||||
scripts/brain/
|
||||
src/
|
||||
consolidate.ts # NEW — Candidate/ProfileDiff types + proposeDiff + applyDiff (pure) + consolidation-state IO
|
||||
cli.ts # EDIT — add `consolidate` dispatch (--gather/--propose/--apply)
|
||||
(types.ts, id.ts, profile.ts, ingest.ts, scaffold.ts, dataRoot.ts unchanged)
|
||||
tests/
|
||||
consolidate.test.ts # NEW — engine: per-rule (SC1a–g) + immutability + round-trip + idempotency
|
||||
consolidate-cli.test.ts # NEW — CLI gating + candidate-validation + sidecar (SC5)
|
||||
hooks/scripts/
|
||||
session-start.mjs # EDIT (existing → pathguard-safe) — scaffold-ensure + consolidation-due nudge
|
||||
__tests__/
|
||||
session-start-brain-consolidation.test.mjs # NEW — SC6 (mirror of session-start-trends-staleness.test.mjs)
|
||||
```
|
||||
|
||||
No new dep (engine is pure TS; the hook stays zero-dep). The brain `test` script already globs `tests/*.test.ts`.
|
||||
|
||||
## 3. Module designs
|
||||
|
||||
### 3.1 `consolidate.ts` — types
|
||||
```ts
|
||||
import { mintEntityId, mintContentId, slugify } from "./id.js";
|
||||
import { SCHEMA_VERSION } from "./types.js";
|
||||
import type { ProfileDoc, ProfileFact, Provenance } from "./types.js";
|
||||
|
||||
export interface Candidate {
|
||||
key: string; // the fact's stable label/topic/concept
|
||||
value: string; // single-line (no embedded newline/CR — profile grammar)
|
||||
provenance: Provenance; // published | human | ai-draft
|
||||
source: string; // e.g. "published:<id>" | "manual"
|
||||
observed_date: string; // YYYY-MM-DD
|
||||
}
|
||||
export interface ProfileDiff {
|
||||
additions: ProfileFact[]; // new dynamic facts (primary + conflict-alt)
|
||||
evidenceBumps: { id: string; newCount: number; last_seen: string }[];
|
||||
promotions: { id: string }[]; // dynamic→static (post-bump count ≥ N)
|
||||
conflicts: { primaryId: string; primaryValue: string; altId: string }[]; // both retained, distinct ids
|
||||
staleFlags: { id: string; last_seen: string; daysStale: number }[];
|
||||
}
|
||||
export interface ConsolidateOpts { promoteThreshold?: number; decayDays?: number } // defaults 3 / 90
|
||||
const OBSERVED_KIND = "observed";
|
||||
```
|
||||
|
||||
### 3.2 `consolidate.ts` — `proposeDiff` (pure; the classification heart) — id model fixes plan-critic B1
|
||||
**Id model (the B1 fix — no duplicate ids):**
|
||||
- A concept's **primary fact** id = `mintEntityId({ kind: OBSERVED_KIND, key })` (key-only → one primary per concept).
|
||||
- A conflict **alt fact** id = `mintContentId(\`observed-alt:${key}::${value}::${observed_date}\`)` (SB-S1's verbatim sha256[:12] — byte-distinct, so primary and alt never collide and two alts for the same concept differ by value/date). Matching always starts from the *candidate's* `key`, so we never need to recover a key from an existing fact.
|
||||
|
||||
```ts
|
||||
proposeDiff({ current, candidates, today, opts }): ProfileDiff
|
||||
```
|
||||
- `N = opts.promoteThreshold ?? 3`, `DECAY = opts.decayDays ?? 90`.
|
||||
- Build a `byId` lookup over `current.dynamic ∪ current.static`. (Folded `profile-field` static seeds use a DIFFERENT kind → their ids never collide with `observed` ids → immutable, SC1g.)
|
||||
- For each candidate:
|
||||
- `provenance === 'ai-draft'` → skip entirely (SC1b).
|
||||
- `primaryId = mintEntityId({kind:OBSERVED_KIND, key})`; `prev = byId.get(primaryId)`.
|
||||
- **no `prev`** → `additions` gets a new dynamic `ProfileFact` (`id:primaryId`, `evidence_count:1`, `first_seen:observed_date`, `last_seen:today`, `status:'active'`, `provenance`) (SC1a).
|
||||
- **`prev`, `prev.value === candidate.value`** → `evidenceBumps` (`newCount = prev.evidence_count+1`, `last_seen:today`); if `prev` is in `dynamic` AND `newCount ≥ N` → also `promotions[{id:primaryId}]` (SC1c/SC1d).
|
||||
- **`prev`, `prev.value !== candidate.value`** → CONFLICT (SC1e): build an alt fact `{id: mintContentId(\`observed-alt:${key}::${candidate.value}::${observed_date}\`), value:candidate.value, dynamic, evidence_count:1, …}`, push it to `additions`, record `conflicts[{primaryId, primaryValue: prev.value, altId}]`, and leave `prev` UNTOUCHED (no bump). **No supersede** (S3). (If the alt id already exists from a prior run → treat as the same-value bump path on the alt id, so re-running is idempotent.)
|
||||
- After the candidate pass: `staleFlags` = every **`current.dynamic`** fact whose `daysSince(last_seen) > DECAY` (SC1f). **Static facts are decay-exempt by design** (promoted = stable). Informational only — no mutation.
|
||||
- **Purity:** never mutate `current`/`candidates` (build new objects); SC2 asserts inputs structurally unchanged.
|
||||
|
||||
### 3.3 `consolidate.ts` — `applyDiff` (pure) + sidecar IO
|
||||
- `applyDiff(current, diff): ProfileDoc` — returns a NEW doc: append `additions` (which already includes both new primary facts AND conflict alt facts) to `dynamic`; apply `evidenceBumps` (update count+last_seen on the fact with the matching id); move `promotions` facts from `dynamic`→`static`. `conflicts[]` is informational (the alt fact is already in `additions`); `staleFlags` cause NO mutation. **Id-uniqueness invariant:** because primary ids (`mintEntityId(key)`) and alt ids (`mintContentId(...)`) are byte-distinct, no two facts share an id — so the bump/promote target is unambiguous and the doc stays well-formed. Output round-trips through `parseProfile`/`serializeProfile` (SC3).
|
||||
- **Sidecar IO** (the only impure bit, kept here for cohesion): `readConsolidationState(): {last_run: string|null}` + `writeConsolidationState(date)` over `dataRoot('brain/consolidation-state.json')` (`{ "last_run": "YYYY-MM-DD" }`). Tiny JSON; absent → `{last_run:null}`.
|
||||
|
||||
### 3.4 `cli.ts` — `consolidate` dispatch (extend the SB-S1 router)
|
||||
Add `command === "consolidate"` with a `--gather` / `--propose` / `--apply` mode (the parseFlags idiom already present):
|
||||
- **`--gather [--json]`** — read-only. Reads the published gold corpus **directly** (`readdir(dataRoot('ingest/published'))` → `parsePublishedRecord` each — `listPublished()` returns no body/`captured_at`, so it can't feed extraction; fixes plan-critic B2), filters to records with `published_date > readConsolidationState().last_run` (null → all), and prints a digest of `{id, published_date, body}` per new record + the current `brain/profile.md` facts (parsed), for the **invoking session** to turn into a `Candidate[]` JSON. **Tributary newest-timestamp reads are CUT** (informational-only, underspecified — the published corpus is the S2 signal; plan-critic M12). Writes nothing.
|
||||
- **`--propose --candidates <file.json>`** — read + **validate** each candidate against the `Candidate` shape: every item has key/value/provenance∈{human,published,ai-draft}/source/observed_date, AND **`key`/`value` contain no newline or carriage-return** (the profile grammar is single-line — a `\n` would corrupt serialization; plan-critic M9). Any violation → non-zero exit, no write (SC5). `proposeDiff` over the current profile. Write `brain/pending-diff.json` (the typed `ProfileDiff`) AND `brain/pending-diff.md` (operator-readable: sections Additions / Evidence-bumps / Promotions / ⚠ Conflicts / Stale, neutral framing). Print both paths. Does NOT touch `profile.md`.
|
||||
- **`--apply --diff <path> --confirm`** — require `--confirm` (else refuse). Read the JSON diff, `parseProfile(brain/profile.md)`, `applyDiff`, write `brain/profile.md` (`serializeProfile`), then `writeConsolidationState(today())`. The ONLY path that writes the profile.
|
||||
- Usage text + exit 2 on misuse. `init`/`ingest`/`published` branches unchanged (regression-guarded).
|
||||
|
||||
### 3.5 `hooks/scripts/session-start.mjs` — scaffold-ensure + consolidation-due nudge (Edit, zero-dep)
|
||||
- Add `readdirSync` to the existing `node:fs` import (line 5).
|
||||
- **Helpers (twins of `trendsNewestCapture`, lines 38–52):**
|
||||
- `brainLastRun(path)` — `JSON.parse(readFileSync)` of `consolidation-state.json` → `last_run` or null (try/catch → null).
|
||||
- `countPublished(dir)` — `existsSync(dir) ? readdirSync(dir).filter(f=>f.endsWith('.md')).length : 0` (no parse — cost-bounded).
|
||||
- **Scaffold-ensure — UNCONDITIONAL** (fixes plan-critic M5: the `reminders`/context block runs only inside `if (existsSync(STATE_FILE))`, so a fresh install (else branch ~line 432) would never scaffold/nudge). Place it next to the M0 migration block (~line 73), BEFORE the state-file branch, so it always runs: `for (const d of ['brain/journal','ingest/inbox','ingest/published']) mkdirSync(join(getDataRoot(''),d),{recursive:true})` (idempotent, zero-dep; confirm `getDataRoot('')` returns the bare data root — step 1). The optional "run `brain init` to seed your profile" nudge is appended to **`context` directly** (not `reminders`) right after scaffold-ensure when `!existsSync(join(getDataRoot('brain'),'profile.md'))`, so it survives the no-state-file path.
|
||||
- **Consolidation-due nudge** (inside the reminders block, beside the trends nudge ~line 342 — an established user with published content has a state file, so the reminders path is correct here). **Use the `\\n` escaped-newline idiom** (the file un-escapes once via `context.replace(/\\n/g,'\n')` at line 479 — a raw `\n` would mis-render; plan-critic M6):
|
||||
```js
|
||||
const published = countPublished(join(getDataRoot('ingest'), 'published'));
|
||||
const lastRun = brainLastRun(join(getDataRoot('brain'), 'consolidation-state.json'));
|
||||
const daysSinceRun = daysSince(lastRun);
|
||||
if (published > 0 && (lastRun === null || (daysSinceRun !== null && daysSinceRun >= 7))) {
|
||||
const since = lastRun === null ? 'never run' : `${daysSinceRun}d ago`;
|
||||
reminders += `- ${published} published post(s) captured, last brain consolidation ${since}. Run \\\`brain consolidate\\\` to evolve your profile.\\n`;
|
||||
}
|
||||
```
|
||||
Null-safe + fires only when published records exist (a never-used brain never nags — the trend-nudge discipline). **No `profile.md` parse.**
|
||||
|
||||
## 4. Files created / edited
|
||||
|
||||
**Created:** `scripts/brain/src/consolidate.ts` · `scripts/brain/tests/{consolidate,consolidate-cli}.test.ts` · `hooks/scripts/__tests__/session-start-brain-consolidation.test.mjs` · `docs/second-brain/consolidation-loop.md`.
|
||||
|
||||
**Edited:** `scripts/brain/src/cli.ts` (+consolidate) · `hooks/scripts/session-start.mjs` (scaffold-ensure + nudge + `readdirSync` import) · `scripts/test-runner.sh` (BRAIN floor + assertion floor) · `docs/second-brain/architecture.md` (flip SB-S2 row).
|
||||
|
||||
**Conditional (Q2 = bump to 0.5.2):** `plugin.json` + README badge + CLAUDE.md header + CHANGELOG; polyrepo release via `release-plugin.mjs` (tag + catalog), pushed per window + confirm.
|
||||
|
||||
## 5. TDD test plan (failing-first) → success criteria
|
||||
|
||||
| Test | Asserts | SC |
|
||||
|---|---|---|
|
||||
| `consolidate.test.ts` | add / reject-ai-draft / evidence-bump / promote-at-3 / **conflict-keep-both with DISTINCT primary+alt ids, old fact untouched** / decay-flag (dynamic only) / **static-decay-exempt** / folded-immutable; input immutability; `applyDiff∘proposeDiff` round-trips via parse/serialize; idempotent (re-run → bumps not dup facts, alt-id stable) | SC1a–g, SC2, SC3, SC4 |
|
||||
| `consolidate-cli.test.ts` | temp-dir: `--gather` reads published bodies (filtered by `published_date>last_run`) + leaves `profile.md` byte-unchanged; `--propose` leaves profile.md unchanged, writes `pending-diff.{md,json}`, REJECTS a malformed candidate file AND a candidate whose value contains `\n` (non-zero, no write); `--apply --confirm` writes profile.md + `consolidation-state.json`; `--apply` without `--confirm` refuses; `init`/`ingest`/`published` still work | SC5 |
|
||||
| `session-start-brain-consolidation.test.mjs` (authored via **Bash-heredoc** — pathguard-safe) | subprocess + isolated HOME/`LINKEDIN_STUDIO_DATA` (trends-staleness harness): nudge fires (rendered on its own line) when published records exist + last_run stale/absent; silent when brain absent / no published; scaffold-ensure runs on the fresh-install (no-state-file) path; hook still emits valid JSON | SC6 |
|
||||
| (separate) `node --test hooks/scripts/__tests__/` | the new SC6 test + all existing hook tests pass — **run as an explicit step, NOT via test-runner.sh** (which runs no hook test; plan-critic B3) | SC6 |
|
||||
| (gate) `scripts/test-runner.sh` | green; floors hold; brain ≥ new floor; counts unchanged; `compile-hooks.py --check` clean if hooks.json regenerated. (No new test-runner section → assertion floor +0.) | SC7 |
|
||||
|
||||
Each test written **before** its module (iron law). **Circuit-breakers:** halt if the engine round-trip (SC3) can't be made green, or if the full gate (step 8) is red.
|
||||
|
||||
## 6. Step sequence (each step: TDD red→green; Checkpoint = commit with the named message; On-failure → revert the named target + halt + report)
|
||||
|
||||
1. **Verify-first:** re-confirm `proposeDiff` inputs against `types.ts` (`ProfileFact`/`ProfileDoc`/`Provenance`) + `mintEntityId`/`mintContentId`/`slugify` signatures + `daysSince`/`getDataRoot('')` behavior in session-start.mjs (does the empty subdir return the bare data root?). **tsx fail-loud:** assert `scripts/brain/node_modules/.bin/tsx` present (the gate brain section *warn-skips* if absent — for this slice that would let the gate go green with SB-S2's tests never run; `npm install` and FAIL if still absent; plan-critic M11). *On-failure: stop, report — the slice cannot be verified.*
|
||||
2. TDD `consolidate.ts` types + `proposeDiff` per-rule (`consolidate.test.ts`, SC1a–g + SC2; incl. the distinct primary/alt id assertion). *Checkpoint: `feat(linkedin-studio): SB-S2 consolidation engine — proposeDiff classification [skip-docs]`. On-failure: revert `scripts/brain/src/consolidate.ts` + its test.*
|
||||
3. TDD `applyDiff` + round-trip + idempotency (SC3/SC4). *Circuit-breaker: if the parse/serialize round-trip can't be made green, the id/shape is wrong — halt.* *Checkpoint: fold into step-2 commit or `feat(...): SB-S2 applyDiff + round-trip [skip-docs]`.*
|
||||
4. TDD sidecar IO (`readConsolidationState`/`writeConsolidationState`) — fold into the consolidate-cli temp-dir tests. *Checkpoint with step 5.*
|
||||
5. Extend `cli.ts` `consolidate` (gather/propose/apply); TDD `consolidate-cli.test.ts` (SC5) incl. candidate validation (shape + no-newline), no-confirm refusal, `--gather` body-read, `init`/`ingest`/`published` regression. *Checkpoint: `feat(linkedin-studio): SB-S2 brain consolidate CLI [skip-docs]`. On-failure: revert `cli.ts` + the cli test.*
|
||||
6. Edit `session-start.mjs` (import `readdirSync`; UNCONDITIONAL scaffold-ensure near the migration block; consolidation-due nudge in the reminders block, `\\n` idiom); author `session-start-brain-consolidation.test.mjs` **via Bash-heredoc** (pathguard-safe); run `node --test hooks/scripts/__tests__/` (the WHOLE hook suite, since the gate doesn't — SC6 + no hook regression). If `hooks.json` shape unchanged (only script body edited) no recompile; else `compile-hooks.py` + verify `--check`. *Checkpoint: `feat(linkedin-studio): SB-S2 session-start scaffold-ensure + consolidation nudge [skip-docs]`. On-failure: `git checkout hooks/scripts/session-start.mjs` (do NOT leave the hook emitting invalid JSON) + halt.*
|
||||
7. Write `docs/second-brain/consolidation-loop.md` (CLI usage, engine rules, the candidate-file schema as the session↔engine contract, the operator-gate, honest limits incl. no-reader-until-S3 + the loop's value-depends-on-extraction caveat). *Checkpoint: `docs(...)`.*
|
||||
8. Bump `scripts/test-runner.sh` `BRAIN_TESTS_FLOOR` to the real `tests N` (assert the brain suite RAN — `BR_TESTS` non-empty, not warn-skipped). **`ASSERT_BASELINE_FLOOR` unchanged (+0): SB-S2 adds NO new test-runner section** (the hook test runs separately, the new TS tests run inside the existing brain `npm test` which contributes one unchanged `pass()` line) — confirm by reading the printed `TOTAL_CHECKS` is unchanged. Run the FULL gate green AND the hook suite green. *Circuit-breaker: no red gate committed.* *Checkpoint: `chore(...): SB-S2 gate brain floor`.*
|
||||
9. Flip `architecture.md` SB-S2 row → landed (exact edit: the `| **SB-S2 — Evolution loop** |` row → add the `✅ *landed*` marker, matching SB-S0/S1 row style); persist brief + plan. **Commit; confirm before push (PUBLIC `open/`), window; STATE.md gitignored.** Conditional 0.5.2 release per Q2 (tag + catalog via `release-plugin.mjs`).
|
||||
|
||||
## 7. Scope fence (echo — SB-S2 does NOT)
|
||||
|
||||
No journal-capture · no new agent (session extracts) · no `profile.md` reader (S3) · no supersede (keep-both only; S3) · no cross-silo id threading (S3) · no `content-history.md` retirement · no AI at session-start (deterministic nudge only) · no auto-apply (only `--apply --confirm` writes profile.md) · no new hook `.mjs` (edit existing session-start.mjs) · no new seam function · no connector (S4) · no GUI.
|
||||
|
||||
## 8. Known limits / deferred (honest)
|
||||
|
||||
- **The loop's VALUE depends on the session's candidate extraction** — the engine guarantees only the mechanics (threshold/conflict/decay/provenance-gate). Garbage candidates → a garbage diff (but the operator gate + the candidate-schema validation catch shape errors, not quality). Documented in `consolidation-loop.md`.
|
||||
- **profile.md has no reader until S3** — S2 evolves an artifact nothing consumes yet; the value is deferred compounding. Stated honestly.
|
||||
- **Evidence inflates only on genuinely-new candidates** — `--gather` is delta-gated (published since `last_run`), so a re-run after `--apply` surfaces nothing; the engine itself would bump again given the same candidate, but the loop never re-feeds one. Documented.
|
||||
- **No supersede / no stale-fact demotion in S2** — conflicts keep both; stale facts are flagged, never auto-removed (operator/ S3).
|
||||
- **Session-start nudge is new-published-count only** — no per-fact staleness at session-start (no zero-dep profile parser).
|
||||
|
||||
## 9. Risks
|
||||
|
||||
- **Duplicate-id corruption (was plan-critic B1) — CLOSED:** primary ids (`mintEntityId(key)`) and conflict-alt ids (`mintContentId('observed-alt:'+key+'::'+value+'::'+date)`) are byte-distinct, so no two facts share an id; the bump/promote target is unambiguous and `parseProfile`/`serializeProfile` stays well-formed. Pinned by SC1e (distinct ids) + SC3 (round-trip) + SC4 (idempotency).
|
||||
- **`--gather` data source (was plan-critic B2) — CLOSED:** gather reads the published files directly (`parsePublishedRecord` → body), not `listPublished()` (which lacks body/`captured_at`); filtered by `published_date > last_run`.
|
||||
- **Hook tests not gate-run (was plan-critic B3):** `scripts/test-runner.sh` runs NO hook test, so SC6 is verified by an explicit `node --test hooks/scripts/__tests__/` step (the whole suite), NOT the structure gate. SC7 reworded; no false "hook tests pass at the gate" claim. **Flagged (not done):** wiring the hook suite into test-runner.sh is a separate hardening opportunity.
|
||||
- **Engine rule ambiguity (conflict vs bump):** deterministic value-equality decides bump-vs-conflict; keep-both is the safe default; each rule pinned by its own SC1x test before the CLI wires it.
|
||||
- **Sidecar reachability (was the brief-review blocker) — CLOSED:** both `--apply` (dataRoot) and session-start (getDataRoot) resolve the SAME data root → the sidecar closes the loop; SC6 sets HOME + data root to separate temp dirs (the trends-test pattern) to prove it.
|
||||
- **Zero-dep hook discipline:** no tsx, no profile parse, bounded readdir, `\\n` idiom; scaffold-ensure runs unconditionally (fresh-install path); SC6 asserts valid JSON output + the nudge rendering.
|
||||
- **Gate floors:** BRAIN floor = actual post-impl count (assert suite RAN, not warn-skipped — step 1 fail-loud on missing tsx); `ASSERT_BASELINE_FLOOR` +0 (no new test-runner section). Confirm `TOTAL_CHECKS` unchanged at step 8.
|
||||
- **Pathguard:** session-start.mjs is an existing file → Edit (safe). The new hook TEST `.mjs` under `hooks/scripts/__tests__/` is authored via **Bash-heredoc** — safe whether or not the active Category-6 guard matches `__tests__/` (the verified llm-security `.mjs` guard matches `hooks/scripts/.*\.mjs`; heredoc sidesteps it regardless).
|
||||
83
docs/second-brain/plan-sb-s3a.md
Normal file
83
docs/second-brain/plan-sb-s3a.md
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
# Plan — SB-S3a: wire the first profile.md READER (`strategy-advisor`)
|
||||
|
||||
> Brief: `docs/second-brain/brief-sb-s3a.md` (APPROVE, FIXes folded). Slice: SB-S3a.
|
||||
> Operator confirmed: **`strategy-advisor`**, READ-only, **manual SC4/SC5** (no behavioural harness).
|
||||
> Order is TDD: the failing lint check lands BEFORE the agent edit (iron law).
|
||||
|
||||
## Goal
|
||||
|
||||
`strategy-advisor` consumes `brain/profile.md` as *evidence-to-test*, guarded by a deterministic wiring lint. No write path, no parser, no tributary/hook change. Smallest end-to-end proof that the second brain feeds generation.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | Why |
|
||||
|------|--------|-----|
|
||||
| `scripts/test-runner.sh` | **NEW Section 16d** (2 unconditional checks) + bump `ASSERT_BASELINE_FLOOR` 78→80 + extend floor-history comment + header enumeration | SC1/SC2/SC6 — the TDD anchor |
|
||||
| `agents/strategy-advisor.md` | add `brain/profile.md` to Step 0 Load Context + a short consumption subsection (anti-sycophancy literal + graceful-absence) | SC1/SC2/SC5 — the wiring itself |
|
||||
| `docs/second-brain/consolidation-loop.md` | **append** an "SB-S3a landed" reader-status line (do NOT rewrite the existing :67 contract prose) | doc reconciliation |
|
||||
|
||||
**Not touched (scope fence):** any `scripts/brain/**` TS · any `hooks/scripts/**` · any tributary JSON · `architecture.md`/`brief.md` (arc-level "motor-only until S3" stays true until all of S3 lands) · `content-optimizer.md` (second reader, not S3a) · CLAUDE.md agent table (responsibility text unchanged — the count/model lint is unaffected; a note is optional and deferred to avoid churn).
|
||||
|
||||
## Step 1 — (RED) Lint: Section 16d + floor bump
|
||||
|
||||
Insert a new section after Section 16c (`test-runner.sh:780`, before Section 17 at `:782`), modelled byte-for-byte on 16c's idiom (literals + `*_wired()` + non-vacuity self-test + real-file grep with `grep -qF`):
|
||||
|
||||
- **Literals:** `READER_LIT='brain/profile.md'`, `ANTISYC_LIT='evidence to TEST'`.
|
||||
- **`reader_wired()`** — text wired iff BOTH literals present (`grep -qF` each; echo twice as 16c does).
|
||||
- **Check A (self-test, unconditional):** a fully-wired probe (`"reads brain/profile.md as evidence to TEST"`) MUST be detected; three under-wired probes MUST be rejected:
|
||||
1. `"reads brain/profile.md but never frames how to weigh it"` (has reader lit, missing anti-syc)
|
||||
2. `"treats facts as evidence to TEST but names no source file"` (has anti-syc, missing reader lit)
|
||||
3. `"reads profile/user-profile.md and tests the evidence"` (decoy: legacy path + lowercase "evidence" — must NOT match either exact literal)
|
||||
→ `pass`/`fail` "profile-reader self-test: full wiring detected; 3 under-wired forms rejected".
|
||||
- **Check B (real-file grep, unconditional):** `grep -qF "$READER_LIT" agents/strategy-advisor.md && grep -qF "$ANTISYC_LIT" agents/strategy-advisor.md` → `pass` "strategy-advisor.md wired to brain profile reader (names '…profile.md', frames 'evidence to TEST')" else `fail`.
|
||||
- **Header enumeration (`:19-35`):** add a sentence — "the brain profile-reader guard (SB-S3a: `strategy-advisor` names `brain/profile.md` AND carries the anti-sycophancy literal `evidence to TEST`, with a non-vacuity self-test) in Section 16d". Range "Sections 8–18" unchanged (16d is in range).
|
||||
- **Floor (`:849-865`):** `ASSERT_BASELINE_FLOOR=78` → `80`; extend the history comment: "+2 for SB-S3a's two UNCONDITIONAL Section-16d checks (profile-reader self-test + strategy-advisor wiring grep) = 80." Both new checks are deps-free → they lift the deps-absent minimum, so the +2 bump is correct (NOT pinned to deps-present TOTAL).
|
||||
|
||||
- **Non-collisions (verified by plan-critic):** the new prose carries no algorithm-magnitude token (Section 8 stat-consistency safe) and no `Significant?(` column (Section 11 render-chain safe); `brain` is absent from Section 13's `BARE_DATA` regex (`test-runner.sh:514`) AND the edit uses the exempt `${LINKEDIN_STUDIO_DATA:-…}` external form → Section 13 data-dir guard safe. Agent count (19) + `strategy-advisor` model/frontmatter unchanged → Sections 2/9/10 unaffected.
|
||||
|
||||
**RED gate:** `bash scripts/test-runner.sh` → Check A passes, **Check B FAILS** (agent not yet wired), floor check passes (TOTAL 95 ≥ 80) → exit 1. This is the required failing test.
|
||||
|
||||
## Step 2 — (GREEN) Wire `strategy-advisor.md`
|
||||
|
||||
1. **Step 0 Load Context** (`strategy-advisor.md:41-47`): insert one line **inside** the fenced code block, immediately BEFORE the closing ``` fence (currently `:47`) — i.e. the new path becomes `:47` and the fence shifts to `:48`. Do NOT place it after the fence (that breaks the code block). The line:
|
||||
`${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/brain/profile.md → evolving second-brain profile (consolidated, evidence-weighted facts about the user)`
|
||||
2. **New subsection** immediately after the Load Context block (after `:49`), e.g. `### Consuming the evolving profile (brain/profile.md)`:
|
||||
- Two layers: `## Static` (stable, high-confidence) vs `## Dynamic` (emerging); each fact carries `evidence_count` + `last_seen` — weight by strength/recency.
|
||||
- **Anti-sycophancy (binding):** "Treat every profile fact as **evidence to TEST**, not flatter — counter-pressure it against analytics/state each time (e.g. *the profile says you lean X, but your imports show Y — test that*). The profile informs; it never dictates or flatters." ← carries the exact literal `evidence to TEST`.
|
||||
- **Graceful absence:** "If `brain/profile.md` is missing or empty (fresh installs have no brain yet), proceed silently on the other context sources — no error, no note about a missing profile."
|
||||
|
||||
**GREEN gate:** `bash scripts/test-runner.sh` → Check A + Check B pass, FAIL=0, **95/0/0**, brain floor 82 untouched.
|
||||
|
||||
## Step 3 — Doc reconciliation
|
||||
|
||||
`consolidation-loop.md:67-68` currently reads *"`brain/profile.md` has no reader yet … wiring content agents/commands to consume it is SB-S3."* — which becomes **false** the moment S3a lands a reader. **No lint guards `docs/`** (Section 11 scans `references/ commands/ skills/ hooks/prompts/ agents/ assets/` only, and `PROP_FORBIDDEN` matches only the significance-verdict column — verified by plan-critic; `grep consolidation-loop test-runner.sh` = 0 hits). So **EDIT the bullet** (it lives under "## Honest limits" — keep it a limit) to the true partial state, e.g.:
|
||||
"**`brain/profile.md` has one reader as of SB-S3a.** S2 evolved the profile motor-only; SB-S3a wired the first consumer — `strategy-advisor` reads it as *evidence-to-test*. Broader consumption (more content agents, a hook-level digest) remains later S3 work, and the profile is still mutated only via `brain consolidate --apply --confirm`."
|
||||
This removes the stale claim rather than appending a self-contradicting line beneath it.
|
||||
|
||||
## Step 4 — Behavioural verification (manual, recorded — SC4/SC5)
|
||||
|
||||
Agent-prompt behaviour is not unit-testable (operator-accepted). Honest procedure, result recorded in STATE/changelog at land:
|
||||
- **SC4 (graceful absence):** with no/empty `brain/profile.md` at the data-root, the wiring degrades silently — verified by inspection of the subsection's graceful-absence clause + (best-effort) a strategy-advisor run on the empty real root → no missing-file noise.
|
||||
- **SC5 (read-back works):** seed a temp profile (e.g. `LINKEDIN_STUDIO_DATA=<tmp> brain init` + a couple of hand-written facts) and run the agent → confirm ≥1 fact surfaces as tested evidence, counter-pressured, not parroted. If the harness cannot fully exercise a per-call data-root, record that honestly and fall back to inspection of the wiring + framing (do NOT overclaim a behavioural pass we did not run).
|
||||
|
||||
## Step 5 — Land
|
||||
|
||||
STATE "Telling" + "👉 NESTE" updated (S3a done → S3b/c/d remain); commit (Conventional, `[skip-docs]` on the code/feat parts; brief+plan+consolidation-loop are docs → judge suffix per the mixed change); **push only inside the window** (`date '+%u %H:%M'` first; `origin` is the PUBLIC `open/` remote → confirm before push). No version bump (S3a is additive within v0.5.2 dev; a release bump is a separate decision).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|----|-------|---------|----------|
|
||||
| SC1+SC2 | reader wired + literal | `bash scripts/test-runner.sh` | Section 16d Check B passes |
|
||||
| SC3 | gate green, brain floor intact | same run | 95/0/0; `BRAIN_TESTS_FLOOR=82` unchanged; hook suite (separate runner) untouched |
|
||||
| SC6 | assertion floor honoured | same run | "assertion-count anti-erosion: 95 … >= floor 80" passes |
|
||||
| (red proof) | failing-test-first | gate run BETWEEN Step 1 and Step 2 | exit 1, exactly Check B fails |
|
||||
| SC4 | graceful absence | manual run, empty root | clean output, no missing-file noise |
|
||||
| SC5 | read-back | manual run, seeded profile | ≥1 fact surfaced as evidence-to-test (or honest fallback recorded) |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — vacuous lint.** Mitigated by the 16c-style non-vacuity self-test with the legacy-path/lowercase decoy.
|
||||
- **R2 — floor false-fail on fresh clone.** Mitigated: both new checks are deps-free (lift the deps-absent minimum), so +2 → 80 is the correct floor; not pinned to deps-present TOTAL.
|
||||
- **R3 — behavioural overclaim.** Mitigated by Step 4's honest-fallback clause (verifiseringsplikt): never report a behavioural pass we did not run.
|
||||
- **R4 — stale doc claim.** `consolidation-loop.md:67` ("no reader yet") becomes false once S3a lands; **no automated guard protects `docs/`** (Section 11 scans non-docs dirs + only the significance column — verified). Mitigated by EDITING :67 to the true partial state (one reader, broader consumption deferred) within the "Honest limits" framing — by discipline, not by a blind append.
|
||||
101
docs/second-brain/plan-sb-s3b.md
Normal file
101
docs/second-brain/plan-sb-s3b.md
Normal file
|
|
@ -0,0 +1,101 @@
|
|||
# Plan — SB-S3b: the supersede arm
|
||||
|
||||
> Brief: `docs/second-brain/brief-sb-s3b.md`. Slice: SB-S3b.
|
||||
> **Operator-confirmed (2026-06-23):** temporal-update→supersede ONLY · winner→dynamic (re-earn) · minimal `supersedes?: string` · retain superseded as audit.
|
||||
> **Light-Voyage:** brief-review APPROVE-WITH-FIXES (8 folded); plan-critic **REVISE → 2 BLOCK + 6 FIX folded below**; scope-guardian **ALIGNED** (0 creep/0 gap, 1 minor SC9 reconciliation folded).
|
||||
> Order is TDD: failing engine unit tests land BEFORE the engine logic (iron law); each rule pinned before the CLI is wired (`brief-sb-s2.md:115`).
|
||||
|
||||
## Goal
|
||||
|
||||
The consolidation engine can **retire** a fact via an operator-confirmed, explicitly-signalled supersede: a temporal-update candidate flips the stale fact to `status: "superseded"` (re-minted to an archival id, **replaced in place**, retained as audit) and installs the new winner under the canonical key-id. Pure, idempotent, deterministic; rides the existing `--propose`/`--apply --confirm` gate; superseded facts never leak back into gather or decay; no grammar change; fully unit-testable.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | SC |
|
||||
|------|--------|-----|
|
||||
| `scripts/brain/src/consolidate.ts` | `Candidate` (+`supersedes?: string`, 33-39) · `ProfileDiff` (+`supersedes: SupersedeOp[]`, 41-47; `SupersedeOp` carries `provenance`) · `proposeDiff` routing fork before keep-both (~120), **value-guarded** + intra-batch `touched`-guard · `applyDiff` **replace-in-place** supersede branch, state-checked, ordered (144-162) · decay scan excludes `superseded` (132-134) · local `archivalId()` helper · engine docstring `:11` | SC1–SC5,7,8,11,12 |
|
||||
| `scripts/brain/src/cli.ts` | `renderDiffMd` (+`## Supersessions (old → new)`, appended last via `section()`, 132-145) · `validateCandidates` (+optional single-line `supersedes`, 147-157) · **`--gather` profileFacts filtered to `status === "active"`** (`:174`) | SC6, SC10 |
|
||||
| `scripts/brain/tests/consolidate.test.ts` | unit tests SC1, SC2, SC3a, SC3b, SC4, SC5, SC7, SC8, SC11, SC12 | engine |
|
||||
| `scripts/brain/tests/consolidate-cli.test.ts` | CLI tests SC6 (render + fidelity non-vacuity + `--confirm` gate), SC10 (gather active-only) | CLI |
|
||||
| `scripts/test-runner.sh` | bump `BRAIN_TESTS_FLOOR` 82→(82+N) + breakdown comment (`:716`); **`ASSERT_BASELINE_FLOOR` UNCHANGED at 80** | SC9 |
|
||||
| `docs/second-brain/consolidation-loop.md` | reconcile "no supersede" claims (`:34`, `:71`) to true state + honest-limit line (engine guarantees mechanics; temporal-update-vs-contradictory is session+operator judgement — the operator gate is the only classification net) | doc |
|
||||
|
||||
**Not touched (scope fence):** `types.ts` (`FactStatus` already has `superseded`) · `profile.ts` (grammar already round-trips `superseded`) · `id.ts` (reuse `mintEntityId`/`mintContentId`; `archivalId` is local to `consolidate.ts`) · `ingest.ts`/`scaffold.ts`/`dataRoot.ts` · `hooks/scripts/**` · `agents/**` (S3a reader unchanged) · `architecture.md` (its `:69` design promise is fulfilled; `:79` "S2 had no supersede" stays historically true — arc-status note deferred, not load-bearing) · condition-dependent/distractor branches (Q-SCOPE).
|
||||
|
||||
## Step 0 — pre-flight (verify baseline, no edits)
|
||||
|
||||
`(cd scripts/brain && npm install)` if `node_modules` absent. Confirm `(cd scripts/brain && npm test)` = **82/82** and `bash scripts/test-runner.sh` green (95/0/0, `BRAIN_TESTS_FLOOR=82`, `ASSERT_BASELINE_FLOOR=80`). Confirm `renderDiffMd`/`--gather`/CLI-propose coverage lives in `consolidate-cli.test.ts` (host SC6/SC10 there). Re-read `applyDiff` (`:144-162`) to see how promotions are removed via `filter` (`:156`) — the supersede branch mirrors that.
|
||||
|
||||
## Step 1 — (RED) types + failing tests
|
||||
|
||||
1. **Type scaffolding only** (suite still compiles, existing 82 green):
|
||||
- `Candidate` (`:33-39`): `supersedes?: string`.
|
||||
- `ProfileDiff` (`:41-47`): `supersedes: SupersedeOp[]`, with `interface SupersedeOp { oldId: string; oldValue: string; newId: string; newValue: string; provenance: Provenance }`.
|
||||
- `proposeDiff` returned object: initialise `supersedes: []` (no routing yet).
|
||||
- `applyDiff`: unchanged for now (ignores the new array).
|
||||
2. **Add failing unit tests** (mirror the SC1e keep-both test at `:78-91`): SC1, SC2, SC3b, SC4, SC8, SC11, SC12 → expected FAIL; SC3a, SC5, SC7 → guards (may pass). SC6, SC10 in `consolidate-cli.test.ts` → FAIL.
|
||||
|
||||
**RED gate (single, unambiguous):** run **`(cd scripts/brain && npm test)` directly** → the new behavioural tests fail, the 82 existing pass. **Do NOT run `bash scripts/test-runner.sh` at RED** — Section 16b (`test-runner.sh:717`) fails the whole gate on a non-zero brain exit, which is design-noise, not signal. The direct brain-suite RED is the failing-test proof. (The gate is run GREEN only, after Step 4.)
|
||||
|
||||
## Step 2 — (GREEN) engine logic
|
||||
|
||||
1. **`proposeDiff` routing fork** — BEFORE the keep-both branch (`~:120`), after the ai-draft reject (`:105`, so a supersede can never carry ai-draft provenance — confirmed):
|
||||
- If `c.supersedes` is set: `targetId = mintEntityId({kind:"observed", key: c.supersedes})`; `target = byId.get(targetId)`.
|
||||
- Route to `supersedes[]` **only when** `target` exists AND `target.status === "active"` AND `target.value !== c.value` AND `targetId` not `touched` this batch. **The `target.value !== c.value` guard is load-bearing** (plan-critic BLOCK 2): a re-sent stale signal whose target already holds the candidate's value is NOT a supersession — it falls through to the ordinary bump path, so SC3a holds.
|
||||
- On route: push `{ oldId: targetId, oldValue: target.value, newId: mintEntityId({kind:"observed", key: c.key}), newValue: c.value, provenance: c.provenance }`; `touched.add(targetId)` + `touched.add(newId)` (intra-batch guard — a later same-key candidate degrades; winner id reserved). Do **not** also push to `additions`. `continue`.
|
||||
- Otherwise (no target / not active / value already equal / touched): fall through to the ordinary add/bump path (SC4 graceful no-target, SC8 intra-batch). A supersede only retires a fact present at batch start (`byId` built once from `current`).
|
||||
2. **`archivalId()` helper** (local to `consolidate.ts`): `archivalId(oldId, oldValue) = mintContentId(\`superseded:${oldId}::${oldValue}\`)`. Seeding with the retired fact's **pre-archival id** (unique among active facts by the no-dup invariant) makes the archival id collision-free in practice (plan-critic NICE 9 — stronger than a key+date seed).
|
||||
3. **`applyDiff` supersede branch — REPLACE IN PLACE, not copy** (`:144-162`; plan-critic BLOCK 1). Explicit ordering:
|
||||
- (a) Compute `passing = diff.supersedes.filter(s => { const f = factById(current, s.oldId); return f && f.status === "active" && f.value === s.oldValue })`. The **`f.value === oldValue` state-check** is the idempotency + stale-diff guard: on re-apply the winner holds `oldId` with `newValue ≠ oldValue` → not passing → skipped (SC3b); a profile mutated under the diff → skipped (R3). Build `retireById = new Map(passing.map(s => [s.oldId, s]))`.
|
||||
- (b) When mapping `current.static` and `current.dynamic`: a fact whose `id ∈ retireById` is **transformed in place** to `{ ...f, id: archivalId(f.id, f.value), status: "superseded" }` (keeps its layer + its OWN `provenance`/`first_seen` — a `published→human` update must NOT rewrite the archived fact's provenance, plan-critic FIX 3). It is **excluded from bump/promote** (a superseded fact is never bumped or promoted — resolves the ordering hazard, plan-critic FIX 6: supersede-wins over a same-id bump/promote in one diff).
|
||||
- (c) Append each winner `{ id: newId, value: newValue, first_seen: today, last_seen: today, evidence_count: 1, provenance, status: "active" }` to **dynamic** (Q-LAYER re-earn). Because (b) vacates `oldId` and the winner re-occupies it (when `key===supersedes`), there is exactly one fact at the canonical key-id (the winner) + one archival fact — **no duplicate id**. Winner `first_seen: today` is intentional (a genuinely new observation; the original first-observation date lives on the archived copy — plan-critic FIX 8a).
|
||||
- `applyDiff` stays **pure** (new `ProfileDoc`); existing bump/promote/addition handling preserved.
|
||||
4. **Decay scan excludes superseded** (`consolidate.ts:132-134`; plan-critic FIX 8c): add `f.status !== "superseded"` to the `current.dynamic` decay filter so a retained dynamic archival fact never throws perpetual `staleFlags`. (Static archival facts are already decay-exempt — dynamic-only scan.)
|
||||
|
||||
**GREEN gate:** `(cd scripts/brain && npm test)` → all engine SCs pass; existing 82 pass.
|
||||
|
||||
## Step 3 — (GREEN) CLI render + validation + gather filter
|
||||
|
||||
1. `renderDiffMd` (`cli.ts:132-145`): add a `## Supersessions (old → new)` section, **appended LAST** (after Stale, `:143`) via the existing `section()` helper, **only when `supersedes.length > 0`** — one line per entry `- \`<oldValue>\` → \`<newValue>\``. Zero supersessions → no section → byte-identical to today (plan-critic FIX 5).
|
||||
2. `validateCandidates` (`:147-157`): if `supersedes` present, assert non-empty single-line string; absent is valid.
|
||||
3. **`--gather` active-only filter** (`cli.ts:174`; plan-critic FIX 8b): `profileFacts` emitted to the extraction session is filtered to `status === "active"` — superseded archival facts must NOT be re-presented as live context (else the slice re-introduces the retired values it exists to retire). One-line filter.
|
||||
4. **CLI tests** (`consolidate-cli.test.ts`): SC6 — (a) `--propose` writes one `## Supersessions` line per entry; (b) **render-fidelity non-vacuity** — 2 entries → 2 lines, 0 entries → no section (decoy); (c) `--apply` without `--confirm` refuses (exit 1, no write); with `--confirm` writes superseded status. SC10 — `--gather` output excludes superseded facts.
|
||||
|
||||
## Step 4 — gate floor + doc reconciliation
|
||||
|
||||
1. `test-runner.sh:716`: `BRAIN_TESTS_FLOOR` 82 → **82 + N** (N = the EXACT counted new brain tests from Steps 1–3 — set after counting, never guessed). Extend the breakdown comment `… + SB-S2 19 + SB-S3b N`. **`ASSERT_BASELINE_FLOOR` stays 80** — no new *unconditional `test-runner.sh` structure-lint* check is added; the render-fidelity check is a brain-suite test counted by `BRAIN_TESTS_FLOOR`. (This reconciles brief SC9/§3.7 "ASSERT bump" wording: the render-fidelity check lives in the brain suite, so `BRAIN_TESTS_FLOOR` is the only floor that moves — see SC9 below.)
|
||||
2. `consolidation-loop.md`: edit `:34` + `:71` to the true state — temporal-update supersede landed in SB-S3b (explicit signal, operator-gated, replace-in-place, audit-retained); contradictory still keep-both; decay still advisory. Add the honest-limit line (engine guarantees mechanics; the temporal-update-vs-contradictory call is session+operator judgement).
|
||||
3. `consolidate.ts:11` docstring: update the "no supersede in S2 — that's S3" line to note S3b implements temporal-update supersede.
|
||||
|
||||
## Step 5 — Land
|
||||
|
||||
STATE "Telling" + "👉 NESTE" updated (S3b done → S3c/S3d remain); brief+plan committed as docs alongside code. Commit (Conventional; `[skip-docs]` on code/feat; brief+plan+consolidation-loop+docstring are docs → feat-dominant `[skip-docs]`). **Push only inside the window** (`date '+%u %H:%M'` first) and **confirm before push** (`origin` is PUBLIC `open/`). No version bump (additive within v0.5.2 dev).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Expected |
|
||||
|----|-------|----------|
|
||||
| SC1 | routing | supersede candidate (target value differs) → 1 `supersedes[]` entry, 0 keep-both |
|
||||
| SC2 | mutation+ids | winner id=`mintEntityId(c.key)`, active, ev=1, value=new; retired→`status:superseded`, id=archival≠key-id, OWN provenance/first_seen kept; both present (no dup at key-id) |
|
||||
| SC3a | propose-idempotent | post-land re-propose (value now matches) → 0 `supersedes[]`, 1 bump — incl. when a stale signal persists (value-guard) |
|
||||
| SC3b | apply-idempotent | re-apply same diff → value-match skip → no double winner, no re-supersede, no dup id |
|
||||
| SC4 | graceful no-target | signal + no active fact at key-id → plain add, no throw |
|
||||
| SC5 | round-trip | `parse(serialize(applyDiff(...)))` deep-equals `applyDiff(...)` (superseded fact from the real path) |
|
||||
| SC6 | CLI render + gate | `## Supersessions` line/entry; 0 entries → no section (byte-identical); `--apply` needs `--confirm` |
|
||||
| SC7 | no regression | contradictory still keep-both; bump/promote/published-only unchanged |
|
||||
| SC8 | intra-batch | two candidates same key / supersede-a-batch-created-fact → deterministic, no dup id |
|
||||
| SC9 | gate green | `bash scripts/test-runner.sh` green; brain suite ≥ new `BRAIN_TESTS_FLOOR`; **`ASSERT_BASELINE_FLOOR` unchanged at 80** |
|
||||
| SC10 | gather active-only | `--gather` output excludes `status:superseded` facts |
|
||||
| SC11 | decay excludes superseded | a retained dynamic superseded fact throws no `staleFlags` |
|
||||
| SC12 | supersede-wins ordering | a diff that bumps/promotes AND supersedes the same id → supersede wins (superseded fact not bumped/promoted) |
|
||||
| (red proof) | failing-test-first | `(cd scripts/brain && npm test)` BETWEEN Step 1 and Step 2 → new SCs fail, 82 pass |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — id corruption (plan-critic BLOCK 1).** `applyDiff` must **replace in place** (filter+map, mirroring promotions `:156`), never copy — else two facts share the canonical key-id. Idempotency/stale guard keys off `oldId` AND `oldValue`. Pinned by SC2 + SC3b.
|
||||
- **R2 — propose self-supersede (plan-critic BLOCK 2).** A persisted signal whose target already holds the candidate's value must NOT route to supersede (value-guard); else spurious archival churn every run. Pinned by SC3a.
|
||||
- **R3 — superseded facts leak back.** `--gather` (FIX 8b) and decay (FIX 8c) must filter `status:active`/exclude `superseded` — else the slice regresses its own goal / spams stale flags. Pinned by SC10 + SC11.
|
||||
- **R4 — stale diff vs mutated profile.** Value-matched state-check skips. Pinned by SC3b/SC4.
|
||||
- **R5 — render JSON↔MD divergence.** SC6 render-fidelity non-vacuity (every entry → a line; 0 → no section).
|
||||
- **R6 — floor-count drift.** `BRAIN_TESTS_FLOOR` bumped by the EXACT counted N; `ASSERT_BASELINE_FLOOR` deliberately unchanged at 80 (no new unconditional structure-lint check). Verified by the green gate.
|
||||
- **R7 — unguarded doc.** No lint protects `docs/`; `consolidation-loop.md` reconciled by discipline (as the S3a `:67` edit).
|
||||
114
docs/second-brain/plan-sb-s3c.md
Normal file
114
docs/second-brain/plan-sb-s3c.md
Normal file
|
|
@ -0,0 +1,114 @@
|
|||
# Plan — SB-S3c: cross-silo id-threading
|
||||
|
||||
> Brief: `docs/second-brain/brief-sb-s3c.md`. Slice: SB-S3c.
|
||||
> **Status:** **LANDED** (operator go 2026-06-23) — brain 113/113, gate 95/0/0, `BRAIN_TESTS_FLOOR` 94→113. **Design:** hub-side threading — `specifics`/`trends` ids onto the brain's published record + a pure analytics resolver. Tributaries untouched.
|
||||
> **Light-Voyage:** brief-review APPROVE-WITH-FIXES (5 FIX folded into the brief; 3 RISK carried here). plan-critic **REVISE → 1 BLOCK + 4 MAJOR + 4 MINOR folded** (Step 1 compile-fixup · Step 2.2 non-throwing parse reader · Step 2.3 producer guard · Step 3.3 floor rationale · Step 3.4 deterministic row-sort · Step 3.5 analytics-root caveat · Step 4.2 `runIngest` signature+call-site · Step 4.3 full-record load + desc sort · R-G sub-count). scope-guardian **ALIGNED** (0 creep / 0 gap).
|
||||
> Order is TDD: failing brain-suite tests land BEFORE the code (iron law); each rule pinned before the CLI is wired.
|
||||
|
||||
## Goal
|
||||
|
||||
A published post records the **raw material it was built from** (`specifics`/`trends` ids, additive + backward-compatible on the post record), and a **pure assembler** joins published-record ↔ analytics-row (by normalized title-prefix + date, with honest confidence tiers) so the north-star query — *which raw material actually performs?* (`specific → post → measured analytics`, `architecture.md:17`) — is **assemblable**. All four tributary schemas stay untouched; the only new disk surface is a **read-only** `brain assemble`. Fully unit/CLI-testable with fixtures.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | SC |
|
||||
|------|--------|-----|
|
||||
| `scripts/brain/src/ingest.ts` | `PublishedRecord` (+`specifics: string[]` +`trends: string[]`, `:27-40`) · `serializePublishedRecord` append the two lines **after `source:`**, **omit-empty** (`:50-58`) · `parsePublishedRecord` read them via a new list-scalar reader, **12-hex-validate each** (reuse the `:82` guard idiom), absent→`[]` (`:75-99`) · `ingestText` (+optional `specifics?`/`trends?`, `:165-180`) | SC1,SC2,SC3,SC4,SC11 |
|
||||
| `scripts/brain/src/assemble.ts` | **NEW.** Pure `assemblePostGraph({records, analytics})` + local `normalize()` + `matchRow()` (tiers) + minimal input type `AnalyticsRowInput` + output types · thin read-only `loadAnalyticsRows()` IO (inline raw-JSON read of `dataRoot('analytics/posts')/*.json` → `AnalyticsBatch.posts[]`; **never** imports the analytics package) | SC6,SC7,SC8 |
|
||||
| `scripts/brain/src/cli.ts` | `collectRepeated(args,key): string[]` helper (leaves `parseFlags` **untouched** — SC12 by construction) · **change `runIngest` signature `(flags)`→`(rest, flags)` AND the `main` dispatch `runIngest(flags)`→`runIngest(rest, flags)`** (`:92`, `:238` — BLOCK 1), thread `collectRepeated(rest,"specific")`/`"trend"` · `runAssemble(flags)` (read-only print, full-record load) · dispatch `+ if (command==="assemble")` (`:240`) · `usage` text (+`assemble`, +ingest flags) | SC5,SC9,SC12 |
|
||||
| `scripts/brain/tests/ingest.test.ts` | SC1 (round-trip w/ ids) · SC2 (byte-identical old-record fixture) · SC3 (parse + producer malformed-id throw) · SC4 (producer threading) · SC11 (dedup/collision/published-only regress) | unit |
|
||||
| `scripts/brain/tests/assemble.test.ts` | **NEW.** SC6 (high) · SC7 (none / below-floor / ellipsis-near-miss / different-date→low) · SC8 (pure+total) | unit |
|
||||
| `scripts/brain/tests/cli.test.ts` | SC5 (repeatable flags) · SC9 (read-only assemble print + missing-dir degrade) · SC12 (single-value flags unregressed) | cli |
|
||||
| `scripts/test-runner.sh` | bump `BRAIN_TESTS_FLOOR` 94→(94+N) + breakdown comment (`:716`); **`ASSERT_BASELINE_FLOOR` UNCHANGED at 80** | SC10 |
|
||||
| `docs/second-brain/consolidation-loop.md` · `architecture.md` · `scripts/brain/src/id.ts` | reconcile status to true state: S3c threads the cross-silo graph; `id.ts:6-7` "SB-S3 will thread" → "S3c threads (post→specifics/trends + analytics resolver)"; `architecture.md:80` SB-S3 row note | doc |
|
||||
| `CLAUDE.md` · STATE | telling/counts: brain test count, S3c landed line; reference-doc count unchanged (briefs aren't counted reference docs) | doc |
|
||||
|
||||
**Not touched (scope fence):** `scripts/trends/**` · `scripts/specifics-bank/**` · `scripts/analytics/**` (READ-only, via inlined JSON — no import, no schema/field change, no base-36-id reconciliation) · `hooks/scripts/**` (no `.mjs`; pathguard untouched) · `agents/**` (no reader wired this slice) · `scripts/brain/src/{consolidate,types,profile,scaffold,dataRoot}.ts` (no consolidation/grammar/scaffold change) · `brain/profile.md` grammar (C-1 — the 7th-token bump stays OUT) · `parseFlags` (untouched — `collectRepeated` is additive) · the plugin command surface (29 unchanged — `brain assemble` is a TS-CLI subcommand, not a `/linkedin:*` command).
|
||||
|
||||
## Step 0 — pre-flight (verify baseline, no edits)
|
||||
|
||||
`(cd scripts/brain && npm install)` if `node_modules` absent. Confirm `(cd scripts/brain && npm test)` = **94/94** and `bash scripts/test-runner.sh` green (95/0/0, `BRAIN_TESTS_FLOOR=94`, `ASSERT_BASELINE_FLOOR=80`). Re-read `ingest.ts:50-99` (serialize/parse round-trip), `cli.ts:44-60,92-116,233-245` (parseFlags/runIngest/main). Confirm **no structure-lint** in `test-runner.sh` asserts the brain-CLI subcommand set (so `assemble` adds no lint debt) and that `specifics-bank/src/bank.ts:52` `normalizeContent` is the idiom to **copy locally** (not import).
|
||||
|
||||
## Step 1 — (RED) types + failing tests
|
||||
|
||||
1. **Type scaffolding + compile-fixups** (suite still compiles, 94 green):
|
||||
- `PublishedRecord` (`ingest.ts:27-40`): `+ specifics: string[]; + trends: string[];` (required).
|
||||
- **Compile-fixup (plan-critic MINOR 4 — REQUIRED fields, so existing full-`PublishedRecord` literals must gain them or the suite won't compile):** add `specifics: [], trends: []` to the literals at `tests/ingest.test.ts:11` (`baseRec`) and `tests/publish.test.ts:78` (`squatter`). Tests that build records via `ingestText`/`parsePublishedRecord` need no change.
|
||||
- `ingestText` opts (`:165-180`): `+ specifics?: string[]; + trends?: string[];`; record literal sets `specifics: opts.specifics ?? []`, `trends: opts.trends ?? []`.
|
||||
- **`parsePublishedRecord` defaults the new fields to `[]` from the start** (absent → `[]`) — so existing round-trips (records carrying `[]`) stay **green** at Step 1 (serialize omits empty, parse sets `[]`, deep-equal holds). The *new behaviour* (emit non-empty, read present values, validate, producer-throw) is Step 2 RED→GREEN.
|
||||
- `assemble.ts`: stub `assemblePostGraph` returning `[]` + the types; `normalize`/`matchRow` declared.
|
||||
2. **Add failing tests** → expected FAIL: SC1 (parse must read **non-empty** lines), SC2 (byte-identical fixture — fails until omit-empty serialize), SC3 (validate-throw), SC4 (producer threading with non-empty ids), SC6/SC7/SC8 (assembler logic), SC5/SC9/SC12 (CLI). Existing 94 (carrying `[]`) PASS.
|
||||
|
||||
**RED gate (single, unambiguous):** run **`(cd scripts/brain && npm test)` directly** → new tests fail, existing 94 **compile** (with the two added literal fields) and **pass** (parse defaults `[]`). **Do NOT run `bash scripts/test-runner.sh` at RED** (Section 16b fails the whole gate on a non-zero brain exit — design-noise, per S3b plan `:38`). The direct brain-suite RED is the failing-test proof; the gate runs GREEN-only after Step 4.
|
||||
|
||||
## Step 2 — (GREEN) the post record carries raw material
|
||||
|
||||
1. **`serializePublishedRecord`** (`ingest.ts:50-58`): after the fixed 5-line header array, conditionally append — **only when non-empty** — `specifics: <id,id>` then `trends: <id,id>` (comma-joined, no spaces), **before** the `SENTINEL`. Empty array → line omitted → **byte-identical 5-line header** (SC2). Order fixed: `…source:` → `specifics:`(if any) → `trends:`(if any) → `---`.
|
||||
2. **`parsePublishedRecord`** (`ingest.ts:75-99`): a **new non-throwing optional-list reader** — **NOT `headerScalar`** (plan-critic MAJOR 1: `headerScalar` `:61-65` THROWS `missing "<key>:" header` on an absent key, which would break **every** pre-S3c record / SC2 / SC11). Instead: `const m = header.match(new RegExp(\`^${key}:\\s*(.*?)\\s*$\`, "m")); if (!m) return []` (absent → `[]`); else split the captured value on `,`, trim, drop empties; **validate each id `/^[0-9a-f]{12}$/`** (the `:82` hex guard) → a non-12-hex entry **throws** `malformed published record: bad <specifics|trends> id <JSON.stringify(entry)>` (never silently dropped). Read from the **header slice only** (pre-sentinel), so a body line that looks like `specifics:` cannot leak.
|
||||
3. **Producer-side validation (SC3 symmetric, plan-critic MAJOR 4 — specify the guard):** in `ingestText` (`:165-180`), BEFORE `mintContentId`, validate each id of `opts.specifics ?? []` and `opts.trends ?? []` against `/^[0-9a-f]{12}$/`; a non-match throws `ingest: bad <specifics|trends> id <JSON.stringify(entry)>`. **Empty/absent arrays skip validation** (no throw) — so `scanInbox` (`:204`, calls `ingestText` with no specifics/trends) and every existing caller stay green (SC11). This adds a throw path to `ingestText`'s previously throw-free contract — that is intended and bounded to malformed-id input.
|
||||
|
||||
**GREEN gate (record):** `(cd scripts/brain && npm test)` → SC1–SC4 + SC11 pass; existing 94 pass.
|
||||
|
||||
## Step 3 — (GREEN) the assembler (the payoff)
|
||||
|
||||
In `assemble.ts`:
|
||||
|
||||
1. **`normalize(s)`** (local copy of the `normalizeContent` idiom — NOT imported): `s.normalize?` → `s.trim().toLowerCase().replace(/\s+/g, " ")`.
|
||||
2. **Title prep:** `nt = stripTrailingEllipsis(normalize(title))` where `stripTrailingEllipsis` removes a trailing `…` or `...` (+ surrounding ws) — this absorbs LinkedIn's truncation marker so a `"…"`-suffixed export title still prefix-matches the body. (Mid-word truncation **without** a marker already prefix-matches — `body.startsWith(title)` stays true — so only the marker needs stripping.)
|
||||
3. **`matchRow(record, row)` → `{ confidence: "high"|"low", row } | null`:**
|
||||
- `nb = normalize(record.body)`; `nt` as above.
|
||||
- `if (nt.length < PREFIX_FLOOR) return null` — **`PREFIX_FLOOR = 24`** (plan-critic MINOR 2 — rationale: the hook quality-rule floor is 110 chars and a LinkedIn export title is the content's opening run; 24 normalized chars (~3–5 words) is the shortest opener specific enough that a prefix-match is not coincidental, while staying well under any real hook. Tunable constant with this one-line justification; below floor → `none`).
|
||||
- `if (!nb.startsWith(nt)) return null` → `none`.
|
||||
- else: `record.published_date === row.publishedDate ? "high" : "low"` (prefix match + same date = high; prefix match + different date = low, surfaced for operator eyeball). *(Note the field is `publishedDate` on the analytics side, `published_date` on the record — RISK B.)*
|
||||
4. **`assemblePostGraph({records, analytics})`:** for each record → `{ contentId: record.id, specifics, trends, match }` where `match` = the **best** of `analytics.map(r => matchRow(record, r))`: prefer `high` over `low`; tie-break by **longest matched `nt`** (plan-critic MINOR 1 — to make the tie fully deterministic regardless of input/file order, the assembler **sorts `analytics` once at entry** by `(publishedDate desc, title asc)` before matching, so an exact length tie resolves stably, not by `readdirSync` order); none qualifying → `match: { confidence: "none" }`. Pure — no FS/clock/network; empty `records`→`[]`; empty `analytics`→every post `none` (SC8). The match carries the **whole row reference** (FIX 4).
|
||||
5. **`loadAnalyticsRows(): AnalyticsRowInput[]`** (thin IO, read-only): `const dir = dataRoot("analytics/posts")`; if absent → `[]` (RISK C, fresh-clone). Read each `*.json`, `JSON.parse`, take `.posts` (an `AnalyticsBatch`), map each to the minimal `{ title, publishedDate, metrics }` (extra fields ignored). A malformed/unreadable file is skipped in a try/catch (mirrors `listPublished` `ingest.ts:230`), never a crash. **No import of the analytics package** (§5 decoupling). **Doc-comment caveat (plan-critic MAJOR 2 / R-B):** this resolves the analytics root via the brain's `dataRoot` (`${LINKEDIN_STUDIO_DATA}/analytics/posts`); the analytics package additionally honours the **deprecated `ANALYTICS_ROOT`** env override (`storage.ts:67-72`) which the brain read path does NOT — so if `ANALYTICS_ROOT` is set to a non-default path, `brain assemble` reads the default root and degrades to every-post-`none`. This is the accepted cost of the §5 no-import decoupling (the M0 default leaves `ANALYTICS_ROOT` unset); name it in the loader doc-comment, do not silently skew.
|
||||
|
||||
**GREEN gate (assembler):** `(cd scripts/brain && npm test)` → SC6/SC7/SC8 pass.
|
||||
|
||||
## Step 4 — (GREEN) CLI: repeatable producer flags + read-only assemble
|
||||
|
||||
1. **`collectRepeated(args, key): string[]`** in `cli.ts` (near `parseFlags`): scan the raw `args` for every `--key <value>` pair (value not starting with `--`) and collect all values. **`parseFlags` is left untouched** → single-value flags parse exactly as today (SC12 by construction).
|
||||
2. **`runIngest` signature + call-site change (plan-critic BLOCK 1 — this is a CHANGE, not an existing affordance):** the current code is `function runIngest(flags)` (`cli.ts:92`) dispatched as `runIngest(flags)` (`main`, `cli.ts:238`) — `rest` is NOT passed today. Change BOTH: the signature to `runIngest(rest: string[], flags)` AND the `main` dispatch to `runIngest(rest, flags)`. Then build `specifics = collectRepeated(rest, "specific")`, `trends = collectRepeated(rest, "trend")`, thread into `ingestText({ …, specifics, trends })`. **`runIngest` reads specifics/trends ONLY via `collectRepeated`, never `flags.specific`/`flags.trend`** (R-E). Empty → `[]` (today's behaviour). Update `usage` (`:71`): `ingest --file <path> [--source <s>] [--date <YYYY-MM-DD>] [--specific <id>]… [--trend <id>]…`.
|
||||
3. **`runAssemble(flags)`** + dispatch `if (command === "assemble") return runAssemble(flags);` (`:240`): load **full `PublishedRecord[]`** via the `runConsolidate --gather` record-load idiom (`cli.ts:170-178` — `parsePublishedRecord` over each `dataRoot('ingest/published')/*.md`, NOT `listPublished` which omits `.body` and sorts ascending — plan-critic MAJOR 3) + `analytics = loadAnalyticsRows()`; run `assemblePostGraph(...)`, then **sort the result newest-first explicitly** (`published_date` desc — do not rely on load order) and print: per post `contentId · published_date · <firstLine>` where `firstLine = record.body.split("\n", 1)[0]` (derived locally), then `specifics: …`, `trends: …`, `analytics: <confidence> [eng <engagementRate>%]` (Fork-4: `engagementRate` headline; whole row available for a future `--verbose`). **Writes nothing** (asserted SC9). Missing `ingest/published/` or `analytics/posts/` → clean empty/partial output, no crash (RISK C). Add `assemble` to `usage`.
|
||||
4. **CLI tests** (`cli.test.ts`): SC5 (`--specific a --specific c --trend b` → record tagged `[a,c]`/`[b]` via round-trip read); SC12 (single-value `--file`/`--source`/`--date` + boolean `--scan-inbox`/`--confirm` parse as today, across subcommands); SC9 (`assemble` over a seeded `LINKEDIN_STUDIO_DATA` tmp dir prints the join + writes nothing; over an empty/missing dir prints clean + no crash).
|
||||
|
||||
**GREEN gate:** `(cd scripts/brain && npm test)` → all SCs pass.
|
||||
|
||||
## Step 5 — gate floor + doc reconciliation
|
||||
|
||||
1. `test-runner.sh:716`: `BRAIN_TESTS_FLOOR` 94 → **94 + N** (N = EXACT counted new brain tests from Steps 1–4 — set after counting, never guessed). Extend the breakdown comment `… + SB-S3c N [ingest(a)+assemble(b)+cli(c)]`. **`ASSERT_BASELINE_FLOOR` stays 80** — no new *unconditional structure-lint* check (all new tests are brain-suite tests, counted by `BRAIN_TESTS_FLOOR`; same idiom as S3b §3.7 / SC10).
|
||||
2. `id.ts:6-7`: "SB-S3 will thread this id through the tributaries" → "SB-S3c threads it (post record carries `specifics`/`trends` ids; analytics joined by resolver)".
|
||||
3. `consolidation-loop.md` + `architecture.md:80`: mark the SB-S3 cross-silo graph as landed for the `specific→post→analytics` spine via the hub-side design; note the resolver's **honest limit** (RISK A): the analytics↔post join is a title-prefix+date heuristic with `high/low/none` tiers — a real-CSV `none` is a normalization-tightening signal, not a guarantee of no match (mirror the `consolidation-loop.md` honest-limit idiom).
|
||||
|
||||
## Step 6 — Land
|
||||
|
||||
STATE "Telling" + "👉 NESTE" updated (S3c done → S3d remains, last); brief+plan committed as docs alongside code. **GREEN gate proof:** `(cd scripts/brain && npm test)` = (94+N)/(94+N) and `bash scripts/test-runner.sh` green. Commit (Conventional; code/feat-dominant → `[skip-docs]`). **Push only inside the window** (`date '+%u %H:%M'` first) and **confirm before push** (`origin` is PUBLIC `open/`). No version bump (additive within v0.5.2 dev). Optional documented end-to-end demo (RISK A) recorded in STATE/changelog.
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Expected |
|
||||
|----|-------|----------|
|
||||
| SC1 | record round-trip | `parse(serialize(rec))===rec` w/ non-empty specifics/trends; order preserved |
|
||||
| SC2 | byte backward-compat | empty arrays → unchanged 5-line header; `serialize(parse(oldText))===oldText` for fixture old record |
|
||||
| SC3 | id validation | non-12-hex specifics/trends id throws on parse AND at `ingestText` producer boundary |
|
||||
| SC4 | producer threading | `ingestText({specifics:[a],trends:[b]})` → round-tripped `[a]`/`[b]`; absent → `[]` |
|
||||
| SC5 | repeatable flags | `--specific a --specific c --trend b` → record `[a,c]`/`[b]` |
|
||||
| SC6 | assemble high | normalized body starts with row title (≥floor) + same date → `high`, whole row attached, specifics/trends surfaced |
|
||||
| SC7 | assemble low/none | no-prefix → none; `<floor` prefix → none; `…`-truncated title + same date → high (ellipsis stripped); prefix + different date → low |
|
||||
| SC8 | pure + total | `records:[]`→`[]`; `analytics:[]`→every post `none`; no throw |
|
||||
| SC9 | read-only CLI | `brain assemble` prints join newest-first + writes nothing; missing dirs → clean, no crash |
|
||||
| SC10 | gate green | `bash scripts/test-runner.sh` green; brain ≥ new `BRAIN_TESTS_FLOOR`; `ASSERT_BASELINE_FLOOR` unchanged at 80 |
|
||||
| SC11 | no regression | ingest dedup/collision/published-only unchanged |
|
||||
| SC12 | single-value flags | `--file`/`--source`/`--date`/`--scan-inbox`/`--confirm` parse as today |
|
||||
| (red proof) | failing-test-first | brain suite BETWEEN Step 1 and Step 2 → new SCs fail, 94 pass |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R-A (from RISK A) — synthetic-only proof.** All SCs use fixtures; only the §Step-6 demo touches a real CSV. A green gate does **not** prove the resolver matches a real LinkedIn title against a real body. Mitigation: keep the demo; treat a demo `none` as a normalization-tightening signal (honest-limit doc line), not a slice failure. `PREFIX_FLOOR`/ellipsis-strip are the tunables.
|
||||
- **R-B (from RISK B) — analytics extraction-point.** `brain assemble` must inline the raw-JSON read (`dataRoot('analytics/posts')`), **not** `loadAllPosts`, and read **`publishedDate`** (not `date`). Pinned in Step 3.5. A try/catch skips malformed batch files. **Root-skew caveat (plan-critic MAJOR 2):** the brain `dataRoot` ignores the analytics package's deprecated `ANALYTICS_ROOT` override (`storage.ts:67-72`); if set, the join silently degrades to every-post-`none`. Accepted cost of the §5 no-import decoupling (M0 default leaves it unset); named in the `loadAnalyticsRows` doc-comment, not silent.
|
||||
- **R-C (from RISK C) — fresh-clone.** `analytics/posts/` is gitignored/absent on a fresh clone → `loadAnalyticsRows()` returns `[]` → every post `none` (SC8), `brain assemble` degrades cleanly (SC9). Confirm no throw on missing dir.
|
||||
- **R-D — round-trip byte-break.** New header lines MUST be omit-empty + appended after `source:`; an interleaved or always-emitted line breaks SC2 silently. Pinned by the SC2 fixture (byte-equality, not just deep-equal).
|
||||
- **R-E — `collectRepeated` vs `parseFlags` divergence.** `parseFlags` still records the *last* `--specific` value (harmless — `runIngest` ignores `flags.specific`/`flags.trend` and uses `collectRepeated`). Pin: `runIngest` reads specifics/trends ONLY via `collectRepeated`, never `flags`.
|
||||
- **R-F — false-positive match.** A generic short opener could prefix-match the wrong post; `PREFIX_FLOOR=24` + same-date→high (else low) guards it; low matches are surfaced (not hidden) so the operator eyeballs. No silent high on a weak match.
|
||||
- **R-G — floor-count drift (plan-critic MINOR 3).** `BRAIN_TESTS_FLOOR` bumped by the EXACT counted N; the breakdown comment's per-file sub-counts `[ingest(a)+assemble(b)+cli(c)]` MUST sum to N (reconcile after counting). `ASSERT_BASELINE_FLOOR` deliberately unchanged at 80. Verified by the green gate; nothing lints the comment (discipline-only, as S3a/S3b).
|
||||
- **R-H — unguarded doc.** No lint protects `docs/`; reconciled by discipline (as S3a/S3b).
|
||||
141
docs/second-brain/plan-sb-s3d.md
Normal file
141
docs/second-brain/plan-sb-s3d.md
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
# Plan — SB-S3d: wire the operations centre (`operations.md` READER + dated anchor)
|
||||
|
||||
> Brief: `docs/second-brain/brief-sb-s3d.md`. Slice: SB-S3d (ops centre only; hygiene+reconciliation → SB-S3e).
|
||||
> Operator confirmed (2026-06-23): **Split, ops-centre first.** Recommendations pending the go-gate: **Option B** (enrich seed) + **advisory** deprecation + **one reader** (`strategy-advisor`).
|
||||
> Order is TDD: the failing lint/seed checks land BEFORE the agent/seed edits (iron law).
|
||||
|
||||
## Goal
|
||||
|
||||
`strategy-advisor` reads `brain/operations.md` and honours the **dated frozen-past-self anchor** — preferring it over older/contradicting profile facts (the guard, advisory) — guarded by a deterministic wiring lint, with an enriched seed that ships the dated-anchor convention. No engine write, no post-silo contact, no parser.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
| File | Change | Why |
|
||||
|------|--------|-----|
|
||||
| `scripts/test-runner.sh` | **NEW Section 16e** (2 unconditional checks) + bump `ASSERT_BASELINE_FLOOR` 80→82 + extend the Section-18 history comment + header enumeration (`:33-36`) | SC1/SC2/SC5 — the TDD anchor |
|
||||
| `scripts/brain/tests/scaffold.test.ts` | **NEW test()** asserting the dated-anchor seed convention (Option B) | SC3 — fails first (RED) |
|
||||
| `scripts/brain/src/scaffold.ts` | enrich `operationsSeed()` (`:52-64`): dated-anchor convention + light Plans/Ideas guidance | SC3 — makes the guard operational |
|
||||
| `agents/strategy-advisor.md` | add `brain/operations.md` to Step 0 Load-Context (`:41-48`) + a consumption subsection after `:63` (anchor authoritative + `deprecates older inferences` literal + graceful absence) | SC1/SC2/SC6/SC7 — the wiring |
|
||||
| `docs/second-brain/architecture.md` | edit the SB-S3 build-row (`:80`) to the post-split truth (S3d ✅ ops · S3e remains hygiene+reconciliation) | doc reconciliation |
|
||||
|
||||
**Not touched (scope fence):** any post silo (`hooks/scripts/state-updater.mjs`, `## Recent Posts`, `content-history*`, analytics, tributary JSON) — that is **S3e** · `consolidate.ts`/`assemble.ts`/`ingest.ts`/`cli.ts` (no engine/graph change; advisory guard only) · `consolidation-loop.md` (profile-motor contract, not this loop) · `content-planner.md` (second reader, not S3d) · CLAUDE.md agent table (count/model lint unaffected — no new agent, frontmatter unchanged) · Section 16d (the profile reader stays exactly as-is).
|
||||
|
||||
## Step 1 — (RED) Lint: Section 16e + floor bump
|
||||
|
||||
Insert a new section after Section 16d (`test-runner.sh:833`, before Section 17 at `:834`), modelled byte-for-byte on 16d's idiom (two exact literals + `*_wired()` + non-vacuity self-test + real-file grep with `grep -qF`):
|
||||
|
||||
- **Literals:** `OPS_LIT='brain/operations.md'`, `ANCHOR_LIT='deprecates older inferences'`.
|
||||
- **`ops_reader_wired()`** — wired iff BOTH literals present. Pin the body VERBATIM from 16d (`test-runner.sh:802`), `grep -qF` (fixed-string — `OPS_LIT` contains `.`/`/`, so `-F` is mandatory or the `.` is a regex wildcard): `echo "$1" | grep -qF "$OPS_LIT" && echo "$1" | grep -qF "$ANCHOR_LIT"` (echo twice — grep consumes stdin).
|
||||
- **Check A (self-test, unconditional):** a fully-wired probe (`"reads brain/operations.md; the anchor deprecates older inferences"`) MUST be detected; three under-wired probes MUST be rejected:
|
||||
1. `reads brain/operations.md but never says the anchor outranks anything` (file lit, missing anchor lit)
|
||||
2. `the anchor deprecates older inferences but names no source file` (anchor lit, missing file lit)
|
||||
3. `reads brain/profile.md and tests the evidence` (**sibling-file decoy** — names the OTHER brain file + no anchor lit; must NOT match either operations literal)
|
||||
→ `pass`/`fail` "ops-reader self-test: full wiring detected; 3 under-wired forms rejected (incl. sibling-file `brain/profile.md` decoy)".
|
||||
- **Check B (real-file grep, unconditional):** `grep -qF "$OPS_LIT" agents/strategy-advisor.md && grep -qF "$ANCHOR_LIT" agents/strategy-advisor.md` → `pass` "strategy-advisor.md wired to brain operations reader (names '…operations.md', frames 'deprecates older inferences')" else `fail`.
|
||||
- **Header enumeration (`:33-37`):** insert the 16e clause **immediately after** "…with a non-vacuity self-test) in Section 16d;" (mid-`:36`), BEFORE the "the assertion-count anti-erosion floor (SC6) in Section 18. All are live below (Sections 8–18)." tail — preserving the 16d → 16e → 18 ordering. Clause: "the brain operations-reader guard (SB-S3d: `strategy-advisor` names `brain/operations.md` AND carries the frozen-past-self literal `deprecates older inferences`, with a non-vacuity self-test) in Section 16e;". The "Sections 8–18" range is unchanged (16e is in range).
|
||||
- **Floor (`:905-913`):** `ASSERT_BASELINE_FLOOR=80` → `82`; extend the history comment: "+2 for SB-S3d's two UNCONDITIONAL Section-16e checks (ops-reader self-test + strategy-advisor ops-wiring grep) = 82." Both new checks are deps-free → they lift the deps-absent minimum, so +2 is correct (NOT pinned to deps-present TOTAL).
|
||||
|
||||
**Non-collisions (verify with plan-critic):** new prose carries no algorithm-magnitude token (Section 8 stat-consistency safe) and no `Significant?(` column (Section 11 render-chain safe); the edit uses the exempt `${LINKEDIN_STUDIO_DATA:-…}` external form → Section 13 data-dir guard safe; agent count (19) + `strategy-advisor` model/frontmatter unchanged → Sections 2/9/10 unaffected; Section 16d's two literals remain present in the (unchanged) profile subsection.
|
||||
|
||||
## Step 2 — (RED) Brain seed test (Option B)
|
||||
|
||||
In `scripts/brain/tests/scaffold.test.ts`, add ONE new `test()` (bumps the suite count → floor +1) asserting the enriched seed — e.g. `test("operations.md ships a dated frozen-past-self anchor convention", ...)`:
|
||||
- `initBrain()`, read `brain/operations.md`.
|
||||
- **The single RED-bearing assertion** is the **date convention**, which is verified ABSENT from the current seed (`grep "As of" scaffold.ts` = 0 → the test fails first): `assert.match(ops, /_As of YYYY-MM-DD:_/)` (or `/As of/` — but the underscore form is the exact seed literal).
|
||||
- Do NOT assert `deprecates older inferences` as the failing literal — it ALREADY ships in the current seed comment (`scaffold.ts:58`, verified `grep -c` = 1), so it passes pre-enrichment and would make the RED gate vacuous. It may be asserted only as an explicit **non-RED companion** (documented as such), or omitted.
|
||||
- Keep matching the existing `Who I am now` / `## Plans` / `## Ideas` anchors so the new case is a strict superset of the old guarantee.
|
||||
- Leave the existing 5 tests untouched (the no-clobber test at `:70-77` already proves idempotency of the new seed).
|
||||
- **Baseline (verified live, 2026-06-23):** brain suite reports `tests 113` and the gate reports 95 checks — so the floor becomes **recorded + 1 = 114** (not a hardcoded guess) and the gate goes 95→97.
|
||||
|
||||
**RED proof (recorded):** `(cd scripts/brain && npm test)` against the un-enriched seed → the new test FAILS (no `_As of` in the seed yet). `bash scripts/test-runner.sh` → Section 16e Check A passes, **Check B FAILS** (agent not yet wired), floor self-count passes (TOTAL 96 ≥ 82). Capture this failing output in STATE/changelog as the iron-law artifact. This is the required failing state.
|
||||
|
||||
## Step 3 — (GREEN) Enrich the seed
|
||||
|
||||
In `scripts/brain/src/scaffold.ts`, rewrite `operationsSeed()` (`:52-64`, the full function incl. the closing brace at `:64` — body string is `:53-63`) to ship the dated-anchor convention + light guidance. **Load-bearing:** the rewrite MUST retain `## Who I am now`, `## Plans`, and `## Ideas` verbatim — the existing anchor test `scaffold.test.ts:48-58` asserts all three, so dropping one regresses that test. Idempotency is untouched: `initBrain` **existence-skips** (writes a seed only when the file is absent, `scaffold.ts:105-113`), so the new seed string never reaches an already-created `operations.md`. Sketch:
|
||||
|
||||
```ts
|
||||
function operationsSeed(): string {
|
||||
return `# Operations
|
||||
|
||||
> The operations centre — where you're headed now, what you're working on, what you might do next.
|
||||
> User-authored: the brain motor never writes here (a \`brain init\` re-run never clobbers your edits).
|
||||
|
||||
## Who I am now (anchor)
|
||||
|
||||
<!-- Frozen-past-self guard: a DATED, user-authored "where I'm headed now" statement.
|
||||
It DEPRECATES older inferences — when a profile.md fact predates or contradicts this
|
||||
anchor, the anchor wins. Re-date it when your direction shifts. -->
|
||||
|
||||
_As of YYYY-MM-DD:_ <one or two sentences on your current direction>
|
||||
|
||||
## Plans
|
||||
|
||||
<!-- Active commitments — what you're working toward now. One per line. -->
|
||||
|
||||
## Ideas
|
||||
|
||||
<!-- Parking lot — possible content/strategy ideas, not yet commitments. One per line. -->
|
||||
`;
|
||||
}
|
||||
```
|
||||
|
||||
**GREEN gate (brain):** `(cd scripts/brain && npm test)` → new test passes; suite reports `tests 114` (verified baseline 113 + 1); bump `BRAIN_TESTS_FLOOR` 113→**114** (`test-runner.sh:716`) + extend its inline breakdown comment ("+ SB-S3d 1 [scaffold dated-anchor seed]"). The no-clobber test (`scaffold.test.ts:70-77`) runs in this same gate and must stay green.
|
||||
|
||||
## Step 4 — (GREEN) Wire `strategy-advisor.md`
|
||||
|
||||
1. **Step 0 Load Context** (`strategy-advisor.md:41-48`): insert one line INSIDE the fenced block, immediately before the closing ``` fence (`:48`), after the profile line (`:47`):
|
||||
`${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/brain/operations.md → the operations centre: dated "who I am now" anchor + plans + ideas (user-authored)`
|
||||
2. **New subsection** immediately after the profile subsection's graceful-absence line (`:63`), e.g. `### Consuming the operations centre (brain/operations.md)`:
|
||||
- **Anchor is authoritative:** "The `## Who I am now` anchor is the user's own dated declaration of current direction. It is authoritative and **deprecates older inferences**: when a `brain/profile.md` Dynamic fact predates the anchor's date or contradicts its stated direction, prefer the anchor and flag the older fact as possibly-stale — do not parrot it." ← carries the exact literal `deprecates older inferences`.
|
||||
- **Inversion (make the contrast explicit):** "Unlike profile facts, which you treat as **evidence to TEST**, the anchor is user-declared direction, not an inference to challenge — honour it. (It is direction, not praise; honouring it never licenses flattery.)"
|
||||
- **Plans / Ideas:** "`## Plans` are active commitments (current intent); `## Ideas` are a parking lot (suggestions, not commitments) — weigh accordingly."
|
||||
- **Graceful absence:** "If `brain/operations.md` (or its anchor) is missing or empty, proceed silently on the other context sources — no error, no note about a missing ops centre."
|
||||
|
||||
**GREEN gate (lint):** `bash scripts/test-runner.sh` → Section 16e Check A + Check B pass, FAIL=0, **97/0/0**, ASSERT floor 82 honoured. (16d still green — profile literals untouched.)
|
||||
|
||||
## Step 5 — Doc reconciliation
|
||||
|
||||
Edit `architecture.md:80` (the SB-S3 build-row) so the post-split status is true: S3a/b/c ✅ as today, then "**S3d ✅ ops centre** — `operations.md` is a read tributary; `strategy-advisor` honours the dated anchor that *deprecates older inferences* (advisory). **S3e remains** — dead `content-history.md` retirement + triple-post (post-tracking↔published↔analytics) reconciliation." No other architecture edit (the §59 guard spec already stands; the §80 row was the stale claim).
|
||||
|
||||
## Step 6 — Behavioural verification (manual, recorded — SC6/SC7)
|
||||
|
||||
Agent-prompt behaviour is not unit-testable (operator-accepted). Honest procedure, result recorded in STATE/changelog at land:
|
||||
- **SC6 (graceful absence):** with no `brain/operations.md` at the data-root, the wiring degrades silently — inspect the subsection's graceful-absence clause + (best-effort) a `strategy-advisor` run on an empty root → no missing-file noise.
|
||||
- **SC7 (guard fires):** seed a temp root (`LINKEDIN_STUDIO_DATA=<tmp> brain init`), write a dated anchor (`_As of 2026-06-20:_ pivoting from X to Z`) and a contradicting older profile fact (`leans toward X`, `last_seen` < 2026-06-20) → run the agent → confirm it prefers the anchor (Z) and flags the older X fact as deprecated, not parroted. If the harness cannot exercise a per-call data-root, record that honestly and fall back to inspection of the wiring + framing (no overclaim).
|
||||
|
||||
## Step 7 — Land
|
||||
|
||||
STATE "Telling" + "👉 NESTE" updated (S3d done → **S3e** is the new last slice); commit. Mixed change: the seed/agent/lint are code → `[skip-docs]` judged per the repo convention; brief+plan+architecture are docs. **Push only inside the window** (`date '+%u %H:%M'` first; `origin` = PUBLIC `open/` → confirm before push). No version bump (S3d additive within v0.5.2 dev; release is a separate decision).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|----|-------|---------|----------|
|
||||
| SC1+SC2 | ops reader wired + literal | `bash scripts/test-runner.sh` | Section 16e Check B passes |
|
||||
| SC3 | seed convention | `(cd scripts/brain && npm test)` | new test passes; floor 114 |
|
||||
| SC4 | gate green | `bash scripts/test-runner.sh` | 97/0/0; 16d still green; hook suite (separate runner) untouched |
|
||||
| SC5 | assertion floor | same run | "anti-erosion: 97 … >= floor 82" passes |
|
||||
| (red proof) | failing-test-first | gate run BETWEEN Step 1/2 and Step 3/4 | exit 1: scaffold test fails (Step 2), 16e Check B fails (Step 1) |
|
||||
| SC6 | graceful absence | manual, empty root | clean output, no missing-file noise |
|
||||
| SC7 | guard fires | manual, seeded anchor+stale fact | anchor wins, older fact flagged (or honest fallback recorded) |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — vacuous lint.** Mitigated by the 16d-style non-vacuity self-test with the sibling-file (`brain/profile.md`) + "outranks anything" decoys.
|
||||
- **R2 — floor false-fail on fresh clone.** Both new lint checks are deps-free (lift the deps-absent minimum) → +2 → 82 correct, not pinned to deps-present TOTAL. The brain floor (114) only checked when the brain suite runs (deps-present); a fresh clone warn-skips it (unchanged mechanism).
|
||||
- **R3 — behavioural overclaim.** Step 6 honest-fallback clause (verifiseringsplikt): never report a guard-fires pass not actually run.
|
||||
- **R4 — anti-sycophancy weakening.** The anchor's "authoritative" stance must not read as "flatter the user." Mitigated by the explicit "direction, not praise" clause + the profile `evidence to TEST` stance staying verbatim (Section 16d unchanged, still grepped).
|
||||
- **R5 — seed breaks idempotency.** Mitigated: the enriched seed is still a static string; `scaffold.test.ts:70-77` (no-clobber) stays green and is run in Step 3's GREEN gate.
|
||||
- **R6 — stale §80 doc.** The build-row claim "S3d remains (ops + hygiene + reconciliation)" becomes false at land; mitigated by EDITING `:80` to the split truth (no automated guard covers `architecture.md`; Section 11 scans non-docs dirs only).
|
||||
|
||||
## Plan-critic — folded
|
||||
|
||||
`voyage:plan-critic` (no blockers; 3 major + 3 minor) folded above:
|
||||
- **Major-1 — vacuous brain RED:** the seed already ships `deprecates older inferences` (`scaffold.ts:58`), so the RED-bearing literal must be the **date convention** `_As of YYYY-MM-DD:_` (verified absent). Folded into Step 2. ✅
|
||||
- **Major-2 — unverified brain floor:** recorded the live count (`tests 113`) → floor = recorded + 1 = 114, not a hardcoded guess. Folded into Step 2/3. ✅
|
||||
- **Major-3 — `grep -qF` fixed-string:** pinned the `ops_reader_wired()` body verbatim from `test-runner.sh:802` (`-F` mandatory — `OPS_LIT` has `.`/`/`). Folded into Step 1. ✅
|
||||
- **Minor-1 — header range:** corrected to `:33-37`; the 16e clause inserts after "…in Section 16d;", before the SC6/Section-18 tail. Folded into Step 1. ✅
|
||||
- **Minor-2 — function range:** cite `operationsSeed()` as `:52-64` incl. the closing brace (body `:53-63`). Folded into Step 3. ✅
|
||||
- **Minor-3 — retained anchors:** the three `##` anchors are load-bearing for `scaffold.test.ts:48-58`; the rewrite retains them verbatim. Folded into Step 3. ✅
|
||||
|
||||
Cross-checks the critic confirmed sound: floor arithmetic (ASSERT 80→82, BRAIN 113→114), the 3 decoys each miss ≥1 literal, the sibling-file decoy `brain/profile.md` cannot match `OPS_LIT`, Section 16d stays green (profile literals untouched; the new subsection adds a *second* `evidence to TEST` occurrence, harmless), and no collision with Sections 8/11/13.
|
||||
158
docs/second-brain/plan-sb-s3e.md
Normal file
158
docs/second-brain/plan-sb-s3e.md
Normal file
|
|
@ -0,0 +1,158 @@
|
|||
# Plan — SB-S3e: hygiene (retire dead `content-history.md`) + triple-post reconciliation (read-side)
|
||||
|
||||
> Brief: `docs/second-brain/brief-sb-s3e.md`. Slice: SB-S3e — the LAST S3 slice (arc complete after; S4 optional).
|
||||
> Recommendations pending the go-gate: **(b) retire** content-history · **(c) read-side joiner** (`brain reconcile`) · build now (not doc-only) · brain-CLI-only (no new plugin command/agent).
|
||||
> Order is TDD: the failing reconcile test + gate self-test land BEFORE the reconcile code (iron law). The retirement is deletion — its proof is grep-zero + the migrate suite staying green.
|
||||
|
||||
## Goal
|
||||
|
||||
Two coupled deliverables: **(b)** the dead, zero-reader `content-history.md` is removed across all eight of its plumbing surfaces (writer prose, template, migration entry+test, gitignore, gate guard, ref-doc, comment); **(c)** a new read-only `brain reconcile` reconciles silo 1 (`## Recent Posts`, auto-tracked creation) against the silo 2↔3 graph, surfacing the coverage gap (created posts never ingested into the brain) — without ever writing the state silo. No state-file write, no `$`-injection surface, no auto-capture.
|
||||
|
||||
## Files touched (exhaustive — for scope-guardian)
|
||||
|
||||
### (b) Retirement
|
||||
|
||||
| File | Change | SC |
|
||||
|------|--------|-----|
|
||||
| `hooks/prompts/state-update-reminder.md` | remove Section 5 "Content History Log" (`:67-86`) — the sole writer | SC1 |
|
||||
| `config/content-history.template.md` | **DELETE** | SC1 |
|
||||
| `hooks/scripts/migrate-data.mjs` | remove the B1 `MOVE_FILES` entry (`:33`) | SC1 |
|
||||
| `hooks/scripts/__tests__/migrate-data.test.mjs` | drop content-history fixture (`:37`), its 2 assertions (`:56`,`:61`), trim test-(a) description (`:50`); cases a–e stay (count unchanged) | SC2 |
|
||||
| `.gitignore` | remove `assets/analytics/content-history.md` (`:44`) | SC1 |
|
||||
| `scripts/test-runner.sh` | narrow `SC2_CLASSES` (`:592`) — drop the `content-history\.md` alternative (check stays → gate count unchanged) | SC1/SC5 |
|
||||
| `references/data-path-convention.md` | remove `content-history.md` from the data-tree diagram (`:26`) | SC1 |
|
||||
| `hooks/scripts/session-start.mjs` | drop `content-history` from the comment list (`:94`) | SC1 |
|
||||
|
||||
### (c) Reconciliation (read-side)
|
||||
|
||||
| File | Change | SC |
|
||||
|------|--------|-----|
|
||||
| `scripts/brain/tests/reconcile.test.ts` | **NEW** — RED-first: coverage tiers + truncated-hook degradation + empty-state | SC3 |
|
||||
| `scripts/brain/src/reconcile.ts` | **NEW** — pure `reconcileRecentPosts` + `parseRecentPosts` + `loadRecentPosts` IO seam | SC3/SC4 |
|
||||
| `scripts/brain/src/cli.ts` | add `reconcile` subcommand (`runReconcile`) + dispatch (`:312`) + usage line | SC4 |
|
||||
| `scripts/test-runner.sh` | **NEW Section 16f** (2 unconditional checks) + `ASSERT_BASELINE_FLOOR` 82→**84** + `BRAIN_TESTS_FLOOR` 114→**recount** + header enumeration | SC4/SC5 |
|
||||
| `docs/second-brain/architecture.md` | mark the SB-S3 build-row **S3e ✅** (arc complete) | SC7 |
|
||||
|
||||
**Not touched (scope fence):** `hooks/scripts/state-updater.mjs`, `## Recent Posts`, `pruneContentHistory` (S3e **reads** silo 1, never writes) · `assemble.ts` join heuristic (reconcile *consumes* the graph) · `consolidate.ts`/`ingest.ts` (no engine change) · `consolidation-loop.md` · CLAUDE.md command/agent tables (no new command/agent; `brain reconcile` is an internal CLI subcommand) · README `:120` conceptual "content-history alignment" (concept, not the file — verify-then-leave).
|
||||
|
||||
## Step 1 — (RED) Reconcile test against the absent module
|
||||
|
||||
`scripts/brain/tests/reconcile.test.ts` — model the IO/pure split on `assemble.test.ts`. The RED is a **logic-RED, not an import-RED**: land a **stub `reconcile.ts`** first that compiles + exports the symbols but mis-classifies (returns `[]` / a wrong tier), so the named tier assertions go red against *logic*, not a `MODULE_NOT_FOUND`. Cases:
|
||||
|
||||
1. **`parseRecentPosts`** extracts `{date, hook, charCount, topic}` from a `## Recent Posts` block whose entries match the **writer's** exact format (`updatePostTracking`, `state-updater.mjs:116`: `- [YYYY-MM-DD] "hook" (chars) - topic`) — the format source of truth, NOT the date-only pruner regex (`:145`). A **golden-string** case feeds a literal writer-produced entry and asserts the round-trip. A non-`## Recent Posts` doc → `[]`; a `$`-bearing topic/hook round-trips verbatim (capture/`match` on read — no `String.replace` → no regex injection).
|
||||
2. **`reconcileRecentPosts({recentPosts, records, graph})`** (the pinned 3-arg signature — `records` carry `body`, which `PostGraphNode` lacks) classifies each silo-1 entry:
|
||||
- **in-graph** — a published record matches (date + hook-prefix on `record.body`) AND its graph node has an analytics match → the full chain is observable.
|
||||
- **in-brain-only** — record matches but its graph node's analytics `confidence==='none'`.
|
||||
- **orphaned-in-state** — no record matches (created via the plugin, never `brain ingest`-ed) → the coverage gap.
|
||||
- **summary** counts (created N · in-graph M · orphaned K).
|
||||
3. **Truncated-hook degradation** — a ≤60-char preview ending `...` strips the marker before prefix-matching (mirror `assemble.ts: stripTrailingEllipsis`); a preview too short to discriminate (below a `PREFIX_FLOOR`-style guard) → `orphaned`, never a false `in-graph`.
|
||||
4. **Empty-state** — empty/absent `## Recent Posts` → empty report, no throw.
|
||||
|
||||
**RED proof (recorded — logic-RED):** with the mis-classifying stub in place, `(cd scripts/brain && npm test)` → the **named tier assertions** fail (e.g. "expected `orphaned-in-state`, got …"). Capture *that* transcript as the iron-law artifact in STATE/changelog — not a bare import error. **Circuit-breaker:** if the RED run is not recorded before the real `reconcile.ts` logic lands, halt and redo Step 1 (the RED proof is the only circuit breaker in this plan).
|
||||
|
||||
## Step 2 — (GREEN) `reconcile.ts` — pure core + parser + IO seam
|
||||
|
||||
`scripts/brain/src/reconcile.ts`:
|
||||
|
||||
- **`parseRecentPosts(stateText: string): RecentPost[]`** — regex the `## Recent Posts` section against the **writer's** format (`updatePostTracking`, `state-updater.mjs:116`: `- [${postDate}] "${hookPreview}" (${charCount}) - ${postTopic}`) → e.g. `/^- \[(\d{4}-\d{2}-\d{2})\] "(.*)" \((\d+)\) - (.+)$/gm`. **NB:** this is NOT `pruneContentHistory`'s regex — that one (`:145`) is **date-only** (`/^- \[(\d{4}-\d{2}-\d{2})\] .+$/gm`) and captures no hook/chars/topic; `parseRecentPosts` is the *first* full-field reader of this section and tracks the writer, not the pruner. Pure.
|
||||
- **`reconcileRecentPosts({recentPosts, records, graph}): ReconcileNode[]` + `summarize(...)`** — pure, no FS/clock. `records` is the **full `PublishedRecord[]`** (carry `body`); `graph` is `PostGraphNode[]` (carry `contentId` + analytics `match`, NO `body`). Reuse `assemble.ts`'s `normalize` / `stripTrailingEllipsis` idiom (brain-local copy, NOT a new shared util — the repo's accepted duplication, `dataRoot.ts` header). Match `recentPost.hook` (ellipsis-stripped, normalized) as a **prefix of** `record.body`; confidence by date equality; below the prefix floor → `orphaned`. Then look up the matched `record.id` in `graph` to read its analytics tier (`in-graph` if the node's `match.confidence !== 'none'`, else `in-brain-only`).
|
||||
- **`loadRecentPosts(): RecentPost[]`** — the new cross-seam IO. Resolve the state file with the **canonical precedence already exported as `data-root.mjs: getStateFile()`** — `process.env.STATE_FILE || join(resolveHome(), ".claude", "linkedin-studio.local.md")` where `resolveHome() = process.env.HOME || process.env.USERPROFILE || homedir()` (`data-root.mjs:17-19`). **NOT bare `homedir()`** — that silently ignores a `HOME`/`USERPROFILE` override the writer honours (`state-updater.mjs:12`); and **NOT `dataRoot()`** (that is the *dir* under `LINKEDIN_STUDIO_DATA`). The brain TS copies `getStateFile()`'s chain because it cannot import the `.mjs`. Read read-only; absent → `[]` (fresh-clone safe). Header caveat: this is a **new** root-skew seam (state file via `STATE_FILE`, a different root than the brain dataRoot) — not a mirror of `assemble.ts`'s `ANALYTICS_ROOT` note.
|
||||
|
||||
**GREEN gate (brain):** `(cd scripts/brain && npm test)` → reconcile cases pass; record the new live `tests` total → `BRAIN_TESTS_FLOOR` 114→**that number** (recorded, not guessed; update the inline breakdown comment `test-runner.sh:718` with "+ SB-S3e N [reconcile]").
|
||||
|
||||
## Step 3 — (GREEN) Wire `brain reconcile` into the CLI
|
||||
|
||||
`scripts/brain/src/cli.ts`:
|
||||
- Add `runReconcile()` (model on `runAssemble`, `:161-191`). **Loader (BLOCKER fix):** do NOT use `listPublished()` — it takes no args and returns body-less `PublishedSummary[]` (`ingest.ts:262-289`). Instead reuse `runAssemble`'s inline loader (`cli.ts:163-174`): `readdirSync(pubDir).filter(.md).map(parsePublishedRecord)` → real `PublishedRecord[]` (with `body`/`specifics`/`trends`). Then `const analytics = loadAnalyticsRows()`, `const graph = assemblePostGraph({records, analytics})`, `const recentPosts = loadRecentPosts()`, and **call the core by its literal name** `reconcileRecentPosts({recentPosts, records, graph})` (no alias/re-export wrapper — Section 16f Check B greps `cli.ts` for the literal `reconcileRecentPosts`). Print per-post lines (`· <date> · <tier> · "<hook>"`) + summary + nudge ("K created posts are not in the brain graph — `brain ingest --file <p>` to feed them"). Read-only.
|
||||
- Dispatch: insert `if (command === "reconcile") return runReconcile(flags);` **after** the `assemble` dispatch line (`:312`, i.e. at the new `:313` before `usage(...)`).
|
||||
- Usage: add `" reconcile"` to the usage block **after** the `" assemble"` line (`:94`).
|
||||
|
||||
**GREEN gate (lint):** Section 16f Check B now passes (see Step 5).
|
||||
|
||||
## Step 4 — (b) Retire `content-history.md` (deletions)
|
||||
|
||||
Apply the eight edits in the (b) table. Order: remove the **writer** (`state-update-reminder.md` Section 5) first, then template, migration entry, gitignore, gate guard, ref-doc, comment. Then trim `migrate-data.test.mjs` — drop the content-history fixture write + the two assertions + the words "content-history moved" from the test-(a) name; cases a–e remain.
|
||||
|
||||
**Gate (hook suite — separate runner), recorded:** `node --test hooks/scripts/__tests__/migrate-data.test.mjs` → cases a–e green (capture the transcript — R8 is demonstrated, not asserted). Confirm cases b–e never expected `analytics/content-history.md` as a present path (they don't — only the deleted case-(a) fixture/assertions referenced it; b–e exercise MOVE_DIRS / idempotency / collision / empty), so removing the B1 MOVE entry cannot regress them. **SC1:** `grep -rn "content-history" hooks/ config/ scripts/test-runner.sh .gitignore references/data-path-convention.md` → zero (the README `:120` *concept* mention is out of this grep set; confirm it is the concept, leave it).
|
||||
|
||||
## Step 5 — (RED→GREEN) Gate Section 16f + floors + SC2 narrow
|
||||
|
||||
Insert **Section 16f** after Section 16e (`test-runner.sh:885`, before Section 17 `:886`), modelled on 16e's two-check idiom (self-test + real-file grep, `grep -qF` fixed-string):
|
||||
|
||||
- **Literals:** `RECON_CLI_LIT='command === "reconcile"'`, `RECON_CORE_LIT='reconcileRecentPosts'`.
|
||||
- **`reconcile_wired()`** — wired iff a probe carries BOTH literals (echo twice; `grep -qF` — both literals contain regex-special chars, so `-F` is mandatory).
|
||||
- **Check A (self-test, unconditional, non-vacuous):** a fully-wired probe MUST be detected; the rejected probes must each miss **exactly one** literal (the load-bearing discriminators), plus a specificity decoy:
|
||||
- **D1 (discriminator):** dispatch present, core call absent — `command === "reconcile"` but no `reconcileRecentPosts` → must REJECT (proves the core literal is load-bearing).
|
||||
- **D2 (discriminator):** core call present, dispatch absent — `reconcileRecentPosts(...)` but no `command === "reconcile"` → must REJECT (proves the dispatch literal is load-bearing).
|
||||
- **D3 (specificity):** a sibling command fully wired to the WRONG domain — `command === "assemble"` + `assemblePostGraph` (neither reconcile literal) → must REJECT (proves the predicate is reconcile-specific, not "any command + any core").
|
||||
→ `pass`/`fail` "reconcile self-test: full wiring detected; D1/D2 single-literal forms + D3 `assemble` specificity decoy rejected".
|
||||
- **Check B (real-file grep, unconditional):** `grep -qF "$RECON_CLI_LIT" scripts/brain/src/cli.ts && grep -qF "$RECON_CORE_LIT" scripts/brain/src/cli.ts` → `pass` "brain CLI wired to reconcile (dispatch + core call by literal name)" else `fail`. Both literals live in `cli.ts` (the dispatch `if` + `runReconcile`'s literal-name call to the core — Step 3 pins "no alias"), so one file carries both → deps-free, like 16e Check B.
|
||||
- **Header enumeration (`:33-39`):** insert the 16f clause **between** the 16e clause's terminal "…in Section 16e;" (`:38`) and the "the assertion-count anti-erosion floor (SC6) in Section 18." tail — preserving 16e → 16f → 18 ordering. The range "Sections 8–18" is unchanged (16f is in range).
|
||||
- **`ASSERT_BASELINE_FLOOR` (`:966`):** 82 → **84** (+2 deps-free unconditional checks — they lift the deps-absent minimum, NOT pinned to the deps-present TOTAL); extend the history comment ("+2 for SB-S3e's two Section-16f checks (reconcile self-test + brain-CLI reconcile grep) = 84").
|
||||
- **`SC2_CLASSES` (`:592`):** delete the `assets/analytics/content-history\.md|` alternative (now retired). The SC2 check is still one pass/fail → gate TOTAL change comes only from 16f's +2.
|
||||
|
||||
**RED proof:** between Steps 1 and 3, a gate run shows Section 16f Check A passing (self-test is self-contained) and **Check B FAILING** (cli.ts not yet wired); after Step 3 both pass.
|
||||
|
||||
**GREEN gate (full):** `bash scripts/test-runner.sh` → FAIL=0; TOTAL recounted **live**. **Recorded pre-change baseline (verified live this session, 2026-06-23): 97/0/0** → expected **99** after 16f's +2 (recount at land, never pin); ASSERT floor 84 honoured; SC2-dry-run green with the narrowed regex.
|
||||
|
||||
## Step 6 — Doc reconciliation
|
||||
|
||||
Edit the `architecture.md` SB-S3 build-row: `S3e ✅` — "dead `content-history.md` retired; `brain reconcile` reconciles post-tracking ↔ published ↔ analytics (read-side; auto-capture = follow-up)" — and note the arc is complete (S4 EØS-connector optional). No other architecture edit.
|
||||
|
||||
**CLAUDE.md surface (open-Q4, resolved in-plan):** `brain reconcile` is an **internal brain-CLI subcommand**, the same class as `brain assemble`/`ingest`/`consolidate` — none of which appear in CLAUDE.md's command/agent tables (those count *plugin* commands/agents). So **no CLAUDE.md count change**; the `brain reconcile` mention lives in the `architecture.md` build-row above. Confirm at the gate, but the default is no-plugin-surface.
|
||||
|
||||
## Step 7 — Behavioural verification (manual, recorded — SC4/SC6)
|
||||
|
||||
Not unit-testable (operator-accepted command-testing gap). Honest procedure, result recorded at land:
|
||||
- **SC4 (e2e):** seed a temp root — a `## Recent Posts` block (via `STATE_FILE`), 1–2 `ingest/published` records (one matching a Recent-Posts entry, one not), an analytics batch — run `brain reconcile`; confirm it prints in-graph / orphaned tiers + the summary + nudge.
|
||||
- **SC6 (root-skew):** with `STATE_FILE=<tmp>` it reads silo 1 from the temp file; unset → falls back to `$HOME/.claude/linkedin-studio.local.md`. **Honesty hedge (verifiseringsplikt):** if the per-call seam can't be exercised, record wiring-inspected-only, NOT a behavioural pass.
|
||||
|
||||
## Step 8 — Land
|
||||
|
||||
STATE "Telling" + "👉 NESTE" updated (S3e done → arc complete; next = strategic checkpoint / product-maturity). Commit. Mixed: reconcile + retirement + gate are code (`[skip-docs]`); brief+plan+architecture are docs. **Push only inside the window** (`date '+%u %H:%M'` first; `origin` = PUBLIC `open/` → confirm before push). No version bump (additive within v0.5.2 dev; release separate).
|
||||
|
||||
## Verification (testable)
|
||||
|
||||
| SC | Check | Command | Expected |
|
||||
|----|-------|---------|----------|
|
||||
| SC1 | content-history retired | `grep -rn content-history hooks/ config/ scripts/test-runner.sh .gitignore references/` + `ls config/content-history.template.md` | zero hits; file gone |
|
||||
| SC2 | migration green | `node --test hooks/scripts/__tests__/migrate-data.test.mjs` | cases a–e pass |
|
||||
| SC3 | reconcile core (RED→GREEN) | `(cd scripts/brain && npm test)` | reconcile cases pass; floor recounted |
|
||||
| SC4 | reconcile wired | `bash scripts/test-runner.sh` + manual `brain reconcile` | Section 16f Check B passes; report prints |
|
||||
| SC5 | gate green + floor | same run | 99/0/0 (confirm live); ASSERT floor 84; SC2-dry-run green |
|
||||
| (red proof) | failing-test-first | gate + brain run BETWEEN Steps 1 and 3 | reconcile test fails; 16f Check B fails |
|
||||
| SC6 | root-skew | manual, `STATE_FILE` set/unset | reads the pointed file / falls back (or honest fallback recorded) |
|
||||
| SC7 | counts reconciled | inspect | architecture S3e ✅; STATE telling updated; no CLAUDE.md command/agent count change |
|
||||
|
||||
## Risks
|
||||
|
||||
- **R1 — vacuous gate.** Mitigated by the 16e-style non-vacuity self-test with the `assemble` sibling-command decoy.
|
||||
- **R2 — seam-mismatch (the load-bearing one).** `loadRecentPosts()` MUST resolve the state FILE via `getStateFile()`'s exact chain — `STATE_FILE || join(resolveHome(), '.claude', 'linkedin-studio.local.md')`, `resolveHome() = HOME||USERPROFILE||homedir()` (`data-root.mjs:17-19,30-31`) — NOT bare `homedir()` (ignores a HOME/USERPROFILE override) and NOT `dataRoot()` (the DIR under `LINKEDIN_STUDIO_DATA`). A divergence silently reads the wrong path. Mitigated: chain copied from the canonical `getStateFile()` (not `state-updater.mjs:12`, which falls through to `''` — a latent bug `data-root.mjs:14-16` warns against); SC6 exercises both `STATE_FILE` branches.
|
||||
- **R3 — parse drift.** The reconcile entry regex must track the **writer** `updatePostTracking` (`state-updater.mjs:116`); if the writer's format changes, the reader silently misses. `pruneContentHistory` (`:145`) is date-only and would NOT catch such drift, so it is not a co-mover. Mitigated: a **golden-string test** feeds a literal writer-format entry through `parseRecentPosts` and asserts the round-trip — pinning the exact write format as the contract.
|
||||
- **R4 — retirement over-reach.** Deleting a *conceptual* "content history" mention (README `:120`, or `pruneContentHistory`'s name) would break live behaviour. Mitigated: the (b) table is file-plumbing-only; the scope fence + SC1's bounded grep set exclude the concept; `pruneContentHistory` (operates on `## Recent Posts`) is explicitly out.
|
||||
- **R5 — `$`-injection on READ.** `parseRecentPosts` uses `match`/capture (read), never `String.replace` with an untrusted replacement — so a `$`-bearing hook/topic round-trips verbatim (Step 1 case 1 pins it). No write path → no Section-13 `state-updater` surface touched.
|
||||
- **R6 — floor false-fail on fresh clone.** 16f's two checks are deps-free (+2 → 84, correct). `BRAIN_TESTS_FLOOR` only checked when the brain suite runs (deps-present); a fresh clone warn-skips it.
|
||||
- **R7 — honest-limit overclaim.** The brief/CLI must state plainly that read-side reports the coverage gap but cannot reconstruct un-captured specifics/trends (auto-capture = follow-up). Mitigated: §3 honest-limits + Step 7 verifiseringsplikt hedge.
|
||||
- **R8 — migrate idempotency.** Removing the B1 MOVE entry must not break the `.migrated` marker / COPY-MOVE classes. Mitigated: cases b–e (idempotent re-run, collision, empty) stay green in Step 4's run.
|
||||
|
||||
## Plan-critic — folded
|
||||
|
||||
`voyage:plan-critic` (3 blockers + 6 major + 4 minor) — each verified directly against the code before folding:
|
||||
|
||||
- **B1 — body-less loader (load-bearing):** `listPublished()` takes no args and returns `PublishedSummary` (`{id,provenance,published_date,firstLine}`, `ingest.ts:262-289`) — NO `body`. The reconcile prefix-match AND `assemblePostGraph` both need `PublishedRecord.body`. **Folded Step 3:** use `runAssemble`'s inline `readdirSync().map(parsePublishedRecord)` loader (`cli.ts:163-174`), which yields real `PublishedRecord[]` (and already builds a `bodyById` Map). ✅
|
||||
- **B2 — core-signature contradiction:** brief said `{recentPosts, graph}`; `PostGraphNode` has no `body` → graph-only can't prefix-match. **Folded** to the 3-arg `{recentPosts, records, graph}` in brief §3(c).1 + plan Steps 1/2 — one signature, agreed across brief/test/core. ✅
|
||||
- **B3 — regex misattribution:** plan cited `pruneContentHistory`'s `:145` regex as full-field; it is **date-only**. **Folded Step 2 + R3:** the parser tracks the **writer** `:116`; `parseRecentPosts` is the first full-field reader; a golden-string test pins the write format. ✅
|
||||
- **M4 — HOME seam:** `loadRecentPosts` must use `getStateFile()`'s `resolveHome()` chain (`HOME||USERPROFILE||homedir()`), not bare `homedir()` (ignores an override the writer honours). **Folded Step 2 + R2**, citing `data-root.mjs:17-19,30-31` as canonical (not `state-updater.mjs:12`, which falls through to `''`). ✅
|
||||
- **M5 — weak gate decoys:** the both-absent `assemble` decoy was trivial. **Folded Step 5:** D1/D2 now miss **exactly one** literal each (the load-bearing discriminators); D3 (`assemble` sibling) is reframed as a specificity decoy. ✅
|
||||
- **M6 — Check-B literal name:** `reconcileRecentPosts` appears in `cli.ts` only if `runReconcile` calls the core by its literal name. **Folded Step 3** ("no alias/wrapper") + Step 5 Check-B wording. ✅
|
||||
- **M7 — unrecorded TOTAL:** **Folded Step 5** — recorded the live pre-change gate **97/0/0 (2026-06-23)**; 99 is the expected post-16f count, recount at land. ✅
|
||||
- **M8 — migrate idempotency asserted not shown:** **Folded Step 4** — require the recorded `node --test migrate-data.test.mjs` green run; confirm cases b–e never expected content-history present. ✅
|
||||
- **M9 — SC1 grep divergence:** the brief's `--include`-filtered grep would skip `.gitignore`/`.md`. **Folded** — brief SC1 unified with the plan's unfiltered explicit-list grep. ✅
|
||||
- **m10 — header insertion point:** **Folded Step 5** — insert the 16f clause between the 16e clause terminal and the Section-18 tail. ✅
|
||||
- **m11 — `:312`/`:94` line refs:** **Folded Step 3** — "after :312" / "after :94" (insertion, not the existing line). ✅
|
||||
- **m12 — CLAUDE.md deferred:** **Folded Step 6** — resolved in-plan: `brain reconcile` is internal brain-CLI (like `assemble`), no CLAUDE.md count change. ✅
|
||||
- **m13 — circuit-breaker:** **Folded Step 1** — "if RED not recorded before reconcile.ts logic lands, halt and redo Step 1." ✅
|
||||
|
||||
**`voyage:scope-guardian`: ALIGNED** — 0 creep / 0 gaps / 0 dependency issues. All 14 IN items map to a step; no NON-GOAL file touched; scope fence consistent (slightly broader/defensive); doc reconciliation proportional. Every file/line/symbol verified live.
|
||||
|
||||
Cross-checks confirmed sound: the (b) retirement arm was already nearly executable (all 8 surfaces verified); floor arithmetic (ASSERT 82→84 from 16f's +2; SC2-narrow nets 0; BRAIN 114→recount); the "read-side does not close the gap" honesty (auto-capture = the real closer, follow-up) stands; `$`-injection is absent on the read path (capture/`match`, never `String.replace`).
|
||||
65
docs/second-brain/research/connector-egress.md
Normal file
65
docs/second-brain/research/connector-egress.md
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
# Research — LinkedIn content/data egress for an individual creator (2026-06-23)
|
||||
|
||||
> Research thread `connector-recon`, 2026-06-23. Verification-duty: every technical claim backed by an official source/URL; unverifiable items marked "IKKE VERIFISERT". This is research INPUT to `../architecture.md` — preserved verbatim so nothing is lost.
|
||||
|
||||
**Bottom line:** For ONE creator pulling their OWN content out of LinkedIn there is exactly one realistic *automated* path, and it is geofenced to **EU/EEA + Switzerland**: the **DMA Member Data Portability API** (self-serve, scope `r_dma_portability_self_serve`). Outside that region the honest answer is **manual native export** for content + **manual analytics CSV download** for per-post metrics. Full hands-off automation is NOT available outside the EU/EEA. Engagement/impression analytics are NOT in any portability/export *content* path — they live only in the separate manual analytics CSV.
|
||||
|
||||
## 1. Path-by-path table
|
||||
|
||||
| # | Path | What it captures (your OWN data) | Verdict | Source |
|
||||
|---|------|----------------------------------|---------|--------|
|
||||
| 1a | **DMA Member Data Portability API — Member Snapshot** (`r_dma_portability_self_serve` / `r_dma_portability_member`) | **Full post text** (`MEMBER_SHARE_INFO`: all shares/re-shares incl. date, URL, shared comments, visibility), **articles** (`ARTICLES`), profile, connections, comments (`ALL_COMMENTS`), reactions you gave (`ALL_LIKES`), reposts, messages — ~60+ domains. **Historical** data included. **NO impression/reach/engagement-received analytics domain exists.** | **AUTO** — but **EU/EEA + Switzerland members ONLY** | learn.microsoft.com/en-us/linkedin/dma/member-data-portability/member-data-portability-member/?view=li-dma-data-portability-2026-05 ; …/shared/snapshot-domain?view=li-dma-data-portability-2026-05 |
|
||||
| 1b | **DMA Member Data Portability API — Member Changelog** | Real-time stream of your interactions (posts created, comments, reactions) from consent onward; **queryable only for the last 28 days** | **AUTO** (rolling 28-day window) — EU/EEA + CH only | (same member page, Changelog section) |
|
||||
| 1c | **Marketing API / Community Management API** | Org/Page-level posting + analytics; NOT a self-serve member-content path | **RISKY/BLOCKED for individuals** — LinkedIn Partner Program approval (legal entity, use-case review, weeks/months) | learn.microsoft.com/en-us/linkedin/marketing/community-management/community-management-overview?view=li-lms-2026-05 ; connectsafely.ai/articles/linkedin-api-complete-guide-2026 |
|
||||
| 2 | **Native "Get a copy of your data"** (Settings → Data Privacy) | ZIP of **CSV + JSON + HTML**: profile, connections, messages, comments, **posts** (Shares), **articles** (HTML folder). Basic categories "within minutes"; **larger archive ~24h** (72h download window). **Saved-items file is URLs+dates only.** **No analytics.** | **MANUAL** | linkedin.com/help/linkedin/answer/a1339364/downloading-your-account-data |
|
||||
| 3a | **Native newsletter content** | **NO native RSS feed.** Subscription = email/notification per edition only. Edition text lives as articles. | **MANUAL** | linkedin.com/help/linkedin/answer/a517925 |
|
||||
| 3b | **Newsletter via your own subscription email** | Each new edition arrives as an **email to you** → ingest your own mailbox / email-to-RSS | **MANUAL (clean)** | linkedin.com/help/linkedin/answer/a517925 |
|
||||
| 3c | **Third-party newsletter→RSS scrapers** (rss.app, linkedin-newsletter-rss) | Scrapes public newsletter/article URL into RSS | **RISKY** (third-party scraping) | rss.app/rss-feed/linkedin ; github.com/chrisns/linkedin-newsletter-rss |
|
||||
| 4a | **Per-post analytics CSV export** (native dashboard "Export") | **Per-post** impressions, unique viewers, clicks, reactions, comments, shares, engagement rate. **Manual, ~1-year range cap.** Profile views NOT exportable. **ONLY path to your engagement metrics.** | **MANUAL** | tryordinal.com/blog/how-to-export-analytics-from-linkedin-to-excel-or-a-csv ; linkedin.com/help/linkedin/answer/a551206/ |
|
||||
| 4b | **Zapier / Make** | Trigger on *your* new posts (publish-side); thin/no historical pull of post bodies+analytics; bound by LinkedIn's official connector scopes | **MANUAL-ish / limited** | connectsafely.ai/articles/is-linkedin-automation-safe-tos-scraping-guide-2026 |
|
||||
| 4c | **Phantombuster / scrapers / browser extensions** | Can extract post text + visible engagement counts | **RISKY** — breaches User Agreement (prohibited automation); active ban-wave enforcement | linkedin.com/help/linkedin/answer/a1341387 ; connectsafely.ai/articles/is-linkedin-automation-safe-tos-scraping-guide-2026 |
|
||||
|
||||
## 2. Single cleanest realistic ingest path for ONE user (own data)
|
||||
|
||||
Splits by region, and even at best it is TWO streams, not one:
|
||||
|
||||
**EU/EEA or Switzerland → near-full automation for CONTENT:**
|
||||
- **Member Data Portability API, self-serve.** Create a dev app **using LinkedIn's mandated default Company Page** ("Member Data Portability (Member) Default Company" — you must NOT create your own page), request the *Member Data Portability (Member)* product, generate a token via the OAuth Token Generator with scope `r_dma_portability_self_serve`, consent, then call **Member Snapshot** (`GET /rest/memberSnapshotData?q=criteria`) for historical full post text (`MEMBER_SHARE_INFO`) + `ARTICLES`, and poll **Member Changelog** for the rolling 28-day live stream. Token valid up to **1 year** before re-consent. Genuinely automatable for a second-brain content store.
|
||||
- **BUT analytics still come from the manual CSV (4a)** — portability exposes reactions/comments *you made*, not impressions/reach *received*.
|
||||
|
||||
**Outside EU/EEA/CH → manual is the honest answer:**
|
||||
- Content: native "Get a copy of your data" (larger archive, ~24h, CSV+JSON+HTML) — paths 2 + 3.
|
||||
- Analytics: native per-post analytics CSV (4a).
|
||||
- Newsletter: own subscription email (3b) or article HTML from the archive.
|
||||
|
||||
**Is full automation possible?** Only inside EU/EEA/CH, and only for content (not received-analytics). Outside, **manual export/paste is the correct, ToS-safe answer.** Scraper/extension/Phantombuster routes are a **User Agreement breach** with active enforcement (LinkedIn's reported Q1-2026: 23.5M automated sessions flagged) — do not bake into the plugin's ingest.
|
||||
|
||||
## 3. What's uncertain / changing (date: 2026-06-23)
|
||||
|
||||
- **Geofence may widen.** Portability API is DMA-driven, explicitly "**at this time**… EU/EEA + Switzerland." Re-check the MS Learn moniker (`li-dma-data-portability-2026-05`) each quarter. IKKE VERIFISERT that it will ever cover non-EU members.
|
||||
- **Analytics-in-portability gap.** No Snapshot domain carries received impressions/reach/engagement-rate (`ALL_LIKES`/`ALL_COMMENTS` = actions *you* took). Treat "portability gives you your analytics" as FALSE until a future domain appears. Native analytics CSV stays the only metrics source.
|
||||
- **Newsletter editions in export/snapshot.** `ARTICLES` covers authored articles; whether every *newsletter edition* is reliably captured as an article vs. needing the email path is **partially uncertain** — native export help does not enumerate newsletter editions. Verify against a real export.
|
||||
- **Saves / dwell** remain non-exportable (consistent with the plugin's existing S16 note). IKKE VERIFISERT any change.
|
||||
- **Versioned API drift.** Snapshot uses `LinkedIn-Version: YYYYMM`; domain list/schemas change month-to-month. Pin a version, re-validate.
|
||||
|
||||
## 4. Sources (accessed 2026-06-23)
|
||||
|
||||
**Official / authoritative:**
|
||||
- Member Data Portability (Member), MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/dma/member-data-portability/member-data-portability-member/?view=li-dma-data-portability-2026-05
|
||||
- Member Snapshot API, MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/dma/member-data-portability/shared/member-snapshot-api?view=li-dma-data-portability-2026-05
|
||||
- Member Snapshot Domain list, MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/dma/member-data-portability/shared/snapshot-domain?view=li-dma-data-portability-2026-05
|
||||
- Member portability APIs, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a6214075
|
||||
- Community Management Overview, MS Learn (2026-05): https://learn.microsoft.com/en-us/linkedin/marketing/community-management/community-management-overview?view=li-lms-2026-05
|
||||
- Download your account data, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a1339364/downloading-your-account-data
|
||||
- Export your Page analytics report, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a551206/
|
||||
- Manage a newsletter, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a517925
|
||||
- Prohibited software and extensions, LinkedIn Help: https://www.linkedin.com/help/linkedin/answer/a1341387
|
||||
- LinkedIn DMA Portability API Terms: https://www.linkedin.com/legal/l/portability-api-terms
|
||||
|
||||
**Secondary (corroborating, not authoritative):**
|
||||
- connectsafely — LinkedIn API 2026: https://connectsafely.ai/articles/linkedin-api-complete-guide-2026
|
||||
- connectsafely — automation/ToS 2026: https://connectsafely.ai/articles/is-linkedin-automation-safe-tos-scraping-guide-2026
|
||||
- tryordinal — Export LinkedIn analytics (Jan 2026): https://www.tryordinal.com/blog/how-to-export-analytics-from-linkedin-to-excel-or-a-csv
|
||||
- rss.app LinkedIn / chrisns/linkedin-newsletter-rss: https://rss.app/rss-feed/linkedin · https://github.com/chrisns/linkedin-newsletter-rss
|
||||
|
||||
**Verification log (key claims → source):** Portability API exists + self-serve scope + EU/EEA+CH-only + 1-yr token → MS Learn member page. Full post text + articles + historical, no analytics domain → MS Learn snapshot-domain list (verbatim `MEMBER_SHARE_INFO`, `ARTICLES`, `ALL_LIKES`, `ALL_COMMENTS`; no impressions domain present). Changelog 28-day window → MS Learn member page. Native export = CSV+JSON+HTML, articles as HTML, ~24h, saved-items URLs-only → LinkedIn Help + corroborating guides. No native newsletter RSS → LinkedIn Help newsletter page (silent on RSS) + third-party scrapers fill the gap. Per-post analytics CSV manual, ~1yr cap, only metrics source → LinkedIn Help + tryordinal. Scraping = User Agreement breach w/ enforcement → LinkedIn Prohibited Software + connectsafely.
|
||||
132
docs/second-brain/research/secondbrain-sota.md
Normal file
132
docs/second-brain/research/secondbrain-sota.md
Normal file
|
|
@ -0,0 +1,132 @@
|
|||
# Research — Second Brain / Personal-Memory SOTA 2026 (synthesis + architecture rec)
|
||||
|
||||
> Research thread `secondbrain-recon`, 2026-06-23. Four streams (agent memory, PKM, profile-evolution, pitfalls) with ESTABLISHED / EMERGING / SPECULATIVE labels. Research INPUT to `../architecture.md` — the architecture's design choices trace to this. Preserved so nothing is lost.
|
||||
|
||||
## PART A — STATE OF THE ART (sourced, calibrated)
|
||||
|
||||
### 1. Agent memory architectures
|
||||
|
||||
**Three-tier taxonomy is the design default** (ESTABLISHED). Episodic (timestamped *what happened*), semantic (distilled *what is true*), procedural (*how to behave*). Formalized for LLM agents by CoALA (arXiv:2309.02427); shared vocabulary across Letta, Mem0, Zep, LangMem. Load-bearing for us: **keep episodic and semantic physically separate** — mixing raw events with distilled facts degrades retrieval both ways. Procedural is rarest; only LangMem makes it first-class.
|
||||
|
||||
**Hybrid vector + structured/graph beats pure vector** (ESTABLISHED). Pure embeddings fail on multi-hop, temporal ("what did the user want in Q1 vs now"), exact-identifier, and contradiction queries. 2026 pattern = query router over vector (semantic) + graph/relational (entities + temporal validity) + BM25 (keyword), fused via Reciprocal Rank Fusion + cross-encoder rerank (+39.7% MRR@3, arXiv:2604.01733). Named systems:
|
||||
- **Mem0** (arXiv:2504.19413, ~48K stars) — vector-primary + entity linking; LLM extracts facts per turn; ~90% token reduction. Benchmark numbers vendor-disputed.
|
||||
- **Zep/Graphiti** (arXiv:2501.13956) — **bi-temporal knowledge graph**: every fact an edge with `valid_from`/`valid_until`. 63.8% vs Mem0's 49.0% on independent LongMemEval. **Most important pattern for us** (staleness).
|
||||
- **Letta/MemGPT** (arXiv:2310.08560) — OS-tiered (core/recall/archival), agent self-manages via tools. 74.0% LoCoMo.
|
||||
- LangMem (EMERGING), Cognee (EMERGING, graph-RAG for docs), A-MEM (EMERGING, NeurIPS 2025).
|
||||
|
||||
**Consolidation = the "reflection/sleep" pattern** (ESTABLISHED original; sleep-time-compute EMERGING). Stanford Generative Agents (arXiv:2304.03442): append-only memory stream → LLM importance-scores each event → periodic reflection synthesizes higher-order insights stored back as semantics. Letta sleep-time compute (arXiv:2504.13171) runs it as background agents during idle so the main loop stays fast.
|
||||
|
||||
**Forgetting is mandatory** (ESTABLISHED TTL/LRU; EMERGING importance-weighted decay). "Append-only memory stores are a trap." Tiers: session ~24h, project ~6mo, core indefinite, cold-archive after 90–180d unused. Mem0 decay is a *retrieval re-ranking* layer (recent → 1.5× boost, unused → 0.3× damp), not deletion. The Generative-Agents **retrieval triad** (recency + importance + relevance) is still the reference scoring formula.
|
||||
|
||||
### 2. PKM methods + AI-native tools
|
||||
- **PARA** (Forte) — top-down, organize-by-*actionability*; strong for ops, no native linking.
|
||||
- **Zettelkasten / Evergreen** (Luhmann/Matuschak) — atomic note (one idea), dense links, **emergent** bottom-up structure; the cognitive work is *writing the note*, which embeddings can't replace.
|
||||
- **LYT/Maps of Content** (Milo) — navigation layer over the graph.
|
||||
|
||||
**2026 consensus is NOT "links win"** — embeddings make folder-vs-link partly moot for *retrieval* while reaffirming neither scales alone. Mem (OpenAI embeddings + Pinecone), Tana (every bullet a graph node, typed Supertags), Obsidian Smart Connections (*local* embeddings — privacy). None *auto-organize* — they auto-*retrieve*. AI collapses CODE's **Organize** step (misfiling recoverable) and accelerates **Capture**; it does NOT do **Distill** (cross-temporal relevance) or **Express** (human synthesis). Sober dissent: "the best memory tool is the one you actually keep feeding" — capture/maintenance friction dominates retrieval sophistication.
|
||||
|
||||
### 3. Profile-evolution / preference-learning
|
||||
**Two-layer profile is standard** (ESTABLISHED): static (stable) + dynamic (evolving). PersonaMem (COLM 2025): frontier models **freeze on the static snapshot, fail to track the dynamic layer** — frozen-past-self is empirically real.
|
||||
|
||||
**A plugin gets only app-level adaptation** (no model-level RLHF/DPO/per-user-LoRA). App-level = **RAG** (retrieve past interactions) + **PAG** (summarize history into a compact profile, inject, re-summarize on update — PURE, SIGIR 2025). **Style/voice = few-shot from the user's own corpus**; content-similar + length-matched selection beats random (arXiv:2509.14543), but fidelity to a private individual's implicit style is limited even at SOTA.
|
||||
|
||||
**Drift guards (the crux):**
|
||||
- **Recency weighting** (ESTABLISHED): `score = α·sim + (1-α)·f_time(Δt)`.
|
||||
- **Event-driven invalidation** (EMERGING): OpenAI "Dreaming" auto-rewrites "going to X in July" → "went to X" after the date (recall 41.5%→82.8%). State-typed facts (active/completed/expired).
|
||||
- **Explicit user curation** (ESTABLISHED): ChatGPT/Claude/Gemini all expose view/edit/delete; ChatGPT (Oct 2025) added **roll-back to prior versions**. Claude is notably transparent about *when* memory influences a reply.
|
||||
- **Contradiction detection** (EMERGING): 4-type typology (MemConflict, arXiv:2605.20926) — temporal-update (supersede), contradictory (keep both timestamped), condition-dependent (scope), distractor (drop). Zep handles structurally via validity intervals.
|
||||
- **Confidence/evidence thresholds before storing** (EMERGING): don't promote a single observation to the stable profile — require confirmation, ≥N independent observations, or a min confidence score. Separate episodic (session) from semantic (cross-session).
|
||||
|
||||
**Sycophancy is the personalization-specific landmine** (ESTABLISHED). RLHF reward models prefer agreement over truth; **providing user memory/profile context measurably INCREASES sycophancy** (arXiv:2509.12517), worse across multi-turn pressure (SYCON Bench). Documented OpenAI insider admission: showing users their own profile assessments triggered backlash, and one response was to make the model *less honest about the user*. Best app-level mitigations: **explicit anti-sycophancy system instruction** ("maintain your position unless given new evidence/logic" — up to 64% improvement on Turn-of-Flip) and **"ask, don't tell"** framing. Users *prefer* sycophancy when offered the choice → must be a built-in default, not a toggle.
|
||||
|
||||
### 4. Pitfalls
|
||||
- **Drift / model collapse** (ESTABLISHED, Shumailov et al., Nature 2024): a profile learning from its *own* AI output narrows and degrades. **Never "replace"; always "accumulate" with provenance.** ACUTE FOR US: a content engine that learns voice from posts *it drafted* collapses the voice toward its own priors. Voice must learn from *actual published, human-edited* output only — the existing `voice-scrubber` already encodes this ("gold standard = approved editions, not the English post corpus").
|
||||
- **Staleness** (ESTABLISHED, STALE benchmark arXiv:2605.06527): *all* tested models+frameworks (incl. Zep, Mem0) fail to notice a valid memory went obsolete. RankSquire 50K-session prod test: 93.4% at deploy → 58.2% at month 3 (41% stale). Mitigate: timestamp every write, surface at retrieval, RAG live sources for fast-changing facts.
|
||||
- **Privacy / memory poisoning** (ESTABLISHED, OWASP **ASI06**): MINJA (NeurIPS 2025) — 98.2% injection success from query-only access; ZombieAgent (Jan 2026) — zero-click persistent injection via ChatGPT Connectors. Persistent poisoning is *worse* than ordinary prompt injection (attack/execution temporally decoupled). **Local-first/plain-file shrinks the blast radius** (no remote exfil endpoint; git = audit/rollback).
|
||||
- **Lock-in** (ESTABLISHED): proprietary embedding stores can't export; plain-text/Markdown + git is the answer ("swap the model, feed it the same Markdown").
|
||||
- **Frozen past self** (ESTABLISHED): profile over-represents who you were, pulls present output backward. Mitigate: time-weighted retrieval + periodic user-authored "who am I now" anchor that deprecates old inferences.
|
||||
- **Context rot / over-retrieval** (ESTABLISHED, Chroma report Jul 2025, all 18 models): more retrieved memory → *lower* accuracy; coherent context can hurt more than shuffled. Strict top-k budgets, multi-signal retrieval, pruning.
|
||||
- **Maintenance graveyard** (ESTABLISHED): 82% abandon second brains within 6 months; ">30 min/week upkeep predicts failure." Retrieval-first, not archive-first. FOR US: the evolution loop must be automated and zero-curation-by-default, curation available but never required.
|
||||
|
||||
## PART B — ARCHITECTURE RECOMMENDATION FOR LINKEDIN STUDIO
|
||||
|
||||
### B.0 What we already have (the silos)
|
||||
Five per-user stores under `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/`, all surviving reinstall, all routed through one tested seam (`hooks/scripts/data-root.mjs` + TS twin `storage.ts:getDataRoot`):
|
||||
1. **voice-samples/** — authentic samples + chronicle-voice-drift-log (style layer)
|
||||
2. **trends/trends.json** — deterministic, provenance-bearing, dedup-by-title+URL
|
||||
3. **specifics-bank/specifics-bank.json** — typed lived-specifics (`number|named-case|what-broke|contrarian|mind-change`), verification-tracked; already self-described as "the first concrete tributary of the broader second-brain architecture"
|
||||
4. **analytics/** — exports, per-post, weekly/monthly reports, content-history
|
||||
5. **drafts/, plans/, profile/user-profile.md, examples/, templates/, audience-insights/**
|
||||
|
||||
The seam is the most valuable thing we own — the SOTA "route all memory I/O through one resolver" pattern is **already built and tested**. We unify + add an evolution loop, not start from zero.
|
||||
|
||||
### B.1 The shape: a thin Markdown hub over typed tributaries (NOT a vector/graph DB)
|
||||
**Recommendation: plain-Markdown+JSON two-layer "second brain," not Pinecone/Neo4j.** The portable, auditable, drift-resistant, zero-new-infra SOTA is plain-text + git (Manus, AGENTS.md, "markdown files are all you need"), augmented with the retrieval-triad + temporal-validity ideas implemented as deterministic JSON/MD. A heavyweight store would be wrong here: it adds infra, breaks portability, and **Claude IS the semantic-retrieval engine** (it reads files in-context). Keep silos as files; add a hub that points at them + a loop that maintains them.
|
||||
|
||||
Two layers, physically separated (episodic/semantic split):
|
||||
```
|
||||
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/
|
||||
brain/
|
||||
profile.md # SEMANTIC, two-layer: §static + §dynamic; each fact carries
|
||||
# value · first_seen · last_seen · evidence_count ·
|
||||
# provenance(human|published|ai-draft) · status(active|superseded)
|
||||
index.md # the HUB / Map-of-Content: one-screen pointer to every tributary
|
||||
# + freshness flags. The "memory AND ops center" front page.
|
||||
operations.md # PLANS/IDEAS: goals, bets, parked ideas, the "who I am now"
|
||||
# restatement anchor (frozen-past-self guard lives here)
|
||||
journal/ # EPISODIC, append-only: YYYY-MM-session.md — raw, never edited,
|
||||
# the source the loop consolidates FROM
|
||||
voice-samples/ # tributary (style) — stays its own store
|
||||
specifics-bank/ # tributary (raw material) — stays its own store
|
||||
trends/ # tributary (external signal) — stays its own store
|
||||
analytics/ # tributary (performance) — stays its own store
|
||||
ingest/ # NEW: raw real LinkedIn output lands here
|
||||
inbox/ # drop-zone: manual import OR connector output
|
||||
published/ # processed, provenance=published — the GOLD signal
|
||||
```
|
||||
|
||||
**Tributaries vs hub — explicit call:**
|
||||
- **Stay separate (tributaries), feed the hub via PAG summaries:** voice-samples, specifics-bank, trends, analytics, ingest. Each has a *different dedupe key + lifecycle* (trends: title+URL, fast decay; specifics-bank: human-only, near-permanent; analytics: numeric, immutable). Forcing one schema would destroy those distinctions. The hub holds a distilled pointer-summary of each.
|
||||
- **Fold INTO the hub:** the flat `profile/user-profile.md` → two-layer `brain/profile.md`. Scattered `audience-insights/`, `examples/` → dynamic-layer sections or tributary-summaries referenced from `index.md`. The **operations/plans center** is genuinely new.
|
||||
|
||||
### B.2 The evolution loop (the compounding mechanism)
|
||||
Run on a cadence (reuse the existing session-start staleness-nudge surface) as a **sleep-time consolidation pass**, not inline cost:
|
||||
1. **Capture (episodic, auto):** real-signal sessions append to `brain/journal/`. The **ingest seam** (`ingest/inbox/`) takes the user's actual published posts — *manual import first* (LinkedIn has no clean self-serve content API; analytics is already manual CSV per CLAUDE.md S16 — so manual is the honest default; any connector is additive). Published output tagged `provenance=published`.
|
||||
2. **Consolidate (reflection):** an Opus agent reads new journal + published + tributary deltas, proposes profile updates as a **diff**, never a silent overwrite. Each candidate carries `evidence_count`, `provenance`, timestamps.
|
||||
3. **Promote with thresholds (anti-overfit):** a candidate enters the **static** layer only on explicit confirmation OR ≥N independent observations; one-offs stay in the **dynamic** layer (which decays). Single weird sessions never reshape identity.
|
||||
4. **Reconcile contradictions:** classify (temporal → supersede + keep old `status: superseded` w/ date; contradictory → both timestamped; condition-dependent → scope; distractor → drop). Zep's validity-interval idea in plain frontmatter.
|
||||
5. **Decay + freshness:** every fact has `last_seen`; retrieval weights recent over old; session-start nudge flags facts not refreshed in T days (extend the existing trend-staleness nudge to the whole brain).
|
||||
6. **Curate (optional, never required):** diff is presentable; user can confirm/edit/delete/roll-back (git = free rollback). Default path needs zero curation — graveyard guard.
|
||||
|
||||
### B.3 Keeping it user-aligned + drift-resistant (the hard part, concrete)
|
||||
- **Provenance-weighted voice learning (THE critical guard):** profile/voice **learn from `provenance=published` only, never `ai-draft`.** Model-collapse guard for our exact risk. Generalizes `voice-scrubber`'s "gold standard = approved editions" to a system-wide invariant: AI-generated content is *never* a training signal for who the user is.
|
||||
- **Anti-sycophancy as built-in default, not toggle:** consolidation + content agents carry an explicit instruction to treat the profile as evidence to *test*, not flatter ("you mentioned X — still true?"), and surface profile-grounded claims as checkable, not asserted. Users prefer sycophancy when given the choice → don't offer it.
|
||||
- **Frozen-past-self guard:** `operations.md` holds a periodic user-authored "where I'm headed now" anchor that deprecates older inferences; time-weighted retrieval; the dynamic layer decays so the present dominates.
|
||||
- **Confidence before commitment:** episodic ≠ semantic; promotion needs evidence density. Highest-leverage anti-overfit mechanism, and cheap (a counter + a threshold).
|
||||
|
||||
### B.4 What's genuinely hard (flagged honestly)
|
||||
1. **Voice fidelity to a private individual is limited even at SOTA** (arXiv:2509.14543) — the real win is grounded *content* (specifics-bank) over mimicked *style*. Manage expectations.
|
||||
2. **Detecting that a high-relevance fact went stale is UNSOLVED field-wide** (STALE: everyone fails). We can timestamp + nudge; we cannot reliably auto-detect "user changed jobs." Keep humans in that loop.
|
||||
3. **Contradiction classification** (real change vs context-scoped vs noise) is EMERGING + error-prone; bias toward "keep both, timestamped, surface the conflict" over silent overwrite.
|
||||
4. **The connector** — no clean LinkedIn self-serve content/analytics API. Build the **manual ingest seam as the contract**; any future connector is a tributary into the same inbox, never a dependency.
|
||||
5. **Sycophancy is structural** — app-level mitigations reduce, not eliminate; personalization context increases it, so counter-pressure it every time the profile is used.
|
||||
6. **Avoiding the graveyard is a PRODUCT problem** — the loop must compound value with ~zero required curation or it dies in 6 months like 82% of second brains.
|
||||
|
||||
### B.5 Why this fits us
|
||||
- Reuses the one tested seam (`getDataRoot`) — SOTA single-resolver is done.
|
||||
- Plain-text + git — portable, auditable, survives reinstall, no new infra; Claude is the semantic-retrieval engine reading files in-context (no vector DB).
|
||||
- specifics-bank + trends already prove the tributary pattern (deterministic, typed, provenance-bearing, dedup-disciplined) — extend, don't reinvent.
|
||||
- The four SOTA ideas that port cleanly to files: episodic/semantic split, provenance-weighting, evidence-threshold promotion, temporal-validity frontmatter. The vector/graph machinery does NOT port and we don't need it.
|
||||
|
||||
**Bottom line (one line):** Build a thin two-layer Markdown hub (`brain/`: semantic `profile.md` + episodic `journal/` + ops `operations.md` + `index.md` MOC) over the existing typed tributaries, fed by a provenance-tagged ingest seam, maintained by a sleep-time consolidation loop with evidence-threshold promotion, temporal-validity reconciliation, and a built-in anti-sycophancy/anti-collapse stance — every silo stays a tributary except the flat `user-profile.md`, which folds in.
|
||||
|
||||
## KEY SOURCES (anchors)
|
||||
- CoALA arXiv:2309.02427 · Generative Agents arXiv:2304.03442 · MemGPT/Letta arXiv:2310.08560 · Letta sleep-time arXiv:2504.13171
|
||||
- Mem0 arXiv:2504.19413 · Zep/Graphiti arXiv:2501.13956 · A-MEM (NeurIPS 2025) arXiv:2502.12110 · Cognee arXiv:2505.24478
|
||||
- PersonaMem (COLM 2025) arXiv:2504.14225 · PURE (SIGIR 2025) · style-imitation arXiv:2509.14543 · MemConflict arXiv:2605.20926
|
||||
- interaction-context-sycophancy arXiv:2509.12517 · OpenAI Dreaming (openai.com/index/chatgpt-memory-dreaming) · Anthropic sycophancy research
|
||||
- Shumailov model collapse (Nature 2024, s41586-024-07566-y) · STALE benchmark arXiv:2605.06527 · MINJA (NeurIPS 2025) arXiv:2503.03704 · OWASP ASI06 · Chroma Context Rot (trychroma.com/research/context-rot) · Lost-in-the-Middle (Liu et al. 2023)
|
||||
- PKM: buildingasecondbrain.com · zettelkasten.de · notes.andymatuschak.org · Mem/Pinecone engineering · Tana · Obsidian Smart Connections
|
||||
- Portability: AGENTS.md (Linux Foundation) · "markdown files are all you need" (voxos.ai, dev.to)
|
||||
|
||||
> NOTE on sources: a few arXiv ids above carry 2025–2026 future-dated identifiers as returned by the research thread. Treat the *patterns* as the durable takeaway; re-verify any specific citation before quoting it externally (verification duty).
|
||||
110
docs/second-brain/research/silo-inventory.md
Normal file
110
docs/second-brain/research/silo-inventory.md
Normal file
|
|
@ -0,0 +1,110 @@
|
|||
# Research — Per-User Data Silo Inventory (LinkedIn Studio, 2026-06-23)
|
||||
|
||||
> Research thread `silo-inventory`, 2026-06-23. Every silo read from the actual files (TS types/store code, .mjs writers, templates, command/agent prose), not inferred. Research INPUT to `../architecture.md`. Preserved so nothing is lost. (file:line refs were accurate at capture time — re-confirm before editing those lines.)
|
||||
|
||||
The unifying seam is the **data-path convention** (`hooks/scripts/data-root.mjs:24` `getDataRoot()` / `scripts/analytics/src/utils/storage.ts getDataRoot`), default `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/`. Twin implementations (Node `.mjs` for hooks, TS for analytics/trends/specifics CLIs) because runtimes differ — `data-root.mjs:1-8` documents the must-stay-in-sync contract. The **M0 migration manifest** (`hooks/scripts/migrate-data.mjs:30-54`) is the single source of truth for what moves out of the plugin tree on session-start.
|
||||
|
||||
## The silos
|
||||
|
||||
### A. State file (`## Recent Posts`, streaks, milestones, first-hour, outreach)
|
||||
- **Location:** `~/.claude/linkedin-studio.local.md` — **NOT** under the data-root seam. Own resolver (`state-updater.mjs:11-12`, `STATE_FILE` override; mirrored in `data-root.mjs:30 getStateFile()`). The data root "mirrors" this file (`data-path-convention.md:15`) but they are two separate locations.
|
||||
- **Format:** YAML frontmatter + markdown sections.
|
||||
- **Schema** (`config/state-file.template.md`): frontmatter — `last_post_date`, `first_post_date`, `last_post_topic`, `posts_this_week`, `weekly_goal`, `current_streak`, `longest_streak`, `current_week`, `last_import_date/week`, `follower_count/target`, `target_date`, `monthly_growth[]`, `projected_10k_date`, `growth_rate_needed`, `next_planned_topic`, `pending_5x5x5`, `content_series_active`, `last_firsthour_date`, `firsthour_active`, `last_outreach_date`, `outreach_active`, `expertise_areas[5]`. Sections — `## Recent Posts` (`- [YYYY-MM-DD] "hook…" (chars) - topic`), `## Session Notes`, `## Planned Content`, `## Milestone Log` (`- [YYYY-MM] count (+delta)`), `## First-Hour Plans`, `## Outreach Pipeline`.
|
||||
- **Writers:** `state-updater.mjs` — `updatePostTracking` (:55), `updateFollowerCount` (:181), `recordFirstHourPlan` (:235), `recordOutreachContact` (:294), `pruneContentHistory` (:138). From Stop hook + `/firsthour`, `/outreach`, `/calendar` publish. **Readers:** SessionStart hook, `/analyze`, `/report`, content-planner.
|
||||
- **Lifecycle:** `## Recent Posts` auto-pruned at 90d on session-start (`state-updater.mjs:138`). Streak resets if post gap >2d (:94). Week rollover resets `posts_this_week` via `week-rollover.mjs`.
|
||||
- **Survives reinstall:** YES (in `~/.claude/`, not plugin tree).
|
||||
|
||||
### B. Trends store
|
||||
- **Location:** `${data}/trends/trends.json` — through seam (`scripts/trends/src/store.ts:180 defaultStorePath()`).
|
||||
- **Format:** JSON (TS-managed), `{ schemaVersion:1, trends:TrendRecord[] }`.
|
||||
- **Schema** (`trends/src/types.ts:26`): `TrendRecord` = `id` (sha256[:12] of normalized title+url = dedupe key), `title`, `url`, `source`, `capturedAt` (ISO), `topics[]`, `summary?`.
|
||||
- **Writers:** CLI `addTrend` (`store.ts:111`), populated by trend-spotter. **Readers:** `queryByTopic` (:137), `history` (:156), `newestCaptureDate` (:171).
|
||||
- **Lifecycle:** Dedup on title+url; re-capture **unions topics**, keeps first-sight provenance. `newestCaptureDate` drives the SessionStart trend-freshness nudge (B-S3). No prune.
|
||||
- **Survives reinstall:** YES.
|
||||
|
||||
### C. Lived-specifics bank
|
||||
- **Location:** `${data}/specifics-bank/specifics-bank.json` — through seam (`scripts/specifics-bank/src/bank.ts:150 defaultBankPath()`).
|
||||
- **Format:** JSON (TS-managed), `{ schemaVersion:1, specifics:Specific[] }`.
|
||||
- **Schema** (`specifics-bank/src/types.ts:46`): `Specific` = `id` (sha256[:12] of normalized content = dedupe key), `type` (`number`|`named-case`|`what-broke`|`contrarian`|`mind-change`|`other`), `content` (operator's own words, never AI-generated — invariant `types.ts:13`), `topicTags[]`, `provenance{capturedAt,source}` (source = edition id/command/"manual"), `verification` (`verified`|`unverified`|`n/a`), `status` (`active`|`archived`).
|
||||
- **Writers:** CLI `addSpecific` (`bank.ts:102`), from `/newsletter` Step 1.5 elicitation. **Readers:** `queryByTopic` (:128) — excludes archived. Companions `binding.ts` (Step 2.5 gate), `kilder.ts`.
|
||||
- **Lifecycle:** Dedup on content; re-capture unions tags. Archive = soft-hide. No time-prune. `verification` gates numbers.
|
||||
- **Survives reinstall:** YES.
|
||||
|
||||
### D. Analytics store
|
||||
- **Location root:** `${data}/analytics/` via `getAnalyticsRoot()` (`scripts/analytics/src/utils/storage.ts:67`) — through seam; legacy `ANALYTICS_ROOT` honored (deprecated). Subdirs: `exports/`, `posts/`, `weekly-reports/`, `monthly-reports/`, `ab-tests/`.
|
||||
- **Format:** JSON (TS-managed) posts/reports; CSV input; markdown ab-tests.
|
||||
- **Schema** (`analytics/src/models/types.ts`): `PostAnalytics` = `id`, `title`, `publishedDate`, `metrics`, `importedAt`, `exportSource` — file `posts/YYYY-MM-DD-<8char>.json` inside `AnalyticsBatch` (`batchId`, `importedAt`, `exportFilename`, `dateRange{from,to}`, `postCount`, `posts[]`). `PostMetrics` = `impressions`, `reactions`, `comments`, `shares`, `clicks`, `engagementRate`, `saves?` — **no `dwell`** (intentional). `WeeklyReport`(`weekly-reports/YYYY-WXX.json`)/`MonthlyReport`(`monthly-reports/YYYY-MM.json`) = `summary{totals incl. totalSaves?, avgEngagementRate, avgImpressionsPerPost}`, `topPerformers[]`, `underperformers[]`/`byWeek[]`, `trends{…percentChange}`, `alerts[]`.
|
||||
- **`saves`** (`parseOptionalCount`, `csv-parser.ts:71`): blank/non-numeric/negative→`undefined` (unknown, never 0); genuine `0` kept; **not folded into `engagementRate`**; surfaced only when ≥1 post carries it.
|
||||
- **Writers:** analytics CLI (`cli.ts`) — `import`→`posts/`, `report`→weekly/monthly; ab-tests written by `/ab-test` prose. **Readers:** CLI report/trends/heatmap subcommands; analytics-interpreter agent (no agent re-reads posts JSON directly — report path is CLI-internal).
|
||||
- **Lifecycle:** Posts deduped by post id on import (latest wins). No time-prune. Reports overwrite by key.
|
||||
- **Survives reinstall:** YES.
|
||||
|
||||
### E. Drafts queue
|
||||
- **Location:** `${data}/drafts/queue.json` — through seam (`queue-manager.mjs:10` `join(getDataRoot('drafts'),'queue.json')`). Migrated `migrate-data.mjs:32`.
|
||||
- **Format:** JSON, `{ version:1, queue:[] }`.
|
||||
- **Schema** (`queue-manager.mjs:63-79`, the actual writer): `id`, `draft_path`, `scheduled_date`, `scheduled_time`, `pillar`, `format`, `hook_preview`, `character_count`, `status` (`'scheduled'` on add; →`posted`/`archived`/`skipped`/`draft` via `queueUpdateStatus`), `created_at`.
|
||||
- **Writers:** `queueAdd`/`queueUpdateStatus` (:63,:82) — `/batch`, `/calendar`. **Readers:** `queueRead`/`Today`/`Upcoming`/`Overdue` (:41-99), SessionStart, `/calendar`, `/pipeline`, ical-generator.
|
||||
- **Lifecycle:** No prune. Dedup-by-id on add (:64). No week reset.
|
||||
- **Survives reinstall:** YES.
|
||||
|
||||
### F. Draft files on disk (week / carousel / multiplatform / repurposed)
|
||||
- **Location (all through seam, confirmed in prose):** `${data}/drafts/week-*/…md` (migrated `migrate-data.mjs:46,129`), `${data}/drafts/carousel-YYYYMMDD-SLUG/slide-N.png` (`commands/carousel.md:133,147`), `${data}/drafts/multiplatform/[platform]-[slug].md` (`commands/multiplatform.md:118`), `${data}/drafts/repurposed/` (`agents/content-repurposer.md:597`). Only `week-*` is in the migration manifest; carousel/multiplatform/repurposed are born external (command/agent prose writes straight to the data root).
|
||||
- **Format:** Markdown (+ YAML frontmatter) / PNG.
|
||||
- **Writers:** creation commands + content-repurposer. **Readers:** `/calendar`, `/pipeline`, repurpose/carousel/video.
|
||||
- **Lifecycle:** No automatic prune — files accumulate; manual cleanup. Dedup by filename/dir slug only.
|
||||
- **Survives reinstall:** YES.
|
||||
|
||||
### G. Content plans
|
||||
- **Location:** `${data}/plans/<weekly|monthly>-plan-*.md` — through seam (`agents/content-planner.md:7,40,464`). Born external (not in manifest).
|
||||
- **Format:** Markdown (+ frontmatter: `plan_date`, `generated`, `status`, `posts_planned`, pillars, `content_mix`).
|
||||
- **Writers:** content-planner agent (:464, can also create Linear issues). **Readers:** content-planner (scans prior plans to avoid repetition :40), SessionStart, `/calendar`.
|
||||
- **Lifecycle:** No prune; planner does content-level dedup vs prior plans. **Survives reinstall:** YES.
|
||||
|
||||
### H. Voice profile + voice-drift log
|
||||
- **Location:** `${data}/voice-samples/authentic-voice-samples.md` (external canonical; in-plugin **placeholder seed** at `assets/voice-samples/authentic-voice-samples.md` with `<!-- VOICE_PLACEHOLDER -->` sentinel; migrated MOVE dropping `.local`, `migrate-data.mjs:31`). Plus `${data}/voice-samples/chronicle-voice-drift-log.md` (external-only, no seed; written/read by `voice-scrubber`).
|
||||
- **Resolution:** `personalization-score.mjs` `pick(dataRel,pluginRel)` (:30-32) prefers external, falls back to seed. Voice scores 25pts iff line-count>50 AND no sentinel.
|
||||
- **Format:** Markdown sections (Core Voice Characteristics, Do's/Don'ts, Signature Phrases, Vocabulary, Language Guidelines, Update Log, auto-appended "Collected Post Samples").
|
||||
- **Writers:** `/setup`/`/onboarding` (overwrite placeholder), voice-trainer (appends samples), voice-scrubber (drift log). **Readers:** `personalization-score.mjs`, `user-prompt-context.mjs`, content commands.
|
||||
- **Survives reinstall:** external instance YES; seed NO (it's the seed).
|
||||
|
||||
### I. User profile
|
||||
- **Location:** `${data}/profile/user-profile.md` (migrated MOVE from `config/user-profile.local.md`, `migrate-data.mjs:34`; template seed `config/user-profile.template.md`). Scored 20pts when <3 `[Your ` placeholders remain (`personalization-score.mjs:54`).
|
||||
- **Format/schema:** Markdown — Personalization Settings (identity, expertise areas, audience, goals), Voice & Style, Strategic Context, **Research Tooling** (which MCPs available), Asset Utilization Preferences.
|
||||
- **Writers:** `/setup`/`/onboarding` + manual edit. **Readers:** `personalization-score.mjs`, `user-prompt-context.mjs`, commands route MCPs from it. **Survives reinstall:** external YES.
|
||||
|
||||
### J. COPY-class scaffolds (examples / audience-insights / templates)
|
||||
Four files ship a tracked seed + canonical external instance (`migrate-data.mjs:49-54 COPY_FILES`; external never clobbered): `${data}/examples/high-engagement-posts.md`, `${data}/audience-insights/{demographics,engagement-patterns}.md`, `${data}/templates/my-post-templates.md`. Markdown. Written by user/voice-trainer; read by `personalization-score.mjs` (pick-resolved) + content commands. **Survives reinstall:** external YES, seed NO.
|
||||
|
||||
### K. content-history.md — vestigial second surface
|
||||
- **Location:** `${data}/analytics/content-history.md` (migrated MOVE, `migrate-data.mjs:33`; template `config/content-history.template.md`). Markdown table (`Date|Hook|Topic|Format|Words|Chars|Source`).
|
||||
- **Status:** No hook/agent code actively writes it — the live content-history surface is the **state file's `## Recent Posts`** (silo A). Migrated-but-superseded scaffold.
|
||||
- **Survives reinstall:** YES (but inactive).
|
||||
|
||||
### L. iCal export
|
||||
`${data}/drafts/week-WXX/schedule.ics`, generated from queue by `ical-generator.mjs` (`/batch`). RFC-5545. Transient/regenerated, not a knowledge store. Survives reinstall: YES.
|
||||
|
||||
## Synthesis
|
||||
|
||||
### Already per-user + reinstall-surviving (the "done" set)
|
||||
A–L all route through the seam or `~/.claude/`. **There is essentially no in-plugin/tmp data left to migrate** — M0 already did it. Only `/tmp/linkedin-hooks/session-active` (a session marker, not data) and the in-plugin **seeds** stay in-tree by design.
|
||||
|
||||
### What is NOT yet unified (the real gap)
|
||||
Per-user but **siloed and heterogeneous** — three storage idioms, two roots, no cross-references:
|
||||
1. **Two roots, not one.** State file at `~/.claude/linkedin-studio.local.md` (own resolver, `state-updater.mjs:12`); everything else under `${data}/`. Unification must decide whether the state file folds in or stays a sibling.
|
||||
2. **Two writer disciplines.** TS-managed JSON (trends, specifics, analytics) with `schemaVersion` + sha256-id dedup + tested pure functions — **vs.** regex markdown-section mutation (state file, `state-updater.mjs`) — **vs.** schema-less free-form markdown written by command/agent prose (drafts, plans, voice, scaffolds).
|
||||
|
||||
### Natural overlaps / relationships
|
||||
- **Posts triple-counted.** A published post lands in three non-referencing places: state `## Recent Posts` (A, live), `analytics/content-history.md` (K, dead), `analytics/posts/*.json` (D, from CSV). Same event, three schemas, no shared id.
|
||||
- **specifics ↔ posts ↔ analytics.** A `Specific` (number/named-case) → a post (F) → a `PostAnalytics` row (D). Nothing links the raw material to the post it seeded or to that post's measured performance — "which raw material performs" is unanswerable today.
|
||||
- **trends ↔ plans ↔ drafts.** trend-spotter captures `TrendRecord` (B); content-planner plan (G) should cite it; draft (F) executes it — all free-text, no id linkage.
|
||||
- **voice ↔ content-history ↔ analytics.** voice-trainer learns from "Collected Post Samples"; the high performers that should train it live in analytics (D) + examples (J) — three disconnected views of "your good posts."
|
||||
- **provenance is the common latent field.** trends (`source`+`capturedAt`), specifics (`provenance{source,capturedAt}`), analytics (`importedAt`,`exportSource`), queue (`created_at`) each independently reinvented capture-provenance under different names.
|
||||
|
||||
### The 5 hardest unification problems
|
||||
1. **Schema-idiom mismatch.** Folding regex-mutated markdown (state file) and prose-written free-form markdown (drafts/plans/voice) into the typed `schemaVersion`+dedup discipline of the JSON stores is the central rewrite. JSON stores (B/C/D) are already second-brain-shaped; the markdown silos are not.
|
||||
2. **Identity / dedup across silos.** Each store has its own id — sha256(title+url) trends, sha256(content) specifics, hash(title+date) analytics, slug ids queue. **No stable cross-silo entity id**, so the post→specific→trend→analytics graph can't be assembled. Needs one canonical id minted at creation and threaded through.
|
||||
3. **Two roots + legacy-alias window.** Reconcile `getStateFile()` with `getDataRoot()` while three deprecated aliases (`ANALYTICS_ROOT`/`STATE_FILE`/`PLUGIN_ROOT`) are honored "for one minor version" (`data-path-convention.md:17`) and **two twin implementations** (`data-root.mjs` ⇄ `storage.ts`) must stay byte-compatible.
|
||||
4. **Triple-recorded post + dead content-history.** K is a migrated-but-unwritten duplicate of A, and neither links to D. Unification must pick one canonical post record and retire/back-fill the other two — a data-reconciliation problem, not just schema.
|
||||
5. **Provenance normalization + lifecycle divergence.** Each silo reinvented provenance under different field names; retention is inconsistent — 90-day prune (state Recent Posts only), soft-archive (specifics), topic-union-merge (trends/specifics), overwrite-by-key (reports), **no prune at all** for drafts/plans/queue/voice/trends. Needs one provenance shape and one coherent retention policy.
|
||||
|
||||
Key files: `hooks/scripts/data-root.mjs`, `hooks/scripts/migrate-data.mjs:30-54`, `hooks/scripts/state-updater.mjs`, `hooks/scripts/queue-manager.mjs:10-91`, `scripts/trends/src/{types,store}.ts`, `scripts/specifics-bank/src/{types,bank}.ts`, `scripts/analytics/src/{utils/storage,models/types,parsers/csv-parser}.ts`, `hooks/scripts/personalization-score.mjs:30-152`, `config/state-file.template.md`, `references/data-path-convention.md`.
|
||||
|
|
@ -63,23 +63,3 @@ If a LinkedIn post was created or finalized in this session, save the full post
|
|||
- **Ask the user for confirmation** before writing: "I'll save this post as a voice sample for drift detection. OK?"
|
||||
- This builds the voice sample library that enables automatic drift scoring (needs 5+ samples for reliable scoring)
|
||||
- The more samples collected, the more accurate the voice-trainer's drift detection becomes
|
||||
|
||||
**5. Content History Log** (if a post was created)
|
||||
|
||||
If a LinkedIn post was created or finalized, append an entry to the content history log:
|
||||
|
||||
- If `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/analytics/content-history.md` does not exist, initialize it from `config/content-history.template.md`
|
||||
- Append a new row to the "## Content Log" table:
|
||||
```
|
||||
| YYYY-MM-DD | "Hook text..." | topic_area | format | word_count | char_count | source |
|
||||
```
|
||||
Where:
|
||||
- `date`: Today's date
|
||||
- `hook`: First 60 characters of the hook line
|
||||
- `topic`: Matching expertise_area value (for pillar tracking)
|
||||
- `format`: post/quick/react/video/pipeline
|
||||
- `word_count`: Word count of the full post
|
||||
- `char_count`: Character count of the full post
|
||||
- `source`: original/url/curated (where the idea came from)
|
||||
- This is append-only — never edit or delete existing entries
|
||||
- This log enables `/linkedin:report` and `analytics-interpreter` to track content production over time without requiring LinkedIn CSV imports
|
||||
|
|
|
|||
|
|
@ -34,7 +34,6 @@ describe('migrateData', () => {
|
|||
if (withRuntime) {
|
||||
writeDeep(join(pluginRoot, 'assets/voice-samples/authentic-voice-samples.local.md'), 'REAL VOICE 227 lines');
|
||||
writeDeep(join(pluginRoot, 'assets/drafts/queue.json'), '{"version":1,"queue":[]}');
|
||||
writeDeep(join(pluginRoot, 'assets/analytics/content-history.md'), '# history');
|
||||
writeDeep(join(pluginRoot, 'assets/analytics/exports/content-2026-W22.csv'), 'a,b\n1,2\n');
|
||||
writeDeep(join(pluginRoot, 'assets/analytics/posts/2026-05-26.json'), '{"x":1}');
|
||||
writeDeep(join(pluginRoot, 'assets/analytics/weekly-reports/2026-W22.json'), '{"w":22}');
|
||||
|
|
@ -47,18 +46,16 @@ describe('migrateData', () => {
|
|||
}
|
||||
}
|
||||
|
||||
test('(a) gitignored runtime files MOVED: source gone, dest byte-equal, voice drops .local, content-history moved', () => {
|
||||
test('(a) gitignored runtime files MOVED: source gone, dest byte-equal, voice drops .local', () => {
|
||||
setup({ withRuntime: true });
|
||||
migrateData({ pluginRoot });
|
||||
|
||||
assert.ok(!existsSync(join(pluginRoot, 'assets/voice-samples/authentic-voice-samples.local.md')), 'voice source gone');
|
||||
assert.ok(!existsSync(join(pluginRoot, 'assets/drafts/queue.json')), 'queue source gone');
|
||||
assert.ok(!existsSync(join(pluginRoot, 'assets/analytics/content-history.md')), 'content-history source gone');
|
||||
assert.ok(!existsSync(join(pluginRoot, 'assets/analytics/exports/content-2026-W22.csv')), 'export source gone');
|
||||
|
||||
assert.equal(readFileSync(join(dataRoot, 'voice-samples/authentic-voice-samples.md'), 'utf-8'), 'REAL VOICE 227 lines');
|
||||
assert.equal(readFileSync(join(dataRoot, 'drafts/queue.json'), 'utf-8'), '{"version":1,"queue":[]}');
|
||||
assert.ok(existsSync(join(dataRoot, 'analytics/content-history.md')));
|
||||
assert.ok(existsSync(join(dataRoot, 'analytics/exports/content-2026-W22.csv')));
|
||||
assert.ok(existsSync(join(dataRoot, 'analytics/posts/2026-05-26.json')));
|
||||
assert.ok(existsSync(join(dataRoot, 'analytics/weekly-reports/2026-W22.json')));
|
||||
|
|
|
|||
|
|
@ -0,0 +1,80 @@
|
|||
import { describe, test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdtempSync, rmSync, mkdirSync, writeFileSync, copyFileSync } from 'node:fs';
|
||||
import { join, dirname } from 'node:path';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
// SB-S2: SessionStart emits a "run brain consolidate" nudge driven by the count of
|
||||
// published gold records + the consolidation-state.json last_run sidecar (read via
|
||||
// getDataRoot — the SAME root the brain CLI --apply writes to). Zero-dep, no
|
||||
// profile.md parse. Subprocess harness with isolated HOME + LINKEDIN_STUDIO_DATA,
|
||||
// mirroring session-start-trends-staleness.test.mjs.
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const hookPath = join(here, '..', 'session-start.mjs');
|
||||
const PLUGIN_ROOT = join(here, '..', '..', '..');
|
||||
const STATE_TEMPLATE = join(PLUGIN_ROOT, 'config', 'state-file.template.md');
|
||||
|
||||
const NUDGE = 'brain consolidate';
|
||||
const isoDaysAgo = (n) => new Date(Date.now() - n * 86400000).toISOString().slice(0, 10);
|
||||
|
||||
const record = (id) =>
|
||||
`id: ${id}\nprovenance: published\npublished_date: 2026-01-01\ncaptured_at: 2026-01-01\nsource: manual\n---\nbody`;
|
||||
|
||||
function runHook({ withState = true, published = [], lastRun = undefined } = {}) {
|
||||
const home = mkdtempSync(join(tmpdir(), 'lis-bc-home-'));
|
||||
const data = mkdtempSync(join(tmpdir(), 'lis-bc-data-'));
|
||||
try {
|
||||
if (withState) {
|
||||
mkdirSync(join(home, '.claude'), { recursive: true });
|
||||
copyFileSync(STATE_TEMPLATE, join(home, '.claude', 'linkedin-studio.local.md'));
|
||||
}
|
||||
if (published.length) {
|
||||
mkdirSync(join(data, 'ingest', 'published'), { recursive: true });
|
||||
for (const id of published) writeFileSync(join(data, 'ingest', 'published', id + '.md'), record(id));
|
||||
}
|
||||
if (lastRun !== undefined) {
|
||||
mkdirSync(join(data, 'brain'), { recursive: true });
|
||||
writeFileSync(join(data, 'brain', 'consolidation-state.json'), JSON.stringify({ last_run: lastRun }));
|
||||
}
|
||||
const stdout = execFileSync('node', [hookPath], {
|
||||
input: '',
|
||||
env: { ...process.env, HOME: home, USERPROFILE: home, LINKEDIN_STUDIO_DATA: data },
|
||||
encoding: 'utf-8',
|
||||
});
|
||||
return JSON.parse(stdout).hookSpecificOutput.additionalContext;
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
rmSync(data, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
describe('session-start — brain consolidation-due nudge (SB-S2)', () => {
|
||||
test('fires when published records exist and consolidation never ran', () => {
|
||||
const ctx = runHook({ published: ['aaaaaaaaaaaa', 'bbbbbbbbbbbb'] });
|
||||
assert.ok(ctx.includes(NUDGE), 'nudge expected when published exist + never consolidated');
|
||||
assert.ok(ctx.includes('2 published'), 'reports the published count');
|
||||
});
|
||||
|
||||
test('fires when last_run is stale (>=7d)', () => {
|
||||
const ctx = runHook({ published: ['aaaaaaaaaaaa'], lastRun: isoDaysAgo(10) });
|
||||
assert.ok(ctx.includes(NUDGE), 'nudge expected for a 10-day-old last_run');
|
||||
});
|
||||
|
||||
test('silent when last_run is fresh (<7d)', () => {
|
||||
const ctx = runHook({ published: ['aaaaaaaaaaaa'], lastRun: isoDaysAgo(2) });
|
||||
assert.ok(!ctx.includes(NUDGE), 'fresh consolidation => no nudge');
|
||||
});
|
||||
|
||||
test('silent when there are no published records (never nags)', () => {
|
||||
const ctx = runHook({ published: [] });
|
||||
assert.ok(!ctx.includes(NUDGE), 'no published => no nudge');
|
||||
});
|
||||
|
||||
test('fresh install (no state file) emits valid JSON and nudges brain init', () => {
|
||||
const ctx = runHook({ withState: false });
|
||||
assert.equal(typeof ctx, 'string', 'valid JSON additionalContext on fresh install');
|
||||
assert.ok(ctx.includes('Brain not initialised'), 'brain-init nudge survives the fresh-install branch');
|
||||
});
|
||||
});
|
||||
77
hooks/scripts/__tests__/session-start-morning-brief.test.mjs
Normal file
77
hooks/scripts/__tests__/session-start-morning-brief.test.mjs
Normal file
|
|
@ -0,0 +1,77 @@
|
|||
import { describe, test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdtempSync, rmSync, mkdirSync, writeFileSync, copyFileSync } from 'node:fs';
|
||||
import { join, dirname } from 'node:path';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
// RE-R2b: SessionStart surfaces the latest dated morning brief — a "## Morning
|
||||
// Brief" block built from the brief's frontmatter (date + pre-rendered summary),
|
||||
// read zero-tsx. session-start is a procedural hook with no exports, so we run it
|
||||
// as a subprocess with an isolated HOME + LINKEDIN_STUDIO_DATA and inspect the
|
||||
// additionalContext. Pattern: session-start-trends-staleness.test.mjs.
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const hookPath = join(here, '..', 'session-start.mjs');
|
||||
const PLUGIN_ROOT = join(here, '..', '..', '..');
|
||||
const STATE_TEMPLATE = join(PLUGIN_ROOT, 'config', 'state-file.template.md');
|
||||
|
||||
const HEADING = '## Morning Brief';
|
||||
// ASCII fixture summary (the hook surfaces whatever the frontmatter carries verbatim).
|
||||
const SUMMARY = '3 fresh signals match your pillars. Top: Alpha (AI, 2d).';
|
||||
|
||||
// Run the hook against an isolated HOME + data root; optionally seed a dated brief
|
||||
// at <data>/trends/morning-brief/<name> — the SAME path defaultBriefDir() resolves
|
||||
// to under this LINKEDIN_STUDIO_DATA (CLI-write path == hook-read path cross-check).
|
||||
function runHook({ briefName }) {
|
||||
const home = mkdtempSync(join(tmpdir(), 'lis-mb-home-'));
|
||||
const data = mkdtempSync(join(tmpdir(), 'lis-mb-data-'));
|
||||
try {
|
||||
mkdirSync(join(home, '.claude'), { recursive: true });
|
||||
copyFileSync(STATE_TEMPLATE, join(home, '.claude', 'linkedin-studio.local.md'));
|
||||
|
||||
if (briefName) {
|
||||
const briefDir = join(data, 'trends', 'morning-brief');
|
||||
mkdirSync(briefDir, { recursive: true });
|
||||
writeFileSync(
|
||||
join(briefDir, briefName),
|
||||
`---\ndate: 2026-06-24\nsummary: ${SUMMARY}\nstore: { trends: 10, matched: 3, fresh: 3 }\nschemaVersion: 1\n---\n\n# Morgen-brief\n\nbody\n`,
|
||||
);
|
||||
}
|
||||
|
||||
const stdout = execFileSync('node', [hookPath], {
|
||||
input: '',
|
||||
env: { ...process.env, HOME: home, USERPROFILE: home, LINKEDIN_STUDIO_DATA: data },
|
||||
encoding: 'utf-8',
|
||||
});
|
||||
return JSON.parse(stdout).hookSpecificOutput.additionalContext;
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
rmSync(data, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
describe('session-start — morning-brief surfacing (RE-R2b)', () => {
|
||||
test('surfaces the latest brief: heading + summary + file pointer', () => {
|
||||
const ctx = runHook({ briefName: '2026-06-24.md' });
|
||||
assert.ok(ctx.includes(HEADING), 'expected the Morning Brief block');
|
||||
assert.ok(ctx.includes('Morning Brief (2026-06-24)'), 'block carries the brief date');
|
||||
assert.ok(ctx.includes(SUMMARY), 'block surfaces the pre-rendered summary');
|
||||
assert.ok(ctx.includes('Full brief:'), 'block carries the full-brief pointer');
|
||||
// The summary must be its OWN line (proves it did not bleed into adjacent lines —
|
||||
// single-line summary + the \n idiom held).
|
||||
assert.ok(ctx.split('\n').includes(SUMMARY), 'summary is a standalone line in the block');
|
||||
});
|
||||
|
||||
test('newest brief wins when several dated files exist', () => {
|
||||
// (single-file harness asserts the surface; lexical name sort picks the newest —
|
||||
// covered structurally by the .md-anchored filter + .sort() in latestMorningBrief.)
|
||||
const ctx = runHook({ briefName: '2026-06-24.md' });
|
||||
assert.ok(ctx.includes('Morning Brief (2026-06-24)'));
|
||||
});
|
||||
|
||||
test('no brief dir -> no block, no crash', () => {
|
||||
const ctx = runHook({ briefName: null });
|
||||
assert.ok(!ctx.includes(HEADING), 'no brief => no Morning Brief block');
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
import { describe, test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdtempSync, rmSync, mkdirSync, writeFileSync, copyFileSync } from 'node:fs';
|
||||
import { join, dirname } from 'node:path';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
// B-S3: SessionStart emits a warn-only "trend signals are N days old" nudge,
|
||||
// driven by the trend store's newest capturedAt, and ONLY when the store already
|
||||
// holds captures. session-start is a procedural hook with no exports, so we run
|
||||
// it as a subprocess with an isolated HOME + LINKEDIN_STUDIO_DATA and inspect the
|
||||
// additionalContext. Pattern: session-start-remember.test.mjs subprocess harness.
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const hookPath = join(here, '..', 'session-start.mjs');
|
||||
const PLUGIN_ROOT = join(here, '..', '..', '..');
|
||||
const STATE_TEMPLATE = join(PLUGIN_ROOT, 'config', 'state-file.template.md');
|
||||
|
||||
const NUDGE = 'Trend signals are';
|
||||
const isoDaysAgo = (n) => new Date(Date.now() - n * 86400000).toISOString().slice(0, 10);
|
||||
|
||||
// Run the hook against an isolated HOME + data root; optionally seed a trend
|
||||
// store. Returns the additionalContext string the hook injected at session start.
|
||||
function runHook({ captures }) {
|
||||
const home = mkdtempSync(join(tmpdir(), 'lis-ts-home-'));
|
||||
const data = mkdtempSync(join(tmpdir(), 'lis-ts-data-'));
|
||||
try {
|
||||
// The reminders block only runs when the state file already exists; the
|
||||
// template is the canonical valid state, so copy it into the isolated HOME.
|
||||
mkdirSync(join(home, '.claude'), { recursive: true });
|
||||
copyFileSync(STATE_TEMPLATE, join(home, '.claude', 'linkedin-studio.local.md'));
|
||||
|
||||
if (captures) {
|
||||
mkdirSync(join(data, 'trends'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(data, 'trends', 'trends.json'),
|
||||
JSON.stringify({ schemaVersion: 1, trends: captures }),
|
||||
);
|
||||
}
|
||||
|
||||
const stdout = execFileSync('node', [hookPath], {
|
||||
input: '',
|
||||
env: { ...process.env, HOME: home, USERPROFILE: home, LINKEDIN_STUDIO_DATA: data },
|
||||
encoding: 'utf-8',
|
||||
});
|
||||
return JSON.parse(stdout).hookSpecificOutput.additionalContext;
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
rmSync(data, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
const trend = (capturedAt) => ({
|
||||
id: 't' + capturedAt,
|
||||
title: 'Some trend ' + capturedAt,
|
||||
url: 'https://example.com/' + capturedAt,
|
||||
source: 'tavily',
|
||||
capturedAt,
|
||||
topics: ['x'],
|
||||
});
|
||||
|
||||
describe('session-start — trend-store staleness nudge (B-S3)', () => {
|
||||
test('fires when newest capture is >=7 days old', () => {
|
||||
const ctx = runHook({ captures: [trend('2026-01-01'), trend(isoDaysAgo(10))] });
|
||||
assert.ok(ctx.includes(NUDGE), 'expected the trend-staleness nudge for a 10-day-old store');
|
||||
});
|
||||
|
||||
test('does NOT fire when newest capture is fresh (<7 days)', () => {
|
||||
const ctx = runHook({ captures: [trend(isoDaysAgo(2))] });
|
||||
assert.ok(!ctx.includes(NUDGE), 'a 2-day-old store must not trigger the nudge');
|
||||
});
|
||||
|
||||
test('does NOT fire (and does not crash) when the store is absent/empty', () => {
|
||||
const absent = runHook({ captures: null });
|
||||
assert.ok(!absent.includes(NUDGE), 'no store => no nudge');
|
||||
const empty = runHook({ captures: [] });
|
||||
assert.ok(!empty.includes(NUDGE), 'empty store => no nudge (never-scanned user is never nagged)');
|
||||
});
|
||||
});
|
||||
|
|
@ -284,41 +284,73 @@ describe('updatePostTracking', () => {
|
|||
});
|
||||
});
|
||||
|
||||
// Fixed synthetic "today" for prune tests: far from every hardcoded SAMPLE_STATE
|
||||
// date, so fixture dates computed relative to it can never collide with them.
|
||||
const FIXED_TODAY = new Date('2027-01-01T00:00:00Z');
|
||||
|
||||
// Days before FIXED_TODAY as YYYY-MM-DD.
|
||||
function daysBeforeFixedToday(days) {
|
||||
const d = new Date(FIXED_TODAY);
|
||||
d.setDate(d.getDate() - days);
|
||||
return d.toISOString().slice(0, 10);
|
||||
}
|
||||
|
||||
// Extract only the Recent Posts section body so assertions cannot accidentally
|
||||
// match dates in the frontmatter or other sections.
|
||||
function recentPostsSection(content) {
|
||||
const match = content.match(/## Recent Posts\n([\s\S]*?)\n## /);
|
||||
assert.ok(match, 'Recent Posts section present');
|
||||
return match[1];
|
||||
}
|
||||
|
||||
describe('pruneContentHistory', () => {
|
||||
test('removes entries older than 90 days', () => {
|
||||
const today = new Date();
|
||||
const old = new Date(today);
|
||||
old.setDate(old.getDate() - 100);
|
||||
const oldDate = old.toISOString().slice(0, 10);
|
||||
test('prunes an old entry that sits BELOW a fresh entry (regression: /m flag truncated the capture to the first line)', () => {
|
||||
// Production prepends new entries at the top, so old entries always sit
|
||||
// BELOW fresh ones. With the /m flag, the $ alternative in the lookahead
|
||||
// matched the end of the FIRST entry line, so only that line was captured
|
||||
// and older entries below were never scanned -> never pruned.
|
||||
const oldDate = daysBeforeFixedToday(400);
|
||||
const freshDate = daysBeforeFixedToday(10);
|
||||
|
||||
const recent = new Date(today);
|
||||
recent.setDate(recent.getDate() - 10);
|
||||
const recentDate = recent.toISOString().slice(0, 10);
|
||||
|
||||
const stateWithOld = SAMPLE_STATE.replace(
|
||||
'## Recent Posts\n\n',
|
||||
`## Recent Posts\n\n- [${oldDate}] "Old post..." (1000) - old topic\n- [${recentDate}] "Recent post..." (1200) - recent topic\n`
|
||||
const state = SAMPLE_STATE.replace(
|
||||
/## Recent Posts\n\n[\s\S]*?(?=## Session Notes)/,
|
||||
`## Recent Posts\n\n- [${freshDate}] "Fresh post..." (1200) - fresh topic\n- [${oldDate}] "Old post..." (1000) - old topic\n\n`
|
||||
);
|
||||
|
||||
const result = pruneContentHistory(stateWithOld, 90);
|
||||
const result = pruneContentHistory(state, 90, FIXED_TODAY);
|
||||
assert.notEqual(result, null, 'an old entry below a fresh one must be pruned');
|
||||
assert.equal(result.pruned, 1);
|
||||
const section = recentPostsSection(result.content);
|
||||
assert.ok(!section.includes(oldDate), 'old entry pruned');
|
||||
assert.ok(section.includes(freshDate), 'fresh entry kept');
|
||||
});
|
||||
|
||||
test('removes entries older than 90 days', () => {
|
||||
const oldDate = daysBeforeFixedToday(100);
|
||||
const recentDate = daysBeforeFixedToday(10);
|
||||
|
||||
const stateWithOld = SAMPLE_STATE.replace(
|
||||
/## Recent Posts\n\n[\s\S]*?(?=## Session Notes)/,
|
||||
`## Recent Posts\n\n- [${oldDate}] "Old post..." (1000) - old topic\n- [${recentDate}] "Recent post..." (1200) - recent topic\n\n`
|
||||
);
|
||||
|
||||
const result = pruneContentHistory(stateWithOld, 90, FIXED_TODAY);
|
||||
assert.notEqual(result, null);
|
||||
assert.equal(result.pruned, 1);
|
||||
assert.ok(!result.content.includes(oldDate));
|
||||
assert.ok(result.content.includes(recentDate));
|
||||
const section = recentPostsSection(result.content);
|
||||
assert.ok(!section.includes(oldDate));
|
||||
assert.ok(section.includes(recentDate));
|
||||
});
|
||||
|
||||
test('preserves entries within 90 days', () => {
|
||||
const today = new Date();
|
||||
const recent = new Date(today);
|
||||
recent.setDate(recent.getDate() - 30);
|
||||
const recentDate = recent.toISOString().slice(0, 10);
|
||||
const recentDate = daysBeforeFixedToday(30);
|
||||
|
||||
const stateWithRecent = SAMPLE_STATE.replace(
|
||||
'## Recent Posts\n\n',
|
||||
`## Recent Posts\n\n- [${recentDate}] "Recent post..." (1200) - topic\n`
|
||||
/## Recent Posts\n\n[\s\S]*?(?=## Session Notes)/,
|
||||
`## Recent Posts\n\n- [${recentDate}] "Recent post..." (1200) - topic\n\n`
|
||||
);
|
||||
|
||||
const result = pruneContentHistory(stateWithRecent, 90);
|
||||
const result = pruneContentHistory(stateWithRecent, 90, FIXED_TODAY);
|
||||
assert.equal(result, null); // nothing to prune
|
||||
});
|
||||
|
||||
|
|
@ -327,22 +359,19 @@ describe('pruneContentHistory', () => {
|
|||
/## Recent Posts\n\n[\s\S]*?(?=## Session Notes)/,
|
||||
'## Recent Posts\n\n'
|
||||
);
|
||||
const result = pruneContentHistory(emptyRecent, 90);
|
||||
const result = pruneContentHistory(emptyRecent, 90, FIXED_TODAY);
|
||||
assert.equal(result, null);
|
||||
});
|
||||
|
||||
test('handles custom maxAgeDays', () => {
|
||||
const today = new Date();
|
||||
const old = new Date(today);
|
||||
old.setDate(old.getDate() - 40);
|
||||
const oldDate = old.toISOString().slice(0, 10);
|
||||
const oldDate = daysBeforeFixedToday(40);
|
||||
|
||||
const stateWithOld = SAMPLE_STATE.replace(
|
||||
'## Recent Posts\n\n',
|
||||
`## Recent Posts\n\n- [${oldDate}] "Somewhat old..." (1000) - topic\n`
|
||||
/## Recent Posts\n\n[\s\S]*?(?=## Session Notes)/,
|
||||
`## Recent Posts\n\n- [${oldDate}] "Somewhat old..." (1000) - topic\n\n`
|
||||
);
|
||||
|
||||
const result = pruneContentHistory(stateWithOld, 30);
|
||||
const result = pruneContentHistory(stateWithOld, 30, FIXED_TODAY);
|
||||
assert.notEqual(result, null);
|
||||
assert.equal(result.pruned, 1);
|
||||
});
|
||||
|
|
@ -352,25 +381,23 @@ describe('pruneContentHistory', () => {
|
|||
// string search has no $1 group, but `$&` still expands to the whole matched
|
||||
// section and `$$` collapses to `$`, so a kept post like "$$ and $& budget"
|
||||
// corrupted state. The replacement function inserts newSection verbatim.
|
||||
const today = new Date();
|
||||
const old = new Date(today); old.setDate(old.getDate() - 100);
|
||||
const oldDate = old.toISOString().slice(0, 10);
|
||||
const recent = new Date(today); recent.setDate(recent.getDate() - 10);
|
||||
const recentDate = recent.toISOString().slice(0, 10);
|
||||
const oldDate = daysBeforeFixedToday(100);
|
||||
const recentDate = daysBeforeFixedToday(10);
|
||||
|
||||
// Build the fixture with a replacement FUNCTION too — a string replacement here
|
||||
// would itself interpret the `$$`/`$&` we are trying to plant (the very bug under
|
||||
// test), corrupting the fixture before pruneContentHistory ever sees it.
|
||||
const stateWithMix = SAMPLE_STATE.replace(
|
||||
'## Recent Posts\n\n',
|
||||
() => `## Recent Posts\n\n- [${oldDate}] "Old..." (1000) - drop me\n- [${recentDate}] "Saved $&100" (1200) - $$ and $& budget\n`
|
||||
/## Recent Posts\n\n[\s\S]*?(?=## Session Notes)/,
|
||||
() => `## Recent Posts\n\n- [${oldDate}] "Old..." (1000) - drop me\n- [${recentDate}] "Saved $&100" (1200) - $$ and $& budget\n\n`
|
||||
);
|
||||
|
||||
const result = pruneContentHistory(stateWithMix, 90);
|
||||
const result = pruneContentHistory(stateWithMix, 90, FIXED_TODAY);
|
||||
assert.notEqual(result, null);
|
||||
assert.equal(result.pruned, 1);
|
||||
assert.ok(!result.content.includes(oldDate), 'old entry pruned');
|
||||
assert.ok(result.content.includes(`- [${recentDate}] "Saved $&100" (1200) - $$ and $& budget`), 'kept $-bearing entry survives verbatim');
|
||||
const section = recentPostsSection(result.content);
|
||||
assert.ok(!section.includes(oldDate), 'old entry pruned');
|
||||
assert.ok(section.includes(`- [${recentDate}] "Saved $&100" (1200) - $$ and $& budget`), 'kept $-bearing entry survives verbatim');
|
||||
const headings = result.content.match(/^## Recent Posts$/gm) || [];
|
||||
assert.equal(headings.length, 1, 'section must not be duplicated by a $& expansion');
|
||||
});
|
||||
|
|
|
|||
|
|
@ -30,7 +30,6 @@ const __dirname = dirname(fileURLToPath(import.meta.url));
|
|||
const MOVE_FILES = [
|
||||
['assets/voice-samples/authentic-voice-samples.local.md', 'voice-samples/authentic-voice-samples.md'], // D6: drop .local
|
||||
['assets/drafts/queue.json', 'drafts/queue.json'],
|
||||
['assets/analytics/content-history.md', 'analytics/content-history.md'], // B1
|
||||
['config/user-profile.local.md', 'profile/user-profile.md'], // D1 (expected-absent today)
|
||||
];
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
// SessionStart hook for linkedin-studio plugin
|
||||
// Reads persistent state and session context, outputs JSON with additionalContext
|
||||
|
||||
import { readFileSync, existsSync, copyFileSync, writeFileSync, mkdirSync } from 'node:fs';
|
||||
import { readFileSync, existsSync, copyFileSync, writeFileSync, mkdirSync, readdirSync } from 'node:fs';
|
||||
import { join, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { calculateScore } from './personalization-score.mjs';
|
||||
|
|
@ -29,6 +29,75 @@ function daysSince(dateStr) {
|
|||
return Math.floor((Date.now() - epoch) / 86400000);
|
||||
}
|
||||
|
||||
// B-S3: newest trend capture (ISO date) from the persistent trend store, read
|
||||
// as raw JSON. The hook must NOT spawn the tsx CLI at session start (slow +
|
||||
// needs node_modules/tsx); the store schema (trends[].capturedAt) is stable and
|
||||
// versioned, so a direct read is safe. Twin of scripts/trends/src/store.ts:
|
||||
// newestCaptureDate (max capturedAt). Returns null when the store is absent,
|
||||
// unreadable, or empty — no captures yet ⇒ no nudge.
|
||||
function trendsNewestCapture(storePath) {
|
||||
if (!existsSync(storePath)) return null;
|
||||
try {
|
||||
const parsed = JSON.parse(readFileSync(storePath, 'utf-8'));
|
||||
const trends = Array.isArray(parsed?.trends) ? parsed.trends : [];
|
||||
let newest = null;
|
||||
for (const t of trends) {
|
||||
const d = typeof t?.capturedAt === 'string' ? t.capturedAt : null;
|
||||
if (d && (newest === null || d > newest)) newest = d;
|
||||
}
|
||||
return newest;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// RE-R2b: the most recent morning brief (date + pre-rendered summary) for
|
||||
// session-start surfacing. Reads the dated Markdown directly — NO tsx (same
|
||||
// zero-dep discipline as trendsNewestCapture above): the brief's frontmatter
|
||||
// carries a single-line `summary` the hook surfaces verbatim (extractYaml's
|
||||
// [^"\n]* + .trim() guarantees date/summary are newline-free). Returns null when
|
||||
// the dir is absent / empty / unreadable — no brief yet => no block.
|
||||
function latestMorningBrief(briefDir) {
|
||||
if (!existsSync(briefDir)) return null;
|
||||
try {
|
||||
const files = readdirSync(briefDir)
|
||||
.filter((f) => /^\d{4}-\d{2}-\d{2}\.md$/.test(f))
|
||||
.sort();
|
||||
const name = files[files.length - 1];
|
||||
if (!name) return null;
|
||||
const content = readFileSync(join(briefDir, name), 'utf-8');
|
||||
return {
|
||||
date: extractYaml(content, 'date'),
|
||||
summary: extractYaml(content, 'summary'),
|
||||
file: join(briefDir, name),
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// SB-S2: brain consolidation freshness. Both read zero-dep through getDataRoot —
|
||||
// the SAME data root the brain CLI's --apply writes to (sidecar reachable by both).
|
||||
// No tsx, no profile.md parse: a readdir count + a tiny JSON read, cost-bounded.
|
||||
function brainLastRun(statePath) {
|
||||
if (!existsSync(statePath)) return null;
|
||||
try {
|
||||
const j = JSON.parse(readFileSync(statePath, 'utf-8'));
|
||||
return typeof j?.last_run === 'string' ? j.last_run : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function countPublished(publishedDir) {
|
||||
if (!existsSync(publishedDir)) return 0;
|
||||
try {
|
||||
return readdirSync(publishedDir).filter((f) => f.endsWith('.md') && !f.startsWith('.')).length;
|
||||
} catch {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
function isoWeek() {
|
||||
const d = new Date();
|
||||
const dayNum = d.getUTCDay() || 7;
|
||||
|
|
@ -47,7 +116,7 @@ let context = '';
|
|||
|
||||
// M0: relocate per-user data to the external root on first run. Idempotent
|
||||
// (silent no-op once .migrated exists); must precede every moved-path read
|
||||
// (voice/profile/scaffold scores, queue, content-history).
|
||||
// (voice/profile/scaffold scores, queue).
|
||||
let m0Migration = { status: 'skip', moved: [], copied: [] };
|
||||
try {
|
||||
m0Migration = migrateData();
|
||||
|
|
@ -55,6 +124,22 @@ try {
|
|||
// Non-critical: never block session start on migration failure.
|
||||
}
|
||||
|
||||
// SB-S2: ensure the brain/ + ingest/ scaffold exists (zero-dep, idempotent).
|
||||
// UNCONDITIONAL — must run on the fresh-install path too (the reminders block
|
||||
// below only runs when the state file exists). Profile.md SEEDING needs the tsx
|
||||
// fold, so it stays the manual `brain init`; here we only mkdir, and remember
|
||||
// whether to nudge (the nudge is appended AFTER the if/else, since the else
|
||||
// branch reassigns `context` and would clobber an early append).
|
||||
let brainProfileMissing = false;
|
||||
try {
|
||||
for (const d of ['brain/journal', 'ingest/inbox', 'ingest/published']) {
|
||||
mkdirSync(join(getDataRoot(''), d), { recursive: true });
|
||||
}
|
||||
brainProfileMissing = !existsSync(join(getDataRoot('brain'), 'profile.md'));
|
||||
} catch {
|
||||
// Non-critical: never block session start on scaffold-ensure failure.
|
||||
}
|
||||
|
||||
if (existsSync(STATE_FILE)) {
|
||||
const stateContent = readFileSync(STATE_FILE, 'utf-8');
|
||||
|
||||
|
|
@ -309,6 +394,27 @@ if (existsSync(STATE_FILE)) {
|
|||
reminders += '- No analytics data imported yet. Run /linkedin:import to start tracking performance.\\n';
|
||||
}
|
||||
|
||||
// Trend-store staleness (B-S3): warn-only, ≥7 days. Fires ONLY when the store
|
||||
// already holds captures (newest === null ⇒ no nudge), so a never-scanned user
|
||||
// is never nagged. Neutral wording — "scan for trends" hits trend-spotter's own
|
||||
// trigger; no hardcoded beat (de-niche invariant, §17 guard).
|
||||
const newestTrendCapture = trendsNewestCapture(join(getDataRoot('trends'), 'trends.json'));
|
||||
const daysSinceTrend = daysSince(newestTrendCapture);
|
||||
if (daysSinceTrend !== null && daysSinceTrend >= 7) {
|
||||
reminders += `- Trend signals are ${daysSinceTrend} days old. Scan for trends to refresh the digest before planning content.\\n`;
|
||||
}
|
||||
|
||||
// Brain consolidation-due (SB-S2): fires only when published gold records exist
|
||||
// and the loop is stale/never-run (a never-used brain never nags). Counts files
|
||||
// + reads the sidecar last_run — no profile.md parse. Zero-dep.
|
||||
const publishedCount = countPublished(join(getDataRoot('ingest'), 'published'));
|
||||
const brainLastRunDate = brainLastRun(join(getDataRoot('brain'), 'consolidation-state.json'));
|
||||
const daysSinceConsolidation = daysSince(brainLastRunDate);
|
||||
if (publishedCount > 0 && (brainLastRunDate === null || (daysSinceConsolidation !== null && daysSinceConsolidation >= 7))) {
|
||||
const since = brainLastRunDate === null ? 'never run' : `${daysSinceConsolidation}d ago`;
|
||||
reminders += `- ${publishedCount} published post(s) captured, last brain consolidation ${since}. Run \`brain consolidate\` to evolve your profile.\\n`;
|
||||
}
|
||||
|
||||
// Milestone reminders
|
||||
if (milestonePhase && followerCount > 0) {
|
||||
if (milestoneStatus === 'SIGNIFICANTLY BEHIND') {
|
||||
|
|
@ -416,6 +522,20 @@ if (existsSync(STATE_FILE)) {
|
|||
}
|
||||
}
|
||||
|
||||
// SB-S2: nudge to seed the brain if it isn't initialised (after the if/else, so
|
||||
// the fresh-install branch's `context = …` reassignment can't clobber it).
|
||||
if (brainProfileMissing) {
|
||||
context += '\\n## Brain\\n- Brain not initialised. Run `brain init` to seed your evolving profile.\\n';
|
||||
}
|
||||
|
||||
// RE-R2b: surface the latest dated morning brief (zero-tsx; summary is a
|
||||
// pre-rendered single line from the brief's frontmatter). Unconditional on a brief
|
||||
// existing, like the brain nudge above (so the fresh-install branch surfaces it too).
|
||||
const latestBrief = latestMorningBrief(join(getDataRoot('trends'), 'morning-brief'));
|
||||
if (latestBrief && latestBrief.summary) {
|
||||
context += `\\n## Morning Brief (${latestBrief.date})\\n${latestBrief.summary}\\n→ Full brief: ${latestBrief.file}\\n`;
|
||||
}
|
||||
|
||||
// Read REMEMBER.md for user session context
|
||||
const rememberFile = join(getDataRoot(), 'REMEMBER.md');
|
||||
const rememberTemplate = join(PLUGIN_ROOT, 'config', 'REMEMBER.template.md');
|
||||
|
|
|
|||
|
|
@ -133,17 +133,21 @@ export function updatePostTracking(stateContent, { postDate, postTopic, hookText
|
|||
* Remove Recent Posts entries older than maxAgeDays.
|
||||
* @param {string} stateContent - Full state file content
|
||||
* @param {number} [maxAgeDays=90]
|
||||
* @param {Date} [today=new Date()] - Injectable clock (tests pass a fixed date)
|
||||
* @returns {{ content: string, pruned: number } | null}
|
||||
*/
|
||||
export function pruneContentHistory(stateContent, maxAgeDays = 90) {
|
||||
const today = new Date();
|
||||
export function pruneContentHistory(stateContent, maxAgeDays = 90, today = new Date()) {
|
||||
const cutoff = new Date(today);
|
||||
cutoff.setDate(cutoff.getDate() - maxAgeDays);
|
||||
const cutoffStr = cutoff.toISOString().slice(0, 10);
|
||||
|
||||
// Find all Recent Posts entries
|
||||
const entryPattern = /^- \[(\d{4}-\d{2}-\d{2})\] .+$/gm;
|
||||
const recentSection = stateContent.match(/## Recent Posts\n\n?([\s\S]*?)(?=\n## [^R]|\n## $|$)/m);
|
||||
// No /m flag: with /m the $ alternative in the lookahead matched the end of
|
||||
// EVERY line, so the lazy capture stopped after the FIRST entry line and
|
||||
// older entries below it were never scanned (and never pruned). Without /m,
|
||||
// $ matches only end-of-string and the capture spans the whole section.
|
||||
const recentSection = stateContent.match(/## Recent Posts\n\n?([\s\S]*?)(?=\n## [^R]|\n## $|$)/);
|
||||
if (!recentSection || !recentSection[1].trim()) return null;
|
||||
|
||||
const sectionContent = recentSection[1];
|
||||
|
|
|
|||
|
|
@ -1,387 +0,0 @@
|
|||
# AI Content Framework
|
||||
|
||||
Specialized framework for creating LinkedIn content about AI topics. Designed for AI advisors, implementers, and strategists who want to build authority in the AI space.
|
||||
|
||||
## The 4 AI Content Pillars
|
||||
|
||||
Structure your AI content around these four pillars for comprehensive coverage:
|
||||
|
||||
### Pillar 1: AI News & Commentary (30-40% of content)
|
||||
|
||||
**Purpose:** Establish yourself as someone who understands what's happening in AI
|
||||
|
||||
**Content types:**
|
||||
- New model releases and capabilities
|
||||
- Company announcements (OpenAI, Anthropic, Microsoft, Google)
|
||||
- Regulatory developments
|
||||
- Industry trends and shifts
|
||||
- Research paper summaries
|
||||
|
||||
**Your angle matters:**
|
||||
- Don't just report news - add perspective
|
||||
- Connect to your expertise area
|
||||
- Explain implications for your audience
|
||||
- Predict what comes next
|
||||
|
||||
**Example transformations:**
|
||||
|
||||
| News Item | Weak Post | Strong Post |
|
||||
|-----------|-----------|-------------|
|
||||
| "GPT-5 released" | "GPT-5 is here! Amazing capabilities!" | "GPT-5 changes the game for enterprise AI. Here's what actually matters for implementation teams..." |
|
||||
| "EU AI Act passed" | "New AI regulations coming" | "The EU AI Act just passed. After reviewing the 200+ pages, here are the 5 requirements that will hit AI projects hardest..." |
|
||||
| "OpenAI acquires company" | "Big acquisition in AI!" | "OpenAI's acquisition of X signals a shift in strategy. Here's what this means for anyone building on their platform..." |
|
||||
|
||||
### Pillar 2: Practical AI Implementation (30-40% of content)
|
||||
|
||||
**Purpose:** Demonstrate that you've actually done the work
|
||||
|
||||
**Content types:**
|
||||
- How-to guides and tutorials
|
||||
- Implementation patterns and anti-patterns
|
||||
- Tool comparisons and recommendations
|
||||
- Architecture decisions and trade-offs
|
||||
- Troubleshooting and problem-solving
|
||||
|
||||
**Key principles:**
|
||||
- Be specific (exact steps, real examples)
|
||||
- Share failures as much as successes
|
||||
- Explain the "why" behind decisions
|
||||
- Make it actionable
|
||||
|
||||
**Example topics:**
|
||||
|
||||
| Category | Example Topics |
|
||||
|----------|----------------|
|
||||
| Implementation | "How we reduced hallucinations by 60% in our RAG system" |
|
||||
| Patterns | "The 3 architecture patterns I use for every AI project" |
|
||||
| Tools | "Copilot Studio vs Power Automate: When to use each" |
|
||||
| Troubleshooting | "Why your AI pilot succeeded but production failed" |
|
||||
| Process | "Our 5-step AI vendor evaluation process" |
|
||||
|
||||
### Pillar 3: AI Strategy & Leadership (20-30% of content)
|
||||
|
||||
**Purpose:** Speak to decision-makers and establish strategic credibility
|
||||
|
||||
**Content types:**
|
||||
- ROI and business case frameworks
|
||||
- Organizational readiness assessments
|
||||
- Change management for AI
|
||||
- Governance and ethics considerations
|
||||
- Leadership perspectives and decisions
|
||||
|
||||
**Target audience:** C-suite, department heads, IT leadership
|
||||
|
||||
**Example topics:**
|
||||
|
||||
| Focus Area | Example Topics |
|
||||
|------------|----------------|
|
||||
| ROI | "How to calculate AI ROI (the honest way)" |
|
||||
| Readiness | "The 5 questions I ask before any AI project" |
|
||||
| Change | "Why your AI project failed (it wasn't the technology)" |
|
||||
| Governance | "Building an AI governance framework that actually works" |
|
||||
| Leadership | "What I tell CEOs who ask 'Should we invest in AI?'" |
|
||||
|
||||
### Pillar 4: AI Tools & Resources (10-20% of content)
|
||||
|
||||
**Purpose:** Provide tangible value and establish generosity
|
||||
|
||||
**Content types:**
|
||||
- Free templates and frameworks
|
||||
- Tool recommendations and reviews
|
||||
- Resource roundups and guides
|
||||
- Skills and capabilities shares
|
||||
- Checklists and cheat sheets
|
||||
|
||||
**Key principles:**
|
||||
- Give away genuinely useful things
|
||||
- Don't gate everything behind email capture
|
||||
- Update regularly as tools change
|
||||
- Focus on tools you actually use
|
||||
|
||||
**Example shares:**
|
||||
|
||||
| Type | Examples |
|
||||
|------|----------|
|
||||
| Templates | "AI project kickoff template (the one I actually use)" |
|
||||
| Checklists | "Pre-deployment AI checklist (20 items)" |
|
||||
| Frameworks | "My vendor evaluation scorecard" |
|
||||
| Guides | "2026 AI tool landscape for enterprise" |
|
||||
| Skills | "Custom Claude Code skill for AI documentation" |
|
||||
|
||||
## AI News Monitoring Routine
|
||||
|
||||
Stay current without drowning in information.
|
||||
|
||||
### Daily Routine (10 minutes)
|
||||
|
||||
**Morning scan:**
|
||||
1. Check top 3 AI news sources (see list below)
|
||||
2. Note 1-2 stories relevant to your expertise
|
||||
3. Add to content ideas if commentary-worthy
|
||||
|
||||
**Key sources for daily scan:**
|
||||
- The Batch (Andrew Ng's newsletter)
|
||||
- AI News (VentureBeat)
|
||||
- Anthropic/OpenAI/Microsoft announcements
|
||||
- r/MachineLearning (top posts)
|
||||
|
||||
### Weekly Routine (30 minutes)
|
||||
|
||||
**Dedicated AI research block:**
|
||||
|
||||
1. **Research papers** (10 min)
|
||||
- ArXiv AI papers (top cited)
|
||||
- Google Research blog
|
||||
- Microsoft Research blog
|
||||
|
||||
2. **Industry analysis** (10 min)
|
||||
- AI-focused podcasts
|
||||
- YouTube channels (AI Explained, Two Minute Papers)
|
||||
- LinkedIn content from top AI voices
|
||||
|
||||
3. **Content planning** (10 min)
|
||||
- Which news items merit posts?
|
||||
- What patterns are emerging?
|
||||
- What's my audience asking about?
|
||||
|
||||
### Sources by Priority
|
||||
|
||||
**Tier 1: Must follow (daily)**
|
||||
- OpenAI blog/announcements
|
||||
- Anthropic blog/announcements
|
||||
- Microsoft AI blog
|
||||
- Google AI blog
|
||||
|
||||
**Tier 2: High value (2-3x/week)**
|
||||
- MIT Technology Review
|
||||
- The Verge AI section
|
||||
- Ars Technica AI
|
||||
- Stratechery (Ben Thompson)
|
||||
|
||||
**Tier 3: Deep dives (weekly)**
|
||||
- ArXiv (cs.AI, cs.CL, cs.LG)
|
||||
- Distill.pub
|
||||
- Papers With Code
|
||||
|
||||
**Tier 4: Community (as needed)**
|
||||
- r/MachineLearning
|
||||
- r/LocalLLaMA
|
||||
- Hacker News AI discussions
|
||||
- AI Twitter/X threads
|
||||
|
||||
## Content Trigger Framework
|
||||
|
||||
Know when AI news warrants a post.
|
||||
|
||||
### High-Priority Triggers (post within 24-48 hours)
|
||||
|
||||
**Always post about:**
|
||||
- Major model releases (GPT-X, Claude X, Gemini X)
|
||||
- Significant capability breakthroughs
|
||||
- Regulatory decisions affecting AI use
|
||||
- Major acquisitions/partnerships
|
||||
- Security vulnerabilities in AI systems
|
||||
|
||||
**Why timing matters:**
|
||||
- First-mover advantage in commentary
|
||||
- Algorithm favors timely content
|
||||
- Establishes you as "in the know"
|
||||
|
||||
### Medium-Priority Triggers (post within week)
|
||||
|
||||
**Consider posting about:**
|
||||
- Research papers with practical implications
|
||||
- Industry reports with notable findings
|
||||
- Tool updates and feature releases
|
||||
- Conference announcements
|
||||
- Company strategy shifts
|
||||
|
||||
### Low-Priority Triggers (optional)
|
||||
|
||||
**Skip or brief mention:**
|
||||
- Incremental updates
|
||||
- Minor funding rounds
|
||||
- Personnel changes (unless significant)
|
||||
- Speculation and rumors
|
||||
- Vendor marketing announcements
|
||||
|
||||
### The Relevance Filter
|
||||
|
||||
**Before posting, ask:**
|
||||
|
||||
1. **Is this relevant to my expertise areas?**
|
||||
- Yes = proceed
|
||||
- No = skip (unless huge news)
|
||||
|
||||
2. **Does my audience care?**
|
||||
- Public sector leaders? Check.
|
||||
- Enterprise AI implementers? Check.
|
||||
- General tech enthusiasts? Maybe skip.
|
||||
|
||||
3. **Can I add unique perspective?**
|
||||
- Have implementation experience? Post.
|
||||
- Just repeating news? Skip or brief.
|
||||
|
||||
4. **Is there urgency?**
|
||||
- Time-sensitive = prioritize
|
||||
- Evergreen = can wait
|
||||
|
||||
## AI-Specific Hook Templates
|
||||
|
||||
Templates optimized for AI content.
|
||||
|
||||
### News Commentary Hooks
|
||||
|
||||
```
|
||||
"[Company] just announced [thing]. Here's what most commentators are missing..."
|
||||
|
||||
"Everyone's talking about [AI development]. After [X] implementations, here's what actually matters..."
|
||||
|
||||
"The [AI announcement] headlines are wrong. The real story is..."
|
||||
|
||||
"[Number] hours after [AI release], here's my first assessment..."
|
||||
|
||||
"While everyone focuses on [obvious thing], the real implication of [news] is..."
|
||||
```
|
||||
|
||||
### Implementation Insight Hooks
|
||||
|
||||
```
|
||||
"We just deployed [AI system] for [use case]. The hardest part wasn't what you'd expect..."
|
||||
|
||||
"After [X] AI projects, I've seen the same pattern [Y]% of the time..."
|
||||
|
||||
"Everyone says [common AI advice]. In practice, the opposite is true..."
|
||||
|
||||
"The difference between AI projects that succeed and fail? It's not the technology..."
|
||||
|
||||
"I just reviewed [X] failed AI projects. They all made this mistake..."
|
||||
```
|
||||
|
||||
### Strategy/Leadership Hooks
|
||||
|
||||
```
|
||||
"Our CEO asked me: 'Should we invest in AI?' Here's what I told her..."
|
||||
|
||||
"Most AI strategies fail for the same reason. Here's the fix..."
|
||||
|
||||
"Before any AI project, I ask these 5 questions. #3 is the killer..."
|
||||
|
||||
"The uncomfortable truth about AI ROI that vendors won't tell you..."
|
||||
|
||||
"What separates AI-ready organizations from the rest? It's not budget..."
|
||||
```
|
||||
|
||||
### Tool/Resource Hooks
|
||||
|
||||
```
|
||||
"I've tested [X] AI tools for [use case]. Here's the winner (and why)..."
|
||||
|
||||
"Free resource: The [framework/template] I use for every [AI task]..."
|
||||
|
||||
"[Tool] vs [Tool]: After using both for [time], here's my verdict..."
|
||||
|
||||
"This [free tool] changed how I approach [AI task]..."
|
||||
|
||||
"I built this [skill/template/framework] for my own use. Now it's yours..."
|
||||
```
|
||||
|
||||
## AI Topic Calendar
|
||||
|
||||
Structure your AI content across the month.
|
||||
|
||||
### Weekly AI Topic Rotation
|
||||
|
||||
| Week | Primary Focus | Secondary Focus |
|
||||
|------|---------------|-----------------|
|
||||
| 1 | News & Commentary | Strategy insight |
|
||||
| 2 | Implementation how-to | Tool/resource |
|
||||
| 3 | News & Commentary | Case study |
|
||||
| 4 | Strategy deep-dive | Tool/resource |
|
||||
|
||||
### Monthly AI Content Mix
|
||||
|
||||
**For 8-12 posts per month:**
|
||||
|
||||
| Pillar | Posts | Examples |
|
||||
|--------|-------|----------|
|
||||
| News & Commentary | 3-4 | News reactions, trend analysis |
|
||||
| Implementation | 3-4 | How-tos, patterns, lessons |
|
||||
| Strategy | 1-2 | Leadership posts, frameworks |
|
||||
| Tools & Resources | 1-2 | Shares, comparisons, giveaways |
|
||||
|
||||
### Seasonal AI Topics
|
||||
|
||||
**Q1 (Jan-Mar):**
|
||||
- Predictions and trends
|
||||
- Budget planning for AI
|
||||
- New year AI resolutions/strategies
|
||||
|
||||
**Q2 (Apr-Jun):**
|
||||
- Conference season coverage
|
||||
- Mid-year assessments
|
||||
- Implementation case studies
|
||||
|
||||
**Q3 (Jul-Sep):**
|
||||
- Summer project retrospectives
|
||||
- H2 planning
|
||||
- Back-to-school AI skills
|
||||
|
||||
**Q4 (Oct-Dec):**
|
||||
- Year-end reflections
|
||||
- Predictions for next year
|
||||
- Budget justification content
|
||||
|
||||
## AI Content Quality Checklist
|
||||
|
||||
Before posting AI content:
|
||||
|
||||
### Accuracy Check
|
||||
- [ ] Claims are factually accurate
|
||||
- [ ] Statistics are sourced and current
|
||||
- [ ] Technical details are correct
|
||||
- [ ] No AI hype or fear-mongering
|
||||
|
||||
### Expertise Signal
|
||||
- [ ] Post demonstrates real experience
|
||||
- [ ] Specific examples included
|
||||
- [ ] Avoids generic AI cliches
|
||||
- [ ] Shows nuanced understanding
|
||||
|
||||
### Audience Value
|
||||
- [ ] Relevant to target audience
|
||||
- [ ] Actionable where appropriate
|
||||
- [ ] Not just information, but insight
|
||||
- [ ] Answers "so what?"
|
||||
|
||||
### Differentiation
|
||||
- [ ] Adds perspective beyond news
|
||||
- [ ] Shows unique angle/experience
|
||||
- [ ] Not duplicating what everyone else says
|
||||
- [ ] Reflects my expertise areas
|
||||
|
||||
## AI Content Anti-Patterns
|
||||
|
||||
**Avoid these common AI content mistakes:**
|
||||
|
||||
| Anti-Pattern | Why It's Bad | Better Approach |
|
||||
|--------------|--------------|-----------------|
|
||||
| "AI will change everything!" | Vague hype | Specific, grounded claims |
|
||||
| "AI is dangerous/scary" | Fear-mongering | Balanced assessment |
|
||||
| Just sharing announcements | No added value | Add your perspective |
|
||||
| "10 AI tools you need" | Generic listicle | Curated with experience |
|
||||
| Jargon-heavy technical posts | Alienates audience | Accessible explanations |
|
||||
| "AI will replace [job]" | Tired take | Nuanced workforce analysis |
|
||||
| Vendor press releases | Looks like promotion | Independent perspective |
|
||||
| Repeating common advice | No differentiation | Counter-conventional takes |
|
||||
|
||||
## Integration with Main Skill
|
||||
|
||||
This framework integrates with the main LinkedIn content skill:
|
||||
|
||||
- **Angles:** AI content uses same 8 angles (content-angles.md)
|
||||
- **Formats:** Follow format guidelines in linkedin-formats.md
|
||||
- **Engagement:** Apply same engagement frameworks
|
||||
- **Growth:** Contributes to overall authority building
|
||||
|
||||
The difference: AI content requires staying current with fast-moving developments and maintaining technical credibility while remaining accessible to non-technical audiences.
|
||||
|
|
@ -113,14 +113,25 @@ improves writing); do not justify it as "reduces reach."
|
|||
|
||||
## The deployed ranking model — what we can and cannot say
|
||||
|
||||
> **An LLM-based relevance-ranking system is live on LinkedIn in 2026.**
|
||||
> **No public name. No deployment date.**
|
||||
> **LinkedIn's feed ranking model has an official name: the Generative Recommender (GR).**
|
||||
> Announced 2026-03-12 on LinkedIn's engineering blog (Hristo Danchev,
|
||||
> [Engineering the next generation of LinkedIn's feed](https://www.linkedin.com/blog/engineering/feed/engineering-the-next-generation-of-linkedins-feed)):
|
||||
> a sequential transformer-based ranker that treats member interaction history as a
|
||||
> timeline, paired with a unified LLM-embedding retrieval system. Rollout announced in
|
||||
> the same post.
|
||||
|
||||
| Claim | Statement | Source | Confidence |
|
||||
|-------|-----------|--------|------------|
|
||||
| A live LLM relevance system exists | Confirmed in direction by LinkedIn's 2026 communications. | LinkedIn comms (2026) | high |
|
||||
| Production name | **Not publishable as fact.** The most-cited arXiv paper (2501.16450) is a Jan-**2025** *pre-production research* model (V1.0, 150B params, offline parity only), **withdrawn 2025-08-23**. A circulating "Generative Recommender / Hristo Danchev" engineering-post citation was independently flagged as **likely fabricated** — do not propagate. | arXiv 2501.16450; Gemini provenance flag | high (on the negative claim) |
|
||||
| Deployment date | No primary source. The "early-2026" date is third-party extrapolation from the paper's Jan-**2025** date. **Do not assert a date.** | — | n/a |
|
||||
| Production name | **Generative Recommender (GR)** — official, primary-source. | LinkedIn engineering blog, 2026-03-12 | high |
|
||||
| LLM-based retrieval | Confirmed: "a unified retrieval system leveraging advances in LLMs to generate a high-quality representation of our members and content." | Same post | high |
|
||||
| Deployment | Rollout announced 2026-03-12 ("rolling out a new advanced ranking system"). Full-coverage completion date not stated — do not assert one. | Same post | high (announcement), n/a (completion) |
|
||||
| "360Brew" as the production name | **Still not publishable.** The arXiv paper (2501.16450) is a Jan-**2025** *pre-production research* model (V1.0, 150B params, offline parity only), **withdrawn 2025-08-23**; the "360Brew" label is third-party and has no official confirmation. GR is the official name. | arXiv 2501.16450 | high (on the negative claim) |
|
||||
|
||||
*Correction note (2026-07-17): this section previously said "No public name. No
|
||||
deployment date." and flagged the Generative Recommender / Hristo Danchev
|
||||
engineering-post citation as likely fabricated. That flag was wrong — and was already
|
||||
wrong at "Last updated 2026-05": the official post had been live since 2026-03-12,
|
||||
two months earlier. The fabrication flag rejected a genuine primary source.*
|
||||
|
||||
## Operational heuristics (directional — test per account)
|
||||
|
||||
|
|
@ -177,10 +188,11 @@ source.)
|
|||
|
||||
---
|
||||
|
||||
*Last updated: 2026-05. Maintained as the single canonical algorithm statement; cite, do
|
||||
not restate.*
|
||||
*Last updated: 2026-07-17 (GR-model correction). Maintained as the single canonical
|
||||
algorithm statement; cite, do not restate.*
|
||||
|
||||
*Sources (per-claim quality/confidence noted inline): arXiv 2501.16450 (pre-production
|
||||
*Sources (per-claim quality/confidence noted inline): LinkedIn Engineering — "Engineering
|
||||
the next generation of LinkedIn's feed" (Hristo Danchev, 2026-03-12); arXiv 2501.16450 (pre-production
|
||||
research paper, withdrawn 2025-08-23); LinkedIn Engineering — "Leveraging Dwell Time" (2024); Tim Jurka, Head of Feed
|
||||
AI (2025-08-11); Laura Lorenzetti, VP & Exec Editor (2026-05-19); Gyanda Sachdeva, VP
|
||||
Product (2026-02-16); Matt Navarra relaying LinkedIn Sr. Director Product (Aug 2025);
|
||||
|
|
|
|||
351
references/content-framework.md
Normal file
351
references/content-framework.md
Normal file
|
|
@ -0,0 +1,351 @@
|
|||
# Content Framework
|
||||
|
||||
A framework for creating LinkedIn content in **your** field — whatever that field is. The structure (pillars, triggers, source tiers, calendar) is domain-general; the subject matter comes from **your content pillars and expertise areas** (loaded from your profile), never baked into this file. The worked examples below deliberately span different domains so you can see the pattern, not inherit someone else's beat.
|
||||
|
||||
> **How to read the examples:** placeholders in `[brackets]` are filled from your own domain. Where a concrete illustration is given, it is an *example from one field* — substitute the equivalent from yours.
|
||||
|
||||
## The 4 Content Pillars
|
||||
|
||||
Structure your content around these four pillars for comprehensive coverage. The percentages are a starting balance, not a rule.
|
||||
|
||||
### Pillar 1: News & Commentary (30-40% of content)
|
||||
|
||||
**Purpose:** Establish yourself as someone who understands what's happening in your field
|
||||
|
||||
**Content types:**
|
||||
- New releases, products, or capabilities in your domain
|
||||
- Notable announcements from the players that matter to your audience
|
||||
- Regulatory or policy developments
|
||||
- Industry trends and shifts
|
||||
- Summaries of new research or reports
|
||||
|
||||
**Your angle matters:**
|
||||
- Don't just report news — add perspective
|
||||
- Connect it to your expertise area
|
||||
- Explain implications for your audience
|
||||
- Predict what comes next
|
||||
|
||||
**Example transformations (different fields, same move):**
|
||||
|
||||
| News Item | Weak Post | Strong Post |
|
||||
|-----------|-----------|-------------|
|
||||
| A major product release in your field | "It's here! Amazing!" | "This changes the calculus for [your audience]. Here's what actually matters when you go to implement it..." |
|
||||
| A new regulation passes | "New rules coming" | "After reading the 200+ pages, here are the 5 requirements that will hit [audience]'s projects hardest..." |
|
||||
| A large acquisition in your sector | "Big deal in [sector]!" | "This acquisition signals a strategy shift. Here's what it means for anyone building on their platform..." |
|
||||
|
||||
### Pillar 2: Practical Implementation (30-40% of content)
|
||||
|
||||
**Purpose:** Demonstrate that you've actually done the work
|
||||
|
||||
**Content types:**
|
||||
- How-to guides and tutorials
|
||||
- Implementation patterns and anti-patterns
|
||||
- Tool comparisons and recommendations
|
||||
- Decisions and trade-offs you actually made
|
||||
- Troubleshooting and problem-solving
|
||||
|
||||
**Key principles:**
|
||||
- Be specific (exact steps, real numbers, real examples)
|
||||
- Share failures as much as successes
|
||||
- Explain the "why" behind decisions
|
||||
- Make it actionable
|
||||
|
||||
**Example topics (spanning fields):**
|
||||
|
||||
| Category | Example Topics |
|
||||
|----------|----------------|
|
||||
| Implementation | "How we cut [a costly metric] by 60% in our [system]" |
|
||||
| Patterns | "The 3 patterns I reach for on every [type of] project" |
|
||||
| Tools | "[Tool A] vs [Tool B]: when to use each" |
|
||||
| Troubleshooting | "Why our pilot succeeded but production failed" |
|
||||
| Process | "Our 5-step vendor evaluation process" |
|
||||
|
||||
### Pillar 3: Strategy & Leadership (20-30% of content)
|
||||
|
||||
**Purpose:** Speak to decision-makers and establish strategic credibility
|
||||
|
||||
**Content types:**
|
||||
- ROI and business-case frameworks
|
||||
- Organizational readiness assessments
|
||||
- Change management
|
||||
- Governance and ethics considerations
|
||||
- Leadership perspectives and decisions
|
||||
|
||||
**Target audience:** C-suite, department heads, the leaders in your space
|
||||
|
||||
**Example topics:**
|
||||
|
||||
| Focus Area | Example Topics |
|
||||
|------------|----------------|
|
||||
| ROI | "How to calculate the ROI of [your initiative] (the honest way)" |
|
||||
| Readiness | "The 5 questions I ask before any [type of] project" |
|
||||
| Change | "Why your [initiative] failed (it wasn't the technology)" |
|
||||
| Governance | "Building a governance framework that actually works" |
|
||||
| Leadership | "What I tell executives who ask 'Should we invest in [X]?'" |
|
||||
|
||||
### Pillar 4: Tools & Resources (10-20% of content)
|
||||
|
||||
**Purpose:** Provide tangible value and establish generosity
|
||||
|
||||
**Content types:**
|
||||
- Free templates and frameworks
|
||||
- Tool recommendations and reviews
|
||||
- Resource roundups and guides
|
||||
- Skills and capabilities you share
|
||||
- Checklists and cheat sheets
|
||||
|
||||
**Key principles:**
|
||||
- Give away genuinely useful things
|
||||
- Don't gate everything behind email capture
|
||||
- Update regularly as the field changes
|
||||
- Focus on tools you actually use
|
||||
|
||||
**Example shares:**
|
||||
|
||||
| Type | Examples |
|
||||
|------|----------|
|
||||
| Templates | "The kickoff template I actually use" |
|
||||
| Checklists | "Pre-deployment checklist (20 items)" |
|
||||
| Frameworks | "My vendor evaluation scorecard" |
|
||||
| Guides | "The [current-year] tool landscape for [your audience]" |
|
||||
| Skills | "A custom tool I built for my own [task]" |
|
||||
|
||||
## Content Monitoring Routine
|
||||
|
||||
Stay current in your field without drowning in information.
|
||||
|
||||
> **Where you look is config, not baked into this file.** The specific sources to monitor — your vendors, regulators, outlets, communities — live in a source list the trend engine loads at runtime: `config/trends-sources.template.md` (shipped generic categories) with a user override at `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/trends/sources.md` (your own niche list, which survives upgrades/reinstalls). Populate the tiers there with **your** domain's sources. The *rhythm* below is what generalizes; the *sources* are yours.
|
||||
|
||||
### Daily Routine (10 minutes)
|
||||
|
||||
**Morning scan:**
|
||||
1. Check your top 3 field sources (your Tier 1 list)
|
||||
2. Note 1-2 stories relevant to your expertise
|
||||
3. Add to content ideas if commentary-worthy
|
||||
|
||||
### Weekly Routine (30 minutes)
|
||||
|
||||
**Dedicated research block:**
|
||||
|
||||
1. **Deep sources** (10 min) — research, primary reports, authoritative analysis in your field
|
||||
2. **Industry analysis** (10 min) — podcasts, channels, and the voices your audience follows
|
||||
3. **Content planning** (10 min)
|
||||
- Which items merit posts?
|
||||
- What patterns are emerging?
|
||||
- What is my audience asking about?
|
||||
|
||||
### Source Tiers (cadence, not a fixed list)
|
||||
|
||||
Group your own sources into four tiers by how fast they move, then poll on that cadence:
|
||||
|
||||
| Tier | What lives here | Cadence |
|
||||
|------|-----------------|---------|
|
||||
| **Tier 1 — Primary / breaking** | first-party announcements, authoritative decisions | daily |
|
||||
| **Tier 2 — Analysis & research** | where developments get interpreted, not just reported | 2-3×/week |
|
||||
| **Tier 3 — Community signals** | where practitioners surface what matters before the press | weekly |
|
||||
| **Tier 4 — Niche & seasonal** | slower sources with predictable cadence | monthly |
|
||||
|
||||
## Content Trigger Framework
|
||||
|
||||
Know when news in your field warrants a post.
|
||||
|
||||
### High-Priority Triggers (post within 24-48 hours)
|
||||
|
||||
**Always consider posting about:**
|
||||
- Major releases or capability breakthroughs in your domain
|
||||
- Regulatory decisions affecting how your audience works
|
||||
- Major acquisitions or partnerships among the players that matter
|
||||
- Security or safety issues in systems your audience relies on
|
||||
|
||||
**Why timing matters:**
|
||||
- First-mover advantage in commentary
|
||||
- The algorithm favors timely content
|
||||
- Establishes you as "in the know"
|
||||
|
||||
### Medium-Priority Triggers (post within a week)
|
||||
|
||||
**Consider posting about:**
|
||||
- Research or reports with practical implications
|
||||
- Tool updates and feature releases
|
||||
- Conference takeaways
|
||||
- Strategy shifts among notable players
|
||||
|
||||
### Low-Priority Triggers (optional)
|
||||
|
||||
**Skip or brief mention:**
|
||||
- Incremental updates
|
||||
- Minor funding rounds
|
||||
- Personnel changes (unless significant)
|
||||
- Speculation and rumors
|
||||
- Vendor marketing announcements
|
||||
|
||||
### The Relevance Filter
|
||||
|
||||
**Before posting, ask:**
|
||||
|
||||
1. **Is this relevant to my expertise areas?** Yes = proceed · No = skip (unless huge news)
|
||||
2. **Does my audience care?** Check it against the audience in your profile — the people you actually write for. If it's adjacent-but-off, maybe skip.
|
||||
3. **Can I add unique perspective?** Direct experience = post · Just repeating news = skip or brief
|
||||
4. **Is there urgency?** Time-sensitive = prioritize · Evergreen = can wait
|
||||
|
||||
## Hook Templates
|
||||
|
||||
Templates for content built on news and expertise. Fill the `[brackets]` from your domain.
|
||||
|
||||
### News Commentary Hooks
|
||||
|
||||
```
|
||||
"[Player] just announced [thing]. Here's what most commentators are missing..."
|
||||
|
||||
"Everyone's talking about [development]. After [X] implementations, here's what actually matters..."
|
||||
|
||||
"The [announcement] headlines are wrong. The real story is..."
|
||||
|
||||
"[Number] hours after [release], here's my first assessment..."
|
||||
|
||||
"While everyone focuses on [obvious thing], the real implication of [news] is..."
|
||||
```
|
||||
|
||||
### Implementation Insight Hooks
|
||||
|
||||
```
|
||||
"We just deployed [system] for [use case]. The hardest part wasn't what you'd expect..."
|
||||
|
||||
"After [X] projects, I've seen the same pattern [Y]% of the time..."
|
||||
|
||||
"Everyone says [common advice]. In practice, the opposite is true..."
|
||||
|
||||
"The difference between projects that succeed and fail? It's not the technology..."
|
||||
|
||||
"I just reviewed [X] failed projects. They all made this mistake..."
|
||||
```
|
||||
|
||||
### Strategy/Leadership Hooks
|
||||
|
||||
```
|
||||
"Our CEO asked me: 'Should we invest in [X]?' Here's what I told her..."
|
||||
|
||||
"Most [domain] strategies fail for the same reason. Here's the fix..."
|
||||
|
||||
"Before any [type of] project, I ask these 5 questions. #3 is the killer..."
|
||||
|
||||
"The uncomfortable truth about [X] ROI that vendors won't tell you..."
|
||||
|
||||
"What separates [X]-ready organizations from the rest? It's not budget..."
|
||||
```
|
||||
|
||||
### Tool/Resource Hooks
|
||||
|
||||
```
|
||||
"I've tested [X] tools for [use case]. Here's the winner (and why)..."
|
||||
|
||||
"Free resource: the [framework/template] I use for every [task]..."
|
||||
|
||||
"[Tool] vs [Tool]: after using both for [time], here's my verdict..."
|
||||
|
||||
"This [free tool] changed how I approach [task]..."
|
||||
|
||||
"I built this [skill/template/framework] for my own use. Now it's yours..."
|
||||
```
|
||||
|
||||
## Topic Calendar
|
||||
|
||||
Structure your content across the month.
|
||||
|
||||
### Weekly Topic Rotation
|
||||
|
||||
| Week | Primary Focus | Secondary Focus |
|
||||
|------|---------------|-----------------|
|
||||
| 1 | News & Commentary | Strategy insight |
|
||||
| 2 | Implementation how-to | Tool/resource |
|
||||
| 3 | News & Commentary | Case study |
|
||||
| 4 | Strategy deep-dive | Tool/resource |
|
||||
|
||||
### Monthly Content Mix
|
||||
|
||||
**For 8-12 posts per month:**
|
||||
|
||||
| Pillar | Posts | Examples |
|
||||
|--------|-------|----------|
|
||||
| News & Commentary | 3-4 | News reactions, trend analysis |
|
||||
| Implementation | 3-4 | How-tos, patterns, lessons |
|
||||
| Strategy | 1-2 | Leadership posts, frameworks |
|
||||
| Tools & Resources | 1-2 | Shares, comparisons, giveaways |
|
||||
|
||||
### Seasonal Topics (rhythm, adapt to your field)
|
||||
|
||||
The calendar rhythm is general; fill it with your domain's events and cycles.
|
||||
|
||||
**Q1 (Jan-Mar):**
|
||||
- Predictions and trends for the year
|
||||
- Budget planning
|
||||
- New-year resolutions/strategies
|
||||
|
||||
**Q2 (Apr-Jun):**
|
||||
- Conference season coverage (your field's events)
|
||||
- Mid-year assessments
|
||||
- Implementation case studies
|
||||
|
||||
**Q3 (Jul-Sep):**
|
||||
- Summer project retrospectives
|
||||
- H2 planning
|
||||
- Skills and fundamentals content
|
||||
|
||||
**Q4 (Oct-Dec):**
|
||||
- Year-end reflections
|
||||
- Predictions for next year
|
||||
- Budget-justification content
|
||||
|
||||
## Content Quality Checklist
|
||||
|
||||
Before posting:
|
||||
|
||||
### Accuracy Check
|
||||
- [ ] Claims are factually accurate
|
||||
- [ ] Statistics are sourced and current
|
||||
- [ ] Technical details are correct
|
||||
- [ ] No hype or fear-mongering
|
||||
|
||||
### Expertise Signal
|
||||
- [ ] Post demonstrates real experience
|
||||
- [ ] Specific examples included
|
||||
- [ ] Avoids generic cliches
|
||||
- [ ] Shows nuanced understanding
|
||||
|
||||
### Audience Value
|
||||
- [ ] Relevant to target audience
|
||||
- [ ] Actionable where appropriate
|
||||
- [ ] Not just information, but insight
|
||||
- [ ] Answers "so what?"
|
||||
|
||||
### Differentiation
|
||||
- [ ] Adds perspective beyond the news
|
||||
- [ ] Shows unique angle/experience
|
||||
- [ ] Not duplicating what everyone else says
|
||||
- [ ] Reflects my expertise areas
|
||||
|
||||
## Content Anti-Patterns
|
||||
|
||||
**Avoid these common mistakes:**
|
||||
|
||||
| Anti-Pattern | Why It's Bad | Better Approach |
|
||||
|--------------|--------------|-----------------|
|
||||
| "[Field] will change everything!" | Vague hype | Specific, grounded claims |
|
||||
| "[Field] is dangerous/scary" | Fear-mongering | Balanced assessment |
|
||||
| Just sharing announcements | No added value | Add your perspective |
|
||||
| "10 [tools] you need" | Generic listicle | Curated with experience |
|
||||
| Jargon-heavy technical posts | Alienates audience | Accessible explanations |
|
||||
| "[X] will replace [job]" | Tired take | Nuanced workforce analysis |
|
||||
| Vendor press releases | Looks like promotion | Independent perspective |
|
||||
| Repeating common advice | No differentiation | Counter-conventional takes |
|
||||
|
||||
## Integration with Main Skill
|
||||
|
||||
This framework integrates with the main LinkedIn content skill:
|
||||
|
||||
- **Angles:** content uses the same 8 angles (content-angles.md)
|
||||
- **Formats:** follow format guidelines in linkedin-formats.md
|
||||
- **Engagement:** apply the same engagement frameworks
|
||||
- **Growth:** contributes to overall authority building
|
||||
|
||||
The difference for fast-moving fields: they require staying current with rapid developments and maintaining technical credibility while remaining accessible to non-specialist audiences.
|
||||
|
|
@ -23,7 +23,7 @@ version. This generalizes the proven newsletter pattern
|
|||
```
|
||||
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/
|
||||
voice-samples/ authentic-voice-samples.md chronicle-voice-drift-log.md
|
||||
analytics/ exports/ posts/ weekly-reports/ monthly-reports/ ab-tests/ content-history.md
|
||||
analytics/ exports/ posts/ weekly-reports/ monthly-reports/ ab-tests/
|
||||
drafts/ queue.json week-*/ carousel/ multiplatform/ repurposed/
|
||||
profile/ user-profile.md
|
||||
frameworks/ <slug>.md
|
||||
|
|
|
|||
|
|
@ -64,7 +64,7 @@ Pattern: Set a scene that resonates
|
|||
|
||||
**10. The Direct Address**
|
||||
Pattern: Speak directly to a specific audience
|
||||
- "If you're an AI leader in the public sector, we need to talk."
|
||||
- "If you run RevOps at a B2B SaaS company, we need to talk."
|
||||
- "To everyone implementing AI right now: Pause and read this."
|
||||
- "Fellow AI advisors: Are we being honest about timelines?"
|
||||
|
||||
|
|
|
|||
72
references/figure-design-guidelines.md
Normal file
72
references/figure-design-guidelines.md
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
# Figure Design Guidelines — coded figures
|
||||
|
||||
Design rules and conventions for **coded figures**: SVG/HTML sources rendered to
|
||||
PNG with `${CLAUDE_PLUGIN_ROOT}/render/build-figur.mjs`. Coded is the **primary
|
||||
route for data figures** (charts, diagrams, comparisons — anything whose content
|
||||
is real numbers or real structure): precision and reproducibility beat generative
|
||||
output, and the figure re-renders identically after a correction. Generative
|
||||
images (`mcp-image`) remain the route for **illustrative** work (cover art, mood,
|
||||
metaphor). Strategy-level guidance on *when* a post needs a visual at all lives in
|
||||
`linkedin-visual-style.md`.
|
||||
|
||||
## The three render targets
|
||||
|
||||
| Target | Size | Use |
|
||||
|--------|------|-----|
|
||||
| `article` (default) | 1200 × content-driven height (aspect derived from the source's `viewBox`/dimensions; fallback 16:9) | inline figures in long-form articles |
|
||||
| `carousel` | 1080 × 1350 (4:5) | carousel/document-post slides |
|
||||
| `single` | 1200 × 1200 (1:1) | standalone feed-post image |
|
||||
|
||||
`--width N` / `--height N` override any target. If an `article`-target source
|
||||
declares no dimensions, the renderer warns and falls back to 16:9 — set an
|
||||
explicit `viewBox` (SVG) or `--height`.
|
||||
|
||||
```bash
|
||||
node "${CLAUDE_PLUGIN_ROOT}/render/build-figur.mjs" figN.svg --target article --out figN.png
|
||||
```
|
||||
|
||||
CLI and importable module (`renderFigure` is async). No npm dependencies; needs
|
||||
headless Chrome (auto-discovered: app bundle → PATH → error with install hint).
|
||||
No network access during render — sources must be local and self-contained.
|
||||
|
||||
## Design rules
|
||||
|
||||
Enforced as **warn-only validation** by the renderer (never a hard fail); treat
|
||||
warnings as a review checklist, not noise:
|
||||
|
||||
1. **Text ≥ 24pt/px effective size.** Figures are read on a phone in the feed —
|
||||
smaller text is illegible at feed scale. (Same floor as the carousel rules in
|
||||
`linkedin-visual-style.md`.)
|
||||
2. **Color budget: max 1 primary + 1 secondary beyond neutrals.** Grays, white,
|
||||
and black are free; every additional hue must earn its place. More colors read
|
||||
as noise, not information.
|
||||
3. **≥ 40 % whitespace — an editorial rule, not machine-checked.** Density is the
|
||||
most common coded-figure failure. If the figure needs a legend to be parsed,
|
||||
it probably needs to be two figures.
|
||||
|
||||
## Brand-token convention
|
||||
|
||||
The renderer injects design tokens as CSS variables (`--figur-*`) so figure
|
||||
sources stay brand-agnostic. Tokens are **user data**, read from the data root
|
||||
(see `data-path-convention.md`):
|
||||
|
||||
```
|
||||
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/profile/brand-tokens.json
|
||||
```
|
||||
|
||||
| Key | CSS variable | Neutral default |
|
||||
|-----|--------------|-----------------|
|
||||
| `background` | `--figur-background` | `#FFFFFF` |
|
||||
| `ink` | `--figur-ink` | `#1A1A1A` |
|
||||
| `muted` | `--figur-muted` | `#5B5B5B` |
|
||||
| `accent` | `--figur-accent` | `#2F6F9F` |
|
||||
| `rule` | `--figur-rule` | `#D9D9D9` |
|
||||
| `fontFamily` | `--figur-fontFamily` | `system-ui, -apple-system, 'Helvetica Neue', Arial, sans-serif` |
|
||||
|
||||
Missing or unparsable file → neutral defaults with a warning. Partial files are
|
||||
merged over the defaults. The user's brand is **data, never hardcoded** — figure
|
||||
sources reference `var(--figur-accent)` etc. and render correctly for any user.
|
||||
|
||||
**Authoring rule:** use the `--figur-*` variables for every color and font in a
|
||||
figure source. A hex value hardcoded in the source defeats the token seam and
|
||||
counts against the color budget.
|
||||
|
|
@ -43,7 +43,7 @@ Key takeaway: [1-sentence summary of why it's worth clicking]
|
|||
|
||||
**Example:**
|
||||
```
|
||||
Here's the Microsoft research paper I mentioned:
|
||||
Here's the research paper I mentioned:
|
||||
[URL]
|
||||
|
||||
Key takeaway: They found that AI assistants improve developer productivity by 26% — but only when the developer already understands the fundamentals.
|
||||
|
|
|
|||
|
|
@ -65,9 +65,9 @@ Optimal content type distribution: 70% Educational (teach, frameworks, how-to),
|
|||
**Used in:** `skills/linkedin-studio/SKILL.md`, `agents/content-planner.md`, `references/linkedin-growth-playbook-2025-2026.md`
|
||||
|
||||
### Content Pillars
|
||||
3-5 core expertise areas that define your LinkedIn focus. Used for topic consistency validation, gap analysis in content planning, and topic-relevance alignment checks. Example for AI content: News, Implementation, Strategy, Tools.
|
||||
3-5 core expertise areas that define your LinkedIn focus. Used for topic consistency validation, gap analysis in content planning, and topic-relevance alignment checks. Example pillar set: News, Implementation, Strategy, Tools.
|
||||
|
||||
**Used in:** `agents/content-planner.md`, `references/ai-content-framework.md`, `skills/linkedin-studio/SKILL.md`
|
||||
**Used in:** `agents/content-planner.md`, `references/content-framework.md`, `skills/linkedin-studio/SKILL.md`
|
||||
|
||||
### CTA (Call-to-Action)
|
||||
Specific, genuine engagement prompt at the end of a post. Must feel natural and offer optionality ("Which strategy has worked for your team?" > "What do you think?"). Creates invitation for the engagement that drives distribution.
|
||||
|
|
|
|||
|
|
@ -66,9 +66,9 @@ Your profile is your landing page. Optimize for the opportunities you want.
|
|||
**Structure:** [Identity] + [Value Proposition] + [Social Proof or Specificity]
|
||||
|
||||
**Examples:**
|
||||
- "AI Implementation Advisor | Helping public sector leaders deploy AI without the hype | 50+ projects delivered"
|
||||
- "Low-Code AI Architect | Building practical AI solutions | Former Microsoft, now independent"
|
||||
- "AI Strategy Consultant | Translating AI hype into business value | Speaker, Author"
|
||||
- "Supply-Chain Analytics Lead | Helping retailers cut stockouts without bloating inventory | 50+ rollouts delivered"
|
||||
- "Cloud Cost Architect | Building practical FinOps for scale-ups | Former Stripe, now independent"
|
||||
- "B2B Growth Consultant | Translating marketing spend into qualified pipeline | Speaker, Author"
|
||||
|
||||
**What to include:**
|
||||
- What you do (clearly)
|
||||
|
|
@ -105,21 +105,21 @@ Your profile is your landing page. Optimize for the opportunities you want.
|
|||
|
||||
**Example section:**
|
||||
```
|
||||
I help public sector leaders implement AI that actually works.
|
||||
I help operations leaders turn messy processes into measurable results.
|
||||
|
||||
After leading AI projects at [Organization] for 5 years, I saw the same pattern:
|
||||
organizations spending millions on AI that never delivered value. Now I help
|
||||
After leading improvement projects at [Organization] for 5 years, I saw the same pattern:
|
||||
organizations spending heavily on tools that never delivered value. Now I help
|
||||
leaders avoid those expensive mistakes.
|
||||
|
||||
What I do:
|
||||
→ AI strategy development for public sector organizations
|
||||
→ Process and workflow strategy for operations teams
|
||||
→ Vendor-neutral technology advisory
|
||||
→ Implementation oversight and quality assurance
|
||||
|
||||
Who I help:
|
||||
→ C-suite executives evaluating AI investments
|
||||
→ Department heads responsible for AI projects
|
||||
→ IT leaders managing AI implementations
|
||||
→ C-suite executives evaluating major investments
|
||||
→ Department heads responsible for transformation projects
|
||||
→ Team leaders managing rollouts
|
||||
|
||||
Track record:
|
||||
→ 50+ AI projects delivered
|
||||
|
|
|
|||
|
|
@ -84,7 +84,7 @@ Hot take: Most "AI strategies" are just PowerPoint decks.
|
|||
```
|
||||
If you could only invest in ONE AI capability this year:
|
||||
|
||||
○ Copilot for productivity
|
||||
○ Off-the-shelf AI assistants
|
||||
○ Custom AI agents
|
||||
○ Data platform modernization
|
||||
○ AI literacy training for all staff
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue