From 4d3b9f471130f12e7198f9b78643fab423e38c31 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Wed, 24 Jun 2026 13:43:02 +0200 Subject: [PATCH] =?UTF-8?q?docs(linkedin-studio):=20RE-R3a=20brief=20+=20p?= =?UTF-8?q?lan=20=E2=80=94=20persist=20relevance=20score=20+=20rank=20morn?= =?UTF-8?q?ing=20brief=20on=20it=20(light-Voyage=20hardened)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit R3 slice 1 (research-deepening): persist a 4-field TrendScore {mode, dimensions, composite, priority} on the store record (schema v2->v3, additive lossless), computed by the already-built score.ts (composite+band, one owner), threaded item->store, and rank rankForBrief on composite first + surface band+mode in renderBrief. Go-gate confirmed (operator "Go"): D1 4-field envelope · D2 composite primary within bucket · D3 first-sight only · D4 one slice · D6 mode shown per body entry. Light-Voyage: scope-guardian ALIGNED (0) / brief-reviewer PROCEED_WITH_RISKS (6 MINOR) / plan-critic REVISE (1 BLOCKER, 4 MAJOR, 4 MINOR) — all folded. Headline fold: the RED proof is now explicitly two-phase (logic-RED for store/brief/cli; stub-first then assertion-RED for score/item, since a missing named import throws at module-load under Node16 ESM, not on assertion). Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01VmHCQjJHUyWwxGAVVjNLgp --- docs/research-engine/brief-re-r3a.md | 357 +++++++++++++++++++++++++++ docs/research-engine/plan-re-r3a.md | 321 ++++++++++++++++++++++++ 2 files changed, 678 insertions(+) create mode 100644 docs/research-engine/brief-re-r3a.md create mode 100644 docs/research-engine/plan-re-r3a.md diff --git a/docs/research-engine/brief-re-r3a.md b/docs/research-engine/brief-re-r3a.md new file mode 100644 index 0000000..2bdb7b0 --- /dev/null +++ b/docs/research-engine/brief-re-r3a.md @@ -0,0 +1,357 @@ +# Brief — RE-R3a: persist the relevance score + rank the morning brief on it + +> **Slice:** RE-R3a (research-engine rung-2, R3 slice 1 — research-*deepening*). R3 ("deepen the +> research engine") is an **arc** of 5 open hulls (substrate §1: autonomous trigger · freshness-as-seen-log · +> relevance/saturation/status scoring · brief history+diff · A1–A4 fan-out). R3a takes the first: the +> **relevance** half of hull 5 (and the remainder of hull 3 — "the store schema lacks fields a brief ranks +> on"). It persists the composite relevance score the `trend-spotter` agent ALREADY computes, onto the store +> record, and makes `rankForBrief` order on it. +> **Predecessor:** RE-R1 (`score.ts`, B2 triage-scorer — built, tested, deterministic) + RE-R2a (`capture` +> bridge + `publishedAt`, schema v1→v2) + RE-R2b (`brief.ts` dated artifact + surfacing). R2b explicitly +> deferred this in its §4: *"the B2 triage scorer stays out of the brief path — its output isn't persisted on +> records yet — that's R3."* R3a is exactly that R3 step. +> **Substrate:** `docs/research-engine-concepts.local.md` §1 hull (3) (schema fields a brief ranks on: +> relevance/...) + (5) (relevance scoring) + §B2 ("scoring/filtering as a gate before expensive work — the +> output is the rank key") + §A2 ("curate/score before synthesis — the writer sees ranked material"). +> **TDD-order:** RED tests land before code — but as **two phases** (light-Voyage BLOCKER fold): the +> store/brief/cli tests are true logic-RED against the pre-edit code (they build fixtures inline, import no new +> symbol); the score/item tests reference not-yet-existing `score.ts` exports, so under Node16 ESM a missing +> named import throws at module-load (not on assertion) — they are RED against **non-throwing stubs** landed as +> the first GREEN-prep sub-step. See plan Step 1. + +## 1. Operator decision context (2026-06-24) + +The research engine is **Tier-1** (operator, 2026-06-23): *"hele min arbeidsflyt hviler på at jeg får en jevn +strøm av gode forslag til tema å skrive om."* RE-R2 made the stream **visible** (a dated morning brief surfaced +at session-start). R2b ranks that brief on **pillar-overlap + recency only** — a coarse proxy for "good topic +to write about." The actual relevance judgment (audience pull, timing, angle potential, authority, depth) lives +in the five 1–10 dimension scores the `trend-spotter` agent produces in Step 2 and pipes to the `score` CLI — +and is then **thrown away** before the trend reaches the store (Step 4.5 builds a *separate*, score-free capture +batch). R3a stops discarding it: persist the composite + band on the record, and rank the brief on composite +first. **The slice the operator chose** ("scoring inn i briefen", 2026-06-24) — the highest-leverage next step +on the core need (better-ordered suggestions), built on already-shipped-but-dormant code (`score.ts` is tested +and unused on records). The bigger R3 arcs (autonomous trigger / seen-log / saturation+status / A1–A4 fan-out) +stay later slices. + +**Go-gate resolutions — CONFIRMED (operator "Go", 2026-06-24; baked into the plan):** **D1** persist the +**4-field** `TrendScore { mode, dimensions, composite, priority }` (composite+priority to rank/display, mode to +disambiguate the instrument, dimensions for audit + lossless re-weight). **D2** composite is the **primary +within-bucket sort** (buckets still assigned by overlap+freshness; composite orders *inside* a bucket). **D3** +score is **first-sight** (set on add, never updated on re-capture — matches the store's provenance discipline; +re-score-on-recapture pairs with the R3b seen-log/status slice). **D4** ship persist+rank as **one** slice (the +operator named the visible payoff; splitting would land an invisible schema-only cut like R2a). + +## 2. The gap — grounded in code + +- **The score the agent computes never reaches the store.** `trend-spotter.md` scores each candidate's five + dimensions and pipes them to the `score` CLI (`agents/trend-spotter.md:134-140`), which returns + `{composite, band}` per candidate (`score.ts:110-122` `triage`). But Step 4.5's capture batch + (`agents/trend-spotter.md:291-298`) is **built separately and carries no score** — `source/title/url/topics/ + publishedAt/summary` only. `TrendItem` (`item.ts:22-39`) and `TrendRecord` (`types.ts:26-48`) have **no score + field**. The relevance judgment is recomputed for the digest and discarded for persistence. +- **`score.ts` is built, tested, deterministic — and unconsumed on records.** It exports `composite()` + (`score.ts:77-88`) and `band()` (`score.ts:91-97`) as pure functions, pinned to the SSOT + (`references/trend-scoring-modes.md`, by `score.test.ts:12-30` weights + the band-string assertions). Nothing + persists their output. `TrendRecord`'s own doc-comment anticipates the field: *"can gain fields (relevance + score, first-mover timing, status) in a later slice"* (`types.ts:21-23`). +- **The brief ranks on a proxy.** `rankForBrief` sorts each bucket `overlap desc → effectiveDate desc → + title asc → url asc` (`brief.ts:94-104`). Overlap (a hard pillar count) is *part* of what the composite + already weights (Pillar Fit 30 %, `trend-scoring-modes.md:43`), but the composite also captures audience/ + timing/angle/authority — signal the brief currently can't see. `brief.ts`'s own header already names this as + the next slice: *"A persisted relevance/saturation score … (R3)"* (`brief.ts:11-14`). + +## 3. Scope — what is IN (RE-R3a) + +### S-score — `scripts/trends/src/score.ts` (EDIT) +- **`export interface TrendScore { mode: ScoreMode; dimensions: DimensionScores; composite: number; priority: + Priority }`** — the persist-ready envelope. Lives in `score.ts` (the score domain owns it); `types.ts` imports + it (one-way: `score.ts` imports nothing — verified leaf, `:1-17` — so no cycle). +- **`export function requiredDimensions(mode: ScoreMode): string[]`** — `Object.keys(WEIGHTS[mode])` + (`score.ts:37-40`). **Contract: ordered** — the keys come back in the SSOT weight-literal order (kortform + `["pillar","audience","timing","angle","authority"]`, long-form `["pillar","depth","angle","authority", + "currency"]`, `score.ts:20-35`); SC1 deep-equals that ordered array, and `score.test` pins the order so a + silent SSOT reorder fails loudly. `normalizeItem` consumes it as a **set** (membership), which is order-safe + either way. `WEIGHTS` stays private; the keys are exposed via this function. +- **`export function scoreEnvelope(mode: ScoreMode, dimensions: DimensionScores): TrendScore`** — composes the + existing pure functions: `const c = composite(dimensions, mode); return { mode, dimensions, composite: c, + priority: band(c).priority }`. **No new arithmetic** — `composite()`+`band()` stay the single owners (SSOT + discipline). It throws (via `composite`, `score.ts:83`) on an out-of-range dimension — that is its + **contract**, exercised directly by SC1/SC2; on the capture path it is unreachable because `normalizeItem` + pre-validates (below). + +### S-types — `scripts/trends/src/types.ts` (EDIT) +- `import type { TrendScore } from "./score.js";` +- `TrendRecord` gains **`score?: TrendScore;`** (optional — pre-R3a records simply lack it; the `add` manual + path and unscored adopters omit it). Doc-comment updated to mark `score` as the now-realized field the + `:21-23` note anticipated. +- **`SCHEMA_VERSION = 2 → 3`** (`types.ts:62`). The bump is the only schema signal; the record shape change is + additive-optional, so the migration is the version-stamp alone (below). + +### S-store — `scripts/trends/src/store.ts` (EDIT) +- `TrendInput` (`store.ts:26-35`) gains **`score?: TrendScore;`** (imported from `score.js`). +- `addTrend` (`store.ts:120-140`): on a **new** record, persist `score` first-sight via the existing + conditional-spread idiom (`...(input.score !== undefined ? { score: input.score } : {})`, mirroring + `publishedAt`/`summary` `:134,136`). On a **duplicate**, score is **NOT** updated (D3 — first-sight, like + `source`/`capturedAt`/first `publishedAt`); topics still union (`:124-126`, unchanged). `AddResult` is + unchanged (no new flag). +- `loadStore` migrate comment (`:79-84`): extend to *"v1→v2→v3 are all purely additive-optional (an old record + is already a valid record that simply lacks the optional field), so the migration is the version stamp alone — + records pass through untouched."* **No code change** to the migration logic (`Math.max(onDisk, SCHEMA_VERSION)` + `:87` already does v2→v3 correctly; `saveStore` `JSON.stringify` `:95` preserves the `score` field — no field + stripping); only `SCHEMA_VERSION` (in `types.ts`) and the comment move. + +### S-item — `scripts/trends/src/item.ts` (EDIT) +- `TrendItem` (`item.ts:22-39`) gains **`score?: { mode: ScoreMode; dimensions: DimensionScores };`** — the + ingress envelope carries the agent's *judgment* (five scores + mode), **not** a precomputed composite (the + store computes it, so the composite has one owner). `import type { ScoreMode, DimensionScores } from + "./score.js"` + `import { requiredDimensions } from "./score.js"`. +- `normalizeItem` (`:86-119`): if `r.score` present, **validate structurally** (returns a structured error into + `errors[]`, never throws — the existing discipline, like the `publishedAt` ISO check `:99-106`): `score` is a + **non-array** object; `mode ∈ {kortform, long-form}`; `dimensions` is a **non-array** object; **each key in + `requiredDimensions(mode)` is present and a number in [1,10]**. On any failure → `errors.push("invalid score: + …")`. On success carry the **validated** `score = { mode, dimensions }` forward (the validated dimensions + object, not raw `r.score.dimensions`). Absent/null/invalid → key omitted. This guarantees the *capture path* + (`cli.ts:246-254`: `normalizeItems` → `itemToInput`) never reaches `composite` with bad dims. +- `itemToInput` (`:129-139`): if `item.score` present → add `score: scoreEnvelope(item.score.mode, + item.score.dimensions)` to the returned `TrendInput` (conditional spread, key omitted when absent). The + item→store bridge is the natural place to turn judgment into the persisted envelope. `itemToInput` is a public + function: called directly (e.g. in a test) with unvalidated dims it **throws by contract** (defense-in-depth); + the no-throw guarantee is a property of the *capture path*, not of `itemToInput` in isolation (§5). + +### S-brief — `scripts/trends/src/brief.ts` (EDIT) +- `rankForBrief` (`:72-114`): **composite becomes the primary within-bucket sort key** (D2). The comparator + (`:94-98`) gains a leading term: + `(b.trend.score?.composite ?? -1) - (a.trend.score?.composite ?? -1) || `. **Sentinel `-1`, not `-Infinity`** — composite is a weighted sum of [1,10] + dims so it is always ≥ 1.0 (min = 1×Σweights = 1.0, verified); `-1` sorts every unscored record below every + scored one and subtracts cleanly (`-Infinity - -Infinity = NaN` would corrupt the comparator). **Buckets are + UNCHANGED** — assignment stays `overlap≥2 & fresh` / `overlap==1 & fresh` / `!fresh` (`:100-104`); composite + only re-orders *within* a bucket. Total order preserved: the `(title,url)` pair is unique per store (it is the + dedupe id, `store.ts:66-68`), so the final `url asc` tie-break makes the order insertion-independent even for + equal composites. +- `renderBrief` (`:152-191`): surface the band **and mode** where a record is scored (so a reader can tell a + kortform "High" from a long-form "High" — the two are different instruments). **Pinned line shapes:** + - Top-entry meta line (`renderTopEntry`, `:135`), scored: + `- Kilde: · Publisert: (d) · () · Pillarer: ` + (the `· ()` token sits between `(d)` and `· Pillarer`); **unscored: unchanged** (no + token). + - Bullet line (`renderBulletEntry`, `:144`), scored: + `- **** — «<matched>» · <date> (<age>d) · <priority> (<mode>) · 🔗 <url>` + (token **before** `· 🔗`); **unscored: unchanged**. + - The `ranking:` frontmatter descriptor (`:160`) → the **exact** string + `composite desc, then pillar-overlap desc, then publishedAt desc (capturedAt fallback); freshDays <N>` + (pinned verbatim; `brief.test` asserts it byte-for-byte). +- `briefSummary` (`:122-130`): the top mention names the **band only** (mode stays a body-entry detail to keep + the one-line headline clean) — fresh>0 with a **scored** top → `… Topp: «<title>» (<pillar> · <priority> · + <age>d).`; fresh>0 with an **unscored** top → `… Topp: «<title>» (<pillar> · <age>d).` (no token). **Still one + line, no `"`, no `\n`** — the `extractYaml` contract (`brief.ts:118-120`) holds; the band strings + (`Immediate`/`High`/…) are bare words. +- `BRIEF_SCHEMA_VERSION` stays **1** (no frontmatter *field* added/removed; the surfacing hook still reads + `date`+`summary`; only the `ranking:` descriptor *string* and body content change). Bumping is an Open Q (§8), + not required for correctness. + +### S-cli — `scripts/trends/src/cli.ts` (EDIT, doc-only behavior) +- The `capture` branch (`:243-269`) folds via `itemToInput` (`:254`) — so once `item.ts` threads `score`, + capture **automatically** persists it with **no logic change**. Update only the header doc-comment + (`:15-21`) to note capture now persists an optional relevance score. The `score` CLI (`:218-241`, the digest + path) and the `add` manual path (`:123-147`, score-free) are unchanged. *(Capture's `{added, merged, + duplicates, errors}` tally is left unchanged — a `scored` count is an Open-Q nice-to-have, §8.)* + +### Wiring (D-default — WIRE, mirrors R2a/R2b Open Q#1) +- `agents/trend-spotter.md` (EDIT): Step 4.5's capture batch (`:291-298`) gains a per-item **`"score": {"mode": + "kortform", "dimensions": {"pillar": N, "audience": N, "timing": N, "angle": N, "authority": N}}`** — the same + five judgment scores the agent computed in Step 2 (`:134`), carried into capture so the store persists them and + the brief ranks on them. Prose explains the carry ("don't discard the Step-2 scores — fold them into the + capture batch"). Mode defaults `kortform`; `long-form` when invoked from `/linkedin:newsletter` (long-form + dims `pillar/depth/angle/authority/currency`). Domain-general (dimensions are the rubric's, pillars are the + user's config; no vendor/sector tokens). Keep the "skip silently if no deps" escape hatch. **Verified + non-vacuous:** `agents/trend-spotter.md` does NOT currently contain the literal `"dimensions"`, so the + Section 16j grep passes only after the wire is added. +- `scripts/trends/README.md` (EDIT): document the item `score` field (judgment in), the persisted `TrendScore` + (composite/priority out), and that the brief now ranks on composite. +- `scripts/test-runner.sh` (EDIT): bump `TRENDS_TESTS_FLOOR` (`:701`, currently 104) to the `tests N` line + reported after Steps 1–6, **append** `+ RE-R3a: score +N` to the inline breakdown comment (`:701`). Add + **Section 16j** ("Trends Score Wiring", RE-R3a) **after Section 16i's closing `echo ""` (~`:1171`), before the + Section 18 block (`:1173`)** (16i is the last 16x before anti-erosion; file order 17→16g→16h→16i→18, + `:947/:1014/:1078/:1122/:1173`). Mirror 16i's shape: **unconditional**, deps-absent-safe `grep -qF` + a + non-vacuity self-test emitting **one** pass/fail (so the count is exact) — (1) self-test; (2) `export + interface TrendScore` in `score.ts`; (3) `score?: TrendScore` in `types.ts`; (4) `"dimensions"` in + `agents/trend-spotter.md`; (5) `score?.composite` in `brief.ts`. **5 unconditional emitters → bump + `ASSERT_BASELINE_FLOOR` 94 → exactly 99** (`:1193`; "live recount" is the safety net, but the expected value + is the pinned 94 + 5 = 99). Update the header-enumeration **prose chain** by inserting the 16j clause between + the 16i clause (`:46-49`) and the Section-18 clause (`:49`), preserving sentence flow. + +## 4. Non-goals — what is OUT (deferred) + +- **Re-score on re-capture** (refresh the score when a trend is re-seen) — **R3b**. R3a is first-sight only + (D3). Re-score pairs naturally with the seen-log/status slice (the Timing dimension decays, so a refresh is a + real improvement — but it expands `addTrend`'s mutation surface and wants the status/lifecycle model alongside). +- **Mode-segmented / mode-normalized ranking** — OUT. R3a ranks **all** records by composite regardless of mode; + a kortform composite and a long-form composite are different instruments (different dimensions, + `trend-scoring-modes.md:50,68`), so the ranking is **mode-blind by design for R3a**. This is acceptable because + (a) almost all records are `kortform` (the default), and (b) the body entry line **shows the mode** (`<priority> + (<mode>)`) so the operator can see when two adjacent entries were scored on different instruments. A + mode-segmented brief (separate sections per mode) or a `--mode` filter is a later refinement. +- **Saturation / status (acted/skipped) / first-mover-as-a-field** (the rest of hull 5) — **R3b+**. R3a does the + **relevance** half of hull 5 only. +- **Autonomous nightly trigger** (cron/launchd, hull 1) — **R3 later**. No scheduler enters the repo. +- **Freshness as a persisted seen-log / dedup-vs-seen (B4)** — **R3 later**. +- **Brief history surfacing / diff ("yesterday vs today", hull 7)** — **R3 later**. +- **Research-deepening A1–A4** (plan → isolated parallel workers → gap loop → curate), adapter sub-agents, MCP + fetch fan-out — **R3 later** (the big slice). +- **A new `score` field in the `add` manual CLI path** — OUT. `add` stays the raw, score-free manual path; only + the normalizing `capture` path carries scores. +- **`BRIEF_SCHEMA_VERSION` bump** — OUT by default (no frontmatter field changes); Open Q#5. +- **New source file / new agent / new command** — none. R3a is all edits to the six existing `src/*.ts` + + one agent + README + gate. Counts stay 27/19/29. + +## 5. Boundaries / invariants (must hold) + +- **TDD iron law (two-phase RED):** the failing tests land **BEFORE** the implementation. `store`/`brief`/`cli` + tests are true logic-RED against the pre-edit code (inline fixtures, no new import). `score`/`item` tests + reference new `score.ts` exports → under Node16 ESM a missing named import throws at module-load, so they are + RED against **non-throwing stubs** landed first (the stubs return wrong-but-present values; the value + assertions then fail). The plan records the RED proof in two phases (Step 1); it does NOT claim a single + "all five fail on assertion before any code" run. +- **One composite owner:** `composite()` + `band()` (`score.ts`) stay the sole arithmetic; `scoreEnvelope` + *composes* them, never re-derives. The agent supplies judgment, the code computes the composite (SSOT + discipline, `score.test.ts:12-30` pins the weights/bands). +- **Purity:** `scoreEnvelope`/`requiredDimensions`/`rankForBrief`/`renderBrief` touch no fs, no clock, no env, + no AI. All fs stays at the CLI edge. +- **No throw on the capture path (not "everywhere"):** `normalizeItem` fully validates the score before + `itemToInput`, so the capture loop (`cli.ts:246-258`) never reaches `composite` with bad dims and never + crashes (a bad score → `errors[]`). `itemToInput`/`scoreEnvelope`/`composite` called **directly** with bad + dims throw by contract — that is the defense-in-depth boundary, asserted (SC2), not a leak. +- **Determinism:** same `(store, pillars, today, freshDays)` → byte-identical brief (the composite sort is a + total order via the unique `(title,url)` final tie-break; `-1` sentinel for unscored is deterministic). +- **Lossless additive migration (both directions):** a v2 store loads as v3 with records **untouched** (no + `score` invented); round-trip writes `schemaVersion: 3`; a v3 store is idempotent; a v3 store's new optional + `score` field **survives a load+resave** (no field stripping, `JSON.stringify` `store.ts:95`). Mirrors the R2a + v1→v2 proof (`store.test.ts:403-476`) + a new field-preservation case. +- **Hook unaffected:** the SessionStart surfacing reads `date`+`summary` only and **never shells out to tsx** + (analytics fresh-clone-crash invariant) — R3a touches neither the hook nor the frontmatter schema, so the + zero-tsx surfacing is unchanged. (No hook test added; the existing hook suite must still pass untouched as a + regression sanity.) +- **Domain-general:** Section 17 de-niche stays green; the `trend-spotter.md` edit carries the rubric's + dimension names + the user's pillars, **no vendor/sector tokens**. +- **No SSOT change:** `references/trend-scoring-modes.md` (weights/bands/actions) untouched; `score.ts` mirrors + it exactly as today. +- **No store-query change:** `queryByTopic`/`history`/`newestCaptureDate` untouched; the brief recomputes + overlap as before (`queryByTopic` NOT refactored). +- **Pathguard:** all edits are to **existing** files (no new `.mjs` under `hooks/scripts/`; no new `.ts` — + R3a adds *no* source file). `.gitignore` already covers `scripts/trends/{node_modules,build}`. +- **Counts** (refs/agents/commands 27/19/29) unchanged. **Recounted live at land**, never pinned/guessed. + +## 6. Success criteria (testable) + +- **SC1 (score envelope)** — `requiredDimensions("kortform")` **deep-equals (ordered)** `["pillar","audience", + "timing","angle","authority"]`; `requiredDimensions("long-form")` deep-equals `["pillar","depth","angle", + "authority","currency"]` (the `WEIGHTS` literal order, `score.ts:20-35`), and `score.test` pins the order so a + SSOT reorder fails. `scoreEnvelope("kortform", {pillar:8,audience:7,timing:9,angle:6,authority:5})` returns + `{ mode:"kortform", dimensions:<the five>, composite: composite(dims,"kortform"), priority: band(composite). + priority }` — composite/priority equal the existing functions' output byte-for-byte (one owner); a bad + dimension makes `scoreEnvelope` throw (via `composite`). +- **SC2 (item validation + bridge + the throw contract)** — `normalizeItem` on an item with a valid `score` + carries the **validated** dims; with a bad `mode`, a missing dimension, a dimension out of [1,10], a non-object + `score`, or an **array** `dimensions` → `{ ok:false, errors:["invalid score: …"] }` (structured, **never + throws**); absent `score` → key omitted. `itemToInput(validItemWithScore, capturedAt)` returns a `TrendInput` + whose `score` is `scoreEnvelope(mode, dimensions)` (composite/priority computed); without a score → no `score` + key; **`itemToInput` called directly with an out-of-range dim throws** (the defense-in-depth contract). +- **SC3 (first-sight persist)** — `addTrend(store, inputWithScore)` on a **new** title+url persists `score` on + the record; re-`addTrend` of the same title+url with a **different** score does **NOT** change the stored + score (first-sight, D3) while topics still union; an input **without** a score adds a score-free record. +- **SC4 (migration v2→v3, both directions)** — a `schemaVersion:2` store with records lacking `score` loads as + **v3**, records intact, **no `score` invented**; round-trip `loadStore→saveStore` writes `schemaVersion:3`; a + v3 store with `score` on records loads idempotent; **a v3 store's `score` field survives load+resave** (field + preservation of a new optional field — not covered by the mirrored v1→v2 block). Mirrors + `store.test.ts:403-476`, **retitled `(RE-R3a / score v2→v3)` with every `schemaVersion` assertion literal + flipped `2`→`3`.** +- **SC5 (brief ranks on composite)** — within a bucket, `rankForBrief` orders **composite desc** first + (a composite-9 record ahead of a composite-6 record at the **same overlap**); an **unscored** record sorts + **after** every scored record in its bucket (the `-1` sentinel) and then by the existing keys; buckets are + unchanged (still overlap+freshness); the order is a **total order** (same-title/diff-url, both unscored → + fixed by `url asc`); same input → byte-identical brief (determinism). +- **SC6 (render surfaces band + mode)** — `renderBrief` emits the **full pinned line shapes** (§3): a scored + top-entry shows `· <priority> (<mode>)` between `(<age>d)` and `· Pillarer`; a scored bullet shows `· + <priority> (<mode>)` before `· 🔗`; an **unscored** entry renders the **unchanged** line (no token) — both + asserted as **full lines, not substrings**. `briefSummary` names the band (no mode) on a scored top, omits the + token on an unscored top, and stays one line with no `"`/`\n` **even when the top title contains a guillemet/ + quote** (the only new code path touching the summary). The `ranking:` frontmatter descriptor equals the pinned + string verbatim. A store whose only fresh match is a **single-pillar unscored** record → `briefSummary` renders + with no `· <priority>` token, one line. +- **SC7 (CLI persists score end-to-end)** — `echo '[{…,"score":{"mode":"kortform","dimensions":{…valid…}}}]' + | … capture --store <tmp>` then `… list --store <tmp> --json` shows the record carrying a `score` with the + computed composite/priority; a batch with one **bad** score → that item in `errors[]`, the valid ones added, + **exit 0** (the run isn't failed). +- **SC8 (gate + wiring + de-niche)** — `bash scripts/test-runner.sh` → `FAIL=0`: trends suite green at the + bumped `TRENDS_TESTS_FLOOR`; new **Section 16j** green (`TrendScore` in `score.ts`, `score?: TrendScore` in + `types.ts`, `"dimensions"` in `trend-spotter.md`, `score?.composite` in `brief.ts`, non-vacuity self-test); + `ASSERT_BASELINE_FLOOR` = **99** (94 + 5); Section 17 de-niche green; counts 27/19/29. + +## 7. Verification + +**Deterministic:** `bash scripts/test-runner.sh` → `FAIL=0`; trends suite ≥ new floor; new Section 16j +self-test + greps pass; `ASSERT_BASELINE_FLOOR` = 99; Section 17 de-niche green; ref/agent/command counts +unchanged. **Regression sanity:** `node --test hooks/scripts/__tests__/` → still green untouched (R3a touches no +hook; adds no hook test). +**Behavioural (manual):** +1. `echo '[{"source":"tavily","title":"A","url":"https://e/a","topics":["ai","gov"],"publishedAt":"<~2d ago>", + "score":{"mode":"kortform","dimensions":{"pillar":9,"audience":8,"timing":9,"angle":7,"authority":6}}}, + {"source":"tavily","title":"B","url":"https://e/b","topics":["ai","gov"],"publishedAt":"<~2d ago>", + "score":{"mode":"kortform","dimensions":{"pillar":6,"audience":5,"timing":6,"angle":5,"authority":5}}}]' + | node --import tsx src/cli.ts capture --store /tmp/r3a.json` — both overlap-2 & fresh, A scored higher. +2. `node --import tsx src/cli.ts list --store /tmp/r3a.json --json` → confirm both records carry `score` + with computed composite/priority. +3. `node --import tsx src/cli.ts brief --pillars ai,gov --store /tmp/r3a.json --out /tmp/r3a-brief --json` → + confirm **A precedes B** in `topMatches` (higher composite, same overlap+freshness), the entry line shows + `· <priority> (kortform)`, and the `summary` names A with its band. +4. Append a bad-score item (`"timing":99`) to the batch and re-`capture` → confirm it lands in `errors[]`, + the valid items still added, exit 0. + +## 8. Open questions for the go-gate — RESOLVED + +D1–D4 confirmed by the operator ("Go", 2026-06-24): **D1** 4-field `TrendScore`; **D2** composite primary within +bucket; **D3** first-sight; **D4** one slice (data-then-visible commit order within it). Two residual cosmetics, +both baked to the recommended default: +- **D5 — `BRIEF_SCHEMA_VERSION` 1→2?** No (no frontmatter field added/removed; the hook reads only + `date`+`summary`). Re-open only if the artifact should self-announce the ranking change. +- **D6 — mode in the per-entry render?** YES (folded from plan-critic #3): the body entry shows `<priority> + (<mode>)`; the summary shows the band only. This makes the mode-blind ranking honest (the reader can see the + instrument). + +## 9. Light-Voyage review — folded + +Three Opus reviewers ran on the drafts, each verifying claims against live code. **scope-guardian: ALIGNED** +(every SC1–SC8 traces to a step; zero creep; all §4 non-goals held; counts 27/19/29 verified live; "no new +source file" verified — exactly 6 `src/*.ts` + 5 `tests/*.test.ts`, all edited, none added; 0 findings). +**brief-reviewer: PROCEED_WITH_RISKS** (all four load-bearing claims — score.ts-is-a-leaf/no-cycle, composite ≥ +1.0, version-stamp-only migration, single-owner arithmetic — verified TRUE; 6 MINOR). **plan-critic: REVISE** +(1 BLOCKER, 4 MAJOR, 4 MINOR). All findings folded; see `plan-re-r3a.md §Plan-critic — folded` for per-step +resolution. Headlines: + +- **[BLOCKER, folded]** the "all five test files fail on assertion after Step 1" RED claim is false for + `score`/`item` under Node16 ESM (a missing named import throws at module-load, not on assertion). → RED is now + **explicitly two-phase**: logic-RED for `store`/`brief`/`cli` against pre-edit code; stub-first then + assertion-RED for `score`/`item` (§5; plan Step 1; the header blockquote). +- **[MAJOR, folded]** the no-throw guarantee was overstated ("unreachable" — but `itemToInput` is public and + throws on direct bad-dim calls). → reworded **path-specific** (no throw on the capture path; direct calls throw + by contract); SC2 asserts both (§5, §6). +- **[MAJOR, folded]** mode-mixing was waved away and "mode shown per entry" contradicted the render spec (which + only showed priority). → the render now shows `<priority> (<mode>)` per body entry (D6); §4 states mode-blind + ranking is accepted for R3a with the mode visible; SC6 asserts the full line incl. mode. +- **[MAJOR, folded]** `requiredDimensions` order contract was ambiguous (SC1 hard-coded arrays vs membership + use). → pinned **ordered** (SC1 deep-equals the SSOT-order array; `score.test` pins order; `normalizeItem` uses + membership) (§3 S-score, SC1). +- **[MAJOR, folded]** `ASSERT_BASELINE_FLOOR` "~99" was not pinned. → pinned **99** (94 + 5 unconditional 16j + emitters; self-test emits one pass/fail like 16i) (§3 wiring, SC8). +- **[MINOR, folded]** SC4 ref `:403-471` stale + pointed at v2 assertions → `:403-476` + "flip every + `schemaVersion` literal 2→3" note (SC4). **[MINOR, folded]** R1 SSOT-pin cite was the doc-comment → now + `score.test.ts:12-30` (§2, §5; plan R1). **[MINOR, folded]** bullet `· <priority>` placement was substring-only + → full pinned line shape, priority+mode before `🔗`, asserted as a full line (§3, SC6). **[MINOR, folded]** + three diverging `ranking:` descriptor strings → one verbatim target, asserted byte-for-byte (§3, SC6). **[MINOR, + folded]** unscored single-match-top summary path untested → added as an SC6 case. **[MINOR, folded]** + `normalizeItem` non-array object case understated → "non-array" added to both object checks + SC2. **[MINOR, + folded]** header-chain line-ref tightened to the 16i clause `:46-49` / Section-18 `:49`. **[MINOR, folded]** R9 + DAG now lists the three new one-way `score.ts ←` edges. **[MINOR, folded]** SC4 forward-compat / + score-survives-round-trip added. **[MINOR, folded]** SC6 quote-safety regression (scored top title with a + guillemet) added. diff --git a/docs/research-engine/plan-re-r3a.md b/docs/research-engine/plan-re-r3a.md new file mode 100644 index 0000000..7a393ae --- /dev/null +++ b/docs/research-engine/plan-re-r3a.md @@ -0,0 +1,321 @@ +# Plan — RE-R3a: persist the relevance score + rank the morning brief on it + +> **Brief:** `docs/research-engine/brief-re-r3a.md`. **Slice:** RE-R3a (research-engine rung-2 — R3 slice 1, +> research-deepening: the relevance half of hull 5 + the hull-3 schema remainder). +> **TDD-order (two-phase RED — light-Voyage BLOCKER fold):** Step 1 records RED in two phases — +> **(A)** true logic-RED for `store`/`brief`/`cli` against the pre-edit code (inline fixtures, no new import); +> **(B)** for `score`/`item`, land non-throwing stubs for the new `score.ts` exports first (Node16 ESM throws a +> missing named import at module-load, not on assertion), then record the value-assertion RED against the stubs. +> Then GREEN: S-score envelope → S-types + S-store (first-sight persist + v2→v3 migrate) → S-item (validate + +> bridge) → S-brief (composite-sort + render band+mode) → S-cli (doc-only) → wire `trend-spotter.md` + README → +> gate floors + Section 16j → behavioural → land. +> **Counts recounted live at land, never pinned/guessed.** +> **Go-gate decisions (confirmed "Go" 2026-06-24):** D1 4-field `TrendScore` · D2 composite primary within +> bucket · D3 first-sight · D4 one slice (data-then-visible commit order) · D6 mode shown in per-entry render. +> **Light-Voyage hardened:** scope-guardian ALIGNED (0) / brief-reviewer PROCEED_WITH_RISKS (6 MINOR) / +> plan-critic REVISE (1 BLOCKER, 4 MAJOR, 4 MINOR) — all folded (see §Plan-critic — folded). + +## Goal + +Stop discarding the relevance judgment the `trend-spotter` agent already computes. Persist a 4-field +`TrendScore { mode, dimensions, composite, priority }` on the store record (schema v2→v3, additive lossless +migrate — the R2a pattern), computed deterministically from the agent's five judgment scores by the already-built +`score.ts` (`composite`+`band`, one owner). Then make `rankForBrief` order each bucket on composite first, and +`renderBrief` surface the band + mode. No re-score-on-recapture, no saturation/status, no scheduler, no new +source file — those stay later R3 slices. + +## Files touched (exhaustive — for scope-guardian) + +| File | Change | SC | +|---|---|---| +| `scripts/trends/src/score.ts` | **EDIT** — `TrendScore` interface + `requiredDimensions(mode)` (ordered) + `scoreEnvelope(mode, dimensions)` (composes the existing `composite`+`band`, no new arithmetic; throws on bad dim by contract) | SC1 | +| `scripts/trends/src/types.ts` | **EDIT** — `import type { TrendScore }`; `TrendRecord.score?: TrendScore`; `SCHEMA_VERSION` 2→3; doc-comment | SC4 | +| `scripts/trends/src/store.ts` | **EDIT** — `TrendInput.score?: TrendScore`; `addTrend` persists `score` first-sight on add (conditional spread), duplicate does NOT update; migrate comment v1→v2→v3 (logic unchanged) | SC3, SC4 | +| `scripts/trends/src/item.ts` | **EDIT** — `TrendItem.score?: {mode,dimensions}`; `normalizeItem` validates (non-array `score`/`dimensions`; mode; the mode's five dims in [1,10]) → structured error, carries validated dims; `itemToInput` carries `scoreEnvelope(...)` (throws by contract on direct bad dims) | SC2 | +| `scripts/trends/src/brief.ts` | **EDIT** — `rankForBrief` comparator: composite primary (`?? -1`), buckets unchanged; `renderBrief`/`renderBulletEntry` surface `· <priority> (<mode>)` (full pinned shapes); `briefSummary` band only; exact `ranking:` descriptor | SC5, SC6 | +| `scripts/trends/src/cli.ts` | **EDIT (doc-only behavior)** — capture persists score automatically via `itemToInput` (no logic change); header doc-comment note | SC7 | +| `scripts/trends/tests/score.test.ts` | **EDIT** — `requiredDimensions` (both modes, ordered + order pinned) + `scoreEnvelope` (composite/priority = existing funcs; bad dim throws) | SC1 | +| `scripts/trends/tests/item.test.ts` | **EDIT** — score validation (valid carried/validated dims; bad mode/missing/out-of-range/non-array/array-dims → structured error, no throw) + `itemToInput` envelope + `itemToInput` direct bad-dim throws | SC2 | +| `scripts/trends/tests/store.test.ts` | **EDIT** — first-sight persist (new persists; duplicate keeps first score, topics union; score-free add) + v2→v3 migration (lossless/idempotent + score-survives-round-trip, mirrors `:403-476` with `2`→`3`) | SC3, SC4 | +| `scripts/trends/tests/brief.test.ts` | **EDIT** — composite-primary within bucket; unscored last (`-1`); total order; full render lines (`· <priority> (<mode>)` scored / unchanged unscored); summary band; unscored single-match-top summary; quote-safe summary; exact `ranking:` string; determinism | SC5, SC6 | +| `scripts/trends/tests/cli.test.ts` | **EDIT** — capture batch with score → record carries computed composite/priority (read back via `list --json`); bad score → `errors[]`, valid added, exit 0 | SC7 | +| `agents/trend-spotter.md` | **EDIT** — Step 4.5 capture batch carries per-item `"score":{"mode":"kortform","dimensions":{…}}`; prose ("carry the Step-2 scores"); domain-general; contains literal `"dimensions"` (currently absent → grep non-vacuous) | SC8 | +| `scripts/trends/README.md` | **EDIT** — item `score` field (judgment in) + persisted `TrendScore` (out) + brief ranks on composite | — | +| `scripts/test-runner.sh` | **EDIT** — `TRENDS_TESTS_FLOOR` 104→recount + breakdown comment (`:701`); NEW unconditional **Section 16j** (after 16i `~:1171` / before 18 `:1173`); `ASSERT_BASELINE_FLOOR` 94→**99**; header-enumeration chain (16i clause `:46-49`, Section-18 clause `:49`) | SC8 | +| `docs/research-engine/{brief,plan}-re-r3a.md` | **NEW** — slice docs (TRACKED, like `docs/second-brain/*`) | — | +| `STATE.md` | **EDIT at land** — Telling-block reconcile (trends floor, ASSERT floor 99, schema v3, gate total). *Land bookkeeping, LOCAL-ONLY.* | — | + +**Not touched (scope fence):** `references/trend-scoring-modes.md` (SSOT weights/bands unchanged) · the +SessionStart hook + its tests (R3a touches no hook; no frontmatter-schema change; no new hook test) · +`queryByTopic`/`history`/`newestCaptureDate` (store query unchanged) · the `score` CLI digest path + the `add` +manual path (`cli.ts`) · `config/*` · no new `.ts`/`.mjs` file · `agents/*` count (19) · `commands/*` (29) · +`references/*` (27) · `.gitignore` (trends lines present). + +## Step 1 — (RED, two phases) failing tests across score/item/store/brief/cli + +RED discipline (R2a/R2b) + the light-Voyage BLOCKER fold: a missing **named** import throws at module-load under +Node16 ESM (`package.json:8` `node --import tsx --test`; `tsconfig.json` `module: Node16`), so `score`/`item` +(which reference new `score.ts` exports) cannot be assertion-RED before those exports exist. Split the RED proof: + +**Phase A — true logic-RED against the pre-edit code** (`store`/`brief`/`cli` build fixtures inline; `TrendStore`/ +`TrendRecord` are `import type`, erased by tsx; they import no new runtime symbol): +- `store.test.ts`: `addTrend(emptyStore, inputWithScore)` → record has `score`; a second `addTrend` (same + title+url, **different** score) → stored score **unchanged**, `added:false`, topics unioned; score-free add → + score-free record. **Migration:** a `schemaVersion:2` store with score-less records → `loadStore` gives + `schemaVersion:3`, records intact, `"score" in record === false` (not invented); round-trip writes + `schemaVersion:3`; a v3 store with `score` is idempotent; **a v3 store's `score` survives load+resave** + (read back, re-save, re-read — `score` byte-identical). *(Mirror the RE-R2a block `store.test.ts:403-476`, + retitled `(RE-R3a / score v2→v3)`, every `schemaVersion` assertion literal flipped `2`→`3`.)* +- `brief.test.ts`: two **same-overlap, same-freshness** records, composites 9 vs 6 → the 9 sorts first in its + bucket; a **scored** vs **unscored** same-bucket pair → scored first (the `-1` sentinel); a same-title/diff-url + **both-unscored** pair → fixed by `url asc` (total order intact); `renderBrief` for a scored top entry contains + the **full line** `… (<age>d) · <priority> (<mode>) · Pillarer: …`, for an unscored one the **unchanged** line + (no token) — assert full lines, not substrings; a scored bullet contains `… (<age>d) · <priority> (<mode>) · + 🔗 …`; `briefSummary` names the band on a scored top, omits it on an unscored top, and is one line with no `"` + **even when the top title contains a guillemet/quote**; a store whose only fresh match is a **single-pillar + unscored** record → summary with no `· <priority>` token; the `ranking:` line equals the pinned string + verbatim; two `renderBrief` calls byte-identical. +- `cli.test.ts`: a `capture` batch (subprocess, `--store` temp) with a valid per-item `score` → a following + `list --store <tmp> --json` shows the record's `score` with the computed composite/priority; a batch with one + bad score (`timing:99`) → JSON `errors[]` non-empty, the valid items added, **exit 0**. + +**Phase B — stub-first, then value-assertion RED** (`score`/`item` reference new exports): +- Land **non-throwing stubs** in `score.ts` so the imports resolve: `requiredDimensions → []`; `scoreEnvelope → + { mode, dimensions, composite: 0, priority: "Skip" }`. (These are the wrong-value stubs Step 2 replaces.) +- `score.test.ts`: `requiredDimensions("kortform")`/`("long-form")` deep-equal the two **ordered** five-key + lists (and a guard that the order matches `Object.keys(KORTFORM_WEIGHTS)` so a SSOT reorder fails); + `scoreEnvelope(mode, dims)` returns `{mode, dimensions, composite, priority}` with composite/priority **equal + to `composite(dims,mode)` / `band(...).priority`** (assert against the existing functions — not hard-coded — + so they share one owner); a bad dim makes `scoreEnvelope` throw (it calls `composite`). Fails on assertion + against the stubs (`[] ≠ expected`, `composite 0 ≠ real`). +- `item.test.ts`: a valid `score` → `normalizeItem` `ok:true` carrying the **validated** dims; bad `mode` / a + missing dimension / a dim `0` or `11` / a non-object `score` / an **array** `dimensions` → `ok:false` with an + `invalid score` error (**does not throw**); absent `score` → key omitted. `itemToInput(validItemWithScore, + "2026-06-24")` → `score` equals `scoreEnvelope(mode,dims)`; without score → no `score`; **`itemToInput` with an + out-of-range dim throws** (the defense-in-depth contract). Fails on assertion against the stubs. + +**RED proof (record in commit, two phases):** Phase A — `(cd scripts/trends && npm test)` before any src edit → +the `store`/`brief`/`cli` new cases fail on **assertion** (logic-RED), not module-not-found. Phase B — after the +non-throwing stubs land, the `score`/`item` cases fail on **value assertion** against the stubs. The plan does +**not** claim a single "all five fail before any code" run. + +## Step 2 — (GREEN) `score.ts` envelope + +In `scripts/trends/src/score.ts`: +- `export interface TrendScore { mode: ScoreMode; dimensions: DimensionScores; composite: number; priority: + Priority }`. +- `export function requiredDimensions(mode: ScoreMode): string[] { return Object.keys(WEIGHTS[mode]); }` — + **ordered** (insertion order of the SSOT weight literal); `score.test` pins the order. +- `export function scoreEnvelope(mode: ScoreMode, dimensions: DimensionScores): TrendScore { const c = + composite(dimensions, mode); return { mode, dimensions, composite: c, priority: band(c).priority }; }` — + composes the existing pure functions; **no new arithmetic**; throws via `composite` on a bad dim (its + contract). Replace the Phase-B stubs. Make `score.test` green. + +## Step 3 — (GREEN) `types.ts` + `store.ts` (schema v3 + first-sight persist) + +- `types.ts`: `import type { TrendScore } from "./score.js";`; add `score?: TrendScore;` to `TrendRecord` + (doc-comment marks it as the realized `:21-23` field); `SCHEMA_VERSION = 3`. +- `store.ts`: add `score?: TrendScore;` to `TrendInput` (`import type { TrendScore } from "./score.js"`); + in `addTrend`'s new-record branch add `...(input.score !== undefined ? { score: input.score } : {})` (after the + `summary` spread `:136`); the **duplicate** branch is unchanged (topics union only — score is first-sight, + D3). Extend the `loadStore` migrate comment to *"v1→v2→v3 all additive-optional … the migration is the version + stamp alone"* — **no logic change** (`Math.max(onDisk, SCHEMA_VERSION)` `:87` already handles v3; `saveStore` + `JSON.stringify` `:95` preserves `score`). Make `store.test` green (first-sight + v2→v3 + survives-round-trip). + +## Step 4 — (GREEN) `item.ts` (validate + bridge) + +- `TrendItem` gains `score?: { mode: ScoreMode; dimensions: DimensionScores };` (`import type { ScoreMode, + DimensionScores } from "./score.js"` + `import { requiredDimensions } from "./score.js"`). +- `normalizeItem`: after the `publishedAt` validation (`:99-106`), if `r.score !== undefined && r.score !== + null`, validate: `score` is a **non-array** object; `mode` ∈ `{kortform, long-form}`; `dimensions` is a + **non-array** object; **for each key in `requiredDimensions(mode)`**, the value is a number in [1,10]. On any + failure `errors.push("invalid score: <reason>")`; on success build `score = { mode, dimensions }` from the + **validated** values (not raw `r.score.dimensions`). Carried into the returned `TrendItem` via conditional + spread (key omitted when absent/invalid). **Never throws** — structured errors only (the `publishedAt` + discipline). +- `itemToInput`: add `...(item.score !== undefined ? { score: scoreEnvelope(item.score.mode, + item.score.dimensions) } : {})` (`import { scoreEnvelope } from "./score.js"`). On the **capture path** the + dims are pre-validated by `normalizeItem`, so `scoreEnvelope`→`composite` cannot throw there; called + **directly** with bad dims it throws by contract (defense-in-depth — SC2 asserts it). Make `item.test` green. + +## Step 5 — (GREEN) `brief.ts` (composite sort + render band+mode) + +- `rankForBrief` comparator (`:94-98`): prepend `(b.trend.score?.composite ?? -1) - (a.trend.score?.composite + ?? -1) ||` before the existing `b.overlap - a.overlap || …`. **Sentinel `-1`** (composite ≥ 1.0 always; `-1` + sorts unscored last and subtracts cleanly). Buckets (`:100-104`) and totals unchanged. +- `renderTopEntry` (`:132-141`): when `e.trend.score` is present, insert `· ${e.trend.score.priority} + (${e.trend.score.mode})` into the meta line **between `(${e.ageDays}d)` and `· Pillarer`** (unscored: + unchanged). `renderBulletEntry` (`:143-145`): when scored, insert `· ${score.priority} (${score.mode})` + **before `· 🔗`** (unscored: unchanged). Both shapes asserted as **full lines** in `brief.test`. +- `briefSummary` (`:122-130`): in the fresh>0 branch, when `top.trend.score` is present include `· + ${top.trend.score.priority}` in the top mention (**band only — no mode** to keep the one-line headline clean): + `… Topp: «<title>» (<pillar> · <priority> · <age>d).`; when the top is unscored omit the token: + `… Topp: «<title>» (<pillar> · <age>d).` Keep it one line, no `"`/`\n`. +- `renderBrief`'s `ranking:` line (`:160`) → the **exact pinned** string + `composite desc, then pillar-overlap desc, then publishedAt desc (capturedAt fallback); freshDays + ${ranking.freshDays}` (asserted byte-for-byte). `BRIEF_SCHEMA_VERSION` stays 1. Make `brief.test` green. + +## Step 6 — (GREEN) `cli.ts` (doc-only) + `cli.test` + +`capture` (`:243-269`) already folds through `itemToInput` (`:254`), which now carries `score` → capture +persists it with **no logic change**. Update only the header doc-comment (`:15-21`): note `capture` persists an +optional relevance score computed from the item's judgment scores. Make `cli.test`'s capture-persists-score + +bad-score-in-errors cases green. *(No `today()` exact-value assertions in `cli.test` — the wall clock is read at +the edge; composite/priority are deterministic and asserted on the read-back record.)* + +## Step 7 — wire `trend-spotter.md` (Open Q#1 default = WIRE) + README + +In `agents/trend-spotter.md` Step 4.5 (`:291-298`), extend each capture-batch item with **`"score": {"mode": +"kortform", "dimensions": {"pillar": N, "audience": N, "timing": N, "angle": N, "authority": N}}`** — the same +five judgment scores computed in Step 2 (`:134`). Add prose: don't discard the Step-2 scores; fold them into the +capture batch so the store persists the relevance assessment and the morning brief ranks on it. Mode defaults +`kortform`; `long-form` when invoked from `/linkedin:newsletter` (the long-form dims are +`pillar/depth/angle/authority/currency`, `trend-scoring-modes.md:59-65`). The replacement prose **must contain +the literal `"dimensions"`** (Section 16j `grep -qF`; verified absent today → non-vacuous). Keep the "skip +silently if no deps" escape hatch + domain-general phrasing (Section 17). Update `scripts/trends/README.md`: the +item `score` field (judgment in), the persisted `TrendScore` (composite/priority out), and that the brief now +ranks on composite. + +## Step 8 — gate: floors + new unconditional Section 16j + +In `scripts/test-runner.sh`: +- Set `TRENDS_TESTS_FLOOR` (`:701`, currently 104) to the **`tests N` line** reported by `(cd scripts/trends && + npm test)` after Steps 1–6 — recounted live, NOT additive-guessed. Stays **inside** the `if [ -x …/tsx ]` + deps guard. **Append** `+ RE-R3a: score +N` to the inline breakdown comment (`:701`) so it can't drift. +- Add **Section 16j** ("Trends Score Wiring", RE-R3a), mirroring Section 16i's shape (`:1122-1171`). **Placement + (verified):** file order is 17→16g→16h→16i→18 (`:947/:1014/:1078/:1122/:1173`), so **16i is the last section + before Section 18** — insert 16j **after 16i's closing `fi`/`echo ""` (~`:1171`), before the Section 18 block + (`:1173`)** (anti-erosion Section 18 must stay last so it counts every prior check). Five **unconditional**, + deps-absent-safe checks (pure `grep -qF`/self-test, no `tsx`), the self-test emitting **one** pass/fail like + 16i: + (1) a non-vacuity self-test (a probe carrying `score?.composite` accepted, one without rejected); + (2) `grep -qF 'export interface TrendScore' scripts/trends/src/score.ts`; + (3) `grep -qF 'score?: TrendScore' scripts/trends/src/types.ts`; + (4) `grep -qF '"dimensions"' agents/trend-spotter.md` (the capture batch carries the judgment); + (5) `grep -qF 'score?.composite' scripts/trends/src/brief.ts` (the brief ranks on it — payoff wired, not + merely doc'd). +- Bump `ASSERT_BASELINE_FLOOR` (`:1193`, currently 94) → **exactly 99** (94 + the 5 new unconditional 16j + emitters; the self-test emits one pass/fail like 16i, so 99 is deterministic — "live recount" is the safety + net, not a guess). Update the **header-enumeration prose chain** by inserting the 16j clause **between** the + 16i clause (`:46-49`) and the Section-18 clause (`:49`), preserving sentence flow (it's prose, not an append). +- **NOT touched here:** the hook suite (no `HOOK_TESTS_FLOOR` in `test-runner.sh`; R3a adds no hook test). It + must still pass untouched (`node --test hooks/scripts/__tests__/`) as a regression sanity at land. + +## Step 9 — behavioural verification + +`(cd scripts/trends && npm install)` if needed, then run brief §7's four behavioural steps (capture A>B with +scores, `list --json` to confirm persisted composite/priority, `brief --json` to confirm A precedes B + the band +in the summary + `· <priority> (kortform)` in the entry line, a bad-score item lands in `errors[]` with exit 0). +Run full `bash scripts/test-runner.sh` → `FAIL=0` (`ASSERT_BASELINE_FLOOR` 99); run `node --test +hooks/scripts/__tests__/` → still green (untouched regression). + +## Step 10 — land + +Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor 99, schema v3, +gate total). Commit order (house style): **(1)** docs commit `docs/research-engine/{brief,plan}-re-r3a.md` (no +suffix, tracked); **(2)** code commit — the six `src/*.ts` + five test files + `agents/trend-spotter.md` + +`scripts/trends/README.md` + `scripts/test-runner.sh` with `[skip-docs]`. (Per D4, the code commit MAY be split +into a data commit [score/types/store/item/cli + their tests] and a visible commit [brief + its tests + agent +wire] if the R2a/R2b two-commit rhythm is preferred.) Push freely (window lifted; gitleaks at commit; `origin` = +PUBLIC `open/` — STATE/`*.local.*` never pushed). No version bump (additive; `v0.5.2` dev). + +## Verification (testable) + +| SC | Check | Command | Expected | +|---|---|---|---| +| — | RED Phase A | `(cd scripts/trends && npm test)` before src edits | store/brief/cli new cases fail on assertion (logic-RED), not module-not-found | +| — | RED Phase B | `npm test` after non-throwing stubs | score/item cases fail on value assertion against stubs (`[] ≠ expected`, `composite 0 ≠ real`) | +| SC1 | score envelope | `npm test` (score.test) | `requiredDimensions` both modes ordered (order pinned); `scoreEnvelope` composite/priority = existing funcs; bad dim throws | +| SC2 | item validate + bridge + contract | `npm test` (item.test) | valid score carried (validated dims); bad mode/missing/out-of-range/non-array/array-dims → structured error (no throw); `itemToInput` envelope; direct bad-dim throws | +| SC3 | first-sight persist | `npm test` (store.test) | new persists score; duplicate keeps first score (topics union); score-free add works | +| SC4 | migration v2→v3 | `npm test` (store.test) | v2 loads as v3, records intact, no score invented; round-trip writes v3; v3 idempotent; score survives load+resave | +| SC5 | brief ranks on composite | `npm test` (brief.test) + manual | composite primary within bucket; unscored last (`-1`); total order; deterministic | +| SC6 | render band+mode | `npm test` (brief.test) | full lines `· <priority> (<mode>)` scored / unchanged unscored; summary band (no mode); unscored single-match summary; quote-safe; exact `ranking:` string | +| SC7 | CLI persists score | `npm test` (cli.test) + manual | capture batch w/ score → record carries composite/priority; bad score → `errors[]`, valid added, exit 0 | +| SC8 | gate + wiring + de-niche | `bash scripts/test-runner.sh` | `FAIL=0`; trends ≥ floor; Section 16j green; `ASSERT_BASELINE_FLOOR`=99; Section 17; counts 27/19/29 | + +## Risks + +- **R1 — composite/band drift from the SSOT** (someone re-implements the math in `scoreEnvelope`). *Mitigated:* + `scoreEnvelope` *composes* `composite()`+`band()` (one owner); SC1 asserts equality against those functions; + `score.test.ts:12-30` (weights) + the band-string assertions already pin them to the SSOT. +- **R2 — a bad score crashes the capture loop.** *Mitigated:* `normalizeItem` fully validates the score (mode + + the mode's five dims in [1,10], non-array objects) → structured error into `errors[]`; on the capture path + `itemToInput`→`composite` is unreachable for bad dims; SC2 + SC7 assert no-throw + `errors[]` routing. + `itemToInput` called **directly** with bad dims throws by contract (SC2) — defense-in-depth, not a leak. +- **R3 — comparator NaN from the unscored sentinel** (`-Infinity - -Infinity`). *Mitigated:* sentinel is `-1` + (composite ≥ 1.0 = min 1×Σweights); subtracts cleanly; SC5 asserts the both-unscored total order holds. +- **R4 — losing the brief's determinism** (composite ties not fully broken). *Mitigated:* the new term is a + leading tie-break; the existing `overlap → effectiveDate → title → url` chain still gives a total order (the + `(title,url)` pair is the unique dedupe id, `store.ts:66-68`); SC5 asserts byte-identical output + the + same-title/diff-url case. +- **R5 — `extractYaml` mis-reads the `summary`** if the band token introduces a `"`/newline. *Mitigated:* the + band strings (`Immediate`/`High`/…) are bare words; the summary stays one line, no `"` — asserted in SC6 even + with a guillemet/quote in the top title; the surfacing hook is untouched. +- **R6 — migration not actually lossless** (a v2 record mutated on load, or a new field stripped on resave). + *Mitigated:* the migration is the version stamp alone (logic unchanged); `saveStore` `JSON.stringify` `:95` + strips nothing; SC4 mirrors the proven R2a `store.test:403-476` and adds a score-survives-round-trip case. +- **R7 — editing `trend-spotter.md` trips the de-niche guard.** *Mitigated:* Section 17 runs in the gate; the + added `score`/`dimensions` are the rubric's generic dimension names + the user's pillars, vendor/sector-free. +- **R8 — new gate checks must survive a deps-absent fresh clone.** *Mitigated:* Section 16j is pure + `grep`/self-test on tracked source (no `tsx`) → unconditional; `TRENDS_TESTS_FLOOR` stays inside the deps guard. +- **R9 — import cycle.** *Mitigated:* `score.ts` imports nothing internal today (`:1-17`, verified leaf). R3a + adds three new **one-way** inbound edges to it — `score.ts ← types.ts` (`TrendScore` type), `score.ts ← + store.ts` (`TrendScore` type on `TrendInput`), `score.ts ← item.ts` (`ScoreMode`/`DimensionScores` types + + `requiredDimensions`/`scoreEnvelope` values). The DAG stays acyclic: `score.ts (leaf) ← {types, store, item, + brief, cli}`, since `score.ts` imports none of them back. +- **R10 — mode-mixing makes the ranking apples-to-oranges** (kortform vs long-form composites ranked together). + *Accepted for R3a, mitigated visibly:* almost all records are `kortform` (the default); the body entry line + **shows the mode** (`<priority> (<mode>)`, D6/SC6) so the operator can see when adjacent entries used different + instruments; a mode-segmented brief is a documented R3-later non-goal (brief §4). + +## Plan-critic — folded + +Three Opus reviewers ran, each verifying claims against live code: **scope-guardian ALIGNED** (0 findings; counts +27/19/29 + "no new source file" verified live); **brief-reviewer PROCEED_WITH_RISKS** (all four load-bearing +claims verified TRUE; 6 MINOR); **plan-critic REVISE** (1 BLOCKER, 4 MAJOR, 4 MINOR; score 72/C). Resolution, +each verified against live code: + +- **[BLOCKER — plan-critic] Step 1 RED-proof self-contradictory for score/item under ESM** (a missing named + import throws at module-load, not on assertion; the stub-first fix inverted the stated ordering). ✅ Step 1 is + now **explicitly two-phase**: Phase A true logic-RED for `store`/`brief`/`cli` against pre-edit code; Phase B + stub-first then value-assertion RED for `score`/`item`. The header blockquote + brief §5 + §TDD-order state it; + the "all five fail before any code" claim is removed. +- **[MAJOR — plan-critic] no-throw guarantee overstated** (`itemToInput` is public; direct bad-dim calls throw). + ✅ Step 4 + R2 + brief §5 reword it **path-specific** (no throw on the capture path because `normalizeItem` + gates it; direct calls throw by contract); SC2 adds a direct-throw assertion + a carries-validated-dims + assertion. +- **[MAJOR — plan-critic] mode-mixing waved away + "mode shown per entry" contradicted the render spec.** ✅ D6: + the render now shows `· <priority> (<mode>)` per body entry (Step 5); brief §4 adds a mode-blind-ranking + non-goal with the rationale (mode visible, mostly kortform); SC6 asserts the full line incl. mode; R10 added. +- **[MAJOR — plan-critic] `requiredDimensions` order contract ambiguous** (SC1 hard-coded arrays vs membership + use). ✅ pinned **ordered** (Step 2 + SC1 deep-equal the SSOT-order array; `score.test` pins the order; + `normalizeItem` consumes as a set). +- **[MAJOR — plan-critic] `ASSERT_BASELINE_FLOOR` "~99" not pinned.** ✅ pinned **99** (94 + 5 unconditional 16j + emitters; the self-test emits one pass/fail like 16i) — Step 8 + SC8 + brief §3. +- **[MINOR — brief-reviewer] SC4 ref `:403-471` stale + v2 assertions** → ✅ `:403-476` + "flip every + `schemaVersion` literal 2→3" note (SC4, Step 1). +- **[MINOR — brief-reviewer] R1 SSOT-pin cite was the doc-comment** (`score.ts:9-13`) → ✅ now `score.test.ts:12-30` + (R1, brief §2/§5). +- **[MINOR — brief-reviewer] bullet `· <priority>` placement substring-only** → ✅ full pinned line shape + (priority+mode before `🔗`), asserted as a full line (Step 5, SC6). +- **[MINOR — brief-reviewer] three diverging `ranking:` descriptor strings** → ✅ one verbatim target string, + asserted byte-for-byte (Step 5, SC6). +- **[MINOR — brief-reviewer] unscored single-match-top summary path untested** → ✅ added as a Phase-A + brief.test case + SC6. +- **[MINOR — brief-reviewer] `normalizeItem` non-array object case understated in the brief** → ✅ "non-array" + added to both `score` and `dimensions` object checks (Step 4, brief §3, SC2 array-dims case). +- **[MINOR — plan-critic] header-chain line-ref `:33-49` loose** → ✅ tightened to the 16i clause `:46-49` / + Section-18 clause `:49` (Step 8). +- **[MINOR — plan-critic] R9 DAG omitted the new `score.ts ←` edges** → ✅ R9 now lists all three one-way edges. +- **[MINOR — plan-critic] SC6 quote-safety regression with the new token** → ✅ SC6 asserts the summary stays + one-line/no-`"` with a scored top title containing a guillemet/quote. +- **[MINOR — plan-critic] SC4 forward-compat / score-survives-round-trip untested** → ✅ added to SC4 + Step 1 + store.test. +- **[plan-critic headless-readiness] N/A** — R3a executes **in-session, operator-driven** (driftsmodell), not as + a headless autonomous run, so per-step revert/halt clauses aren't needed (R1/R2a/R2b had none either). + +**scope-guardian — ALIGNED:** every SC1–SC8 traces to a step; zero creep; all §4 non-goals held (no +re-score-on-recapture, no saturation/status/first-mover field, no scheduler, no seen-log, no brief-diff, no +A1–A4, no mode-filter, no `score` in the `add` path, no new source file/agent/command); counts 27/19/29 verified +live; exactly 6 `src/*.ts` + 5 `tests/*.test.ts`, all edited, none added.