linkedin-studio/docs/research-engine/brief-re-r3b.md
Kjell Tore Guttormsen c40b937856 docs(linkedin-studio): RE-R3b brief + plan — trend lifecycle (re-score · status · seen-log), light-Voyage hardened
Slice (a) of the full-R3 build-out: the lifecycle layer over the trend store.
- re-score on re-capture (last-wins; R3a's explicit deferral)
- status new/acted/skipped (act/skip/reset CLI; brief excludes handled)
- seen-log surfacedCount/lastSurfacedAt (per-day idempotent, brief-recorded)

Architecture confirmed via AskUserQuestion: on-record seen-log + brief records
surfacing (rankForBrief stays pure, --no-mark dry-run) · last-score-wins ·
exclude acted/skipped. score.ts + item.ts untouched (re-score reuses the R3a
capture path); touched: types/store/brief/cli + schema v3->v4.

Light-Voyage hardened (3 Opus reviewers vs live code): scope-guardian ALIGNED;
brief-reviewer PROCEED_WITH_RISKS; plan-critic PROCEED_WITH_RISKS (78/B). All
folded — incl. the MAJOR (the brief CLI store binding hoist) + the v3->v4
migration-block reconcile (premise-verified before drafting).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011vmzxpsFpc8q19LaogAWLD
2026-06-26 00:44:47 +02:00

37 KiB

Brief — RE-R3b: trend lifecycle — re-score on re-capture · status (acted/skipped) · seen-log

Slice: RE-R3b (research-engine rung-2, R3 slice 2 — the lifecycle slice: what happens to a trend AFTER first capture). R3 ("deepen the research engine") is an arc of 5 open hulls (substrate §1). R3a took the relevance half of hull 5 (persist the score, rank on it). R3b takes the rest of the lifecycle of a trend: (i) re-score on re-capture (R3a's explicit deferral — hull 3 remainder), (ii) a status lifecycle new/acted/skipped (hull 5), and (iii) a seen-logsurfacedCount/lastSurfacedAt accumulated on each record as the temporal foundation slices (c)+(b) build on (hull 5, B4 dedup-state). Predecessor: RE-R3a (score?: TrendScore persisted first-sight; rankForBrief orders on composite; renderBrief surfaces band+mode) + RE-R2b (brief.ts dated artifact + surfacing) + RE-R2a (capture bridge). R3a §4 deferred this exactly: "Re-score on re-capture … R3b. R3a is first-sight only (D3). Re-score pairs naturally with the seen-log/status slice." — R3b is that paired slice. Substrate: docs/research-engine-concepts.local.md §1 hull (5) (status/lifecycle: acted/skipped) + remainder of (3) (status as a schema field) + §B4 ("freshness window + dedup-state (append-only seen-log → don't re-surface the same item)"). The freshness window already exists (freshDays, R2b); R3b adds the dedup-state (status as the hard dedup; surfacedCount as the soft signal). TDD-order: RED before code, two phases (light-Voyage BLOCKER fold, inherited from R3a): the re-score + migration parts of store.test, all of brief.test, and cli.test are true logic-RED against the pre-edit code (inline fixtures / behaviour change / subprocess — no new import); the setStatus/markSurfaced/ effectiveStatus tests reference not-yet-existing store.ts exports, so under Node16 ESM a missing named import throws at module-load (not on assertion) — they are RED against non-throwing stubs landed first. See plan Step 1.

1. Operator decision context (2026-06-25)

The research engine is Tier-1 (operator, 2026-06-23). R1→R3a built the deterministic spine: item-schema + triage scorer (R1) → capture bridge (R2a) → dated morning brief + surfacing (R2b) → persisted relevance score + composite ranking (R3a). What the spine still lacks is memory of a trend's life after first sight: the score is frozen at first capture even as timing decays; a trend the operator already wrote about (or deliberately passed on) re-tops tomorrow's brief unchanged; and nothing records that a trend has been surfaced N times without action. The morning brief is meant to be a work queue, but today it is amnesiac — it cannot tell a fresh unhandled signal from one the operator dealt with yesterday.

R3b closes that gap with the lifecycle layer the operator chose as slice (a) of the full-R3 build-out (2026-06-24, "ALLE gjenstående R3-slices … i rekkefølge (a) → (c) → (b) → (d) → (e)"). It is the fundament for everything temporal: the autonomous trigger (c) must never automate a loop that re-surfaces handled items — so it depends on (a)'s status+seen-log; saturation/first-mover (b) is only meaningful with accumulated seen-data — which (a) starts accumulating. R3b is deliberately first in the sequence: correctness of the lifecycle model before any automation reads it.

Architectural decisions — CONFIRMED (operator, AskUserQuestion 2026-06-25; baked into the plan):

  • A1 — seen-log form = on-record + the brief records surfacing. Three new optional fields on TrendRecord (status, surfacedCount, lastSurfacedAt); the brief CLI, after the pure rankForBrief computes the ranking, records surfacing on the rendered trends and re-saves the store. rankForBrief stays pure (mutation only at the CLI edge). The store stays the single source of truth — no separate seen-items.md. A --no-mark flag gives a side-effect-free dry run. (This is exactly what slice (c) will automate and slice (b) will read.)
  • A2 — re-score on re-capture = last-score-wins. On a duplicate capture carrying a fresh score, the stored score is replaced by the freshly-computed envelope (composite re-derived by the one owner, composite()+band()). score becomes the one deliberately-mutable field; provenance (source, capturedAt, first publishedAt) stays first-sight. A re-score does NOT reset status — an acted/skipped decision sticks. (Rationale: the Timing dimension decays, so the newer judgment — even a lower one — is the truer one; monotone "only if higher" would freeze stale optimism.)
  • A3 — acted/skipped are EXCLUDED from the brief. rankForBrief drops every record whose effective status is not new from all three buckets — the brief is a work queue, not an archive. Full history stays available via list/query.

2. The gap — grounded in code

  • The score is frozen at first sight, even as timing decays. addTrend's duplicate branch (store.ts:127-131) unions topics and returns — it never touches score (R3a's D3, first-sight only). The capture path already carries a fresh score on every re-capture (item.ts:192 itemToInputscoreEnvelope; cli.ts:257 folds it through addTrend), so the fresh judgment reaches addTrend and is silently discarded for any trend already in the store. A trend re-polled a week later still ranks on its week-old Timing score.
  • A handled trend re-tops the brief unchanged. rankForBrief (brief.ts:82-92) iterates every store record, dropping only off-pillar ones (overlap === 0, :89). There is no notion of "I already wrote about this" — an acted trend with a high composite re-sorts to the top of topMatches tomorrow exactly as it did today. TrendRecord has no status field (types.ts:29-59); the doc-comment anticipates it: "can gain fields (…, status) in a later slice" (types.ts:22).
  • Nothing records that a trend has been surfaced. The brief is a pure read (brief.ts:1-15: "No fs, no clock, no AI"); generating it leaves no trace on the store. There is no surfacedCount/lastSurfacedAt — so a future autonomous loop (slice c) has no way to know a trend was already shown, and saturation (slice b) has no accumulated signal to read. B4's dedup-state (docs/research-engine-concepts.local.md:63) does not exist yet.
  • The CLI has no lifecycle verbs. cli.ts exposes add/query/list/status/normalize/score/ capture/brief (:5-13) — all capture/read. There is no way for the operator to mark a trend acted or skipped.

3. Scope — what is IN (RE-R3b)

  • export type TrendStatus = "new" | "acted" | "skipped"; — the lifecycle states.
  • TrendRecord gains three optional fields (all absent on pre-R3b records, all additive):
    • status?: TrendStatus; — lifecycle. Absent ⇒ "new" (back-compat); set only by act/skip/reset, never on capture (a freshly-captured trend is implicitly new).
    • surfacedCount?: number; — the seen-log count: how many distinct days this trend has appeared in a generated brief. Absent ⇒ 0. Incremented (per-day-idempotent) by the brief CLI.
    • lastSurfacedAt?: string; — ISO date of the most recent surfacing. Absent ⇒ never. The per-day idempotency key (re-running today's brief does not re-increment).
  • Doc-comment: mark status/surfacedCount/lastSurfacedAt as the now-realized lifecycle fields the :22 note anticipated.
  • SCHEMA_VERSION = 3 → 4 (types.ts:73). Additive-optional; the migration is the version-stamp alone (below), identical to v1→v2→v3.
  • export function effectiveStatus(t: TrendRecord): TrendStatusreturn t.status ?? "new";. The single reader of the absent-⇒-new convention (pure; consumed by addTrend audit, brief, and the CLI). Imports TrendStatus from ./types.js (type-only).
  • Re-score in addTrend's duplicate branch (:127-131, A2): after the topic union, if input.score !== undefined and it differs from existing.score (compared via JSON.stringify — the envelope is built in a fixed key order by scoreEnvelope, so the compare is stable), set existing.score = input.score and mark the record changed. AddResult.merged is broadened to "the existing record was mutated — topics unioned and/or score refreshed"; merged is true iff either changed (a re-capture with an identical score → merged:false, no false-positive). status/surfacedCount/lastSurfacedAt are NOT touched on re-capture (A2: re-score doesn't reset status; surfacing is the brief's job, not capture's). The new-record branch (:132-144) is unchanged — a new record omits all three lifecycle fields (status absent ⇒ new; never surfaced; no input.status exists on the capture path).
  • export function setStatus(store: TrendStore, id: string, status: TrendStatus): { store: TrendStore; found: boolean } — find the record by id; if absent return { store, found: false } (no throw); else set t.status = status (set explicitly, including "new" for a reset) and return { store, found: true }. Mutates in place + returns the same store (the addTrend idiom). Pure (no fs).
  • export function markSurfaced(store: TrendStore, ids: string[], today: string): { store: TrendStore; marked: number } — for each record whose id is in ids and whose lastSurfacedAt !== today (per-day idempotent), set surfacedCount = (surfacedCount ?? 0) + 1 and lastSurfacedAt = today; count it. Records already surfaced today, or not in ids, are untouched. Pure (no fs; today injected by the caller, like capturedAt). Returns the count actually incremented.
  • AddResult keeps its 2-flag shape { store, added, merged } (no new flag — merged is broadened, not joined). TrendInput is unchanged (no status/surfaced* input — lifecycle is set post-capture, not ingested).
  • loadStore migrate comment (:82-88): extend the enumeration to "v1→v2→v3→v4 are all purely additive-optional". No code change (Math.max(onDisk, SCHEMA_VERSION) :91 already stamps v4; saveStore JSON.stringify :99 preserves the three new fields). Only SCHEMA_VERSION (in types.ts) and the comment move.
  • rankForBrief excludes handled trends (A3): in the entry loop (:82-92), add if (effectiveStatus(trend) !== "new") continue; immediately before the overlap === 0 check (so acted/skipped never enter any bucket). Import effectiveStatus from ./store.js (brief.ts already imports defaultStorePath from there — :19; the edge stays one-way, no cycle). totals.trends still counts the full inventory (store.trends.length, :116) — honest "of N in store"; totals.matched/fresh naturally reflect the post-filter entries.
  • renderBrief/renderTopEntry/renderBulletEntry surface the trend id + a surfaced marker (so the operator can act on an entry, and a re-surfaced item is honest). Pinned line shapes:
    • A shared surfacedToken(e) helper (mirrors scoreToken, :142-145): · sett <surfacedCount>x when surfacedCount >= 2, else "" (only a genuinely re-surfaced item is flagged; this is a saturation hint, not the saturation scoring of slice b). Semantic (folded — plan-critic #3): the count is PRIOR-DAY — the brief renders from surfacedCount before the CLI records today's surfacing (the mutation runs after renderBrief), so · sett Nx means "shown on N prior distinct days" (today's appearance is recorded but not yet counted in this render). The >= 2 floor therefore means "already shown on ≥2 earlier days". This is documented in the README + asserted by a unit test that sets surfacedCount directly (the cross-day behaviour is exercised by behavioural step §7).
    • Top-entry meta line (renderTopEntry, :150): append ** · \`** at the end (after Pillarer: …), and surfacedToken(e)after thescoreToken: - Kilde: · Publisert: (d) · Pillarer: · ```
    • Bullet line (renderBulletEntry, :159): append ** · \`** at the end (after 🔗 ), with surfacedTokenafterscoreToken: -