docs(linkedin-studio): RE-R3e brief + plan — brief history + day-over-day diff (hull #7), light-Voyage folded
Closes hull #7 ("ingen brief-historikk"): each morning brief records the trend ids it showed (surfaced: frontmatter, BRIEF_SCHEMA_VERSION 1->2; store schema stays 4) and renders "Nytt siden sist" against the most recent prior brief. Pure render-time diff (brief.ts stays store/fs-free; the dir+file reads live at the cli.ts edge). Zero new source/test files — all EDITs. Light-Voyage (3 Opus reviewers — scope-guardian MIXED, brief-reviewer PROCEED_WITH_RISKS, plan-critic REWORK 0.88) folded into brief #9 / plan Plan-critic. Converged on 2 MAJOR + 4 MINOR, all re-verified against live code: - MAJOR-1: brief.test.ts:574 assert.equal(BRIEF_SCHEMA_VERSION, 1) is a hard literal outside the frontmatter set -> Step 3 flips it to 2 with the bump. - MAJOR-2: cli.ts:350 const summary = briefSummary(ranking) left unthreaded -> day-2 --json.summary would lose the marker the file carries (breaks the cli.test.ts:268 one-source invariant); Step 4 threads diff -> briefSummary. - M1 import type for the BriefDiff interface; M2 SC9 rename-real-write (no hand-fixture); M3 SC1 cross-partition disjointness wording; M4 empty surfaced: contradiction reworded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011vmzxpsFpc8q19LaogAWLD
This commit is contained in:
parent
2a8459c674
commit
e0b191db0c
2 changed files with 830 additions and 0 deletions
429
docs/research-engine/brief-re-r3e.md
Normal file
429
docs/research-engine/brief-re-r3e.md
Normal file
|
|
@ -0,0 +1,429 @@
|
||||||
|
# Brief — RE-R3e: brief history + day-over-day diff (R3 slice d)
|
||||||
|
|
||||||
|
> **Slice:** RE-R3e (research-engine rung-2, R3 slice **(d)** in the operator's `(a)→(c)→(b)→(d)→(e)`
|
||||||
|
> sequence — the doc is numbered r3e by creation order, the concept is slice (d)). It closes **hull #7**
|
||||||
|
> (substrate §1 hull list: *"ingen brief-historikk"*): the dated morning brief already writes one Markdown
|
||||||
|
> file per day (`morning-brief/YYYY-MM-DD.md`), but the file is **prose-only** — nothing records *which*
|
||||||
|
> trends a brief showed in a machine-readable form, and no run reads yesterday's brief, so the engine cannot
|
||||||
|
> answer the one question a daily motor exists to answer: **"what is new since I last looked?"**
|
||||||
|
> **Predecessor:** RE-R2b (the dated brief artifact + `surfacedIds(ranking)` — the exact set a brief shows:
|
||||||
|
> `brief.ts:305`) + RE-R3b (the per-day-idempotent seen-log `surfacedCount`/`lastSurfacedAt`, which already
|
||||||
|
> records *that* a trend was surfaced but not *with which cohort*) + RE-R3c (the autonomous trigger that makes
|
||||||
|
> the dated files **accumulate day-over-day on their own** — the dependency that makes a day-over-day diff a real
|
||||||
|
> signal, not a once-in-a-while comparison) + RE-R3d (the temporal overlay — the within-brief recency/saturation
|
||||||
|
> class the diff is orthogonal to).
|
||||||
|
> **Substrate:** `docs/research-engine-concepts.local.md` §1 hull **#7** (*"ingen brief-historikk"*) + §B3
|
||||||
|
> (*"Dated-digest som flat plain-text-artefakt … diffbar, grep-bar, lenkbar … Senere sesjon laster «gårsdagens
|
||||||
|
> brief» trivielt"* — the dated file is **explicitly designed to be diffed**, R3e is the diff B3 anticipated) +
|
||||||
|
> §B4 (*"append-only seen-log → ikke re-overflate samme sak"* — R3e is the per-cohort complement: not "have I
|
||||||
|
> seen this ever" but "was this in the PRIOR brief").
|
||||||
|
> **The core decisions (operator-confirmed, AskUserQuestion 2026-06-26 — baked):**
|
||||||
|
> - **SD1 — persist membership in the brief's own frontmatter, NOT a sidecar.** Each brief writes a single
|
||||||
|
> `surfaced: <id-csv>` line into its YAML frontmatter — the ids it actually showed (`surfacedIds(ranking)`).
|
||||||
|
> This keeps **one self-describing, grep-bar artifact** per day (B3), mirrors the existing `store: { … }`
|
||||||
|
> frontmatter idiom, and is **hook-safe** (the SessionStart `extractYaml` is `^summary:`-anchored and
|
||||||
|
> line-scoped — a new `surfaced:` line cannot perturb it). The diff reads the prior brief's `surfaced:` line
|
||||||
|
> via one pure regex. **No sidecar `.json`; no second artifact.** `BRIEF_SCHEMA_VERSION` bumps **1 → 2** (the
|
||||||
|
> frontmatter gained a field — the first bump since R2b; the store's `SCHEMA_VERSION` stays **4**).
|
||||||
|
> - **SD2 — `added` with titles + `dropped` as a count; `brief.ts` stays store-free.** The diff is the symmetric
|
||||||
|
> set difference of today's surfaced ids against the prior brief's: **added** (in today, not prior — the
|
||||||
|
> headline "what's new", rendered with titles resolved from the ranking the brief already holds), **carried**
|
||||||
|
> (in both), **dropped** (in prior, not today). `added` is the value; `dropped`/`carried` render as a one-line
|
||||||
|
> tally (counts). The dropped ids are **not** resolved against the store for an acted/skipped/aged *reason* —
|
||||||
|
> that would require injecting store records into the render and is the one explicit follow-up (§4). The render
|
||||||
|
> needs **only the ranking** it already has → `brief.ts` stays pure (no store, no fs). The framing is **honest**:
|
||||||
|
> "ikke vist i dag" (not shown today), never "resolved" (which the count cannot prove).
|
||||||
|
> **TDD-order (two-phase RED, light-Voyage discipline, inherited from R3c/R3d):** Phase A — `diffSurfaced`,
|
||||||
|
> `parseSurfacedFrontmatter`, `selectPriorBriefFile` (+ the `BriefDiff` type) are NEW named exports of the
|
||||||
|
> EXISTING `brief.ts`; under Node16 ESM a missing named import throws at module-load (every `brief.test.ts` test
|
||||||
|
> would error, not assert), so land **non-throwing stubs** FIRST (`diffSurfaced → {priorDate:null,added:[],
|
||||||
|
> carried:[],dropped:[]}`, `parseSurfacedFrontmatter → []`, `selectPriorBriefFile → null`; `renderBrief` gains an
|
||||||
|
> optional `diff` param it ignores in the stub), then record value-assertion RED against them (the stubs' constant
|
||||||
|
> returns fail the diff/parse/select/section/marker assertions — true assertion-RED). Phase B — the CLI two-day
|
||||||
|
> diff test is value-RED against the existing `brief` handler (today it writes no `surfaced:` line, reads no prior
|
||||||
|
> brief, and its `--json` carries no `diff` key → the day-2 diff assertions fail). See plan Step 1.
|
||||||
|
|
||||||
|
## 1. Operator decision context (2026-06-26)
|
||||||
|
|
||||||
|
The research engine is **Tier-1** (operator, 2026-06-23). R1→R3d built the deterministic spine, the trend's life
|
||||||
|
after capture, the autonomy that runs it, and the within-brief temporal overlay: item-schema + triage (R1) →
|
||||||
|
capture bridge (R2a) → dated morning brief + surfacing (R2b) → persisted relevance + composite ranking (R3a) →
|
||||||
|
status lifecycle + seen-log + re-score (R3b) → autonomous trigger + headless entry (R3c) → temporal overlay
|
||||||
|
(first-mover + saturation, R3d). The brief now **regenerates itself every morning** (R3c) and ranks each trend by
|
||||||
|
its frozen relevance composite, its pillar overlap, and a live first-mover/saturation class (R3d). But every
|
||||||
|
morning's brief is a **standalone snapshot**: it cannot say *"these three are new since yesterday; the two you
|
||||||
|
saw yesterday are gone."* The accumulated dated files are a pile of snapshots, not a **history with a diff**.
|
||||||
|
|
||||||
|
R3e closes **hull #7** — the **brief history + day-over-day diff** — which the operator chose as slice **(d)** of
|
||||||
|
the full-R3 build-out (2026-06-24, *"ALLE gjenstående R3-slices … i rekkefølge (a) → (c) → (b) → (d) → (e)"*). It
|
||||||
|
is sequenced **after** R3c for a load-bearing reason: a day-over-day diff is only meaningful when a brief is
|
||||||
|
**produced every day on its own** — R3c's nightly trigger is what makes "yesterday's brief" reliably exist. (c)
|
||||||
|
makes (d) a real signal; without the daily trigger, "since last brief" could mean "since whenever the operator
|
||||||
|
last happened to ask."
|
||||||
|
|
||||||
|
**The concrete value — turning a pile of snapshots into a feed.** A daily motor's job is to surface the *delta*:
|
||||||
|
the operator does not want to re-read the full ranked list every morning and diff it in their head — they want
|
||||||
|
the engine to say **"3 nye siden i går"** at the top of the brief (and on the one-line SessionStart surfacing,
|
||||||
|
for free). R3e makes the dated file a genuine **history rung**: each brief records what it showed
|
||||||
|
(`surfaced:` frontmatter), and the next brief reads the most recent prior one and renders **"Nytt siden sist."**
|
||||||
|
This is the smallest honest step from *"a brief is written daily"* (R3c) to *"the brief tells me what changed"*
|
||||||
|
(the point of a feed) — and it is exactly the diffable dated-digest §B3 said the artifact was designed to be.
|
||||||
|
|
||||||
|
## 2. The gap — grounded in code
|
||||||
|
|
||||||
|
- **The dated brief is prose-only; membership is not machine-readable.** `cli.ts:341-343` writes
|
||||||
|
`morning-brief/${day}.md` from `renderBrief(ranking)`; the body embeds each trend's `id` inside a rendered
|
||||||
|
bullet (`brief.ts:242`, `:251` — `` `${e.trend.id}` ``), but there is **no structured record** of *the set a
|
||||||
|
brief showed*. `surfacedIds(ranking)` (`brief.ts:305`) computes that set and feeds it to `markSurfaced`
|
||||||
|
(`cli.ts:348`), but it is **never persisted to the artifact** — so a later run that wants "what did yesterday's
|
||||||
|
brief show" would have to scrape prose. The seen-log (`surfacedCount`/`lastSurfacedAt`, R3b) records *that* and
|
||||||
|
*how many days* a trend was surfaced, but **not which cohort it appeared with** — it cannot reconstruct
|
||||||
|
"yesterday's brief contained {A, B, C}."
|
||||||
|
- **No run reads a prior brief.** The only consumer of the dated files is the SessionStart hook's
|
||||||
|
`latestMorningBrief` (`session-start.mjs:60-77`), which reads the **single newest** file's `date`+`summary` and
|
||||||
|
surfaces it verbatim. Nothing reads the **second-newest** to compare. There is no diff, anywhere.
|
||||||
|
- **"New since last" is not derivable from the store alone.** The seen-log gives "never surfaced ever"
|
||||||
|
(`surfacedCount` absent ⇒ a first-ever sighting) — but that is **not** "new since the last brief": a trend
|
||||||
|
surfaced once three days ago, absent from yesterday's brief, reappearing today is *new to yesterday's reader*
|
||||||
|
yet has `surfacedCount 1` (not 0). Only a **per-brief membership record** (the prior `surfaced:` set) answers
|
||||||
|
"was this in the immediately-prior brief," and only it can compute **dropped** (in the prior cohort, gone
|
||||||
|
today) — which the store cannot express at all. This is precisely the gap §B3's "diffbar … dated-digest" and
|
||||||
|
§1 hull #7 name.
|
||||||
|
- **The brief's own summary cannot signal a delta.** `briefSummary` (`brief.ts:204-216`) describes today's top
|
||||||
|
match in isolation; the SessionStart surfacing (`session-start.mjs:534-536`) shows that line verbatim. There is
|
||||||
|
no "N nye siden sist" the operator could see *without opening the file* — the one number a feed leads with.
|
||||||
|
|
||||||
|
## 3. Scope — what is IN (RE-R3e)
|
||||||
|
|
||||||
|
**Zero new source/test files** (the two tracked slice docs aside). R3e is pure EDITs: the diff lives beside
|
||||||
|
`surfacedIds` in `brief.ts` (the module that already owns the brief's pure read logic — surgical, no new module
|
||||||
|
to wire into the gate), its unit tests in `brief.test.ts`, its CLI wiring (prior-file discovery) in
|
||||||
|
`cli.ts`/`cli.test.ts`, plus the wiring docs + gate.
|
||||||
|
|
||||||
|
### S-history — `scripts/trends/src/brief.ts` (EDIT) — the pure diff + the persisted membership
|
||||||
|
|
||||||
|
- **`BRIEF_SCHEMA_VERSION` bumps 1 → 2** (`brief.ts:23`) — the frontmatter gained the `surfaced:` field. (The
|
||||||
|
store's `SCHEMA_VERSION` is untouched at **4** — R3e adds **no store field**; the membership lives in the
|
||||||
|
artifact, the diff is derived.)
|
||||||
|
- **`export interface BriefDiff { priorDate: string | null; added: string[]; carried: string[]; dropped: string[] }`**
|
||||||
|
— `priorDate` = the date of the brief diffed against (`null` ⇒ no prior brief, i.e. the first ever / a fresh
|
||||||
|
data dir); `added`/`carried`/`dropped` = the three partitions of the set difference, each **order-stable**
|
||||||
|
(added/carried preserve today's `surfacedIds` order; dropped preserves the prior set's order).
|
||||||
|
- **`export function diffSurfaced(currentIds: string[], priorIds: string[], priorDate: string | null): BriefDiff`**
|
||||||
|
— pure (no clock/fs/env; both id lists + the prior date are injected by the CLI edge). `added` = `currentIds`
|
||||||
|
not in `priorIds`; `carried` = `currentIds` in `priorIds`; `dropped` = `priorIds` not in `currentIds`. Uses a
|
||||||
|
`Set` for membership; preserves input order in the output arrays. When `priorIds` is empty (first brief),
|
||||||
|
`added === currentIds` and `dropped === []`.
|
||||||
|
- **`export function parseSurfacedFrontmatter(md: string): string[]`** — pure; extracts the `surfaced:` value
|
||||||
|
from a brief's full text via a single line-anchored regex (mirrors the hook's `extractYaml` idiom:
|
||||||
|
`/^surfaced: *([^\n]*)/m`), splits on `,`, trims, drops empties. Returns `[]` when the line is **absent, blank,
|
||||||
|
or malformed** (a pre-R3e brief, or a hand-edited file) — degrades to "empty prior," never throws. Real ids are
|
||||||
|
comma-free hex (`store.ts:69-72`), so the CSV is unambiguous.
|
||||||
|
- **`export function selectPriorBriefFile(filenames: string[], today: string): string | null`** — pure; from a
|
||||||
|
directory listing, returns the **lexicographically greatest** filename matching `^\d{4}-\d{2}-\d{2}\.md$` whose
|
||||||
|
date is **strictly less than** `today` (ISO dates sort lexicographically, so string compare = date compare),
|
||||||
|
else `null`. Mirrors the hook's `latestMorningBrief` filter+sort (`session-start.mjs:63-66`) but **excludes
|
||||||
|
today and any future-dated file** — so a same-day re-run diffs against the true previous day, not its own
|
||||||
|
just-written file (the byte-determinism guarantee, SC8).
|
||||||
|
- **`renderBrief` gains an optional `diff` param** (`brief.ts:259`): `renderBrief(ranking: BriefRanking, diff?:
|
||||||
|
BriefDiff)`. Two additive emissions:
|
||||||
|
- **Frontmatter `surfaced:` line** — `surfaced: ${surfacedIds(ranking).join(",")}` inserted **before**
|
||||||
|
`schemaVersion:` (always emitted, even for an empty store → `surfaced: ` blank; this is the record the *next*
|
||||||
|
day's diff reads, independent of whether *today* had a prior). `schemaVersion:` now renders **2**.
|
||||||
|
- **A `## 🆕 Nytt siden sist` section**, placed **after the intro line and before `## 🎯 Topp-treff`** (the
|
||||||
|
delta leads, then the full ranked list). Branches (all deterministic):
|
||||||
|
- **no diff arg / `priorDate === null` with added** → `_Første brief — alt nedenfor er nytt._`
|
||||||
|
- **`priorDate === null` with no added** (empty first brief) → `_Første brief._`
|
||||||
|
- **`priorDate !== null`, `added` non-empty** → one bullet per added id, its entry resolved from the ranking
|
||||||
|
(title + matched pillars + date + link + id, reusing the bullet idiom), then a tally line
|
||||||
|
`_${carried.length} båret over, ${dropped.length} ikke vist i dag._`
|
||||||
|
- **`priorDate !== null`, `added` empty** → `_Ingenting nytt siden ${priorDate}._` (+ the same tally line)
|
||||||
|
- When `diff` is omitted (a bare `renderBrief(ranking)` call, e.g. a unit test that does not exercise the diff),
|
||||||
|
it defaults to the empty diff (`{priorDate:null,added:[],carried:[],dropped:[]}`) → the **`_Første brief._`**
|
||||||
|
section branch (`priorDate===null`, `added` empty). The `surfaced:` frontmatter line is **independent of the
|
||||||
|
diff** — always `surfacedIds(ranking).join(",")` (blank **only** for an empty store), so a non-empty ranking
|
||||||
|
still emits its real surfaced ids. (This keeps existing single-arg call sites compiling and semantically valid.)
|
||||||
|
- **`briefSummary` gains an optional `diff` param** (`brief.ts:204`): `briefSummary(ranking, diff?)`. When `diff`
|
||||||
|
is present, `priorDate !== null`, and `added.length > 0`, it appends ` ${added.length} nye siden sist.` to the
|
||||||
|
one-line headline — so the **SessionStart hook surfaces the delta for free** (it already shows the `summary:`
|
||||||
|
line verbatim; no hook edit). The marker is suppressed on the first brief (`priorDate === null`) and when
|
||||||
|
nothing is new (no noise). It carries **no double-quote and no newline** (the `^summary: *"?([^"\n]*)"?`
|
||||||
|
hook-regex invariant, `brief.ts:200-203`). `renderBrief` passes its `diff` through to `briefSummary` so the
|
||||||
|
frontmatter `summary:` and the `--json summary` agree.
|
||||||
|
|
||||||
|
### S-cli — `scripts/trends/src/cli.ts` (EDIT) — prior-brief discovery + the diff in `--json`
|
||||||
|
|
||||||
|
- **The `brief` handler discovers the prior brief and computes the diff** (between the ranking at `cli.ts:339`
|
||||||
|
and the render at `:340`): `readdirSync(outDir)` (guarded by `existsSync` — a first run has no dir) →
|
||||||
|
`selectPriorBriefFile(files, day)` → if found, `readFileSync` it and `parseSurfacedFrontmatter` → build
|
||||||
|
`diffSurfaced(surfacedIds(ranking), priorIds, priorDate)`; on any fs error, degrade to the empty-prior diff
|
||||||
|
(`priorDate: null`). Pass the diff into `renderBrief(ranking, diff)`. Adds `readdirSync` to the existing
|
||||||
|
`node:fs` import (`cli.ts:51`).
|
||||||
|
- **`--json` gains a `diff` object** (`cli.ts:352`): `diff: { priorDate, added: added.length, carried:
|
||||||
|
carried.length, dropped: dropped.length }` — counts, not id lists (the headless `run-daily.sh` collapses
|
||||||
|
`--json` to one cron-log line). The non-JSON console line (`cli.ts:355`) appends `, N nye siden sist` when
|
||||||
|
`added > 0 && priorDate !== null`.
|
||||||
|
- **`--no-mark` is unchanged in meaning** — it still governs only the **store** seen-log write (`cli.ts:347-349`).
|
||||||
|
The artifact's `surfaced:` frontmatter records what the brief showed **regardless** of `--no-mark` (it is a
|
||||||
|
property of the rendered brief, not of the store mutation). No new flag.
|
||||||
|
|
||||||
|
### Wiring (D-default — WIRE, mirrors R3a/R3b/R3c/R3d)
|
||||||
|
|
||||||
|
- `agents/trend-spotter.md` (EDIT, **prose-only, minimal**): one line — the morning brief now records which trends
|
||||||
|
it showed (frontmatter `surfaced:`) and renders a **day-over-day diff** ("Nytt siden sist") against the most
|
||||||
|
recent prior brief — no new capture step; the agent's polling/capture path is unchanged. Domain-general (no
|
||||||
|
vendor/sector token).
|
||||||
|
- `scripts/trends/README.md` (EDIT): add a `## Brief history + diff (RE-R3e)` section between the R3d temporal-
|
||||||
|
overlay section and `## Tests`: the `surfaced:` frontmatter record, the `selectPriorBriefFile` prior-discovery
|
||||||
|
(strict `< today`, same-day re-run determinism), the `diffSurfaced` partitions, the section + the summary
|
||||||
|
marker, and the `BRIEF_SCHEMA_VERSION 1→2` boundary (artifact-only; store `SCHEMA_VERSION` stays 4).
|
||||||
|
- `scripts/test-runner.sh` (EDIT): bump `TRENDS_TESTS_FLOOR` (**live `:716`**, currently **216**) to the `tests N`
|
||||||
|
line reported after the suite runs — **recounted live**, **append** `+ RE-R3e: brief +N, cli +N (brief history
|
||||||
|
+ diff)` to the inline breakdown comment. Add **Section 16n** ("Trends Brief History / Diff", RE-R3e) **between
|
||||||
|
Section 16m's closing `echo ""` and the Section 18 header** (anti-erosion must stay last). Mirror 16m's shape:
|
||||||
|
**unconditional**, deps-absent-safe (`grep -qF` + a non-vacuity self-test emitting **one** pass/fail). **6
|
||||||
|
emitters** (all on tracked source, no `tsx`, all literals **ASCII** — the section header emoji `🆕` is **never**
|
||||||
|
grepped; the shell stays ASCII-clean for bash 3.2 `set -u`): (1) self-test; (2) `export function diffSurfaced`
|
||||||
|
in `brief.ts`; (3) `parseSurfacedFrontmatter` in `brief.ts`; (4) the section header literal `Nytt siden sist`
|
||||||
|
in `brief.ts`; (5) `selectPriorBriefFile` in `cli.ts` (the diff wiring); (6) the frontmatter emit `surfaced: `
|
||||||
|
in `brief.ts`. **6 unconditional emitters → bump `ASSERT_BASELINE_FLOOR` 117 → exactly 123** (**live `:1473`**;
|
||||||
|
"live recount" is the safety net; the expected value is the pinned 117 + 6). Insert the 16n clause into the
|
||||||
|
**header-enumeration prose chain (`:53-64`)** before "…the assertion-count anti-erosion floor (SC6) in Section
|
||||||
|
18," and **append the R3e (→123) narration** to the Section-18 floor-history comment (which ends "= 117").
|
||||||
|
|
||||||
|
## 4. Non-goals — what is OUT (deferred)
|
||||||
|
|
||||||
|
- **A sidecar `.json` membership manifest** — OUT (SD1). Membership lives in the brief's own frontmatter
|
||||||
|
(`surfaced:`), keeping one self-describing artifact (B3). No second file per day.
|
||||||
|
- **`dropped` resolved to an acted/skipped/aged *reason*** — OUT (SD2), and the **one explicit follow-up**.
|
||||||
|
`dropped` renders as a **count** ("N ikke vist i dag"). Labelling *why* each dropped id left (acted/skipped via
|
||||||
|
`status`, or aged past `freshDays`) would require injecting the store records into the render — `brief.ts` would
|
||||||
|
no longer be store-free. Honest framing for R3e: "ikke vist i dag," never "resolved." A small, clearly-scoped
|
||||||
|
follow-up (CLI resolves dropped ids → `{title, status}` and passes them to a richer render) if the loop-closing
|
||||||
|
signal proves worth the coupling.
|
||||||
|
- **A browsable history INDEX file** (e.g. a rolling `history.md` of all past briefs) — OUT. The dated files +
|
||||||
|
the `surfaced:` frontmatter **are** the history (grep-bar, lenkbar — B3); an index is a presentation nicety, not
|
||||||
|
a capability gap.
|
||||||
|
- **A SessionStart hook change to render the diff** — OUT. The "N nye siden sist" marker rides the **existing**
|
||||||
|
`summary:` surfacing (`session-start.mjs:534-536`) — no hook edit, no hook test, no new frontmatter field the
|
||||||
|
hook must learn. (The hook still reads only `date`+`summary`.)
|
||||||
|
- **`schedule` / `run-daily.sh` changes** — OUT. The nightly run calls `brief` (`run-daily.sh:33`), which now
|
||||||
|
computes the diff internally → the scheduled brief gets "Nytt siden sist" **automatically**, with **no**
|
||||||
|
scheduler edit (no R3c regression surface).
|
||||||
|
- **A new store field / schema bump / store mutation for the diff** — OUT. `SCHEMA_VERSION` stays **4**;
|
||||||
|
`types.ts`/`store.ts` are **untouched**. The membership is an artifact property; the diff is derived at the CLI
|
||||||
|
edge.
|
||||||
|
- **"New" defined as first-ever-sighting (`surfacedCount === 0`)** — OUT (rejected as less correct). R3e's "new"
|
||||||
|
is **relative to the immediately-prior brief** (artifact diff), which also flags a trend *re-emerging* after a
|
||||||
|
gap — the honest meaning of "siden sist." (`surfacedCount` stays the R3d saturation input, a different
|
||||||
|
question.)
|
||||||
|
- **Diffing against an arbitrary historical brief (`--since <date>`)** — OUT. R3e diffs against the **most recent
|
||||||
|
prior** brief only (the "since last" a daily feed needs). An arbitrary baseline is a later nicety.
|
||||||
|
- **A new module / new test file** — none. The diff lives in `brief.ts`; its tests in `brief.test.ts`. (No
|
||||||
|
pathguard surface — all EDITs.)
|
||||||
|
- **New agent / new command / new reference doc** — none. R3e EDITs `brief.ts` + `cli.ts` + their tests + one
|
||||||
|
agent (prose) + README + gate. Counts stay **29/19/27**. (The SSOT `trend-scoring-modes.md` is **not** touched —
|
||||||
|
the diff is not a scoring concern; scope fence.)
|
||||||
|
|
||||||
|
## 5. Boundaries / invariants (must hold)
|
||||||
|
|
||||||
|
- **TDD iron law (two-phase RED):** failing tests land **BEFORE** implementation. Phase A — the new named exports
|
||||||
|
(`diffSurfaced`/`parseSurfacedFrontmatter`/`selectPriorBriefFile`/`BriefDiff`) need non-throwing **stubs** first
|
||||||
|
(Node16 ESM throws a missing named import at module-load), THEN value-assertion RED against the constant stubs.
|
||||||
|
Phase B — the CLI two-day diff test is value-RED against the existing handler (no `surfaced:` write, no prior
|
||||||
|
read, no `diff` in `--json` today). The plan does **not** claim a single "everything fails before any code" run.
|
||||||
|
- **`brief.ts` stays pure** (no clock, no fs, no env, no AI): `diffSurfaced`/`parseSurfacedFrontmatter`/
|
||||||
|
`selectPriorBriefFile` all take strings/arrays and return values — the directory read + file read live in
|
||||||
|
`cli.ts` (the edge), exactly like `today`/`pillars` are injected. The module's "No fs, no clock, no AI, no
|
||||||
|
network" header claim is preserved.
|
||||||
|
- **Determinism of the brief:** given `(store, pillars, today, freshDays, firstMoverDays, saturationAt, diff)` the
|
||||||
|
rendered `.md` is **byte-identical** (the diff is now an injected input, like `today`). Critically, a **same-day
|
||||||
|
re-run is byte-identical**: `selectPriorBriefFile` excludes `${today}.md` (strict `<`), so the re-run diffs
|
||||||
|
against the same previous day's brief and re-writes the same `surfaced:` line (R3c SC7 preserved).
|
||||||
|
- **`surfaced:` records the shown set, `--no-mark`-independent:** the frontmatter line is `surfacedIds(ranking)`
|
||||||
|
joined — what the brief *showed* — regardless of whether the store seen-log was written (`--no-mark` governs the
|
||||||
|
store mutation only). The artifact is always self-consistent.
|
||||||
|
- **Frozen composite + temporal overlay untouched (R3a + R3d):** R3e adds **no** `cmp` key and changes **no**
|
||||||
|
ranking — `rankForBrief` is unchanged. The diff is a **post-ranking, render-time** layer over the same surfaced
|
||||||
|
set. `score.composite` is never read for mutation; the R3d `temporal` overlay is orthogonal (it orders within
|
||||||
|
the brief; the diff compares across briefs).
|
||||||
|
- **Schema boundary:** `BRIEF_SCHEMA_VERSION` bumps **1 → 2** (the artifact's frontmatter gained `surfaced:`);
|
||||||
|
the store's `SCHEMA_VERSION` stays **4**; `types.ts`/`store.ts`/`score.ts`/`item.ts`/`schedule.ts`/
|
||||||
|
`run-daily.sh` are **untouched**.
|
||||||
|
- **Hook unaffected:** the SessionStart surfacing reads `date`+`summary` only (`session-start.mjs:60-77`). The new
|
||||||
|
`surfaced:` frontmatter line is `^surfaced:`-keyed (the `^summary:`-anchored, line-scoped `extractYaml` cannot
|
||||||
|
match it), and the `summary:` marker carries no `"`/`\n` — so the regex still captures the summary whole. R3e
|
||||||
|
touches neither the hook nor the field set the hook reads; the hook suite must still pass untouched (regression
|
||||||
|
sanity; R3e adds no hook test).
|
||||||
|
- **ASCII-only gate literals:** `scripts/test-runner.sh` must stay ASCII (a multibyte char crashes bash 3.2 under
|
||||||
|
`set -u`). The Section-16n sentinels grep the **ASCII** literals (`export function diffSurfaced`,
|
||||||
|
`parseSurfacedFrontmatter`, `Nytt siden sist`, `selectPriorBriefFile`, `surfaced: `) — **never** the `🆕` emoji
|
||||||
|
(which lives only in `brief.ts` source + rendered output, asserted by the TS tests, not by the shell gate).
|
||||||
|
- **Domain-general:** no hard-coded user/repo path, no vendor/sector token in any edit. The section header +
|
||||||
|
Norwegian copy (`Nytt siden sist`, `båret over`, `ikke vist i dag`, `nye siden sist`, `Første brief`) are
|
||||||
|
domain-general UI copy (the brief's existing language); pillars/topics remain config. Section 17 de-niche stays
|
||||||
|
green.
|
||||||
|
- **Pathguard:** R3e adds **no new files** — every change is an EDIT of an existing file (write-allowed). (No
|
||||||
|
`.mjs`-under-`hooks/scripts/` surface, no new `scripts/` file.)
|
||||||
|
- **Counts** (refs/agents/commands 27/19/29) unchanged — **recounted live at land**, never pinned/guessed.
|
||||||
|
|
||||||
|
## 6. Success criteria (testable)
|
||||||
|
|
||||||
|
- **SC1 (diffSurfaced — partitions + order + empty prior)** — `diffSurfaced(["a","b","c"], ["b","c","d"], "2026-
|
||||||
|
06-25")` → `{priorDate:"2026-06-25", added:["a"], carried:["b","c"], dropped:["d"]}` (added/carried in current
|
||||||
|
order, dropped in prior order). `diffSurfaced(["a","b"], [], null)` → `{priorDate:null, added:["a","b"],
|
||||||
|
carried:[], dropped:[]}` (empty prior ⇒ everything added). Pure: same inputs → same output. The three
|
||||||
|
partitions are **mutually disjoint** — `Set` membership is binary (an id is in `priorIds` or not), so each id
|
||||||
|
lands in exactly one of added/carried and dropped is disjoint from both; within each list, order and any
|
||||||
|
duplicates **mirror the input** (`surfacedIds` yields **distinct** ids in production, so within-list dups never
|
||||||
|
arise — the cross-partition exclusivity is the real invariant, not within-list dedup).
|
||||||
|
- **SC2 (parseSurfacedFrontmatter — read + degrade)** — parses `surfaced: 1a2b,3c4d,5e6f` (in a full frontmatter
|
||||||
|
block) → `["1a2b","3c4d","5e6f"]`; a **blank** `surfaced: ` → `[]`; an **absent** `surfaced:` line (a pre-R3e
|
||||||
|
brief) → `[]`; whitespace around ids is trimmed; the `summary:`/`store:`/`date:` lines are **not** mismatched
|
||||||
|
(line-anchored). Never throws on malformed input.
|
||||||
|
- **SC3 (selectPriorBriefFile — strict-prior selection)** — from `["2026-06-24.md","2026-06-25.md","2026-06-26.md",
|
||||||
|
"README.md","2026-06-30.md"]` with `today="2026-06-26"` → `"2026-06-25.md"` (greatest `< today`; **excludes**
|
||||||
|
today `2026-06-26.md` and the future `2026-06-30.md`; ignores the non-dated `README.md`). Empty list, or no file
|
||||||
|
`< today`, → `null`.
|
||||||
|
- **SC4 (frontmatter `surfaced:` line + round-trip)** — `renderBrief(ranking, diff)` emits exactly one
|
||||||
|
`^surfaced: <csv>$` line, equal to `surfacedIds(ranking).join(",")`, positioned before `schemaVersion: 2`; an
|
||||||
|
empty-store brief emits `surfaced: ` (blank); `parseSurfacedFrontmatter(renderBrief(r, d))` round-trips to
|
||||||
|
`surfacedIds(r)`. `schemaVersion:` renders `2`.
|
||||||
|
- **SC5 (Nytt siden sist section — all four branches)** — the rendered body contains `## 🆕 Nytt siden sist`;
|
||||||
|
with `priorDate:null` + added → `Første brief — alt nedenfor er nytt`; with `priorDate:null` + no added (empty
|
||||||
|
store) → `Første brief.`; with a prior + `added` → one bullet per added entry (its **title** present, resolved
|
||||||
|
from the ranking) + `N båret over, M ikke vist i dag`; with a prior + no added → `Ingenting nytt siden <date>`
|
||||||
|
+ the tally. The section precedes `## 🎯 Topp-treff`.
|
||||||
|
- **SC6 (summary delta marker)** — `briefSummary(ranking, diff)` with `priorDate !== null` and `added.length > 0`
|
||||||
|
ends with ` ${added.length} nye siden sist.`; with `priorDate:null` (first brief) or `added.length === 0`, the
|
||||||
|
marker is **absent** (and `briefSummary(ranking)` with no diff === `briefSummary(ranking, emptyDiff)` — no
|
||||||
|
marker, so the existing `frontmatter summary === briefSummary(r)` test stays green). The summary contains **no**
|
||||||
|
`"` and **no** `\n`.
|
||||||
|
- **SC7 (schema boundary)** — `BRIEF_SCHEMA_VERSION === 2`; `SCHEMA_VERSION === 4`; `types.ts` untouched; a
|
||||||
|
`brief` run does **not** change any record's `score.composite` (only `surfacedCount`/`lastSurfacedAt` move — the
|
||||||
|
existing R3b behaviour, since `rankForBrief`/`markSurfaced` are unchanged).
|
||||||
|
- **SC8 (determinism, incl. same-day re-run)** — two `brief` renders with the same `(store, pillars, today,
|
||||||
|
opts, diff)` → byte-identical `.md`. End-to-end via the CLI: running `brief` **twice on the same day** (the
|
||||||
|
second after the first wrote `${day}.md`) → byte-identical files, because `selectPriorBriefFile` excludes the
|
||||||
|
same-day file and picks the same prior day.
|
||||||
|
- **SC9 (CLI diff wiring — two-day sequence)** — write a day-1 brief (records `surfaced:` for its cohort), then a
|
||||||
|
day-2 brief over a store with one **new** trend: the day-2 `.md` `## 🆕 Nytt siden sist (<day-1>)` section lists
|
||||||
|
the new trend, the day-2 `--json` carries `diff: { priorDate:<day-1>, added:≥1, carried:…, dropped:… }`, and the
|
||||||
|
console line appends `N nye siden sist`. A **first** run (empty dir) → `diff.priorDate === null`. A **custom
|
||||||
|
`--out`** isolates discovery to that dir (the diff reads prior briefs only from `outDir`).
|
||||||
|
- **SC10 (gate + wiring + de-niche)** — `bash scripts/test-runner.sh` → `FAIL=0`: trends suite green at the
|
||||||
|
bumped `TRENDS_TESTS_FLOOR`; new **Section 16n** green (the six ASCII sentinels + non-vacuity self-test);
|
||||||
|
`ASSERT_BASELINE_FLOOR` = **123** (117 + 6); Section 17 de-niche green; counts 29/19/27; the hook suite still
|
||||||
|
green untouched (`node --test hooks/scripts/__tests__/*.test.mjs`).
|
||||||
|
|
||||||
|
## 7. Verification
|
||||||
|
|
||||||
|
**Deterministic:** `bash scripts/test-runner.sh` → `FAIL=0`; trends suite ≥ new floor; Section 16n self-test +
|
||||||
|
greps pass; `ASSERT_BASELINE_FLOOR` = 123; Section 17 de-niche green; ref/agent/command counts unchanged.
|
||||||
|
**Regression sanity:** `node --test hooks/scripts/__tests__/*.test.mjs` → still green untouched (R3e touches no
|
||||||
|
hook). The R3c suite (`schedule.test.ts`/`run-daily.test.ts`) still green untouched (`schedule.ts`/`run-daily.sh`
|
||||||
|
not edited).
|
||||||
|
**Behavioural (manual):**
|
||||||
|
1. `D=/tmp/r3e-mb-$$; S=/tmp/r3e-$$.json` (unique dir, no `rm`). Seed a store with two on-pillar trends and run a
|
||||||
|
day-1 brief: `node --import tsx src/cli.ts brief --pillars ai,gov --out "$D" --store "$S"` → inspect the `.md`:
|
||||||
|
the frontmatter carries `surfaced: <ids>` and `schemaVersion: 2`; the `## 🆕 Nytt siden sist` section says
|
||||||
|
`Første brief — alt nedenfor er nytt`.
|
||||||
|
2. `capture` a third on-pillar trend into the same store, then run a day-2 brief **with a later `today`** (seed via
|
||||||
|
a second dated file is not possible — use `--out "$D"` so day-1's `${day}.md` is the prior; on a real next-day
|
||||||
|
run the date advances): inspect the new `.md` → `## 🆕 Nytt siden sist (<prior-date>)` lists the new trend with
|
||||||
|
its title, then `N båret over, M ikke vist i dag`; the `--json` shows `diff.added ≥ 1`.
|
||||||
|
3. Re-run the **same-day** brief → the written `.md` is **byte-identical** (`diff` against the same prior file;
|
||||||
|
`surfaced:` re-written identically) — `cmp` the two files.
|
||||||
|
4. Confirm the seeded records' `score.composite` values are unchanged after the briefs (only `surfacedCount`/
|
||||||
|
`lastSurfacedAt` advance) — the diff never mutates the store ranking.
|
||||||
|
5. Confirm a pre-R3e brief (no `surfaced:` line) as the prior → `parseSurfacedFrontmatter` returns `[]` → every
|
||||||
|
trend reads as `added` (graceful degrade, no crash).
|
||||||
|
|
||||||
|
## 8. Open questions for the go-gate
|
||||||
|
|
||||||
|
Two architectural decisions are **CONFIRMED** (operator, AskUserQuestion 2026-06-26): **SD1** persist membership
|
||||||
|
in the brief's frontmatter (no sidecar); **SD2** `added` with titles + `dropped` as a count (`brief.ts`
|
||||||
|
store-free). Residual decisions, all baked to the recommended default — confirm or redirect with "Go":
|
||||||
|
- **D1 — "new" = relative to the immediately-prior brief (artifact diff), not first-ever (`surfacedCount 0`)?**
|
||||||
|
YES (rec). It is the honest meaning of "siden sist" and catches re-emergence; it is also what unlocks `dropped`.
|
||||||
|
Re-open only to redefine "new" as first-ever.
|
||||||
|
- **D2 — diff against the most recent prior brief only (no `--since` baseline)?** YES (rec). The "since last" a
|
||||||
|
daily feed needs. Re-open only to add an arbitrary historical baseline.
|
||||||
|
- **D3 — `## 🆕 Nytt siden sist` placed before `## 🎯 Topp-treff` (delta leads)?** YES (rec). The one thing a feed
|
||||||
|
leads with. Re-open only to place it after the ranked list (appendix) or omit the header on a first brief.
|
||||||
|
- **D4 — summary marker ` N nye siden sist.` (suppressed on first brief / when nothing new)?** YES (rec). The
|
||||||
|
delta the SessionStart hook surfaces for free, with no hook edit. Drop only to keep the summary minimal.
|
||||||
|
- **D5 — `surfaced:` frontmatter always emitted (incl. `--no-mark`, incl. empty store → blank)?** YES (rec). It is
|
||||||
|
the record the *next* diff reads; gating it on `--no-mark` or non-empty would silently break tomorrow's diff.
|
||||||
|
Re-open only to gate it.
|
||||||
|
- **D6 — `dropped`/`carried` render as a one-line count (no titles); `dropped` framed "ikke vist i dag"?** YES
|
||||||
|
(rec, = SD2). Keeps `brief.ts` store-free; honest (a count cannot prove "resolved"). Re-open only to pull the
|
||||||
|
reason-labeled follow-up into R3e now.
|
||||||
|
- **D7 — `BRIEF_SCHEMA_VERSION` 1 → 2; store `SCHEMA_VERSION` stays 4?** YES (rec). The frontmatter gained a
|
||||||
|
field (the first artifact-schema change since R2b); no store field. Re-open only to add a store field instead.
|
||||||
|
- **D8 — README gets the R3e section; `trend-spotter.md` gets one prose line; the SSOT is NOT touched?** YES
|
||||||
|
(rec). The diff is not a scoring concern — touching `trend-scoring-modes.md` would be scope creep. Re-open only
|
||||||
|
to add an SSOT note.
|
||||||
|
- **D9 — commit split?** Docs commit first, then **one** code commit (rec) — the diff (helpers + render + CLI
|
||||||
|
wiring + gate) is one coherent feature. Re-open only for a helpers-then-wiring split.
|
||||||
|
|
||||||
|
## 9. Light-Voyage review — folded
|
||||||
|
|
||||||
|
Three Opus reviewers ran COLD on this brief + the plan against live `scripts/trends/` code (scope-guardian,
|
||||||
|
brief-reviewer, plan-critic — the R3c/R3d discipline). **Verdicts:** scope-guardian **MIXED** · brief-reviewer
|
||||||
|
**PROCEED_WITH_RISKS** · plan-critic **REWORK (0.88)**. They **converged on 2 MAJOR** (both re-verified against
|
||||||
|
live code before folding) + 4 MINOR. Every line-cite, the floors, the regex/lex/hook safety, and the §3 scope
|
||||||
|
fence were **confirmed correct** by all three and left untouched.
|
||||||
|
|
||||||
|
**MAJOR-1 — a hard schema literal breaks at the 1→2 bump (the §6/Step-1 "no existing assertion breaks" scoping
|
||||||
|
missed it).** `tests/brief.test.ts:574` is `assert.equal(BRIEF_SCHEMA_VERSION, 1)` — a **hard literal**, not the
|
||||||
|
constant-tracking RegExp at `:163` (`new RegExp("\\nschemaVersion: " + BRIEF_SCHEMA_VERSION + "\\n")` auto-tracks).
|
||||||
|
It lives in the `rankForBrief — no schema/score mutation` block (`:568-577`), **outside** the frontmatter tests
|
||||||
|
§6/plan-Step-1 enumerated, so the "every frontmatter assertion auto-tracks" claim did not cover it. **Resolution:**
|
||||||
|
the GREEN schema bump (`BRIEF_SCHEMA_VERSION = 2`) must **also flip `:574` → `, 2)`** in the same step (plan
|
||||||
|
Step 3). `:575` (`assert.equal(SCHEMA_VERSION, 4)`, the store schema) is untouched. Folded into plan Step 3 + R1 +
|
||||||
|
Step 1's enumeration. (Swept live: `:574` is the *only* hard `BRIEF_SCHEMA_VERSION` literal in the suite;
|
||||||
|
`cli.test.ts:102` asserts the **store** `persisted.schemaVersion === SCHEMA_VERSION` (4) — unrelated, stays green.)
|
||||||
|
|
||||||
|
**MAJOR-2 — the `--json summary` would diverge from the file frontmatter on day-2 (the "one source" invariant).**
|
||||||
|
The file's frontmatter `summary:` is built inside `renderBrief` (`brief.ts:265`), which Step 3 routes through
|
||||||
|
`briefSummary(ranking, diff)` → on day-2 it carries the ` N nye siden sist.` marker. But the CLI's `--json`
|
||||||
|
`summary` field reads a **separate** `const summary = briefSummary(ranking)` (`cli.ts:350`, comment `// SAME source
|
||||||
|
the frontmatter carries`) that Step 4 left unthreaded → no marker. `cli.test.ts:268` asserts
|
||||||
|
`fileFrontmatter.summary === json.summary` ("one source") → would **break** on day-2. **Resolution:** Step 4
|
||||||
|
changes `cli.ts:350` to `briefSummary(ranking, diff)` (the `diff` is in scope — Step 4 computes it between the
|
||||||
|
ranking at `:339` and the render at `:340`). **Safe on day-1:** `priorDate === null` ⇒ the marker is suppressed ⇒
|
||||||
|
byte-identical to today's string. Folded into plan Step 4 + the files-touched table.
|
||||||
|
|
||||||
|
**MINOR (folded):**
|
||||||
|
- **M1 — `BriefDiff` is an interface (type-only export).** Anywhere it is referenced as a type (tests or `cli.ts`),
|
||||||
|
import it with **`import type { BriefDiff }`**, never a value import — under Node16 ESM + tsx a type-only export
|
||||||
|
is stripped from the emitted JS, so a value-import named binding fails to resolve at **module-load** (the same
|
||||||
|
Phase-A hazard as a missing named import). The plan's Step-4 code does **not** annotate `: BriefDiff` (it infers
|
||||||
|
from `diffSurfaced`'s return) and the SC tests pass object literals — so in practice no `BriefDiff` import is
|
||||||
|
needed; the rule is the guardrail if one is added. Folded into plan Step 2/4.
|
||||||
|
- **M2 — SC9's prior brief is a real renamed brief, not a hand-fixture (rename-real-write).** Replace the "pre-write
|
||||||
|
a `<prior-date>.md` fixture carrying a `surfaced:` line" mechanism (plan Step 4 note / Phase B / Step 7) with:
|
||||||
|
run `brief` once (writes `${today}.md` with a genuine `surfaced:` line), `mv ${today}.md → 2026-06-20.md` (a fixed
|
||||||
|
past date) **in the same `--out`**, then run `brief` again. This (a) closes the write→read loop **clock-free** (no
|
||||||
|
`today()` manipulation), and (b) **guarantees the prior `surfaced:` ids are real store ids** (they came from a
|
||||||
|
real run), so `carried` is non-trivial and `added` is *exactly* the newly-captured trend — a hand-fixture risks an
|
||||||
|
id mismatch that makes everything read as added/dropped (a weaker, possibly-vacuous test). Folded into plan Step 4
|
||||||
|
note, Phase B, Step 7, SC9.
|
||||||
|
- **M3 — SC1 "repeated id" wording.** `diffSurfaced` uses `currentIds.filter(...)`/`priorIds.filter(...)`, which
|
||||||
|
**preserve** within-list duplicates — so "not double-counted" is wrong as within-list dedup. What the `Set`
|
||||||
|
membership actually guarantees is **cross-partition disjointness** (added/carried/dropped are mutually exclusive).
|
||||||
|
Reworded in §6 SC1 + plan Step 1/verification. (Production `surfacedIds` yields distinct ids, so within-list dups
|
||||||
|
never arise.)
|
||||||
|
- **M4 — §3 "empty `surfaced:`" self-contradiction.** The bare `renderBrief(ranking)` default-diff prose said it
|
||||||
|
yields "an empty `surfaced:` reflecting the ranking" — contradictory: `surfaced:` is always
|
||||||
|
`surfacedIds(ranking).join(",")` (**non-empty** for a non-empty ranking; blank only for an empty store),
|
||||||
|
independent of the diff; the default empty diff only drives the **`_Første brief._`** section branch. Reworded in
|
||||||
|
§3 (the `renderBrief` default-diff bullet).
|
||||||
|
|
||||||
|
**Confirmed correct by all three — left untouched:** every line-cite (`brief.ts:23/204/265/305`,
|
||||||
|
`cli.ts:339/340/350/352`, `session-start.mjs:60-77/534-536`); the floors (`TRENDS_TESTS_FLOOR` 216 @ live `:716`,
|
||||||
|
`ASSERT_BASELINE_FLOOR` 117 @ live `:1473` → 123 = 117 + 6 unconditional 16n emitters); Section 16m is the last
|
||||||
|
trends section (16n sits between its `echo ""` and Section 18); the 6 ASCII sentinels are non-vacuous; the
|
||||||
|
`/^surfaced: *([^\n]*)/m` regex, the ISO-lex compare, the `^surfaced:` ≠ `^summary:` hook-safety, the `--json`
|
||||||
|
shape, and the same-day-determinism strict `<`.
|
||||||
401
docs/research-engine/plan-re-r3e.md
Normal file
401
docs/research-engine/plan-re-r3e.md
Normal file
|
|
@ -0,0 +1,401 @@
|
||||||
|
# Plan — RE-R3e: brief history + day-over-day diff (R3 slice d)
|
||||||
|
|
||||||
|
> **Brief:** `docs/research-engine/brief-re-r3e.md`. **Slice:** RE-R3e (research-engine rung-2 — R3 slice **(d)**:
|
||||||
|
> brief history + day-over-day diff). Closes hull **#7** (*"ingen brief-historikk"*) — each brief records the
|
||||||
|
> trends it showed (`surfaced:` frontmatter) and renders **"Nytt siden sist"** against the most recent prior
|
||||||
|
> brief. **Zero new files** — pure EDITs (the two tracked slice docs aside).
|
||||||
|
> **TDD-order (two-phase RED — light-Voyage discipline, inherited):** Step 1 records RED in two phases — **(A)**
|
||||||
|
> `diffSurfaced`/`parseSurfacedFrontmatter`/`selectPriorBriefFile` (+ `BriefDiff`) are NEW named exports of the
|
||||||
|
> EXISTING `brief.ts`; Node16 ESM throws a missing named import at module-load (every `brief.test.ts` test would
|
||||||
|
> error, not assert), so land **non-throwing stubs** first (constant returns; `renderBrief`/`briefSummary` gain an
|
||||||
|
> optional `diff` param ignored by the stub render), then record value-assertion RED against them (the constant
|
||||||
|
> stubs fail the diff/parse/select/section/marker assertions); **(B)** the CLI two-day diff test is value-RED
|
||||||
|
> against the existing `brief` handler (no `surfaced:` write, no prior read, no `diff` in `--json` today).
|
||||||
|
> Then GREEN: real `diffSurfaced`/`parseSurfacedFrontmatter`/`selectPriorBriefFile` → `surfaced:` frontmatter +
|
||||||
|
> `BRIEF_SCHEMA_VERSION` 1→2 → `## 🆕 Nytt siden sist` section → summary marker → `cli.ts` prior-discovery +
|
||||||
|
> `--json diff` → wire `trend-spotter.md` (prose) + README → gate floors + Section 16n → behavioural → land.
|
||||||
|
> **Counts recounted live at land, never pinned/guessed.**
|
||||||
|
> **Architectural decisions (CONFIRMED, AskUserQuestion 2026-06-26):** SD1 frontmatter `surfaced:` (no sidecar) ·
|
||||||
|
> SD2 `added` w/ titles + `dropped` as a count (`brief.ts` store-free). Go-gate D1–D9 baked to recommended
|
||||||
|
> defaults (brief §8).
|
||||||
|
> **Light-Voyage:** scope-guardian / brief-reviewer / plan-critic to run on these drafts; findings folded in
|
||||||
|
> §Plan-critic — folded before the code commit.
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Turn the dated morning brief from a **standalone daily snapshot** into a **history rung with a day-over-day diff**.
|
||||||
|
Each brief persists the set of trend ids it showed into its YAML frontmatter (`surfaced: <id-csv>`), bumping
|
||||||
|
`BRIEF_SCHEMA_VERSION` 1→2 (the store's `SCHEMA_VERSION` stays 4 — **no store field**). The next brief discovers
|
||||||
|
the most recent **prior** dated file (strictly `< today`), parses its `surfaced:` line, and renders a
|
||||||
|
**`## 🆕 Nytt siden sist`** section — `added` (in today, not prior — the headline, with titles resolved from the
|
||||||
|
ranking), `carried`/`dropped` as a one-line count — plus a ` N nye siden sist.` marker on the one-line summary the
|
||||||
|
SessionStart hook already surfaces. The diff is a **pure, render-time** layer: `rankForBrief` and the R3a
|
||||||
|
composite / R3d temporal overlay are **unchanged**; `brief.ts` stays **store-free and fs-free** (the directory +
|
||||||
|
file reads live at the `cli.ts` edge, injected like `today`/`pillars`). **No new agent/command/reference/module/
|
||||||
|
file**; `types.ts`/`store.ts`/`score.ts`/`item.ts`/`schedule.ts`/`run-daily.sh` + the hook untouched.
|
||||||
|
|
||||||
|
## Files touched (exhaustive — for scope-guardian)
|
||||||
|
|
||||||
|
| File | Change | SC |
|
||||||
|
|---|---|---|
|
||||||
|
| `scripts/trends/src/brief.ts` | **EDIT** — `BRIEF_SCHEMA_VERSION` 1→2; add `BriefDiff` interface + pure `diffSurfaced(currentIds, priorIds, priorDate)` + `parseSurfacedFrontmatter(md)` + `selectPriorBriefFile(filenames, today)`; `renderBrief` gains optional `diff?` → emits `surfaced:` frontmatter line (before `schemaVersion:`) + the `## 🆕 Nytt siden sist` section (before Topp-treff); `briefSummary` gains optional `diff?` → ` N nye siden sist.` marker; `renderBrief` passes `diff` through to `briefSummary` | SC1–SC8 |
|
||||||
|
| `scripts/trends/src/cli.ts` | **EDIT** — `brief` handler: `readdirSync(outDir)` (existsSync-guarded) → `selectPriorBriefFile` → `readFileSync` + `parseSurfacedFrontmatter` → `diffSurfaced(surfacedIds(ranking), priorIds, priorDate)` → `renderBrief(ranking, diff)`; thread `diff` into the shared summary at `:350` (`briefSummary(ranking, diff)` — one-source, MAJOR-2); `--json` gains `diff:{priorDate,added,carried,dropped}` (counts); console line appends `, N nye siden sist`; add `readdirSync` to the `node:fs` import | SC9 |
|
||||||
|
| `scripts/trends/tests/brief.test.ts` | **EDIT** — `diffSurfaced` unit (SC1), `parseSurfacedFrontmatter` unit (SC2), `selectPriorBriefFile` unit (SC3), `surfaced:` frontmatter + round-trip (SC4), `## 🆕 Nytt siden sist` four branches (SC5), summary marker + the `briefSummary(r)===briefSummary(r,empty)` invariant (SC6), `BRIEF_SCHEMA_VERSION===2` (SC7), determinism-with-diff (SC8). **Stubs imported in Phase A.** No existing assertion breaks (verified §Step 1) | SC1–SC8 |
|
||||||
|
| `scripts/trends/tests/cli.test.ts` | **EDIT** — subprocess two-day sequence: day-1 brief writes `surfaced:`; day-2 over a +1-trend store → `Nytt siden sist (<day1>)` lists the new trend, `--json diff.added≥1`, console `N nye siden sist`; first run → `diff.priorDate===null`; custom `--out` isolates discovery | SC9 |
|
||||||
|
| `scripts/trends/src/types.ts` · `store.ts` · `score.ts` · `item.ts` · `schedule.ts` · `run-daily.sh` | **UNTOUCHED** — no data-shape/scoring/store/scheduler change (`SCHEMA_VERSION` 4 held in `types.ts`; the R3c scheduler suite + R3d ranking stay green untouched). Listed to assert they are *not* in scope. | — |
|
||||||
|
| `hooks/scripts/session-start.mjs` + its tests | **UNTOUCHED** — the `surfaced:` frontmatter line is `^surfaced:`-keyed (the `^summary:`-anchored `extractYaml` cannot match it); the summary marker is `"`/`\n`-free. No hook edit/test; the hook suite is a land-time regression check. | — |
|
||||||
|
| `agents/trend-spotter.md` | **EDIT (prose-only, minimal)** — one line: the brief now records its shown set + renders a day-over-day diff ("Nytt siden sist"); capture path unchanged. Domain-general. | — |
|
||||||
|
| `scripts/trends/README.md` | **EDIT** — new `## Brief history + diff (RE-R3e)` between the R3d temporal-overlay section and `## Tests`: `surfaced:` record, `selectPriorBriefFile` strict-prior + same-day determinism, `diffSurfaced` partitions, section + summary marker, `BRIEF_SCHEMA_VERSION 1→2` (artifact-only) | — |
|
||||||
|
| `scripts/test-runner.sh` | **EDIT** — `TRENDS_TESTS_FLOOR` (live `:716`, 216)→recount + breakdown comment; NEW unconditional **Section 16n** between 16m's `echo ""` and Section 18; `ASSERT_BASELINE_FLOOR` (live `:1473`) 117→**123**; header-enum chain (`:53-64`) + Section-18 floor-history narration (ends "= 117") | SC10 |
|
||||||
|
| `docs/research-engine/{brief,plan}-re-r3e.md` | **NEW** — slice docs (TRACKED, like `docs/second-brain/*`) | — |
|
||||||
|
| `STATE.md` | **EDIT at land** — Telling-block reconcile (trends floor, ASSERT floor 123, gate total; `BRIEF_SCHEMA_VERSION` 1→2; store schema unchanged v4; correct stale line-cites to live `:716`/`:1473`). *Land bookkeeping, LOCAL-ONLY.* | — |
|
||||||
|
|
||||||
|
**Not touched (scope fence):** `types.ts`/`store.ts`/`score.ts`/`item.ts` (no data-shape/scoring change;
|
||||||
|
`SCHEMA_VERSION` 4) · `schedule.ts`/`run-daily.sh` + their tests (R3c untouched — the nightly run gets the diff
|
||||||
|
internally; no scheduler edit) · the SessionStart hook + its tests (R3e adds no field the hook reads; `surfaced:`
|
||||||
|
+ marker are regex-safe) · `references/trend-scoring-modes.md` (the diff is not a scoring concern) · `config/*` ·
|
||||||
|
`commands/*` (29) · `agents/*` count (19 — `trend-spotter.md` is a prose EDIT) · `references/*` count (27) ·
|
||||||
|
`.gitignore` (no new artifact — the brief files already live under the gitignored data dir). `SCHEMA_VERSION` (4).
|
||||||
|
|
||||||
|
## Step 1 — (RED, two phases) failing tests across brief/cli
|
||||||
|
|
||||||
|
**Phase A — stub-first, then value-assertion RED** (`brief.test.ts` imports the new `brief.ts` exports):
|
||||||
|
- Land **non-throwing stubs** so the static imports resolve (Node16 ESM links named imports before any test runs):
|
||||||
|
in `brief.ts` — add `export interface BriefDiff`, and
|
||||||
|
- `export function diffSurfaced(): BriefDiff { return { priorDate: null, added: [], carried: [], dropped: [] }; }`
|
||||||
|
- `export function parseSurfacedFrontmatter(): string[] { return []; }`
|
||||||
|
- `export function selectPriorBriefFile(): string | null { return null; }`
|
||||||
|
(each a constant stub ignoring its args). Add an **optional** `diff?: BriefDiff` param to `renderBrief` **and**
|
||||||
|
`briefSummary`, **wired but inert** in the stub: `renderBrief` does NOT yet emit the `surfaced:` line or the
|
||||||
|
section; `briefSummary` does NOT yet emit the marker. (Keeps the static signatures stable for the RED tests
|
||||||
|
while the *behaviour* is still absent → value-RED, not type-RED.) `BRIEF_SCHEMA_VERSION` is still **1** in Phase
|
||||||
|
A (so the SC7 `=== 2` assertion is RED).
|
||||||
|
- `brief.test.ts` (value-RED against the stubs):
|
||||||
|
- **diffSurfaced (SC1)** — `diffSurfaced(["a","b","c"],["b","c","d"],"2026-06-25")` expects
|
||||||
|
`{priorDate:"2026-06-25",added:["a"],carried:["b","c"],dropped:["d"]}` (stub returns all-empty/null → RED);
|
||||||
|
`diffSurfaced(["a","b"],[],null)` expects `{priorDate:null,added:["a","b"],carried:[],dropped:[]}`; a repeated
|
||||||
|
id not double-counted.
|
||||||
|
- **parseSurfacedFrontmatter (SC2)** — a full frontmatter string with `surfaced: 1a2b,3c4d,5e6f` → `["1a2b",
|
||||||
|
"3c4d","5e6f"]` (stub `[]` → RED); blank `surfaced: ` → `[]`; absent line → `[]`; trims whitespace; does not
|
||||||
|
match `summary:`/`store:`.
|
||||||
|
- **selectPriorBriefFile (SC3)** — `(["2026-06-24.md","2026-06-25.md","2026-06-26.md","README.md","2026-06-30.md"],
|
||||||
|
"2026-06-26")` → `"2026-06-25.md"` (stub `null` → RED); empty list / none `< today` → `null`.
|
||||||
|
- **frontmatter `surfaced:` + round-trip (SC4)** — `renderBrief(r, diff)` includes `\nsurfaced: ` + the
|
||||||
|
`surfacedIds(r).join(",")` value, before `\nschemaVersion: 2\n`; `parseSurfacedFrontmatter(renderBrief(r,d))
|
||||||
|
=== surfacedIds(r)`; empty store → `surfaced: ` blank. Stub render emits neither the line nor `schemaVersion:
|
||||||
|
2` → RED.
|
||||||
|
- **section (SC5)** — `renderBrief(r, diff)` contains `## 🆕 Nytt siden sist`; the four branches (first-brief+
|
||||||
|
added → `Første brief — alt nedenfor er nytt`; empty first → `Første brief.`; prior+added → the added title +
|
||||||
|
`båret over` + `ikke vist i dag`; prior+no-added → `Ingenting nytt siden <date>`). Stub render omits the
|
||||||
|
section → RED. **Assert the section precedes `## 🎯 Topp-treff`** (index check).
|
||||||
|
- **summary marker (SC6)** — `briefSummary(r, {priorDate:"2026-06-25",added:["x"],carried:[],dropped:[]})` ends
|
||||||
|
with ` 1 nye siden sist.` (stub omits → RED); `briefSummary(r, {priorDate:null,…})` and `briefSummary(r,
|
||||||
|
{…,added:[]})` have no marker; **`briefSummary(r) === briefSummary(r, emptyDiff)`** (the invariant that keeps
|
||||||
|
the existing `:166-171` test green); no `"`/`\n`.
|
||||||
|
- **schema (SC7)** — `BRIEF_SCHEMA_VERSION === 2` (stub still 1 → RED); `SCHEMA_VERSION === 4`.
|
||||||
|
- **determinism (SC8)** — `renderBrief(r, d) === renderBrief(r, d)`; with a fixed `d`, stable bytes.
|
||||||
|
- **Existing assertions — one hard literal flips with the bump; the rest auto-track (verified live, MAJOR-1):**
|
||||||
|
the frontmatter tests are `:158-160` (`startsWith "---\n"` — unaffected), `:161-164` (`schemaVersion: ` built
|
||||||
|
from the **imported** `BRIEF_SCHEMA_VERSION` constant via RegExp `:163` — auto-tracks 1→2; `date:`/`store:`
|
||||||
|
unaffected), `:166-173` (`summary: === briefSummary(r)` — preserved by the SC6 invariant: with no prior the
|
||||||
|
marker is suppressed, so `briefSummary(r)===briefSummary(r,emptyDiff)`), the determinism pair `:182-184` (both
|
||||||
|
sides default to the empty diff → still equal), and the `ranking:` descriptor tests (**descriptor unchanged by
|
||||||
|
R3e** → unaffected). **The one break:** `:574` `assert.equal(BRIEF_SCHEMA_VERSION, 1)` — a **hard literal** in
|
||||||
|
the `rankForBrief — no schema/score mutation` block (`:568-577`), **outside** the frontmatter set, which the
|
||||||
|
GREEN bump must flip to `, 2)` in **Step 3** (`:575` `assert.equal(SCHEMA_VERSION, 4)` stays — store schema
|
||||||
|
untouched). Swept: `:574` is the *only* hard `BRIEF_SCHEMA_VERSION` literal in the suite. The new section is
|
||||||
|
**additive**, asserted only by new tests; **no test pins the intro→Topp-treff adjacency** (verified — all body
|
||||||
|
assertions are substring/`match`).
|
||||||
|
|
||||||
|
**Phase B — subprocess value-RED against the existing CLI handler** (no new import; the handler ignores prior
|
||||||
|
briefs today):
|
||||||
|
- `cli.test.ts` — a **two-day** sequence sharing one `--out <tmp>` dir: (1) seed a store, run `brief --pillars
|
||||||
|
ai,gov --out <tmp> --store <s> --json` → today the written `.md` has **no `surfaced:` line** and the `--json`
|
||||||
|
has **no `diff` key** → RED against the day-1 assertions (`surfaced:` present, `--json.diff.priorDate === null`).
|
||||||
|
(2) `capture` one new on-pillar trend, then **rename** the day-1 `.md` to a fixed past date (`mv` it to
|
||||||
|
`2026-06-20.md` in the same `<tmp>` — the **rename-real-write** mechanism, M2) and run a second `brief` → against
|
||||||
|
the *existing* handler it reads no prior, renders no `Nytt siden sist (<date>)` section, `--json` has no `diff` →
|
||||||
|
RED. **Diff content lives in the written `.md` body**, so the assertion `readFileSync(path)` checks `## 🆕 Nytt
|
||||||
|
siden sist`; the `diff` counts are read from `--json`.
|
||||||
|
|
||||||
|
**RED proof (record in commit, two phases):** Phase A — after the non-throwing stubs land, `(cd scripts/trends &&
|
||||||
|
npm test)` fails the diff/parse/select/frontmatter/section/marker/schema cases on **value** assertions against the
|
||||||
|
constant stubs (not module-not-found). Phase B — the `cli.test` two-day cases fail on the missing `surfaced:`/
|
||||||
|
`diff`/section. The plan does **not** claim a single "everything fails before any code" run.
|
||||||
|
|
||||||
|
## Step 2 — (GREEN) `brief.ts` — the three pure helpers
|
||||||
|
|
||||||
|
Replace the Phase-A stubs with the real, pure implementations (all no clock/fs/env):
|
||||||
|
- `export interface BriefDiff { priorDate: string | null; added: string[]; carried: string[]; dropped: string[]; }`
|
||||||
|
- `diffSurfaced(currentIds, priorIds, priorDate)`:
|
||||||
|
```ts
|
||||||
|
const prior = new Set(priorIds);
|
||||||
|
const cur = new Set(currentIds);
|
||||||
|
return {
|
||||||
|
priorDate,
|
||||||
|
added: currentIds.filter((id) => !prior.has(id)),
|
||||||
|
carried: currentIds.filter((id) => prior.has(id)),
|
||||||
|
dropped: priorIds.filter((id) => !cur.has(id)),
|
||||||
|
};
|
||||||
|
```
|
||||||
|
Order-stable (filters preserve input order); empty `priorIds` ⇒ `added===currentIds`, `dropped===[]`.
|
||||||
|
- `parseSurfacedFrontmatter(md)`:
|
||||||
|
```ts
|
||||||
|
const m = md.match(/^surfaced: *([^\n]*)/m);
|
||||||
|
if (!m) return [];
|
||||||
|
return m[1].split(",").map((s) => s.trim()).filter((s) => s.length > 0);
|
||||||
|
```
|
||||||
|
Absent/blank/malformed → `[]` (mirrors the hook's `extractYaml` line-anchoring; never throws).
|
||||||
|
- `selectPriorBriefFile(filenames, today)`:
|
||||||
|
```ts
|
||||||
|
const todayFile = `${today}.md`;
|
||||||
|
return (
|
||||||
|
filenames
|
||||||
|
.filter((f) => /^\d{4}-\d{2}-\d{2}\.md$/.test(f) && f < todayFile)
|
||||||
|
.sort()
|
||||||
|
.pop() ?? null
|
||||||
|
);
|
||||||
|
```
|
||||||
|
ISO dates sort lexicographically, so `f < todayFile` = date `< today` (strict — excludes today + future);
|
||||||
|
greatest remaining = the most recent prior. Mirrors `session-start.mjs:63-66`, minus today.
|
||||||
|
Make the Phase-A diff/parse/select cases green.
|
||||||
|
|
||||||
|
## Step 3 — (GREEN) `brief.ts` — frontmatter `surfaced:` + schema bump + the section + the marker
|
||||||
|
|
||||||
|
- **`BRIEF_SCHEMA_VERSION = 2`** (`brief.ts:23`) — **and flip the one hard test literal in the same step**
|
||||||
|
(MAJOR-1): `tests/brief.test.ts:574` `assert.equal(BRIEF_SCHEMA_VERSION, 1)` → `assert.equal(BRIEF_SCHEMA_VERSION,
|
||||||
|
2)`. (The RegExp at `:163` and the new SC7 already track the constant; `:575` `assert.equal(SCHEMA_VERSION, 4)`
|
||||||
|
is untouched.)
|
||||||
|
- **`renderBrief(ranking, diff: BriefDiff = { priorDate: null, added: [], carried: [], dropped: [] })`** — the
|
||||||
|
default empty diff keeps single-arg call sites valid. Two additive emissions:
|
||||||
|
- In the frontmatter block (`brief.ts:263-269`), insert **before** the `schemaVersion:` line:
|
||||||
|
`lines.push(\`surfaced: ${surfacedIds(ranking).join(",")}\`);` (empty store ⇒ `surfaced: ` blank). The
|
||||||
|
`schemaVersion:` line now renders `2` via the bumped constant.
|
||||||
|
- After the intro line (`brief.ts:273-276`) and **before** `## 🎯 Topp-treff` (`:278`), emit the section:
|
||||||
|
```ts
|
||||||
|
lines.push("## 🆕 Nytt siden sist");
|
||||||
|
if (diff.priorDate === null) {
|
||||||
|
lines.push(diff.added.length > 0 ? "_Første brief — alt nedenfor er nytt._" : "_Første brief._", "");
|
||||||
|
} else if (diff.added.length === 0) {
|
||||||
|
lines.push(`_Ingenting nytt siden ${diff.priorDate}._`,
|
||||||
|
`_${diff.carried.length} båret over, ${diff.dropped.length} ikke vist i dag._`, "");
|
||||||
|
} else {
|
||||||
|
const byId = new Map(
|
||||||
|
[...ranking.topMatches, ...ranking.singleMatches, ...ranking.olderMatched].map((e) => [e.trend.id, e]),
|
||||||
|
);
|
||||||
|
for (const id of diff.added) {
|
||||||
|
const e = byId.get(id);
|
||||||
|
if (e) lines.push(renderBulletEntry(e));
|
||||||
|
}
|
||||||
|
lines.push(`_${diff.carried.length} båret over, ${diff.dropped.length} ikke vist i dag._`, "");
|
||||||
|
}
|
||||||
|
```
|
||||||
|
(`renderBulletEntry` is the existing bullet renderer — reused, no new format.) An added id always resolves
|
||||||
|
(added ⊆ surfacedIds ⊆ ranking entries); the `if (e)` guard keeps it total.
|
||||||
|
- Pass the diff through: `briefSummary(ranking, diff)` at the frontmatter `summary:` line (`brief.ts:265`).
|
||||||
|
- **`briefSummary(ranking, diff?: BriefDiff)`** (`brief.ts:204`): after building the headline, append the marker:
|
||||||
|
```ts
|
||||||
|
const delta = diff && diff.priorDate !== null && diff.added.length > 0
|
||||||
|
? ` ${diff.added.length} nye siden sist.` : "";
|
||||||
|
return `${...existing headline...}${delta}`;
|
||||||
|
```
|
||||||
|
Suppressed on the first brief / when nothing new; carries no `"`/`\n` (the existing summary already guarantees
|
||||||
|
this — the marker adds only digits + ASCII words + a period). The no-diff call (`briefSummary(ranking)`) yields
|
||||||
|
exactly the pre-R3e string (the SC6 invariant).
|
||||||
|
Make the Phase-A frontmatter/section/marker/schema cases green.
|
||||||
|
|
||||||
|
## Step 4 — (GREEN) `cli.ts` — prior-brief discovery + the diff in `--json`
|
||||||
|
|
||||||
|
- Add `readdirSync` to the `node:fs` import (`cli.ts:51`).
|
||||||
|
- In the `brief` handler, **between** the ranking (`cli.ts:339`) and the render (`:340`):
|
||||||
|
```ts
|
||||||
|
const todayIds = surfacedIds(ranking);
|
||||||
|
let priorIds: string[] = [];
|
||||||
|
let priorDate: string | null = null;
|
||||||
|
try {
|
||||||
|
if (existsSync(outDir)) {
|
||||||
|
const priorFile = selectPriorBriefFile(readdirSync(outDir), day);
|
||||||
|
if (priorFile) {
|
||||||
|
priorIds = parseSurfacedFrontmatter(readFileSync(join(outDir, priorFile), "utf8"));
|
||||||
|
priorDate = priorFile.slice(0, 10);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch { priorIds = []; priorDate = null; } // unreadable prior ⇒ first-brief path
|
||||||
|
const diff = diffSurfaced(todayIds, priorIds, priorDate);
|
||||||
|
const md = renderBrief(ranking, diff);
|
||||||
|
```
|
||||||
|
- Import `diffSurfaced`, `parseSurfacedFrontmatter`, `selectPriorBriefFile` from `./brief.js` (`cli.ts:71`). (If
|
||||||
|
any code annotates `: BriefDiff`, import it via **`import type`** — it is an interface, stripped from emitted JS,
|
||||||
|
so a value-import fails at module-load (M1); the code above infers the type from `diffSurfaced`'s return, so no
|
||||||
|
`BriefDiff` import is actually needed.)
|
||||||
|
- **Thread the diff into the shared summary** (`cli.ts:350`, MAJOR-2): change `const summary = briefSummary(ranking)`
|
||||||
|
→ `const summary = briefSummary(ranking, diff)`. The frontmatter `summary:` (built inside `renderBrief` →
|
||||||
|
`briefSummary(ranking, diff)`) and the `--json` `summary` (read from this var) must stay **one source**
|
||||||
|
(`cli.test.ts:268`); without this, day-2's file carries the ` N nye siden sist.` marker but `--json.summary`
|
||||||
|
would not. **Safe on day-1:** `priorDate === null` ⇒ marker suppressed ⇒ byte-identical to the pre-R3e string.
|
||||||
|
- `--json` (`cli.ts:352`): add `diff: { priorDate: diff.priorDate, added: diff.added.length, carried:
|
||||||
|
diff.carried.length, dropped: diff.dropped.length }`.
|
||||||
|
- The non-JSON console line (`cli.ts:355`): append `${diff.added.length > 0 && diff.priorDate !== null ? \`, ${diff.added.length} nye siden sist\` : ""}`.
|
||||||
|
- **No new flag, no new exit code.** **Note (Phase-B test mechanism — rename-real-write, M2):** `today()` is
|
||||||
|
wall-clock, so a same-process two-day sequence cannot advance the date. Rather than hand-author a `<prior>.md`
|
||||||
|
fixture (which risks an id mismatch — its `surfaced:` ids would not be real store ids, so every trend reads as
|
||||||
|
added/dropped, a weak/vacuous test), the cli.test **runs `brief` for real** (writing `${today}.md` with a genuine
|
||||||
|
`surfaced:` line = `surfacedIds(ranking)`), **renames** it to a fixed past date (`mv ${out}/${today}.md
|
||||||
|
${out}/2026-06-20.md`), then runs `brief` again in the same `--out`. The second run discovers `2026-06-20.md` as
|
||||||
|
the strict-prior, parses its **real** ids, and diffs against today's cohort — proving discovery + parse + diff
|
||||||
|
clock-free, with `carried`/`added` that are *exactly* right (id-matched). Capture one new on-pillar trend between
|
||||||
|
the runs → that trend is the sole `added`.
|
||||||
|
Make the Phase-B cli cases green.
|
||||||
|
|
||||||
|
## Step 5 — wire `trend-spotter.md` (prose) + README
|
||||||
|
|
||||||
|
- `agents/trend-spotter.md` — one prose line (no batch-shape change): the morning brief now **records the trends
|
||||||
|
it showed** (frontmatter `surfaced:`) and renders a **day-over-day diff** ("Nytt siden sist") against the most
|
||||||
|
recent prior brief — no new capture step; the polling/capture path is unchanged. Domain-general (Section 17).
|
||||||
|
- `scripts/trends/README.md` — add `## Brief history + diff (RE-R3e)` between the R3d temporal-overlay section and
|
||||||
|
`## Tests`: the `surfaced:` frontmatter record (one self-describing artifact, `BRIEF_SCHEMA_VERSION 1→2`,
|
||||||
|
store `SCHEMA_VERSION` stays 4), `selectPriorBriefFile` strict-`< today` discovery (same-day re-run
|
||||||
|
determinism), the `diffSurfaced` partitions (added/carried/dropped), the `## 🆕 Nytt siden sist` section
|
||||||
|
(added with titles, carried/dropped as a count, "ikke vist i dag" framing) + the ` N nye siden sist.` summary
|
||||||
|
marker the SessionStart hook surfaces for free.
|
||||||
|
|
||||||
|
## Step 6 — gate: floors + new unconditional Section 16n
|
||||||
|
|
||||||
|
In `scripts/test-runner.sh`:
|
||||||
|
- Set `TRENDS_TESTS_FLOOR` (live **`:716`**, currently **216**) to the **`tests N`** line reported by `(cd
|
||||||
|
scripts/trends && npm test)` after Steps 1–5 — recounted live, NOT additive-guessed. Stays inside the deps
|
||||||
|
guard. **Append** `+ RE-R3e: brief +N, cli +N (brief history + diff)` to the inline breakdown comment.
|
||||||
|
- Add **Section 16n** ("Trends Brief History / Diff", RE-R3e), mirroring Section 16m (unconditional,
|
||||||
|
deps-absent-safe, pure `grep -qF`/self-test, no `tsx`, **all literals ASCII** — the `🆕` emoji is NEVER grepped;
|
||||||
|
the shell stays ASCII-clean for bash 3.2 `set -u`). **Placement (verify live):** between Section 16m's trailing
|
||||||
|
`echo ""` and the Section 18 header — anti-erosion stays last. Six **unconditional** checks, the self-test
|
||||||
|
emitting **one** pass/fail like 16m:
|
||||||
|
(1) a non-vacuity self-test (a probe carrying `diffSurfaced` accepted, one without rejected);
|
||||||
|
(2) `grep -qF 'export function diffSurfaced' scripts/trends/src/brief.ts`;
|
||||||
|
(3) `grep -qF 'parseSurfacedFrontmatter' scripts/trends/src/brief.ts`;
|
||||||
|
(4) `grep -qF 'Nytt siden sist' scripts/trends/src/brief.ts` (the section header literal — ASCII portion only);
|
||||||
|
(5) `grep -qF 'selectPriorBriefFile' scripts/trends/src/cli.ts` (the diff wiring in the CLI);
|
||||||
|
(6) `grep -qF 'surfaced: ' scripts/trends/src/brief.ts` (the frontmatter emit).
|
||||||
|
- Bump `ASSERT_BASELINE_FLOOR` (live **`:1473`**, currently **117**) → **exactly 123** (117 + the 6 new
|
||||||
|
unconditional 16n emitters; the self-test emits one pass/fail like 16m, so 123 is deterministic — "live recount"
|
||||||
|
is the safety net, not a guess). Insert the 16n clause into the **header-enumeration prose chain (`:53-64`)**
|
||||||
|
before "…the assertion-count anti-erosion floor (SC6) in Section 18," preserving sentence flow. **Append** the
|
||||||
|
RE-R3e (→123) narration to the **Section-18 floor-history comment** (which ends "= 117").
|
||||||
|
- **NOT touched here:** the hook suite (no `HOOK_TESTS_FLOOR` in `test-runner.sh`; R3e adds no hook test). It must
|
||||||
|
still pass untouched (`node --test hooks/scripts/__tests__/*.test.mjs`) as a regression sanity at land.
|
||||||
|
|
||||||
|
## Step 7 — behavioural verification
|
||||||
|
|
||||||
|
`(cd scripts/trends && npm install)` if needed, then run brief §7's five behavioural steps with a **unique tmp dir
|
||||||
|
(no `rm`)**: `D=/tmp/r3e-mb-$$; S=/tmp/r3e-$$.json`. Seed an on-pillar store; `brief --pillars … --out "$D"
|
||||||
|
--store "$S"` → frontmatter carries `surfaced: <ids>` + `schemaVersion: 2`, the section says `Første brief — alt
|
||||||
|
nedenfor er nytt`; **rename that real brief to a fixed past date** (`mv "$D/$(ls "$D")" "$D/2026-06-20.md"`),
|
||||||
|
`capture` a new on-pillar trend, and re-run `brief` → `## 🆕 Nytt siden sist (2026-06-20)` lists the added trend +
|
||||||
|
`N båret over, M ikke vist i dag`, `--json diff.added ≥ 1` (the **rename-real-write** path, M2 — the prior's ids
|
||||||
|
are real, so `carried`/`added` are id-exact); same-day re-run → `cmp` the two `${day}.md` byte-identical; confirm
|
||||||
|
`score.composite` unchanged after the briefs (only `surfacedCount`/`lastSurfacedAt` advance); strip the `surfaced:`
|
||||||
|
line from the renamed prior (a pre-R3e brief) and re-run → every trend reads as added (graceful degrade). Run
|
||||||
|
full `bash scripts/test-runner.sh` → `FAIL=0` (`ASSERT_BASELINE_FLOOR` 123, trends ≥ new floor, Section 16n green,
|
||||||
|
Section 17 de-niche green, counts 29/19/27); run `node --test hooks/scripts/__tests__/*.test.mjs` → still green
|
||||||
|
(untouched regression); confirm `schedule.test`/`run-daily.test` still green (R3c untouched).
|
||||||
|
|
||||||
|
## Step 8 — land
|
||||||
|
|
||||||
|
Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor 123, gate total;
|
||||||
|
`BRIEF_SCHEMA_VERSION` 1→2; store schema unchanged v4; **correct the stale `:713`/`:1403` cites to live
|
||||||
|
`:716`/`:1473`**). Commit order (house style): **(1)** docs commit `docs/research-engine/{brief,plan}-re-r3e.md`
|
||||||
|
(no suffix, tracked); **(2)** code commit — `brief.ts` + `cli.ts` + `brief.test.ts` + `cli.test.ts` +
|
||||||
|
`agents/trend-spotter.md` + `scripts/trends/README.md` + `scripts/test-runner.sh` with `[skip-docs]` (D9: single
|
||||||
|
code commit — the diff is one coherent feature). Push freely (window lifted; gitleaks at commit; `origin` =
|
||||||
|
PUBLIC `open/` — STATE/`*.local.*` never pushed). No version bump (additive; `v0.5.2` dev) — note
|
||||||
|
`BRIEF_SCHEMA_VERSION` 1→2 is the **artifact** schema, not the plugin version.
|
||||||
|
|
||||||
|
## Verification (testable)
|
||||||
|
|
||||||
|
| SC | Check | Command | Expected |
|
||||||
|
|---|---|---|---|
|
||||||
|
| — | RED Phase A | `(cd scripts/trends && npm test)` after the stubs | diff/parse/select/frontmatter/section/marker/schema cases fail on **value** assertions vs the constant stubs (not module-not-found) |
|
||||||
|
| — | RED Phase B | `npm test` (cli.test) before the wiring | the two-day cases fail on the missing `surfaced:`/`diff`/section |
|
||||||
|
| SC1 | diffSurfaced partitions | `npm test` (brief.test) | added/carried/dropped order-stable; empty prior ⇒ all added; repeated id once |
|
||||||
|
| SC2 | parseSurfacedFrontmatter | `npm test` (brief.test) | csv → ids; blank/absent/malformed → `[]`; line-anchored (no `summary:` mismatch) |
|
||||||
|
| SC3 | selectPriorBriefFile | `npm test` (brief.test) | greatest `< today`; excludes today + future; ignores non-dated; none → `null` |
|
||||||
|
| SC4 | `surfaced:` + round-trip | `npm test` (brief.test) | one `surfaced: <csv>` line = `surfacedIds(r).join(",")`, before `schemaVersion: 2`; round-trips; empty store → blank |
|
||||||
|
| SC5 | Nytt siden sist (4 branches) | `npm test` (brief.test) | `## 🆕 Nytt siden sist`; first-brief/empty/added/no-added branches; section precedes Topp-treff |
|
||||||
|
| SC6 | summary delta marker | `npm test` (brief.test) | ` N nye siden sist.` when prior+added; absent on first/no-added; `briefSummary(r)===briefSummary(r,empty)`; no `"`/`\n` |
|
||||||
|
| SC7 | schema boundary | `npm test` (brief.test) | `BRIEF_SCHEMA_VERSION` 2; `SCHEMA_VERSION` 4; no `score.composite` mutation after a brief |
|
||||||
|
| SC8 | determinism + same-day | `npm test` (brief.test) | same `(store,pillars,today,opts,diff)` → byte-identical; same-day re-run picks the same prior (strict `<`) |
|
||||||
|
| SC9 | CLI diff wiring | `npm test` (cli.test) | two-day (**rename-real-write**: real day-1 brief renamed to `2026-06-20.md`): `Nytt siden sist (<day1>)` lists the new trend; `--json diff` counts; first run `priorDate null`; `--out` isolates |
|
||||||
|
| SC10 | gate + wiring + de-niche | `bash scripts/test-runner.sh` | FAIL=0; trends ≥ floor; Section 16n green; `ASSERT_BASELINE_FLOOR`=123; Section 17; counts 29/19/27; hook suite green |
|
||||||
|
|
||||||
|
## Risks
|
||||||
|
|
||||||
|
- **R1 — adding `surfaced:` / bumping the schema breaks pinned tests.** *Mitigated (verified live):* the
|
||||||
|
frontmatter `schemaVersion:` test uses the **imported** `BRIEF_SCHEMA_VERSION` constant (auto-tracks 1→2); the
|
||||||
|
`summary:`-equality test is preserved by the SC6 invariant (`briefSummary(r)===briefSummary(r,empty)`);
|
||||||
|
`startsWith "---\n"` and the `date:`/`ranking:` tests are unaffected. **The one hard break (MAJOR-1):** `:574`
|
||||||
|
`assert.equal(BRIEF_SCHEMA_VERSION, 1)` — a literal **outside** the frontmatter set — which Step 3 flips to
|
||||||
|
`, 2)` with the bump (`:575` `SCHEMA_VERSION === 4` stays). Step 1 enumerates every surviving assertion + this
|
||||||
|
one break.
|
||||||
|
- **R2 — inserting a section before Topp-treff breaks an ordering assertion.** *Mitigated (verified):* no existing
|
||||||
|
test pins the intro→Topp-treff adjacency (all body assertions are substring/`match`); the new section is
|
||||||
|
asserted only by new tests. SC5 pins the section-before-Topp index in the new suite.
|
||||||
|
- **R3 — the `surfaced:` CSV could collide with a comma in an id.** *Mitigated:* real ids are 12-hex
|
||||||
|
(`store.ts:69-72`) — comma-free; the join/split is unambiguous in production. The brief.test fixtures use
|
||||||
|
comma-free ids for the round-trip. (Test `mkTrend` ids are `title|url`; the round-trip unit uses clean ids.)
|
||||||
|
- **R4 — same-day re-run picks its own just-written file → self-diff (empty) → non-deterministic vs the first
|
||||||
|
run.** *Mitigated:* `selectPriorBriefFile` filters `f < ${today}.md` (strict), so the same-day file is excluded
|
||||||
|
and the re-run picks the same true-prior → byte-identical (SC8). This is the exact R3c SC7 guarantee, preserved.
|
||||||
|
- **R5 — a malformed / hand-edited / pre-R3e prior brief crashes the diff.** *Mitigated:* `parseSurfacedFrontmatter`
|
||||||
|
returns `[]` on absent/blank/malformed (never throws); the cli `try/catch` degrades any fs error to the
|
||||||
|
empty-prior (first-brief) path. SC2 + behavioural step 5 cover it.
|
||||||
|
- **R6 — the summary marker breaks the SessionStart `extractYaml` regex.** *Mitigated:* ` N nye siden sist.` is
|
||||||
|
digits + ASCII words + a period — no `"`, no `\n`; SC6 asserts the invariant; the hook suite is a land-time
|
||||||
|
regression check. The hook reads `date`+`summary` only; `surfaced:` is `^surfaced:`-keyed (the `^summary:`
|
||||||
|
regex cannot match it).
|
||||||
|
- **R7 — the `🆕` emoji in the gate crashes bash 3.2 `set -u`.** *Mitigated:* Section 16n greps ONLY ASCII
|
||||||
|
literals (`export function diffSurfaced`, `parseSurfacedFrontmatter`, `Nytt siden sist`, `selectPriorBriefFile`,
|
||||||
|
`surfaced: `); the emoji lives only in `brief.ts` source + rendered output, asserted by the TS tests.
|
||||||
|
- **R8 — `BRIEF_SCHEMA_VERSION` bump misread as a store-schema/plugin-version change.** *Mitigated:* it is the
|
||||||
|
**artifact** frontmatter version (`brief.ts:23`, distinct from the store's `SCHEMA_VERSION` — the comment says
|
||||||
|
so); store `SCHEMA_VERSION` stays 4; no plugin version bump (additive). README + §4 state the boundary.
|
||||||
|
- **R9 — the diff couples `brief.ts` to fs (directory read) → breaks the purity claim.** *Mitigated (SD2):* the
|
||||||
|
three new helpers are pure (string/array in, value out); the `readdirSync`/`readFileSync` live in `cli.ts` (the
|
||||||
|
edge), injected exactly like `today`/`pillars`. `brief.ts`'s "No fs" header claim holds.
|
||||||
|
- **R10 — STATE's pinned floor line-cites (`:713`/`:1403`) are stale (live `:716`/`:1473`).** *Mitigated:* caught
|
||||||
|
at brief time (the lines drifted when R3d added Section 16m + the floor-history narration); the plan cites live
|
||||||
|
values; Step 8 corrects STATE.
|
||||||
|
- **R11 — a diff test that passes WITHOUT the feature (vacuous GREEN).** *Mitigated:* SC1/SC2/SC3 are unit tests
|
||||||
|
of pure functions whose stubs return constants (true value-RED in Phase A); SC9's day-2 asserts a **specific**
|
||||||
|
added trend appears in `Nytt siden sist (<day1>)` AND the `--json diff.added` count — both absent in the
|
||||||
|
flag-ignoring handler (Phase B RED). No fixture is ordered-by-accident.
|
||||||
|
- **R12 — `--no-mark` desync: the artifact records `surfaced:` but the store seen-log is not written.** *Accepted/
|
||||||
|
intended:* `surfaced:` is a property of the rendered brief (what it showed), `--no-mark` governs only the store
|
||||||
|
mutation. The next diff reads the **artifact**, so it is correct regardless of `--no-mark`. SC4 (frontmatter)
|
||||||
|
and the existing `--no-mark` test (`cli.test:392-398`) both hold.
|
||||||
|
|
||||||
|
## Plan-critic — folded
|
||||||
|
|
||||||
|
Three Opus reviewers (scope-guardian, brief-reviewer, plan-critic) ran COLD on the brief + this plan against live
|
||||||
|
`scripts/trends/`. **Verdicts:** scope-guardian **MIXED** · brief-reviewer **PROCEED_WITH_RISKS** · plan-critic
|
||||||
|
**REWORK (0.88)** — **converged on 2 MAJOR + 4 MINOR** (all re-verified against live code before folding; full
|
||||||
|
rationale in `brief-re-r3e.md §9`).
|
||||||
|
|
||||||
|
- **MAJOR-1** — `tests/brief.test.ts:574` `assert.equal(BRIEF_SCHEMA_VERSION, 1)` is a **hard literal** (outside
|
||||||
|
the frontmatter set §Step-1 enumerated; the `:163` RegExp auto-tracks). **Folded:** Step 3 flips it to `, 2)`
|
||||||
|
with the constant bump; Step 1's enumeration + R1 now name it; `:575` (`SCHEMA_VERSION === 4`) stays.
|
||||||
|
- **MAJOR-2** — `cli.ts:350` `const summary = briefSummary(ranking)` was left unthreaded → day-2 `--json.summary`
|
||||||
|
would lose the marker the file's frontmatter carries, breaking the `cli.test.ts:268` "one source" invariant.
|
||||||
|
**Folded:** Step 4 changes it to `briefSummary(ranking, diff)` (safe day-1; `priorDate===null` suppresses the
|
||||||
|
marker).
|
||||||
|
- **M1 (MINOR)** — `BriefDiff` is type-only → `import type` if referenced (Step 2/4); the Step-4 code infers it, so
|
||||||
|
no import is actually needed.
|
||||||
|
- **M2 (MINOR)** — SC9 uses **rename-real-write** (run `brief`, `mv ${day}.md → 2026-06-20.md`, re-run), not a
|
||||||
|
hand-fixture → clock-free + id-exact prior. **Folded:** Step 4 note, Phase B, Step 7, SC9 row.
|
||||||
|
- **M3 (MINOR)** — SC1 "repeated id not double-counted" reworded → **cross-partition disjointness** (filters
|
||||||
|
preserve within-list dups; production ids are distinct). Brief §6 SC1 + Step 1.
|
||||||
|
- **M4 (MINOR)** — brief §3 "empty `surfaced:`" contradiction reworded (the `surfaced:` line is diff-independent =
|
||||||
|
`surfacedIds(ranking)`; the default empty diff only drives the `_Første brief._` section). Brief §3.
|
||||||
|
|
||||||
|
**Confirmed correct (untouched):** all line-cites, the floors (216 @ `:716`; 117 → 123 @ `:1473`), Section 16m as
|
||||||
|
the last trends section, the 6 ASCII sentinels' non-vacuity, the regex/ISO-lex/hook-safety/`--json`-shape/same-day
|
||||||
|
strict-`<` — verified by all three.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue