linkedin-studio/docs/research-engine/plan-re-r1.md

Plan — RE-R1: item-schema (B1) + triage-scorer (B2) as tested code

Brief: docs/research-engine/brief-re-r1.md. Slice: RE-R1 (research-engine rung-2, slice 1). TDD-order: RED (item + score tests as logic-RED) → GREEN (item.ts, score.ts) → GREEN (CLI + cli tests) → wire trend-spotter prose → gate floors → behavioural → land. Counts recounted live at land, never pinned/guessed. Light-Voyage hardened: scope-guardian ALIGNED; brief-reviewer + plan-critic findings folded (see §Plan-critic — folded).

Goal

Move the research engine's deterministic core — the canonical item envelope (B1) and the composite/band/threshold arithmetic (B2) — out of agents/trend-spotter.md prose into pure, tested TypeScript under scripts/trends/, behind a CLI seam the agent and a future orchestrator call. No store-schema change; the five judgment scores stay with the model; wiring normalizeItem into the store is R2.

Files touched (exhaustive — for scope-guardian)

File	Change	SC
scripts/trends/src/item.ts	NEW — TrendItem ingress envelope + normalizeItem / normalizeItems (pure, validating; no id)	SC1
scripts/trends/src/score.ts	NEW — ScoreMode, per-mode weight consts (mirror SSOT), composite, band, triage	SC2, SC3, SC4
scripts/trends/src/cli.ts	EDIT — add normalize + score subcommands (stdin JSON in, JSON out, exit 2 on bad invocation)	SC5
scripts/trends/tests/item.test.ts	NEW — normalize: required-field errors, whitespace/topic dedupe, publishedAt validate, batch partition	SC1
scripts/trends/tests/score.test.ts	NEW — golden composite (both modes: all-tens=10.0 + {10,8,6,4,2}=7.0 + weights-sum-1.0 + pinned weights), range guard, band boundaries + pinned action strings, triage gate/rank	SC2, SC3, SC4
scripts/trends/tests/cli.test.ts	NEW — subprocess: normalize/score happy path (stdin→JSON) + exit-2 bad invocation	SC5
agents/trend-spotter.md	EDIT — replace L124-137 "compute composite/bands yourself" with a pointer naming src/cli.ts score as the deterministic-step owner; domain-general	SC7
scripts/test-runner.sh	EDIT — TRENDS_TESTS_FLOOR 24→recount (stays inside deps guard); NEW unconditional Section 16g (before Section 18); ASSERT_BASELINE_FLOOR 84→recount; header enumeration	SC6, SC7
docs/research-engine/{brief,plan}-re-r1.md	NEW — slice docs (TRACKED, like docs/second-brain/*)	—
STATE.md	EDIT at land — Telling-block reconcile (floors, gate total). Land bookkeeping, not slice scope; LOCAL-ONLY.	—

Not touched (scope fence): scripts/trends/src/{store.ts,types.ts} (no schema change, SCHEMA_VERSION stays 1) · references/trend-scoring-modes.md (SSOT unchanged) · references/* (no new ref doc) · config/trends-sources.template.md (source list — not wired in R1) · agents/* count (19) · commands/* (29) · hooks/** · .gitignore (trends lines already present).

Step 1 — (RED) failing tests for B1 + B2

Write tests/item.test.ts and tests/score.test.ts against not-yet-existing modules. Make them logic-RED, not import-RED, with a stub strategy split by assertion type:

• arithmetic / "returns X" tests → stub returns a wrong constant (so the equality assertion fails on value, not on a throw);
• "should throw" validation tests → stub returns a valid-looking value (so the assert.throws fails because nothing threw).

item.test.ts: well-formed raw → canonical item (fields verbatim; topics normalized+deduped); missing/empty source|title|url → {ok:false} naming the field; whitespace collapse via the same normalization as store.normalizeField; publishedAt present-and-ISO → kept, absent → undefined, present-and-invalid → {ok:false}; normalizeItems partitions a batch into {items, errors}; the canonical item carries no id.

score.test.ts: composite for both modes on (a) all-tens → exactly 10.0, (b) the asymmetric vector {10,8,6,4,2} in dimension order → 7.0; a dimension = 0 or 11 → throws; a pinned-weights assertion (each mode's five constants equal the SSOT values, Σ=1.0); band at 8.0 / 6.0 / 4.0 / 2.0 / below → correct priority + the exact SSOT action string (kortform + long-form); triage with threshold 4.0 → kept (≥4.0, sorted composite-desc, annotated composite+band) and dropped (<4.0).

RED proof (record in commit): (cd scripts/trends && npm test) → new cases fail with assertion errors (not module-not-found).

Step 2 — (GREEN) implement item.ts

Implement the TrendItem ingress envelope {source,title,url,publishedAt?,topics[],summary?} + normalizeItem/normalizeItems to pass Step 1's item cases. Reuse normalizeField (import from ./store.js) for whitespace; topic normalize + dedupe. publishedAt validated with a strict ISO-date check; carried, not persisted (comment: the source's publish date, forward-compat for B4 freshness — distinct from the store's capturedAt). Do not derive or carry id — the store owns it via addTrend→trendId; the envelope has no id field. Do not wire to the store — the item→TrendInput bridge (capturedAt injection) is R2.

Step 3 — (GREEN) implement score.ts

Encode the two weight-sets as const records mirroring trend-scoring-modes.md (header comment: the SSOT is the human source, "ordering is the signal, not a measured coefficient", + the SSOT path). Implement composite (validate each dimension ∈[1,10], weighted sum), band (the five-range map → {priority, kortformAction, longformAction} using the exact SSOT action strings), and triage (kept/dropped + composite-desc sort + per-item composite/band annotation). Make Step 1's score cases green (incl. the pinned-weights + pinned-action-string assertions).

Step 4 — (GREEN) CLI subcommands + cli.test.ts

Add normalize and score to cli.ts's main dispatch. Both read the JSON payload from stdin (not a flag — the existing --json is an output toggle and must keep that meaning) and print JSON to stdout. normalize → canonical items / {ok:false} error entries. score → triage with --mode/--threshold, prints {kept, dropped}. Exit 2 on a malformed invocation (unparseable stdin, missing required flag) via the existing usage() path; 0 otherwise. Write tests/cli.test.ts (subprocess: spawn node --import tsx src/cli.ts <sub> with a piped stdin payload) covering a happy path + an exit-2 bad-invocation for each new subcommand.

Step 5 — wire trend-spotter.md (prose pointer)

Replace the L124-137 "score 5 dims, take the weighted composite, apply the bands yourself" instruction with: the agent supplies the five 1–10 judgment scores, then pipes them to scripts/trends/src/cli.ts score (the deterministic owner of composite + bands + threshold). The replacement prose must contain the literal src/cli.ts score (the exact string Step 6's grep -qF matches). Keep it domain-general — no vendor/sector tokens (Section 17). The agent still owns mode selection and the qualitative scoring.

Step 6 — gate: floors + new unconditional section

In scripts/test-runner.sh:

• Bump TRENDS_TESTS_FLOOR 24 → live recount after Steps 1–4 (24 + new item/score/cli cases). It stays inside the if [ -x scripts/trends/node_modules/.bin/tsx ] deps guard (conditional — do not hoist it out; that would break fresh-clone safety).
• Add Section 16g (label it 16g; place it after Section 17 / before Section 18, since Section 18 anti-erosion must run last). Three unconditional, deps-absent-safe checks (pure grep, no tsx): (1) score.ts encodes both kortform and long-form weight-sets; (2) grep -qF "src/cli.ts score" agents/trend-spotter.md; (3) a non-vacuity self-test for those greps (house pattern, per Sections 16c–17).
• Bump ASSERT_BASELINE_FLOOR 84 → live recount (= 84 + the count of new unconditional pass/fail emitters in 16g; expected ~87 with the self-test, recounted at land, not pinned). Update the section-header enumeration comment.

Step 7 — behavioural verification

(cd scripts/trends && npm install) if needed, then run (verified invocation form, not npm run start): echo '<3-item sample>' | node --import tsx src/cli.ts normalize and echo '<scored sample>' | node --import tsx src/cli.ts score --mode kortform --threshold 4.0; confirm by hand that one ≥4.0 item is kept (correct band/action) and one <4.0 is dropped. Run full bash scripts/test-runner.sh → FAIL=0.

Step 8 — land

Recount all touched floors live; reconcile STATE.md "Telling" block (trends N/N, ASSERT floor, gate total). Commit order (house style: feat … [skip-docs] for code; plain for docs): (1) docs commit docs/research-engine/{brief,plan}-re-r1.md (no suffix, tracked); (2) code commit scripts/trends/* + agents/trend-spotter.md + scripts/test-runner.sh with [skip-docs]. Check the push window (date '+%u %H:%M'); origin is the PUBLIC open/ remote → confirm with operator before push. No version bump (additive; v0.5.2 dev).

Verification (testable)

SC	Check	Command	Expected
—	RED proof	(cd scripts/trends && npm test) after Step 1	new cases fail on assertion (logic-RED), not module-not-found
SC1	normalize	npm test (item.test.ts)	required-field {ok:false} + dedupe + publishedAt + batch-partition cases green; no id on item
SC2	composite	npm test (score.test.ts)	all-tens=10.0 both modes; {10,8,6,4,2}=7.0 both modes; pinned weights; range guard throws
SC3	bands	npm test (score.test.ts)	8.0/6.0/4.0/2.0 boundaries → correct priority + exact SSOT action string
SC4	triage	npm test (score.test.ts)	kept ≥ threshold ranked desc + annotated; dropped below
SC5	CLI	npm test (cli.test.ts) + manual stdin run	JSON out; exit 2 malformed invocation / 0 well-formed
SC6	gate	bash scripts/test-runner.sh	FAIL=0; trends ≥ new floor; ASSERT floor = live recount
SC7	wiring + de-niche	Section 16g greps + Section 17	src/cli.ts score present in trend-spotter.md; no vendor/sector tokens; counts 19/29/27

Risks

• R1 — SSOT/code drift. Weights, band thresholds, AND the ten action strings now live in both trend-scoring-modes.md and score.ts. Mitigated: score.test.ts pins all three (weights + Σ=1.0 + thresholds + exact action strings) against the SSOT values, with an SSOT-path comment naming the markdown as the human source. (A markdown-table-parsing cross-check is deferred — out of scope.)
• R2 — publishedAt carried but not persisted; capturedAt not on the item. Could read as a dangling field. Mitigated: explicit comment (source publish-date, forward-compat for B4) + the brief's non-goal; the store bridge (capturedAt injection) is explicitly R2; the scorer does not depend on either.
• R3 — editing trend-spotter.md could trip the de-niche guard. Mitigated: Section 17 runs in the gate; replacement prose is pillar-driven and vendor/sector-free.
• R4 — new gate checks must survive a deps-absent fresh clone. Mitigated: the three new checks are pure grep/self-test on tracked source files (no tsx), so they are unconditional and safe; the arithmetic proof stays inside the deps-gated suite; TRENDS_TESTS_FLOOR stays inside the deps guard.
• R5 — default threshold (4.0) may not match operator intent. Mitigated: single config param; brief open question #1 settles it at the go-gate.
• R6 — CLI --json semantic collision. Mitigated: new subcommands take payload via stdin, leaving --json as the existing output toggle; cli.test.ts encodes the stdin contract.

Plan-critic — folded

plan-critic returned REVISE (1 blocker, 6 majors, 4 minors); brief-reviewer PROCEED_WITH_RISKS; scope-guardian ALIGNED. Resolution, each verified against live code:

• [BLOCKER] TrendInput shape mismatch (store.ts:26-33 requires capturedAt, no publishedAt). ✅ Step 2 no longer claims a direct map; the item→store bridge is deferred to R2; envelope carries no id.
• [MAJOR] gate-section placement / Section-18-last ✅ Step 6 pins 16g, before Section 18.
• [MAJOR] ASSERT_BASELINE_FLOOR hard-pinned 86 ✅ Step 6 now live recount (~+3 with the house self-test), not pinned.
• [MAJOR] TRENDS_TESTS_FLOOR could be hoisted out of the deps guard ✅ Step 6 states it stays conditional.
• [MAJOR] CLI --json input/output overload (cli.ts:79 output toggle) ✅ Step 4 reads payload from stdin.
• [MAJOR] band action-string drift unguarded ✅ Step 1/3 + R1 pin the thresholds + action strings.
• [MAJOR] Step 5/6 grep literal unpinned ✅ pinned to src/cli.ts score in both steps.
• [MINOR] RED-stub strategy ✅ Step 1 splits stub by assertion type. [MINOR] npm run start unverified ✅ Step 7 uses node --import tsx src/cli.ts. [MINOR] commit grouping ✅ Step 8 fixes order. [MINOR] empty folded sections ✅ filled.
• [scope-guardian MINOR] STATE.md + config/trends-sources.template.md ✅ STATE.md added as a land-bookkeeping row; template added to the scope fence.

scope-guardian — ALIGNED: every SC1–SC7 traces to a step; zero scope creep; every NON-goal respected.