open/linkedin-studio
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions

feat(linkedin-studio): RE-R1 — item-schema (B1) + triage-scorer (B2) as tested code behind CLI seam [skip-docs]

Browse source 
Lift the research engine's deterministic core out of agents/trend-spotter.md prose
into pure, tested TypeScript under scripts/trends/, behind a CLI seam the agent calls.

- B1 src/item.ts: TrendItem ingress envelope + normalizeItem/normalizeItems
  (required-field validation, topic normalize+dedupe via store's normalizeField,
  optional publishedAt ISO-validate). No id (store derives it); no store bridge
  (capturedAt injection is R2).
- B2 src/score.ts: per-mode weight consts mirroring the SSOT
  (references/trend-scoring-modes.md), composite (weighted sum, [1,10] guard),
  band (5-band map + exact SSOT action strings), triage (keep>=threshold, rank desc,
  annotate composite+band). Owns ONLY the arithmetic; the five judgment scores stay
  model-side.
- CLI normalize/score: JSON payload on STDIN, JSON to stdout (the existing --json
  output toggle is untouched); exit 2 on bad invocation, 0 otherwise.
- Wire trend-spotter.md to name 'src/cli.ts score' as the deterministic-step owner
  (prose pointer; the agent still supplies the five scores). Domain-general.
- Gate: TRENDS_TESTS_FLOOR 24->62; new unconditional Section 16g (score.ts both-mode
  weight-sets + trend-spotter scorer-pointer + non-vacuity self-test);
  ASSERT_BASELINE_FLOOR 84->87.

TDD: logic-RED proven (33/34 item+score fail on assertions, not module-not-found),
then GREEN (trends suite 62/62); CLI RED 2/4 -> GREEN 4/4. Full gate 102/0/0.
No store-schema change (SCHEMA_VERSION stays 1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VmHCQjJHUyWwxGAVVjNLgp
This commit is contained in:
Kjell Tore Guttormsen 2026-06-24 10:09:45 +02:00
commit 24775f4493
8 changed files with 793 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

11
agents/trend-spotter.md
View file

@ -131,10 +131,13 @@ query bank: target a source or topic from the list (`"[Tier-1 source] latest"`,
invoked from `/linkedin:newsletter`) or asks for it explicitly. Depth potential enters at 25 %
and timing drops to 10 % — a chronicle rewards substance and a durable angle over speed.
Score each candidate's five dimensions 1–10 per the mode's table, take the weighted composite
(both modes stay on the same 0–10 scale), and rank highest-first. The composite→action bands
(Immediate / High / Medium / Low / Skip) live in that same reference — use them; do not restate
the thresholds here.
Score each candidate's five dimensions 1–10 per the mode's table — that qualitative judgment is
yours. The deterministic step that follows is NOT: pipe the scored candidates (JSON on stdin) to the
scorer CLI `${CLAUDE_PLUGIN_ROOT}/scripts/trends/src/cli.ts score` (`--mode kortform|long-form
[--threshold N]`), the single owner of the weighted composite, the composite→action bands
(Immediate / High / Medium / Low / Skip), and the keep/drop threshold. It returns the kept candidates
ranked highest-first, each annotated with its composite + band. Do not recompute the composite or
restate the band thresholds here — supply the five judgment scores and let the scorer rank and triage.
## Trend Opportunity Assessment

77
scripts/test-runner.sh
View file

@ -38,7 +38,10 @@
# inferences', with a non-vacuity self-test) in Section 16e; the brain reconcile-wiring
# guard (SB-S3e: scripts/brain/src/cli.ts dispatches `reconcile` AND calls the core
# reconcileRecentPosts by literal name, with a non-vacuity self-test) in Section 16f;
# the assertion-count anti-erosion floor (SC6) in Section 18. All are live below (Sections 8–18).
# the trends-scorer wiring guard (RE-R1: scripts/trends/src/score.ts encodes both mode
# weight-sets AND agents/trend-spotter.md references the scorer CLI 'src/cli.ts score',
# with a non-vacuity self-test) in Section 16g; the assertion-count anti-erosion floor
# (SC6) in Section 18. All are live below (Sections 8–18).
#
# Usage: bash scripts/test-runner.sh
# bash 3.2-safe: plain arrays only, no `declare -A`, no `mapfile`/`readarray`.
@ -689,7 +692,7 @@ if [ -x "$TR_DIR/node_modules/.bin/tsx" ]; then
TR_OUT=$( set +e; (cd "$TR_DIR" && npm test) 2>&1; echo "TR_EXIT:$?" )
TR_EXIT=$(echo "$TR_OUT" | grep -oE 'TR_EXIT:[0-9]+' | grep -oE '[0-9]+' | head -1)
TR_TESTS=$(echo "$TR_OUT" | grep -oE 'tests [0-9]+' | grep -oE '[0-9]+' | tail -1)
TRENDS_TESTS_FLOOR=24 # B-S3: +3 newestCaptureDate tests (staleness signal)
TRENDS_TESTS_FLOOR=62 # store 24 + RE-R1: item 18 + score 16 + cli 4 (item-schema + triage-scorer)
if [ "$TR_EXIT" = "0" ] && [ -n "$TR_TESTS" ] && [ "$TR_TESTS" -ge "$TRENDS_TESTS_FLOOR" ]; then
pass "trends-store suite green: $TR_TESTS tests pass (floor $TRENDS_TESTS_FLOOR)"
else
@ -1002,6 +1005,70 @@ done
echo ""
# --- Section 16g: Trends Scorer Wiring (research-engine RE-R1 / B2) ---
echo "--- Trends Scorer Wiring ---"
# RE-R1 lifts the composite/band/threshold arithmetic out of trend-spotter.md prose into
# tested code (scripts/trends/src/score.ts) behind a CLI seam. Two literals must hold,
# grepped EXACT (grep -F), deps-absent-safe (pure grep, no tsx):
# (1) score.ts encodes BOTH mode weight-sets (the 'kortform' + 'long-form' literals), so a
# silent collapse to one mode fails here (the per-mode arithmetic itself is unit-tested
# in score.test.ts, behind the deps guard / trends-suite floor);
# (2) agents/trend-spotter.md references the scorer CLI by the literal 'src/cli.ts score' —
# the lift is real and grep-able, not merely documented.
# Non-vacuity self-test mirrors Sections 16c-17: the weight-set predicate (AND of both mode
# literals) must accept a both-modes probe and reject single-mode probes; the wiring predicate
# must accept a probe carrying the scorer-pointer literal and reject one without it. Labelled
# 16g but placed after Section 17 / before Section 18 (anti-erosion must run last so it sees
# every prior check). UNCONDITIONAL (no tsx) -> counts toward ASSERT_BASELINE_FLOOR.
WEIGHT_KORT_LIT='kortform'
WEIGHT_LONG_LIT='long-form'
SCORER_WIRE_LIT='src/cli.ts score'
weights_both_modes() { # $1 = text; true iff BOTH mode literals present (echo twice — grep consumes stdin)
echo "$1" | grep -qF "$WEIGHT_KORT_LIT" && echo "$1" | grep -qF "$WEIGHT_LONG_LIT"
}
G16_SELFTEST_OK=1
if ! weights_both_modes 'the kortform weight-set and the long-form weight-set are both encoded'; then
G16_SELFTEST_OK=0; echo " non-vacuity FAIL: a both-modes weight probe was not detected"
fi
while IFS= read -r probe; do
[ -z "$probe" ] && continue
if weights_both_modes "$probe"; then
G16_SELFTEST_OK=0; echo " false-positive FAIL: single-mode weight probe accepted -> $probe"
fi
done <<'NEGATIVE16G'
only the kortform weight-set is present here
only the long-form weight-set is present here
NEGATIVE16G
if ! echo 'pipe the scores to src/cli.ts score for the composite' | grep -qF "$SCORER_WIRE_LIT"; then
G16_SELFTEST_OK=0; echo " non-vacuity FAIL: a wired scorer-pointer probe was not detected"
fi
if echo 'the agent computes the composite itself' | grep -qF "$SCORER_WIRE_LIT"; then
G16_SELFTEST_OK=0; echo " false-positive FAIL: an unwired probe matched the scorer pointer"
fi
if [ "$G16_SELFTEST_OK" -eq 1 ]; then
pass "trends-scorer self-test: both-modes weight predicate + scorer-pointer predicate detect wiring, reject the under-wired forms"
else
fail "trends-scorer self-test failed — the scorer-wiring lint is vacuous or over-eager"
fi
SCORE_TS="scripts/trends/src/score.ts"
if grep -qF "$WEIGHT_KORT_LIT" "$SCORE_TS" 2>/dev/null && grep -qF "$WEIGHT_LONG_LIT" "$SCORE_TS" 2>/dev/null; then
pass "score.ts encodes both mode weight-sets ('$WEIGHT_KORT_LIT' + '$WEIGHT_LONG_LIT')"
else
fail "score.ts missing a mode weight-set — needs both '$WEIGHT_KORT_LIT' and '$WEIGHT_LONG_LIT' in $SCORE_TS"
fi
if grep -qF "$SCORER_WIRE_LIT" agents/trend-spotter.md; then
pass "trend-spotter.md references the scorer CLI ('$SCORER_WIRE_LIT') as the deterministic-step owner"
else
fail "trend-spotter.md does not reference the scorer CLI — add a '$SCORER_WIRE_LIT' pointer (RE-R1 lift)"
fi
echo ""
# --- Section 18: Assertion-Count Anti-Erosion (SC6) ---
# The lint self-modifies its own checks, so a green run could mask a silently dropped
# assertion. Pin the total pass()+fail() invocations as a monotonic floor; the count
@ -1011,12 +1078,14 @@ echo ""
# +2 for SB-S3a's two UNCONDITIONAL Section-16d checks (profile-reader self-test +
# strategy-advisor wiring grep) = 80; +2 for SB-S3d's two UNCONDITIONAL Section-16e
# checks (ops-reader self-test + strategy-advisor ops-wiring grep) = 82; +2 for SB-S3e's
# two UNCONDITIONAL Section-16f checks (reconcile self-test + brain-CLI reconcile grep) = 84.
# two UNCONDITIONAL Section-16f checks (reconcile self-test + brain-CLI reconcile grep) = 84;
# +3 for RE-R1's three UNCONDITIONAL Section-16g checks (trends-scorer self-test + score.ts
# both-modes weight-set grep + trend-spotter scorer-pointer grep) = 87.
# NB: the floor tracks the deps-absent MINIMUM (conditional TS suites warn-skip and drop
# the count), so it is bumped only by UNCONDITIONAL new checks — NOT pinned to the
# deps-present TOTAL_CHECKS (that would zero the warn-skip margin and false-fail a fresh
# clone). Runs last so TOTAL_CHECKS sees every prior check.
ASSERT_BASELINE_FLOOR=84
ASSERT_BASELINE_FLOOR=87
TOTAL_CHECKS=$((PASS + FAIL))
if [ "$TOTAL_CHECKS" -ge "$ASSERT_BASELINE_FLOOR" ]; then
pass "assertion-count anti-erosion: $TOTAL_CHECKS checks >= baseline floor $ASSERT_BASELINE_FLOOR"

66
scripts/trends/src/cli.ts
View file

@ -7,14 +7,23 @@
* node --import tsx src/cli.ts query --topics <a,b> [--store <path>] [--json]
* node --import tsx src/cli.ts list [--since <YYYY-MM-DD>] [--limit <n>] [--store <path>] [--json]
* node --import tsx src/cli.ts status [--store <path>] [--json]
* echo '<raw item|batch>' | node --import tsx src/cli.ts normalize
* echo '<scored candidates>' | node --import tsx src/cli.ts score [--mode kortform|long-form] [--threshold N]
*
* The capture agent (research-engine) calls `add` to fold a freshly-polled trend
* into the store, and `query`/`list` to reason over accumulated history. The
* polling + relevance-scoring itself lives upstream; this is the deterministic store.
*
* Exit code: 0 on success, 2 on usage error.
* `normalize` + `score` (RE-R1) are the deterministic research-engine seam: both read
* their JSON PAYLOAD FROM STDIN (so they do not overload `--json`, which stays an
* output toggle) and print JSON to stdout. `normalize` validates raw items into the
* canonical envelope; `score` triages scored candidates (composite/band/threshold).
*
* Exit code: 0 on success, 2 on usage error (incl. unparseable stdin / bad flag).
*/
import { readFileSync } from "node:fs";
import {
addTrend,
defaultStorePath,
@ -24,6 +33,9 @@ import {
queryByTopic,
saveStore,
} from "./store.js";
import { normalizeItem, normalizeItems } from "./item.js";
import { triage } from "./score.js";
import type { ScoreMode } from "./score.js";
function parseFlags(args: string[]): Record<string, string> {
const out: Record<string, string> = {};
@ -58,11 +70,29 @@ function usage(msg: string): never {
' add --title "<t>" --url "<u>" --topics <a,b> [--source <s>] [--summary "<s>"] [--store <path>]\n' +
" query --topics <a,b> [--store <path>] [--json]\n" +
" list [--since <YYYY-MM-DD>] [--limit <n>] [--store <path>] [--json]\n" +
" status [--store <path>] [--json]",
" status [--store <path>] [--json]\n" +
" normalize < raw-item-or-batch.json\n" +
" score [--mode kortform|long-form] [--threshold N] < scored-candidates.json",
);
process.exit(2);
}
/** Read the full JSON payload from stdin, or exit 2 if it is empty/unparseable. */
function readStdinJson(): unknown {
let raw = "";
try {
raw = readFileSync(0, "utf8").trim();
} catch {
raw = "";
}
if (raw.length === 0) usage("expected a JSON payload on stdin");
try {
return JSON.parse(raw);
} catch {
usage("stdin is not valid JSON");
}
}
function today(): string {
return new Date().toISOString().slice(0, 10);
}
@ -166,6 +196,38 @@ function main(): void {
return;
}
if (command === "normalize") {
const payload = readStdinJson();
const out = Array.isArray(payload) ? normalizeItems(payload) : normalizeItem(payload);
console.log(JSON.stringify(out, null, 2));
return;
}
if (command === "score") {
const mode = flags.mode && flags.mode !== "true" ? flags.mode : "kortform";
if (mode !== "kortform" && mode !== "long-form") {
usage('score --mode must be "kortform" or "long-form"');
}
let threshold = 4.0;
if (flags.threshold && flags.threshold !== "true") {
const t = Number.parseFloat(flags.threshold);
if (Number.isNaN(t)) usage("--threshold must be a number");
threshold = t;
}
const payload = readStdinJson();
if (!Array.isArray(payload)) usage("score expects a JSON array of scored candidates on stdin");
try {
const result = triage(payload as Array<{ scores: Record<string, number> }>, {
mode: mode as ScoreMode,
threshold,
});
console.log(JSON.stringify(result, null, 2));
} catch (e) {
usage(`scoring failed: ${(e as Error).message}`);
}
return;
}
usage(command ? `unknown command: ${command}` : "no command given");
}

130
scripts/trends/src/item.ts Normal file
View file

@ -0,0 +1,130 @@
/**
* Canonical ingress item schema + normalizer for the research engine (RE-R1, B1).
*
* A `TrendItem` is the ONE envelope every source/adapter emits before a trend reaches
* the store — "the one schema downstream never branches on". This module validates +
* normalizes that envelope deterministically (no AI, no network): required-field
* validation, topic normalize + dedupe, optional publishedAt ISO validation. It is the
* trend-side twin of the store's own normalize/dedupe discipline (scripts/trends/src/store.ts).
*
* Scope (RE-R1): the validated envelope + normalizer ONLY. The item->store bridge —
* injecting the store's `capturedAt` and persisting `publishedAt` — is R2 orchestration
* and lives in the CLI/agent layer, not here. The envelope carries NO `id`: the store
* derives it via addTrend->trendId, so an id here would be a second source of truth.
*
* GENERIC BY ARCHITECTURE: nothing niche-specific lives here. Which topics matter and
* which sources to poll are decided upstream (config/profile + the capture agent).
*/
import { normalizeField } from "./store.js";
export interface TrendItem {
/** Capture origin: a research-MCP name ("tavily"), "websearch", or "manual". Stored VERBATIM. */
source: string;
/** The trend headline, VERBATIM (case + spacing preserved — the store keeps it verbatim too). */
title: string;
/** The source URL, VERBATIM (case-sensitive paths must survive). */
url: string;
/**
* The SOURCE's own publish date (ISO-8601), validated-if-present. Carried for
* forward-compat (B4 freshness) — distinct from the store's `capturedAt`, and NOT
* persisted in R1. Absent -> the key is omitted.
*/
publishedAt?: string;
/** Topic tags, normalized (lowercase + whitespace-collapsed via normalizeField) + deduped. */
topics: string[];
/** Optional short summary, VERBATIM. Absent/blank -> the key is omitted. */
summary?: string;
}
export type NormalizeResult = { ok: true; item: TrendItem } | { ok: false; errors: string[] };
/** One failed entry in a batch: its index in the input + the field errors. */
export interface ItemError {
index: number;
errors: string[];
}
const REQUIRED_FIELDS = ["source", "title", "url"] as const;
/** Strict ISO-8601: a calendar date (YYYY-MM-DD), optionally with a time/zone. Rejects impossible dates. */
function isValidIso(value: string): boolean {
if (!/^\d{4}-\d{2}-\d{2}([T ]\d{2}:\d{2}(:\d{2}(\.\d+)?)?(Z|[+-]\d{2}:\d{2})?)?$/.test(value)) {
return false;
}
const ms = Date.parse(value);
if (Number.isNaN(ms)) return false;
// Round-trip the date part: catches 2026-02-30 / out-of-range that the regex lets through.
return new Date(ms).toISOString().slice(0, 10) === value.slice(0, 10);
}
function isNonEmptyString(v: unknown): v is string {
return typeof v === "string" && v.trim().length > 0;
}
/** Normalize each topic via the store's normalizeField, drop blanks, dedupe (first-seen order). */
function normalizeTopics(raw: unknown): string[] {
if (!Array.isArray(raw)) return [];
const out: string[] = [];
const seen = new Set<string>();
for (const t of raw) {
if (typeof t !== "string") continue;
const norm = normalizeField(t);
if (norm.length === 0 || seen.has(norm)) continue;
seen.add(norm);
out.push(norm);
}
return out;
}
/**
* Validate + normalize one raw item into the canonical envelope. Pure. Returns a
* structured error (never a silent partial) when a required field is missing/empty
* or publishedAt is present-but-invalid.
*/
export function normalizeItem(raw: unknown): NormalizeResult {
if (typeof raw !== "object" || raw === null || Array.isArray(raw)) {
return { ok: false, errors: ["raw item must be an object"] };
}
const r = raw as Record<string, unknown>;
const errors: string[] = [];
for (const field of REQUIRED_FIELDS) {
if (!isNonEmptyString(r[field])) {
errors.push(`missing or empty required field: ${field}`);
}
}
let publishedAt: string | undefined;
if (r.publishedAt !== undefined && r.publishedAt !== null) {
if (typeof r.publishedAt !== "string" || !isValidIso(r.publishedAt)) {
errors.push(`invalid publishedAt (expected an ISO-8601 date): ${String(r.publishedAt)}`);
} else {
publishedAt = r.publishedAt;
}
}
if (errors.length > 0) return { ok: false, errors };
const item: TrendItem = {
source: r.source as string,
title: r.title as string,
url: r.url as string,
topics: normalizeTopics(r.topics),
...(publishedAt !== undefined ? { publishedAt } : {}),
...(isNonEmptyString(r.summary) ? { summary: r.summary as string } : {}),
};
return { ok: true, item };
}
/** Partition a raw batch into normalized items + per-index errors (never throws). */
export function normalizeItems(raw: unknown[]): { items: TrendItem[]; errors: ItemError[] } {
const items: TrendItem[] = [];
const errors: ItemError[] = [];
raw.forEach((entry, index) => {
const res = normalizeItem(entry);
if (res.ok) items.push(res.item);
else errors.push({ index, errors: res.errors });
});
return { items, errors };
}

122
scripts/trends/src/score.ts Normal file
View file

@ -0,0 +1,122 @@
/**
* Deterministic triage scorer for the research engine (RE-R1, B2).
*
* Owns ONLY the arithmetic the SSOT (references/trend-scoring-modes.md) defines: the
* per-mode weighted composite, the composite->band map, and the threshold triage.
* Producing the five 1-10 dimension scores stays MODEL JUDGMENT by design — this module
* never scores; it only combines + classifies + ranks. No AI, no network: pure and tested.
*
* SSOT discipline: the weights, the four band thresholds, and the ten band action strings
* below MIRROR references/trend-scoring-modes.md (the human source of truth). score.test.ts
* pins all three against the SSOT values so silent drift in any of them fails loudly. The
* ORDERING of the weights is the signal; the exact percentages are a documented choice, not
* a measured coefficient (SSOT "How to read this file").
*/
export type ScoreMode = "kortform" | "long-form";
export type DimensionScores = Record<string, number>;
/** kortform weights (SSOT "Mode: kortform"). Sigma = 1.0. */
export const KORTFORM_WEIGHTS = {
pillar: 0.3,
audience: 0.25,
timing: 0.2,
angle: 0.15,
authority: 0.1,
} as const;
/** long-form weights (SSOT "Mode: long-form"). Sigma = 1.0. */
export const LONG_FORM_WEIGHTS = {
pillar: 0.3,
depth: 0.25,
angle: 0.2,
authority: 0.15,
currency: 0.1,
} as const;
const WEIGHTS: Record<ScoreMode, Record<string, number>> = {
kortform: KORTFORM_WEIGHTS,
"long-form": LONG_FORM_WEIGHTS,
};
export type Priority = "Immediate" | "High" | "Medium" | "Low" | "Skip";
export interface Band {
priority: Priority;
kortformAction: string;
longformAction: string;
}
/**
* Composite->band map (SSOT "Composite -> action"). Descending by `min`; the first band
* whose `min` the composite reaches wins. Thresholds + action strings are pinned by
* score.test.ts against the SSOT, so any drift here fails the gate.
*/
const BANDS: ReadonlyArray<{ readonly min: number } & Band> = [
{ min: 8.0, priority: "Immediate", kortformAction: "Draft within 24h", longformAction: "Promote to the edition backlog now" },
{ min: 6.0, priority: "High", kortformAction: "Publish within 48–72h", longformAction: "Strong edition candidate — schedule it" },
{ min: 4.0, priority: "Medium", kortformAction: "Add to this week's calendar", longformAction: "Hold as a backlog candidate, revisit" },
{ min: 2.0, priority: "Low", kortformAction: "Note, skip for now", longformAction: "Park unless the angle sharpens" },
{ min: 0, priority: "Skip", kortformAction: "Off positioning", longformAction: "Off positioning" },
];
function round1(x: number): number {
return Math.round(x * 10) / 10;
}
function toBand(b: { readonly min: number } & Band): Band {
return { priority: b.priority, kortformAction: b.kortformAction, longformAction: b.longformAction };
}
/**
* Weighted composite on the shared 0-10 scale, rounded to 1 decimal (the SSOT's display
* granularity). Validates each of the mode's five dimensions in [1,10]; a missing or
* out-of-range dimension throws — the scores are model output, and a bad one is a contract
* violation, not a value to silently clamp.
*/
export function composite(scores: DimensionScores, mode: ScoreMode): number {
const weights = WEIGHTS[mode];
let sum = 0;
for (const [dim, weight] of Object.entries(weights)) {
const value = scores[dim];
if (typeof value !== "number" || Number.isNaN(value) || value < 1 || value > 10) {
throw new RangeError(`dimension "${dim}" must be a number in [1,10] (got ${String(value)})`);
}
sum += value * weight;
}
return round1(sum);
}
/** Map a composite to its priority band + the mode-specific action strings. */
export function band(composite: number): Band {
for (const b of BANDS) {
if (composite >= b.min) return toBand(b);
}
// composite < 0 (off the scale) — classify as Skip rather than throw; band is a classifier.
return toBand(BANDS[BANDS.length - 1]);
}
export interface TriageOptions {
mode: ScoreMode;
threshold: number;
}
export type Triaged<T> = T & { composite: number; band: Band };
/**
* Score each candidate, keep composite >= threshold (ranked composite-desc), drop below
* (also composite-desc). Each returned entry is annotated with its composite + band. Pure.
*/
export function triage<T extends { scores: DimensionScores }>(
candidates: T[],
opts: TriageOptions,
): { kept: Array<Triaged<T>>; dropped: Array<Triaged<T>> } {
const annotated: Array<Triaged<T>> = candidates.map((c) => {
const comp = composite(c.scores, opts.mode);
return { ...c, composite: comp, band: band(comp) };
});
const byCompositeDesc = (a: Triaged<T>, b: Triaged<T>) => b.composite - a.composite;
const kept = annotated.filter((a) => a.composite >= opts.threshold).sort(byCompositeDesc);
const dropped = annotated.filter((a) => a.composite < opts.threshold).sort(byCompositeDesc);
return { kept, dropped };
}

70
scripts/trends/tests/cli.test.ts Normal file
View file

@ -0,0 +1,70 @@
import { describe, test } from "node:test";
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";
import { fileURLToPath } from "node:url";
// Resolve the package root (scripts/trends) so the subprocess `src/cli.ts` path + the
// `tsx` loader resolve regardless of the runner's cwd.
const trendsDir = fileURLToPath(new URL("..", import.meta.url));
function run(args: string[], input: string): { status: number | null; stdout: string } {
const res = spawnSync("node", ["--import", "tsx", "src/cli.ts", ...args], {
input,
encoding: "utf8",
cwd: trendsDir,
});
return { status: res.status, stdout: res.stdout };
}
describe("trends CLI — normalize/score subcommands (RE-R1 / Step 4)", () => {
describe("normalize (stdin JSON in, JSON out)", () => {
test("happy path: a JSON batch on stdin -> exit 0 + {items,errors} JSON", () => {
const batch = JSON.stringify([
{ source: "tavily", title: "Good", url: "https://example.com/a", topics: ["AI", "ai"] },
{ source: "tavily", title: "", url: "https://example.com/b", topics: ["x"] }, // bad: empty title
]);
const { status, stdout } = run(["normalize"], batch);
assert.equal(status, 0);
const out = JSON.parse(stdout);
assert.equal(out.items.length, 1);
assert.deepEqual(out.items[0].topics, ["ai"]); // deduped + lowercased
assert.equal(out.errors.length, 1);
assert.equal(out.errors[0].index, 1);
});
test("bad invocation: unparseable stdin -> exit 2", () => {
const { status } = run(["normalize"], "not json at all");
assert.equal(status, 2);
});
});
describe("score (stdin JSON in, JSON out)", () => {
test("happy path: scored candidates on stdin -> exit 0 + {kept,dropped} JSON", () => {
const candidates = JSON.stringify([
{ id: "high", scores: { pillar: 8, audience: 8, timing: 8, angle: 8, authority: 8 } }, // 8.0
{ id: "low", scores: { pillar: 2, audience: 2, timing: 2, angle: 2, authority: 2 } }, // 2.0
]);
const { status, stdout } = run(["score", "--mode", "kortform", "--threshold", "4.0"], candidates);
assert.equal(status, 0);
const out = JSON.parse(stdout);
assert.deepEqual(
out.kept.map((k: { id: string }) => k.id),
["high"],
);
assert.equal(out.kept[0].composite, 8.0);
assert.equal(out.kept[0].band.priority, "Immediate");
assert.deepEqual(
out.dropped.map((d: { id: string }) => d.id),
["low"],
);
});
test("bad invocation: an unknown --mode -> exit 2", () => {
const candidates = JSON.stringify([
{ id: "x", scores: { pillar: 5, audience: 5, timing: 5, angle: 5, authority: 5 } },
]);
const { status } = run(["score", "--mode", "bogus"], candidates);
assert.equal(status, 2);
});
});
});

182
scripts/trends/tests/item.test.ts Normal file
View file

@ -0,0 +1,182 @@
import { describe, test } from "node:test";
import assert from "node:assert/strict";
import { normalizeItem, normalizeItems } from "../src/item.js";
import { normalizeField } from "../src/store.js";
describe("trends item normalizer (RE-R1 / B1)", () => {
describe("normalizeItem — well-formed", () => {
test("a well-formed raw item normalizes to a canonical item (string fields verbatim)", () => {
const raw = {
source: "tavily",
title: "OpenAI ships a new reasoning model",
url: "https://example.com/Article-Path",
topics: ["ai", "reasoning"],
summary: "A short summary.",
};
const res = normalizeItem(raw);
assert.equal(res.ok, true);
if (!res.ok) return;
assert.equal(res.item.source, "tavily");
assert.equal(res.item.title, "OpenAI ships a new reasoning model"); // verbatim, case preserved
assert.equal(res.item.url, "https://example.com/Article-Path"); // verbatim, case-sensitive path
assert.equal(res.item.summary, "A short summary.");
assert.deepEqual(res.item.topics, ["ai", "reasoning"]);
});
test("topics are normalized (lowercase + whitespace) and deduped, order-stable", () => {
const res = normalizeItem({
source: "manual",
title: "T",
url: "https://example.com/t",
topics: ["AI", " Machine Learning ", "ai", "Machine Learning"],
});
assert.equal(res.ok, true);
if (!res.ok) return;
// "AI"/"ai" dedupe -> "ai"; " Machine Learning "/"Machine Learning" dedupe -> "machine learning"
assert.deepEqual(res.item.topics, ["ai", "machine learning"]);
// each topic equals store.normalizeField of the raw form (the same normalization)
assert.equal(res.item.topics[1], normalizeField(" Machine Learning "));
});
test("the canonical item carries NO id (the store derives it via addTrend->trendId)", () => {
const res = normalizeItem({
source: "tavily",
title: "No id here",
url: "https://example.com/x",
topics: ["x"],
});
assert.equal(res.ok, true);
if (!res.ok) return;
assert.equal((res.item as Record<string, unknown>).id, undefined);
assert.equal(Object.prototype.hasOwnProperty.call(res.item, "id"), false);
});
test("summary is optional — absent -> no summary key", () => {
const res = normalizeItem({ source: "manual", title: "T", url: "https://example.com/t", topics: ["x"] });
assert.equal(res.ok, true);
if (!res.ok) return;
assert.equal("summary" in res.item, false);
});
test("topics absent -> empty topics array", () => {
const res = normalizeItem({ source: "manual", title: "T", url: "https://example.com/t" });
assert.equal(res.ok, true);
if (!res.ok) return;
assert.deepEqual(res.item.topics, []);
});
});
describe("normalizeItem — required-field validation", () => {
for (const field of ["source", "title", "url"] as const) {
test(`missing ${field} -> {ok:false} naming the field`, () => {
const base: Record<string, unknown> = {
source: "tavily",
title: "T",
url: "https://example.com/t",
topics: ["x"],
};
delete base[field];
const res = normalizeItem(base);
assert.equal(res.ok, false);
if (res.ok) return;
assert.ok(
res.errors.some((e) => e.includes(field)),
`error should name ${field}: ${res.errors.join("; ")}`,
);
});
test(`empty/whitespace ${field} -> {ok:false} naming the field (no silent partial)`, () => {
const base: Record<string, unknown> = {
source: "tavily",
title: "T",
url: "https://example.com/t",
topics: ["x"],
};
base[field] = " ";
const res = normalizeItem(base);
assert.equal(res.ok, false);
if (res.ok) return;
assert.ok(res.errors.some((e) => e.includes(field)));
});
}
test("a non-object raw -> {ok:false}", () => {
const res = normalizeItem("not an object" as unknown);
assert.equal(res.ok, false);
});
});
describe("normalizeItem — publishedAt", () => {
test("present and valid ISO date -> kept", () => {
const res = normalizeItem({
source: "tavily",
title: "T",
url: "https://example.com/t",
topics: ["x"],
publishedAt: "2026-06-20",
});
assert.equal(res.ok, true);
if (!res.ok) return;
assert.equal(res.item.publishedAt, "2026-06-20");
});
test("absent -> undefined (no key)", () => {
const res = normalizeItem({ source: "tavily", title: "T", url: "https://example.com/t", topics: ["x"] });
assert.equal(res.ok, true);
if (!res.ok) return;
assert.equal(res.item.publishedAt, undefined);
assert.equal("publishedAt" in res.item, false);
});
test("present but invalid -> {ok:false} naming publishedAt", () => {
const res = normalizeItem({
source: "tavily",
title: "T",
url: "https://example.com/t",
topics: ["x"],
publishedAt: "not-a-date",
});
assert.equal(res.ok, false);
if (res.ok) return;
assert.ok(res.errors.some((e) => e.includes("publishedAt")));
});
test("present but impossible calendar date -> {ok:false}", () => {
const res = normalizeItem({
source: "tavily",
title: "T",
url: "https://example.com/t",
topics: ["x"],
publishedAt: "2026-13-45",
});
assert.equal(res.ok, false);
});
});
describe("normalizeItems — batch partition", () => {
test("partitions a batch into {items, errors} with error indices", () => {
const raw = [
{ source: "tavily", title: "Good A", url: "https://example.com/a", topics: ["x"] },
{ source: "tavily", title: "", url: "https://example.com/b", topics: ["y"] }, // bad: empty title
{ source: "manual", title: "Good C", url: "https://example.com/c", topics: ["z", "z"] },
];
const { items, errors } = normalizeItems(raw);
assert.equal(items.length, 2);
assert.equal(errors.length, 1);
assert.equal(errors[0].index, 1);
assert.ok(errors[0].errors.some((e) => e.includes("title")));
assert.deepEqual(
items.map((i) => i.title),
["Good A", "Good C"],
);
assert.deepEqual(items[1].topics, ["z"]); // deduped
});
test("an empty batch -> empty partition", () => {
const { items, errors } = normalizeItems([]);
assert.deepEqual(items, []);
assert.deepEqual(errors, []);
});
});
});

145
scripts/trends/tests/score.test.ts Normal file
View file

@ -0,0 +1,145 @@
import { describe, test } from "node:test";
import assert from "node:assert/strict";
import { KORTFORM_WEIGHTS, LONG_FORM_WEIGHTS, composite, band, triage } from "../src/score.js";
const r1 = (x: number) => Math.round(x * 10) / 10;
const sum = (o: Record<string, number>) => Object.values(o).reduce((a, b) => a + b, 0);
describe("trends scorer (RE-R1 / B2)", () => {
// SSOT: references/trend-scoring-modes.md — weights, band thresholds, and action
// strings are pinned here so silent drift in any of them fails loudly.
describe("pinned weights (SSOT)", () => {
test("kortform weights match the SSOT and sum to 1.0", () => {
assert.equal(KORTFORM_WEIGHTS.pillar, 0.3);
assert.equal(KORTFORM_WEIGHTS.audience, 0.25);
assert.equal(KORTFORM_WEIGHTS.timing, 0.2);
assert.equal(KORTFORM_WEIGHTS.angle, 0.15);
assert.equal(KORTFORM_WEIGHTS.authority, 0.1);
assert.equal(r1(sum(KORTFORM_WEIGHTS)), 1.0);
});
test("long-form weights match the SSOT and sum to 1.0", () => {
assert.equal(LONG_FORM_WEIGHTS.pillar, 0.3);
assert.equal(LONG_FORM_WEIGHTS.depth, 0.25);
assert.equal(LONG_FORM_WEIGHTS.angle, 0.2);
assert.equal(LONG_FORM_WEIGHTS.authority, 0.15);
assert.equal(LONG_FORM_WEIGHTS.currency, 0.1);
assert.equal(r1(sum(LONG_FORM_WEIGHTS)), 1.0);
});
});
describe("composite", () => {
test("all-tens -> exactly 10.0 (proves Sigma weights = 1.0) for both modes", () => {
const kort = { pillar: 10, audience: 10, timing: 10, angle: 10, authority: 10 };
const long = { pillar: 10, depth: 10, angle: 10, authority: 10, currency: 10 };
assert.equal(composite(kort, "kortform"), 10.0);
assert.equal(composite(long, "long-form"), 10.0);
});
test("asymmetric golden vector {10,8,6,4,2} in dimension order -> 7.0 for both modes", () => {
// 10*.30 + 8*.25 + 6*.20 + 4*.15 + 2*.10 = 3.0 + 2.0 + 1.2 + 0.6 + 0.2 = 7.0
const kort = { pillar: 10, audience: 8, timing: 6, angle: 4, authority: 2 };
const long = { pillar: 10, depth: 8, angle: 6, authority: 4, currency: 2 };
assert.equal(composite(kort, "kortform"), 7.0);
assert.equal(composite(long, "long-form"), 7.0);
});
test("a dimension below 1 throws", () => {
const kort = { pillar: 0, audience: 5, timing: 5, angle: 5, authority: 5 };
assert.throws(() => composite(kort, "kortform"), /range|1.*10|dimension/i);
});
test("a dimension above 10 throws", () => {
const kort = { pillar: 11, audience: 5, timing: 5, angle: 5, authority: 5 };
assert.throws(() => composite(kort, "kortform"));
});
test("a missing dimension throws", () => {
const kort = { pillar: 5, audience: 5, timing: 5, angle: 5 }; // authority missing
assert.throws(() => composite(kort as Record<string, number>, "kortform"));
});
});
describe("band — boundaries + exact SSOT action strings", () => {
test("8.0 -> Immediate", () => {
const b = band(8.0);
assert.equal(b.priority, "Immediate");
assert.equal(b.kortformAction, "Draft within 24h");
assert.equal(b.longformAction, "Promote to the edition backlog now");
});
test("6.0 -> High", () => {
const b = band(6.0);
assert.equal(b.priority, "High");
assert.equal(b.kortformAction, "Publish within 48–72h");
assert.equal(b.longformAction, "Strong edition candidate — schedule it");
});
test("4.0 -> Medium", () => {
const b = band(4.0);
assert.equal(b.priority, "Medium");
assert.equal(b.kortformAction, "Add to this week's calendar");
assert.equal(b.longformAction, "Hold as a backlog candidate, revisit");
});
test("2.0 -> Low", () => {
const b = band(2.0);
assert.equal(b.priority, "Low");
assert.equal(b.kortformAction, "Note, skip for now");
assert.equal(b.longformAction, "Park unless the angle sharpens");
});
test("below 2.0 -> Skip", () => {
const b = band(1.9);
assert.equal(b.priority, "Skip");
assert.equal(b.kortformAction, "Off positioning");
assert.equal(b.longformAction, "Off positioning");
});
test("just below a boundary lands in the lower band (7.9->High, 5.9->Medium, 3.9->Low)", () => {
assert.equal(band(7.9).priority, "High");
assert.equal(band(5.9).priority, "Medium");
assert.equal(band(3.9).priority, "Low");
});
});
describe("triage", () => {
const candidates = [
{ id: "low", scores: { pillar: 2, audience: 2, timing: 2, angle: 2, authority: 2 } }, // 2.0
{ id: "high", scores: { pillar: 8, audience: 8, timing: 8, angle: 8, authority: 8 } }, // 8.0
{ id: "mid", scores: { pillar: 5, audience: 5, timing: 5, angle: 5, authority: 5 } }, // 5.0
{ id: "below", scores: { pillar: 3, audience: 3, timing: 3, angle: 3, authority: 3 } }, // 3.0
];
test("keeps composite >= threshold, drops below, ranks kept composite-desc, annotates", () => {
const { kept, dropped } = triage(candidates, { mode: "kortform", threshold: 4.0 });
assert.deepEqual(
kept.map((k) => k.id),
["high", "mid"],
); // 8.0, 5.0 desc; both >= 4.0
assert.deepEqual(
dropped.map((d) => d.id).sort(),
["below", "low"],
); // 3.0, 2.0 < 4.0
assert.equal(kept[0].composite, 8.0);
assert.equal(kept[0].band.priority, "Immediate");
assert.equal(kept[1].composite, 5.0);
assert.equal(kept[1].band.priority, "Medium");
});
test("threshold is inclusive (composite == threshold is kept)", () => {
const { kept } = triage(candidates, { mode: "kortform", threshold: 5.0 });
assert.deepEqual(
kept.map((k) => k.id),
["high", "mid"],
); // mid == 5.0 kept
});
test("an empty candidate list -> empty kept/dropped", () => {
const { kept, dropped } = triage([], { mode: "kortform", threshold: 4.0 });
assert.deepEqual(kept, []);
assert.deepEqual(dropped, []);
});
});
});