fix(linkedin-studio): S11 — model-name declaration drift + model-consistency lint guard
Cold full-brief re-review (S10) reached a class the S7->S9 algorithm-stat lens never did: BLOCKER — post-feedback-monitor published as Haiku in four surfaces (README:259, skills/linkedin-studio:159 with wrong color Green too, skills/linkedin-analytics:41, agents-capability-matrix:20) while agents/post-feedback-monitor.md runs Opus. v4.0.0's Opus promotion never reached the user-facing tables. Synced all to Opus/Lime. Refreshed agents-capability-matrix.md (frozen at the v2.0 14-agent era): header 14->19, +5 missing longform agents, tier counts Opus 2->8 / Haiku 1->0, longform-gate diagram updated to the real 8-Opus-agent chain. MAJOR — de-branded docs/plan-fullspektrum-innholdsmotor.md:70 (model brand + jan-2026 asserted as fact -> no-name/no-month relevance-model phrasing). It was the only tracked survivor; the rest live in gitignored ROADMAP.md / .claude/research/ (not shipped, out of honesty scope). META — added Section 10 model-consistency guard (scripts/check-model-consistency.mjs): each agents/*.md model: must match every surface declaration AND the canonical rosters must list all 19 agents. Permanent non-vacuity self-test + e2e mutation-proven. Pre-patch sweep confirmed post-feedback-monitor was the sole drifted agent (89 model rows, 0 other mismatches). test-runner.sh 68/0/0, node --test 94/94. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
853cad3ade
commit
433c2efb3d
8 changed files with 223 additions and 17 deletions
|
|
@ -1,6 +1,6 @@
|
|||
# LinkedIn Studio Plugin (v4.0.0)
|
||||
|
||||
Full-spectrum LinkedIn content engine — short-form feed posts, carousels, video scripts, and long-form newsletter editions — with the 2026 relevance-ranking model baked in. **v4.0.0** is an **audit-remediation release (Voyage Phase 0–3)**: a critical self-review found overclaiming (tracking/analytics/review-independence the plugin couldn't deliver), dormant capability (11 agents never invoked by any command), and structural rot (a dead lint, a self-contradicting algorithm claim, an unpublishable model brand/date in user copy). The fix wires **all 11 orphaned agents** (no deletions → 19 agents), adds **`/linkedin:firsthour`** (→ 27 commands) + a short-form de-AI gate + a video quality gate, promotes `post-feedback-monitor` to Opus, makes the newsletter-distribution / profile-SEO / outreach surfaces honest, **reconciles the algorithm signals to one sourced statement** (no model name or date; `references/algorithm-signals-reference.md` is the single source of truth), fixes the analytics fresh-clone crash, closes the voice-profile leak (placeholder + sentinel + gitignore), and rebuilds the structure lint with version/count/stat-consistency guards. Breaking — reinstall/reload required for the newly-wired agents; consolidates the v3.0.0 identity break (slug, agent namespace, state-file path). v2.0.0 consolidated the surface (27 commands → 24, 16 agents → 14) while adding the long-form `/linkedin:newsletter` orchestrator + two longform-quality gate agents (`fact-checker`, `persona-reviewer`). v2.1.0 added two gates BEFORE prose (Step 2.5 skeleton + Step 3a spine prose) + a third `persona-reviewer` mode (`skjelett`). v2.2.0 hardened the longform gates with the lessons from the second production run (Seres-serien): blocking persona hard-fails, a post-cutoff fact-check mandate, a `voice-scrubber` agent, render+annotate operator gates, and STATE.md-reconciled edition state. v2.3.0 made **visual assets an explicit pipeline phase** — Step 7.5 (visual-assets) between annotation (Step 7) and lock (Step 8): cover (+ optional inline figures) or a carousel deck, generated (default `mcp-image`, external `cover-raw.png` accepted) and operator-gated BEFORE lock so `render/build-linkedin.mjs` picks up `cover.png` at lock without a post-lock re-render. **v2.4.0** makes an **editor's craft gate an explicit pipeline phase** — new **Step 5.5 (editorial-review)** between fact-check (Step 5) and the persona sweep (Step 6): a new **`editorial-reviewer` agent** (Opus) judges **craft** (prosa-håndverk + narrativ-arkitektur), not reader-response, returning ≤10 flags (BLOCK/REWORK/NICE) as direction, **operator-gated via `SendUserFile` BEFORE the persona sweep** so the personas measure resonance instead of stumbling on craft noise. Motivated by Del 4: every persona reported PASS, yet the editor found 8 fresh points on first reading, ~6/8 of them craft/architecture blind spots no agent measured. Mirrors the Maskinrommet writing-contract §C2. Pipeline 14 → 15 phases; agents 15 → 16; additive `editorialReview` state. Doc/orchestration-only for the wiring (the new agent + its fasit fixture + lint test are the only new files); commands unchanged (24). **v3.1.0 (Endring 9)** adds an **adversarial review package** run COLD on a frozen draft — new **Step 6.5 (headless-review)** between the persona sweep (Step 6) and lock, plus a standalone **`/linkedin:headless-review`** command (run in a fresh session for maximum isolation): three new headless archetypes — **`content-reviewer`** (argument integrity), **`language-reviewer`** (Norwegian language), **`fact-reviewer`** (cold re-verification incl. claims a late pivot bolted on) — plus `persona-reviewer` in resonance + conversion modes, all with NO drafting-session context (the independence layer the in-session gates structurally cannot be). v3.1.0 also adds **`/linkedin:pivot`** (re-opens cleared gates after a late change + a >20 %/>2-section pivot-detection gate at lock) and **per-artifact personas** (`articles.NN.personas` — one or more readers configurable per edition, resolved edition-state → series file → plugin library → interactive). Pipeline 15 → 16 phases; agents 16 → 19; commands 24 → 26; additive `personas` / `pivots` / `headlessReview` state. Motivated by Del 4: the in-session editor + persona sweep shared the drafting session's framing-bias, so the version that shipped was never independently re-reviewed.
|
||||
Full-spectrum LinkedIn content engine — short-form feed posts, carousels, video scripts, and long-form newsletter editions — with the 2026 relevance-ranking model baked in. **v4.0.0** is an **audit-remediation release (Voyage Phase 0–3)**: a critical self-review found overclaiming (tracking/analytics/review-independence the plugin couldn't deliver), dormant capability (11 agents never invoked by any command), and structural rot (a dead lint, a self-contradicting algorithm claim, an unpublishable model brand/date in user copy). The fix wires **all 11 orphaned agents** (no deletions → 19 agents), adds **`/linkedin:firsthour`** (→ 27 commands) + a short-form de-AI gate + a video quality gate, promotes `post-feedback-monitor` to Opus, makes the newsletter-distribution / profile-SEO / outreach surfaces honest, **reconciles the algorithm signals to one sourced statement** (no model name or date; `references/algorithm-signals-reference.md` is the single source of truth), fixes the analytics fresh-clone crash, closes the voice-profile leak (placeholder + sentinel + gitignore), and rebuilds the structure lint with version/count/stat/model-consistency guards (each agent's frontmatter model must match every surface declaration). Breaking — reinstall/reload required for the newly-wired agents; consolidates the v3.0.0 identity break (slug, agent namespace, state-file path). v2.0.0 consolidated the surface (27 commands → 24, 16 agents → 14) while adding the long-form `/linkedin:newsletter` orchestrator + two longform-quality gate agents (`fact-checker`, `persona-reviewer`). v2.1.0 added two gates BEFORE prose (Step 2.5 skeleton + Step 3a spine prose) + a third `persona-reviewer` mode (`skjelett`). v2.2.0 hardened the longform gates with the lessons from the second production run (Seres-serien): blocking persona hard-fails, a post-cutoff fact-check mandate, a `voice-scrubber` agent, render+annotate operator gates, and STATE.md-reconciled edition state. v2.3.0 made **visual assets an explicit pipeline phase** — Step 7.5 (visual-assets) between annotation (Step 7) and lock (Step 8): cover (+ optional inline figures) or a carousel deck, generated (default `mcp-image`, external `cover-raw.png` accepted) and operator-gated BEFORE lock so `render/build-linkedin.mjs` picks up `cover.png` at lock without a post-lock re-render. **v2.4.0** makes an **editor's craft gate an explicit pipeline phase** — new **Step 5.5 (editorial-review)** between fact-check (Step 5) and the persona sweep (Step 6): a new **`editorial-reviewer` agent** (Opus) judges **craft** (prosa-håndverk + narrativ-arkitektur), not reader-response, returning ≤10 flags (BLOCK/REWORK/NICE) as direction, **operator-gated via `SendUserFile` BEFORE the persona sweep** so the personas measure resonance instead of stumbling on craft noise. Motivated by Del 4: every persona reported PASS, yet the editor found 8 fresh points on first reading, ~6/8 of them craft/architecture blind spots no agent measured. Mirrors the Maskinrommet writing-contract §C2. Pipeline 14 → 15 phases; agents 15 → 16; additive `editorialReview` state. Doc/orchestration-only for the wiring (the new agent + its fasit fixture + lint test are the only new files); commands unchanged (24). **v3.1.0 (Endring 9)** adds an **adversarial review package** run COLD on a frozen draft — new **Step 6.5 (headless-review)** between the persona sweep (Step 6) and lock, plus a standalone **`/linkedin:headless-review`** command (run in a fresh session for maximum isolation): three new headless archetypes — **`content-reviewer`** (argument integrity), **`language-reviewer`** (Norwegian language), **`fact-reviewer`** (cold re-verification incl. claims a late pivot bolted on) — plus `persona-reviewer` in resonance + conversion modes, all with NO drafting-session context (the independence layer the in-session gates structurally cannot be). v3.1.0 also adds **`/linkedin:pivot`** (re-opens cleared gates after a late change + a >20 %/>2-section pivot-detection gate at lock) and **per-artifact personas** (`articles.NN.personas` — one or more readers configurable per edition, resolved edition-state → series file → plugin library → interactive). Pipeline 15 → 16 phases; agents 16 → 19; commands 24 → 26; additive `personas` / `pivots` / `headlessReview` state. Motivated by Del 4: the in-session editor + persona sweep shared the drafting session's framing-bias, so the version that shipped was never independently re-reviewed.
|
||||
|
||||
## Architecture
|
||||
|
||||
|
|
|
|||
|
|
@ -256,7 +256,7 @@ The plugin delegates specialized work to 19 purpose-built agents. Each agent has
|
|||
| `trend-spotter` | Sonnet | White | Trending topics, opportunity scoring, first-mover assessment |
|
||||
| `voice-trainer` | Sonnet | Pink | Voice profile building and drift detection |
|
||||
| `differentiation-checker` | Sonnet | Gray | Originality scoring and commodity content detection |
|
||||
| `post-feedback-monitor` | Haiku | Lime | Post-publish 48h monitoring and real-time interventions |
|
||||
| `post-feedback-monitor` | Opus | Lime | Post-publish 48h monitoring and real-time interventions |
|
||||
| `video-scripter` | Sonnet | Violet | Video script creation with pacing and visual cues |
|
||||
| `fact-checker` | Opus | Brown | Factual-claim verification against primary/credible sources, post-cutoff web-search mandate (longform) |
|
||||
| `editorial-reviewer` | Opus | Orange | Editor's craft gate (Step 5.5, before persona sweep): **prosa-håndverk** + **narrativ-arkitektur**, ≤10 flags BLOCK/REWORK/NICE as direction, operator-gated via `SendUserFile`; mirrors Maskinrommet §C2 (longform) |
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Agent Capability Matrix
|
||||
|
||||
14 specialized agents for LinkedIn thought leadership. Each agent has a focused responsibility, defined model, and unique color for visual identification.
|
||||
19 specialized agents for LinkedIn content (short-form feed + long-form newsletter). Each agent has a focused responsibility, defined model, and unique color for visual identification.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
|
|
@ -17,13 +17,18 @@
|
|||
| voice-trainer | Sonnet | Pink | Voice profile building and drift detection |
|
||||
| differentiation-checker | Sonnet | Gray | Originality scoring and commodity detection |
|
||||
| video-scripter | Sonnet | Violet | Video script creation with pacing and visual cues |
|
||||
| post-feedback-monitor | Haiku | Lime | Post-publish 48h monitoring and real-time interventions |
|
||||
| post-feedback-monitor | Opus | Lime | Post-publish 48h monitoring and real-time interventions |
|
||||
| fact-checker | Opus | Brown | Factual-claim verification against primary/credible sources (longform) |
|
||||
| persona-reviewer | Opus | Olive | Reader-persona resonance + hook-conversion gate (longform) |
|
||||
| editorial-reviewer | Opus | Orange | Editor's craft gate: prosa-håndverk + narrativ-arkitektur (longform) |
|
||||
| persona-reviewer | Opus | Olive | Reader-persona skeleton + resonance + hook-conversion gate (longform) |
|
||||
| voice-scrubber | Opus | Red | De-AI scrub + Norwegian-chronicle voice-drift correction (longform) |
|
||||
| content-reviewer | Opus | Maroon | Cold argument-integrity review (C1–C5) on a frozen draft (longform) |
|
||||
| language-reviewer | Opus | Navy | Cold Norwegian-language review (L1–L5) on a frozen draft (longform) |
|
||||
| fact-reviewer | Opus | Gold | Cold fact re-verification (F1–F4) + pivot-risk on a frozen draft (longform) |
|
||||
|
||||
## Capability Matrix
|
||||
|
||||
Capabilities mapped across agents. **P** = Primary, **S** = Secondary/Supporting.
|
||||
Capabilities mapped across the 14 content-production agents (the columns below). **P** = Primary, **S** = Secondary/Supporting. The five remaining agents — editorial-reviewer, voice-scrubber, content-reviewer, language-reviewer, fact-reviewer — are ordered long-form quality gates rather than content-capability agents; they are documented in **Longform Quality Gates** below and listed in the Quick Reference above.
|
||||
|
||||
| Capability | optimizer | strategy | analytics | engage | planner | network | repurpose | trends | voice | diff-check | video | post-monitor | fact-check | persona-rev |
|
||||
|-----------|:---------:|:--------:|:---------:|:------:|:-------:|:-------:|:---------:|:------:|:-----:|:----------:|:-----:|:------------:|:----------:|:-----------:|
|
||||
|
|
@ -108,13 +113,19 @@ How agents collaborate in the end-to-end content lifecycle:
|
|||
|
||||
### Longform Quality Gates (newsletter)
|
||||
|
||||
For longform editions, two additional Opus agents run BEFORE lock:
|
||||
For longform editions, eight Opus agents run as ordered gates BEFORE lock:
|
||||
|
||||
```
|
||||
draft ─▸ fact-checker ─▸ persona-reviewer ─▸ LOCK ─▸ delivery
|
||||
(primary-source (resonance +
|
||||
verification) hook-conversion
|
||||
gate)
|
||||
draft
|
||||
─▸ fact-checker (primary-source verification, post-cutoff web search)
|
||||
─▸ editorial-reviewer (craft: prosa-håndverk + narrativ-arkitektur, Step 5.5)
|
||||
─▸ persona-reviewer (skeleton → resonance → hook-conversion)
|
||||
─▸ voice-scrubber (de-AI + Norwegian-chronicle voice)
|
||||
─▸ headless review (Step 6.5 — COLD on the frozen draft):
|
||||
content-reviewer (argument integrity C1–C5)
|
||||
language-reviewer (Norwegian language L1–L5)
|
||||
fact-reviewer (cold re-verification F1–F4 + pivot-risk)
|
||||
─▸ LOCK ─▸ delivery
|
||||
```
|
||||
|
||||
### Parallel Support Agents
|
||||
|
|
@ -159,6 +170,5 @@ content-repurposer ────▸ Post-publish: extends content lifecycle
|
|||
|
||||
| Model | Agents | Why |
|
||||
|-------|--------|-----|
|
||||
| **Opus** | 2 agents (fact-checker, persona-reviewer) | Longform judgment: factual verification, reader-persona resonance |
|
||||
| **Opus** | 8 agents (fact-checker, editorial-reviewer, persona-reviewer, voice-scrubber, content-reviewer, language-reviewer, fact-reviewer, post-feedback-monitor) | Longform judgment + 48h post-monitoring: factual verification, craft, resonance, voice, cold adversarial re-review, real-time intervention |
|
||||
| **Sonnet** | 11 agents | Complex reasoning: optimization, strategy, analysis, scoring, scripting, comment targeting |
|
||||
| **Haiku** | 1 agent (post-feedback-monitor) | Lighter task: post monitoring with anomaly detection |
|
||||
|
|
|
|||
|
|
@ -67,7 +67,7 @@ Nøkkelfiler i svv-memory for dypere kontekst: `project_kronikk_produksjonsprose
|
|||
- **POST.html «alt på ett sted»** — ett selvforsynt publiseringsark per edition, generert av `build-linkedin.mjs` (0.6). Inneholder i publiseringsrekkefølge: dato + klokkeslett (+ ev. ferskvare-banner), tittel/SEO, cover-filnavn + credit + caption, delingstekst inkl. hashtags, første kommentar, ev. carousel-referanse, og brødteksten som rik tekst klar til innliming. Formålet: legg inn én edition i én operasjon uten å hoppe mellom filer.
|
||||
- **Ferskvare-flagg** — markering i POST.html av tids-sensitive tall (f.eks. en selskaps-verdsettelse) som MÅ re-verifiseres på publiseringsdagen. En publiseringsdag-sjekkliste, ikke en tekstsvakhet.
|
||||
- **voice-profil / voice-samples** — operatørens skrivestemme. Lagret i `config/user-profile.local.md` + `assets/voice-samples/` i pluginen. **Leses ALLTID før innholdsproduksjon** (eksisterende LTL-regel).
|
||||
- **360Brew** — LinkedIns rangerings-/anbefalingsmodell (oppdatering jan. 2026). LTL optimaliserer innhold mot dens signaler.
|
||||
- **Relevans-/rangeringsmodell** — LinkedIns 2026 relevans-/rangeringsmodell (modellnavn og oppdateringsdato ikke publiserbart som faktum; se `references/algorithm-signals-reference.md`). LTL optimaliserer innhold mot dens signaler.
|
||||
- **5x5x5** — eksisterende LTL-engasjementstaktikk (kommenter/engasjer i et mønster rundt egen post). Ikke relevant for langform.
|
||||
- **CEA** — kommenteringsmetode i `comment-strategist`-agenten. Ikke relevant for langform.
|
||||
- **PreToolUse-gate / content-gatekeeper / voice-guardian** — eksisterende LTL-hooks som gir rådgivende kvalitets-/stemme-advarsler når kortform-innhold skrives. Kalibrert for feed-poster.
|
||||
|
|
|
|||
174
plugins/linkedin-studio/scripts/check-model-consistency.mjs
Normal file
174
plugins/linkedin-studio/scripts/check-model-consistency.mjs
Normal file
|
|
@ -0,0 +1,174 @@
|
|||
#!/usr/bin/env node
|
||||
// Agent Model-Consistency guard (remediation S11).
|
||||
//
|
||||
// The source of truth for an agent's model is its own frontmatter
|
||||
// (`agents/<name>.md` → `model:`). Every user-facing surface that DECLARES an
|
||||
// agent's model in a table — README.md, CLAUDE.md, docs/agents-capability-matrix.md,
|
||||
// and the skills/*/SKILL.md rosters — must declare that same model. The v4.0.0
|
||||
// Opus promotion of `post-feedback-monitor` reached the agent frontmatter and
|
||||
// CLAUDE.md but NOT README/SKILL/matrix, so the README publicly stated a false
|
||||
// fact (Haiku) about a shipped Opus agent. The structure lint had version/count/
|
||||
// stat guards but no per-agent model-consistency guard, so nothing failed on it.
|
||||
// This closes that meta-gap: agent-model drift now fails the same suite that
|
||||
// defines the registration contract.
|
||||
//
|
||||
// Two checks:
|
||||
// 1. MODEL-CORRECTNESS — every agent row in ANY model-table surface (canonical
|
||||
// rosters + the curated domain SKILLs) must declare the frontmatter model.
|
||||
// 2. ROSTER-COMPLETENESS — the canonical complete-roster surfaces must mention
|
||||
// EVERY agent (this is what catches the matrix frozen at "14 specialized
|
||||
// agents"; the domain SKILLs are deliberately curated subsets and are
|
||||
// exempt from completeness, checked for correctness only).
|
||||
//
|
||||
// A permanent non-vacuity self-test runs BEFORE the real scan on every
|
||||
// invocation (mirrors Section 8's STALE_STATS self-test): a checker that cannot
|
||||
// catch a deliberately-mismatched probe — or that false-flags a correct one —
|
||||
// is not enforcing the criterion, so it fails the suite instead of silently
|
||||
// certifying nothing. This is the S7→S10 lesson (a proof run once by hand and
|
||||
// never committed lets a survivor slip) applied to the model axis.
|
||||
//
|
||||
// Zero dependencies (node:fs only). bash 3.2-safe caller: invoked from
|
||||
// scripts/test-runner.sh Section 10, exit code mapped to pass/fail.
|
||||
|
||||
import { readdirSync, readFileSync, existsSync } from "node:fs";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { dirname, join } from "node:path";
|
||||
|
||||
const ROOT = join(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const MODEL_RE = /\b(opus|sonnet|haiku)\b/i;
|
||||
|
||||
// Canonical surfaces that MUST list every agent (complete rosters).
|
||||
const CANONICAL = [
|
||||
"README.md",
|
||||
"CLAUDE.md",
|
||||
"docs/agents-capability-matrix.md",
|
||||
"skills/linkedin-studio/SKILL.md",
|
||||
];
|
||||
// Additional surfaces that declare some agent models but are curated subsets
|
||||
// (per-domain SKILLs). Checked for model-correctness only, not completeness.
|
||||
const SUBSET = [
|
||||
"skills/linkedin-analytics/SKILL.md",
|
||||
"skills/linkedin-content-creation/SKILL.md",
|
||||
"skills/linkedin-networking/SKILL.md",
|
||||
"skills/linkedin-strategy/SKILL.md",
|
||||
"skills/linkedin-voice/SKILL.md",
|
||||
];
|
||||
|
||||
// --- Truth from agent frontmatter ---
|
||||
function loadTruth() {
|
||||
const truth = {};
|
||||
for (const f of readdirSync(join(ROOT, "agents")).filter((x) => x.endsWith(".md"))) {
|
||||
const fm = readFileSync(join(ROOT, "agents", f), "utf8").split(/^---$/m)[1] || "";
|
||||
const model = ((fm.match(/^model:\s*(.+)$/m) || [])[1] || "").trim().toLowerCase();
|
||||
truth[f.replace(/\.md$/, "")] = model;
|
||||
}
|
||||
return truth;
|
||||
}
|
||||
|
||||
// --- Core, testable primitives (exercised by the self-test on synthetic input) ---
|
||||
|
||||
// Every table row that names an agent AND carries a short model cell must match.
|
||||
function modelMismatches(text, truth) {
|
||||
const names = Object.keys(truth);
|
||||
const out = [];
|
||||
text.split("\n").forEach((ln, i) => {
|
||||
if (!ln.includes("|")) return;
|
||||
const cells = ln.split("|").map((c) => c.trim());
|
||||
const nameCell = cells.find((c) => names.includes(c.replace(/[`*]/g, "").trim()));
|
||||
if (!nameCell) return;
|
||||
const agent = nameCell.replace(/[`*]/g, "").trim();
|
||||
const modelCell = cells.find((c) => MODEL_RE.test(c) && c.length < 12);
|
||||
if (!modelCell) return;
|
||||
const declared = (modelCell.match(MODEL_RE) || [])[1].toLowerCase();
|
||||
if (declared !== truth[agent]) {
|
||||
out.push({ line: i + 1, agent, declared, truth: truth[agent] });
|
||||
}
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
// Agents not mentioned anywhere in a canonical surface (word-boundaried so
|
||||
// `content-reviewer` does not satisfy `content-repurposer`).
|
||||
function missingAgents(text, names) {
|
||||
return names.filter((n) => !new RegExp("(^|[^a-z-])" + n + "([^a-z-]|$)").test(text));
|
||||
}
|
||||
|
||||
// --- Permanent non-vacuity self-test (runs every invocation, before the scan) ---
|
||||
function selfTest(truth) {
|
||||
const names = Object.keys(truth);
|
||||
const a = names[0];
|
||||
const cap = (m) => m.charAt(0).toUpperCase() + m.slice(1);
|
||||
const other = truth[a] === "opus" ? "sonnet" : "opus";
|
||||
const failures = [];
|
||||
|
||||
// POSITIVE (correctness): a row with the WRONG model must be flagged.
|
||||
const wrongRow = `| \`${a}\` | ${cap(other)} | Lime | desc |`;
|
||||
if (modelMismatches(wrongRow, truth).length !== 1) {
|
||||
failures.push("model mismatch probe not caught");
|
||||
}
|
||||
// NEGATIVE (correctness): a row with the CORRECT model must NOT be flagged.
|
||||
const rightRow = `| \`${a}\` | ${cap(truth[a])} | Lime | desc |`;
|
||||
if (modelMismatches(rightRow, truth).length !== 0) {
|
||||
failures.push("correct model probe false-flagged");
|
||||
}
|
||||
// POSITIVE (completeness): a roster missing one agent must be flagged.
|
||||
const rosterMissing = names.slice(1).map((n) => `\`${n}\``).join(" ");
|
||||
if (!missingAgents(rosterMissing, names).includes(a)) {
|
||||
failures.push("missing-agent probe not caught");
|
||||
}
|
||||
// NEGATIVE (completeness): a full roster must NOT be flagged.
|
||||
const fullRoster = names.map((n) => `\`${n}\``).join(" ");
|
||||
if (missingAgents(fullRoster, names).length !== 0) {
|
||||
failures.push("full roster false-flagged");
|
||||
}
|
||||
return failures;
|
||||
}
|
||||
|
||||
// --- Main ---
|
||||
function main() {
|
||||
const truth = loadTruth();
|
||||
const names = Object.keys(truth);
|
||||
|
||||
const stFailures = selfTest(truth);
|
||||
if (stFailures.length > 0) {
|
||||
console.log("SELFTEST FAIL — model-consistency guard is not enforcing the criterion:");
|
||||
stFailures.forEach((f) => console.log(" - " + f));
|
||||
process.exit(1);
|
||||
}
|
||||
console.log(
|
||||
`self-test OK: model-mismatch + missing-agent probes caught, correct probes ignored (truth = ${names.length} agents)`,
|
||||
);
|
||||
|
||||
const problems = [];
|
||||
|
||||
// Check 1: model-correctness across every model-table surface.
|
||||
for (const rel of [...CANONICAL, ...SUBSET]) {
|
||||
if (!existsSync(join(ROOT, rel))) continue;
|
||||
const text = readFileSync(join(ROOT, rel), "utf8");
|
||||
for (const m of modelMismatches(text, truth)) {
|
||||
problems.push(`${rel}:${m.line} ${m.agent} declared=${m.declared} but frontmatter=${m.truth}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Check 2: roster-completeness on the canonical surfaces.
|
||||
for (const rel of CANONICAL) {
|
||||
if (!existsSync(join(ROOT, rel))) {
|
||||
problems.push(`${rel} MISSING (canonical roster surface)`);
|
||||
continue;
|
||||
}
|
||||
const missing = missingAgents(readFileSync(join(ROOT, rel), "utf8"), names);
|
||||
if (missing.length > 0) {
|
||||
problems.push(`${rel} does not list ${missing.length} agent(s): ${missing.join(", ")}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (problems.length === 0) {
|
||||
console.log(`model-consistency OK: ${names.length} agents, all surface declarations match frontmatter`);
|
||||
process.exit(0);
|
||||
}
|
||||
console.log("model-consistency FAIL — agent model/roster drift:");
|
||||
problems.forEach((p) => console.log(" ✗ " + p));
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
main();
|
||||
|
|
@ -10,7 +10,9 @@
|
|||
#
|
||||
# The stat-consistency grep (one magnitude per algorithm effect across the
|
||||
# tree) was added in remediation Step 3; the version-consistency grep in
|
||||
# Step 21. Both are live below (Sections 8 and 9).
|
||||
# Step 21; the agent model-consistency guard (each agents/<name>.md frontmatter
|
||||
# model: must match every surface declaration, and canonical rosters must list
|
||||
# every agent) in S11. All three are live below (Sections 8, 9 and 10).
|
||||
#
|
||||
# Usage: bash scripts/test-runner.sh
|
||||
# bash 3.2-safe: plain arrays only, no `declare -A`, no `mapfile`/`readarray`.
|
||||
|
|
@ -319,6 +321,26 @@ fi
|
|||
|
||||
echo ""
|
||||
|
||||
# --- Section 10: Agent Model-Consistency ---
|
||||
echo "--- Agent Model-Consistency ---"
|
||||
|
||||
# Each agents/<name>.md frontmatter `model:` is the source of truth; every
|
||||
# surface that DECLARES an agent's model (README, CLAUDE.md, the capability
|
||||
# matrix, the SKILL rosters) must match it, and the canonical rosters must list
|
||||
# every agent. Added in S11 after a cold full-brief review found
|
||||
# post-feedback-monitor published as Haiku across four surfaces while the agent
|
||||
# runs Opus — declaration drift the version/count/stat guards could not see. The
|
||||
# checker self-tests its own non-vacuity on every run (see the .mjs header):
|
||||
# a deliberately-mismatched probe must be caught and a correct one ignored, else
|
||||
# the suite fails instead of certifying an unenforced criterion.
|
||||
if node scripts/check-model-consistency.mjs; then
|
||||
pass "agent model-consistency: all surface declarations match frontmatter + canonical rosters complete"
|
||||
else
|
||||
fail "agent model-consistency drift — see check-model-consistency.mjs output above"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
|
||||
# --- Summary ---
|
||||
echo "================================================"
|
||||
echo "RESULTS"
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@ This skill covers everything related to LinkedIn analytics, performance measurem
|
|||
|-------|-------|----------------|
|
||||
| `analytics-interpreter` | Sonnet | Audience pattern analysis + weekly/monthly performance reports (interpret/report modes) |
|
||||
| `trend-spotter` | Sonnet | Trending topics + opportunity scores |
|
||||
| `post-feedback-monitor` | Haiku | Post-publish 48h monitoring, anomaly detection |
|
||||
| `post-feedback-monitor` | Opus | Post-publish 48h monitoring, anomaly detection |
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -156,7 +156,7 @@ These rules apply to ALL content created by any skill or command:
|
|||
| `trend-spotter` | Sonnet | Cyan | Trending topics + opportunity scores |
|
||||
| `voice-trainer` | Sonnet | Magenta | Voice profile building + drift detection |
|
||||
| `differentiation-checker` | Sonnet | Blue | Originality scoring + commodity detection |
|
||||
| `post-feedback-monitor` | Haiku | Green | Post-publish 48h monitoring, real-time interventions |
|
||||
| `post-feedback-monitor` | Opus | Lime | Post-publish 48h monitoring, real-time interventions |
|
||||
| `video-scripter` | Sonnet | Violet | Video script creation with pacing + visual cues |
|
||||
| `fact-checker` | Opus | Brown | Factual-claim verification against primary/credible sources (longform) |
|
||||
| `persona-reviewer` | Opus | Olive | Reader-persona skeleton + resonance + hook-conversion gate (longform) |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue