feat(linkedin-studio): N17 — baseline-motor (median + variansbånd + minimum-N-refusal) [skip-docs]
Every reading now leads with "vs your own baseline", and no verdict is given when N is too small to carry one. - stats.ts: median + medianAbsoluteDeviation (robust pair; mean/stddev stay for the alert engine, which wants outlier sensitivity), rollingBaseline with a 10-post positional window, median ± 1·MAD band floored at 0, and a typed insufficient-data refusal below MIN_BASELINE_N=5. readAgainstBaseline returns above/within/below-band, or no-verdict when the baseline was refused. - baselineByGroup + buildBaselineBlock: per-format/per-pillar baselines, each judged on its own N; the reported period is excluded from its own baseline and compared on its median, not its mean. - queue-join.ts (new): read-only date join supplying format/pillar from the post queue. Every ambiguity resolves to unlabelled, an entry labels at most one post, and a missing/broken queue degrades to no labels. - weekly/monthly reports attach the block unconditionally (refusal included); optional in the types, so pre-N17 reports load unchanged. - CLI: report output leads with the baseline; new `baseline [--by format|pillar]` verb with coverage reporting. - report.md leads with baseline framing and prints the code's reading rather than judging the band by eye; WoW loses to the baseline on disagreement. analyze.md Step 2a tests whether the drop is real before diagnosing it. TDD: 58 analytics tests written red first (144 -> 202). test-runner Section 16x, 23 unconditional checks + self-test (247 -> 270; anti-erosion floor 228 -> 251). tsc clean. All suites green: trends 300, brain 134, editions 72, specifics-bank 45, contract-gate 33, hooks 191, tests 35, render 60. Also closes the OKF phase-4 scope follow-up in docs/okf-ingestion/plan.md §8 (coord round 2026-07-25): phase 4 tracks the contract, parse is in scope, and our claim on read_concept/navigate_bundle is withdrawn as unnecessary. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxvWAjte7vPcF79QeSRvRJ
This commit is contained in:
parent
63506f7d5c
commit
e2ad190dda
15 changed files with 1723 additions and 14 deletions
|
|
@ -2709,6 +2709,227 @@ fi
|
|||
|
||||
echo ""
|
||||
|
||||
# --- Section 16x: Baseline Engine - Median, Band, Refusal (N17 / E#5) ---
|
||||
echo "--- Baseline Engine: median + band + minimum-N refusal (N17) ---"
|
||||
|
||||
# At a normal cadence a week holds two or three posts, so an absolute total answers
|
||||
# nothing on its own and a week-over-week percentage answers worse than nothing: it
|
||||
# swings on sample composition and reads as a trend. The engine's whole value is in
|
||||
# two disciplines, and BOTH fail silently if eroded:
|
||||
# (E#5 robust) median and MAD, never mean and stddev. One viral post would
|
||||
# otherwise define "normal" as a number no ordinary post reaches -
|
||||
# and the operator would read every ordinary week as a decline.
|
||||
# (E#5 refusal) below the minimum N there is NO verdict. This is the check most
|
||||
# likely to be "helpfully" removed later, because a refusal looks
|
||||
# like missing functionality rather than the feature it is.
|
||||
# (E#5 honesty) the baseline excludes the reported period (else the week is partly
|
||||
# compared with itself), the band is floored at 0, an unlabelled post
|
||||
# is never bucketed, and each group is judged on its OWN n.
|
||||
# (rendering) the report LEADS with the baseline and prints the code's reading
|
||||
# rather than eyeballing the band; the refusal is never softened
|
||||
# into a hedge.
|
||||
STATS_N17="scripts/analytics/src/utils/stats.ts"
|
||||
TYPES_N17="scripts/analytics/src/models/types.ts"
|
||||
QJOIN_N17="scripts/analytics/src/utils/queue-join.ts"
|
||||
WEEKLY_N17="scripts/analytics/src/reports/weekly.ts"
|
||||
MONTHLY_N17="scripts/analytics/src/reports/monthly.ts"
|
||||
RPT_N17="commands/report.md"
|
||||
ANA_N17="commands/analyze.md"
|
||||
|
||||
refusal_honest() { # $1 = text; honest iff it REFUSES a verdict, NAMES the threshold, and FORBIDS a substitute
|
||||
echo "$1" | grep -qF "insufficient-data" \
|
||||
&& echo "$1" | grep -qF "without" \
|
||||
&& echo "$1" | grep -qF "do not call it a trend"
|
||||
}
|
||||
|
||||
RF_SELFTEST_OK=1
|
||||
if ! refusal_honest "when status is insufficient-data, report the numbers without a verdict and do not call it a trend"; then
|
||||
RF_SELFTEST_OK=0; echo " non-vacuity FAIL: a fully-honest refusal probe was not detected"
|
||||
fi
|
||||
while IFS= read -r probe; do
|
||||
[ -z "$probe" ] && continue
|
||||
if refusal_honest "$probe"; then
|
||||
RF_SELFTEST_OK=0; echo " false-positive FAIL: under-specified refusal probe accepted -> $probe"
|
||||
fi
|
||||
done <<'NEGATIVE16X'
|
||||
when data is thin, report the numbers without a verdict and do not call it a trend
|
||||
status insufficient-data means show the week-over-week percentage without a verdict instead
|
||||
insufficient-data: say the trend is early but real, without a firm verdict
|
||||
NEGATIVE16X
|
||||
if [ "$RF_SELFTEST_OK" -eq 1 ]; then
|
||||
pass "refusal self-test: predicate needs the status token + no-verdict + no-trend-substitute (1 accepted, 3 under-specified rejected)"
|
||||
else
|
||||
fail "refusal self-test failed - the N17 refusal lint is vacuous or over-eager"
|
||||
fi
|
||||
|
||||
# (E#5 robust) the robust primitives exist and are documented as the outlier-proof pair
|
||||
if grep -qF "export function median" "$STATS_N17" 2>/dev/null \
|
||||
&& grep -qF "export function medianAbsoluteDeviation" "$STATS_N17" 2>/dev/null; then
|
||||
pass "stats exposes median + MAD, the outlier-robust pair the baseline is built on (E#5)"
|
||||
else
|
||||
fail "$STATS_N17 has no median/MAD - the baseline would rest on mean/stddev (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 robust) the reason mean/stddev were NOT used is written down, so it survives a refactor
|
||||
if grep -qF "one viral post" "$STATS_N17" 2>/dev/null; then
|
||||
pass "stats records WHY the baseline is median-based (one viral post must not define normal)"
|
||||
else
|
||||
fail "$STATS_N17 does not state why median/MAD replace mean/stddev - the choice will be undone (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 robust) absent data stays absent: an empty history is not a zero baseline
|
||||
if grep -qF "number | undefined" "$STATS_N17" 2>/dev/null \
|
||||
&& grep -qF "0 would be a fabricated baseline" "$STATS_N17" 2>/dev/null; then
|
||||
pass "median returns unknown for an empty history, never 0 (same discipline as the reach roll-up)"
|
||||
else
|
||||
fail "$STATS_N17 may return 0 for an empty history - a fabricated baseline (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 refusal) the threshold is a named constant with a stated rationale, and overridable
|
||||
if grep -qF "export const MIN_BASELINE_N" "$STATS_N17" 2>/dev/null \
|
||||
&& grep -qF "documented default, overridable per call" "$STATS_N17" 2>/dev/null; then
|
||||
pass "the minimum-N threshold is a named, justified, overridable constant (E#5)"
|
||||
else
|
||||
fail "$STATS_N17 has no justified MIN_BASELINE_N - the threshold is a magic number (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 refusal) the window is in POSTS, and the reason a calendar window was rejected is kept
|
||||
if grep -qF "export const BASELINE_WINDOW" "$STATS_N17" 2>/dev/null \
|
||||
&& grep -qF "silently shrinks" "$STATS_N17" 2>/dev/null; then
|
||||
pass "the window is counted in posts, with the calendar-window failure mode documented"
|
||||
else
|
||||
fail "$STATS_N17 window is unnamed or undocumented - a calendar window will creep back in (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 refusal) refusal is a first-class RESULT, not an exception or a null
|
||||
if grep -qF '"insufficient-data"' "$TYPES_N17" 2>/dev/null \
|
||||
&& grep -qF "BaselineRefusal" "$TYPES_N17" 2>/dev/null; then
|
||||
pass "the refusal is a typed result the caller must handle, not a null or a throw (E#5)"
|
||||
else
|
||||
fail "$TYPES_N17 has no BaselineRefusal type - a refusal can be lost at the call site (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 refusal) a refused baseline cannot be read as a verdict downstream
|
||||
if grep -qF '"no-verdict"' "$TYPES_N17" 2>/dev/null \
|
||||
&& grep -qF "readAgainstBaseline" "$STATS_N17" 2>/dev/null; then
|
||||
pass "a refused baseline propagates as no-verdict through the reading function (E#5)"
|
||||
else
|
||||
fail "$STATS_N17 reading path can produce a verdict from a refused baseline (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 honesty) the reported period is EXCLUDED from its own baseline
|
||||
if grep -qF "STRICTLY BEFORE" "$STATS_N17" 2>/dev/null \
|
||||
&& grep -qF "historyBefore" "$TYPES_N17" 2>/dev/null; then
|
||||
pass "the baseline excludes the reported period, so the comparison is not partly with itself"
|
||||
else
|
||||
fail "$STATS_N17 does not exclude the reported period from its own baseline (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 honesty) the period is compared on its MEDIAN, not its mean
|
||||
if grep -qF "period's MEDIAN, not its mean" "$STATS_N17" 2>/dev/null; then
|
||||
pass "the period is read against the band on its median (one viral post cannot fake a trend)"
|
||||
else
|
||||
fail "$STATS_N17 may compare a period mean against a median band (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 honesty) the band cannot go negative on a non-negative metric
|
||||
if grep -qF "Math.max(0, mid - k * mad)" "$STATS_N17" 2>/dev/null; then
|
||||
pass "the band's lower bound is floored at 0 for naturally non-negative metrics"
|
||||
else
|
||||
fail "$STATS_N17 band can present a negative lower bound (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 honesty) per-group refusal: each group judged on its own n, no phantom groups
|
||||
if grep -qF "export function baselineByGroup" "$STATS_N17" 2>/dev/null \
|
||||
&& grep -qF "judged on ITS OWN count" "$STATS_N17" 2>/dev/null; then
|
||||
pass "grouped baselines are judged per group, so a barely-used format gets a refusal (E#5)"
|
||||
else
|
||||
fail "$STATS_N17 grouped baseline can borrow the overall n for a thin group (E#5)"
|
||||
fi
|
||||
|
||||
# (E#5 honesty) an unlabelled post is skipped, never collected into a group called "undefined"
|
||||
if grep -qF "never collected into a phantom group" "$STATS_N17" 2>/dev/null; then
|
||||
pass "unlabelled posts are skipped rather than bucketed as a format that does not exist"
|
||||
else
|
||||
fail "$STATS_N17 may bucket unlabelled posts into a phantom group (E#5)"
|
||||
fi
|
||||
|
||||
# (join) the format/pillar join refuses to guess, and never reuses one entry twice
|
||||
if grep -qF "resolves to UNLABELLED" "$QJOIN_N17" 2>/dev/null \
|
||||
&& grep -qF "labels at most one post" "$QJOIN_N17" 2>/dev/null; then
|
||||
pass "the queue join leaves ambiguity unlabelled and never double-counts an entry (E#5)"
|
||||
else
|
||||
fail "$QJOIN_N17 join may guess a label or reuse an entry - it would poison the baseline (E#5)"
|
||||
fi
|
||||
|
||||
# (join) a missing or broken queue degrades to "no labels", never takes the report down
|
||||
if grep -qF "degrade to \"no labels\"" "$QJOIN_N17" 2>/dev/null; then
|
||||
pass "a missing/malformed queue yields no labels instead of failing the report"
|
||||
else
|
||||
fail "$QJOIN_N17 does not document the degrade-to-unlabelled contract (E#5)"
|
||||
fi
|
||||
|
||||
# (wiring) both period reports carry the block, refusal included - absence is ambiguous
|
||||
if grep -qF "buildBaselineBlock" "$WEEKLY_N17" 2>/dev/null \
|
||||
&& grep -qF "buildBaselineBlock" "$MONTHLY_N17" 2>/dev/null; then
|
||||
pass "weekly and monthly reports both attach a baseline block (E#5)"
|
||||
else
|
||||
fail "a period report does not attach a baseline block - the framing is unavailable (E#5)"
|
||||
fi
|
||||
|
||||
if grep -qF "refusal included" "$WEEKLY_N17" 2>/dev/null; then
|
||||
pass "the weekly block is always present, refusal included (absent != pre-N17 report)"
|
||||
else
|
||||
fail "$WEEKLY_N17 may omit the baseline block, which is indistinguishable from an old report"
|
||||
fi
|
||||
|
||||
# (rendering) the report LEADS with the baseline and does not compute the verdict itself
|
||||
if grep -qF "LEADS with the operator's own baseline" "$RPT_N17" 2>/dev/null \
|
||||
&& grep -qF "Do not compute the verdict yourself" "$RPT_N17" 2>/dev/null; then
|
||||
pass "/linkedin:report leads with the baseline and prints the code's reading (E#5)"
|
||||
else
|
||||
fail "$RPT_N17 does not lead with the baseline, or judges the band by eye (E#5)"
|
||||
fi
|
||||
|
||||
# (rendering) within-band is stated as normal variation, not dressed up as momentum
|
||||
if grep -qF "normal variation" "$RPT_N17" 2>/dev/null \
|
||||
&& grep -qF "not a trend" "$RPT_N17" 2>/dev/null; then
|
||||
pass "an in-band week is reported as normal variation rather than as momentum"
|
||||
else
|
||||
fail "$RPT_N17 may present in-band variation as a trend (E#5)"
|
||||
fi
|
||||
|
||||
# (rendering) the refusal is honest: named status, no verdict, no substitute metric
|
||||
if refusal_honest "$(cat "$RPT_N17" 2>/dev/null)"; then
|
||||
pass "/linkedin:report refuses the verdict on thin data without substituting a percentage (E#5)"
|
||||
else
|
||||
fail "report.md softens or skips the minimum-N refusal (E#5)"
|
||||
fi
|
||||
|
||||
# (rendering) week-over-week loses to the baseline when they disagree
|
||||
if grep -qF "the baseline wins" "$RPT_N17" 2>/dev/null; then
|
||||
pass "report.md resolves a WoW-vs-baseline disagreement in the baseline's favour (E#5)"
|
||||
else
|
||||
fail "$RPT_N17 does not say which frame wins when WoW and baseline disagree (E#5)"
|
||||
fi
|
||||
|
||||
# (rendering) the diagnostic surface verifies the problem is real BEFORE diagnosing it
|
||||
if grep -qF "establish whether there is a problem at all" "$ANA_N17" 2>/dev/null \
|
||||
&& grep -qF "within-band" "$ANA_N17" 2>/dev/null; then
|
||||
pass "/linkedin:analyze checks the drop against the band before diagnosing it (E#5)"
|
||||
else
|
||||
fail "$ANA_N17 diagnoses a reported drop without first testing it against the baseline (E#5)"
|
||||
fi
|
||||
|
||||
# (rendering) analyze refuses to upgrade a refusal into a diagnosis
|
||||
if grep -qF "Never upgrade a refusal into a verdict" "$ANA_N17" 2>/dev/null; then
|
||||
pass "analyze keeps a no-verdict provisional instead of diagnosing on self-report alone"
|
||||
else
|
||||
fail "$ANA_N17 may turn a no-verdict into a confident diagnosis (E#5)"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
|
||||
# --- Section 18: Assertion-Count Anti-Erosion (SC6) ---
|
||||
# The lint self-modifies its own checks, so a green run could mask a silently dropped
|
||||
# assertion. Pin the total pass()+fail() invocations as a monotonic floor; the count
|
||||
|
|
@ -2779,12 +3000,22 @@ echo ""
|
|||
# weighted roll-up/zero-weight grep + import reach-column grep + report reach-split/
|
||||
# never-estimate compound grep + report reach do-next grep + A2-F11 gated-update
|
||||
# compound grep + shipped-template write-ban grep + boundary-map reach/unverified grep +
|
||||
# boundary-map dwell/saves grep + data-README entry-rules grep) = 228.
|
||||
# boundary-map dwell/saves grep + data-README entry-rules grep) = 228; +23 for N17's
|
||||
# twenty-three UNCONDITIONAL Section-16x checks (refusal self-test + median/MAD grep +
|
||||
# why-median rationale grep + empty-history-unknown grep + MIN_BASELINE_N justified-
|
||||
# constant grep + BASELINE_WINDOW posts-not-calendar grep + BaselineRefusal typed-result
|
||||
# grep + no-verdict propagation grep + excludes-reported-period grep + period-median-not-
|
||||
# mean grep + band-floored-at-0 grep + per-group own-n grep + no-phantom-group grep +
|
||||
# join-unlabelled/no-reuse grep + join-degrade-to-no-labels grep + weekly+monthly block
|
||||
# wiring grep + always-present/refusal-included grep + report leads-with-baseline/no-eyeball
|
||||
# compound grep + normal-variation-not-trend grep + report refusal-honest compound grep +
|
||||
# WoW-loses-to-baseline grep + analyze verify-before-diagnose grep + analyze never-upgrade-
|
||||
# refusal grep) = 251.
|
||||
# NB: the floor tracks the deps-absent MINIMUM (conditional TS suites warn-skip and drop
|
||||
# the count), so it is bumped only by UNCONDITIONAL new checks — NOT pinned to the
|
||||
# deps-present TOTAL_CHECKS (that would zero the warn-skip margin and false-fail a fresh
|
||||
# clone). Runs last so TOTAL_CHECKS sees every prior check.
|
||||
ASSERT_BASELINE_FLOOR=228
|
||||
ASSERT_BASELINE_FLOOR=251
|
||||
TOTAL_CHECKS=$((PASS + FAIL))
|
||||
if [ "$TOTAL_CHECKS" -ge "$ASSERT_BASELINE_FLOOR" ]; then
|
||||
pass "assertion-count anti-erosion: $TOTAL_CHECKS checks >= baseline floor $ASSERT_BASELINE_FLOOR"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue