open/voyage
1
0
Fork 0
Code Issues Releases 1 Activity

fix(storm-measure): restore the pre-registered OR decision rule

Browse source 
The brief pre-registers "median forbedring >= 30 % pa (a) eller (b) ->
adopt. < 15 % -> decline." decideVerdict() required BOTH metrics to clear
the adopt bar and BOTH to fall under the decline bar, a rule introduced at
execution time (plan Step 11 restated the brief's wording unchanged).

The divergence is behavioural, not cosmetic: sources +90% / dimensions +10%
is adopt under the brief and was inconclusive in code; sources +2% /
dimensions +20% is decline under the brief and was inconclusive in code.

Adopt is evaluated before decline, so the OR bars do not conflict where they
overlap. docs/storm-measurement.md §2 and the test pins follow the same rule.

The stricter AND rule may well be the better decision procedure — but
changing it after the fact is exactly what pre-registration exists to
prevent. Review finding c37bf50d.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011vPSXe88qp5aqWUqbDNWoF
This commit is contained in:
Kjell Tore Guttormsen 2026-08-12 22:15:07 +02:00
commit 4c4457f6e9
3 changed files with 34 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

20
tests/scripts/storm-measure.test.mjs
View file

@ -168,8 +168,8 @@ test('decideVerdict: at and above the adopt threshold', () => {
assert.equal(decideVerdict(0.55, 0.44), 'adopt');
});
test('decideVerdict: just below the adopt threshold is inconclusive, not adopt', () => {
assert.equal(decideVerdict(ADOPT_THRESHOLD - 0.0001, 0.9), 'inconclusive');
test('decideVerdict: both metrics between the bars is inconclusive, not adopt', () => {
assert.equal(decideVerdict(ADOPT_THRESHOLD - 0.0001, 0.2), 'inconclusive');
});
test('decideVerdict: below the decline threshold on both metrics declines', () => {
@ -181,8 +181,20 @@ test('decideVerdict: at the decline threshold is inconclusive, not decline', ()
assert.equal(decideVerdict(DECLINE_THRESHOLD, DECLINE_THRESHOLD), 'inconclusive');
});
test('decideVerdict: adopt needs BOTH metrics — one strong metric does not carry a weak one', () => {
assert.equal(decideVerdict(0.90, 0.10), 'inconclusive');
// The brief pre-registers "median forbedring >= 30 % på (a) eller (b) → adopt.
// < 15 % → decline." — OR on both sides, with adopt evaluated first.
test('decideVerdict: adopt needs EITHER metric — one strong metric carries a weak one', () => {
assert.equal(decideVerdict(0.90, 0.10), 'adopt');
assert.equal(decideVerdict(0.10, 0.90), 'adopt');
});
test('decideVerdict: either metric below the decline bar declines', () => {
assert.equal(decideVerdict(0.02, 0.20), 'decline');
assert.equal(decideVerdict(0.20, 0.02), 'decline');
});
test('decideVerdict: adopt outranks decline when one metric clears and the other is under the decline bar', () => {
assert.equal(decideVerdict(0.90, 0.10), 'adopt');
});
test('decideVerdict: a null gain is insufficient data, never a decline', () => {