open/voyage
1
0
Fork 0
Code Issues Releases 1 Activity

fix(storm-measure): restore the pre-registered OR decision rule

Browse source 
The brief pre-registers "median forbedring >= 30 % pa (a) eller (b) ->
adopt. < 15 % -> decline." decideVerdict() required BOTH metrics to clear
the adopt bar and BOTH to fall under the decline bar, a rule introduced at
execution time (plan Step 11 restated the brief's wording unchanged).

The divergence is behavioural, not cosmetic: sources +90% / dimensions +10%
is adopt under the brief and was inconclusive in code; sources +2% /
dimensions +20% is decline under the brief and was inconclusive in code.

Adopt is evaluated before decline, so the OR bars do not conflict where they
overlap. docs/storm-measurement.md §2 and the test pins follow the same rule.

The stricter AND rule may well be the better decision procedure — but
changing it after the fact is exactly what pre-registration exists to
prevent. Review finding c37bf50d.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011vPSXe88qp5aqWUqbDNWoF
This commit is contained in:
Kjell Tore Guttormsen 2026-08-12 22:15:07 +02:00
commit 4c4457f6e9
3 changed files with 34 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

15
docs/storm-measurement.md
View file

@ -31,14 +31,17 @@ below `effort: high`.
## 2. Pre-registered thresholds
Both metrics must clear the bar. A strong result on one axis does not carry a
weak result on the other — the loop's claim is breadth on both.
Either metric clearing the bar is enough. The brief pre-registers "median
forbedring ≥ 30 % på (a) eller (b) → adopt. < 15 % → decline" — a breadth win
on one axis counts, because either axis widening is the effect the loop claims.
Adopt is evaluated first, so a run that clears the adopt bar on one metric is
an adopt even when the other metric sits under the decline bar.
| Median gain (BOTH metrics) | Verdict | Action |
| Median gain | Verdict | Action |
|---|---|---|
| ≥ 30% | **adopt** | Flip the `VOYAGE_STORM_ENABLED` default (see §5) |
| < 15% | **decline** | Leave the mechanism default-off. This is a **no-op**: nothing is rolled back |
| 15% – 30% | **inconclusive** | Keep default-off, gather more runs, re-measure |
| ≥ 30% on **either** metric | **adopt** | Flip the `VOYAGE_STORM_ENABLED` default (see §5) |
| < 15% on **either** metric (and no adopt) | **decline** | Leave the mechanism default-off. This is a **no-op**: nothing is rolled back |
| both metrics in 15% – 30% | **inconclusive** | Keep default-off, gather more runs, re-measure |
| either arm empty | **insufficient-data** | Not a decline — measure more |
`ADOPT_THRESHOLD = 0.30` and `DECLINE_THRESHOLD = 0.15` are exported constants