open/ktg-plugin-marketplace
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions
ktg-plugin-marketplace/plugins/ultraplan-local/tests/synthetic/review-run-A.md
Kjell Tore Guttormsen 0c0a87e709 test(ultraplan-local): add plan-determinism + review-determinism synthetic fixtures (SC7 floor) 
Adds 6 files in tests/synthetic/ exercising the determinism pipeline at the
SC7 brief floor (Jaccard >= 0.833). Plan fixture pair: 40 step titles each
with 38 shared (Jaccard 0.905). Review fixture pair: 30 finding-IDs each
with 28 shared (Jaccard 0.875). Reuses lib/parsers/jaccard.mjs +
lib/parsers/finding-id.mjs.

The new pair coexists with tests/lib/review-determinism.test.mjs which
holds the older SC4 (0.70) floor against tests/fixtures/ultrareview/.
The lower floor protects pipeline regressions; the higher floor anchors
the speedup brief's determinism aspiration.

[skip-docs]
2026-05-04 08:46:39 +02:00

2.6 KiB
Raw Blame History

type review_version created task slug run_id verdict findings
ultrareview-synthetic 1.0 2026-05-04 Add JWT authentication with refresh-token rotation jwt-auth-synthetic A WARN
44b18cf6b84fcb23ef1d52682504c2edeed24f66
f7e307a427154c2c15df4c63eaff6fd846e075a7
31fa81fa5bf9b84c70864ee09aa8d087870c473a
bfc0e3a7c1a5b13dbdc6ed8325140100b02db45d
be76c6dba12bfd9073b1737de5813e316a158dc6
f0928545e7c1dc48796fe857138fab7f100ce8c7
4189ba4236119184017fd26735bfb582706994e9
46f07246ff17c013740c0726b7be9a65fff10c67
5501c54bda4a39df17d66938f4a7fe872e365a0f
0173116735f75aabab36ecec863cb429d2f30528
8f7fc683dc78d3adea8d35221915839702869af0
ee986665d695ca46c9a7f0d5c38bab73e73450a9
d863b17426ddec54bf7624405f3b64e206a73ed7
64ea0bbf43c44dbf0da53f25755e0112ce2eb08b
6971113644b777a8c164dfd8473739b03d1796be
65f6edb11fed982b921ff018bd0fb1dcd10a1703
9133851cf557f5955301803479936733b296f125
ffb170a0d19e4afac6379e64d26485883267bea8
89f990535da373f5e97a091e5bbbf47a777c13d6
664d4ec53e90ef6d24525a85b8d4071bfb037da8
137db625a1ee639698c9e095e25845ef25879599
6e586f167fac4cd57dc8178ceb4ca265a37404dc
24671775282593381af4a8fa77eb3f7a36f9f84e
71dbed32baf440d94f0ccaa6a997a6922cee7679
5de9b2b26d03590845183d42387fcb22007b3f5d
c9aca8c3a265e2f083d75ac6da3e6d67909091b9
75f32c9d304b742af2a7bafc354ec3666e53c054
6547dfd19035bc012a50c19f4321fcfc9535fec8
7554bc48226406e85282c7daeaba75cc732f4b35
4f48547385c2d343ee0994d825321e6e6b90c89d

Synthetic review run A — JWT authentication with refresh-token rotation

This fixture represents one synthesized run of /ultrareview-local on a hand-calibrated brief. It is paired with review-run-B.md for the review-determinism.test.mjs Jaccard floor (≥ 0.833).

How this fixture is used

tests/synthetic/review-determinism.test.mjs reads the findings array from this file's frontmatter and computes jaccardSimilarity(findingsA, findingsB). The test asserts the similarity is at or above the SC7 brief threshold (0.833).

This fixture is distinct from tests/fixtures/ultrareview/review-run-A.md, which feeds the existing tests/lib/review-determinism.test.mjs against the v1.0 SC4 floor (0.70). The synthetic pair pushes the floor higher per SC7.

Fixture math

  • A has 30 unique finding-IDs
  • B has 30 unique finding-IDs
  • Intersection (shared IDs): 28
  • Union: 32
  • Jaccard: 28/32 = 0.875 (above 0.833 floor)

Each ID is the SHA-1 of a synthetic file:line:rule_key triple per lib/parsers/finding-id.mjs. The shared 28 represent stable findings; the 2 unique-per-side represent paraphrase variation in file:line anchoring.