Kjell Tore Guttormsen
d7751c0b9a
hit@8 = 5 of 6, every hit at rank 1, against a chance baseline of 1.35 of 6 over a denominator of 629 concepts per question. Wall time 0.51-0.56 s per question; spent 17 970 - 74 838 bytes against a 120 000 limit. Two things this measurement did NOT establish, both in the report: - BOTH known-negative controls FAILED. A question the bundle has no answer to still returns eight excerpts, because no natural Norwegian question is lexically disjoint from a 629-concept corpus under a four-character shared-prefix rule -- measured per token, the interrogative `hvor` reaches 40 concepts, `brukes` 83. So `no_lexical_match` works per concept and not as a whole-question gate: an empty excerpt list is evidence of absence, a full one is not evidence of presence. The fix is named (rarity weighting) and NOT built, because this step's fence freezes the instrument before it is measured. - The question texts were written during execution, after the ranker existed. The plan recorded the gold documents' SIZE profile -- its per-row baselines sum to 1.35 and the sizes used here reproduce that exactly, which is an independent check that this is the set the plan profiled -- but it recorded no question texts, and three of six gold documents could not be pinned uniquely from the sizes. Not a blind evaluation, and the report says so. The scorer is a tool, not a script in a document: `tools/okf_consume_measure.py` takes the gold set as an INPUT because it is tracked in a public repository and an answer key names a consumer's documents. hit_rank, both chance baselines and the document-size census are unit-tested; the corpus run is a measurement. Public-file rule, checked with a pattern DERIVED from the corpus's own 39 document names rather than hand-picked, and shown able to find first (67 hits on the bundle's own index): zero corpus document names in any tracked file in this repository. One leak was found and removed on the way -- a corpus concept name in a code comment and a hardcoded corpus path in a test. Suite run after git add: 1230 passed, mypy --strict clean on 27 files, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
403 B
403 B
| type | title | source_file | source_sha256 | ingested_at | adjudication | bundle_id | verified | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| reference | Pristabell | Pristabell.xlsx | 1111111111111111111111111111111111111111111111111111111111111111 | 2026-09-01T00:00:00Z | proposed | consume-fixture |
|
Pristabell
Prisene fylles ut i dette skjemaet. Summen av alle poster overfoeres til tilbudsbrevet.