feat(p16): the stress judge -- and the order's own (a) was a gate that could only be green
Session 102's criterion ((a) built on the right fasit concept OR refused anchored, (b') names it,
(c) zero hallucinations) was adjudicated BY HAND. Measured 14.09: nothing in the tree read
contexts/<set>/fasit.json against an outbox at all, so "provable against the base" had no
repeatable form. portfolio_optimiser.stress reads ONLY artefacts that already exist -- the
per-approach proposal/outcome pair and {run_id}-debate.json -- so no run gains a field.
MEASURED BEFORE BUILDING: the order defines grounded as "OPENED or CITED", but on the S2c path
run_project stamps citations = bundle_citations(bundle), one per context file. On n100-2023 that
is 446 citations over 446 concepts, and 6 of 6 fasit paths are already "cited" before a single
model call. Honouring it literally would be the repo's own vacuous-gate class inside the gate
built to catch it, so a citation grounds an approach only under a NARROWED list (a declared
pre-pass cut); both halves are reported either way. (b') was checked for the same vacuity and is
clean -- snippets are bodies, ref/title live in frontmatter (0 of 446 n100 bodies carry
"Krav 4.1.2-1") -- so the order's definition stands.
A2: unanswerable questions had no runnable form (po is not a lookup tool), so they become a FOURTH
commissioned approach per set whose cost line the base carries no ground for, and fasit.json
carries must_refuse INSTEAD of unanswerable -- one form, never two copies of one fact. Rule U is
untouched and its known-positive is still red.
Load-bearing MEASURED (20 arms), eleven mutations all red on their own arm, green control 1663/5
(from 1643/5, superset, 0 removed), golden demo-transcript.stdout BYTE-UNCHANGED
(shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
f13dc64a0a
commit
f21007c858
13 changed files with 1008 additions and 87 deletions
|
|
@ -18,13 +18,22 @@ rather than vacuously green: "the anchor was not found" is equally true of a bas
|
|||
read (Verifiseringsloven, ansikt 4).
|
||||
|
||||
**Rule U** — the measurable form of "the base cannot answer this" (documented in
|
||||
``docs/2026-09-12-p14-kontekstsett.md § 2.3``): each unanswerable question declares >= 1 ``anchor``,
|
||||
``docs/2026-09-12-p14-kontekstsett.md § 2.3``): each ``must_refuse`` row declares >= 1 ``anchor``,
|
||||
a lowercase word of >= 4 characters, and is admitted **iff every anchor is absent — case-insensitive
|
||||
substring — from the WHOLE text (frontmatter + body) of EVERY concept document in the base**. Not
|
||||
"shares no keyword with any title": a tunnel question shares "tunnel" with hundreds of titles and
|
||||
that proves nothing. What makes a question unanswerable is that the base lacks the SUBJECT, and the
|
||||
anchor is that subject. Titles alone would be a proxy the full text costs nothing more to replace
|
||||
(measured: 0.77 s for r761-2025, the largest base).
|
||||
|
||||
**P16 A2 moved rule U from ``unanswerable`` to ``must_refuse``, and that is ONE form rather than
|
||||
two.** ``po`` is not a lookup tool (D-1), so an "unanswerable question" had no runnable form: no
|
||||
execution path ever consumed those rows, and the falsification half of session 102's criterion was
|
||||
therefore unprovable by a run. The same fact now rides a FOURTH commissioned approach per set
|
||||
(``a4-…``) whose cost line the base carries no ground for, and ``must_refuse`` names it by
|
||||
``approach_id`` plus the same anchors. Keeping the questions beside it as a second key would be two
|
||||
copies of one fact (kø-(p)) — the questions survive inside the row's ``rationale``, which nothing
|
||||
keys on. Rule U itself is UNCHANGED, and its known-positive is still red.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
|
@ -143,7 +152,7 @@ def anchors_are_absent(
|
|||
if not concepts:
|
||||
raise ValueError("rule U ran against ZERO concepts: absence here is unmeasured, not false")
|
||||
if not anchors:
|
||||
raise ValueError("an unanswerable question declares no anchors, so nothing was checked")
|
||||
raise ValueError("a must_refuse row declares no anchors, so nothing was checked")
|
||||
carried = []
|
||||
for anchor in anchors:
|
||||
if anchor != anchor.lower() or len(anchor) < _MIN_ANCHOR_CHARS:
|
||||
|
|
@ -217,9 +226,14 @@ def test_the_fasit_names_every_commissioned_approach(set_dir: Path) -> None:
|
|||
fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))
|
||||
mandate = load_mandate(set_dir / "mandate.json")
|
||||
cited = {row["approach_id"] for row in fasit["must_cite"]}
|
||||
assert cited == {a.id for a in mandate.approaches}
|
||||
refused = {row["approach_id"] for row in fasit["must_refuse"]}
|
||||
assert not (cited & refused), "an approach is either answerable or the falsification arm"
|
||||
assert cited | refused == {a.id for a in mandate.approaches}
|
||||
assert fasit["honesty"].strip(), "DEL 2(iii): what in this set is constructed must be stated"
|
||||
assert len(fasit["unanswerable"]) >= 2, "the order asks for at least two per set"
|
||||
assert refused, "P16 A2: every set carries the falsification arm"
|
||||
for row in fasit["must_refuse"]:
|
||||
assert len(row["anchors"]) >= 2, "the order asks for at least two per set"
|
||||
assert row["rationale"].strip(), "why the base cannot ground it must be stated"
|
||||
assert (set_dir / "docs").is_dir(), "the form declares a docs/ directory even when it is empty"
|
||||
|
||||
|
||||
|
|
@ -263,11 +277,11 @@ def test_c_rule_u_every_unanswerable_question_is_unanswerable(set_dir: Path) ->
|
|||
)
|
||||
|
||||
fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))
|
||||
for row in fasit["unanswerable"]:
|
||||
for row in fasit["must_refuse"]:
|
||||
carried = anchors_are_absent(row["anchors"], concepts)
|
||||
assert not carried, (
|
||||
f"{set_dir.name}: {declared['name']} DOES carry {carried} over {len(concepts)} "
|
||||
f"concepts, so {row['question']!r} is not unanswerable by rule U"
|
||||
f"concepts, so {row['approach_id']!r} is not un-groundable by rule U"
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue