Kjell Tore Guttormsen
7b4f85d77c
Four paid stress rounds ran entirely UN-ANCHORED, all of them, because the one file loader reads cost-baseline.json out of the BUNDLE and no vegnormal ships one: N100, N200, N500 and R761 are knowledge, and knowledge carries requirements, never amounts. The validator's stage 0 -- the one stage that tells an invented cost line from a line this project actually buys -- was skipped in every single run, so "validated" could not mean what it says. P20 G1/G2 measured real R761 process numbers (12.11 three times on Soraasen, 1.1.1 on Lindaas) validating with amounts nobody had anywhere. --cost-baseline FILE is PM decision (e), taken over the three alternatives P20 wrote down. A LOADED object, never a path (prepass_payload's rule): the CLI owns the file and loads it ONCE, so the notice, the stamp and every base of an --across-bundle pass all descend from one read. ONE parse, two doors -- load_cost_baseline delegates to load_cost_baseline_file -- while safe_resolve stays on the bundle door alone, because a project's own schedule is legitimately outside every base. No tolerant twin: this path exists only because an operator NAMED a file. DEL B: five anchored context sets, a1-a3 with their line and a4 with none, so stage 0 is what catches the falsification arm. THE ORDER'S OWN ARM (h) WAS FELLED BY MEASUREMENT: "no baseline code is a requirement number the base declares" is measured 0 of 4 on the project-coded sets and 5 of 5 on kontrakt-sorasen -- which is what R761 Prosesskoden IS, a bill of quantities priced BY process code. The complement keeps both, and the order's own mutation still bites. DEL B3: the judge reports anchored (off the run's own stamp), priced per row, and WHICH falsifier caught the falsification arm. Load-bearing MEASURED, five mutations all red against the WHOLE suite, green control 1850/5 (from 1809/5, superset, 0 removed), golden byte-unchanged: A3(i) the flag is read but the baseline is unused (3 red) . A3(ii) only the first base gets it (1) . A3(iii) report_forbidden drops it (1) . B2(i) a4 gets a line (1, arm (g) alone) . B2(ii) a code swapped to 12.11 (2, arms (f) and (h)). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
661 lines
33 KiB
Python
661 lines
33 KiB
Python
"""P14 — the stress-test context sets, gated (order ``20260912T202210Z``).
|
|
|
|
Each set under ``contexts/<project-id>/`` commissions ONE run against ONE knowledge base:
|
|
``mandate.json`` (the ``Mandate`` schema verbatim), ``bundle.txt`` (which base, and the id that base
|
|
declares) and ``fasit.json`` (what a right answer MUST cite, what the base cannot answer, and what
|
|
in the set is constructed rather than real).
|
|
|
|
**Five arms, and the split between them is a measurement rather than a taste.** Two are
|
|
unconditional and can never be silently absent — a mandate that does not load, and a mandate routed
|
|
at a base the set is not for. Three need the base itself, which lives OUTSIDE this repository
|
|
(``PORTFOLIO_VEGNORMAL_ROOT``): they SKIP when the root is missing, exactly as MAJOR-3's ceiling
|
|
gate could not take K2 as a test dependency, and for the same published-package reason — a hard
|
|
failure would break ``uv run pytest`` for any external recipient of the ``git archive HEAD``
|
|
handover. The skip NAMES the root it looked for.
|
|
|
|
**Every bundle-reading arm carries its own denominator.** A scan that sees zero concepts is RED
|
|
rather than vacuously green: "the anchor was not found" is equally true of a base that was never
|
|
read (Verifiseringsloven, ansikt 4).
|
|
|
|
**Rule U** — the measurable form of "the base cannot answer this" (documented in
|
|
``docs/2026-09-12-p14-kontekstsett.md § 2.3``): each ``must_refuse`` row declares >= 1 ``anchor``,
|
|
a lowercase word of >= 4 characters, and is admitted **iff every anchor is absent — case-insensitive
|
|
substring — from the WHOLE text (frontmatter + body) of EVERY concept document in the base**. Not
|
|
"shares no keyword with any title": a tunnel question shares "tunnel" with hundreds of titles and
|
|
that proves nothing. What makes a question unanswerable is that the base lacks the SUBJECT, and the
|
|
anchor is that subject. Titles alone would be a proxy the full text costs nothing more to replace
|
|
(measured: 0.77 s for r761-2025, the largest base).
|
|
|
|
**P16 A2 moved rule U from ``unanswerable`` to ``must_refuse``, and that is ONE form rather than
|
|
two.** ``po`` is not a lookup tool (D-1), so an "unanswerable question" had no runnable form: no
|
|
execution path ever consumed those rows, and the falsification half of session 102's criterion was
|
|
therefore unprovable by a run. The same fact now rides a FOURTH commissioned approach per set
|
|
(``a4-…``) whose cost line the base carries no ground for, and ``must_refuse`` names it by
|
|
``approach_id`` plus the same anchors. Keeping the questions beside it as a second key would be two
|
|
copies of one fact (kø-(p)) — the questions survive inside the row's ``rationale``, which nothing
|
|
keys on. Rule U itself is UNCHANGED, and its known-positive is still red.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import os
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
from pydantic import ValidationError
|
|
|
|
from portfolio_optimiser import okf
|
|
from portfolio_optimiser.ir import CostBaseline, CostBaselineLine
|
|
from portfolio_optimiser.mandate import load_mandate
|
|
from portfolio_optimiser.stress import read_bundle_declarations
|
|
|
|
_REPO_ROOT = Path(__file__).resolve().parent.parent
|
|
_CONTEXT_ROOT = _REPO_ROOT / "contexts"
|
|
|
|
#: Where the vegnormal bases are mounted. A SYMBOLIC name in ``bundle.txt`` is resolved against
|
|
#: this, never an absolute path in the set: this repository is published, and an absolute path
|
|
#: would pin a set to one machine's home directory and ride out in the handover archive.
|
|
_DEFAULT_BUNDLE_ROOT = Path.home() / "repos" / "vegnormal-okf" / "build" / "ferdig"
|
|
|
|
#: The concept types the four bases declare. ``index.md`` carries none of them — it is navigation,
|
|
#: not content — which is why the file count and the concept count differ.
|
|
_CONCEPT_TYPES = {"Krav", "Prosess", "Kapittel", "Normal", "Håndbok"}
|
|
|
|
_MIN_ANCHOR_CHARS = 4
|
|
|
|
|
|
def own_frontmatter(path: Path) -> dict[str, str]:
|
|
"""The concept's OWN frontmatter: top-level keys only, FIRST occurrence winning.
|
|
|
|
**P15 (2026-09-13) fixed the finding this helper was written against.** Before P15,
|
|
``okf.parse_frontmatter`` was linewise and last-write-wins over EVERY line regardless of
|
|
indentation, so a nested block overwrote a top-level key of the same name. Every vegnormal
|
|
concept ends its frontmatter with
|
|
|
|
sources:
|
|
- resource: https://…
|
|
title: N500:2024
|
|
|
|
and the indented ``title`` used to replace the concept's own. MEASURED on n500-2024 before the
|
|
fix: ``okf.navigate_bundle`` yielded 270 concept files carrying **1 distinct title**
|
|
(``N500:2024``, 270 times). ``okf.parse_frontmatter`` now makes indentation load-bearing —
|
|
a top-level (unindented) key always wins over a nested one of the same name — and re-measured
|
|
AFTER the fix, the same base's 269 ``krav/N500`` documents carry **269 distinct titles**.
|
|
|
|
**This helper still isn't a plain call to ``okf.parse_frontmatter``, and that remains
|
|
measured rather than assumed:** ``own_frontmatter`` also strips one layer of enclosing
|
|
``'`` quotes (``.strip("'")``) so a value matches the fasit's stored plain-text title
|
|
verbatim, while ``okf.parse_frontmatter`` deliberately leaves scalars quoted — unquoting is
|
|
``okf.unquote_scalar``'s ONE job (D1/(a)/(i)), and a second copy of that rule here would be
|
|
the drifting one. Re-measured across all four bases (29 500 field reads: ``type``, ``title``,
|
|
``req_number``, ``prosessnr`` on every concept file) the two now agree EXACTLY except for
|
|
quoted scalars (2 728 of 29 500 checks — every one a quote-stripping difference, none a value
|
|
difference), so this helper stays for that one reason, not for the nested-override bug P15
|
|
closed.
|
|
|
|
Uses ``okf._split_frontmatter`` deliberately: it is the module's ONE place ``---`` is compared
|
|
(B4), and a second delimiter rule here would be the copy that drifts.
|
|
"""
|
|
out: dict[str, str] = {}
|
|
for line in okf._split_frontmatter(path.read_text(encoding="utf-8"))[0]:
|
|
if not line or line[0].isspace() or line.lstrip().startswith("-"):
|
|
continue
|
|
key, sep, value = line.partition(":")
|
|
if sep and key.strip() not in out:
|
|
out[key.strip()] = value.strip().strip("'")
|
|
return out
|
|
|
|
|
|
def _bundle_root() -> Path:
|
|
return Path(os.environ.get("PORTFOLIO_VEGNORMAL_ROOT", str(_DEFAULT_BUNDLE_ROOT)))
|
|
|
|
|
|
#: The ONE reader, imported from production rather than copied here (P17b). It used to be a
|
|
#: private copy in this file and a second, looser one inside ``stress.main`` — and the multi-base
|
|
#: form is exactly the change that would have let the two drift into different answers about one
|
|
#: set. A set declaring ONE base is one block, so the four pre-P17b files parse unchanged.
|
|
read_bundle_txt = read_bundle_declarations
|
|
|
|
|
|
def scan_concepts(base: Path) -> list[tuple[str, dict[str, str], str]]:
|
|
"""Every concept document in a base: bundle-relative path, frontmatter, lowercased full text.
|
|
|
|
Built from the declared ``type``, never from the directory listing: ``index.md`` is navigation
|
|
and would otherwise be counted as content.
|
|
"""
|
|
found: list[tuple[str, dict[str, str], str]] = []
|
|
for path in sorted(base.rglob("*.md")):
|
|
text = path.read_text(encoding="utf-8")
|
|
frontmatter = own_frontmatter(path)
|
|
if frontmatter.get("type", "") in _CONCEPT_TYPES:
|
|
found.append((path.relative_to(base).as_posix(), frontmatter, text.lower()))
|
|
return found
|
|
|
|
|
|
def anchors_are_absent(
|
|
anchors: list[str], concepts: list[tuple[str, dict[str, str], str]]
|
|
) -> list[str]:
|
|
"""Rule U: return the anchors the base DOES carry (empty == the question is admitted).
|
|
|
|
:raises ValueError: an empty scan, or an anchor that is not a usable one. Both are refusals
|
|
rather than a quiet pass — a rule that cannot fail proves nothing.
|
|
"""
|
|
if not concepts:
|
|
raise ValueError("rule U ran against ZERO concepts: absence here is unmeasured, not false")
|
|
if not anchors:
|
|
raise ValueError("a must_refuse row declares no anchors, so nothing was checked")
|
|
carried = []
|
|
for anchor in anchors:
|
|
if anchor != anchor.lower() or len(anchor) < _MIN_ANCHOR_CHARS:
|
|
raise ValueError(
|
|
f"anchor {anchor!r} must be lowercase and at least {_MIN_ANCHOR_CHARS} characters"
|
|
)
|
|
if any(anchor in text for _, _, text in concepts):
|
|
carried.append(anchor)
|
|
return carried
|
|
|
|
|
|
def context_sets() -> list[Path]:
|
|
return (
|
|
sorted(p for p in _CONTEXT_ROOT.iterdir() if p.is_dir()) if _CONTEXT_ROOT.is_dir() else []
|
|
)
|
|
|
|
|
|
_SETS = context_sets()
|
|
_SET_IDS = [p.name for p in _SETS]
|
|
|
|
|
|
def _require_base(declared: dict[str, str]) -> Path:
|
|
root = _bundle_root()
|
|
base = root / declared["name"]
|
|
if not base.is_dir():
|
|
pytest.skip(
|
|
f"knowledge base {declared['name']!r} not mounted under {root} (PORTFOLIO_VEGNORMAL_ROOT)"
|
|
)
|
|
return base
|
|
|
|
|
|
def _base_by_approach(set_dir: Path) -> dict[str, Path]:
|
|
"""Which MOUNTED base each approach was routed at (P17b).
|
|
|
|
Read off the mandate's ``bundle_id`` and the set's own declarations — the routing has exactly
|
|
one home, and a second per-row key in the fasit would be the copy free to drift.
|
|
"""
|
|
by_id = {block["bundle_id"]: block for block in read_bundle_txt(set_dir / "bundle.txt")}
|
|
out: dict[str, Path] = {}
|
|
for approach in load_mandate(set_dir / "mandate.json").approaches:
|
|
block = by_id.get(approach.bundle_id)
|
|
assert block is not None, f"{approach.id} routes at an undeclared base"
|
|
out[approach.id] = _require_base(block)
|
|
return out
|
|
|
|
|
|
# --------------------------------------------------------------------------------------------
|
|
# The sets exist at all. Without this, every parametrised arm below would collapse to zero cases
|
|
# and the file would pass by having nothing to say.
|
|
# --------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_the_five_context_sets_are_present() -> None:
|
|
assert len(_SETS) == 5, f"expected five context sets under {_CONTEXT_ROOT}, found {_SET_IDS}"
|
|
|
|
|
|
# --------------------------------------------------------------------------------------------
|
|
# (a) + (d): unconditional — no knowledge base needed.
|
|
# --------------------------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
|
|
def test_a_mandate_loads_fail_fast(set_dir: Path) -> None:
|
|
mandate = load_mandate(set_dir / "mandate.json")
|
|
assert mandate.objective
|
|
assert mandate.success_criteria, f"{set_dir.name} states no success criteria to judge it by"
|
|
assert 2 <= len(mandate.approaches) <= 4, "the order asks for 2-4 approaches per set"
|
|
for approach in mandate.approaches:
|
|
assert approach.affected_codes, f"{approach.id} names no affected_codes"
|
|
assert approach.claimed_saving_nok is not None, f"{approach.id} states no estimate"
|
|
|
|
|
|
@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
|
|
def test_d_every_approach_is_routed_at_this_sets_own_base(set_dir: Path) -> None:
|
|
"""Every approach names ONE of the set's declared bases, and every declared base is named.
|
|
|
|
Both halves are the claim. The first is the original: an approach routed at a base the set is
|
|
not for would be evaluated against a corpus nobody commissioned. The second arrived with the
|
|
multi-base form (P17b) and is what keeps the declaration honest the other way — a base listed
|
|
in ``bundle.txt`` that no approach names is never run (``route_by_bundle``'s own rule, a run
|
|
costs money and the commission ordered nothing for it), so a set declaring it would be
|
|
describing a pass wider than the one it commissions.
|
|
"""
|
|
declared = read_bundle_txt(set_dir / "bundle.txt")
|
|
ids = {block["bundle_id"] for block in declared}
|
|
routed = {approach.bundle_id for approach in load_mandate(set_dir / "mandate.json").approaches}
|
|
assert routed <= ids, (
|
|
f"{set_dir.name}: approaches route at {sorted(routed - ids)}, which the set does not "
|
|
f"declare (declared: {sorted(ids)})"
|
|
)
|
|
assert ids <= routed, (
|
|
f"{set_dir.name}: declares {sorted(ids - routed)} that no approach names, so the set "
|
|
"describes a wider pass than it commissions"
|
|
)
|
|
|
|
|
|
@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
|
|
def test_the_fasit_names_every_commissioned_approach(set_dir: Path) -> None:
|
|
fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))
|
|
mandate = load_mandate(set_dir / "mandate.json")
|
|
cited = {row["approach_id"] for row in fasit["must_cite"]}
|
|
refused = {row["approach_id"] for row in fasit["must_refuse"]}
|
|
assert not (cited & refused), "an approach is either answerable or the falsification arm"
|
|
assert cited | refused == {a.id for a in mandate.approaches}
|
|
assert fasit["honesty"].strip(), "DEL 2(iii): what in this set is constructed must be stated"
|
|
assert refused, "P16 A2: every set carries the falsification arm"
|
|
for row in fasit["must_refuse"]:
|
|
assert len(row["anchors"]) >= 2, "the order asks for at least two per set"
|
|
assert row["rationale"].strip(), "why the base cannot ground it must be stated"
|
|
assert (set_dir / "docs").is_dir(), "the form declares a docs/ directory even when it is empty"
|
|
|
|
|
|
# --------------------------------------------------------------------------------------------
|
|
# (b) + (c) + (e): these read the base itself and SKIP when it is not mounted.
|
|
# --------------------------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
|
|
def test_b_every_fasit_concept_is_in_the_base_as_recorded(set_dir: Path) -> None:
|
|
"""Every cited concept is in the base ITS OWN approach was routed at (P17b).
|
|
|
|
Resolving per approach rather than per set is the multi-base half: in a set spanning two
|
|
bases, checking every path against one of them would fail half the fasit while proving
|
|
nothing about the other, and checking against "either" would let a path meant for N200 be
|
|
satisfied by a coincidence in R761.
|
|
"""
|
|
bases = _base_by_approach(set_dir)
|
|
fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))
|
|
|
|
seen = 0
|
|
for row in fasit["must_cite"]:
|
|
assert row["concepts"], f"{row['approach_id']} cites nothing a right answer must reach"
|
|
base = bases[row["approach_id"]]
|
|
for concept in row["concepts"]:
|
|
path = base / concept["path"]
|
|
assert path.is_file(), f"{set_dir.name}: {concept['path']} is not in {base.name}"
|
|
frontmatter = own_frontmatter(path)
|
|
assert frontmatter.get("title", "") == concept["title"], (
|
|
f"{concept['path']}: the base's own title has drifted from the fasit"
|
|
)
|
|
if concept.get("ref"):
|
|
actual = frontmatter.get("req_number") or frontmatter.get("prosessnr", "")
|
|
assert actual == concept["ref"], (
|
|
f"{concept['path']}: the base's own reference has drifted from the fasit"
|
|
)
|
|
seen += 1
|
|
assert seen > 0, "the fasit named no concepts at all, so nothing was verified"
|
|
|
|
|
|
@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
|
|
def test_c_rule_u_every_unanswerable_question_is_unanswerable(set_dir: Path) -> None:
|
|
"""Rule U over EVERY base the set declares, as ONE scan.
|
|
|
|
For a multi-base set "the base cannot answer this" becomes "NEITHER base can", and the union
|
|
is the honest reading: an anchor absent from N200 but present in R761 is a question the pass
|
|
as a whole CAN reach. MEASURED 15.09 and the reason this is not a formality — ``enhetspris``
|
|
is absent from n200-2024 and carried by 70 of r761-2025's 2 756 concepts, so an anchor set
|
|
admitted per base would have admitted a question the pass could ground.
|
|
"""
|
|
declared = read_bundle_txt(set_dir / "bundle.txt")
|
|
concepts: list[tuple[str, dict[str, str], str]] = []
|
|
names = []
|
|
for block in declared:
|
|
base = _require_base(block)
|
|
names.append(block["name"])
|
|
concepts += scan_concepts(base)
|
|
assert len(concepts) >= 100, (
|
|
f"{', '.join(names)} scanned to {len(concepts)} concepts — too few to be the base(s)"
|
|
)
|
|
|
|
fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))
|
|
for row in fasit["must_refuse"]:
|
|
carried = anchors_are_absent(row["anchors"], concepts)
|
|
assert not carried, (
|
|
f"{set_dir.name}: {', '.join(names)} DOES carry {carried} over {len(concepts)} "
|
|
f"concepts, so {row['approach_id']!r} is not un-groundable by rule U"
|
|
)
|
|
|
|
|
|
@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
|
|
def test_e_the_declared_bundle_id_is_the_bases_own(set_dir: Path) -> None:
|
|
for block in read_bundle_txt(set_dir / "bundle.txt"):
|
|
base = _require_base(block)
|
|
resolved = okf.reconcile_bundle_id(base)
|
|
assert resolved.id == block["bundle_id"], (
|
|
f"{set_dir.name}: bundle.txt declares {block['bundle_id']!r} for {block['name']} but "
|
|
f"the base resolves to {resolved.id!r} (origin {resolved.origin})"
|
|
)
|
|
|
|
|
|
# --------------------------------------------------------------------------------------------
|
|
# KNOWN-POSITIVES (DEL 3): a deliberately broken set must make EXACTLY the arm that guards it red.
|
|
# Without these, a check that can only pass is indistinguishable from a check that never runs.
|
|
# --------------------------------------------------------------------------------------------
|
|
|
|
|
|
def _broken_set(tmp_path: Path, *, mandate: dict, bundle: str, fasit: dict) -> Path:
|
|
set_dir = tmp_path / "broken-set"
|
|
(set_dir / "docs").mkdir(parents=True)
|
|
(set_dir / "mandate.json").write_text(json.dumps(mandate), encoding="utf-8")
|
|
(set_dir / "bundle.txt").write_text(bundle, encoding="utf-8")
|
|
(set_dir / "fasit.json").write_text(json.dumps(fasit), encoding="utf-8")
|
|
return set_dir
|
|
|
|
|
|
_GOOD_MANDATE = {
|
|
"objective": "Reduce cost on a synthetic project",
|
|
"success_criteria": "at least one approach validates",
|
|
"approaches": [
|
|
{
|
|
"id": "a1",
|
|
"label": "One",
|
|
"affected_codes": ["X-1"],
|
|
"claimed_saving_nok": 1.0,
|
|
"bundle_id": "vegnormal-n500-2024",
|
|
},
|
|
{
|
|
"id": "a2",
|
|
"label": "Two",
|
|
"affected_codes": ["X-2"],
|
|
"claimed_saving_nok": 2.0,
|
|
"bundle_id": "vegnormal-n500-2024",
|
|
},
|
|
],
|
|
}
|
|
_GOOD_BUNDLE_TXT = "name: n500-2024\nbundle_id: vegnormal-n500-2024\n"
|
|
|
|
|
|
def test_known_positive_a_a_malformed_mandate_is_refused(tmp_path: Path) -> None:
|
|
broken = dict(_GOOD_MANDATE)
|
|
broken["approaches"] = [
|
|
dict(_GOOD_MANDATE["approaches"][0]),
|
|
dict(_GOOD_MANDATE["approaches"][0]),
|
|
]
|
|
set_dir = _broken_set(tmp_path, mandate=broken, bundle=_GOOD_BUNDLE_TXT, fasit={})
|
|
with pytest.raises(ValidationError):
|
|
load_mandate(set_dir / "mandate.json")
|
|
|
|
|
|
def test_known_positive_d_a_mandate_routed_at_another_base_is_caught(tmp_path: Path) -> None:
|
|
broken = json.loads(json.dumps(_GOOD_MANDATE))
|
|
broken["approaches"][1]["bundle_id"] = "vegnormal-n100-2023"
|
|
set_dir = _broken_set(tmp_path, mandate=broken, bundle=_GOOD_BUNDLE_TXT, fasit={})
|
|
declared = read_bundle_txt(set_dir / "bundle.txt")
|
|
ids = {block["bundle_id"] for block in declared}
|
|
routed = {a.bundle_id for a in load_mandate(set_dir / "mandate.json").approaches}
|
|
# The SAME two set relations arm (d) asserts, and the broken set must fail the first of them:
|
|
# an approach routed at a base the set does not declare.
|
|
assert not routed <= ids
|
|
assert sorted(routed - ids) == ["vegnormal-n100-2023"]
|
|
|
|
|
|
def test_known_positive_c_an_anchor_the_base_carries_is_reported() -> None:
|
|
concepts = [("a.md", {"type": "Krav"}, "en tunnel med ventilasjon og belysning")]
|
|
assert anchors_are_absent(["enhetspris"], concepts) == []
|
|
assert anchors_are_absent(["ventilasjon"], concepts) == ["ventilasjon"]
|
|
|
|
|
|
def test_known_positive_c_an_empty_scan_is_refused_never_vacuously_absent() -> None:
|
|
with pytest.raises(ValueError, match="ZERO concepts"):
|
|
anchors_are_absent(["enhetspris"], [])
|
|
|
|
|
|
def test_known_positive_c_an_unusable_anchor_is_refused() -> None:
|
|
concepts = [("a.md", {"type": "Krav"}, "tekst")]
|
|
with pytest.raises(ValueError, match="at least"):
|
|
anchors_are_absent(["vei"], concepts)
|
|
with pytest.raises(ValueError, match="lowercase"):
|
|
anchors_are_absent(["Enhetspris"], concepts)
|
|
with pytest.raises(ValueError, match="no anchors"):
|
|
anchors_are_absent([], concepts)
|
|
|
|
|
|
def test_known_positive_b_a_fasit_path_the_base_does_not_carry_is_caught(tmp_path: Path) -> None:
|
|
base = tmp_path / "base"
|
|
(base / "krav").mkdir(parents=True)
|
|
(base / "krav" / "real.md").write_text(
|
|
"---\ntype: Krav\ntitle: Ekte krav\nreq_number: Krav 1.1—1\n---\n\nkropp\n",
|
|
encoding="utf-8",
|
|
)
|
|
assert (base / "krav" / "real.md").is_file()
|
|
assert not (base / "krav" / "invented.md").is_file()
|
|
assert own_frontmatter(base / "krav" / "real.md")["title"] == "Ekte krav"
|
|
|
|
|
|
def test_known_positive_bundle_txt_must_declare_both_keys(tmp_path: Path) -> None:
|
|
path = tmp_path / "bundle.txt"
|
|
path.write_text("name: n500-2024\n", encoding="utf-8")
|
|
with pytest.raises(ValueError, match="bundle_id"):
|
|
read_bundle_txt(path)
|
|
|
|
|
|
def test_known_positive_a_second_block_needs_its_own_bundle_id(tmp_path: Path) -> None:
|
|
"""P17b: each ``name:`` OPENS a block, and each block closes with its own id.
|
|
|
|
The half a single-base file cannot exercise: a reader that flattened the file into one
|
|
mapping would let the FIRST block's ``bundle_id`` satisfy the second, and the second base
|
|
would then be addressed under the first one's name.
|
|
"""
|
|
path = tmp_path / "bundle.txt"
|
|
path.write_text(
|
|
"name: n200-2024\nbundle_id: vegnormal-n200-2024\nname: r761-2025\n", encoding="utf-8"
|
|
)
|
|
with pytest.raises(ValueError, match="bundle_id"):
|
|
read_bundle_txt(path)
|
|
|
|
|
|
def test_a_multi_base_bundle_txt_parses_into_one_block_per_base(tmp_path: Path) -> None:
|
|
path = tmp_path / "bundle.txt"
|
|
path.write_text(
|
|
"name: n200-2024\nbundle_id: vegnormal-n200-2024\n"
|
|
"name: r761-2025\nbundle_id: vegnormal-r761-2025\n",
|
|
encoding="utf-8",
|
|
)
|
|
assert read_bundle_txt(path) == (
|
|
{"name": "n200-2024", "bundle_id": "vegnormal-n200-2024"},
|
|
{"name": "r761-2025", "bundle_id": "vegnormal-r761-2025"},
|
|
)
|
|
|
|
|
|
@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
|
|
def test_the_fasit_titles_are_distinct_not_the_collapsed_sources_title(set_dir: Path) -> None:
|
|
"""The fasit's recorded titles must tell the cited concepts APART.
|
|
|
|
Paired with ``own_frontmatter``'s measurement, this is what keeps arm (b) from being vacuous: if
|
|
the recorded titles were ``parse_frontmatter``'s PRE-P15 titles, every one of them would be the
|
|
base's ``sources`` title and the assert would hold against any concept in the base.
|
|
|
|
**Former tripwire, INVERTED 2026-09-13 by P15 (deliberately, per that order — not deleted).**
|
|
Until P15 the second half asserted that ``okf.parse_frontmatter`` DID still collapse the
|
|
titles, as a red flag that would fire the day the production bug was fixed here instead of at
|
|
the call site. P15 fixed it AT THE SOURCE (``okf._frontmatter_from_text``: a top-level key now
|
|
always wins over a nested one of the same name), so the second half now asserts the opposite —
|
|
that ``parse_frontmatter`` agrees with the fasit's own distinct titles — as a live regression
|
|
guard against the collapse coming back."""
|
|
bases = _base_by_approach(set_dir)
|
|
fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))
|
|
|
|
cited = [(bases[row["approach_id"]], c) for row in fasit["must_cite"] for c in row["concepts"]]
|
|
assert len({c["title"] for _, c in cited}) == len(cited), (
|
|
"recorded titles do not tell them apart"
|
|
)
|
|
|
|
titles = {okf.parse_frontmatter(base / c["path"]).get("title", "") for base, c in cited}
|
|
assert len(titles) == len(cited), (
|
|
"okf.parse_frontmatter collapsed these titles onto the sources block again — the P15 fix "
|
|
"in okf._frontmatter_from_text has regressed"
|
|
)
|
|
|
|
|
|
# --------------------------------------------------------------------------------------------
|
|
# (f) + (g) + (h): the set is ANCHORED (P21 B2).
|
|
#
|
|
# The measured reason these exist. Four paid rounds ran entirely un-anchored, because the only
|
|
# file loader reads ``cost-baseline.json`` out of the BUNDLE and no road normal carries prices — a
|
|
# vegnormal is knowledge, the price belongs to the PROJECT. With ``--cost-baseline`` the project
|
|
# supplies its own schedule, so the validator's stage 0 judges again: (f) every answerable approach
|
|
# has a line to reconcile against, and (g) the falsification arm has NONE, so the code it proposes
|
|
# is refused as "not in the project's cost baseline" — by stage 0, the one stage that can tell an
|
|
# invented line from a real one, instead of by the weaker downstream gates.
|
|
#
|
|
# (f) and (g) are SEPARATE arms rather than one loop over all approaches, because they are opposite
|
|
# claims about opposite rows: a single arm asserting "exactly the non-refuse codes are present"
|
|
# would go red for either defect and name neither.
|
|
# --------------------------------------------------------------------------------------------
|
|
|
|
|
|
def _set_baseline(set_dir: Path) -> CostBaseline:
|
|
return okf.load_cost_baseline_file(str(set_dir / "cost-baseline.json"))
|
|
|
|
|
|
@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
|
|
def test_f_every_answerable_approach_has_a_cost_line(set_dir: Path) -> None:
|
|
"""Unconditional — the schedule is the PROJECT's and needs no knowledge base to read.
|
|
|
|
The total is asserted against the approach's own estimate as well as the code's presence:
|
|
``SavingsProposal`` refuses ``claimed_saving_nok > sum(affected_items.total)``, so a line that
|
|
exists but is smaller than the saving commissioned against it would make the approach
|
|
unbuildable — a set that looks anchored and cannot be run.
|
|
"""
|
|
baseline = _set_baseline(set_dir)
|
|
assert 4 <= len(baseline.items) <= 8, (
|
|
f"{set_dir.name}: {len(baseline.items)} cost lines — the order asks for 4-8"
|
|
)
|
|
assert baseline.project_id == set_dir.name, (
|
|
f"{set_dir.name}: the schedule names project {baseline.project_id!r}"
|
|
)
|
|
fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))
|
|
refused = {row["approach_id"] for row in fasit["must_refuse"]}
|
|
for approach in load_mandate(set_dir / "mandate.json").approaches:
|
|
if approach.id in refused:
|
|
continue
|
|
missing = [c for c in approach.affected_codes if c not in baseline.items]
|
|
assert not missing, (
|
|
f"{set_dir.name}: {approach.id} is answerable but {missing} carry no line in the "
|
|
f"project's schedule ({sorted(baseline.items)})"
|
|
)
|
|
total = sum(
|
|
baseline.items[c].quantity * baseline.items[c].unit_cost
|
|
for c in approach.affected_codes
|
|
)
|
|
assert approach.claimed_saving_nok is not None
|
|
assert total >= approach.claimed_saving_nok, (
|
|
f"{set_dir.name}: {approach.id} claims {approach.claimed_saving_nok:g} against lines "
|
|
f"totalling {total:g} — no proposal on it can satisfy claimed <= total"
|
|
)
|
|
|
|
|
|
@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
|
|
def test_g_the_falsification_arm_has_no_cost_line(set_dir: Path) -> None:
|
|
"""The ``must_refuse`` approach's code is ABSENT, so stage 0 is what catches it.
|
|
|
|
This is the half that makes the anchoring worth measuring rather than just present: rule U
|
|
already says the base carries no GROUND for that line, and P7's stage 0b says the identifier is
|
|
ungrounded in the delivered input — but neither of those is the stage that knows what this
|
|
project actually buys. Stage 0 is, and it can only speak when the schedule exists.
|
|
"""
|
|
baseline = _set_baseline(set_dir)
|
|
fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))
|
|
by_id = {a.id: a for a in load_mandate(set_dir / "mandate.json").approaches}
|
|
assert fasit["must_refuse"], f"{set_dir.name} declares no falsification arm"
|
|
for row in fasit["must_refuse"]:
|
|
approach = by_id[row["approach_id"]]
|
|
carried = [c for c in approach.affected_codes if c in baseline.items]
|
|
assert not carried, (
|
|
f"{set_dir.name}: the falsification arm {approach.id} carries {carried} in the "
|
|
"project's schedule, so stage 0 would ACCEPT the line it exists to refuse"
|
|
)
|
|
|
|
|
|
@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
|
|
def test_h_no_cost_line_smuggles_in_an_uncommissioned_requirement_number(set_dir: Path) -> None:
|
|
"""No line of the schedule is a reference number the base declares AND nobody commissions.
|
|
|
|
**The order words this arm as "no baseline code is a requirement number the base declares", and
|
|
that rule was FELLED BY MEASUREMENT before anything was built on it.** Measured 15.09 against
|
|
``okf.declared_reference_numbers`` over the four mounted bases: the four project-coded sets
|
|
carry 0 such codes, and ``kontrakt-sorasen-2027`` carries FIVE of five — ``12.1``, ``12.12``,
|
|
``22.1``, ``52.11``, ``51.1`` are real R761 ``prosessnr``. That is not an accident in the set;
|
|
it is what R761 Prosesskoden IS. A Norwegian road contract's bill of quantities is priced BY
|
|
process code, so the project's schedule and the corpus's vocabulary share an identifier
|
|
namespace by design — and the order's rule would have forced a rewrite of the ONE set P20's
|
|
decision (e) was chosen to preserve.
|
|
|
|
The COMPLEMENT keeps both: a schedule may price what the commission names, and may not
|
|
INTRODUCE a corpus identifier as a cost line nobody ordered. The order's own mutation still
|
|
bites — swapping a code for ``12.11`` (a declared ``prosessnr`` no approach commissions) goes
|
|
red here — while the five real process codes pass because an approach names each of them.
|
|
|
|
Needs the base (the vocabulary is the base's), so it SKIPS with the root named.
|
|
"""
|
|
baseline = _set_baseline(set_dir)
|
|
commissioned = {
|
|
code
|
|
for approach in load_mandate(set_dir / "mandate.json").approaches
|
|
for code in approach.affected_codes
|
|
}
|
|
declared: set[str] = set()
|
|
concepts = 0
|
|
for block in read_bundle_txt(set_dir / "bundle.txt"):
|
|
bundle = okf.navigate_bundle(str(_require_base(block)))
|
|
for f in bundle.context_files:
|
|
concepts += 1
|
|
declared |= set(okf.declared_reference_numbers(f))
|
|
assert concepts >= 100, (
|
|
f"{set_dir.name}: scanned {concepts} concepts — too few to be the base(s)"
|
|
)
|
|
assert declared, f"{set_dir.name}: the base(s) declare NO reference numbers — nothing to test"
|
|
smuggled = sorted(c for c in baseline.items if c in declared and c not in commissioned)
|
|
assert not smuggled, (
|
|
f"{set_dir.name}: cost line(s) {smuggled} are reference numbers the knowledge base "
|
|
"declares and no approach commissions — the schedule would be introducing the corpus's "
|
|
"own identifiers as prices nobody ordered"
|
|
)
|
|
|
|
|
|
def test_known_positive_f_a_missing_cost_line_is_caught(tmp_path: Path) -> None:
|
|
baseline = _set_baseline(_CONTEXT_ROOT / "gate-nordvik-2027")
|
|
assert "GATE-KRYSS-01" in baseline.items
|
|
stripped = CostBaseline(
|
|
project_id=baseline.project_id,
|
|
items={k: v for k, v in baseline.items.items() if k != "GATE-KRYSS-01"},
|
|
)
|
|
assert "GATE-KRYSS-01" not in stripped.items
|
|
|
|
|
|
def test_known_positive_g_a_line_for_the_falsification_arm_is_caught() -> None:
|
|
"""The order's mutation (i): give a4 a line, and (g)'s assertion must fail on this set."""
|
|
baseline = _set_baseline(_CONTEXT_ROOT / "gate-nordvik-2027")
|
|
priced = dict(baseline.items)
|
|
priced["GATE-GANG-ENHET"] = CostBaselineLine(quantity=6, unit_cost=50_000.0)
|
|
fasit = json.loads((_CONTEXT_ROOT / "gate-nordvik-2027" / "fasit.json").read_text("utf-8"))
|
|
by_id = {
|
|
a.id: a
|
|
for a in load_mandate(_CONTEXT_ROOT / "gate-nordvik-2027" / "mandate.json").approaches
|
|
}
|
|
for row in fasit["must_refuse"]:
|
|
carried = [c for c in by_id[row["approach_id"]].affected_codes if c in priced]
|
|
assert carried == ["GATE-GANG-ENHET"]
|
|
|
|
|
|
def test_known_positive_h_an_uncommissioned_requirement_number_is_caught() -> None:
|
|
"""The order's mutation (ii): swap a code for ``12.11``.
|
|
|
|
Driven against a KNOWN vocabulary rather than the mounted base, so this known-positive runs
|
|
unconditionally — a control that skipped with the base would leave the arm's discriminator
|
|
unproven on exactly the machines that cannot run the arm.
|
|
"""
|
|
declared = {"12.1", "12.11", "12.12"}
|
|
commissioned = {"12.1", "12.12"}
|
|
assert sorted(c for c in {"12.1", "12.12"} if c in declared and c not in commissioned) == []
|
|
assert sorted(c for c in {"12.1", "12.11"} if c in declared and c not in commissioned) == [
|
|
"12.11"
|
|
]
|