portfolio-optimiser/tests/test_context_sets_loadbearing.py

"""P14 — the stress-test context sets, gated (order ``20260912T202210Z``).

Each set under ``contexts/<project-id>/`` commissions ONE run against ONE knowledge base:
``mandate.json`` (the ``Mandate`` schema verbatim), ``bundle.txt`` (which base, and the id that base
declares) and ``fasit.json`` (what a right answer MUST cite, what the base cannot answer, and what
in the set is constructed rather than real).

**Five arms, and the split between them is a measurement rather than a taste.** Two are
unconditional and can never be silently absent — a mandate that does not load, and a mandate routed
at a base the set is not for. Three need the base itself, which lives OUTSIDE this repository
(``PORTFOLIO_VEGNORMAL_ROOT``): they SKIP when the root is missing, exactly as MAJOR-3's ceiling
gate could not take K2 as a test dependency, and for the same published-package reason — a hard
failure would break ``uv run pytest`` for any external recipient of the ``git archive HEAD``
handover. The skip NAMES the root it looked for.

**Every bundle-reading arm carries its own denominator.** A scan that sees zero concepts is RED
rather than vacuously green: "the anchor was not found" is equally true of a base that was never
read (Verifiseringsloven, ansikt 4).

**Rule U** — the measurable form of "the base cannot answer this" (documented in
``docs/2026-09-12-p14-kontekstsett.md § 2.3``): each ``must_refuse`` row declares >= 1 ``anchor``,
a lowercase word of >= 4 characters, and is admitted **iff every anchor is absent — case-insensitive
substring — from the WHOLE text (frontmatter + body) of EVERY concept document in the base**. Not
"shares no keyword with any title": a tunnel question shares "tunnel" with hundreds of titles and
that proves nothing. What makes a question unanswerable is that the base lacks the SUBJECT, and the
anchor is that subject. Titles alone would be a proxy the full text costs nothing more to replace
(measured: 0.77 s for r761-2025, the largest base).

**P16 A2 moved rule U from ``unanswerable`` to ``must_refuse``, and that is ONE form rather than
two.** ``po`` is not a lookup tool (D-1), so an "unanswerable question" had no runnable form: no
execution path ever consumed those rows, and the falsification half of session 102's criterion was
therefore unprovable by a run. The same fact now rides a FOURTH commissioned approach per set
(``a4-…``) whose cost line the base carries no ground for, and ``must_refuse`` names it by
``approach_id`` plus the same anchors. Keeping the questions beside it as a second key would be two
copies of one fact (kø-(p)) — the questions survive inside the row's ``rationale``, which nothing
keys on. Rule U itself is UNCHANGED, and its known-positive is still red.
"""

from __future__ import annotations

import json
import os
from pathlib import Path

import pytest
from pydantic import ValidationError

from portfolio_optimiser import okf
from portfolio_optimiser.mandate import load_mandate

_REPO_ROOT = Path(__file__).resolve().parent.parent
_CONTEXT_ROOT = _REPO_ROOT / "contexts"

#: Where the vegnormal bases are mounted. A SYMBOLIC name in ``bundle.txt`` is resolved against
#: this, never an absolute path in the set: this repository is published, and an absolute path
#: would pin a set to one machine's home directory and ride out in the handover archive.
_DEFAULT_BUNDLE_ROOT = Path.home() / "repos" / "vegnormal-okf" / "build" / "ferdig"

#: The concept types the four bases declare. ``index.md`` carries none of them — it is navigation,
#: not content — which is why the file count and the concept count differ.
_CONCEPT_TYPES = {"Krav", "Prosess", "Kapittel", "Normal", "Håndbok"}

_MIN_ANCHOR_CHARS = 4


def own_frontmatter(path: Path) -> dict[str, str]:
    """The concept's OWN frontmatter: top-level keys only, FIRST occurrence winning.

    **P15 (2026-09-13) fixed the finding this helper was written against.** Before P15,
    ``okf.parse_frontmatter`` was linewise and last-write-wins over EVERY line regardless of
    indentation, so a nested block overwrote a top-level key of the same name. Every vegnormal
    concept ends its frontmatter with

        sources:
          - resource: https://…
            title: N500:2024

    and the indented ``title`` used to replace the concept's own. MEASURED on n500-2024 before the
    fix: ``okf.navigate_bundle`` yielded 270 concept files carrying **1 distinct title**
    (``N500:2024``, 270 times). ``okf.parse_frontmatter`` now makes indentation load-bearing —
    a top-level (unindented) key always wins over a nested one of the same name — and re-measured
    AFTER the fix, the same base's 269 ``krav/N500`` documents carry **269 distinct titles**.

    **This helper still isn't a plain call to ``okf.parse_frontmatter``, and that remains
    measured rather than assumed:** ``own_frontmatter`` also strips one layer of enclosing
    ``'`` quotes (``.strip("'")``) so a value matches the fasit's stored plain-text title
    verbatim, while ``okf.parse_frontmatter`` deliberately leaves scalars quoted — unquoting is
    ``okf.unquote_scalar``'s ONE job (D1/(a)/(i)), and a second copy of that rule here would be
    the drifting one. Re-measured across all four bases (29 500 field reads: ``type``, ``title``,
    ``req_number``, ``prosessnr`` on every concept file) the two now agree EXACTLY except for
    quoted scalars (2 728 of 29 500 checks — every one a quote-stripping difference, none a value
    difference), so this helper stays for that one reason, not for the nested-override bug P15
    closed.

    Uses ``okf._split_frontmatter`` deliberately: it is the module's ONE place ``---`` is compared
    (B4), and a second delimiter rule here would be the copy that drifts.
    """
    out: dict[str, str] = {}
    for line in okf._split_frontmatter(path.read_text(encoding="utf-8"))[0]:
        if not line or line[0].isspace() or line.lstrip().startswith("-"):
            continue
        key, sep, value = line.partition(":")
        if sep and key.strip() not in out:
            out[key.strip()] = value.strip().strip("'")
    return out


def _bundle_root() -> Path:
    return Path(os.environ.get("PORTFOLIO_VEGNORMAL_ROOT", str(_DEFAULT_BUNDLE_ROOT)))


def read_bundle_txt(path: Path) -> dict[str, str]:
    """Parse a set's ``bundle.txt``: ``key: value`` lines, nothing else."""
    out: dict[str, str] = {}
    for line in path.read_text(encoding="utf-8").splitlines():
        line = line.strip()
        if not line or line.startswith("#"):
            continue
        if ": " not in line:
            raise ValueError(f"malformed bundle.txt line in {path}: {line!r}")
        key, value = line.split(": ", 1)
        out[key.strip()] = value.strip()
    for required in ("name", "bundle_id"):
        if required not in out:
            raise ValueError(f"{path} declares no {required!r}")
    return out


def scan_concepts(base: Path) -> list[tuple[str, dict[str, str], str]]:
    """Every concept document in a base: bundle-relative path, frontmatter, lowercased full text.

    Built from the declared ``type``, never from the directory listing: ``index.md`` is navigation
    and would otherwise be counted as content.
    """
    found: list[tuple[str, dict[str, str], str]] = []
    for path in sorted(base.rglob("*.md")):
        text = path.read_text(encoding="utf-8")
        frontmatter = own_frontmatter(path)
        if frontmatter.get("type", "") in _CONCEPT_TYPES:
            found.append((path.relative_to(base).as_posix(), frontmatter, text.lower()))
    return found


def anchors_are_absent(
    anchors: list[str], concepts: list[tuple[str, dict[str, str], str]]
) -> list[str]:
    """Rule U: return the anchors the base DOES carry (empty == the question is admitted).

    :raises ValueError: an empty scan, or an anchor that is not a usable one. Both are refusals
        rather than a quiet pass — a rule that cannot fail proves nothing.
    """
    if not concepts:
        raise ValueError("rule U ran against ZERO concepts: absence here is unmeasured, not false")
    if not anchors:
        raise ValueError("a must_refuse row declares no anchors, so nothing was checked")
    carried = []
    for anchor in anchors:
        if anchor != anchor.lower() or len(anchor) < _MIN_ANCHOR_CHARS:
            raise ValueError(
                f"anchor {anchor!r} must be lowercase and at least {_MIN_ANCHOR_CHARS} characters"
            )
        if any(anchor in text for _, _, text in concepts):
            carried.append(anchor)
    return carried


def context_sets() -> list[Path]:
    return (
        sorted(p for p in _CONTEXT_ROOT.iterdir() if p.is_dir()) if _CONTEXT_ROOT.is_dir() else []
    )


_SETS = context_sets()
_SET_IDS = [p.name for p in _SETS]


def _require_base(declared: dict[str, str]) -> Path:
    root = _bundle_root()
    base = root / declared["name"]
    if not base.is_dir():
        pytest.skip(
            f"knowledge base {declared['name']!r} not mounted under {root} (PORTFOLIO_VEGNORMAL_ROOT)"
        )
    return base


# --------------------------------------------------------------------------------------------
# The sets exist at all. Without this, every parametrised arm below would collapse to zero cases
# and the file would pass by having nothing to say.
# --------------------------------------------------------------------------------------------


def test_the_four_context_sets_are_present() -> None:
    assert len(_SETS) == 4, f"expected four context sets under {_CONTEXT_ROOT}, found {_SET_IDS}"


# --------------------------------------------------------------------------------------------
# (a) + (d): unconditional — no knowledge base needed.
# --------------------------------------------------------------------------------------------


@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
def test_a_mandate_loads_fail_fast(set_dir: Path) -> None:
    mandate = load_mandate(set_dir / "mandate.json")
    assert mandate.objective
    assert mandate.success_criteria, f"{set_dir.name} states no success criteria to judge it by"
    assert 2 <= len(mandate.approaches) <= 4, "the order asks for 2-4 approaches per set"
    for approach in mandate.approaches:
        assert approach.affected_codes, f"{approach.id} names no affected_codes"
        assert approach.claimed_saving_nok is not None, f"{approach.id} states no estimate"


@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
def test_d_every_approach_is_routed_at_this_sets_own_base(set_dir: Path) -> None:
    declared = read_bundle_txt(set_dir / "bundle.txt")
    mandate = load_mandate(set_dir / "mandate.json")
    for approach in mandate.approaches:
        assert approach.bundle_id == declared["bundle_id"], (
            f"{set_dir.name}/{approach.id} routes at {approach.bundle_id!r} but the set declares "
            f"{declared['bundle_id']!r}"
        )


@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
def test_the_fasit_names_every_commissioned_approach(set_dir: Path) -> None:
    fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))
    mandate = load_mandate(set_dir / "mandate.json")
    cited = {row["approach_id"] for row in fasit["must_cite"]}
    refused = {row["approach_id"] for row in fasit["must_refuse"]}
    assert not (cited & refused), "an approach is either answerable or the falsification arm"
    assert cited | refused == {a.id for a in mandate.approaches}
    assert fasit["honesty"].strip(), "DEL 2(iii): what in this set is constructed must be stated"
    assert refused, "P16 A2: every set carries the falsification arm"
    for row in fasit["must_refuse"]:
        assert len(row["anchors"]) >= 2, "the order asks for at least two per set"
        assert row["rationale"].strip(), "why the base cannot ground it must be stated"
    assert (set_dir / "docs").is_dir(), "the form declares a docs/ directory even when it is empty"


# --------------------------------------------------------------------------------------------
# (b) + (c) + (e): these read the base itself and SKIP when it is not mounted.
# --------------------------------------------------------------------------------------------


@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
def test_b_every_fasit_concept_is_in_the_base_as_recorded(set_dir: Path) -> None:
    declared = read_bundle_txt(set_dir / "bundle.txt")
    base = _require_base(declared)
    fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))

    seen = 0
    for row in fasit["must_cite"]:
        assert row["concepts"], f"{row['approach_id']} cites nothing a right answer must reach"
        for concept in row["concepts"]:
            path = base / concept["path"]
            assert path.is_file(), f"{set_dir.name}: {concept['path']} is not in {declared['name']}"
            frontmatter = own_frontmatter(path)
            assert frontmatter.get("title", "") == concept["title"], (
                f"{concept['path']}: the base's own title has drifted from the fasit"
            )
            if concept.get("ref"):
                actual = frontmatter.get("req_number") or frontmatter.get("prosessnr", "")
                assert actual == concept["ref"], (
                    f"{concept['path']}: the base's own reference has drifted from the fasit"
                )
            seen += 1
    assert seen > 0, "the fasit named no concepts at all, so nothing was verified"


@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
def test_c_rule_u_every_unanswerable_question_is_unanswerable(set_dir: Path) -> None:
    declared = read_bundle_txt(set_dir / "bundle.txt")
    base = _require_base(declared)
    concepts = scan_concepts(base)
    assert len(concepts) >= 100, (
        f"{declared['name']} scanned to {len(concepts)} concepts — too few to be the base itself"
    )

    fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))
    for row in fasit["must_refuse"]:
        carried = anchors_are_absent(row["anchors"], concepts)
        assert not carried, (
            f"{set_dir.name}: {declared['name']} DOES carry {carried} over {len(concepts)} "
            f"concepts, so {row['approach_id']!r} is not un-groundable by rule U"
        )


@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
def test_e_the_declared_bundle_id_is_the_bases_own(set_dir: Path) -> None:
    declared = read_bundle_txt(set_dir / "bundle.txt")
    base = _require_base(declared)
    resolved = okf.reconcile_bundle_id(base)
    assert resolved.id == declared["bundle_id"], (
        f"{set_dir.name}: bundle.txt declares {declared['bundle_id']!r} but the base resolves to "
        f"{resolved.id!r} (origin {resolved.origin})"
    )


# --------------------------------------------------------------------------------------------
# KNOWN-POSITIVES (DEL 3): a deliberately broken set must make EXACTLY the arm that guards it red.
# Without these, a check that can only pass is indistinguishable from a check that never runs.
# --------------------------------------------------------------------------------------------


def _broken_set(tmp_path: Path, *, mandate: dict, bundle: str, fasit: dict) -> Path:
    set_dir = tmp_path / "broken-set"
    (set_dir / "docs").mkdir(parents=True)
    (set_dir / "mandate.json").write_text(json.dumps(mandate), encoding="utf-8")
    (set_dir / "bundle.txt").write_text(bundle, encoding="utf-8")
    (set_dir / "fasit.json").write_text(json.dumps(fasit), encoding="utf-8")
    return set_dir


_GOOD_MANDATE = {
    "objective": "Reduce cost on a synthetic project",
    "success_criteria": "at least one approach validates",
    "approaches": [
        {
            "id": "a1",
            "label": "One",
            "affected_codes": ["X-1"],
            "claimed_saving_nok": 1.0,
            "bundle_id": "vegnormal-n500-2024",
        },
        {
            "id": "a2",
            "label": "Two",
            "affected_codes": ["X-2"],
            "claimed_saving_nok": 2.0,
            "bundle_id": "vegnormal-n500-2024",
        },
    ],
}
_GOOD_BUNDLE_TXT = "name: n500-2024\nbundle_id: vegnormal-n500-2024\n"


def test_known_positive_a_a_malformed_mandate_is_refused(tmp_path: Path) -> None:
    broken = dict(_GOOD_MANDATE)
    broken["approaches"] = [
        dict(_GOOD_MANDATE["approaches"][0]),
        dict(_GOOD_MANDATE["approaches"][0]),
    ]
    set_dir = _broken_set(tmp_path, mandate=broken, bundle=_GOOD_BUNDLE_TXT, fasit={})
    with pytest.raises(ValidationError):
        load_mandate(set_dir / "mandate.json")


def test_known_positive_d_a_mandate_routed_at_another_base_is_caught(tmp_path: Path) -> None:
    broken = json.loads(json.dumps(_GOOD_MANDATE))
    broken["approaches"][1]["bundle_id"] = "vegnormal-n100-2023"
    set_dir = _broken_set(tmp_path, mandate=broken, bundle=_GOOD_BUNDLE_TXT, fasit={})
    declared = read_bundle_txt(set_dir / "bundle.txt")
    mandate = load_mandate(set_dir / "mandate.json")
    assert any(a.bundle_id != declared["bundle_id"] for a in mandate.approaches)


def test_known_positive_c_an_anchor_the_base_carries_is_reported() -> None:
    concepts = [("a.md", {"type": "Krav"}, "en tunnel med ventilasjon og belysning")]
    assert anchors_are_absent(["enhetspris"], concepts) == []
    assert anchors_are_absent(["ventilasjon"], concepts) == ["ventilasjon"]


def test_known_positive_c_an_empty_scan_is_refused_never_vacuously_absent() -> None:
    with pytest.raises(ValueError, match="ZERO concepts"):
        anchors_are_absent(["enhetspris"], [])


def test_known_positive_c_an_unusable_anchor_is_refused() -> None:
    concepts = [("a.md", {"type": "Krav"}, "tekst")]
    with pytest.raises(ValueError, match="at least"):
        anchors_are_absent(["vei"], concepts)
    with pytest.raises(ValueError, match="lowercase"):
        anchors_are_absent(["Enhetspris"], concepts)
    with pytest.raises(ValueError, match="no anchors"):
        anchors_are_absent([], concepts)


def test_known_positive_b_a_fasit_path_the_base_does_not_carry_is_caught(tmp_path: Path) -> None:
    base = tmp_path / "base"
    (base / "krav").mkdir(parents=True)
    (base / "krav" / "real.md").write_text(
        "---\ntype: Krav\ntitle: Ekte krav\nreq_number: Krav 1.1—1\n---\n\nkropp\n",
        encoding="utf-8",
    )
    assert (base / "krav" / "real.md").is_file()
    assert not (base / "krav" / "invented.md").is_file()
    assert own_frontmatter(base / "krav" / "real.md")["title"] == "Ekte krav"


def test_known_positive_bundle_txt_must_declare_both_keys(tmp_path: Path) -> None:
    path = tmp_path / "bundle.txt"
    path.write_text("name: n500-2024\n", encoding="utf-8")
    with pytest.raises(ValueError, match="bundle_id"):
        read_bundle_txt(path)


@pytest.mark.parametrize("set_dir", _SETS, ids=_SET_IDS)
def test_the_fasit_titles_are_distinct_not_the_collapsed_sources_title(set_dir: Path) -> None:
    """The fasit's recorded titles must tell the cited concepts APART.

    Paired with ``own_frontmatter``'s measurement, this is what keeps arm (b) from being vacuous: if
    the recorded titles were ``parse_frontmatter``'s PRE-P15 titles, every one of them would be the
    base's ``sources`` title and the assert would hold against any concept in the base.

    **Former tripwire, INVERTED 2026-09-13 by P15 (deliberately, per that order — not deleted).**
    Until P15 the second half asserted that ``okf.parse_frontmatter`` DID still collapse the
    titles, as a red flag that would fire the day the production bug was fixed here instead of at
    the call site. P15 fixed it AT THE SOURCE (``okf._frontmatter_from_text``: a top-level key now
    always wins over a nested one of the same name), so the second half now asserts the opposite —
    that ``parse_frontmatter`` agrees with the fasit's own distinct titles — as a live regression
    guard against the collapse coming back."""
    declared = read_bundle_txt(set_dir / "bundle.txt")
    base = _require_base(declared)
    fasit = json.loads((set_dir / "fasit.json").read_text(encoding="utf-8"))

    cited = [c for row in fasit["must_cite"] for c in row["concepts"]]
    assert len({c["title"] for c in cited}) == len(cited), "recorded titles do not tell them apart"

    titles = {okf.parse_frontmatter(base / c["path"]).get("title", "") for c in cited}
    assert len(titles) == len(cited), (
        "okf.parse_frontmatter collapsed these titles onto the sources block again — the P15 fix "
        "in okf._frontmatter_from_text has regressed"
    )