[skip-docs] PM addendum 5 to order 20260902T151931Z-250257273, reported at the close of B4 and deliberately left unfixed there as outside that order. evidence_for(path, key="sources") raised ValueError. The tier was derived unconditionally through trust_tier, which refuses an entry that names no `by` actor -- correctly, because a trust level derived from an entry identifying nobody mints the provenance it claims to read. But `by` is required of a VERIFICATION entry (SPEC 5.2), not of every provenance key: the agreed segmented form carries segment_id/source_offset and a sources list carries id/resource. The guard belonging to one key was being applied to all of them. trust_tier is UNCHANGED. evidence_for stops applying it to keys it was never about: _TIERED_KEY names the one key SPEC 5.3 tiers, and every other key comes back with state / reason / items_seen / entries and tier=None. admits_falsification moved WITH it, and that is the same fact rather than scope: it read `tier != "unverified"`, and None != "unverified" is TRUE, so a present `sources` list would have cleared a threshold about verification. It now names the tiers that clear it. For every value reachable before this change both spellings are identical, which is why the existing K5 arms stay green. Measured, and it corrected the test: an actorless `verified` value never reaches trust_tier through evidence_for at all -- the decoder refuses that shape first as unreadable / unsupported-flow. Both guards are now asserted where each actually lives instead of one asserted where it is not. Load-bearing measured, four mutations, all red against the WHOLE suite: derive the tier unconditionally (3 red) - revert the K5 threshold (1, the consequence arm alone) - "fix" it by never tiering at all (3, including two pre-existing falsification arms) - loosen trust_tier instead (2, including its own pre-existing gate). Green control 1208 passed / 5 skipped; golden demo-transcript.stdout unchanged (ea8c534773acdbe41ae68f2c55724d69aaf8be4f).
143 lines
6.4 KiB
Python
143 lines
6.4 KiB
Python
"""``evidence_for``'s ``key`` parameter works for keys other than ``verified``.
|
|
|
|
Reported at the close of B4 (session 76, PM addendum 5 to order
|
|
``20260902T151931Z-250257273``) and NOT fixed there, because it was outside that order:
|
|
|
|
evidence_for(path, key="sources") -> ValueError
|
|
|
|
``tier`` was derived UNCONDITIONALLY via ``trust_tier``, which refuses an entry that names no
|
|
``by`` actor — correctly, because deriving a trust level from an entry identifying nobody would
|
|
mint the provenance it claims to read. But ``by`` is required of a ``verified`` entry (SPEC §5.2),
|
|
not of every provenance key: the agreed segmented form carries ``segment_id``/``source_offset``,
|
|
and a ``sources`` list carries ``id``/``resource``. So the guard belonging to ONE key was applied
|
|
to ALL of them, and any other key whose value decoded cleanly crashed.
|
|
|
|
**It was never measured because it was never exercised:** all 11 call sites in the repo use the
|
|
default, so the parameter had no test of its own. A parameter no test drives is a parameter that
|
|
rots — the same class as a field no artefact carries.
|
|
|
|
The fix keeps the guard exactly where SPEC §5.3 puts it. ``trust_tier`` is UNCHANGED and still
|
|
refuses an actor-less ``verified`` entry; ``evidence_for`` derives a tier only for the key §5.3
|
|
tiers, and answers every other key with ``state`` / ``reason`` / ``items_seen`` / ``entries`` and
|
|
``tier=None``.
|
|
|
|
``admits_falsification`` had to move WITH it, and that is not scope creep — it is the same fact.
|
|
It read ``tier != "unverified"``, and ``None != "unverified"`` is true, so a present ``sources``
|
|
key would have admitted falsification: a document would have cleared the K5 threshold on evidence
|
|
that is not a verification at all. The threshold now names the tiers that clear it. For every
|
|
value reachable before this change the two spellings are identical, which is why the existing K5
|
|
arms stay green — the tightening is only for the state this change makes reachable.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
from portfolio_optimiser import okf
|
|
|
|
_TWO_SOURCES = (
|
|
"---\n"
|
|
"type: concept\n"
|
|
"title: Tunnelbelysning\n"
|
|
"sources: [{ id: n100, resource: vegnormal }, { id: ipmvp, resource: efficiency-valuation }]\n"
|
|
"verified: { by: human:kjell, at: 2026-09-02 }\n"
|
|
"---\n"
|
|
"\nInnhold.\n"
|
|
)
|
|
|
|
|
|
def _concept(tmp_path: Path, text: str = _TWO_SOURCES) -> Path:
|
|
path = tmp_path / "konsept.md"
|
|
path.write_text(text, encoding="utf-8")
|
|
return path
|
|
|
|
|
|
def test_a_non_verified_key_is_read_instead_of_raising(tmp_path: Path) -> None:
|
|
"""THE GOAL, in the reporter's own words: ``key="sources"`` on a concept with two sources
|
|
returns, and says it saw two.
|
|
|
|
Detach point: derive the tier unconditionally again → ``ValueError`` (RED).
|
|
"""
|
|
evidence = okf.evidence_for(_concept(tmp_path), key="sources")
|
|
|
|
assert evidence.state == "present"
|
|
assert evidence.items_seen == 2
|
|
assert [e["id"] for e in evidence.entries] == ["n100", "ipmvp"]
|
|
|
|
|
|
def test_a_non_verified_key_carries_no_tier(tmp_path: Path) -> None:
|
|
"""SPEC §5.3 tiers ``verified``. A tier on anything else would be a trust level derived from
|
|
a key that says nothing about trust — asserted, never inferred."""
|
|
assert okf.evidence_for(_concept(tmp_path), key="sources").tier is None
|
|
|
|
|
|
def test_a_non_verified_key_does_not_clear_the_k5_threshold(tmp_path: Path) -> None:
|
|
"""The consequence arm. ``tier != "unverified"`` is TRUE for ``None``, so without moving the
|
|
threshold with the fix, a present ``sources`` list would admit falsification — a document
|
|
clearing K5 on evidence that is not a verification.
|
|
|
|
Detach point: revert the threshold to ``!= "unverified"`` → RED here, green everywhere else.
|
|
"""
|
|
evidence = okf.evidence_for(_concept(tmp_path), key="sources")
|
|
|
|
assert evidence.state == "present", (
|
|
"the arm must run against a PRESENT key, or it proves nothing"
|
|
)
|
|
assert okf.admits_falsification(evidence) is False
|
|
|
|
|
|
def test_the_default_key_still_tiers_and_still_clears(tmp_path: Path) -> None:
|
|
"""The CONTROL. A fix that stopped tiering ``verified`` would pass all three arms above and
|
|
disable the whole falsification threshold."""
|
|
evidence = okf.evidence_for(_concept(tmp_path))
|
|
|
|
assert evidence.state == "present"
|
|
assert evidence.tier == "human-reviewed"
|
|
assert okf.admits_falsification(evidence) is True
|
|
|
|
|
|
def test_an_actorless_verified_entry_is_still_refused(tmp_path: Path) -> None:
|
|
"""The guard stays exactly where SPEC §5.2 puts it. Loosening ``trust_tier`` itself would have
|
|
fixed the crash by minting a trust tier for an entry that identifies nobody — the defect
|
|
``trust_tier``'s own docstring exists to refuse.
|
|
|
|
**A premise of this arm was wrong when first written, and the measurement corrected it:** an
|
|
actorless ``verified`` value never reaches ``trust_tier`` through ``evidence_for`` at all — the
|
|
decoder refuses that shape first, as ``unreadable`` / ``unsupported-flow`` (measured). So BOTH
|
|
guards are asserted where each actually lives, rather than one asserted where it is not.
|
|
"""
|
|
path = _concept(
|
|
tmp_path,
|
|
"---\ntype: concept\nverified: { at: 2026-09-02 }\n---\n\nInnhold.\n",
|
|
)
|
|
|
|
evidence = okf.evidence_for(path)
|
|
assert (evidence.state, evidence.reason, evidence.tier) == (
|
|
"unreadable",
|
|
"unsupported-flow",
|
|
None,
|
|
), "the decoder refuses an actorless verified value before any tier is derived"
|
|
assert okf.admits_falsification(evidence) is False
|
|
|
|
# And the public guard one layer down is untouched: called DIRECTLY with the shape the decoder
|
|
# would never hand it, ``trust_tier`` still refuses rather than tiering nobody.
|
|
with pytest.raises(ValueError):
|
|
okf.trust_tier(({"at": "2026-09-02"},))
|
|
|
|
|
|
def test_the_three_non_present_states_are_unchanged_for_any_key(tmp_path: Path) -> None:
|
|
"""Absent and unreadable never tiered anything, and must still not — the fix touches the
|
|
PRESENT arm alone."""
|
|
absent = okf.evidence_for(_concept(tmp_path), key="adjudication")
|
|
assert (absent.state, absent.tier, absent.items_seen) == ("absent", None, 0)
|
|
|
|
blocked = _concept(
|
|
tmp_path,
|
|
"---\ntype: concept\nsources:\n - id: n100\n resource: vegnormal\n---\n\nInnhold.\n",
|
|
)
|
|
unreadable = okf.evidence_for(blocked, key="sources")
|
|
assert unreadable.state == "unreadable"
|
|
assert unreadable.tier is None
|
|
assert unreadable.reason == "block-sequence"
|