Kjell Tore Guttormsen
b8b83fb267
The four arms that were red on assert, in the order the reader meets them.
A DANGLING SYMLINK as the round directory now refuses instead of tracebacking.
exists() FOLLOWS a link, so a dangling one answers False and slipped straight past
"a round is never overwritten"; the build then died on the filesystem's own
FileExistsError. is_symlink() is checked FIRST, the message says what a link would cost
(the round's content would sit somewhere the gate does not measure), and the link is
left exactly as it was found — nothing is written, exit code 1 like every other refusal.
ONE CITATION LIST SHARED BY EVERY PROPOSAL is now stated once. The cause was measured
before anything was written, because "the builder reads the wrong field" and "the outbox
says the same thing five times" want opposite fixes: in all four archived runs every
proposal carries a byte-identical 270-citation list — the run's whole retrieved context,
stamped once per proposal. No report can make that quote say something about the
individual measure. So when every proposal carries the same list, the report says so
once, says what the list actually is ("hva kjøringen leste, ikke hva det enkelte tiltaket
bygger på"), and drops the five copies. When the lists differ, nothing changes: the quote
and its COUNT stay under each proposal, which is where they mean something.
THE SAME COST LINE ON BOTH SIDES OF THE VERDICT is named where it happens. The 19.09
report refused TUN-LYS-01 under one label and validated the same line under another and
said nothing, so a reader met two figures for one budget line with no way to see they
collided. Both sides now carry the sentence, in the run's own row order.
A REMOVED APPROACH is shown by the label the expert saw, with the id in parentheses. It
is the one row whose human name is not in this run's coverage, so the label is read from
the coverage inside the PREVIOUS round's own outbox — derived from what the round already
carries, not a new column in outcome.json. An unreadable coverage falls back on the bare
id; a missing label is not a reason to refuse a round.
39 of 39 arms green. Nothing here touches what the gate reads: outcome.json keeps its
four columns and the builder still never writes the attestation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
754 lines
33 KiB
Python
754 lines
33 KiB
Python
"""From one run's outbox to a round directory the v1 gate can read.
|
||
|
||
One command, deterministic, offline: no model call, no network. It takes a finished run's outbox
|
||
and a round number and writes ``<rounds-dir>/<n>/`` in the shape ``v1_gate --help`` states —
|
||
``outbox/`` copied from the run, ``outcome.json`` DERIVED from that copy, and ``report.md``, the
|
||
one artefact in the round a domain expert is meant to read and correct.
|
||
|
||
Measured 19.09: nothing bound a run to a round directory (every mention of ``v1-rounds`` in the
|
||
repository was in the gate, its tests, ``.gitignore``, the ledger and a deck), and nothing under
|
||
``src/`` wrote markdown at all. Round 0 therefore counted 0 of 3 because it could not be MADE,
|
||
which is a different failure from a round nobody had held.
|
||
|
||
What it verifies, it verifies with the gate's own functions rather than with a second copy:
|
||
``verify_run`` for whether the run stands up to itself, ``stage_of`` for column (c),
|
||
``parse_time`` for the run's stamp, ``safe_rounds_dir`` for where a round may be written. A
|
||
builder that judged its output by its own rules could write rounds the reader refuses.
|
||
|
||
Two things it never does, and both are the point. It never writes the operator's attestation
|
||
file: that file is the statement that a round was actually held, and a builder that could produce
|
||
it would put the gate's one un-computable step back inside the machine. And it never invents —
|
||
``ran_at`` is an argument because no outbox artefact carries a clock (they are byte-deterministic
|
||
by contract), and ``feedback_ids`` stays empty because no run records which feedback item produced
|
||
which row. An empty list is the honest reading of a run that tracked nothing.
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import argparse
|
||
import json
|
||
import shutil
|
||
import sys
|
||
from collections.abc import Mapping, Sequence
|
||
from dataclasses import dataclass
|
||
from pathlib import Path
|
||
from typing import Any
|
||
|
||
from portfolio_optimiser.evals.v1_gate import (
|
||
DEFAULT_ROUNDS_DIR,
|
||
RUN_OUTBOX,
|
||
parse_time,
|
||
row_changed,
|
||
safe_rounds_dir,
|
||
stage_of,
|
||
verify_run,
|
||
)
|
||
from portfolio_optimiser.ledger import to_ore
|
||
|
||
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
||
_COVERAGE_SUFFIX = "-coverage.json"
|
||
#: The last round the gate reads (``ROUNDS_CONTRACT``): 0 is the baseline, 1-3 carry feedback.
|
||
_LAST_ROUND = 3
|
||
#: A citation is shown to place the claim, not to reproduce the source. Longer than this and the
|
||
#: report stops being something a person reads in one pass.
|
||
_SNIPPET_MAX = 220
|
||
|
||
|
||
class RoundBuildError(Exception):
|
||
"""A round that cannot be built from what is on disk, with the reason a reader can act on."""
|
||
|
||
|
||
@dataclass(frozen=True)
|
||
class Run:
|
||
"""One finished run, as its outbox presents it — already checked against itself."""
|
||
|
||
outbox: Path
|
||
run_id: str
|
||
rows: tuple[dict[str, Any], ...]
|
||
stop_reason: str
|
||
artefacts: tuple[str, ...]
|
||
ignored: tuple[str, ...]
|
||
proposals: dict[str, dict[str, Any]]
|
||
|
||
|
||
@dataclass(frozen=True)
|
||
class Built:
|
||
"""What one build produced, in numbers the caller can check against the source directory."""
|
||
|
||
round_dir: Path
|
||
run_id: str
|
||
copied: tuple[str, ...]
|
||
ignored: tuple[str, ...]
|
||
evaluated: tuple[str, ...]
|
||
not_evaluated: tuple[str, ...]
|
||
validated_ore: int
|
||
|
||
|
||
#: Every rejection stage ``validator.rejection_stage`` can name, in the words a domain expert
|
||
#: reads — plus the two labels a coverage row carries that are not rejections. A stage added to
|
||
#: the validator without a sentence here reaches the expert as a bare identifier, so the test
|
||
#: suite pins this table against the validator's own list rather than against itself.
|
||
#:
|
||
#: Every sentence is "<kort navn>: <forklaring>", because both halves are needed in different
|
||
#: places — the full sentence heads a refusal, the short name fits inside a one-line diff of what
|
||
#: changed since the previous round. A raw ``stage4-p90`` in either is an internal identifier
|
||
#: reaching a reader who has no way to look it up.
|
||
STAGE_PROSE: dict[str, str] = {
|
||
"stage0-baseline": (
|
||
"prosjektets egen kostnadsbasis: en kostnadslinje tiltaket bygger på stemmer ikke med "
|
||
"det prosjektet faktisk har budsjettert"
|
||
),
|
||
"stage0b-grounding": (
|
||
"forankringen i kunnskapsbasen: en kode tiltaket bygger på står ikke i teksten som ble "
|
||
"levert til kjøringen"
|
||
),
|
||
"stage4-p90": (
|
||
"usikkerhetsberegningen: den påståtte besparelsen er større enn det de gunstigste ti "
|
||
"prosentene av utfallene gir"
|
||
),
|
||
"stage4b-nominal": (
|
||
"det nominelt mulige: den påståtte besparelsen er større enn tiltaket kan gi selv i "
|
||
"beste fall"
|
||
),
|
||
"stage5-method-cap": (
|
||
"metodetaket: metoden gir erfaringsmessig ikke en så stor andel av kostnaden tilbake"
|
||
),
|
||
"unsupported": (
|
||
"manglende krav: tallene holdt, men ingen krav i kunnskapsbasen ble erklært å binde "
|
||
"tiltaket, så retningen står uten hjemmel"
|
||
),
|
||
"other": ("en kontroll rapporten ikke kjenner navnet på: begrunnelsen står ordrett under"),
|
||
"not_evaluated": ("ingenting: kjøringen stoppet før tilnærmingen ble vurdert i det hele tatt"),
|
||
}
|
||
|
||
ROUND_BUILD_CONTRACT = """\
|
||
Skriver EN runde i formen v1-gaten leser (se `v1_gate --help` for hele kontrakten):
|
||
|
||
<rounds-dir>/<n>/outbox/ kjøringens egne artefakter, KOPIERT hit (aldri lenket)
|
||
<rounds-dir>/<n>/outcome.json utledet av coverage + artefaktene, aldri håndskrevet
|
||
<rounds-dir>/<n>/report.md rapporten fagpersonen leser og retter
|
||
<rounds-dir>/<n>/feedback.json tilbakemeldingen runden svarer på (n = 1, 2, 3)
|
||
|
||
To filer skriver denne kommandoen ALDRI, og begge er med vilje:
|
||
|
||
<rounds-dir>/<n>/attestering.txt operatørens bekreftelse på at runden faktisk ble holdt.
|
||
Uten den stopper gaten på FORM OK, IKKE BEVIST — og det er riktig: ingen filer kan vise
|
||
at en kjøring skjedde. Formen står i `v1_gate --help`.
|
||
<rounds-dir>/3/report.kept.md runde 3-rapporten slik fagpersonen beholdt den.
|
||
|
||
--ran-at er påkrevd fordi ingen artefakt i utboksen bærer en klokke: filene er
|
||
byte-deterministiske med vilje. Tidspunktet er operatørens opplysning, ikke en måling.
|
||
|
||
feedback_ids står tomt på hver rad. Ingen kjøring sporer i dag hvilken tilbakemelding som
|
||
førte til hvilken rad, og binderen finner ikke på en sporing som ikke finnes.
|
||
"""
|
||
|
||
|
||
# ---------------------------------------------------------------------------------------------
|
||
# Reading the run
|
||
# ---------------------------------------------------------------------------------------------
|
||
|
||
|
||
def _read_json(path: Path) -> Any:
|
||
try:
|
||
return json.loads(path.read_text(encoding="utf-8"))
|
||
except FileNotFoundError as exc:
|
||
raise RoundBuildError(f"{path} mangler") from exc
|
||
except (ValueError, OSError) as exc:
|
||
raise RoundBuildError(f"{path} kan ikke leses ({exc!r})") from exc
|
||
|
||
|
||
def read_run(outbox_dir: Path) -> Run:
|
||
"""The run an outbox directory holds, or ``RoundBuildError`` saying why it holds none.
|
||
|
||
The run is checked against ITSELF here, before anything is written, with the gate's own
|
||
``verify_run``: an artefact that contradicts its coverage row, a family with a missing half,
|
||
and a stray artefact naming an approach nobody commissioned are all refused at the source. A
|
||
builder that copied first and let the reader find those would hand the operator a round that
|
||
is red for something this function already knew.
|
||
"""
|
||
if not outbox_dir.is_dir():
|
||
raise RoundBuildError(f"{outbox_dir} er ingen katalog")
|
||
covers = sorted(p.name for p in outbox_dir.iterdir() if p.name.endswith(_COVERAGE_SUFFIX))
|
||
if not covers:
|
||
raise RoundBuildError(
|
||
f"{outbox_dir} har ingen <kjøring>-coverage.json. Den fila skrives bare når "
|
||
"kjøringen hadde BÅDE --mandate og --outbox-dir, så denne utboksen kan ikke si "
|
||
"hvilke tilnærminger som ble bestilt eller hvorfor hver av dem endte som den gjorde. "
|
||
"En runde kan ikke bygges av en kjøring uten mandat."
|
||
)
|
||
if len(covers) > 1:
|
||
raise RoundBuildError(
|
||
f"{outbox_dir} holder {len(covers)} kjøringer ({', '.join(covers)}) — en runde "
|
||
"presenterer ÉN kjøring, og hvilken kan ikke avgjøres av sorteringsrekkefølge"
|
||
)
|
||
run_id = covers[0][: -len(_COVERAGE_SUFFIX)]
|
||
payload = _read_json(outbox_dir / covers[0])
|
||
try:
|
||
rows = [dict(row) for row in payload["rows"]]
|
||
stamped = str(payload["run_id"])
|
||
stop_reason = str(payload.get("stop_reason", ""))
|
||
ids = [str(row["id"]) for row in rows]
|
||
except (KeyError, TypeError) as exc:
|
||
raise RoundBuildError(f"{covers[0]} mangler et felt ({exc!r})") from exc
|
||
if stamped != run_id:
|
||
raise RoundBuildError(f"{covers[0]} er stemplet kjøring {stamped!r}, ikke {run_id!r}")
|
||
if not rows:
|
||
raise RoundBuildError(f"{covers[0]}: kjøringen evaluerte ingen tilnærming")
|
||
if len(set(ids)) != len(ids):
|
||
raise RoundBuildError(f"{covers[0]}: samme tilnærming står flere ganger")
|
||
|
||
unheld = verify_run(outbox_dir, run_id, rows)
|
||
if unheld:
|
||
raise RoundBuildError(f"kjøringen {run_id!r} står ikke inne for seg selv — {unheld}")
|
||
|
||
artefacts, ignored = [], []
|
||
for path in sorted(outbox_dir.iterdir()):
|
||
if path.is_file() and path.name.startswith(f"{run_id}-") and path.suffix == ".json":
|
||
artefacts.append(path.name)
|
||
else:
|
||
ignored.append(path.name)
|
||
proposals = {
|
||
aid: _read_json(outbox_dir / f"{run_id}-{aid}-proposal.json")
|
||
for aid, row in zip(ids, rows, strict=True)
|
||
if row["status"] != "not_evaluated"
|
||
}
|
||
return Run(
|
||
outbox=outbox_dir,
|
||
run_id=run_id,
|
||
rows=tuple(rows),
|
||
stop_reason=stop_reason,
|
||
artefacts=tuple(artefacts),
|
||
ignored=tuple(ignored),
|
||
proposals=proposals,
|
||
)
|
||
|
||
|
||
# ---------------------------------------------------------------------------------------------
|
||
# The outcome file
|
||
# ---------------------------------------------------------------------------------------------
|
||
|
||
|
||
def derive_outcome(
|
||
run: Run, *, ran_at: str, previous: Mapping[str, Any] | None = None
|
||
) -> dict[str, Any]:
|
||
"""``outcome.json`` computed from the run's own coverage and per-approach artefacts.
|
||
|
||
Every column is derived: (b) and (d) from the coverage row the artefacts were just checked
|
||
against, (c) from the gate's ``stage_of``. ``removed`` is the set difference against the
|
||
PREVIOUS round's outcome — derivation, not declaration. The one thing no file knows is when
|
||
the run happened, and that is why ``ran_at`` is an argument.
|
||
"""
|
||
if parse_time(ran_at) is None:
|
||
raise RoundBuildError(
|
||
f"--ran-at {ran_at!r} er ikke et ISO-8601-tidspunkt med tidssone "
|
||
"(f.eks. 2026-09-18T11:20:00+02:00). Gaten leser dette som kjøringens tidspunkt."
|
||
)
|
||
approaches = [
|
||
{
|
||
"id": str(row["id"]),
|
||
"validated": row["status"] == "validated",
|
||
"stage": stage_of(str(row["status"]), str(row.get("detail", ""))),
|
||
"validated_nok": row.get("saving_nok") if row["status"] == "validated" else None,
|
||
"feedback_ids": [],
|
||
}
|
||
for row in run.rows
|
||
]
|
||
here = {row["id"] for row in approaches}
|
||
gone = (
|
||
[
|
||
{"id": str(row["id"]), "feedback_ids": []}
|
||
for row in previous.get("approaches", ())
|
||
if str(row["id"]) not in here
|
||
]
|
||
if previous is not None
|
||
else []
|
||
)
|
||
return {"run_id": run.run_id, "ran_at": ran_at, "approaches": approaches, "removed": gone}
|
||
|
||
|
||
def validated_ore(outcome: Mapping[str, Any]) -> int:
|
||
"""The round's validated saving in øre — quantized PER AMOUNT and summed as integers (kø-(p)).
|
||
|
||
``ledger.to_ore``'s rule, not a second one: three 60000.005 NOK lines are 18000003 øre this
|
||
way and 18000001 if the floats are summed first, and each row is a real amount. Only
|
||
``validated`` rows count; an ``unsupported`` row's numbers held but its direction has no
|
||
hjemmel, and a rejected row's figure is a claim the validator refused.
|
||
"""
|
||
return sum(
|
||
to_ore(float(row["validated_nok"]))
|
||
for row in outcome["approaches"]
|
||
if row["validated"] and row["validated_nok"] is not None
|
||
)
|
||
|
||
|
||
# ---------------------------------------------------------------------------------------------
|
||
# The report — the one artefact a person reads
|
||
# ---------------------------------------------------------------------------------------------
|
||
|
||
|
||
def _kroner(ore: int) -> str:
|
||
sign = "-" if ore < 0 else ""
|
||
whole, rest = divmod(abs(ore), 100)
|
||
return f"{sign}{whole:,}".replace(",", " ") + f",{rest:02d}"
|
||
|
||
|
||
def _antall(value: float) -> str:
|
||
text = f"{value:,.2f}".replace(",", " ").replace(".", ",")
|
||
return text[:-3] if text.endswith(",00") else text
|
||
|
||
|
||
def _stage_short(stage: str) -> str:
|
||
"""The stage's short name: the clause before the colon of its sentence, never the id."""
|
||
return STAGE_PROSE[stage].split(":", 1)[0]
|
||
|
||
|
||
def _status_word(row: Mapping[str, Any]) -> str:
|
||
return {
|
||
"validated": "validert",
|
||
"rejected": "avvist",
|
||
"unsupported": "validert, men uten erklært krav",
|
||
"not_evaluated": "ikke vurdert",
|
||
}.get(str(row["status"]), str(row["status"]))
|
||
|
||
|
||
def _citations_of(payload: Mapping[str, Any]) -> list[dict[str, Any]]:
|
||
return [dict(c) for c in payload.get("provenance", {}).get("citations", ())]
|
||
|
||
|
||
def _citation_lines(citations: Sequence[Mapping[str, Any]], *, prefix: str) -> list[str]:
|
||
"""One quote with the COUNT of places it stands for, or nothing when there is no quote.
|
||
|
||
The count is part of the citation: a run that cited 446 places and one that cited a single
|
||
place both show one quote here, and a reader who cannot tell them apart cannot tell a
|
||
grounded proposal from a decorated one."""
|
||
if not citations:
|
||
return []
|
||
snippet = " ".join(str(citations[0].get("snippet", "")).split())
|
||
if len(snippet) > _SNIPPET_MAX:
|
||
snippet = snippet[:_SNIPPET_MAX].rstrip() + " …"
|
||
if not snippet:
|
||
return []
|
||
return [
|
||
"",
|
||
f"{prefix} (1 av {len(citations)} siterte steder): «{snippet}» — "
|
||
f"{citations[0].get('file', 'ukjent fil')}",
|
||
]
|
||
|
||
|
||
def _one_list_for_every_proposal(run: Run) -> list[dict[str, Any]] | None:
|
||
"""The citation list EVERY proposal in the run carries, when they all carry the same one.
|
||
|
||
Measured 19.09 on the four archived runs: all five proposals of each carried byte-identical
|
||
270-citation lists — the run's whole retrieved context, stamped once per proposal. The cause
|
||
is in the outbox, not in this module reading a wrong field, so no report can make that quote
|
||
say something about the individual measure. What a report can do is stop repeating it five
|
||
times and say once what it actually is."""
|
||
lists = [_citations_of(run.proposals[aid]) for aid in run.proposals]
|
||
if len(lists) < 2 or not lists[0]:
|
||
return None
|
||
keyed = {json.dumps(one, ensure_ascii=False, sort_keys=True) for one in lists}
|
||
return lists[0] if len(keyed) == 1 else None
|
||
|
||
|
||
def _cost_line_notes(run: Run) -> dict[str, list[str]]:
|
||
"""For every cost line a HELD and a FELL approach both touch, the sentence that says so.
|
||
|
||
Measured on the 19.09 report: ``TUN-LYS-01`` was refused under one label and validated under
|
||
another, and the report said nothing — so the reader met two figures for one budget line
|
||
with no way to see that they were the same line. Said on BOTH sides, in the run's own row
|
||
order, or the reader has to find it by comparing sections."""
|
||
status_of = {str(row["id"]): str(row["status"]) for row in run.rows}
|
||
label_of = {str(row["id"]): str(row["label"]) for row in run.rows}
|
||
touching: dict[str, list[str]] = {}
|
||
for row in run.rows:
|
||
aid = str(row["id"])
|
||
payload = run.proposals.get(aid)
|
||
if payload is None:
|
||
continue
|
||
for item in dict(payload.get("proposal", {})).get("affected_items", ()):
|
||
touching.setdefault(str(item["code"]), []).append(aid)
|
||
notes: dict[str, list[str]] = {}
|
||
for code, involved in touching.items():
|
||
held = [a for a in involved if status_of[a] == "validated"]
|
||
fell = [a for a in involved if status_of[a] in ("rejected", "unsupported")]
|
||
if not held or not fell:
|
||
continue
|
||
for aid in involved:
|
||
other_side = fell if status_of[aid] == "validated" else held
|
||
others = [o for o in other_side if o != aid]
|
||
if not others:
|
||
continue
|
||
what = "ble avvist" if status_of[aid] == "validated" else "holdt kontrollen"
|
||
notes.setdefault(aid, []).append(
|
||
f"Merk: kostnadslinjen {code} står også i "
|
||
f"{', '.join('«' + label_of[o] + '»' for o in others)}, som {what}. Samme "
|
||
"kostnadslinje står på begge sider av dommen, og beløpene kan ikke legges "
|
||
"sammen før du har avgjort hvilket tiltak som gjelder."
|
||
)
|
||
return notes
|
||
|
||
|
||
def _lines_about(
|
||
run: Run, aid: str, *, shared_source: bool, notes: Sequence[str] = ()
|
||
) -> list[str]:
|
||
"""The proposal's own words: what it proposed, which cost lines it touched, and one source.
|
||
|
||
``shared_source`` drops the quote here because the run gave every proposal the SAME one and
|
||
the report states it once instead (``_one_list_for_every_proposal``)."""
|
||
payload = run.proposals.get(aid)
|
||
if payload is None:
|
||
return []
|
||
ir = dict(payload.get("proposal", {}))
|
||
out: list[str] = []
|
||
if ir.get("measure"):
|
||
out += ["", str(ir["measure"])]
|
||
items = [
|
||
f"{item['code']} ({_antall(float(item['quantity']))} × "
|
||
f"{_kroner(to_ore(float(item['unit_cost'])))} kroner)"
|
||
for item in ir.get("affected_items", ())
|
||
]
|
||
if items:
|
||
out += ["", "Berørte kostnadslinjer: " + "; ".join(items) + "."]
|
||
for note in notes:
|
||
out += ["", note]
|
||
if not shared_source:
|
||
out += _citation_lines(_citations_of(payload), prefix="Kilde")
|
||
return out
|
||
|
||
|
||
def _change_prose(before: Mapping[str, Any], after: Mapping[str, Any]) -> str:
|
||
def side(row: Mapping[str, Any]) -> str:
|
||
if row["validated"]:
|
||
return f"validert til {_kroner(to_ore(float(row['validated_nok'])))} kroner"
|
||
stage = str(row.get("stage") or "")
|
||
return "ikke vurdert" if stage == "not_evaluated" else f"avvist ({_stage_short(stage)})"
|
||
|
||
return f"{side(before)} → {side(after)}"
|
||
|
||
|
||
def build_report(
|
||
run: Run,
|
||
n: int,
|
||
outcome: Mapping[str, Any],
|
||
*,
|
||
previous: Mapping[str, Any] | None = None,
|
||
previous_labels: Mapping[str, str] | None = None,
|
||
) -> str:
|
||
"""``report.md`` — what a domain expert reads and corrects, in Norwegian prose.
|
||
|
||
Deterministic for a given (outbox, round, previous): the sections are a fixed sequence and
|
||
every list follows the coverage's OWN order. Row 4 of the gate counts content lines in order,
|
||
so a report that reshuffled between builds would read as an expert's edit.
|
||
"""
|
||
rows = {str(row["id"]): row for row in outcome["approaches"]}
|
||
shared = _one_list_for_every_proposal(run)
|
||
notes = _cost_line_notes(run)
|
||
fell = [row for row in run.rows if str(row["status"]) in ("rejected", "unsupported")]
|
||
held = [row for row in run.rows if str(row["status"]) == "validated"]
|
||
missed = [row for row in run.rows if str(row["status"]) == "not_evaluated"]
|
||
|
||
out: list[str] = [f"# Rapport fra runde {n} — kjøring {run.run_id}", ""]
|
||
out += [
|
||
"Denne rapporten er bygget maskinelt fra kjøringens egen utboks; ingen modell har",
|
||
"skrevet den. Den sier hvilke kostnadstiltak systemet foreslo, hvilke som holdt den",
|
||
"deterministiske kontrollen, og hvorfor de øvrige falt. Rett den fritt — det du beholder",
|
||
"og det du skriver om, er selve målingen.",
|
||
"",
|
||
]
|
||
|
||
out += ["## Hva ble vurdert", ""]
|
||
stopped = (
|
||
f" Kjøringen stoppet før den var ferdig ({run.stop_reason})." if run.stop_reason else ""
|
||
)
|
||
out += [
|
||
f"Kommisjonen ba om {len(run.rows)} tilnærminger, og kjøringen rakk "
|
||
f"{len(run.rows) - len(missed)} av dem.{stopped}",
|
||
"",
|
||
]
|
||
out += [f"- **{row['label']}** — {_status_word(row)}" for row in run.rows]
|
||
out += [""]
|
||
if shared is not None:
|
||
out += [
|
||
f"Alle {len(run.proposals)} forslagene i denne kjøringen viser til NØYAKTIG samme",
|
||
"kildeliste, i samme rekkefølge. Listen er hva kjøringen leste, ikke hva det enkelte",
|
||
"tiltaket bygger på, så den står her én gang i stedet for under hvert forslag.",
|
||
]
|
||
out += _citation_lines(shared, prefix="Felles kilde")
|
||
out += [""]
|
||
|
||
out += ["## Hva holdt, og hva det er verdt", ""]
|
||
if held:
|
||
out += [
|
||
f"{len(held)} av {len(run.rows)} tilnærminger holdt kontrollen. Samlet validert "
|
||
f"besparelse: {_kroner(validated_ore(outcome))} kroner.",
|
||
"",
|
||
]
|
||
for row in held:
|
||
aid = str(row["id"])
|
||
amount = _kroner(to_ore(float(rows[aid]["validated_nok"])))
|
||
out += [f"### {row['label']} — {amount} kroner"]
|
||
out += _lines_about(
|
||
run, aid, shared_source=shared is not None, notes=notes.get(aid, ())
|
||
)
|
||
out += [""]
|
||
else:
|
||
out += ["Ingen tilnærming holdt kontrollen i denne kjøringen.", ""]
|
||
|
||
out += ["## Hva falt, og hvorfor", ""]
|
||
if fell:
|
||
out += [f"{len(fell)} tilnærminger ble ikke godtatt.", ""]
|
||
for row in fell:
|
||
aid = str(row["id"])
|
||
out += [f"### {row['label']} — falt på {STAGE_PROSE[str(rows[aid]['stage'])]}"]
|
||
out += _lines_about(
|
||
run, aid, shared_source=shared is not None, notes=notes.get(aid, ())
|
||
)
|
||
out += ["", f"Kontrollens egen begrunnelse, ordrett: «{row.get('detail', '')}»", ""]
|
||
else:
|
||
out += ["Ingen tilnærming ble avvist i denne kjøringen.", ""]
|
||
|
||
out += ["## Hva kjøringen aldri rakk", ""]
|
||
if missed:
|
||
out += [
|
||
f"{len(missed)} tilnærminger ble aldri vurdert. De er verken godtatt eller avvist.",
|
||
"",
|
||
]
|
||
out += [
|
||
f"- **{row['label']}** — {row.get('detail') or 'ingen grunn oppgitt'}" for row in missed
|
||
]
|
||
out += [""]
|
||
else:
|
||
out += ["Kjøringen rakk alle tilnærmingene kommisjonen ba om.", ""]
|
||
|
||
if n >= 1:
|
||
out += ["## Endret siden forrige runde", ""]
|
||
out += _changed_section(run, n, outcome, previous or {}, previous_labels or {})
|
||
|
||
out += ["## Slik leser du tallene", ""]
|
||
out += [
|
||
"- «Validert» betyr at forslagets egne tall holdt en deterministisk kontroll mot",
|
||
" prosjektets kostnadsbasis og en usikkerhetsberegning. Det er ikke en beslutning om å",
|
||
" gjennomføre tiltaket, og ingen har vurdert om tiltaket er faglig forsvarlig.",
|
||
"- «Validert, men uten erklært krav» betyr at tallene holdt, men at ingen krav i",
|
||
" kunnskapsbasen ble erklært å binde tiltaket. Beløpet telles ikke med i summen.",
|
||
"- Beløpene er kvantisert til hele øre per beløp før de summeres, aldri etter.",
|
||
"- Om kjøringen faktisk ble gjort, og når, står i ingen fil her. Det bekrefter du selv.",
|
||
"",
|
||
]
|
||
return "\n".join(out).rstrip("\n") + "\n"
|
||
|
||
|
||
def _changed_section(
|
||
run: Run,
|
||
n: int,
|
||
outcome: Mapping[str, Any],
|
||
previous: Mapping[str, Any],
|
||
previous_labels: Mapping[str, str],
|
||
) -> list[str]:
|
||
"""Every row that moved against round n-1, and for each one whether a feedback id explains it.
|
||
|
||
Model noise between two runs looks exactly like an answered objection, so a row no feedback id
|
||
accounts for has to SAY so. Today that is every row: no run records the tracking.
|
||
"""
|
||
before = {str(row["id"]): row for row in previous.get("approaches", ())}
|
||
label_of = {str(row["id"]): str(row["label"]) for row in run.rows}
|
||
|
||
def why(row: Mapping[str, Any]) -> str:
|
||
ids = list(row.get("feedback_ids", ()))
|
||
return f"Utløst av {', '.join(ids)}" if ids else "Ingen tilbakemelding forklarer dette"
|
||
|
||
out = [f"Sammenlignet med runde {n - 1}:", ""]
|
||
moved = 0
|
||
for row in outcome["approaches"]:
|
||
aid = str(row["id"])
|
||
was = before.get(aid)
|
||
name = label_of.get(aid, aid)
|
||
if was is None:
|
||
moved += 1
|
||
out.append(f"- **{name}** — ny i denne runden. {why(row)}.")
|
||
elif row_changed(was, row):
|
||
moved += 1
|
||
out.append(f"- **{name}** — {_change_prose(was, row)}. {why(row)}.")
|
||
for row in outcome["removed"]:
|
||
moved += 1
|
||
# The one row whose human name is NOT in this run's coverage — it is only in the round
|
||
# it disappeared from. A bare id here would hand the expert an identifier they have no
|
||
# way to look up, in the section that exists for them to judge what moved.
|
||
gone = str(row["id"])
|
||
known = previous_labels.get(gone)
|
||
shown = f"**{known}** ({gone})" if known else f"**{gone}**"
|
||
out.append(
|
||
f"- {shown} — tilnærmingen er borte: den står ikke i denne kjøringens "
|
||
f"liste i det hele tatt. {why(row)}."
|
||
)
|
||
if not moved:
|
||
out.append("- Ingenting endret seg i (a)-(d) over støygrensen.")
|
||
out.append("")
|
||
return out
|
||
|
||
|
||
# ---------------------------------------------------------------------------------------------
|
||
# Building the round
|
||
# ---------------------------------------------------------------------------------------------
|
||
|
||
|
||
def _labels_of(round_dir: Path, outcome: Mapping[str, Any]) -> dict[str, str]:
|
||
"""The labels a round SHOWED, read from the coverage file inside that round's own outbox.
|
||
|
||
Derived from what the round already carries rather than declared: ``outcome.json`` keeps its
|
||
four columns, and an approach that is gone from THIS run's coverage is still named in the
|
||
round it vanished from. An unreadable or absent coverage leaves the mapping empty, and the
|
||
caller falls back on the bare id — a missing label is not a reason to refuse a round."""
|
||
name = f"{outcome.get('run_id', '')}{_COVERAGE_SUFFIX}"
|
||
try:
|
||
rows = json.loads((round_dir / RUN_OUTBOX / name).read_text(encoding="utf-8"))["rows"]
|
||
return {str(row["id"]): str(row["label"]) for row in rows}
|
||
except (OSError, ValueError, KeyError, TypeError):
|
||
return {}
|
||
|
||
|
||
def _dump(payload: Mapping[str, Any]) -> str:
|
||
"""Byte-deterministic on-disk form, mirroring ``outbox._dump`` with Norwegian text kept as is."""
|
||
return json.dumps(payload, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
|
||
|
||
|
||
def build_round(
|
||
outbox_dir: Path,
|
||
rounds_dir: Path,
|
||
n: int,
|
||
*,
|
||
ran_at: str,
|
||
feedback: Path | None = None,
|
||
) -> Built:
|
||
"""Write ``<rounds-dir>/<n>/`` from ``outbox_dir``, or refuse without touching the tree.
|
||
|
||
Every check that can fail runs BEFORE the first byte is written, so a refusal never leaves a
|
||
half-built round for the next command to read as a whole one.
|
||
"""
|
||
if not 0 <= n <= _LAST_ROUND:
|
||
raise RoundBuildError(f"runde {n} finnes ikke i kontrakten — gaten leser 0 til 3")
|
||
if not safe_rounds_dir(rounds_dir):
|
||
raise RoundBuildError(
|
||
f"{rounds_dir} ligger i repoet uten å være ignorert av gitignore — fagpersonens "
|
||
"tilbakemelding kunne da bli committet til den offentlige remoten"
|
||
)
|
||
round_dir = rounds_dir / str(n)
|
||
# BEFORE ``exists()``, because ``exists()`` FOLLOWS the link: a DANGLING link answers False
|
||
# and slipped past the guard below, after which the build died on the filesystem's own
|
||
# FileExistsError (measured 19.09). A refusal has to be a refusal in the form the operator
|
||
# sees, and the link is left exactly as it was found.
|
||
if round_dir.is_symlink():
|
||
raise RoundBuildError(
|
||
f"{round_dir} er en lenke ({round_dir.readlink()}), ikke en katalog. En runde er skrevet "
|
||
"i rundekatalogen selv — en lenke ville latt rundens innhold ligge et sted gaten "
|
||
"ikke måler, og en hengende lenke ville dessuten sluppet forbi «overskrives aldri». "
|
||
"Fjern lenken selv om runden skal bygges her."
|
||
)
|
||
if round_dir.exists():
|
||
raise RoundBuildError(
|
||
f"{round_dir} finnes allerede. En runde holder fagpersonens egen tilbakemelding og "
|
||
"rettinger; den overskrives aldri. Flytt eller slett den selv om den skal bygges om."
|
||
)
|
||
if n == 0 and feedback is not None:
|
||
raise RoundBuildError(
|
||
"runde 0 er grunnkjøringen og svarer på ingenting — det finnes ingen rapport noen "
|
||
"kan ha kommentert ennå, så den tar ingen feedback.json"
|
||
)
|
||
previous: dict[str, Any] | None = None
|
||
previous_labels: dict[str, str] = {}
|
||
if n >= 1:
|
||
if feedback is None:
|
||
raise RoundBuildError(
|
||
f"runde {n} er tilbakemelding på rapport {n - 1} og så en kjøring: den trenger "
|
||
"fagpersonens feedback.json (--feedback)"
|
||
)
|
||
if not feedback.is_file():
|
||
raise RoundBuildError(f"--feedback {feedback} finnes ikke")
|
||
earlier = rounds_dir / str(n - 1) / "outcome.json"
|
||
if not earlier.is_file():
|
||
raise RoundBuildError(
|
||
f"runde {n} måles mot runde {n - 1}, og {earlier} finnes ikke — bygg den runden "
|
||
"først"
|
||
)
|
||
previous = _read_json(earlier)
|
||
previous_labels = _labels_of(rounds_dir / str(n - 1), previous)
|
||
|
||
run = read_run(outbox_dir)
|
||
outcome = derive_outcome(run, ran_at=ran_at, previous=previous)
|
||
report = build_report(run, n, outcome, previous=previous, previous_labels=previous_labels)
|
||
|
||
outbox = round_dir / RUN_OUTBOX
|
||
outbox.mkdir(parents=True)
|
||
for name in run.artefacts:
|
||
shutil.copyfile(run.outbox / name, outbox / name)
|
||
(round_dir / "outcome.json").write_text(_dump(outcome), encoding="utf-8")
|
||
(round_dir / "report.md").write_text(report, encoding="utf-8")
|
||
if feedback is not None:
|
||
shutil.copyfile(feedback, round_dir / "feedback.json")
|
||
|
||
return Built(
|
||
round_dir=round_dir,
|
||
run_id=run.run_id,
|
||
copied=run.artefacts,
|
||
ignored=run.ignored,
|
||
evaluated=tuple(str(r["id"]) for r in run.rows if str(r["status"]) != "not_evaluated"),
|
||
not_evaluated=tuple(str(r["id"]) for r in run.rows if str(r["status"]) == "not_evaluated"),
|
||
validated_ore=validated_ore(outcome),
|
||
)
|
||
|
||
|
||
def main(argv: Sequence[str] | None = None) -> int:
|
||
"""Command-line front door; 0 on a built round, 1 on a refusal, 2 on wrong usage."""
|
||
parser = argparse.ArgumentParser(
|
||
prog="python -m portfolio_optimiser.evals.round_builder",
|
||
description="Bygg én rundekatalog av en ferdig kjørings utboks, i formen v1-gaten leser. "
|
||
"Deterministisk og offline: ingen modellkall, intet nett, ingen klokke.",
|
||
epilog=ROUND_BUILD_CONTRACT,
|
||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||
)
|
||
parser.add_argument("--outbox", required=True, help="kjøringens utboks-katalog")
|
||
parser.add_argument("--round", type=int, required=True, help="rundenummeret (0-3)")
|
||
parser.add_argument(
|
||
"--rounds-dir",
|
||
default=None,
|
||
help=f"rundekatalogen (default {DEFAULT_ROUNDS_DIR}/, gitignored)",
|
||
)
|
||
parser.add_argument(
|
||
"--ran-at",
|
||
required=True,
|
||
help="da kjøringen ble gjort, ISO-8601 MED tidssone. Påkrevd fordi ingen artefakt i "
|
||
"utboksen bærer en klokke — dette er din opplysning, ikke en måling",
|
||
)
|
||
parser.add_argument(
|
||
"--feedback", default=None, help="fagpersonens feedback.json (påkrevd for runde 1-3)"
|
||
)
|
||
args = parser.parse_args(argv)
|
||
|
||
rounds_dir = Path(args.rounds_dir) if args.rounds_dir else _REPO_ROOT / DEFAULT_ROUNDS_DIR
|
||
try:
|
||
built = build_round(
|
||
Path(args.outbox),
|
||
rounds_dir,
|
||
args.round,
|
||
ran_at=args.ran_at,
|
||
feedback=Path(args.feedback) if args.feedback else None,
|
||
)
|
||
except RoundBuildError as refused:
|
||
print(f"runden ble ikke bygget: {refused}", file=sys.stderr)
|
||
return 1
|
||
print(f"{built.round_dir} bygget av kjøring {built.run_id}")
|
||
print(
|
||
f" {len(built.copied)} artefakt(er) kopiert, {len(built.ignored)} fil(er) utenfor "
|
||
f"kjøringen ble stående igjen: {', '.join(built.ignored) or 'ingen'}"
|
||
)
|
||
print(
|
||
f" {len(built.evaluated)} tilnærming(er) vurdert, {len(built.not_evaluated)} ikke; "
|
||
f"validert besparelse {_kroner(built.validated_ore)} kroner"
|
||
)
|
||
print(" attesteringen skriver denne kommandoen aldri — gaten stopper på FORM OK uten den")
|
||
return 0
|
||
|
||
|
||
if __name__ == "__main__": # pragma: no cover - exercised by a subprocess test
|
||
raise SystemExit(main())
|