open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

fix(round-builder): a link is refused with a reason, an identical quote is said once, a shared cost line is named on both sides [skip-docs]

Browse source 
The four arms that were red on assert, in the order the reader meets them.

A DANGLING SYMLINK as the round directory now refuses instead of tracebacking.
exists() FOLLOWS a link, so a dangling one answers False and slipped straight past
"a round is never overwritten"; the build then died on the filesystem's own
FileExistsError. is_symlink() is checked FIRST, the message says what a link would cost
(the round's content would sit somewhere the gate does not measure), and the link is
left exactly as it was found — nothing is written, exit code 1 like every other refusal.

ONE CITATION LIST SHARED BY EVERY PROPOSAL is now stated once. The cause was measured
before anything was written, because "the builder reads the wrong field" and "the outbox
says the same thing five times" want opposite fixes: in all four archived runs every
proposal carries a byte-identical 270-citation list — the run's whole retrieved context,
stamped once per proposal. No report can make that quote say something about the
individual measure. So when every proposal carries the same list, the report says so
once, says what the list actually is ("hva kjøringen leste, ikke hva det enkelte tiltaket
bygger på"), and drops the five copies. When the lists differ, nothing changes: the quote
and its COUNT stay under each proposal, which is where they mean something.

THE SAME COST LINE ON BOTH SIDES OF THE VERDICT is named where it happens. The 19.09
report refused TUN-LYS-01 under one label and validated the same line under another and
said nothing, so a reader met two figures for one budget line with no way to see they
collided. Both sides now carry the sentence, in the run's own row order.

A REMOVED APPROACH is shown by the label the expert saw, with the id in parentheses. It
is the one row whose human name is not in this run's coverage, so the label is read from
the coverage inside the PREVIOUS round's own outbox — derived from what the round already
carries, not a new column in outcome.json. An unreadable coverage falls back on the bare
id; a missing label is not a reason to refuse a round.

39 of 39 arms green. Nothing here touches what the gate reads: outcome.json keeps its
four columns and the builder still never writes the attestation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 10:20:30 +02:00
commit b8b83fb267
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
1 changed files with 152 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

175
src/portfolio_optimiser/evals/round_builder.py
View file

@ -312,8 +312,90 @@ def _status_word(row: Mapping[str, Any]) -> str:
}.get(str(row["status"]), str(row["status"]))
def _lines_about(run: Run, aid: str) -> list[str]:
"""The proposal's own words: what it proposed, which cost lines it touched, and one source."""
def _citations_of(payload: Mapping[str, Any]) -> list[dict[str, Any]]:
return [dict(c) for c in payload.get("provenance", {}).get("citations", ())]
def _citation_lines(citations: Sequence[Mapping[str, Any]], *, prefix: str) -> list[str]:
"""One quote with the COUNT of places it stands for, or nothing when there is no quote.
The count is part of the citation: a run that cited 446 places and one that cited a single
place both show one quote here, and a reader who cannot tell them apart cannot tell a
grounded proposal from a decorated one."""
if not citations:
return []
snippet = " ".join(str(citations[0].get("snippet", "")).split())
if len(snippet) > _SNIPPET_MAX:
snippet = snippet[:_SNIPPET_MAX].rstrip() + " …"
if not snippet:
return []
return [
"",
f"{prefix} (1 av {len(citations)} siterte steder): «{snippet}» — "
f"{citations[0].get('file', 'ukjent fil')}",
]
def _one_list_for_every_proposal(run: Run) -> list[dict[str, Any]] | None:
"""The citation list EVERY proposal in the run carries, when they all carry the same one.
Measured 19.09 on the four archived runs: all five proposals of each carried byte-identical
270-citation lists — the run's whole retrieved context, stamped once per proposal. The cause
is in the outbox, not in this module reading a wrong field, so no report can make that quote
say something about the individual measure. What a report can do is stop repeating it five
times and say once what it actually is."""
lists = [_citations_of(run.proposals[aid]) for aid in run.proposals]
if len(lists) < 2 or not lists[0]:
return None
keyed = {json.dumps(one, ensure_ascii=False, sort_keys=True) for one in lists}
return lists[0] if len(keyed) == 1 else None
def _cost_line_notes(run: Run) -> dict[str, list[str]]:
"""For every cost line a HELD and a FELL approach both touch, the sentence that says so.
Measured on the 19.09 report: ``TUN-LYS-01`` was refused under one label and validated under
another, and the report said nothing — so the reader met two figures for one budget line
with no way to see that they were the same line. Said on BOTH sides, in the run's own row
order, or the reader has to find it by comparing sections."""
status_of = {str(row["id"]): str(row["status"]) for row in run.rows}
label_of = {str(row["id"]): str(row["label"]) for row in run.rows}
touching: dict[str, list[str]] = {}
for row in run.rows:
aid = str(row["id"])
payload = run.proposals.get(aid)
if payload is None:
continue
for item in dict(payload.get("proposal", {})).get("affected_items", ()):
touching.setdefault(str(item["code"]), []).append(aid)
notes: dict[str, list[str]] = {}
for code, involved in touching.items():
held = [a for a in involved if status_of[a] == "validated"]
fell = [a for a in involved if status_of[a] in ("rejected", "unsupported")]
if not held or not fell:
continue
for aid in involved:
other_side = fell if status_of[aid] == "validated" else held
others = [o for o in other_side if o != aid]
if not others:
continue
what = "ble avvist" if status_of[aid] == "validated" else "holdt kontrollen"
notes.setdefault(aid, []).append(
f"Merk: kostnadslinjen {code} står også i "
f"{', '.join('«' + label_of[o] + '»' for o in others)}, som {what}. Samme "
"kostnadslinje står på begge sider av dommen, og beløpene kan ikke legges "
"sammen før du har avgjort hvilket tiltak som gjelder."
)
return notes
def _lines_about(
run: Run, aid: str, *, shared_source: bool, notes: Sequence[str] = ()
) -> list[str]:
"""The proposal's own words: what it proposed, which cost lines it touched, and one source.
``shared_source`` drops the quote here because the run gave every proposal the SAME one and
the report states it once instead (``_one_list_for_every_proposal``)."""
payload = run.proposals.get(aid)
if payload is None:
return []
@ -328,20 +410,10 @@ def _lines_about(run: Run, aid: str) -> list[str]:
]
if items:
out += ["", "Berørte kostnadslinjer: " + "; ".join(items) + "."]
citations = list(payload.get("provenance", {}).get("citations", ()))
if citations:
snippet = " ".join(str(citations[0].get("snippet", "")).split())
if len(snippet) > _SNIPPET_MAX:
snippet = snippet[:_SNIPPET_MAX].rstrip() + " …"
if snippet:
# The COUNT is part of the citation: a run that cited 446 places and one that cited
# one both show a single quote here, and a reader who cannot tell them apart cannot
# tell a grounded proposal from a decorated one.
out += [
"",
f"Kilde (1 av {len(citations)} siterte steder): «{snippet}» — "
f"{citations[0].get('file', 'ukjent fil')}",
]
for note in notes:
out += ["", note]
if not shared_source:
out += _citation_lines(_citations_of(payload), prefix="Kilde")
return out
@ -356,7 +428,12 @@ def _change_prose(before: Mapping[str, Any], after: Mapping[str, Any]) -> str:
def build_report(
run: Run, n: int, outcome: Mapping[str, Any], *, previous: Mapping[str, Any] | None = None
run: Run,
n: int,
outcome: Mapping[str, Any],
*,
previous: Mapping[str, Any] | None = None,
previous_labels: Mapping[str, str] | None = None,
) -> str:
"""``report.md`` — what a domain expert reads and corrects, in Norwegian prose.
@ -365,6 +442,8 @@ def build_report(
so a report that reshuffled between builds would read as an expert's edit.
"""
rows = {str(row["id"]): row for row in outcome["approaches"]}
shared = _one_list_for_every_proposal(run)
notes = _cost_line_notes(run)
fell = [row for row in run.rows if str(row["status"]) in ("rejected", "unsupported")]
held = [row for row in run.rows if str(row["status"]) == "validated"]
missed = [row for row in run.rows if str(row["status"]) == "not_evaluated"]
@ -389,6 +468,14 @@ def build_report(
]
out += [f"- **{row['label']}** — {_status_word(row)}" for row in run.rows]
out += [""]
if shared is not None:
out += [
f"Alle {len(run.proposals)} forslagene i denne kjøringen viser til NØYAKTIG samme",
"kildeliste, i samme rekkefølge. Listen er hva kjøringen leste, ikke hva det enkelte",
"tiltaket bygger på, så den står her én gang i stedet for under hvert forslag.",
]
out += _citation_lines(shared, prefix="Felles kilde")
out += [""]
out += ["## Hva holdt, og hva det er verdt", ""]
if held:
@ -401,7 +488,9 @@ def build_report(
aid = str(row["id"])
amount = _kroner(to_ore(float(rows[aid]["validated_nok"])))
out += [f"### {row['label']} — {amount} kroner"]
out += _lines_about(run, aid)
out += _lines_about(
run, aid, shared_source=shared is not None, notes=notes.get(aid, ())
)
out += [""]
else:
out += ["Ingen tilnærming holdt kontrollen i denne kjøringen.", ""]
@ -412,7 +501,9 @@ def build_report(
for row in fell:
aid = str(row["id"])
out += [f"### {row['label']} — falt på {STAGE_PROSE[str(rows[aid]['stage'])]}"]
out += _lines_about(run, aid)
out += _lines_about(
run, aid, shared_source=shared is not None, notes=notes.get(aid, ())
)
out += ["", f"Kontrollens egen begrunnelse, ordrett: «{row.get('detail', '')}»", ""]
else:
out += ["Ingen tilnærming ble avvist i denne kjøringen.", ""]
@ -432,7 +523,7 @@ def build_report(
if n >= 1:
out += ["## Endret siden forrige runde", ""]
out += _changed_section(run, n, outcome, previous or {})
out += _changed_section(run, n, outcome, previous or {}, previous_labels or {})
out += ["## Slik leser du tallene", ""]
out += [
@ -449,7 +540,11 @@ def build_report(
def _changed_section(
run: Run, n: int, outcome: Mapping[str, Any], previous: Mapping[str, Any]
run: Run,
n: int,
outcome: Mapping[str, Any],
previous: Mapping[str, Any],
previous_labels: Mapping[str, str],
) -> list[str]:
"""Every row that moved against round n-1, and for each one whether a feedback id explains it.
@ -477,8 +572,14 @@ def _changed_section(
out.append(f"- **{name}** — {_change_prose(was, row)}. {why(row)}.")
for row in outcome["removed"]:
moved += 1
# The one row whose human name is NOT in this run's coverage — it is only in the round
# it disappeared from. A bare id here would hand the expert an identifier they have no
# way to look up, in the section that exists for them to judge what moved.
gone = str(row["id"])
known = previous_labels.get(gone)
shown = f"**{known}** ({gone})" if known else f"**{gone}**"
out.append(
f"- **{row['id']}** — tilnærmingen er borte: den står ikke i denne kjøringens "
f"- {shown} — tilnærmingen er borte: den står ikke i denne kjøringens "
f"liste i det hele tatt. {why(row)}."
)
if not moved:
@ -492,6 +593,21 @@ def _changed_section(
# ---------------------------------------------------------------------------------------------
def _labels_of(round_dir: Path, outcome: Mapping[str, Any]) -> dict[str, str]:
"""The labels a round SHOWED, read from the coverage file inside that round's own outbox.
Derived from what the round already carries rather than declared: ``outcome.json`` keeps its
four columns, and an approach that is gone from THIS run's coverage is still named in the
round it vanished from. An unreadable or absent coverage leaves the mapping empty, and the
caller falls back on the bare id — a missing label is not a reason to refuse a round."""
name = f"{outcome.get('run_id', '')}{_COVERAGE_SUFFIX}"
try:
rows = json.loads((round_dir / RUN_OUTBOX / name).read_text(encoding="utf-8"))["rows"]
return {str(row["id"]): str(row["label"]) for row in rows}
except (OSError, ValueError, KeyError, TypeError):
return {}
def _dump(payload: Mapping[str, Any]) -> str:
"""Byte-deterministic on-disk form, mirroring ``outbox._dump`` with Norwegian text kept as is."""
return json.dumps(payload, ensure_ascii=False, indent=2, sort_keys=True) + "\n"
@ -518,6 +634,17 @@ def build_round(
"tilbakemelding kunne da bli committet til den offentlige remoten"
)
round_dir = rounds_dir / str(n)
# BEFORE ``exists()``, because ``exists()`` FOLLOWS the link: a DANGLING link answers False
# and slipped past the guard below, after which the build died on the filesystem's own
# FileExistsError (measured 19.09). A refusal has to be a refusal in the form the operator
# sees, and the link is left exactly as it was found.
if round_dir.is_symlink():
raise RoundBuildError(
f"{round_dir} er en lenke ({round_dir.readlink()}), ikke en katalog. En runde er skrevet "
"i rundekatalogen selv — en lenke ville latt rundens innhold ligge et sted gaten "
"ikke måler, og en hengende lenke ville dessuten sluppet forbi «overskrives aldri». "
"Fjern lenken selv om runden skal bygges her."
)
if round_dir.exists():
raise RoundBuildError(
f"{round_dir} finnes allerede. En runde holder fagpersonens egen tilbakemelding og "
@ -529,6 +656,7 @@ def build_round(
"kan ha kommentert ennå, så den tar ingen feedback.json"
)
previous: dict[str, Any] | None = None
previous_labels: dict[str, str] = {}
if n >= 1:
if feedback is None:
raise RoundBuildError(
@ -544,10 +672,11 @@ def build_round(
"først"
)
previous = _read_json(earlier)
previous_labels = _labels_of(rounds_dir / str(n - 1), previous)
run = read_run(outbox_dir)
outcome = derive_outcome(run, ran_at=ran_at, previous=previous)
report = build_report(run, n, outcome, previous=previous)
report = build_report(run, n, outcome, previous=previous, previous_labels=previous_labels)
outbox = round_dir / RUN_OUTBOX
outbox.mkdir(parents=True)