open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(tracing): U14 - sporing er opt-in, og "av" betyr at MAF aldri kalles (ORDRE 20260823T165757Z)

Browse source 
PORTFOLIO_OTEL er eneste bryter, lest paa truthiness. Uten den kalles
configure_otel_providers ikke i det hele tatt: spans lages fortsatt
(ENABLE_INSTRUMENTATION defaulter True) og kastes, saa ingenting KAN forlate
prosessen. `console` skriver spans til stderr - demoens stdout er byte-identisk
med fasiten, maalt. `otlp` eksporterer over nett, og kun mot et endepunkt
operatoeren selv har navngitt.

To regler er MAALT, ikke valgt (observability.py:849 bygger exporter-lista i
fast rekkefoelge):

- enable_console_exporters sendes EKSPLISITT False i begge moduser. Overlatt til
  miljoeet faller den tilbake paa ENABLE_CONSOLE_EXPORTERS, hvis
  ConsoleSpanExporter skriver til STDOUT - nettopp det S6 maalte som oedeleggende
  for goldenen.
- `console` NEKTER naar en OTEL_EXPORTER_OTLP_*_ENDPOINT finnes: env-avledede
  exportere bygges UBETINGET og FOER vaare, saa ordet "console" ville vaert en
  usann paastand om hvor kjoeringens innhold tok veien. Validering, ALDRI
  reparasjon - vi fjerner ikke operatoerens variabel bak ryggen paa dem.

Tre kallsteder (run.main, simulation.main, hosting.main): demoen er et skriptet
bevis, ikke produktet, og en soem bare demoen naar ville latt de to inngangene en
virksomhet faktisk kjoerer vaere usporbare. tracing_notice er ENESTE renderer og
returnerer None naar sporing er av - omisjon, aldri tom rad.

IKKE bygget, med grunn: PLAN_CREATED/REPLANNED/PROGRESS_LEDGER_UPDATED hoerer til
sloeyfa U4 bygger; en emitter uten kallsted er en form gjettet i stedet for maalt.
OTLP-exporter-PAKKENE er bevisst ikke deklarert (egress + grpc/protobuf-vekt i et
publisert wheel); uttalt i README/DEPLOY/env.template.

Ny dep: opentelemetry-sdk>=1.42,<2 (operatoerbeslutning 2, 23.08). EN pakke, ikke
to - ConsoleSpanExporter bor inne i sdk-en. opentelemetry-api fulgte med
1.42.1 -> 1.44.0, maalt uskadelig.

Load-bearing MAALT (tests/test_tracing_loadbearing.py), ni mutasjoner alle roede
mot HELE suiten + groenn kontroll 943/5. Tre av de roede bor i tester som fantes
fra foer (golden-transkriptets fire-linjers stderr + portefoelje-CLI-ens stille
pass), altsaa er omisjons-regelen gatet av uavhengige vitner.

Golden ea8c534... uendret. mypy src + ruff rene.

Ordren tar ogsaa de fire operatoerbeslutningene inn i planens paragraf F.
Laasen paa orchestrations 1.0.1 er ENDELIG (operatoerbekreftelse 23.08), ikke
midlertidig: spike-ordrens "revert hvis E7 staar" er overstyrt av den senere
beslutningen, som betinget paa groenn suite - ikke paa E7.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-08-23 20:51:46 +02:00
commit 4a19d39e63
12 changed files with 928 additions and 16 deletions
Download patch file Download diff file Expand all files Collapse all files

35
CLAUDE.md
View file

@ -737,6 +737,41 @@ Python ≥3.10. MAF (`agent-framework-core` 1.9.0). Pakkehåndtering: `uv`. To b
hvordan prosessen driftes er mottakerens valg. `git archive HEAD` leser HEAD, ikke arbeidstreet, hvordan prosessen driftes er mottakerens valg. `git archive HEAD` leser HEAD, ikke arbeidstreet,
så gaten er ekte men forsinket med én commit (funn 35). Load-bearing MÅLT så gaten er ekte men forsinket med én commit (funn 35). Load-bearing MÅLT
(`tests/test_handover_package_loadbearing.py`). (`tests/test_handover_package_loadbearing.py`).
- **Sporing er OPT-IN, og «av» betyr at MAF ALDRI kalles (U14, økt 55):** `PORTFOLIO_OTEL` leses på
**truthiness** (4b-regelen) og er ENESTE bryter; uten den kalles `configure_otel_providers` ikke i
det hele tatt — spans LAGES fortsatt (`ENABLE_INSTRUMENTATION` defaulter `True`,
`observability.py:697`) og kastes, så ingenting KAN forlate prosessen. Et kall med tom
exporter-liste ville derimot installert providere og lest hver `OTEL_EXPORTER_OTLP_*` i det
omkringliggende miljøet — «av» må være fravær av kall, ikke kall uten innhold. **To regler er
MÅLT, ikke valgt** (`observability.py:849` bygger exporter-lista i fast rekkefølge: (1) env-avledede
OTLP-exportere UBETINGET, (2) de innsendte, (3) `ConsoleSpanExporter()` — default-sink **stdout** —
når `enable_console_exporters` er sann fra argument ELLER `ENABLE_CONSOLE_EXPORTERS`): (a)
`enable_console_exporters=False` sendes EKSPLISITT i BEGGE moduser, ellers gir en operatør med den
variabelen eksportert et span-dump på stdout — nøyaktig det S6 målte som ødeleggende for
golden-transkriptet; (b) `console` NEKTER når en OTLP-endepunkt-variabel finnes, fordi steg (1)
ville lagt til en nettverks-exporter ordet «console» lover ikke er der. **Validering, ALDRI
reparasjon** — vi fjerner ikke operatørens miljøvariabel bak ryggen på dem
(`write_concept_file`-regelen); nekten NAVNGIR variabelen. `otlp` uten deklarert endepunkt nektes
også: providere med ingenting å eksportere til er en kjøring som SER sporet ut og ikke er det. En
ukjent verdi nektes ved navn, aldri stille fallback til av. **`tracing_notice` er ENESTE renderer**,
tar den alt oppløste `TracingSetup` og returnerer `None` når sporing er av — omisjon, aldri tom rad
(`announce`-regelen), og her bærende utover stil: demoens pinnede stderr er FIRE linjer. **Tre
kallsteder, ikke ett** (`run.main`, `simulation.main`, `hosting.main`): demoen er et skriptet bevis,
ikke produktet, og en søm bare demoen når ville latt de to inngangene en virksomhet faktisk kjører
være usporbare. **OTLP-exporter-PAKKENE er BEVISST ikke deklarert** (egress + grpc/protobuf-vekt i
et publisert wheel; MAF raiser selv en `ImportError` som navngir pakka) — uttalt ærlighets-grense.
**`PLAN_CREATED`/`REPLANNED`/`PROGRESS_LEDGER_UPDATED`-eventene planen navngir er IKKE bygget:**
de hører til utforskningssløyfa (U4) som ikke finnes ennå, og en emitter skrevet før kallstedet er
en form gjettet i stedet for målt. Load-bearing MÅLT
(`tests/test_tracing_loadbearing.py`), ni mutasjoner alle røde mot HELE suiten + grønn kontroll
943/5: exporteren tar sin stdout-default (4 røde) · `enable_console_exporters` overlatt til miljøet
(3 røde — inkludert den ATFERDSMESSIGE, som kjører demoen med variabelen eksportert; uten den ville
raden bare vært en keyword-assert) · detach console-nekten (4 røde) · detach otlp-endepunkt-nekten
(1 rød) · ukjent modus faller stille til av (2 røde) · «av» kaller MAF likevel + renderer returnerer
alltid en linje (9 røde, hvorav TRE i tester som fantes fra før — `test_golden_transcript` sin
fire-linjers stderr og `test_portfolio_cli_offline`s stille-pass — altså er omisjonen gatet av
uavhengige vitner) · detach demo-wiringen + detach CLI-wiringen (5 røde) · detach hosting-wiringen
(1 rød, KUN subprosess-testen — P4-presedensen).
- **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet. - **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet.
- Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase. - Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase.

26
DEPLOY.md
View file

@ -90,7 +90,7 @@ You need, in Microsoft Foundry:
--- ---
## 5. Configuration — the two variables that matter ## 5. Configuration — the two variables that matter, and one that is optional
**5.1 The project endpoint.** Resolved as the first non-empty of: **5.1 The project endpoint.** Resolved as the first non-empty of:
@ -128,7 +128,29 @@ export PORTFOLIO_MODEL_MAP=/path/to/model_map.json
If you skip this, the service starts and answers `/readiness` — and fails every invocation. That If you skip this, the service starts and answers `/readiness` — and fails every invocation. That
is the single most likely first-deployment failure, which is why it has its own section. is the single most likely first-deployment failure, which is why it has its own section.
**5.3 Verify before you pay for a model call:** **5.3 Tracing — optional, and off until you ask.** `PORTFOLIO_OTEL` is the only switch:
```bash
export PORTFOLIO_OTEL=console # OpenTelemetry spans on STDERR — in a container, that is your log
```
Unset, no OpenTelemetry provider is configured at all: the spans are still created and then
discarded, so nothing can leave the process. `console` writes them to stderr and leaves stdout
byte-for-byte as it was, so you can trace a run without changing what the run prints. `otlp` sends
them to a collector, and only to an endpoint you name yourself:
```bash
export PORTFOLIO_OTEL=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT=http://your-collector:4317
pip install opentelemetry-exporter-otlp-proto-grpc # not a declared dependency — egress is yours to opt into
```
Asking for `console` while an `OTEL_EXPORTER_OTLP_*_ENDPOINT` variable is set is **refused**, not
quietly honoured: those exporters are constructed unconditionally by the framework underneath, so
the run would have shipped its contents to the collector while calling itself console-only. The
service names its tracing destination on stderr at startup, before the first span exists.
**5.4 Verify before you pay for a model call:**
```bash ```bash
uv run python -m portfolio_optimiser.preflight --profile azure uv run python -m portfolio_optimiser.preflight --profile azure

10
README.md
View file

@ -377,6 +377,16 @@ when the seam is detached, so the loop cannot silently degrade into theater.
Deployment names are never committed: the role→model map ships `REPLACE-WITH-*` placeholders that Deployment names are never committed: the role→model map ships `REPLACE-WITH-*` placeholders that
fail fast, and `PORTFOLIO_MODEL_MAP` points at an out-of-tree map that wins over the bundled one. fail fast, and `PORTFOLIO_MODEL_MAP` points at an out-of-tree map that wins over the bundled one.
- **Tracing:** off unless `PORTFOLIO_OTEL` holds a value, and then it says where the spans go
before it emits one. `console` writes them to **stderr**, so a traced run's stdout is
byte-identical to an untraced one; `otlp` exports them over the network, and only to an endpoint
you named in one of the standard `OTEL_EXPORTER_OTLP_*_ENDPOINT` variables. Asking for `console`
while such a variable is set is refused rather than quietly honoured — those exporters are built
unconditionally by the framework underneath, so "console" would have been a false statement about
where the run's contents went. With the variable unset, no provider is configured at all: spans
are still made and discarded, and nothing can leave the process. The OTLP exporter packages are
not declared dependencies (they are egress, and heavy in a published wheel); install one yourself
if you use that mode.
- **Run:** the `run.py` CLI has **three modes** — a documented partition, since one invocation - **Run:** the `run.py` CLI has **three modes** — a documented partition, since one invocation
cannot exercise every flag: cannot exercise every flag:
- **Single-project** — `PROJECT_ID --docs-dir <dir>`, plus optional `--bundle-dir`, - **Single-project** — `PROJECT_ID --docs-dir <dir>`, plus optional `--bundle-dir`,

50
docs/plan/2026-08-23-magentic-utforskningssloeyfe.md
View file

@ -488,16 +488,48 @@ samme stack (core 1.9.0 / orch 1.0.1 / `opentelemetry-sdk` 1.44.0; **ingenting l
shasum). U14 er altså én økts arbeid — forutsatt at exporteren konstrueres eksplisitt mot stderr, shasum). U14 er altså én økts arbeid — forutsatt at exporteren konstrueres eksplisitt mot stderr,
aldri via flagget. aldri via flagget.
**U14 er LANDET (økt 55, ordre `20260823T165757Z-6762213199`).** Operatøren tok alle fire
beslutningene som anbefalt 23.08, og bekreftet samme dag at **låsen på orchestrations 1.0.1 er
ENDELIG, ikke midlertidig** — spike-ordrens «revert hvis E7 står» er overstyrt av den senere
beslutningen, som betinget på grønn suite (920/5, målt) og ikke på E7. Levert:
* `opentelemetry-sdk>=1.42,<2` som DEKLARERT dependency (beslutning 2). Én ny pakke, ikke to:
`ConsoleSpanExporter` bor inne i sdk-en. `opentelemetry-api` fulgte med fra 1.42.1 til 1.44.0 —
målt uskadelig (golden `ea8c534…` uendret før og etter).
* `src/portfolio_optimiser/tracing.py` — `PORTFOLIO_OTEL` på truthiness, modusene `console`
(spans → stderr) og `otlp` (kun mot et navngitt endepunkt), `tracing_notice` som eneste renderer.
* Wiret i TRE inngangspunkter: `run.main`, `simulation.main`, `hosting.main`.
* **S6-forbeholdet er implementert som skrevet:** `ConsoleSpanExporter(out=…)`, aldri
`enable_console_exporters=True`. Målt: demoen under `PORTFOLIO_OTEL=console` gir `workflow.run`-
spans på stderr og stdout BYTE-IDENTISK med fasiten; uten variabelen null spans.
* **To ting E-tabellen ikke forutså, begge lukket:** (i) MAF bygger env-avledede OTLP-exportere
UBETINGET, så `console` NEKTER når en `OTEL_EXPORTER_OTLP_*_ENDPOINT` er satt — ellers var
«console» en usann påstand om hvor kjøringens innhold tok veien; (ii) `enable_console_exporters`
faller tilbake på `ENABLE_CONSOLE_EXPORTERS` i miljøet, så den pinnes eksplisitt `False` i begge
moduser. Begge er MÅLT som defektklasser, ikke antatt.
* **IKKE bygget, med grunn:** `PLAN_CREATED`/`REPLANNED`/`PROGRESS_LEDGER_UPDATED` — de hører til
sløyfa U4 bygger, og en emitter uten kallsted er en form gjettet i stedet for målt. De landes i
økt 56–57, på sømmen U14 nettopp la. OTLP-exporter-PAKKENE er bevisst ikke deklarert (egress +
grpc/protobuf-vekt i et publisert wheel); uttalt ærlighets-grense i README/DEPLOY/env.template.
Suite etter U14: **943 passed / 5 skipped**, golden `ea8c534…` uendret, `mypy src` + `ruff` rene.
**Beslutninger som trenger operatøren (speiles i avslutningsblokken):** **Beslutninger som trenger operatøren (speiles i avslutningsblokken):**
1. **Versjon — premisset har flyttet seg.** Ordrens «felles hvis» sa: står E7 i 1.0.1, revert til 1. **Versjon — AVGJORT 23.08: låsen står på 1.0.1, ENDELIG.** Bakgrunnen: ordrens «felles hvis» sa
1.0.0. E7 STÅR — men 1.0.1 viste seg å fikse noe større enn det som ble håpet: E2/E4, altså at hvis E7 står i 1.0.1, revert til 1.0.0. E7 STÅR — men 1.0.1 fikset noe større enn det som ble
«en plausibel fasit produsert av null arbeid». Låsen står derfor på **1.0.1** i påvente av håpet (E2/E4, «en plausibel fasit produsert av null arbeid»). Operatøren aksepterte 23.08 at
operatøren. Alternativene: bli på 1.0.1 (anbefalt) · revert til 1.0.0 etter ordrens bokstav · gevinsten veier tyngre enn ordrens bokstav, og at betingelsen som gjelder er «grønn suite»
core-bump til 1.15 nå (egen økt). (målt 920/5), ikke E7. **Ikke midlertidig, og ikke å gjenåpne uten en ny måling.** Bumpen til
2. U14 som deklarert avhengighet (`opentelemetry-sdk` + exporter) i wheel/handover — ja/nei. 1.1.x / core 1.15 er fortsatt en EGEN økt etter denne planen.
3. HITL-leveranse: synkron-først (anbefalt) vs. asynkron i samme plan — **S4 grønn, så asynkron 2. **U14 som deklarert avhengighet — AVGJORT 23.08: ja.** Landet i økt 55; se «U14 er LANDET»
er nå teknisk mulig**; valget er ren rekkefølge, ikke lenger risiko. over. Kostnaden ble ÉN pakke (`opentelemetry-sdk`), ikke to.
4. Commons-amendment «Step 0 — Explore» — sende forslag nå (anbefalt: ja, via coord, uten å vente). 3. **HITL — AVGJORT 23.08: synkron-først.** Merk at ordrens BEGRUNNELSE er utdatert mens
konklusjonen står: den sier «A4 er umålt og kun kode-lest», men A4 ble MÅLT GRØNN i økt 54, så
asynkron er teknisk mulig. Valget er dermed ren rekkefølge, ikke risiko — asynkron U13 forblir
pkt. 3 i rekkefølgen, betinget av S4 som nå er grønn.
4. **Commons-amendment «Step 0 — Explore» — AVGJORT 23.08: sendt.** `.claude` sendte forslaget til
`portfolio-optimiser-commons` via coord. Vi venter IKKE: planen slår selv fast at ingen amendment
er påkrevd, og «Step 0» er foreslått som VALGFRI. Kommer den, er den en tilføyelse.
--- ---

13
env.template
View file

@ -28,3 +28,16 @@ PORTFOLIO_FOUNDRY_PROJECT_ENDPOINT= # tenant-specific Foundry project endpoint
# Hosted only: Foundry injects FOUNDRY_PROJECT_ENDPOINT itself, and the framework reads it as a # Hosted only: Foundry injects FOUNDRY_PROJECT_ENDPOINT itself, and the framework reads it as a
# fallback. The name above wins whenever it holds a non-empty value, so an explicit export is never # fallback. The name above wins whenever it holds a non-empty value, so an explicit export is never
# shadowed by a platform default — and an exported-but-empty name falls through rather than masking. # shadowed by a platform default — and an exported-but-empty name falls through rather than masking.
# --- Tracing (U14) — OFF unless this variable holds a value ---
# Unset or empty: OpenTelemetry providers are NOT configured at all. Spans are still made (MAF's
# ENABLE_INSTRUMENTATION defaults to true) and simply discarded, so nothing can leave the process.
# console -> spans are written to STDERR. stdout is byte-identical to an untraced run, which is
# what keeps the pinned demo transcript usable. Refused if any OTEL_EXPORTER_OTLP_*
# endpoint is also set: those exporters are built unconditionally, so "console"
# would otherwise be a false statement about where the run's contents went.
# otlp -> spans are EXPORTED OVER THE NETWORK, and only to an endpoint you named below.
# Requires an OTLP exporter package, which this framework deliberately does NOT
# declare (`pip install opentelemetry-exporter-otlp-proto-grpc`).
# PORTFOLIO_OTEL=console
# OTEL_EXPORTER_OTLP_ENDPOINT= # required by PORTFOLIO_OTEL=otlp; egress — name it yourself

13
pyproject.toml
View file

@ -28,7 +28,18 @@ dependencies = [
# `uv run mypy src` outright. Raise both floors together or not at all. # `uv run mypy src` outright. Raise both floors together or not at all.
"numpy>=2.0,<2.3", "numpy>=2.0,<2.3",
"llm-ingestion-okf", # Door A ingest (§4–§6) — the shared implementation of shared/ingest-spec.md; zero runtime deps, MAF-free (D7) "llm-ingestion-okf", # Door A ingest (§4–§6) — the shared implementation of shared/ingest-spec.md; zero runtime deps, MAF-free (D7)
"llm-ingestion-guard", # Door A content gate — scans materialized concepts before they reach the bundle; zero runtime deps, MAF-free (D7) "llm-ingestion-guard",
# U14 (økt 55, operatørbeslutning 23.08): the tracing seam's ONE new runtime dependency.
# `opentelemetry-api` was already here transitively via agent-framework-core, but the SDK is
# what `configure_otel_providers()` needs — without it MAF raises ModuleNotFoundError, so the
# seam cannot land as an opt-in without declaring it. The console exporter this framework uses
# (`opentelemetry.sdk.trace.export.ConsoleSpanExporter`) ships INSIDE this package, so console
# mode costs exactly one declaration. The OTLP exporter packages are deliberately NOT declared:
# they are egress, they drag grpc/protobuf into a published wheel, and MAF already raises a
# named ImportError telling the operator which one to install. Stated honesty limit, not an
# oversight — see tracing.py.
"opentelemetry-sdk>=1.42,<2",
# Door A content gate — scans materialized concepts before they reach the bundle; zero runtime deps, MAF-free (D7)
] ]
# Console entry points — the install surface a fresh clone gets from `uv sync` (P4 pkt. 5). # Console entry points — the install surface a fresh clone gets from `uv sync` (P4 pkt. 5).

14
src/portfolio_optimiser/hosting.py
View file

@ -65,11 +65,13 @@ import asyncio
import json import json
import os import os
import signal import signal
import sys
from typing import Any from typing import Any
from portfolio_optimiser.budget import BudgetExceeded from portfolio_optimiser.budget import BudgetExceeded
from portfolio_optimiser.outbox import outcome_payload from portfolio_optimiser.outbox import outcome_payload
from portfolio_optimiser.run import RunResult, run_project from portfolio_optimiser.run import RunResult, run_project
from portfolio_optimiser.tracing import configure_tracing, tracing_notice
DEFAULT_PORT = 8088 DEFAULT_PORT = 8088
_HOSTED_DEFAULT_PROFILE = "azure" _HOSTED_DEFAULT_PROFILE = "azure"
@ -257,5 +259,15 @@ async def _serve_until_sigterm() -> None:
def main() -> None: def main() -> None:
"""Serve the hosted-agent contract until SIGTERM (bind 0.0.0.0 — the platform """Serve the hosted-agent contract until SIGTERM (bind 0.0.0.0 — the platform
terminates TLS in front of us), then exit 0.""" terminates TLS in front of us), then exit 0.
U14: the tracing seam is installed before the loop starts and announced on stderr, which in a
container IS the log. This is the entry where "an organisation must be able to see what the run
did" is actually cashed — the demo is a scripted proof, not the product. A malformed
``PORTFOLIO_OTEL`` propagates: a server whose telemetry cannot be configured as asked must not
start and then look healthy on ``/readiness``."""
setup = configure_tracing()
notice = tracing_notice(setup)
if notice is not None:
print(notice, file=sys.stderr)
asyncio.run(_serve_until_sigterm()) asyncio.run(_serve_until_sigterm())

16
src/portfolio_optimiser/run.py
View file

@ -75,6 +75,7 @@ from portfolio_optimiser.mcp_tools import (
) )
from portfolio_optimiser.provenance import ProvenanceStamp from portfolio_optimiser.provenance import ProvenanceStamp
from portfolio_optimiser.reference_domain import Project, load_reference_projects from portfolio_optimiser.reference_domain import Project, load_reference_projects
from portfolio_optimiser.tracing import TracingConfigError, configure_tracing, tracing_notice
from portfolio_optimiser.validator import Rejection, ValidatedProposal, baseline_from_project from portfolio_optimiser.validator import Rejection, ValidatedProposal, baseline_from_project
from portfolio_optimiser import okf, outbox from portfolio_optimiser import okf, outbox
from portfolio_optimiser.semretrieval import ( from portfolio_optimiser.semretrieval import (
@ -1506,6 +1507,21 @@ def main(argv: list[str] | None = None) -> int:
) )
args = parser.parse_args(argv) args = parser.parse_args(argv)
# U14: the tracing seam, resolved FIRST — ahead of every branch that can return, because MAF's
# contract is "call once at startup, before any telemetry is captured". Without PORTFOLIO_OTEL
# this configures nothing at all and prints nothing (omission, never an empty row), so every
# existing stderr expectation in the suite is untouched. A malformed request exits through this
# CLI's own refusal surface (printed line + rc 1) rather than as a traceback: it is something
# the operator exported and can fix, which is exactly what that surface is for.
try:
tracing_setup = configure_tracing()
except TracingConfigError as exc:
print(f"run refused: {exc}", file=sys.stderr)
return 1
tracing_line = tracing_notice(tracing_setup)
if tracing_line is not None:
print(tracing_line, file=sys.stderr)
# S5.4: read-only value-report dispatch — placed FIRST (right after parse_args, BEFORE the # S5.4: read-only value-report dispatch — placed FIRST (right after parse_args, BEFORE the
# mode-exclusivity block below) so it returns before any model/portfolio path can start and no # mode-exclusivity block below) so it returns before any model/portfolio path can start and no
# later branch can shadow it (the bare `--ledger`-outside-portfolio refusal at the elif below is # later branch can shadow it (the bare `--ledger`-outside-portfolio refusal at the elif below is

11
src/portfolio_optimiser/simulation.py
View file

@ -43,6 +43,7 @@ from portfolio_optimiser.ir import AffectedItem, CostBaseline, CostBaselineLine,
from portfolio_optimiser.persona import load_persona_example from portfolio_optimiser.persona import load_persona_example
from portfolio_optimiser.run import RunResult, run_project from portfolio_optimiser.run import RunResult, run_project
from portfolio_optimiser.shared_root import shared_root from portfolio_optimiser.shared_root import shared_root
from portfolio_optimiser.tracing import configure_tracing, tracing_notice
from portfolio_optimiser.validator import ValidatedProposal from portfolio_optimiser.validator import ValidatedProposal
from portfolio_optimiser.verdicts import ( from portfolio_optimiser.verdicts import (
Verdict, Verdict,
@ -817,6 +818,16 @@ def main(argv: list[str] | None = None) -> int: # pragma: no cover - console tr
# library consumer of this module keeps its own logging configuration. # library consumer of this module keeps its own logging configuration.
quiet_expected_round_cap_notice() quiet_expected_round_cap_notice()
# U14: install OpenTelemetry providers if — and only if — PORTFOLIO_OTEL asks for it, BEFORE
# any workflow runs (MAF's own "call once at startup, before telemetry is captured"). Absent
# the variable this configures nothing at all, which is what keeps the pinned stdout AND the
# four-line pinned stderr byte-identical; console mode writes spans to stderr, never stdout.
# The announcement goes to stderr for the same reason.
setup = configure_tracing()
notice = tracing_notice(setup)
if notice is not None:
print(notice, file=sys.stderr)
work = tempfile.mkdtemp(prefix="po-sim-") work = tempfile.mkdtemp(prefix="po-sim-")
# THE call site (P3, GO): the demo runs the DELIVERED bundle, which ships its own # THE call site (P3, GO): the demo runs the DELIVERED bundle, which ships its own
# `cost-baseline.json` — so the gate is anchored on numbers a domain team wrote, not on numbers # `cost-baseline.json` — so the gate is anchored on numbers a domain team wrote, not on numbers

235
src/portfolio_optimiser/tracing.py Normal file
View file

@ -0,0 +1,235 @@
"""U14 — the opt-in OpenTelemetry seam: one variable, two sinks, and no silent egress.
A run of this framework already *makes* spans: MAF's ``ENABLE_INSTRUMENTATION`` defaults to True
(``observability.py:697``), so every workflow, executor and chat call is instrumented — and, with
no provider configured, every one of those spans is discarded. This module is the provider, and
nothing else. It exists because the exploration loop being built on top of it (U4) hands a manager
the freedom to choose its own next step, and the programme's ordering rule is that nothing which
grants that freedom lands before the freedom can be *watched*. An organisation cannot be asked to
trust an autonomous exploration it cannot read afterwards.
**The contract, in one paragraph.** ``PORTFOLIO_OTEL`` is read on truthiness. Absent or empty:
``configure_otel_providers`` is not called at all, so nothing is configured and nothing can leave —
"off" means off, not "on, exporting to nowhere". ``console``: spans are written to **stderr**, so a
traced run and an untraced run print byte-identical stdout and the pinned demo transcript survives.
``otlp``: spans go over the network, and ONLY when the operator has named a destination in one of
the standard ``OTEL_EXPORTER_OTLP_*_ENDPOINT`` variables. Anything else is refused by name.
**Two rules here are measurements, not preferences.** ``configure_otel_providers`` composes its
exporter list in a fixed order (``observability.py:849``):
1. exporters derived from the standard ``OTEL_EXPORTER_OTLP_*`` variables — **unconditionally**,
2. the exporters passed in as ``exporters=``,
3. a ``ConsoleSpanExporter()`` — default sink ``sys.stdout`` — when ``enable_console_exporters``
is true, taken from the argument *or*, if that is ``None``, from ``ENABLE_CONSOLE_EXPORTERS``
in the environment.
Step 3 is why ``enable_console_exporters=False`` is passed explicitly in **both** modes: left to
the environment, an operator with that variable exported gets a span dump on stdout, which spike
S6 measured as destroying the golden transcript outright. Step 1 is why console mode **refuses**
when an OTLP endpoint variable is present: the word "console" would otherwise be a false statement
about where the run's contents went. The refusal names the variable and leaves it alone —
validation, never repair, the same rule ``write_concept_file`` and ``load_optional_cost_baseline``
follow. Unsetting an operator's environment behind their back would be a fix that hides its own
cause.
**What is deliberately absent.** The OTLP exporter *packages*
(``opentelemetry-exporter-otlp-proto-grpc`` / ``-http``) are not declared dependencies. They are
egress, they drag grpc and protobuf into a published wheel for a mode that is off by default, and
MAF already raises an ``ImportError`` that names the package to install. Stated honesty limit:
``PORTFOLIO_OTEL=otlp`` works only after the operator installs one of them. Equally absent are the
``PLAN_CREATED`` / ``REPLANNED`` / ``PROGRESS_LEDGER_UPDATED`` events the plan names — they belong
to the exploration loop (U4), which does not exist yet, and an emitter written before its call site
is a shape guessed rather than measured.
MAF-touching by construction, so this module never enters the framework-neutral context layer
(``okf.py``); the ``test_okf_is_maf_free`` guard keeps that boundary.
"""
from __future__ import annotations
import os
from collections.abc import Callable, Mapping
from dataclasses import dataclass
from typing import IO, Any, Final
#: The one variable that turns tracing on. Read on TRUTHINESS, never presence (the 4b rule): an
#: exported-but-empty value is a shell accident, and treating it as a request would turn
#: ``export PORTFOLIO_OTEL=`` into a fail-fast on an unknown mode.
TRACING_ENV: Final = "PORTFOLIO_OTEL"
MODE_CONSOLE: Final = "console"
MODE_OTLP: Final = "otlp"
#: The closed set. A value outside it is refused by name rather than falling back to off — an
#: operator who asked for a trace and mistyped would otherwise get the black box this seam exists
#: to remove, arrived at by accident and without a word.
_MODES: Final = (MODE_CONSOLE, MODE_OTLP)
#: Every standard variable that makes ``_get_exporters_from_env`` construct a NETWORK exporter.
#: All four are checked, not just the base one: a run configured only via
#: ``OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`` exports exactly the signal this seam is about.
_OTLP_ENDPOINT_ENVS: Final = (
"OTEL_EXPORTER_OTLP_ENDPOINT",
"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
)
class TracingConfigError(ValueError):
"""A tracing request that cannot be honoured as stated.
``ValueError`` so the hosted flate maps it to 400 through the existing arm rather than needing
a new one: it is a malformed request about this process's configuration, not a crash.
"""
@dataclass(frozen=True)
class TracingSetup:
"""What a run resolved about its own tracing — the single source both callers read.
``mode`` is ``None`` when tracing is off. ``endpoints`` carries ``(variable, value)`` pairs and
is non-empty only in OTLP mode, so the announcement can name the variable the operator edits
rather than only the URL. Both the configuring and the announcing descend from this one value:
a renderer that re-read the environment would be a second resolution of the same rule, free to
disagree with the run it describes (the ``cost_baseline_notice`` rule).
"""
mode: str | None
endpoints: tuple[tuple[str, str], ...]
def declared_otlp_endpoints(env: Mapping[str, str]) -> tuple[tuple[str, str], ...]:
"""Return the OTLP endpoint variables the environment actually declares, in a fixed order.
Truthiness again: an exported-but-empty endpoint variable declares nothing, and MAF's own
``os.getenv`` reads would skip it too, so treating it as a declaration would make console mode
refuse over a destination that does not exist.
"""
return tuple(
(name, env[name].strip()) for name in _OTLP_ENDPOINT_ENVS if env.get(name, "").strip()
)
def resolve_tracing_mode(env: Mapping[str, str]) -> str | None:
"""Read ``PORTFOLIO_OTEL`` into the closed mode set, or ``None`` when tracing is off."""
raw = env.get(TRACING_ENV, "").strip()
if not raw:
return None
if raw not in _MODES:
raise TracingConfigError(
f"{TRACING_ENV}={raw!r} is not a tracing mode. Allowed: "
+ ", ".join(repr(mode) for mode in _MODES)
+ f". Unset {TRACING_ENV} to run without tracing."
)
return raw
def configure_tracing(
*,
env: Mapping[str, str] | None = None,
stream: IO[str] | None = None,
configure: Callable[..., None] | None = None,
) -> TracingSetup:
"""Install OpenTelemetry providers for this process if — and only if — asked to.
Call ONCE, at process startup, before any telemetry is captured (MAF's own instruction; a
second call is a no-op behind its ``_executed_setup`` guard). Returns what was resolved, so the
caller can announce it without re-reading anything.
``configure`` is injectable for the same reason ``run._default_factory`` is: it is the one seam
a test can observe without installing global providers into the pytest process. The real proof
that the seam works is a subprocess running the actual demo, not this argument.
"""
env = os.environ if env is None else env
mode = resolve_tracing_mode(env)
endpoints = declared_otlp_endpoints(env)
if mode is None:
# Not "configure with nothing" — NOT CALLING is what makes "off" mean nothing can leave.
# A call with an empty exporter list would still install providers and re-read every
# OTEL_EXPORTER_OTLP_* variable in the ambient environment.
return TracingSetup(mode=None, endpoints=())
if configure is None: # pragma: no cover - trivial default resolution
from agent_framework.observability import configure_otel_providers
configure = configure_otel_providers
if mode == MODE_CONSOLE:
if endpoints:
named = ", ".join(name for name, _ in endpoints)
raise TracingConfigError(
f"{TRACING_ENV}={MODE_CONSOLE} promises that spans stay in this process, but the "
f"environment declares a network exporter: {named}. OpenTelemetry exporters are "
"built from those variables unconditionally, so the run would also ship its spans "
f"over the wire. Unset them, or ask for {TRACING_ENV}={MODE_OTLP} and say so."
)
# Imported here rather than at module scope: the exporter is constructed only in this
# branch, and every importer of the CLI would otherwise pay for a mode that is off by
# default.
from opentelemetry.sdk.trace.export import ConsoleSpanExporter
# `out=stream` is the whole of console mode's safety. The default sink is sys.stdout, and
# stdout is byte-pinned by tests/golden/demo-transcript.stdout.
exporters: list[Any] | None = [ConsoleSpanExporter(out=stream or _default_stream())]
else:
if not endpoints:
raise TracingConfigError(
f"{TRACING_ENV}={MODE_OTLP} was requested but no endpoint is declared. Set one of: "
+ ", ".join(_OTLP_ENDPOINT_ENVS)
+ ". Configuring providers with nowhere to export would produce a run that looks "
"traced and is not."
)
# No exporter of our own: MAF builds them from the standard variables, and a hand-rolled
# second one would be the duplicate free to drift from the OTel spec.
exporters = None
# `enable_console_exporters=False` is EXPLICIT in both modes, and load-bearing in both: left as
# None it falls back to ENABLE_CONSOLE_EXPORTERS in the environment, whose console exporter
# writes to stdout.
configure(enable_console_exporters=False, exporters=exporters)
return TracingSetup(mode=mode, endpoints=endpoints)
def _default_stream() -> IO[str]:
"""``sys.stderr`` resolved at CALL time, not import time.
The demo replaces neither, but a caller that redirects ``sys.stderr`` before startup should get
the redirected one — an import-time binding would have captured whatever was current when the
module was first imported.
"""
import sys
return sys.stderr
def tracing_notice(setup: TracingSetup) -> str | None:
"""Render what a run says about its own tracing, or ``None`` when there is nothing to say.
ONE renderer with N call sites, never N copies of the wording (kø-(p)), and it takes the
already-resolved ``TracingSetup`` rather than an environment: the printed line and the
providers that were installed then descend from the same single resolution.
``None`` when tracing is off — omission, never an empty row (``mandate.announce``'s rule, the
one ``cost_baseline_notice`` and ``skipped_links_notice`` follow). Here it is load-bearing past
style: the pinned demo stderr is four lines, and a "tracing: off" row would have made it five.
The OTLP form names the VARIABLE beside the value, because the variable is what the operator
edits — and prints one row per declared endpoint rather than only the first, so a run exporting
logs and traces to different collectors declares both. English, like every other line this CLI
prints.
"""
if setup.mode is None:
return None
if setup.mode == MODE_CONSOLE:
return (
f" Tracing: {TRACING_ENV}={MODE_CONSOLE} — OpenTelemetry spans are written to stderr; "
"nothing leaves this process"
)
rows = "\n".join(f" {name} = {value}" for name, value in setup.endpoints)
return (
f" Tracing: {TRACING_ENV}={MODE_OTLP} — OpenTelemetry spans are EXPORTED OVER THE NETWORK "
f"to the endpoints declared below\n{rows}"
)

486
tests/test_tracing_loadbearing.py Normal file
View file

@ -0,0 +1,486 @@
"""U14 (økt 55) — the tracing seam: opt-in OpenTelemetry, and no silent egress.
**Why the seam exists.** The exploration loop this program is building hands a manager the freedom
to decide what to look at next. The programme's load-bearing ordering rule is that *nothing which
gives the manager more freedom lands before we can see what it did with it* — a loop without a
trace is a black box, and a black box is not something an organisation can be asked to trust. The
measured starting point (spike S6, økt 54) is that MAF's own Magentic code emits **zero** spans of
its own, while ``ENABLE_INSTRUMENTATION`` defaults to ``True``, so spans are already being *made*
today and thrown away for want of a provider. This seam is the provider — nothing more.
**What is measured here, and what is deliberately not.**
* Measured: that tracing is OFF unless asked for (and then MAF is never called at all), that
console mode writes to **stderr** so the pinned demo transcript stays byte-identical, that a
console run cannot silently also ship spans over the wire, and that the OTLP mode refuses to
pretend when no endpoint was declared.
* Not built: the ``PLAN_CREATED`` / ``REPLANNED`` / ``PROGRESS_LEDGER_UPDATED`` events the plan
names. They belong to a loop that does not exist yet (U4, økt 56-57). An emitter with no call
site is a shape guessed instead of measured, and this repo has paid for that guess before.
**The measurement that shaped the code, not a preference.** ``configure_otel_providers`` composes
its exporter list in a fixed order (``observability.py:849``): (1) exporters derived from the
standard ``OTEL_EXPORTER_OTLP_*`` environment variables, **unconditionally**, (2) exporters passed
in, (3) a ``ConsoleSpanExporter()`` — whose default sink is **stdout** — if
``enable_console_exporters`` is true, from the argument *or* from ``ENABLE_CONSOLE_EXPORTERS`` in
the environment. Two consequences drive two of the tests below:
1. ``enable_console_exporters`` must be passed **explicitly False**, in both modes. Left to the
environment, an operator with ``ENABLE_CONSOLE_EXPORTERS=true`` exported gets a stdout span
dump — which is precisely the thing S6 measured as destroying the golden transcript.
2. Console mode must **refuse** when an OTLP endpoint variable is set, because step (1) would add
a network exporter that the word "console" promises is not there. Validation, NEVER repair: we
do not unset the operator's environment behind their back (the ``write_concept_file`` rule).
**The subprocess arm is the measurement; the spy arms only prove wiring.** Every unit test here
injects a recorder in place of ``configure_otel_providers``, and a suite made only of those would
prove that we call *something* named right — the exact vacuity 4b's credential tests were rewritten
to escape. So the real arm runs the actual demo as a subprocess under ``PORTFOLIO_OTEL=console``
and reads what came out: at least one ``workflow.run`` span on stderr, and stdout byte-identical to
``tests/golden/demo-transcript.stdout``. Its control runs the same demo with the variable absent
and requires **no** span — without that control, "spans appeared" could not be attributed to the
variable rather than to something the demo does anyway.
"""
from __future__ import annotations
import io
import os
import re
import subprocess
import sys
from pathlib import Path
from typing import Any
import pytest
from portfolio_optimiser import tracing
_GOLDEN_STDOUT = Path(__file__).resolve().parent / "golden" / "demo-transcript.stdout"
class _ConfigureRecorder:
"""Stand-in for ``agent_framework.observability.configure_otel_providers``.
Records every call verbatim. The point of recording rather than counting is that the two
keyword arguments carry the whole safety property: which exporters were handed over, and
whether the console (stdout) exporters were left to the environment to decide.
"""
def __init__(self) -> None:
self.calls: list[dict[str, Any]] = []
def __call__(self, **kwargs: Any) -> None:
self.calls.append(kwargs)
# ---------------------------------------------------------------------------------------------
# OFF by default — and "off" means MAF is never called, not called-with-nothing
# ---------------------------------------------------------------------------------------------
def test_absent_variable_configures_nothing_at_all() -> None:
"""T1: no ``PORTFOLIO_OTEL`` → mode ``None`` and ``configure_otel_providers`` is NEVER called.
The distinction matters: a call with an empty exporter list would still install global
providers and re-read every ``OTEL_EXPORTER_OTLP_*`` variable in the ambient environment. Not
calling is the only shape under which "tracing is off" also means "nothing can leave".
"""
spy = _ConfigureRecorder()
setup = tracing.configure_tracing(env={}, configure=spy)
assert setup.mode is None
assert setup.endpoints == ()
assert spy.calls == []
@pytest.mark.parametrize("raw", ["", " ", "\t\n"])
def test_variable_is_read_on_truthiness_not_presence(raw: str) -> None:
"""T2: an exported-but-empty ``PORTFOLIO_OTEL`` is a shell accident, not a request.
Same rule as ``FOUNDRY_HOSTING_ENVIRONMENT`` (Fase 4b) and ``PORT`` (Fase 4d). Reading this on
presence would turn ``export PORTFOLIO_OTEL=`` into a fail-fast on an unknown mode.
"""
spy = _ConfigureRecorder()
setup = tracing.configure_tracing(env={tracing.TRACING_ENV: raw}, configure=spy)
assert setup.mode is None
assert spy.calls == []
# ---------------------------------------------------------------------------------------------
# console mode — spans to stderr, stdout untouched
# ---------------------------------------------------------------------------------------------
def test_console_mode_exports_to_the_given_stream_and_never_to_stdout() -> None:
"""T3: console mode hands MAF exactly one exporter, writing to the stream we chose.
``ConsoleSpanExporter``'s default sink is ``sys.stdout``; S6 measured that letting it take that
default destroys the pinned transcript. So the exporter is constructed explicitly against the
stream, and ``enable_console_exporters`` is pinned False so that the environment cannot add a
second, stdout-bound one behind it.
"""
from opentelemetry.sdk.trace.export import ConsoleSpanExporter
stream = io.StringIO()
spy = _ConfigureRecorder()
setup = tracing.configure_tracing(
env={tracing.TRACING_ENV: "console"}, stream=stream, configure=spy
)
assert setup.mode == tracing.MODE_CONSOLE
assert len(spy.calls) == 1
call = spy.calls[0]
assert call["enable_console_exporters"] is False
exporters = call["exporters"]
assert len(exporters) == 1
exporter = exporters[0]
assert isinstance(exporter, ConsoleSpanExporter)
assert exporter.out is stream
@pytest.mark.parametrize(
"endpoint_var",
[
"OTEL_EXPORTER_OTLP_ENDPOINT",
"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
],
)
def test_console_mode_refuses_when_the_environment_declares_a_network_exporter(
endpoint_var: str,
) -> None:
"""T4: ``console`` + any OTLP endpoint variable → ``TracingConfigError``, and MAF is not called.
This is the no-silent-egress test. ``_configure`` adds env-derived exporters unconditionally and
BEFORE ours, so the word "console" would have been a lie about where the spans went. Refusal
names the offending variable, because the operator has to be able to find it; it does NOT unset
it (validation, never repair).
"""
spy = _ConfigureRecorder()
env = {tracing.TRACING_ENV: "console", endpoint_var: "http://collector.example:4317"}
with pytest.raises(tracing.TracingConfigError) as excinfo:
tracing.configure_tracing(env=env, configure=spy)
assert endpoint_var in str(excinfo.value)
assert spy.calls == []
# ---------------------------------------------------------------------------------------------
# otlp mode — egress, and only when it was asked for by name
# ---------------------------------------------------------------------------------------------
def test_otlp_mode_without_a_declared_endpoint_is_refused() -> None:
"""T5: ``otlp`` with no endpoint variable → refusal, not a provider that exports nowhere.
``configure_otel_providers()`` with nothing to export to succeeds silently and installs
providers whose spans go into the void. An operator who typed ``PORTFOLIO_OTEL=otlp`` asked to
see the run somewhere; answering with a working-looking no-op is the failure mode this repo
calls "a gate that can only be green".
"""
spy = _ConfigureRecorder()
with pytest.raises(tracing.TracingConfigError) as excinfo:
tracing.configure_tracing(env={tracing.TRACING_ENV: "otlp"}, configure=spy)
assert "OTEL_EXPORTER_OTLP_ENDPOINT" in str(excinfo.value)
assert spy.calls == []
def test_otlp_mode_leaves_the_exporter_to_maf_and_still_pins_the_console_flag() -> None:
"""T6: ``otlp`` + a declared endpoint → MAF builds the network exporter from the environment.
We pass no exporters of our own: MAF's step (1) already reads the standard variables, and a
second, hand-rolled OTLP exporter would be the duplicate free to drift from the spec. What we
still pin is ``enable_console_exporters=False`` — otherwise an operator running OTLP with
``ENABLE_CONSOLE_EXPORTERS`` exported also gets a stdout dump.
"""
spy = _ConfigureRecorder()
env = {
tracing.TRACING_ENV: "otlp",
"OTEL_EXPORTER_OTLP_ENDPOINT": "http://collector.example:4317",
}
setup = tracing.configure_tracing(env=env, configure=spy)
assert setup.mode == tracing.MODE_OTLP
assert setup.endpoints == (("OTEL_EXPORTER_OTLP_ENDPOINT", "http://collector.example:4317"),)
assert len(spy.calls) == 1
assert spy.calls[0]["enable_console_exporters"] is False
assert spy.calls[0]["exporters"] is None
def test_an_unknown_mode_is_refused_by_name() -> None:
"""T7: a typo is a refusal that lists the closed set, never a silent fallback to off.
Falling back to off would mean an operator who asked for tracing and mistyped gets a run with
no trace and no complaint — the black box the seam exists to remove, arrived at by accident.
"""
spy = _ConfigureRecorder()
with pytest.raises(tracing.TracingConfigError) as excinfo:
tracing.configure_tracing(env={tracing.TRACING_ENV: "jaeger"}, configure=spy)
message = str(excinfo.value)
assert "jaeger" in message
assert tracing.MODE_CONSOLE in message and tracing.MODE_OTLP in message
assert spy.calls == []
# ---------------------------------------------------------------------------------------------
# The announcement — one renderer, and omission rather than an empty row
# ---------------------------------------------------------------------------------------------
def test_notice_is_omitted_when_tracing_is_off() -> None:
"""T8a: no tracing → no line at all (``mandate.announce``'s rule).
The same rule ``cost_baseline_notice`` and ``skipped_links_notice`` follow, and it is
load-bearing beyond style here: the pinned demo stderr is four lines, and a "tracing: off" row
would have made it five.
"""
assert tracing.tracing_notice(tracing.TracingSetup(mode=None, endpoints=())) is None
def test_notice_names_the_sink_for_console_and_the_endpoint_for_otlp() -> None:
"""T8b: the announcement names where spans go, BEFORE the first one is emitted.
Same discipline as the MCP announcement (``mcp_tools``): every destination is named up front,
so there is no undeclared egress. The OTLP line names the *variable* as well as the value,
because that is what the operator edits.
"""
console = tracing.tracing_notice(tracing.TracingSetup(mode=tracing.MODE_CONSOLE, endpoints=()))
assert console is not None and "stderr" in console
otlp = tracing.tracing_notice(
tracing.TracingSetup(
mode=tracing.MODE_OTLP,
endpoints=(("OTEL_EXPORTER_OTLP_ENDPOINT", "http://collector.example:4317"),),
)
)
assert otlp is not None
assert "http://collector.example:4317" in otlp
assert "OTEL_EXPORTER_OTLP_ENDPOINT" in otlp
# ---------------------------------------------------------------------------------------------
# The real arm: the actual demo, in a subprocess, with nothing patched
# ---------------------------------------------------------------------------------------------
_SPAN_MARKER = re.compile(r'"name": "workflow\.run"')
def _run_demo(extra_env: dict[str, str]) -> subprocess.CompletedProcess[str]:
"""Run the real demo the same way the golden-transcript module does.
The ``-m`` form (no PATH assumption) and a pinned ``PYTHONIOENCODING`` — without the latter the
stdout comparison would be measuring the operator's locale rather than the program.
"""
proc = subprocess.run(
[sys.executable, "-m", "portfolio_optimiser.simulation"],
capture_output=True,
text=True,
encoding="utf-8",
env={**os.environ, "PYTHONIOENCODING": "utf-8", **extra_env},
check=False,
)
assert proc.returncode == 0, proc.stderr
return proc
@pytest.fixture(scope="module")
def traced_demo() -> subprocess.CompletedProcess[str]:
return _run_demo({tracing.TRACING_ENV: "console"})
@pytest.fixture(scope="module")
def untraced_demo() -> subprocess.CompletedProcess[str]:
return _run_demo({tracing.TRACING_ENV: ""})
def test_traced_demo_emits_workflow_run_spans_on_stderr(
traced_demo: subprocess.CompletedProcess[str],
) -> None:
"""T9a: the plan's U14 criterion, half one — ``PORTFOLIO_OTEL=console`` produces real spans.
``workflow.run`` is asserted specifically rather than "some span": it is the span that says a
workflow was executed, which is the thing a reader of the trace is looking for. S6 measured two
of them for this demo; the assertion is ``>= 1`` because the count is a property of the demo's
scripted script, not of the seam.
"""
assert len(_SPAN_MARKER.findall(traced_demo.stderr)) >= 1
def test_traced_demo_leaves_stdout_byte_identical_to_the_golden_transcript(
traced_demo: subprocess.CompletedProcess[str],
) -> None:
"""T9b: the plan's U14 criterion, half two — turning tracing ON does not move one byte of stdout.
This is what makes the seam safe to ship: the operator can trace a live run on stage without
the transcript they rehearsed against changing under them. It is also the test that fails if
the exporter is ever allowed to take its stdout default.
"""
assert traced_demo.stdout == _GOLDEN_STDOUT.read_text(encoding="utf-8")
def test_untraced_demo_emits_no_spans_at_all(
untraced_demo: subprocess.CompletedProcess[str],
) -> None:
"""T9c (control): without the variable there is NO span — so T9a measured the variable.
Without this, "spans on stderr" would be consistent with a demo that emits them regardless, and
the seam would be unproven while looking proven.
"""
assert _SPAN_MARKER.findall(untraced_demo.stderr) == []
def test_traced_demo_announces_where_the_spans_go(
traced_demo: subprocess.CompletedProcess[str],
) -> None:
"""T9d: the announcement reaches the operator, not just the renderer's return value.
The wiring half of T8: a renderer nobody prints is the silent-success shape this repo keeps
finding. Only a subprocess run can catch a detached ``print`` in ``main()`` (the P4 precedent).
"""
assert "PORTFOLIO_OTEL" in traced_demo.stderr
# ---------------------------------------------------------------------------------------------
# The other two process entries — the CLI and the hosted service
# ---------------------------------------------------------------------------------------------
#
# The demo is a scripted proof, not the product. Wiring the seam only there would leave the two
# entries an organisation actually runs — `portfolio-optimiser` on a terminal and `python main.py`
# in a container — untraceable, which is the state U14 exists to end. Both are exercised as
# SUBPROCESSES for the P4 reason: a detached call inside a `main()` is invisible to every in-process
# test, because no in-process test calls `main()`.
def _run_cli(argv: list[str], extra_env: dict[str, str]) -> subprocess.CompletedProcess[str]:
"""Drive the CLI's refusal path — the cheapest argv that reaches past ``parse_args``.
``--json`` without ``--report`` is an existing, documented rc-1 refusal. It is used here as a
carrier, not as the thing under test: it proves that the tracing seam is resolved BEFORE any
branch of the CLI can return, which is what "call once at startup" requires. No model is
contacted and no workflow is built, so the arm stays free.
"""
return subprocess.run(
[sys.executable, "-m", "portfolio_optimiser.run", *argv],
capture_output=True,
text=True,
encoding="utf-8",
env={**os.environ, "PYTHONIOENCODING": "utf-8", **extra_env},
check=False,
)
def test_cli_announces_tracing_before_it_refuses_anything() -> None:
"""T10: ``run.main`` resolves and announces the seam, ahead of every other branch.
RED if the call is detached from ``main()`` or moved below a ``return``.
"""
proc = _run_cli(["--json"], {tracing.TRACING_ENV: "console"})
assert proc.returncode == 1, proc.stderr
assert "PORTFOLIO_OTEL=console" in proc.stderr
assert "run refused: --json requires --report" in proc.stderr
def test_cli_says_nothing_about_tracing_when_it_is_off() -> None:
"""T10 control: the same argv without the variable prints the refusal and NOTHING else.
Without this, T10 could not distinguish an announcement caused by the variable from a banner
the CLI prints unconditionally — and an unconditional banner would have changed every existing
stderr expectation in the suite.
"""
proc = _run_cli(["--json"], {tracing.TRACING_ENV: ""})
assert proc.returncode == 1
assert "PORTFOLIO_OTEL" not in proc.stderr
def test_cli_refuses_an_unusable_tracing_request_as_a_run_refusal() -> None:
"""T11: a malformed ``PORTFOLIO_OTEL`` exits through the CLI's own rc-1 refusal surface.
Not a traceback: this repo's CLI contract is that a refusal is a printed line and rc 1, and a
configuration the operator can fix belongs there. The measured content matters too — the line
must name the variable, since the operator is looking for something they exported, not for a
flag they typed.
"""
proc = _run_cli(["--json"], {tracing.TRACING_ENV: "jaeger"})
assert proc.returncode == 1
assert "run refused" in proc.stderr
assert "PORTFOLIO_OTEL" in proc.stderr
assert proc.stderr.count("Traceback") == 0
def test_hosted_entrypoint_announces_tracing_on_stderr() -> None:
"""T12: ``python main.py`` — the ONE start command DEPLOY.md prints — carries the seam too.
The hosted flate is where "an enterprise must be able to see what the run did" is actually
cashed: a container's stderr is its log. Served, then stopped with SIGTERM exactly as the
existing entrypoint test does, so this measures the real process rather than an import.
"""
import socket
import time
import urllib.error
import urllib.request
with socket.socket() as probe:
probe.bind(("127.0.0.1", 0))
port = probe.getsockname()[1]
repo_root = Path(__file__).resolve().parents[1]
proc = subprocess.Popen(
[sys.executable, str(repo_root / "main.py")],
env={**os.environ, "PORT": str(port), tracing.TRACING_ENV: "console"},
cwd=repo_root,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
text=True,
encoding="utf-8",
)
try:
deadline = time.monotonic() + 60
up = False
while time.monotonic() < deadline:
try:
with urllib.request.urlopen(
f"http://127.0.0.1:{port}/readiness", timeout=5
) as resp:
if resp.status == 200:
up = True
break
except (urllib.error.URLError, OSError):
time.sleep(0.2)
assert up, "main.py never served /readiness"
import signal as _signal
proc.send_signal(_signal.SIGTERM)
stderr = proc.communicate(timeout=15)[1]
assert proc.returncode == 0
finally:
if proc.poll() is None:
proc.kill()
proc.wait()
assert "PORTFOLIO_OTEL=console" in stderr
def test_console_mode_survives_an_operator_who_exported_enable_console_exporters() -> None:
"""T13: ``ENABLE_CONSOLE_EXPORTERS=true`` in the environment does NOT reach stdout.
The behavioural half of ``enable_console_exporters=False``. Asserting only the keyword argument
(T3, T6) proves what we passed, never what it prevents — and what it prevents is MAF's step (3)
adding a second ``ConsoleSpanExporter()`` whose default sink is stdout, which is the exact shape
S6 measured as destroying the pinned transcript. Left as ``None``, the flag falls back to this
variable, so this run is the one that would break.
"""
proc = _run_demo({tracing.TRACING_ENV: "console", "ENABLE_CONSOLE_EXPORTERS": "true"})
assert proc.stdout == _GOLDEN_STDOUT.read_text(encoding="utf-8")
assert len(_SPAN_MARKER.findall(proc.stderr)) >= 1

35
uv.lock generated
View file

@ -1448,14 +1448,41 @@ wheels = [
[[package]] [[package]]
name = "opentelemetry-api" name = "opentelemetry-api"
version = "1.42.1" version = "1.44.0"
source = { registry = "https://pypi.org/simple" } source = { registry = "https://pypi.org/simple" }
dependencies = [ dependencies = [
{ name = "typing-extensions" }, { name = "typing-extensions" },
] ]
sdist = { url = "https://files.pythonhosted.org/packages/b4/1c/125e1c936c0873796771b7f04f6c93b9f1bf5d424cea90fda94a99f61da8/opentelemetry_api-1.42.1.tar.gz", hash = "sha256:56c63bea9f77b62856be8c47600474acad853b2924b99b1687c4cb6297166716", size = 72296, upload-time = "2026-05-21T16:32:49.335Z" } sdist = { url = "https://files.pythonhosted.org/packages/ee/8b/aa9e2d8b8dfa7c946f7dec5d1f8f6ba8eca062f43509a06bdb5ce93d26c0/opentelemetry_api-1.44.0.tar.gz", hash = "sha256:67647e5e9566edcf421166fdf022b3537f818635daa852b289e34604dc6fb33a", size = 72406, upload-time = "2026-07-16T15:25:32.678Z" }
wheels = [ wheels = [
{ url = "https://files.pythonhosted.org/packages/a3/ca/9520cc1f3dfbbd03ac5903bbf55833e257bc64b1cf30fa8b0d6df374d821/opentelemetry_api-1.42.1-py3-none-any.whl", hash = "sha256:51a69edacadbc03a8950ace1c4c21099cacc538820ac2c9e36277e78cebba714", size = 61311, upload-time = "2026-05-21T16:32:28.822Z" }, { url = "https://files.pythonhosted.org/packages/ca/6f/a04e900f465ff3221ccc395522503e2d10e79fa21f2723c8e177aae1e0d1/opentelemetry_api-1.44.0-py3-none-any.whl", hash = "sha256:94b98c893a91b88657eaac1e3ba89618cdb85be6918196705354f34728b2cdef", size = 60018, upload-time = "2026-07-16T15:25:11.657Z" },
]
[[package]]
name = "opentelemetry-sdk"
version = "1.44.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "opentelemetry-api" },
{ name = "opentelemetry-semantic-conventions" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/5d/77/a6592cbc7c8d9bcc9d6757a9df45e04a7c585e3e6e7a13456da522b21109/opentelemetry_sdk-1.44.0.tar.gz", hash = "sha256:cebe7f65dc12f26ead75c6064de12fd2a9052e5060c0272d402cfa203aae123b", size = 208624, upload-time = "2026-07-16T15:25:46.078Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e7/23/ff077e61886ee020a17ce9c8b6fa11c601c8d8345b09ea24f605445df62a/opentelemetry_sdk-1.44.0-py3-none-any.whl", hash = "sha256:df081c4c6bcfdb1211e3e86140376792643128a25f8d72d1d27675936e7e96ad", size = 137221, upload-time = "2026-07-16T15:25:29.534Z" },
]
[[package]]
name = "opentelemetry-semantic-conventions"
version = "0.65b0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "opentelemetry-api" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/8f/73/0cbdebcb4cf545fdd328da14f5137e37d0770c3f26185e478b0d15d94f50/opentelemetry_semantic_conventions-0.65b0.tar.gz", hash = "sha256:f9b2b81e9d5b64f11bc952075e7e9c7fb0aab075c7fd1c46d597f1b919852d60", size = 148774, upload-time = "2026-07-16T15:25:46.902Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a6/0e/49df70d9b81fb5cbae4bbf2a49d865b09bcbcbc4eb53f5851b1027738d78/opentelemetry_semantic_conventions-0.65b0-py3-none-any.whl", hash = "sha256:1cacde7b0ad306f84c5ef08c3dbe1bbaf20165bba6f8bff43b670e555a086bcb", size = 204645, upload-time = "2026-07-16T15:25:30.688Z" },
] ]
[[package]] [[package]]
@ -1500,6 +1527,7 @@ dependencies = [
{ name = "llm-ingestion-okf" }, { name = "llm-ingestion-okf" },
{ name = "mcp" }, { name = "mcp" },
{ name = "numpy" }, { name = "numpy" },
{ name = "opentelemetry-sdk" },
{ name = "pulp" }, { name = "pulp" },
{ name = "pydantic" }, { name = "pydantic" },
] ]
@ -1524,6 +1552,7 @@ requires-dist = [
{ name = "llm-ingestion-okf", git = "https://git.fromaitochitta.com/open/llm-ingestion-okf.git?rev=v0.3.2" }, { name = "llm-ingestion-okf", git = "https://git.fromaitochitta.com/open/llm-ingestion-okf.git?rev=v0.3.2" },
{ name = "mcp", specifier = ">=1.28.0" }, { name = "mcp", specifier = ">=1.28.0" },
{ name = "numpy", specifier = ">=2.0,<2.3" }, { name = "numpy", specifier = ">=2.0,<2.3" },
{ name = "opentelemetry-sdk", specifier = ">=1.42,<2" },
{ name = "pulp", specifier = ">=2.8" }, { name = "pulp", specifier = ">=2.8" },
{ name = "pydantic", specifier = ">=2.11,<3" }, { name = "pydantic", specifier = ">=2.11,<3" },
] ]