portfolio-optimiser/docs/extending.md

Extending the framework (extension points)

portfolio-optimiser is a generic core with explicit config seams (D4/D5, 90 %-prinsippet): you onboard a new project, a new data source, or a new model-map without editing the core src/portfolio_optimiser/*.py. The three guides below name the exact seam for each.

Honesty note (rent teknisk rammeverk). The bundled reference domain (data/reference_projects.json + data/docs/<id>/) is a set of SYNTHETIC, AI-authored fixtures — fictional construction-cost projects, dummy estimates, and placeholder verdict_input decisions. They are flagged in each file's _note. A production deployer replaces the data source with their own and supplies Layer-2 verdicts via real HITL (fageksperter), not static config. The static verdict_input field is a test-fixture convenience that stands in for the durable HITL verdict in the offline synthetic framework.

Legg til eget prosjekt

A new project is config + docs only — no code change (this is exercised by the SC1 test test_e_new_project_flows_through_via_config_only + the test_f_no_hardcoded_project_ids_in_src guard, which fails if any project id leaks into src/).

1. Append an entry to src/portfolio_optimiser/data/reference_projects.json with the full key set: id, name, description, currency, cost_items, docs_dir, and verdict_input ({"decision", "rationale"}).
   • docs_dir is a path relative to the package data/ root (e.g. "docs/MY-PROJECT"); the loader (reference_domain.load_reference_projects) resolves it to an absolute path.
   • verdict_input carries the (synthetic) Layer-2 decision/rationale; flag it in the file's _note as synthetic if it is not a real expert verdict.
2. Create the bundled docs folder src/portfolio_optimiser/data/docs/<id>/ with at least one text file whose content names the cost-saving measure/terms (so retrieve_chunks returns at least one citable chunk).
3. Run the project: run_portfolio(["MY-PROJECT"], "local", client_factory=...) (or include it in the default fan-out by passing no project_ids).

Legg til egen datakilde

The retriever (retrieval.py / datasource.py) reads a local docs folder per project, selected by the project's docs_dir in reference_projects.json. To point a project at your own data, change its docs_dir to your folder and drop your cost documentation there — the citation seam ({file, locator, snippet, score}) is identical on the in-process tool and MCP paths. The folder is boundary-checked (fail-closed) against path traversal, so keep documents inside the configured docs_dir. A real deployer swaps the bundled synthetic docs for their own source.

Legg til egen modell-map

Model choice is config, not code (B12): src/portfolio_optimiser/data/model_map.json maps profile -> role -> model/deployment (resolve_model(profile, role)). To use your own models:

• Edit the local block to your local model ids (Ollama/LM Studio), and/or
• Edit the azure block to your Foundry deployment names (the placeholders REPLACE-WITH-FOUNDRY-DEPLOYMENT are tenant-specific — replace them or supply via env).

The role keys (proposer, checker, default) let you assign a distinct model per debate role; default is the fallback when a role is unmapped.

Legg til en ingest-kilde (http-familien som worked example)

The ingest layer (ingest.py, spec shared/ingest-spec.md) is a second, distinct source seam from the retriever above: one JSON manifest per source coupling declares a source.type and a list of extractions; materialize(...) runs the connector for that type and writes an OKF bundle. The spec ships three source families — file/CSV (I2), sql (I4), and http (I6) — and http is the framework's worked extension-point example: it shows exactly how a third, network-transport family plugs into the same connector / materialization / gate contracts.

The http manifest contract (spec §4):

• source.base_url — the endpoint root; it must not embed credentials (userinfo is rejected at schema validation).
• source.credential_ref — the name of an environment variable holding the secret, resolved at run time (sent as Authorization: Bearer <secret>); the secret is never read from the manifest, never logged, never stamped in the generated frontmatter. null → no auth.
• each extraction's query is joined onto base_url and its response body is rendered verbatim inside a fenced code block (not a markdown table — a raw | or \ survives un-escaped); max_rows caps the response line count, fail-fast (never silent truncation).

Network is a per-run grant, never a manifest field (spec §8). materialize refuses an http source unless the caller passes allow_network=True:

# refused fail-fast — the manifest cannot grant itself network access:
materialize(manifest, bundle_dir, ingested_at="2026-07-04T12:00:00Z")
# opted in explicitly by the operator for this run:
materialize(manifest, bundle_dir, ingested_at="2026-07-04T12:00:00Z", allow_network=True)

This is the local-only default, no silent egress principle made mechanical: configuration is data that cannot escalate its own authority; only the runtime allow_network grant can. The transport itself is an injectable seam — materialize(..., http_get=<callable>) swaps the GET implementation (the default _urllib_get is the only socket path). The golden case (examples/ingest-golden-http/) and every test inject a canned get over committed fixture payloads, so the suite runs offline against a local mock — no live source, no credentials.

D7 sibling hook — MCP as an extension of this family

The spec (§4) documents an MCP-based connector as an extension of the http family, not a new client wired into the optimiser run path — that stays a Non-Goal here: the in-process FunctionTool seam is the default in the run path, and MCP is demonstrated (via build_mcp_server in datasource.py), not wired in.

Where the D7 sibling stands (målbilde §11 boundary). The Claude Agent SDK sibling (D7) built the file/CSV and SQL connectors — mirroring I3/I5 — with bit-identical golden extractions. HTTP and MCP are demonstrated on the MAF side only (MAF-only), against a local mock; the sibling ships no network connector and no live-source integration. On D7 the in-process server hook is create_sdk_mcp_server(name, version="1.0.0", tools=...) -> McpSdkServerConfig (package claude-agent-sdk) — verified 2026-07-04 against the official Claude Agent SDK Python docs — but that is a documented hook a deployer would reach for, not a shipped D7 connector; no D7 HTTP or MCP session is planned. A deployer who wants a network- or MCP-mediated source extends this family behind the same explicit, per-run network grant; nothing here contacts a live endpoint.

Bevisst ikke bygget (90 %-kuttlista)

Per the design philosophy (a ~90 % generic core with clear extension points — we do not chase the last 10 %), the following are deliberate cuts, not roadmap debt. Each is extension territory for a deployer, with the seam named:

• B10 — full verdict-conflict taxonomy. Chosen minimal semantics (documented in verdicts.py + README): the in-memory store is first-write-wins per verdict id; the disk layers (write_verdict, promote_verdict) are last-write-wins per file. The full taxonomy (rejection categories + a rule for conflicting expert verdicts) is deferred until real experts produce conflicting verdicts.
• B11 — expert notification. run_project(notify=...) is a stub seam (run.py): pass any callable; no delivery mechanism (e-mail/Teams/webhook) ships with the core.
• U12 — checkpointing / crash-survival of a run. A run either completes or is re-run; the async verdict inbox (step 7) is the resumable boundary, not intra-run state.
• U14 — OpenTelemetry / observability. Provenance stamping is the audit trail the core ships; OTEL wiring is a deployer concern.
• Concurrent fan-out. run_portfolio iterates projects sequentially by design (fresh per-project execution state; one threaded VerdictStore); parallel orchestration is left to the deployer and would need budget-cap coordination.