open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 1 Activity
portfolio-optimiser/docs/extending.md

13 KiB
Raw Blame History

Extending the framework (extension points)

portfolio-optimiser is a generic core with explicit config seams (D4/D5, 90 %-prinsippet): you onboard a new project, a new data source, or a new model-map without editing the core src/portfolio_optimiser/*.py. The three guides below name the exact seam for each.

Honesty note (rent teknisk rammeverk). The bundled reference domain (data/reference_projects.json + data/docs/<id>/) is a set of SYNTHETIC, AI-authored fixtures — fictional construction-cost projects, dummy estimates, and placeholder verdict_input decisions. They are flagged in each file's _note. A production deployer replaces the data source with their own and supplies Layer-2 verdicts via real HITL (fageksperter), not static config. The static verdict_input field is a test-fixture convenience that stands in for the durable HITL verdict in the offline synthetic framework.

Legg til eget prosjekt

A new project is config + docs only — no code change (this is exercised by the SC1 test test_e_new_project_flows_through_via_config_only + the test_f_no_hardcoded_project_ids_in_src guard, which fails if any project id leaks into src/).

  1. Append an entry to src/portfolio_optimiser/data/reference_projects.json with the full key set: id, name, description, currency, cost_items, docs_dir, and verdict_input ({"decision", "rationale"}).
    • docs_dir is a path relative to the package data/ root (e.g. "docs/MY-PROJECT"); the loader (reference_domain.load_reference_projects) resolves it to an absolute path.
    • verdict_input carries the (synthetic) Layer-2 decision/rationale; flag it in the file's _note as synthetic if it is not a real expert verdict.
  2. Create the bundled docs folder src/portfolio_optimiser/data/docs/<id>/ with at least one text file whose content names the cost-saving measure/terms (so retrieve_chunks returns at least one citable chunk).
  3. Run the project: run_portfolio(["MY-PROJECT"], "local", client_factory=...) (or include it in the default fan-out by passing no project_ids).

Legg til egen datakilde

The retriever (retrieval.py / datasource.py) reads a local docs folder per project, selected by the project's docs_dir in reference_projects.json. To point a project at your own data, change its docs_dir to your folder and drop your cost documentation there — the citation seam ({file, locator, snippet, score}) is identical on the in-process tool and MCP paths. The folder is boundary-checked (fail-closed) against path traversal, so keep documents inside the configured docs_dir. A real deployer swaps the bundled synthetic docs for their own source.

Legg til egen modell-map

Model choice is config, not code (B12): src/portfolio_optimiser/data/model_map.json maps profile -> role -> model/deployment (resolve_model(profile, role)). To use your own models:

  • Edit the local block to your local model ids (Ollama/LM Studio), and/or
  • Edit the azure block to your Foundry deployment names (the placeholders REPLACE-WITH-FOUNDRY-DEPLOYMENT are tenant-specific — replace them or supply via env).

The role keys (proposer, checker, default) let you assign a distinct model per debate role; default is the fallback when a role is unmapped.

Legg til en ingest-kilde (http-familien som worked example)

The ingest layer (ingest.py, spec shared/ingest-spec.md) is a second, distinct source seam from the retriever above: one JSON manifest per source coupling declares a source.type and a list of extractions; materialize(...) runs the connector for that type and writes an OKF bundle.

Where to change it (2026-07-20): ingest.py is a thin adapter — Door A is implemented by the shared llm-ingestion-okf library (git-pinned to v0.3.1), so connectors, materialization and index generation improve in ONE place across every consumer. shared/ingest-spec.md remains the normative spec — the library implements it, it does not replace it, and spec changes go via commons. The API below is unchanged; only the implementation moved. Note that Door A is ungated: it calls no security guard before writing to disk, so gating untrusted content is the caller's responsibility (see docs/plan/2026-07-16-llm-ingestion-guard-inclusion.md). The spec ships three source families — file/CSV (I2), sql (I4), and http (I6) — and http is the framework's worked extension-point example: it shows exactly how a third, network-transport family plugs into the same connector / materialization / gate contracts.

The http manifest contract (spec §4):

  • source.base_url — the endpoint root; it must not embed credentials (userinfo is rejected at schema validation).
  • source.credential_ref — the name of an environment variable holding the secret, resolved at run time (sent as Authorization: Bearer <secret>); the secret is never read from the manifest, never logged, never stamped in the generated frontmatter. null → no auth.
  • each extraction's query is joined onto base_url and its response body is rendered verbatim inside a fenced code block (not a markdown table — a raw | or \ survives un-escaped); max_rows caps the response line count, fail-fast (never silent truncation).

Network is a per-run grant, never a manifest field (spec §8). materialize refuses an http source unless the caller passes allow_network=True:

# refused fail-fast — the manifest cannot grant itself network access:
materialize(manifest, bundle_dir, ingested_at="2026-07-04T12:00:00Z")
# opted in explicitly by the operator for this run:
materialize(manifest, bundle_dir, ingested_at="2026-07-04T12:00:00Z", allow_network=True)

This is the local-only default, no silent egress principle made mechanical: configuration is data that cannot escalate its own authority; only the runtime allow_network grant can. The transport itself is an injectable seam — materialize(..., http_get=<callable>) swaps the GET implementation (the default _urllib_get is the only socket path). The golden case (examples/ingest-golden-http/) and every test inject a canned get over committed fixture payloads, so the suite runs offline against a local mock — no live source, no credentials.

D7 sibling hook — MCP as an extension of this family

The spec (§4) documents an MCP-based connector as an extension of the http family, not a new client wired into the optimiser run path — that stays a Non-Goal here: the in-process FunctionTool seam is the default in the run path, and MCP is demonstrated (via build_mcp_server in datasource.py), not wired in.

Where the D7 sibling stands (målbilde §11 boundary). The Claude Agent SDK sibling (D7) built the file/CSV and SQL connectors — mirroring I3/I5 — with bit-identical golden extractions. HTTP and MCP are demonstrated on the MAF side only (MAF-only), against a local mock; the sibling ships no network connector and no live-source integration. On D7 the in-process server hook is create_sdk_mcp_server(name, version="1.0.0", tools=...) -> McpSdkServerConfig (package claude-agent-sdk) — verified 2026-07-04 against the official Claude Agent SDK Python docs — but that is a documented hook a deployer would reach for, not a shipped D7 connector; no D7 HTTP or MCP session is planned. A deployer who wants a network- or MCP-mediated source extends this family behind the same explicit, per-run network grant; nothing here contacts a live endpoint.

Bytt ut henteren (Embedder / Retriever, S3.1)

How prior verdicts are ranked is a seam, not a hard-coded sort. semretrieval.py declares two protocols and the store delegates to them:

  • Retriever — rank(query, candidates, k) -> list[Verdict]. VerdictStore.retriever defaults to None, which means StructuralRetriever: the same weighted structural score and (-similarity, id) ordering the store used before the seam existed. Assign your own object with a rank method to replace ranking wholesale.
  • Embedder — __call__(features) -> np.ndarray. HybridRanker(embedder, similarity, weight) blends weight * cosine + (1 - weight) * structural; both terms live in [0, 1], so weight means what it reads as (SEMANTIC_WEIGHT_DEFAULT = 0.5).
store.retriever = HybridRanker(MyEmbedder(), similarity, weight=0.3)

Note that similarity is injected, not imported by semretrieval. That is deliberate: verdicts.py imports agent_framework, and injecting the score keeps the retrieval layer free of it (guarded by tests/test_semretrieval_loadbearing.py, which ranks in a subprocess and then asserts verdicts never entered sys.modules). Keep that property if you extend the module.

A real embeddings client is a config-only extension point — it is not built here. The shipped FakeEmbedder is a deterministic sha256 projection with no semantics; it exists so the seam is exercisable offline at zero cost, and the load-bearing proof is that removing the cosine term flips the ranking, not that the projection is meaningful. A deployer supplying a real client owns its network egress, cost, and the fact that it embeds proposal text — which is why the hybrid is opt-in (--semantic-retrieval) rather than the default. If you wire one in, note that semretrieval currently imports no network module at all, and a guard test asserts exactly that; a real client belongs behind the Embedder protocol in your module, not inside this one.

Optional persistence — an unwired authoring primitive. save_vector_store(dir, verdicts, embedder) / load_vector_store(dir) write and read a byte-deterministic vectors.npy + vectors.jsonl pair (sorted by verdict id, atomic replace). A missing store loads as None; a row/line mismatch raises rather than silently mis-ranking. *.npy is gitignored.

They have no caller in src/, and ranking never reads a persisted matrix: HybridRanker.rank re-invokes the embedder for every candidate on every call, so a persisted store would be bypassed even if one existed. They are offered to extenders, on the same footing as write_verdict, promote_verdict and build_mcp_server — public primitives the core deliberately does not wire into run_project. Wiring the cache in only becomes worth anything once the ranker is changed to consult it, which is a redesign rather than a hookup.

Known limitation: the empty-store branch hardcodes EMBED_DIM, so a third-party Embedder of a different dimension writes a shape-inconsistent empty store. Stated, not fixed.

Bevisst ikke bygget (90 %-kuttlista)

Per the design philosophy (a ~90 % generic core with clear extension points — we do not chase the last 10 %), the following are deliberate cuts, not roadmap debt. Each is extension territory for a deployer, with the seam named:

  • B10 — full verdict-conflict taxonomy. Chosen minimal semantics (documented in verdicts.py + README): the in-memory store is first-write-wins per verdict id; the disk layers (write_verdict, promote_verdict) are last-write-wins per file. The full taxonomy (rejection categories + a rule for conflicting expert verdicts) is deferred until real experts produce conflicting verdicts.
  • B11 — expert notification. run_project(notify=...) remains a plain-callable seam (run.py auto-wires no default notifier), but the core now ships the declared Notifier contract (notify.py, exported from the package top) with three implementations: ConsoleNotifier, FileNotifier (byte-deterministic JSONL), and WebhookNotifier — plus a fail-fast build_notifier(config, *, allow_egress=...) factory. The webhook is the ONLY egress point and is fail-closed behind an explicit per-run allow_egress=True opt-in (a code kwarg, never a config field — mirroring the ingest layer's allow_network). SSRF guards, HMAC signing, and auth headers remain deployer-owned extension points on the injectable WebhookPost transport seam. Webhook URLs must start with http:// or https:// — a scheme-less URL is rejected fail-fast at config construction.
  • U12 — checkpointing / crash-survival of a run. A run either completes or is re-run; the async verdict inbox (step 7) is the resumable boundary, not intra-run state.
  • U14 — OpenTelemetry / observability. Provenance stamping is the audit trail the core ships; OTEL wiring is a deployer concern.
  • Concurrent fan-out. run_portfolio iterates projects sequentially by design (fresh per-project execution state; one threaded VerdictStore); parallel orchestration is left to the deployer and would need budget-cap coordination.