feat(fase5): add the long/async verdict file inbox (Steg 7 resumable feedback)
The short loop captured the expert verdict inline into an in-memory store, so a verdict arriving days/weeks later in a separate run could not influence any future hypothesis (målbilde §5 row 7). Steg 7 adds the long timescale: run_project gains an opt-in verdict_dir async inbox that load_verdicts_from_dir -> store.add MERGES into the store BEFORE the Step-1 ExpeL fold, so a verdict dropped after an earlier run reaches a separate, later run's hypothesis — fully resumable across runs separated in time. - verdicts.py: verdict_to_dict / verdict_from_dict (id read verbatim, never re-minted), write_verdict (public authoring primitive, NOT wired into run_project — system reads the folder, expert/persona writes it, §3 role split), tolerant load_verdicts_from_dir (missing/foreign/half-written files skipped, not raised — RAW layer per §10 R2), VerdictStore.from_dir. - run.py: verdict_dir kwarg; ingest-merge block after load_contracts (merge not replace keeps run_portfolio's cross-project threading; store.add idempotent on content-hash id; no change to the fold). CLI --bundle-dir/--verdict-dir thread the long loop to the console entry. No auto-persist of the run's own captured verdict (outbox/Steg 8). - Load-bearing PAIR (test_step7_async_loop_loadbearing.py): a verdict dropped after run A must reach run B's prompt (run B uses a FRESH store -> the transfer is the file loop, not in-memory carryover); empty-inbox control proves causality. Marker = a realization value absent from the bundle (not the seed's 0.82). Proven RED on ingest detach. Suite 138 -> 140 passed, 4 skipped; mypy + ruff check clean. Målbilde treated as frozen (no §3/§5/§7 edit). Step 8 (gated wiki promotion) remains. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MHR8iKxJRxDiDfNw8HZmWE
This commit is contained in:
parent
f7c81b45ec
commit
e2861cac0c
5 changed files with 297 additions and 6 deletions
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Generic, open framework on **Microsoft Agent Framework (MAF)** for finding cost-savings / efficiency proposals *within* each project of a portfolio of independent projects. Multiple agents collaborate to generate candidate proposals; a mandatory deterministic validator (solver + Monte Carlo) decides the numbers; domain experts review via human-in-the-loop, and the system learns from their verdicts.
|
||||
|
||||
> **Status:** Early development. The deterministic backbone is solid; the agentic learning loop is being wired one load-bearing seam at a time — **Step 1 (OKF-navigated context → hypothesis) is wired**, with the verdict layer kept out of the read-context (see below). Not yet end-to-end usable.
|
||||
> **Status:** Early development. The deterministic backbone is solid; the agentic learning loop is being wired one load-bearing seam at a time — **Steps 1, 3/4, 5 and 7 are wired** (OKF-navigated context, checker gate, informed refinement, and the async file feedback loop; see below). Step 8 (gated wiki promotion) remains. Not yet end-to-end usable.
|
||||
|
||||
> **Disclaimer — technical framework only.** This project is a *technical framework*. Organizations that deploy it are themselves responsible for ensuring a valid processing purpose and for any required assessments (DPIA, risk/ROS, security reviews, etc.). The framework ships technical affordances (local-only mode, provenance/audit logging, no silent data egress) to *enable* compliant use, but makes no compliance guarantees.
|
||||
|
||||
|
|
@ -20,7 +20,8 @@ The mandatory deterministic backbone (validator + budget meter + provenance) is
|
|||
- Two load-bearing tests fail when the seam is detached: the realization signal must reach the prompt via the fold (`tests/test_step1_expel_loadbearing.py`), and must never leak via context (`tests/test_okf.py::test_bundle_context_excludes_verdict_layer`).
|
||||
- **Steps 3/4 — checker gates the reasoning (wired).** Two falsifiers now act on the same candidate: the deterministic validator gates the **numbers** (blocking, as before), and the maker-checker's checker gates the **reasoning** (target picture §2/§6). The checker ends its turn with a `VERDICT: APPROVE` / `VERDICT: REJECT — <reason>` line; an explicit reject blocks an otherwise-validated proposal (`run_project` surfaces both debate participants via `output_from=agents` and overrides the outcome to a checker-sourced `Rejection`). The gate is opt-in-reject (fail-open on a missing marker), and `provenance.validator_decision` stays honest — it reflects the validator only, never the checker. Load-bearing: `tests/test_checker_gate_loadbearing.py` goes red on *either* detach (revert `output_from`, or drop the override).
|
||||
- **Step 5 — informed refinement (wired).** The proposer's bounded retry is no longer blind: the validator's *previous* `Rejection.reason` is fed into the next attempt's prompt (`generate.py` `generate_via_llm` → `_build_messages(prior_rejection=...)`), so the model corrects against the falsification instead of re-answering identically (target picture §5/§7). It carries only the most-recent reason (never an accumulated history, never the rejected proposal JSON) and runs under the *existing* `max_attempts` + token-meter cap — no new loop, so "improve until good enough" without a ceiling stays impossible. The only per-attempt falsifier here is the validator; seeding generation with the checker's critique is a run-level, separately-scoped concern and is deliberately not done here. Load-bearing: `tests/test_step5_refine_loadbearing.py` goes red when the reason is detached from the prompt (the outcome never flips *and* the verbatim-reason assertion fails), with a bounded control proving the loop still stops at `max_attempts`.
|
||||
- **Steps 7–8** (async file feedback, gated wiki promotion) — not yet wired.
|
||||
- **Step 7 — async file feedback loop (wired).** `run_project(..., verdict_dir=...)` adds the long feedback timescale (target picture §3/§7): an expert/persona drops a verdict file (plain JSON — the raw output layer, §10 R2) into an inbox folder *after* a run, and a separate, later run ingests it — merged into the store *before* the Step-1 fold — so a verdict that landed out of band reaches the next hypothesis. The loop is fully resumable across runs separated in time; no live session is assumed. The system *reads* the folder, the expert/persona *writes* it (§3 role split), so `run_project` deliberately does **not** persist its own captured verdict back (that is the outbox / Step-8 concern). Ingestion is tolerant (a missing folder, foreign or half-written files are skipped, not raised) and **merges** (never replaces), preserving `run_portfolio`'s cross-project store. Reachable from the CLI via `--bundle-dir --verdict-dir`. Load-bearing: `tests/test_step7_async_loop_loadbearing.py` — a verdict dropped after run A must reach run B's prompt (run B uses a *fresh* store, so the transfer is the file loop, not in-memory carryover), with an empty-inbox control proving causality.
|
||||
- **Step 8** (gated wiki promotion) — not yet wired.
|
||||
|
||||
## Docs
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue