Kjell Tore Guttormsen
da93a68ce7
The last ungated build session: the operator now drives the whole build from the command line, and the documents claim exactly what the code does (§1). run.py becomes the collecting entrance. Exactly one of --bundle (one project) or --portfolio (N projects from a schema-validated reference config, with --verdict-dir as the portfolio-level expert inbox) is required; both and neither are refused. --goals loads a goal contract and checks it against --ledger's realized sum BEFORE the first model call: the §8 caps bound spend, the goal bounds achievement, so a hard target the book already meets stops the run at exit 4 without constructing a client. A soft target reached is a flag and the run continues; an absent ledger is an empty book, so the goal is still evaluated, never skipped. The one declared goal also drives --value-report's goal progress — one contract, never two figures that can disagree. The portfolio path persists nothing (K3 returns typed results; the outbox names pairs by run_id, which a portfolio pass has none of). Rather than accept --out/--outbox/--run-id/--value-report/--inbox/--live-dry-run there and silently ignore them, the entrance refuses them and says why. run_portfolio is imported lazily — portfolio.py imports this module, so a module-level import is circular. Three seams, each detach-proven RED: - unwire the goal check → the run proceeds and spends → red - unwire the portfolio branch → the configured projects never run → red - document a flag no CLI offers → the README honesty grep goes red That last one is the doc-sync made load-bearing: the test reads README.md, collects every --flag it documents (excluding third-party dev-tooling lines) and asserts each exists in the --help of a CLI the README names. The drift it exists to close was real — README claimed 562 tests, CHANGELOG claimed 265, actual 597. Docs synced to the code: README gains an operator-CLI section and honest goal/ portfolio descriptions, CHANGELOG is rewritten to what actually shipped, and docs/oppskrift-kunnskapsbase.md delivers D-H point 1 — the documented team process for building a knowledge base, with the honest 1–2 week expectation and every factory-dependent step (verdict translation, demo path) marked NOT BUILT. 597 passed · ruff clean · mypy strict clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MQu2xxwedckjU56byu1aUG
163 lines
9.2 KiB
Markdown
163 lines
9.2 KiB
Markdown
# Recipe: building a knowledge base
|
||
|
||
*(Norwegian title kept from the programme plan — «oppskrift» = recipe. Contents in English,
|
||
like the rest of this repo's documentation.)*
|
||
|
||
This is the documented **team process** for building the OKF bundles this framework reads.
|
||
It is a recipe, not a wizard: there is deliberately no onboarding interview and no guided
|
||
verdict command. Both were evaluated and rejected — a setup this consequential is done by
|
||
people talking to each other, and a wizard would hide exactly the decisions that need
|
||
deciding.
|
||
|
||
**Honest expectation: 1–2 weeks of dedicated work** for a knowledge base worth running
|
||
against. The quality of that investment decides the quality of everything downstream. A
|
||
thin bundle produces confident nonsense; the deterministic validator will catch arithmetic,
|
||
but it cannot catch a domain the base never described.
|
||
|
||
**Honesty rule (method spec §1):** this document separates what this repo *does today* from
|
||
what the process *will* look like once the bundle factory exists. Sections marked
|
||
**NOT BUILT** describe a target arrangement, not a capability you can use.
|
||
|
||
## 1. The team
|
||
|
||
Setup is always done by a small team, never by one person and never by a tool alone:
|
||
|
||
| Role | Brings | Owns |
|
||
|---|---|---|
|
||
| Technical | the file formats, the ingest layer, the run entrance, the caps | the bundle's *shape* |
|
||
| Domain expert(s) | what is actually true in this domain, and what the numbers hide | the bundle's *content* |
|
||
|
||
The split matters because the two failure modes are different. A technically perfect bundle
|
||
with no domain substance runs cleanly and tells you nothing. A domain-rich pile of documents
|
||
that never navigates is invisible to the loop.
|
||
|
||
## 2. What a knowledge base is here
|
||
|
||
One **OKF bundle** per project: a directory of plain markdown, plus the IR projection the
|
||
deterministic validator consumes. From the shared micro example
|
||
([`shared/examples/bygg-energi-mikro/`](../shared/examples/bygg-energi-mikro/)):
|
||
|
||
```
|
||
bygg-energi-mikro/
|
||
index.md # REQUIRED entry point; links to everything else
|
||
bygg-kontor-nord.md # type: project — the asset and its baseline
|
||
tiltak-led-retrofit.md # type: hypothesis — the candidate measure + parameters
|
||
metode-ipmvp-a.md # type: methodology — how savings are measured
|
||
kilder-realiseringsgap.md # type: reference — sourced literature
|
||
verdict-led-fro.md # type: verdict — an expert judgement (see §5)
|
||
validator-input.json # the IR projection the validator consumes
|
||
```
|
||
|
||
Four rules the code actually enforces — verify against these, not against intent:
|
||
|
||
1. **`index.md` is the entry point.** A bundle without one fails fast (`navigate_bundle`).
|
||
2. **The index's cross-links define the bundle.** Navigation follows the links in `index.md`;
|
||
a file nobody links to is a file the model never sees. Links containing a path separator
|
||
are skipped (fail-closed boundary rule), so keep concept files flat and beside the index.
|
||
3. **Every non-index file needs a `type` in its frontmatter.** `project`, `hypothesis`,
|
||
`methodology`, `reference`, `verdict` are the types in use.
|
||
4. **`type: verdict` files are excluded from the read-context.** A verdict reaches the next
|
||
hypothesis only through the experience fold, never by leaking through context. This is a
|
||
seam, not a preference — see the load-bearing tests.
|
||
|
||
`validator-input.json` is separate and **required by the run entrance**: it is the candidate
|
||
measure projected into the cost-IR (`project_id`, `measure`, `affected_items` with quantity
|
||
and unit cost, `claimed_saving_nok`). Writing it is technical work, and it is where the
|
||
domain expert's parameters become numbers the validator can refuse.
|
||
|
||
## 3. The process, week by week
|
||
|
||
**Day 1–2 — scope and one skeleton.** Pick *one* project and *one* candidate measure. Build
|
||
the bundle above end to end, however thin. Run it:
|
||
|
||
```bash
|
||
uv run python -m portfolio_optimiser_claude.preflight
|
||
uv run python -m portfolio_optimiser_claude.costsim --projects 1
|
||
uv run python -m portfolio_optimiser_claude.run --bundle <dir> --live-dry-run \
|
||
--outbox <dir> --run-id skeleton-1
|
||
```
|
||
|
||
The drill builds everything a real run builds and stops before the first model call, so the
|
||
rig is proven before anyone spends anything. Do not proceed to content until this is clean.
|
||
|
||
**Day 3–7 — content, expert-led.** The domain experts fill in what the skeleton only named:
|
||
the baseline the asset actually has, the method by which a saving is measured, and — the part
|
||
that matters most — the *sourced* reference material describing where modelled numbers and
|
||
realized numbers diverge. In the micro example this is `kilder-realiseringsgap.md`, and it is
|
||
the reason the learning loop has anything to learn.
|
||
|
||
**Day 8–10 — seed the judgement layer.** Add at least one `type: verdict` file encoding what
|
||
the experts know that the model does not: the correction they would apply and *why*. Without
|
||
this the first runs have no experience to fold, and the loop is a validator with extra steps.
|
||
|
||
**Ongoing — sufficiency review.** Ask the base directly: *is this comprehensive enough to be
|
||
useful, and what is missing?* The answer becomes the domain experts' work list. This review
|
||
is human-run today; an automated evaluator is part of the factory (**NOT BUILT**, §7).
|
||
|
||
## 4. The domain expert's interface
|
||
|
||
The expert is asked exactly three things: **what** to contribute, **how**, and **where**.
|
||
They deliver files in their own formats — notes, spreadsheets, reports, memos — and are never
|
||
asked to fill in a schema or hand-write JSON.
|
||
|
||
**Today:** a technical team member transcribes those contributions into bundle files. This is
|
||
the interim arrangement, and it is real work — budget for it in the 1–2 weeks. The AI
|
||
translator that would do it is **NOT BUILT** (§7).
|
||
|
||
**Tabular sources are the exception, and they are automated.** If the contribution is a CSV
|
||
catalogue or a local SQL database, the ingest layer materializes an OKF bundle from it
|
||
deterministically, with provenance frontmatter and a `generated: true` honesty marker. That
|
||
path makes zero model calls and touches no network. See
|
||
[`docs/extending.md`](extending.md).
|
||
|
||
## 5. Verdicts — the loop's fuel
|
||
|
||
After a run, an expert judges the proposal. The verdict is a plain-JSON file dropped into an
|
||
inbox folder; a later run merges it before the fold. The shape (§4.2) is deliberately small:
|
||
an `id`, a `decision`, a `rationale`, and the proposal features the verdict is about.
|
||
|
||
**Today:** the same transcription applies — the expert judges in their own words, a technical
|
||
team member writes the verdict file. The decision is the human's in both arrangements; only
|
||
the typing changes.
|
||
|
||
**NOT BUILT — the target arrangement.** The expert delivers a free-format file and the
|
||
factory AI-translates it into the strict verdict format, because at volume manual translation
|
||
is unrealistic. Approval in practice then becomes **spot checks** in the generated bundle,
|
||
which only works with one guard in place: *the structured verdict must always point back at
|
||
the expert's original file*, so a spot check can compare what the expert wrote against what
|
||
the system understood. **The verdict is the human's judgement; the AI is a translator.** Any
|
||
implementation of this that drops the provenance pointer breaks the arrangement.
|
||
|
||
## 6. Reading and editing bundles
|
||
|
||
Bundles are plain markdown with frontmatter, so **Obsidian or VS Code is the reading solution**
|
||
— Obsidian in particular follows the cross-links the way the framework navigates them, which
|
||
makes "what will the model actually see?" answerable by clicking. A dedicated reading view is
|
||
a later building block in the toolkit repo, not a prerequisite.
|
||
|
||
## 7. What is not built
|
||
|
||
Named here so the recipe cannot be mistaken for a description of shipped capability (§1):
|
||
|
||
- **The bundle factory** — the tooling that would translate free-format expert contributions
|
||
(and verdicts) into bundle files. It lives in a separate toolkit repo that **does not exist
|
||
yet**. Everything above marked NOT BUILT depends on it.
|
||
- **The demo path** — fresh clone → unzip a worked example into a bundle inbox → the factory
|
||
builds it → the whole loop runs. Factory-dependent, and honest only once the factory is
|
||
real: the point of that demo is that it shows the actual process with finished content.
|
||
- **A dedicated reading view** for bundles (see §6).
|
||
- **A dimension catalogue** — the formalized scoping of measure types and cost codes. Today
|
||
the routing table treats a measure as a config string; the catalogue is gated on a pending
|
||
amendment to the shared spec.
|
||
|
||
## 8. Checklist before the first real run
|
||
|
||
- [ ] `index.md` exists and links every concept file that should be visible.
|
||
- [ ] Every non-index file has a `type` in its frontmatter.
|
||
- [ ] `validator-input.json` matches the hypothesis file's parameters.
|
||
- [ ] At least one `type: verdict` file encodes real expert correction.
|
||
- [ ] Reference material is sourced — a claim without a source is a claim the experts cannot
|
||
defend when the model repeats it back.
|
||
- [ ] `preflight` is clean and `costsim` gives a cost you have accepted.
|
||
- [ ] `--live-dry-run` completes and its captured artifacts look right.
|
||
- [ ] Stop criteria and a budget cap are set — the framework refuses to start without them.
|