Add durable configurable hypothesis budget accounting

docs/budget-manager-v1.md

@@ -0,0 +1,192 @@
+# Budget Manager v1
+
+This milestone implements deterministic accounting for the initial hypotheses.
+It does not run investigation tools, invoke an LLM for evidence assessment, build
+diagnoses, or start an investigation loop. `docs/architecture.md` remains the
+authoritative specification, particularly sections 16, 18, 19, 25 and 31.
+
+## Configuration and deployment tuning
+
+Copy `config/budget.example.toml` to the ignored `config/budget.local.toml` and
+load it with `BudgetConfig.load(path)`. This is a separate config from the initial
+hypothesis allocations in `config/analysis.example.toml`. These are conservative
+starting values, not format constants or empirically calibrated scoring rules:
+
+| Setting | Default | Meaning |
+| --- | --- | --- |
+| `default_action_cost` | 10 | Positive integer cost for an action without an override |
+| `action_costs` | empty in Python | Per-action positive integer costs; example TOML includes future tool names |
+| `max_reward_bps` | 8000 | Maximum refund in basis points (8000 = 80%); range 0–9999 |
+| `new_evidence_score` | 100 | Deterministic score for unseen, nonempty evidence; range 0–100 |
+| `low_information_threshold` | 20 | Information scores at or below this value count as low gain |
+| `max_low_information_steps` | 3 | Consecutive low-gain settlements before abandonment |
+
+The example config assigns `source.read_context` a cost of 5, `source.find_text`
+and `database.lookup` 10, and `research.lookup` 20. These names describe accounting
+overrides only: they neither implement tools nor authorize arbitrary operations.
+The future Tool Router must independently validate every action.
+
+The complete effective policy is frozen in the first budget event of an analysis
+revision. Changing the config cannot alter an existing policy, refill balances,
+undo expenses, or reopen terminal accounts. An existing snapshot or identical
+operation can be replayed with a different config; new operations reject with
+`BUDGET_POLICY_CHANGED`. Use the frozen config to continue an existing revision;
+new revisions can adopt the tuned config. Every event includes the effective
+policy, so historical accounting remains reproducible after deployments change.
+
+Tune costs, refund ceiling, novelty score, threshold and consecutive-step limit
+using actual result quality and cost data. The initial hypothesis allocation
+remains separately configurable. No automatic online adaptation is implemented.
+The invariants below deliberately cannot be disabled by configuration.
+
+## Calculation and invariants
+
+The unit is `investigation_units`, unrelated to LLM token counts, money or
+provider retry counts. The LLM can supply only a bounded semantic score and a
+reason through the future trusted controller. It cannot choose actual costs,
+refunds, remaining balances, or budget policy.
+
+For each completed action:
+
+```text
+information = min(deterministic_novelty, semantic_score)
+reward = floor(cost * max_reward_bps * information / 1_000_000)
+reward = min(reward, cost - 1)
+remaining = initial - total_spent + total_rewarded
+```
+
+Both scores range from 0 to 100. Full configured reimbursement requires both
+scores to reach 100. A score is an ordinal accounting input, not a confidence
+percentage. For example, cost 10 and both scores 100 produce reward 8 under the
+default policy; a semantic score of 50 produces reward 4. Even the most generous
+allowed policy always has `0 <= reward < cost`: each settled action consumes at
+least one whole unit. Thus a hypothesis cannot fund infinite investigation.
+
+Novelty v1 hashes canonical JSON evidence (sorted object keys, preserved array
+order). New nonempty evidence gets `new_evidence_score`; empty or already seen
+evidence gets zero. Deduplication spans hypotheses and prior analysis revisions
+in the same case package. Fresh revision budgets do not make old evidence novel.
+Transport metadata and query IDs should be kept outside the evidence payload.
+This detects exact normalized content reuse, not semantic equivalence. Future
+EvidenceStore/scoring work may provide richer deterministic novelty signals;
+this milestone does not pretend to assess factual correctness or entailment.
+
+The controller supplies JSON evidence and safe references to existing package
+sources. Source hashes, evidence, computed novelty, semantic assessment reason,
+cost and reward are all retained. Accounting checks references and integrity, not
+whether an assessment is scientifically justified. Tests simulate assessments;
+no live model or paid provider is used.
+
+## Durable controller API
+
+```python
+from viano_bug_analysis.budget import BudgetConfig, BudgetManager
+
+budget = BudgetManager(store, BudgetConfig.load("config/budget.local.toml"))
+budget.initialize(case_id, package_id)
+budget.reserve(case_id, package_id, "H1", "query-0001", "source.read_context")
+# A future controller invokes the allowed read-only tool after reservation.
+budget.record_result(
+    case_id, package_id, "H1", "query-0001",
+    {"finding": "Example stored observation"}, source_refs=[source_ref],
+)
+# In the next iteration the controller validates a bounded semantic assessment.
+result = budget.settle(
+    case_id, package_id, "H1", "query-0001",
+    semantic_score=50, reason="Example assessment of this stored result",
+)
+```
+
+- `initialize` verifies a completed current-revision hypothesis batch, registry,
+  allocations, context and original successful LLM response. It creates equal
+  balances from the controller-owned allocations; it cannot allocate more IDs.
+- `reserve` durably debits the configured cost before external work. Unknown
+  hypotheses, blank IDs, unaffordable costs and terminal accounts are rejected.
+  Only one unsettled action per hypothesis is allowed. Different hypotheses can
+  have their own pending actions; action IDs are scoped to revision/hypothesis.
+- `record_result` persists evidence, source references and their hashes. It
+  changes the action to `awaiting_assessment`, with no immediate reimbursement.
+- `settle` accepts only an integer score 0–100 and a nonblank reason. The score is
+  bound to the already stored action result, and the manager computes the reward.
+- `fail` closes a reserved action without reimbursement. Failed or uncertain
+  executions still cost budget and count as low information. A pending action
+  survives restart and blocks another reservation; the manager never silently
+  cancels it or retries external work. The controller decides whether the same
+  external operation can safely resume or must be closed with `fail`.
+- `reject` marks an account terminal without refunding or deleting its history;
+  a pending action must first be settled or failed.
+- `snapshot` validates/replays the ledger without changing budgets.
+
+Identical repeated operations return the current package/state with `reused=True`.
+Changed inputs under the same operation/hypothesis/action identity are rejected.
+Repeated initialization cannot refill balances, including after index-write
+failure, service restart, transport under another filename or config changes.
+`BudgetResult` returns the package path, revision, event reference and current
+state. Its `investigable` property lists nonterminal accounts without pending
+actions; `ready_for_evaluation` is true only when all accounts are terminal and
+no action awaits completion. An empty investigable list alone is not sufficient
+while asynchronous actions are pending.
+
+## Status and lifecycle boundaries
+
+Budget account status is separate from qualitative hypothesis confidence and
+the immutable hypothesis birth/creation registry. The manager owns `new`,
+`investigating`, `budget_exhausted`, `abandoned_low_information_gain` and the
+controller-requested `rejected` budget eligibility marker. It does not infer
+`supported`/`plausible`, diagnose causes, create replacements or restore slots.
+The birth document's `budget_accounting = not_started` describes generation
+time and remains immutable; `artifacts.budget_ledger` is authoritative for the
+subsequent accounting state.
+
+The cheapest cost in the configured override/default table is the affordability
+floor. Immediately after initialization and after settlement, a balance below
+that floor becomes `budget_exhausted`. Pending actions are not marked exhausted
+before their possible reward is evaluated. If affordable but the consecutive
+low-gain limit is reached, the account becomes `abandoned_low_information_gain`.
+A higher information score resets that counter. Exhaustion takes precedence if
+both conditions occur. Request-specific insufficient funds reject that request;
+a cheaper permitted action can still be considered.
+
+Mutations are allowed only in `GENERATING_HYPOTHESES` or
+`INVESTIGATING_HYPOTHESES`. Budget initialization does not start investigation
+or transition the case. An exhausted set exposes eligibility for final
+evaluation, but does not create a diagnosis or route the case to Human Review.
+Those decisions belong to the future controller/evaluator. Reopening retains
+old artifacts and starts accounting from the new revision's accepted allocations.
+
+## Persistence, validation and limits
+
+Each operation appends one immutable event/snapshot:
+
+```text
+analysis/budgets/0001/events/00000001.json  # policy + initial balances
+analysis/budgets/0001/events/00000002.json  # reservation
+analysis/budgets/0001/events/00000003.json  # result
+analysis/budgets/0001/events/00000004.json  # assessment + computed reimbursement
+```
+
+The manifest's `artifacts.budget_ledger` points to the latest event. Each event
+has format/version, identities, original hypothesis/registry hashes, sequence,
+parent/hash, effective policy, request and fully derived accounting state. The
+existing CheckpointStore publishes event, checkpoint and pointer atomically,
+increases `package_generation`, preserves unknown files and checks the prepared
+generation. A per-case budget lock prevents duplicate dispatch; short existing
+case/worker locks protect publication. No lock waits on an external tool or LLM.
+
+Replay verifies event schemas, checkpoint bindings, immutable origins, lineage,
+source hashes and every computed state. Pointer deletion, changed allocations,
+costs, rewards, balances or broken lineage are controlled errors, never a reset.
+Checks detect inconsistent artifacts; hashes are not digital signatures and do
+not authenticate a maliciously rewritten complete case history.
+
+`budget-policy-v1`, `budget-request-v1` and `budget-event-v1` JSON schemas are
+bundled with the Python package. Byte/event limits in `BudgetLimits` are adjustable
+operational safeguards, independent of format v1: defaults are 2 MiB per event,
+1 MiB per evidence payload and 10,000 events across retained budget revisions.
+Limits reject explicitly rather than truncating history or dropping expenses.
+Snapshots contain cumulative state, so long-lived cases may need larger limits;
+incremental replay optimizations are a later performance task.
+
+The CaseAssembler excludes checkpoint-declared budget events from raw case input
+identity, preserving replay of earlier phases. Undeclared/unknown files remain
+preserved and part of raw input inventory as before.