VianoBugAnalysis/docs/budget-manager-v1.md

Budget Manager v1

This milestone implements deterministic accounting for the initial hypotheses. It does not run investigation tools, invoke an LLM for evidence assessment, build diagnoses, or start an investigation loop. docs/architecture.md remains the authoritative specification, particularly sections 16, 18, 19, 25 and 31.

Configuration and deployment tuning

Copy config/budget.example.toml to the ignored config/budget.local.toml and load it with BudgetConfig.load(path). This is a separate config from the initial hypothesis allocations in config/analysis.example.toml. These are conservative starting values, not format constants or empirically calibrated scoring rules:

Setting	Default	Meaning
default_action_cost	10	Positive integer cost for an action without an override
action_costs	empty in Python	Per-action positive integer costs; example TOML includes future tool names
max_reward_bps	8000	Maximum refund in basis points (8000 = 80%); range 0–9999
new_evidence_score	100	Deterministic score for unseen, nonempty evidence; range 0–100
low_information_threshold	20	Information scores at or below this value count as low gain
max_low_information_steps	3	Consecutive low-gain settlements before abandonment

The example config assigns source.read_context a cost of 5, source.find_text and database.lookup 10, and research.lookup 20. These names describe accounting overrides only: they neither implement tools nor authorize arbitrary operations. The future Tool Router must independently validate every action.

The complete effective policy is frozen in the first budget event of an analysis revision. Changing the config cannot alter an existing policy, refill balances, undo expenses, or reopen terminal accounts. An existing snapshot or identical operation can be replayed with a different config; new operations reject with BUDGET_POLICY_CHANGED. Use the frozen config to continue an existing revision; new revisions can adopt the tuned config. Every event includes the effective policy, so historical accounting remains reproducible after deployments change.

Tune costs, refund ceiling, novelty score, threshold and consecutive-step limit using actual result quality and cost data. The initial hypothesis allocation remains separately configurable. No automatic online adaptation is implemented. The invariants below deliberately cannot be disabled by configuration.

Calculation and invariants

The unit is investigation_units, unrelated to LLM token counts, money or provider retry counts. The LLM can supply only a bounded semantic score and a reason through the future trusted controller. It cannot choose actual costs, refunds, remaining balances, or budget policy.

For each completed action:

information = min(deterministic_novelty, semantic_score)
reward = floor(cost * max_reward_bps * information / 1_000_000)
reward = min(reward, cost - 1)
remaining = initial - total_spent + total_rewarded

Both scores range from 0 to 100. Full configured reimbursement requires both scores to reach 100. A score is an ordinal accounting input, not a confidence percentage. For example, cost 10 and both scores 100 produce reward 8 under the default policy; a semantic score of 50 produces reward 4. Even the most generous allowed policy always has 0 <= reward < cost: each settled action consumes at least one whole unit. Thus a hypothesis cannot fund infinite investigation.

Novelty v1 hashes canonical JSON evidence (sorted object keys, preserved array order). New nonempty evidence gets new_evidence_score; empty or already seen evidence gets zero. Deduplication spans hypotheses and prior analysis revisions in the same case package. Fresh revision budgets do not make old evidence novel. Transport metadata and query IDs should be kept outside the evidence payload. This detects exact normalized content reuse, not semantic equivalence. Future EvidenceStore/scoring work may provide richer deterministic novelty signals; this milestone does not pretend to assess factual correctness or entailment.

The controller supplies JSON evidence and safe references to existing package sources. Source hashes, evidence, computed novelty, semantic assessment reason, cost and reward are all retained. Accounting checks references and integrity, not whether an assessment is scientifically justified. Tests simulate assessments; no live model or paid provider is used.

Durable controller API

from viano_bug_analysis.budget import BudgetConfig, BudgetManager

budget = BudgetManager(store, BudgetConfig.load("config/budget.local.toml"))
budget.initialize(case_id, package_id)
budget.reserve(case_id, package_id, "H1", "query-0001", "source.read_context")
# A future controller invokes the allowed read-only tool after reservation.
budget.record_result(
    case_id, package_id, "H1", "query-0001",
    {"finding": "Example stored observation"}, source_refs=[source_ref],
)
# In the next iteration the controller validates a bounded semantic assessment.
result = budget.settle(
    case_id, package_id, "H1", "query-0001",
    semantic_score=50, reason="Example assessment of this stored result",
)

• initialize verifies a completed current-revision hypothesis batch, registry, allocations, context and original successful LLM response. It creates equal balances from the controller-owned allocations; it cannot allocate more IDs.
• reserve durably debits the configured cost before external work. Unknown hypotheses, blank IDs, unaffordable costs and terminal accounts are rejected. Only one unsettled action per hypothesis is allowed. Different hypotheses can have their own pending actions; action IDs are scoped to revision/hypothesis.
• record_result persists evidence, source references and their hashes. It changes the action to awaiting_assessment, with no immediate reimbursement.
• settle accepts only an integer score 0–100 and a nonblank reason. The score is bound to the already stored action result, and the manager computes the reward.
• fail closes a reserved action without reimbursement. Failed or uncertain executions still cost budget and count as low information. A pending action survives restart and blocks another reservation; the manager never silently cancels it or retries external work. The controller decides whether the same external operation can safely resume or must be closed with fail.
• reject marks an account terminal without refunding or deleting its history; a pending action must first be settled or failed.
• snapshot validates/replays the ledger without changing budgets.

Identical repeated operations return the current package/state with reused=True. Changed inputs under the same operation/hypothesis/action identity are rejected. Repeated initialization cannot refill balances, including after index-write failure, service restart, transport under another filename or config changes. BudgetResult returns the package path, revision, event reference and current state. Its investigable property lists nonterminal accounts without pending actions; ready_for_evaluation is true only when all accounts are terminal and no action awaits completion. An empty investigable list alone is not sufficient while asynchronous actions are pending.

Status and lifecycle boundaries

Budget account status is separate from qualitative hypothesis confidence and the immutable hypothesis birth/creation registry. The manager owns new, investigating, budget_exhausted, abandoned_low_information_gain and the controller-requested rejected budget eligibility marker. It does not infer supported/plausible, diagnose causes, create replacements or restore slots. The birth document's budget_accounting = not_started describes generation time and remains immutable; artifacts.budget_ledger is authoritative for the subsequent accounting state.

The cheapest cost in the configured override/default table is the affordability floor. Immediately after initialization and after settlement, a balance below that floor becomes budget_exhausted. Pending actions are not marked exhausted before their possible reward is evaluated. If affordable but the consecutive low-gain limit is reached, the account becomes abandoned_low_information_gain. A higher information score resets that counter. Exhaustion takes precedence if both conditions occur. Request-specific insufficient funds reject that request; a cheaper permitted action can still be considered.

Mutations are allowed only in GENERATING_HYPOTHESES or INVESTIGATING_HYPOTHESES. Budget initialization does not start investigation or transition the case. An exhausted set exposes eligibility for final evaluation, but does not create a diagnosis or route the case to Human Review. Those decisions belong to the future controller/evaluator. Reopening retains old artifacts and starts accounting from the new revision's accepted allocations.

Persistence, validation and limits

Each operation appends one immutable event/snapshot:

analysis/budgets/0001/events/00000001.json  # policy + initial balances
analysis/budgets/0001/events/00000002.json  # reservation
analysis/budgets/0001/events/00000003.json  # result
analysis/budgets/0001/events/00000004.json  # assessment + computed reimbursement

The manifest's artifacts.budget_ledger points to the latest event. Each event has format/version, identities, original hypothesis/registry hashes, sequence, parent/hash, effective policy, request and fully derived accounting state. The existing CheckpointStore publishes event, checkpoint and pointer atomically, increases package_generation, preserves unknown files and checks the prepared generation. A per-case budget lock prevents duplicate dispatch; short existing case/worker locks protect publication. No lock waits on an external tool or LLM.

Replay verifies event schemas, checkpoint bindings, immutable origins, lineage, source hashes and every computed state. Pointer deletion, changed allocations, costs, rewards, balances or broken lineage are controlled errors, never a reset. Checks detect inconsistent artifacts; hashes are not digital signatures and do not authenticate a maliciously rewritten complete case history.

budget-policy-v1, budget-request-v1 and budget-event-v1 JSON schemas are bundled with the Python package. Byte/event limits in BudgetLimits are adjustable operational safeguards, independent of format v1: defaults are 2 MiB per event, 1 MiB per evidence payload and 10,000 events across retained budget revisions. Limits reject explicitly rather than truncating history or dropping expenses. Snapshots contain cumulative state, so long-lived cases may need larger limits; incremental replay optimizations are a later performance task.

The CaseAssembler excludes checkpoint-declared budget events from raw case input identity, preserving replay of earlier phases. Undeclared/unknown files remain preserved and part of raw input inventory as before.

For asynchronous external testing, see async-tests-v1.md. BLOCKED pauses the case, not accounting history: balances and pending reservations survive unchanged. Confirmed feedback resumes the same revision; only subsequent result assessment changes its budget. Human/automated tests have separately configurable explicit costs, with no idle-time charges.