VianoBugAnalysis/docs/investigation-loop-v1.md

# LLM investigation loop v1

The Investigator connects the existing Hypothesis Builder, LLMGateway, snapshot
Tool Router, EvidenceStore, BudgetManager and asynchronous test coordinator.
`docs/architecture.md` remains unchanged and authoritative. This phase performs
investigation, not final diagnosis or Change Request generation. It never writes
project source, SQL, patches, branches, builds or lab/client instructions for
direct execution.

## API and dispatch

```python
from viano_bug_analysis.analysis import Investigator, InvestigationConfig

loop = Investigator(
    store, gateway,
    InvestigationConfig.load("config/investigation.local.toml"),
    router=router,  # Supply the existing Vcore schema adapter where needed.
)
result = await loop.run(case_id, package_id)
```

`step` processes at most one model decision and its confirmed local execution.
`run` repeats eligible steps up to `max_steps_per_dispatch`, then returns
`YIELDED`. Callers can explicitly bound a dispatch through `max_steps`.
`PROGRESSED`, `REJECTED`, `YIELDED`, `RETRY_PENDING`, `DEFERRED`, `BLOCKED` and
`COMPLETED` describe scheduling outcomes. Unsafe input, stale state, exceeded
operational limits or changed policies raise a controlled `InvestigationError`;
duplicate concurrent dispatch raises the existing nonblocking `LockBusy`.
The server worker/scheduler remains a separate integration: no continuous daemon,
sleeping retry loop or UI is introduced.

Input must have an accepted current-revision interpretation and hypothesis batch.
The Investigator verifies the existing budget origins and canonical case freshness;
it does not silently build missing earlier phases or reopen changed raw inputs.
Existing pending actions without a controller-owned plan/last evidence binding
are rejected explicitly. Existing initialized budgets are reused, not refilled.
When no budget config is explicitly supplied, the stored effective budget policy
is adopted automatically for subsequent accounting. An explicit changed config
still follows BudgetManager's policy-change rejection rules.

## Deterministic cycle

1. Give each hypothesis one initial turn, then prefer higher confidence with
   numeric-ID ties. A reviewed early completion request may defer alternatives;
   see [Early diagnosis review](early-diagnosis-v1.md).
2. Build a bounded context from canonical case, accepted interpretation, current
   hypothesis states, questions, notes, budget policy/balances, current-revision
   stored evidence and the previous result. First iteration has no prior result.
3. Persist that context and its hash before the model call. `investigation_v1`
   is retained for historical requests. New prompts extend the output contract
   with reviewable early diagnosis and assessment-only deferral.
4. Execute one eligible LLM round through the existing persisted failover/retry
   gateway. No global worker lock spans the external call.
5. Validate the whole reply: schema, references, confidence, question updates,
   assessment binding, tool scope/arguments and projected affordability/status.
6. Persist the accepted decision as an execution plan before applying effects.
7. Settle the previous evidence's bounded semantic assessment, if present; then
   reserve the next action and perform exactly one allowed query or test handoff.
8. Persist raw reply and normalized evidence, then account its deterministic
   novelty through `record_evidence`. Semantic assessment occurs in a later
   model iteration, not on the result's publication step.
9. Publish the updated hypothesis state and scheduling position.

Validation simulates settlement and reservation with the same deterministic budget
rules used by execution. A valid assessment with an invalid next action is wholly
rejected: it receives neither a refund nor partial question/confidence updates.
Local preflight reads bounded snapshots but does not dispatch query handlers or
perform external work. Runtime failures after a valid plan are recoverable from
that plan; they are distinct from technically invalid model replies.

## Model/controller contract

Each reply has exactly `assessment`, `update` and `next`:

- `assessment`: null on first iteration; otherwise integer semantic score 0–100,
  concrete reason and valid evidence IDs including the previous result.
- `update`: qualitative confidence (`unassessed`, `weak`, `plausible`, `supported`,
  `strongly_supported`, `weakened`), known evidence IDs, existing open questions
  to resolve, new nonduplicate questions and an audit note.
- `next`: one existing tool operation/arguments, an external human/automated test
  descriptor, or `finish`/`reject` with a reason.

Evidence citations must exist in the supplied current-revision context. Supported
confidence requires nonempty stored findings. Resolved questions require evidence
and cannot be unknown or already resolved. A strong-support update combined with
rejection is inconsistent and rejected. These checks establish technical grounding,
not scientific proof that the model's causal assessment is correct.

The controller assigns action/request IDs, deadlines, checkpoint IDs, question
IDs, costs, refunds and balances. They are not model-controlled output fields.
The first loop controller works over the accepted initial hypotheses. Dynamic
`hypothesis.create`, cross-hypothesis mutation and arbitrary controller commands
are not exposed; creation limits and immutable birth records remain unchanged.
Adding replacement hypotheses requires a dedicated creation/budget/registry
contract rather than granting a model extra allocations through this schema.

All source, database, human feedback and prior model notes are untrusted data.
Symbol occurrences and partial/empty queries retain their existing scope warnings.
No conclusions about current live server behavior are inferred from old snapshots.
The durable context includes current-revision evidence. The opt-in project-research
prompt uses a compact view documented in [project-research-v1.md](project-research-v1.md).
If the configured byte bound is exceeded, dispatch rejects explicitly;
operators must align context, gateway request and artifact limits.

## Budget and stopping

Every investigation query/test uses a positive configured cost. Refunds remain
strictly below cost; deterministic novelty and semantic utility must both justify
them. Empty/duplicate findings earn no novelty reward, overlaps receive the
existing item-based cap, and low-information/exhausted accounts are terminal.
Model token usage and monetary metrics remain separate from investigation units.

The new additive BudgetManager `complete` operation marks budget eligibility as
`investigation_finished`, preserves the balance and does not label a supported
hypothesis as factually rejected. Pending actions must first be assessed/closed.
The original hypothesis birth artifact is immutable; current qualitative states
live in the controller's versioned snapshots.

Logical defaults, frozen per analysis revision:

| Config | Default | Meaning |
| --- | --- | --- |
| `max_iterations_per_hypothesis` | 50 | Maximum research-producing model iterations; one final assessment/closure is permitted |
| `max_invalid_responses` | 3 | Consecutive rejected replies before blocking the case for technical correction |
| `max_questions_per_hypothesis` | 30 | Bound on accumulated question records |
| `test_deadline_seconds` | 21600 | Controller-owned external test deadline |
| `max_steps_per_dispatch` | 8 | Fair scheduler yield boundary |

An invalid reply changes only retry/audit bookkeeping. A following prompt includes
its safe rejection code. Repeated invalid replies cause `BLOCKED` with
`INVALID_LLM_RESPONSES`, requiring explicit operator resume after correction.
The hypothesis stays active; pending results, reservations and remaining credit
are preserved. No budget completion or zero-score settlement is fabricated.
The same initial hypotheses cannot be regenerated to reset their limits.

The iteration cap prevents free model-only research loops even when credit is
large. The final extra iteration can assess a pending result but may not start
another action. Questions, earlier notes and evidence remain in immutable history.

Once all accounts/dispositions are terminal, the case enters `BUILDING_DIAGNOSIS`
with final investigation state and deterministic `summary.md`. It does not create
a diagnosis, select a root cause or route a Coding-Agent task. The summary reports
balances, confidence, dispositions, evidence and outstanding questions. Its
countercheck marker records actions on alternatives; it is not semantic proof
that an alternative was meaningfully disproved. The later evaluator must still
consider uncertainty, evidence quality and missing counterchecks.

## Provider failures and external tests

Unreachable providers use existing ordered failover, configured delay and attempt
count (default 10). Pending retries return immediately with `retry_not_before`.
Exhaustion returns `BLOCKED`, preserving budgets, pending evidence and the frozen
context. Known credential, HTTP configuration, privacy/adapter and unexpected
provider failures also block the case rather than abandon its hypotheses.

After the external problem is fixed and the due blocked case is explicitly
resumed through the lifecycle, a caller can supply a stable `retry_id` to `step`
or `run`. This opens a new transport operation for an exhausted/configuration-
failed call; it does not reset investigation iterations, credit or evidence.
It cannot override an ordinary provider backoff or reset model-invalid counters.
Saved successful execution plans are completed without another model call, even
when transport configuration changes after their publication.

Human/automated test descriptors require explicitly configured action costs.
The controller freezes a deadline, reserves credit, stores the exact request and
delegates to `AsyncTestCoordinator`. The case returns
`BLOCKED/AWAITING_TEST_FEEDBACK` without parking a worker. No time-based charges
are applied. Authenticated, confirmed feedback resumes the same revision; the
loop imports it as evidence and later evaluates it. Web delivery/authentication,
bot steering, claims/cancellation and deadlines' notifications remain external
future adapters. See [async-tests-v1.md](async-tests-v1.md).

## Persistence and recovery

Context and state files are appended under `analysis/investigation/NNNN/`.
`manifest.artifacts.investigation_state` and `investigation_summary` are pointers.
Each state checkpoint has monotonically increasing sequence, parent ref and hash
binding; context hashes, identity and selected hypothesis are checked on replay.
Accepted plans are additionally bound to the original successful LLM call,
operation, saved request, prompt/schema/context and exact output document.

Reservation, result publication, evidence accounting, assessment and final state
publication are separately idempotent. After a crash, an accepted plan resumes
those steps; existing evidence is reused, not queried again or assigned new IDs.
Repeated completion repairs/validates stored output without new provider calls.
Pointer deletion or changed policy cannot silently restart the investigation.
CaseAssembler excludes only checkpoint-declared investigation artifacts, so
expected controller work does not make the accepted case stale.

`InvestigationLimits` are independent operational defaults: 1 MiB context,
4 MiB JSON artifact and 10,000 state checkpoints. They are adjustable safety
bounds, not permanent format limits. Compiled static schema validators are reused.
CheckpointStore caches validated generations only behind a newly computed full
archive-byte hash and unchanged limits. PackageReader additionally takes a bounded
immutable ZIP snapshot and reuses detached validation metadata for identical
snapshot hashes/limits, with a bounded entry/metadata cache. No timestamp-based
trust, extraction, skipped changed-file validation or raw archive retention in
the cache is introduced. Static bundled schema changes require process restart.

Tests exercise round-robin investigations, real read-only queries, semantic
assessment/novelty, bounded failures, limits, reference validation, questions,
provider retries/config errors, actual local HTTP transport, portable resume,
crashes after evidence publication, external test confirmation and byte-preserving
output. The BUG-20 integration uses actual wowdiag/wowsrc data and simulated model
decisions; it tests orchestration, not real model reasoning or discovery of the
known fix. No paid/cloud model is called by these tests.

Question resolution is idempotent: reconfirming an already resolved question
does not invalidate an otherwise valid decision and needs no new evidence.
Resolving any still-open question requires evidence references. Unknown question
IDs invalidate the entire decision before assessments, tools or budget changes
are applied. Mixing old confirmations with new resolutions follows the same
rules; existing question history is preserved.