Add independently reviewed early investigation completion

This commit is contained in:
Nico committed 2026-10-15 15:58:40 +02:00
1 parent c44ccd5ce3
commit 5082354bc3
23 files changed
+779 -28

No files matched your search

+64
View File
@@ -0,0 +1,64 @@
# Early diagnosis review v1
An investigation may request independent final evaluation before all hypothesis
accounts become terminal. This is not acceptance of a cause or a coding task.
The service performs no patch, SQL write or client automation.
## Request and prerequisites
New investigation prompts (`investigation_v2`, `investigation_project_v3`) offer
`next.kind = request_diagnosis`, with `reason` and `countercheck_refs`.
The selected hypothesis must be `strongly_supported`, cite concrete nonempty
investigation evidence of its own, and cite an actual result from an alternative.
An alternative already rated supported/strongly_supported prevents a request.
The controller validates IDs and structure; whether the evidence establishes
causality and whether a negative result has sufficient coverage remain explicit
responsibilities of the independent final evaluator.
A request first becomes `preparing`. Outstanding results must be assessed before
review: the controller schedules their hypotheses, and permits `defer` (assessment
without closing the hypothesis) or a justified `reject`. No fresh research action
is launched while preparing. `defer` is not available during ordinary research.
All action reservations must be settled before entering BUILDING_DIAGNOSIS.
No account is artificially exhausted and no alternative is implicitly refuted.
The requested state includes the leading hypothesis, evidence, countercheck,
reason and explicitly deferred alternatives. The existing confidence scheduler
and its initial turn for each hypothesis remain in use outside preparation.
## Independent decision and continuation
`diagnosis_early_v1` evaluates every hypothesis, all evidence and remaining
uncertainty independently. Acceptance requires the requested hypothesis,
`strongly_supported` confidence, direct evidence for it, an adequate investigated
alternative countercheck and no other evaluation marked supported. Ordinary
final diagnosis validation remains mandatory. An accepted review creates the
normal diagnosis artifact; the existing separate Change Request and output
validators still decide whether a coding task can be issued. Missing coordinates
or other implementation prerequisites must remain explicit missing information.
An inconclusive review, insufficient confidence, competing support or exhausted
invalid-response retries declines the request. The case returns to
INVESTIGATING_HYPOTHESES in the same analysis revision. The immutable review reason
is included in subsequent investigation context. No failure report or coding
task is issued by this rejection. Account balances, assessed results, questions
and evidence survive. The only reverse lifecycle transition is guarded by the
persisted pending request and its declined review. It is not a general rewind.
Each review has its own context, calls and error namespace under
`analysis/diagnosis/NNNN/early/NNNN/`. Historical requests and prompts stay intact;
completed acceptance still uses the canonical diagnosis completion checkpoint.
A process restart reuses the persisted model call rather than requesting it again.
Pending old investigation prompts retain their original protocol. Existing loops
adopt early-review support at their next decision boundary.
## Limits
`[investigation].max_early_reviews = 3` is a configurable operational default,
frozen with the revision policy. Older policies without it use three. Reaching
the limit disables further early requests, not ordinary investigation or normal
final evaluation. A declined request also needs new evidence content before
another request; a fresh ID for identical content does not suffice.
This prevents repeated unchanged final evaluations. It is independent of per
hypothesis research budgets, token limits and provider retry configuration.
Confidence levels are qualitative ratings, not calibrated probabilities.
+6 -4
View File
@@ -41,13 +41,15 @@ still follows BudgetManager's policy-change rejection rules.
## Deterministic cycle
1. Select an eligible hypothesis in numeric-ID round-robin order. No selected
cause terminates the case early; alternatives remain scheduled.
1. Give each hypothesis one initial turn, then prefer higher confidence with
numeric-ID ties. A reviewed early completion request may defer alternatives;
see [Early diagnosis review](early-diagnosis-v1.md).
2. Build a bounded context from canonical case, accepted interpretation, current
hypothesis states, questions, notes, budget policy/balances, current-revision
stored evidence and the previous result. First iteration has no prior result.
3. Persist that context and its hash before the model call. `investigation_v1`
prompt and `investigation-output-v1` schema define the model contract.
is retained for historical requests. New prompts extend the output contract
with reviewable early diagnosis and assessment-only deferral.
4. Execute one eligible LLM round through the existing persisted failover/retry
gateway. No global worker lock spans the external call.
5. Validate the whole reply: schema, references, confidence, question updates,
@@ -58,7 +60,7 @@ still follows BudgetManager's policy-change rejection rules.
8. Persist raw reply and normalized evidence, then account its deterministic
novelty through `record_evidence`. Semantic assessment occurs in a later
model iteration, not on the result's publication step.
9. Publish the updated hypothesis state and advance the round-robin cursor.
9. Publish the updated hypothesis state and scheduling position.
Validation simulates settlement and reservation with the same deterministic budget
rules used by execution. A valid assessment with an invalid next action is wholly