Add independently reviewed early investigation completion
This commit is contained in:
1 parent
c44ccd5ce3
commit
5082354bc3
23 files changed
+779
-28
No files matched your search
@@ -0,0 +1,64 @@
|
||||
# Early diagnosis review v1
|
||||
|
||||
An investigation may request independent final evaluation before all hypothesis
|
||||
accounts become terminal. This is not acceptance of a cause or a coding task.
|
||||
The service performs no patch, SQL write or client automation.
|
||||
|
||||
## Request and prerequisites
|
||||
|
||||
New investigation prompts (`investigation_v2`, `investigation_project_v3`) offer
|
||||
`next.kind = request_diagnosis`, with `reason` and `countercheck_refs`.
|
||||
The selected hypothesis must be `strongly_supported`, cite concrete nonempty
|
||||
investigation evidence of its own, and cite an actual result from an alternative.
|
||||
An alternative already rated supported/strongly_supported prevents a request.
|
||||
The controller validates IDs and structure; whether the evidence establishes
|
||||
causality and whether a negative result has sufficient coverage remain explicit
|
||||
responsibilities of the independent final evaluator.
|
||||
|
||||
A request first becomes `preparing`. Outstanding results must be assessed before
|
||||
review: the controller schedules their hypotheses, and permits `defer` (assessment
|
||||
without closing the hypothesis) or a justified `reject`. No fresh research action
|
||||
is launched while preparing. `defer` is not available during ordinary research.
|
||||
All action reservations must be settled before entering BUILDING_DIAGNOSIS.
|
||||
No account is artificially exhausted and no alternative is implicitly refuted.
|
||||
|
||||
The requested state includes the leading hypothesis, evidence, countercheck,
|
||||
reason and explicitly deferred alternatives. The existing confidence scheduler
|
||||
and its initial turn for each hypothesis remain in use outside preparation.
|
||||
|
||||
## Independent decision and continuation
|
||||
|
||||
`diagnosis_early_v1` evaluates every hypothesis, all evidence and remaining
|
||||
uncertainty independently. Acceptance requires the requested hypothesis,
|
||||
`strongly_supported` confidence, direct evidence for it, an adequate investigated
|
||||
alternative countercheck and no other evaluation marked supported. Ordinary
|
||||
final diagnosis validation remains mandatory. An accepted review creates the
|
||||
normal diagnosis artifact; the existing separate Change Request and output
|
||||
validators still decide whether a coding task can be issued. Missing coordinates
|
||||
or other implementation prerequisites must remain explicit missing information.
|
||||
|
||||
An inconclusive review, insufficient confidence, competing support or exhausted
|
||||
invalid-response retries declines the request. The case returns to
|
||||
INVESTIGATING_HYPOTHESES in the same analysis revision. The immutable review reason
|
||||
is included in subsequent investigation context. No failure report or coding
|
||||
task is issued by this rejection. Account balances, assessed results, questions
|
||||
and evidence survive. The only reverse lifecycle transition is guarded by the
|
||||
persisted pending request and its declined review. It is not a general rewind.
|
||||
|
||||
Each review has its own context, calls and error namespace under
|
||||
`analysis/diagnosis/NNNN/early/NNNN/`. Historical requests and prompts stay intact;
|
||||
completed acceptance still uses the canonical diagnosis completion checkpoint.
|
||||
A process restart reuses the persisted model call rather than requesting it again.
|
||||
Pending old investigation prompts retain their original protocol. Existing loops
|
||||
adopt early-review support at their next decision boundary.
|
||||
|
||||
## Limits
|
||||
|
||||
`[investigation].max_early_reviews = 3` is a configurable operational default,
|
||||
frozen with the revision policy. Older policies without it use three. Reaching
|
||||
the limit disables further early requests, not ordinary investigation or normal
|
||||
final evaluation. A declined request also needs new evidence content before
|
||||
another request; a fresh ID for identical content does not suffice.
|
||||
This prevents repeated unchanged final evaluations. It is independent of per
|
||||
hypothesis research budgets, token limits and provider retry configuration.
|
||||
Confidence levels are qualitative ratings, not calibrated probabilities.
|
||||
@@ -41,13 +41,15 @@ still follows BudgetManager's policy-change rejection rules.
|
||||
|
||||
## Deterministic cycle
|
||||
|
||||
1. Select an eligible hypothesis in numeric-ID round-robin order. No selected
|
||||
cause terminates the case early; alternatives remain scheduled.
|
||||
1. Give each hypothesis one initial turn, then prefer higher confidence with
|
||||
numeric-ID ties. A reviewed early completion request may defer alternatives;
|
||||
see [Early diagnosis review](early-diagnosis-v1.md).
|
||||
2. Build a bounded context from canonical case, accepted interpretation, current
|
||||
hypothesis states, questions, notes, budget policy/balances, current-revision
|
||||
stored evidence and the previous result. First iteration has no prior result.
|
||||
3. Persist that context and its hash before the model call. `investigation_v1`
|
||||
prompt and `investigation-output-v1` schema define the model contract.
|
||||
is retained for historical requests. New prompts extend the output contract
|
||||
with reviewable early diagnosis and assessment-only deferral.
|
||||
4. Execute one eligible LLM round through the existing persisted failover/retry
|
||||
gateway. No global worker lock spans the external call.
|
||||
5. Validate the whole reply: schema, references, confidence, question updates,
|
||||
@@ -58,7 +60,7 @@ still follows BudgetManager's policy-change rejection rules.
|
||||
8. Persist raw reply and normalized evidence, then account its deterministic
|
||||
novelty through `record_evidence`. Semantic assessment occurs in a later
|
||||
model iteration, not on the result's publication step.
|
||||
9. Publish the updated hypothesis state and advance the round-robin cursor.
|
||||
9. Publish the updated hypothesis state and scheduling position.
|
||||
|
||||
Validation simulates settlement and reservation with the same deterministic budget
|
||||
rules used by execution. A valid assessment with an invalid next action is wholly
|
||||
|
||||
Reference in new issue
Block a user