Allow shared evidence and bounded causal diagnosis with retry feedback
This commit is contained in:
1 parent
5082354bc3
commit
c1f44e4e02
21 files changed
+727
-41
No files matched your search
@@ -29,9 +29,10 @@ uncertainty, final reason and recommended human-review actions.
|
||||
|
||||
A confirmed result requires supported/strongly_supported confidence, a supported
|
||||
selected hypothesis that was not factually rejected, nonempty stored root-cause
|
||||
findings also cited by its evaluation, and at least one alternative with actual
|
||||
investigation actions and a reasoned countercheck citing evidence originating
|
||||
from that alternative. Unknown/duplicate/missing hypothesis IDs and fabricated
|
||||
findings also cited by its evaluation, and at least one alternative with a
|
||||
reasoned countercheck citing actual shared findings. Evidence provenance does
|
||||
not restrict which hypotheses may use it. See [Decision v2](diagnosis-decision-v2.md)
|
||||
for explicit scope limits, contradictions and retry feedback. Unknown/duplicate/missing hypothesis IDs and fabricated
|
||||
or empty support references are rejected. Loop confidence and its countercheck
|
||||
marker alone never authorize a diagnosis. Partial/empty snapshots are not proof
|
||||
of absence. The model must distinguish intended external behavior from local
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
# Evidence-grounded diagnosis decision v2
|
||||
|
||||
The decision is whether an adequately supported cause justifies a bounded,
|
||||
verifiable change, not whether every conceivable cause has been excluded.
|
||||
There remains at least one targeted alternative countercheck; unlimited
|
||||
investigation of theoretical or newly proposed hypotheses is not required.
|
||||
|
||||
Evidence IDs are usable across hypotheses, including root-cause support and
|
||||
counterchecks. The record's hypothesis ID is provenance only. Technical checks
|
||||
still require actual referenced findings; a schema-only capture is insufficient
|
||||
for causal support or a meaningful countercheck. The evaluator must explain
|
||||
which prerequisite/prediction a finding tests and respect its coverage. Absence
|
||||
within a complete filtered snapshot is not absence of all possible runtime paths.
|
||||
|
||||
New diagnosis contexts freeze evaluation_protocol=2. diagnosis_v2 and
|
||||
diagnosis_early_v2 require decision_basis: causal_explanation, scope_limits,
|
||||
sufficient_for_change and contradictions. Each material contradiction cites
|
||||
known evidence and records its resolution and reason. A confirmed output cannot
|
||||
claim sufficient support while marking a material contradiction unresolved.
|
||||
Unfinished alternatives require explicit remaining_uncertainty; another supported
|
||||
hypothesis is not automatically a contradiction or an obligation to research it.
|
||||
|
||||
The deterministic validator checks structure, identities, references, nonempty
|
||||
findings, consistency and declared unresolved contradictions. Semantic relevance,
|
||||
whether findings are actually contradictory, adequacy of a countercheck, and
|
||||
whether prose overstates evidence remain LLM judgments. The structure does not
|
||||
prove that a model has detected every contradiction or told the truth. Prompts
|
||||
explicitly reject unjustified universal exclusion and require scoped claims.
|
||||
|
||||
Cause confidence and implementation readiness remain separate. Change Request
|
||||
and output checks are unchanged; missing implementation data must not be invented.
|
||||
This does not add dynamic hypothesis creation or change investigation budgets.
|
||||
|
||||
Every new diagnosis retry includes prior failure codes, call references and
|
||||
correction guidance from the same review namespace. It never overwrites the
|
||||
frozen base context. The precise request is reproducible after a restart and
|
||||
when validating accepted output. A pending historical call retains its old
|
||||
prompt/hash; after a historical rejection the next fresh attempt uses v2 with
|
||||
feedback. Completed historical diagnoses remain replayable.
|
||||
|
||||
The current live case is not rewritten or auto-approved by this change. Its
|
||||
stored response can be structurally tested offline under shared-evidence rules;
|
||||
that is not a new semantic review of its reasoning. No extra summary LLM is used.
|
||||
@@ -8,9 +8,12 @@ The service performs no patch, SQL write or client automation.
|
||||
|
||||
New investigation prompts (`investigation_v2`, `investigation_project_v3`) offer
|
||||
`next.kind = request_diagnosis`, with `reason` and `countercheck_refs`.
|
||||
Current prompts are `investigation_v3` / `investigation_project_v4`; older
|
||||
persisted prompt versions remain reproducible.
|
||||
The selected hypothesis must be `strongly_supported`, cite concrete nonempty
|
||||
investigation evidence of its own, and cite an actual result from an alternative.
|
||||
An alternative already rated supported/strongly_supported prevents a request.
|
||||
investigation evidence, and cite findings that meaningfully test an alternative.
|
||||
Evidence is shared: collecting hypothesis IDs never restrict its use. Another
|
||||
supported alternative does not by itself establish a material contradiction.
|
||||
The controller validates IDs and structure; whether the evidence establishes
|
||||
causality and whether a negative result has sufficient coverage remain explicit
|
||||
responsibilities of the independent final evaluator.
|
||||
@@ -28,16 +31,16 @@ and its initial turn for each hypothesis remain in use outside preparation.
|
||||
|
||||
## Independent decision and continuation
|
||||
|
||||
`diagnosis_early_v1` evaluates every hypothesis, all evidence and remaining
|
||||
`diagnosis_early_v2` evaluates every hypothesis, all evidence and remaining
|
||||
uncertainty independently. Acceptance requires the requested hypothesis,
|
||||
`strongly_supported` confidence, direct evidence for it, an adequate investigated
|
||||
alternative countercheck and no other evaluation marked supported. Ordinary
|
||||
`strongly_supported` confidence, direct evidence for it, an adequate evidence-grounded
|
||||
alternative countercheck, and no explicitly unresolved material contradiction. Ordinary
|
||||
final diagnosis validation remains mandatory. An accepted review creates the
|
||||
normal diagnosis artifact; the existing separate Change Request and output
|
||||
validators still decide whether a coding task can be issued. Missing coordinates
|
||||
or other implementation prerequisites must remain explicit missing information.
|
||||
|
||||
An inconclusive review, insufficient confidence, competing support or exhausted
|
||||
An inconclusive review, insufficient confidence, unresolved material counterevidence or exhausted
|
||||
invalid-response retries declines the request. The case returns to
|
||||
INVESTIGATING_HYPOTHESES in the same analysis revision. The immutable review reason
|
||||
is included in subsequent investigation context. No failure report or coding
|
||||
|
||||
Reference in new issue
Block a user