VianoBugAnalysis/docs/causal-confidence-and-local-performance.md

# Causal confidence and local checkpoint performance

The stable-case-cache Bug 17 trial reached 20 dispatches without a fatal error,
but kept selecting a "missing/mismatched quest accept hook" hypothesis while
finding a matching hook, the relevant quest ID, dispatch and AI creation. Novel
information about working implementation was being confused with evidence of
the hypothesized defect. This is a reasoning-quality issue, independent of
cache hits. No new paid model run was made for this change.

## Explicit causal review, without another LLM call

New project investigation contexts use `investigation_project_v12`. Each existing
model answer now includes a short `update.causal_review`:

- `effect`: supports / contradicts / mixed / neutral, for the exact selected mechanism;
- `support`, `against`: up to eight known stored evidence IDs on each side;
- `reason`: at most 600 characters explaining the causal relation.

The review covers accumulated evidence, not only the most recent query. A high
`assessment.semantic_score` means informative evidence, not positive causal
support. Locating a normal function/trigger/ID must not support its absence.
Untraced links are uncertainty, not proof of a defect. A complete scoped empty
query can support missing-data hypotheses; schema evidence can establish a schema
defect but does not establish runtime database contents. No Vcore/WoW-specific
entity, ID, implementation pattern or natural-language truth classifier is
hardcoded into the controller.

The controller validates references, their presence in `update.evidence_refs`,
and the declared direction's required support/against lists. Unknown references
or invalid reviews reject the entire decision before tools, assessment settlement
or spending. Once valid, deterministic calibration applies:

| Declared cumulative effect | Effective controller confidence |
| --- | --- |
| contradicts | weakened |
| neutral or mixed, with requested supported/strongly_supported | plausible |
| otherwise | requested confidence |

Calibration is not another invalid-answer/retry loop: a contradictory high
confidence claim is capped in the accepted controller update. The original model
answer remains immutable. Requested confidence, effective confidence and the
review are retained in assessment notes. Existing confidence scheduling then
prefers stronger alternatives after bootstrap; it does not exhaust the weakened
hypothesis's budget just to finish it. Useful counterevidence can still receive
ordinary information rewards, with all budget invariants unchanged.

Early diagnosis uses the effective confidence, so contradiction/mixed/neutral
cannot bypass its strong-evidence gate. Independent final diagnosis remains in
place. The model still has to classify the meaning of evidence correctly: this
contract makes that judgment explicit and auditable, not infallible. Quality
needs verification in a later real model run. Older prompt contracts, saved
answers and replay schemas retain their old behaviour.

Stable aliases and the full-history cache partition continue in v12. No historic
research payload is filtered, and no additional model call, summary or paid
diagnostic review is introduced. The new short output review has some token cost;
it is bounded and intended to avoid much more expensive misdirected iterations.

## Local performance bottleneck

The measured trial had 148 durable ZIP generations: 4,427,901,136 bytes in total.
The self-contained current g148 package was 36,556,503 bytes. `latest()` hashed
every historical ZIP on every lookup and revalidated all of them after restart.
Budget snapshots and checkpoint operations invoked this repeatedly, producing
growing local costs unrelated to provider inference.

Normal `CheckpointStore.latest()` now selects the newest managed-name candidate
and fully validates **that package**, including manifest identity/generation,
all archive contents, configured safety limits and the complete embedded
checkpoint history. Names select only a candidate; they do not establish case
identity or authorize contents. Symlinks and malformed managed generation names
remain rejected. Numeric selection supports generations beyond eight digits.
Fresh full-byte hashing still protects cached validation; size/mtime is never
trusted. A corrupt or misidentified head is an error, never a rollback.

Old ZIP snapshots remain untouched. Their embedded history/content is already
carried by the head, as required for a portable case. Normal lookup no longer
audits the physical health of redundant older ZIP files. Corruption confined to
an obsolete snapshot does not block a valid current case. Explicit
`latest(case_id, package_id, audit_history=True)` retains the former full-snapshot
audit and will detect it. This separates current-case validation from backup
auditing; it does not delete history or relax current package validation.

`checkpoint_lookup` is also timed through existing operational telemetry, so
future trials can distinguish lookup time from other nested operations. Nested
operation durations must not be summed as independent wall time.

## Offline checks and limits of the measurement

A cold `BudgetManager.snapshot` on the archived g148 case, with cProfile enabled,
changed from **38.836 seconds / 99,054,413 function calls** to
**0.653 seconds / 1,805,962 calls** after this change. Both measurements include
the same read/replay operation and fresh CheckpointStore instances. OS caches,
profiling and machine load affect timings; this is a local diagnostic comparison,
not a prediction of end-to-end trial speed or API costs.

Regression tests cover calibrated confidence and scheduling, high-value
counterevidence/rewards, empty-query evidence, atomic invalid-reference rejection,
old contracts, same-size/same-mtime tampering, corrupt heads, forged newer names,
head-only reads, and explicit full-history audits. The original trial package
and its existing investigation state are not rewritten or resumed by this fix.