6.6 KiB
Stable case context cache v1
This milestone extends the working explicit cache from instructions/schema to the stable investigation case. It does not add another LLM call, summary, conversation session, historical-evidence filter or recall action. The portable package and complete archived request remain authoritative. No architecture document or system boundary is changed.
Stable and live input
The native Gemini transport partitions eligible investigation input into:
| Cached, unchanged prefix | Complete live input |
|---|---|
| Original system instructions and output schema | Selected hypothesis |
| Case/package identity, analysis revision and original case generation | Every hypothesis's current state, notes and questions |
Readable observations, facts and resolved entities (sources) |
Budget balances, actions and previous-result pointer |
| Case interpretation | Every stored research evidence payload, including other hypotheses |
| Static analysis/budget policy and project tool bindings | Early-review state, rejection feedback and future unknown fields |
| Bounded unread-source catalogs, explicitly marked as metadata | Shared provenance definitions not needed by the prefix |
This partitions the existing complete model view; it does not replace the canonical case with a different truth. Existing phase-selection and source/code consolidation policies remain unchanged. No new evidence selection is introduced.
llm/stable_context.py implements a provider-independent JSON partition and
inverse. The prefix contains an encoding version, a SHA-256 content ID and the
stable input. The live document binds to that ID. Top-level input keys are
disjoint. Shared metadata required by the prefix is included transitively; the
remaining definitions stay live. Both sides can cite definitions in the stable
part. The inverse rejects mismatched hashes, overlapping keys and provenance
redefinitions. Database row values are literal data, not metadata references.
For every partition, a runtime assertion reconstructs and compares the complete original user JSON. A changed shape or unresolved provenance falls back to the complete original view. The adapter tells the model how the two parts form one input. No field is silently overridden. The full-input fallback includes both parts, so a cache rejection never removes information. The original stored request is not mutated or replaced by a remote cache handle.
Stable aliases and replay
Sequential P/M aliases could shift when unrelated evidence introduced earlier
identifiers. Merely splitting that representation caused five different prefixes
across Bug 17's thirteen requests. New investigation_project_v11 uses
deterministic P/M aliases based on SHA-256 content (128-bit labels with explicit
collision detection). Large provenance objects are consistently shared even
before another occurrence arrives. Adding evidence, changing the selected
hypothesis or adjusting its budget therefore does not rename the case sources.
Source/fact/entity/evidence IDs remain unchanged. Alias expansion in tool requests is still validated against the exact saved context, including FILE references. It never grants an unvalidated source path. All source line blocks, DB row values, provenance scope and negative results retain their existing meaning. P/M labels remain scoped to the complete current request and case; they are not evidence IDs or provider conversation state.
Older prompt files and encodings are preserved. An already saved v10 or earlier request is reconstructed using its original version, not rewritten to v11. New project investigation contexts use v11 and retain full history. Old focused v8/v9 requests are never interpreted as full-history cache candidates.
Configuration and lifecycle
Native Gemini defaults to cache_context_version = "stable_case_v1". This
version is pinned in configuration snapshots because it changes the wire view.
Use cache_context_version = "instructions_v1" to reproduce the pre-partition
native configuration snapshot for a frozen operation. Other adapters still
receive the complete canonical request; their default snapshots are unchanged.
Use a fresh trial for a comparison, rather than changing a pending operation's
pinned configuration.
The exact prefix (including case identity and revision), system instructions, model, endpoint, provider identity, credential fingerprint and TTL determine cache reuse. Any changed stable content or interpretation creates a new prefix. Handles are disposable in-process optimizations; expiry, restart and loss of a handle are handled by reconstructing from the package. Existing bounded cache creation, failure handling, strict-test mode and provider failover remain in force. Explicit storage billing still applies. Final diagnosis remains an independent request and is not placed in an investigation conversation.
Offline measurement
python -m viano_bug_analysis.llm.cache_measurement --package PACKAGE.zip --output REPORT.json
This command validates/reads the package without contacting a provider. It pairs saved requests with their original contexts, rebuilds detached v11 views, partitions them and proves exact reconstruction for every measured request. It emits aggregate sizes and content hashes, not source text, observations, credentials or provider cache handles. Package bytes remain unchanged.
Checked on the archived trial generations below:
| Trial | Requests | Unique new prefixes | Historical live characters | New live characters | Reduction |
|---|---|---|---|---|---|
| Bug 17, cache-required trial g110 | 13 | 1 | 863,639 | 368,911 | 57.28% |
| Bug 20, full-run trial g93 | 9 | 1 | 570,203 | 161,741 | 71.63% |
All 22 v11 inputs reconstructed exactly. Reports: Bug 17, Bug 20.
These figures describe the separately transmitted live portion under cache reuse, including JSON transport encoding. The stable information is moved to the cached prefix, not deleted. Initial creation still transmits it, prefix storage and cached-token reads still cost money, TTL may require recreation, and longer content-based aliases add some representation overhead. Historical Bug 20 used an older presentation policy; the report marks that its comparison is to a rebuilt v11 view. These are character measurements, not provider-token measurements, a quality benchmark or a guaranteed billing reduction. No paid API call was made for this milestone.
The next separate step can consider rolling evidence caches. This milestone deliberately keeps all growing research and assessments live on every call.