115 lines
6.6 KiB
Markdown
115 lines
6.6 KiB
Markdown
# Stable case context cache v1
|
|
|
|
This milestone extends the working explicit cache from instructions/schema to
|
|
the stable investigation case. It does not add another LLM call, summary,
|
|
conversation session, historical-evidence filter or recall action. The portable
|
|
package and complete archived request remain authoritative. No architecture
|
|
document or system boundary is changed.
|
|
|
|
## Stable and live input
|
|
|
|
The native Gemini transport partitions eligible investigation input into:
|
|
|
|
| Cached, unchanged prefix | Complete live input |
|
|
| --- | --- |
|
|
| Original system instructions and output schema | Selected hypothesis |
|
|
| Case/package identity, analysis revision and original case generation | Every hypothesis's current state, notes and questions |
|
|
| Readable observations, facts and resolved entities (`sources`) | Budget balances, actions and previous-result pointer |
|
|
| Case interpretation | **Every stored research evidence payload**, including other hypotheses |
|
|
| Static analysis/budget policy and project tool bindings | Early-review state, rejection feedback and future unknown fields |
|
|
| Bounded unread-source catalogs, explicitly marked as metadata | Shared provenance definitions not needed by the prefix |
|
|
|
|
This partitions the existing complete model view; it does not replace the
|
|
canonical case with a different truth. Existing phase-selection and source/code
|
|
consolidation policies remain unchanged. No new evidence selection is introduced.
|
|
|
|
`llm/stable_context.py` implements a provider-independent JSON partition and
|
|
inverse. The prefix contains an encoding version, a SHA-256 content ID and the
|
|
stable input. The live document binds to that ID. Top-level input keys are
|
|
disjoint. Shared metadata required by the prefix is included transitively; the
|
|
remaining definitions stay live. Both sides can cite definitions in the stable
|
|
part. The inverse rejects mismatched hashes, overlapping keys and provenance
|
|
redefinitions. Database row values are literal data, not metadata references.
|
|
|
|
For every partition, a runtime assertion reconstructs and compares the complete
|
|
original user JSON. A changed shape or unresolved provenance falls back to the
|
|
complete original view. The adapter tells the model how the two parts form one
|
|
input. No field is silently overridden. The full-input fallback includes both
|
|
parts, so a cache rejection never removes information. The original stored
|
|
request is not mutated or replaced by a remote cache handle.
|
|
|
|
## Stable aliases and replay
|
|
|
|
Sequential P/M aliases could shift when unrelated evidence introduced earlier
|
|
identifiers. Merely splitting that representation caused five different prefixes
|
|
across Bug 17's thirteen requests. New `investigation_project_v11` uses
|
|
deterministic P/M aliases based on SHA-256 content (128-bit labels with explicit
|
|
collision detection). Large provenance objects are consistently shared even
|
|
before another occurrence arrives. Adding evidence, changing the selected
|
|
hypothesis or adjusting its budget therefore does not rename the case sources.
|
|
|
|
Source/fact/entity/evidence IDs remain unchanged. Alias expansion in tool
|
|
requests is still validated against the exact saved context, including FILE
|
|
references. It never grants an unvalidated source path. All source line blocks,
|
|
DB row values, provenance scope and negative results retain their existing
|
|
meaning. P/M labels remain scoped to the complete current request and case;
|
|
they are not evidence IDs or provider conversation state.
|
|
|
|
Older prompt files and encodings are preserved. An already saved v10 or earlier
|
|
request is reconstructed using its original version, not rewritten to v11.
|
|
New project investigation contexts use v11 and retain full history. Old focused
|
|
v8/v9 requests are never interpreted as full-history cache candidates.
|
|
|
|
## Configuration and lifecycle
|
|
|
|
Native Gemini defaults to `cache_context_version = "stable_case_v1"`. This
|
|
version is pinned in configuration snapshots because it changes the wire view.
|
|
Use `cache_context_version = "instructions_v1"` to reproduce the pre-partition
|
|
native configuration snapshot for a frozen operation. Other adapters still
|
|
receive the complete canonical request; their default snapshots are unchanged.
|
|
Use a fresh trial for a comparison, rather than changing a pending operation's
|
|
pinned configuration.
|
|
|
|
The exact prefix (including case identity and revision), system instructions,
|
|
model, endpoint, provider identity, credential fingerprint and TTL determine
|
|
cache reuse. Any changed stable content or interpretation creates a new prefix.
|
|
Handles are disposable in-process optimizations; expiry, restart and loss of a
|
|
handle are handled by reconstructing from the package. Existing bounded cache
|
|
creation, failure handling, strict-test mode and provider failover remain in
|
|
force. Explicit storage billing still applies. Final diagnosis remains an
|
|
independent request and is not placed in an investigation conversation.
|
|
|
|
## Offline measurement
|
|
|
|
`python -m viano_bug_analysis.llm.cache_measurement --package PACKAGE.zip --output REPORT.json`
|
|
|
|
This command validates/reads the package without contacting a provider. It pairs
|
|
saved requests with their original contexts, rebuilds detached v11 views,
|
|
partitions them and proves exact reconstruction for every measured request.
|
|
It emits aggregate sizes and content hashes, not source text, observations,
|
|
credentials or provider cache handles. Package bytes remain unchanged.
|
|
|
|
Checked on the archived trial generations below:
|
|
|
|
| Trial | Requests | Unique new prefixes | Historical live characters | New live characters | Reduction |
|
|
| --- | ---: | ---: | ---: | ---: | ---: |
|
|
| Bug 17, cache-required trial g110 | 13 | 1 | 863,639 | 368,911 | 57.28% |
|
|
| Bug 20, full-run trial g93 | 9 | 1 | 570,203 | 161,741 | 71.63% |
|
|
|
|
All 22 v11 inputs reconstructed exactly. Reports:
|
|
[Bug 17](measurements/stable-case-cache-20261016/bug17.json),
|
|
[Bug 20](measurements/stable-case-cache-20261016/bug20.json).
|
|
|
|
These figures describe the separately transmitted **live portion** under cache
|
|
reuse, including JSON transport encoding. The stable information is moved to
|
|
the cached prefix, not deleted. Initial creation still transmits it, prefix
|
|
storage and cached-token reads still cost money, TTL may require recreation,
|
|
and longer content-based aliases add some representation overhead. Historical
|
|
Bug 20 used an older presentation policy; the report marks that its comparison
|
|
is to a rebuilt v11 view. These are character measurements, not provider-token
|
|
measurements, a quality benchmark or a guaranteed billing reduction. No paid API
|
|
call was made for this milestone.
|
|
|
|
The next separate step can consider rolling evidence caches. This milestone
|
|
deliberately keeps **all growing research and assessments live on every call**.
|