Add optional Gemini context caching and cache usage accounting

docs/llm-context-caching.md

@@ -0,0 +1,131 @@
+# Optional context caching and usage accounting
+
+The package remains the complete, authoritative record. Saved requests, evidence,
+responses, checkpoints and provider configuration references remain portable.
+No cache handle, credential or provider conversation ID becomes required case
+state. `docs/architecture.md` and the investigation full-history policy are
+unchanged. All evidence remains in every investigation input.
+
+## First implementation
+
+The existing `openai_compatible` adapter now reads
+`usage.prompt_tokens_details.cached_tokens` and
+`usage.completion_tokens_details.reasoning_tokens`. This exposes implicit cache
+hits when the provider actually reports them, without changing existing requests
+or enabling explicit caching. An absent count means unknown, not zero.
+
+The optional `gemini` adapter uses the native GenerateContent API. It moves the
+unchanged system instructions and authoritative output schema to the beginning
+of the wire request. Only an exact top-level schema duplicate is removed from
+the live user JSON; no case facts, identifiers, evidence or assessments are
+removed. The archived request is not changed. This is deterministic transport
+preparation, not another LLM summary.
+
+With `context_cache = true`, this instruction/schema prefix can be stored as a
+Gemini `cachedContents` resource. Case-specific input and all research history
+are still sent in full. The prefix is independent of the currently selected
+hypothesis. Reuse requires an exact match of endpoint, provider ID, model,
+credential fingerprint, prefix and TTL. The adapter retains at most eight
+entries (including short-lived failed-creation entries), in process memory only.
+Different phases have different instructions/schema and therefore different
+cache entries. Final diagnosis remains an independent request, without a prior
+investigation conversation.
+
+This intentionally does **not** implement a provider conversation/session or
+cache the growing evidence history. It establishes measured caching without
+changing what the model knows. A later session adapter must preserve full replay,
+provider failover and an independent final evaluation; its quality and total
+cost need a separate comparison. Caching itself promises no reasoning-quality
+improvement.
+
+## Configuration and fallback
+
+See `config/llm.gemini-cache.example.toml`. Existing local configuration files
+are not edited. The native base URL ends in `/v1beta`, **not** `/v1beta/openai`.
+`context_cache` defaults to false and is allowed only for the native adapter.
+`cached_input_cost_per_million` can also be set for OpenAI-compatible providers
+that report cache usage. Prices remain optional, model-specific configuration.
+New default fields are omitted from configuration snapshots to preserve frozen
+operation hashes and replay of old cases.
+
+`cache_min_prefix_chars` is an eligibility heuristic, not a token count or API
+minimum. A short prefix may therefore obtain no explicit cache saving. The
+provider remains authoritative about minimum tokens and model support. There
+is no padding to meet a minimum. TTL defaults to five minutes; expiry is not
+extended automatically. Restarting a worker rebuilds a cache from complete
+requests rather than depending on a remote handle.
+
+Cache creation has a small bounded part of the overall request deadline (at
+most five seconds and one quarter of the provider timeout). Rejection, timeout,
+unsupported caching or an invalid cache-create response falls back to the full
+request. Failed creation is suppressed for sixty seconds. A cached generation
+rejected with HTTP 400, 403 or 404 invalidates the handle and retries once with
+the full request, within the same overall deadline. Other failures use normal
+provider failover/retry. No unbounded transport retry loop is introduced.
+Successful allocations expire remotely by TTL even if the worker exits. Keys
+are sent in headers; redirects and ambient proxies are disabled. Native
+`reasoning_effort` uses Gemini `thinkingLevel` (model support must be checked);
+`none` is rejected rather than silently mapped to another setting.
+
+## Reading the measurements
+
+Each round's metadata and safe `llm_round` telemetry can include:
+
+| Field | Meaning |
+| --- | --- |
+| `input_tokens` | Total effective input, including cached input |
+| `cached_input_tokens` | Provider-reported cached part |
+| `uncached_input_tokens` | Total minus cached part, only if both are known and consistent |
+| `output_tokens` | Billed generated tokens; native Gemini candidates plus thoughts |
+| `reasoning_tokens` | Reported thinking subset, not added to output cost a second time |
+| `estimated_input_cost` | Uncached input estimate |
+| `estimated_cached_input_cost` | Cached input estimate |
+| `estimated_output_cost` | Output estimate, including thinking |
+| `estimated_cost` | Generation estimate; excludes separately reported cache storage |
+| `cache_status` | `disabled`, `below_minimum`, `created`, `reused`, or `fallback` |
+| `cache_created_tokens` | Newly allocated tokens, recorded once per allocation |
+| `estimated_cache_storage_cost` | Newly allocated tokens × configured TTL hours × storage rate / 1M |
+
+Token counters are available as `vba_llm_tokens_total{direction="..."}`; the
+directions overlap (total input includes cached input, output includes thinking).
+Do not sum all directions. Cost components use
+`vba_llm_cost_component_total{component="...",currency="..."}`. The storage
+component is distinct from generation costs. Unknown values remain null and
+usage-unknown counters identify incomplete accounting.
+
+Storage is an estimate for the **whole reserved TTL**, counted on creation only,
+not on every hit. It is not a provider invoice. A cache created before a
+temporary generation failure is recorded in the failed provider attempt's
+`cache_usage`; ordinary unsuccessful generation attempts can still have unknown
+usage. A crash between remote allocation and local persistence can also leave
+unrecorded charges. Never treat these estimates as a precise account balance.
+If cache creation loses its response, newly allocated tokens remain unknown;
+they are not reported as zero.
+
+For old configurations with no cache price and no reported cached count, the
+legacy full-input-price estimate remains available. With a known positive cache
+count and no cache rate, or with a configured cache rate and missing cache usage,
+the exact generation estimate is unknown. No fake discount is inferred.
+
+For an economic comparison, use measured cached/uncached input and output
+estimates plus newly allocated storage, latency and outcome quality. There is
+no program-side Euro abort cap. The existing local configuration remains usable
+for collecting implicit-hit measurements before choosing the native adapter.
+No paid call is needed to run the offline protocol/accounting regression tests.
+
+An offline inspection of the archived BUG-000017 selected-context trial found an
+instruction/schema prefix of 18,703 characters for `investigation_project_v9`,
+with 59,900 characters remaining live in the inspected request. That prefix
+passes the default character heuristic; only a real provider response can
+confirm token eligibility, hits and savings. The smaller entity-resolution
+prefixes (3,730 / 4,340 characters) do not pass it. No live API call was made
+for this inspection; it is not a new quality or cost benchmark.
+
+## API sources checked 2026-10-16
+
+- [Gemini caching guide](https://ai.google.dev/gemini-api/docs/generate-content/caching)
+- [CachedContents reference](https://ai.google.dev/api/caching)
+- [GenerateContent usage metadata](https://ai.google.dev/api/generate-content#UsageMetadata)
+- [Thinking configuration](https://ai.google.dev/gemini-api/docs/generate-content/thinking)
+- [Model-specific pricing](https://ai.google.dev/gemini-api/docs/pricing)
+- [Interactions session API, for a separate future adapter](https://ai.google.dev/gemini-api/docs/interactions-overview)