Files
VianoBugAnalysis/docs/llm-context-efficiency-v1.md

6.4 KiB
Raw Permalink Blame History

Identity context and reasoning configuration

Entity-resolution protocol 3 omits sources of kind unread_reference and fact_reference from plan and assessment messages. Those sources cannot ground identity searches. All readable sources (including human observations, reports, quest descriptions, entities, technical facts and uncertainty) remain unchanged. Assessment retains all candidates, captures, truncation and coverage information. No canonical case data or archive member is removed. Final interpretation is unchanged. The full context remains persisted; protocol 1/2 reconstruct their original requests. New cases select protocol 3. Existing cases keep their frozen protocol, so use a fresh trial to measure the reduced prompts.

On the quality-diagnostics-20261015 sample, message characters fall from 130430 to 44913 for planning (65.6%) and from 137708 to 52191 for assessment (62.1%). These are offline character measurements, not measured API tokens or cost savings. No LLM quality equivalence is claimed before a fresh live trial.

Provider settings

max_output_tokens already configures the output limit using token_parameter. Optional reasoning_effort is now forwarded as a top-level chat-completions field. Allowed values: none, minimal, low, medium, high; actual support depends on provider/model. Omission preserves provider defaults and historical configuration snapshot hashes. An explicit value is part of the persisted configuration snapshot. There is no automatic downgrade or silent retry when a provider rejects a setting. This is a reasoning level, not an exact thinking-token quota. It does not change investigation credits, API account credit, timeout or package-format limits.

Google documents support and model-dependent mapping here: https://ai.google.dev/gemini-api/docs/openai#thinking

Example provider settings for the next quality trial:

max_output_tokens = 16384
reasoning_effort = "medium"

The output limit is a ceiling, not a request to generate that many tokens. Thinking and output accounting are provider-dependent. More output capacity can increase cost; monitor actual usage and finish diagnostics. No raw reasoning is requested or persisted by these settings.

Identity-only reasoning override

Each provider optionally accepts entity_resolution_reasoning_effort with the same validated values as reasoning_effort. It applies only to versioned entity_resolution_plan_vN and entity_resolution_assessment_vN requests. All interpretation, hypothesis and investigation requests retain the provider's normal reasoning_effort. Omission preserves previous configuration snapshots.

reasoning_effort = "medium"
entity_resolution_reasoning_effort = "low"
max_output_tokens = 16384

The override is recorded in configuration snapshots and their hashes. Existing pending calls cannot silently change configuration mid-retry. A new entity assessment attempt uses the new settings; accepted plan and previous assessment artifacts remain unchanged and are checked without repeating their API calls. No attempt counters or investigation budgets are reset.

Compact interpretation protocol 6

Fresh entity-resolution inputs freeze interpretation protocol 6. Its prompt receives a deterministic view of the full archived context. Only long storage identifier fields containing digest material (40+ hexadecimal characters) use short P aliases. Equal identifiers retain equal aliases; different identifiers remain distinct. Filenames are retained as labels where applicable. Meaningful source paths without digest material, human/descriptive text and database row objects remain unchanged. This is not a semantic summary or evidence ranking.

Repeated origin/provenance objects are stored once in input.shared_metadata and referenced with M metadata aliases. Their author, timestamp, snapshot scope and other fields remain available; these aliases are not evidence source IDs. All B/O/C/T/E/F/N/D/X source IDs and source kinds remain present, including unread sources. Query filters, rows, completeness/truncation, next offsets, conflicting names and uncertainty remain present. JSON whitespace is reduced in this new prompt only. The stored full context, canonical case, raw archives and outputs are unchanged. Existing protocols 1–5 preserve their original request bytes.

On the completed typed-names-20261015 sample, the initial interpretation messages previously contained 151857 characters; the compact representation contains 83681 characters (44.9% fewer). These sizes are measured offline; this is not a claim about exact Gemini tokens, cost, or live quality. The next fresh LLM trial must verify role/ID correctness, item-name uncertainty, visibility prerequisites and investigation directions. No LLM pre-summary and no extra model calls are introduced. Name-resolution prompts are unchanged.

Hypothesis generation v2

New hypothesis generation checkpoints freeze prompt_version=hypotheses_v2 in the archived hypothesis context. Existing contexts without that field reconstruct v1 requests and retain their hypotheses and budgets unchanged. V2 applies the same deterministic compact presentation to the entire hypothesis input, retaining case sources, accepted interpretation sources, prior proposals and budget policy. No additional LLM call or semantic summary is introduced.

On the saved stable-resolution-20261015 generation, initial message characters fall from 153692 to 86127 (44.0%). Actual provider tokens and quality require a fresh live comparison; no token or monetary saving is guaranteed by character size.

V2 explicitly distinguishes supplied evidence from conditional causal mechanisms. Untested item/aura associations remain permissible research questions. A nearby name must not be presented as an established ID binding. Concrete API/enum/action numbers require an explicit mapping in cited readable local evidence; otherwise the model asks which implementation realizes the conceptual operation. Existing interpretation speculation is not upgraded to a verified fact. The prompts do not prescribe a known correct cause or specific test-case IDs.

These are prompting constraints, not a complete semantic fact checker over free text. Existing deterministic source/count/budget validation remains in place. A later live trial must assess whether this prevents the observed unsupported item equivalence and action-number examples while preserving causal diversity.