4.4 KiB
Cross-result investigation consolidation
Historical v6 format; the current measurement CLI defaults to v7. Use
--target-version investigation_project_v6 to reproduce this version.
Only new project-backed investigation requests use investigation_project_v6.
Other LLM phases, tool result persistence, budgets and lifecycle are unchanged.
Frozen v4/v5 contexts retain their prompts, views and request hashes. The new
algorithm operates after the v5 view, before request freezing and hashing.
File identity and shared code
A FILE entry identifies the exact combination of observed file metadata (including path and content hash/blob) and pinned capture origin. Equal paths with different hashes, revisions or origins remain distinct. Captures without an origin or content identity are not assumed to describe the same snapshot. The existing shared_metadata table continues to carry repeated provenance; FILE entries reuse these exact provenance references rather than inventing one.
Source captures keep their evidence ID, query, coverage, truncation, paging and
requested/returned bounds. Their items reference shared_sources through
shared_source_lines_v1. Each reference contains FILE ID and ordered slices
[start,count] of positions in the shared line table, not source line numbers.
The table retains actual line numbers, text and truncation flags. Equal records
are transmitted once across captures, including overlapping source intervals.
Absent line numbers remain gaps. Repeated rows and original ordering reconstruct
exactly. Different records at the same line are both retained and explicitly
listed under conflicting_lines, even if the purported file hash is identical.
No observation wins silently.
Read-query paths use path_ref only when exactly one FILE identity matches the path and origin. FILE metadata contains the path. The controller can resolve known FILE aliases in source.read paths using only the active context, followed by the existing path, repository and read-only validation. Unknown aliases grant no access and remain subject to ordinary tool rejection. Existing P aliases keep their original allocation and meaning.
Equal results and different queries
Large identical result objects may share a RESULT entry. Queries and origins stay attached to each evidence record: equal returned DB rows do not imply equal filters, snapshots or coverage. Only exact typed JSON equality is used. False, zero, null and absent fields are not interchangeable. No older evidence or counterevidence is filtered out, and no extra summarization/model call is added.
The final candidate is used only if its canonical model text is smaller than the v5 view; otherwise the v5 encoding is retained within the v6 prompt. This avoids expanding a request merely to introduce reference tables. It is a deterministic representation choice, not a monetary limit or investigation stopping rule.
restore_view reconstructs the previous working view, including original item
order, duplicate records, query paths, actual values and every evidence ID.
Offline benchmarks compare canonical JSON after reconstruction, not only hashes
or field counts. The system instructions were shortened while retaining the
existing task, evidence, budget, tool and early-review rules and describing all
new and fallback encodings. Output schemas and controller gates are unchanged.
Measurements and limits
scripts/measure_investigation_context.py PACKAGE now defaults to v6, with
--target-version investigation_project_v5 available for comparison. It reports
user text and total message text (including system instructions), component
sizes and exact reconstruction. No provider request is made. Stored contexts
include rejected/unused attempts; these are not billed-call totals.
On the local archives:
| Trial | Stored contexts | v5 user text | v6 user text | v5 all messages | v6 all messages |
|---|---|---|---|---|---|
| Bug 17 | 14 | 1,017,342 | 1,007,291 | 1,125,982 | 1,087,343 |
| Bug 20 | 9 | 549,750 | 549,750 | 619,590 | 601,212 |
Additional full-message reductions are 3.43% and 2.97%, respectively. Bug 17's captured source intervals were mostly adjacent rather than overlapping; Bug 20 had little source content to consolidate. The earlier v5 reduction must not be counted again as new savings. Repeated/overlapping source workloads benefit more, as tested separately. Actual provider tokens, cost and reasoning quality still need evaluation in a later live run. Remaining unique code/history is still sent.