64 lines
3.7 KiB
Markdown
64 lines
3.7 KiB
Markdown
# Investigation context v5
|
|
|
|
Historical v5 format; the current measurement CLI defaults to v7. Use
|
|
`--target-version investigation_project_v5` to reproduce this version.
|
|
|
|
New project-backed investigation contexts use `investigation_project_v5`.
|
|
Existing frozen contexts keep their version and exact request reconstruction.
|
|
No model/provider, hypothesis budgets, money limits, lifecycle or evidence
|
|
validation gates change. The complete archive retains all original evidence.
|
|
|
|
## Lossless representation before relevance filtering
|
|
|
|
Source query rows repeated file paths, hashes, sizes and other metadata for every
|
|
line. V5 factors identical fields into `common`, records the variable field names
|
|
in `columns`, and transmits ordered values in `rows` with encoding
|
|
`source_line_table_v1`. Merging common fields and each column/value row recreates
|
|
every original item. Line numbers, text and text-truncation flags remain intact.
|
|
|
|
Uniform database row results use `database_row_table_v1`: `common` contains item
|
|
metadata, and columns/rows reconstruct the original nested `row` dictionaries.
|
|
Actual scalar types, nulls, values and ordering are preserved. Nonuniform results
|
|
stay in their original representation; absent fields are never converted to null.
|
|
Query filters, snapshot origin, coverage, paging and truncation remain attached
|
|
to their original capture and evidence ID. No older findings or counterevidence
|
|
are removed. Source alias allocation remains the existing v4 allocation.
|
|
|
|
No additional LLM summary or call is used. Evidence recall/relevance filtering is
|
|
not needed for this lossless step and has not been introduced. This does not yet
|
|
solve the cost of resending large quantities of unique historical code.
|
|
|
|
## Source range behavior
|
|
|
|
Source reads retain the configurable maximum requested line count and reject
|
|
invalid/reversed/noninteger ranges. A valid bounded range crossing EOF returns
|
|
the available tail. A start past EOF returns an empty selected-lines result.
|
|
Results explicitly include total_lines, eof, requested bounds and actual returned
|
|
bounds (null for empty results). EOF is not truncation by a safety limit and does
|
|
not prove absence in other files or earlier portions of this file. Path and
|
|
pinned-Git-object safety checks are unchanged. Capture schema fields are additive
|
|
and optional for compatibility with existing captures.
|
|
|
|
New prompt guidance prioritizes targeted symbol/search/registration queries,
|
|
existing results and explicit EOF metadata over sequential whole-file reading.
|
|
It also asks the model to distinguish observation/deployment/capture/source times
|
|
and to preserve inconsistencies rather than invent an absent implementation.
|
|
There is no deterministic claim that a reported bug is fixed merely because code
|
|
exists in a newer snapshot.
|
|
|
|
## Offline measurement
|
|
|
|
`scripts/measure_investigation_context.py PACKAGE` reconstructs v4 and v5 requests
|
|
from stored contexts using PackageReader validation, checks that table decoding
|
|
exactly equals the original working view, and reports per-context character sizes.
|
|
It never calls a provider and does not modify the package. Stored contexts include
|
|
rejected/unused attempts; totals are not billed request counts or token estimates.
|
|
|
|
The Bug 17 archive's 14 contexts shrink from 1,961,028 to 1,017,342 user-message
|
|
characters (48.12%). Bug 20's nine contexts shrink from 563,131 to 549,750 (2.38%).
|
|
Every transformed view passes exact reconstruction. This difference is expected:
|
|
the former contains substantial repeated source-line metadata; the latter mostly
|
|
contains small database captures. Thus the earlier 70% reduction goal is not yet
|
|
met by this first safe step. Actual tokens, cost and reasoning quality need a
|
|
later live test; no paid experiments were run during implementation.
|