8.2 KiB
EvidenceStore v1
This milestone persists validated tool replies and confirmed test feedback as portable evidence. It adds no model calls, autonomous investigation, diagnosis, patch generation, external research service, web UI or bot steering. The authoritative architecture is unchanged.
Controller API
from viano_bug_analysis.evidence import EvidenceStore
# Reserve each requested action through BudgetManager before the query.
reply = router.query(budget.snapshot(case_id, package_id).package, request)
evidence = EvidenceStore(store, router=router)
saved = evidence.store_tool_results(
case_id, package_id, "iteration-0001", request, reply,
)
budget.record_evidence(case_id, package_id, "H1", "query-0001", saved.record_refs[0])
# A future validated controller assessment supplies the bounded semantic score.
budget.settle(case_id, package_id, "H1", "query-0001",
semantic_score=50, reason="Assessment of the stored evidence")
The caller's router must use the same trusted descriptor adapter and operational limits as the original query. Before publication, EvidenceStore deterministically recomputes the whole batch against the current package and compares the complete reply. Invented snippets, rows, counts, provenance, changed identities or stale package generations reject the entire batch. Each result must match a still reserved budget action and operation. The full batch is published atomically; no partial evidence records or ghost IDs survive rejection.
store_feedback(case_id, package_id, operation_id, feedback_ref) imports an
already checkpoint-confirmed async test reply, bound to the original hypothesis,
action and pending reservation. Draft/unconfirmed feedback is not evidence.
Feedback and attachment hashes are verified; no new test request is submitted.
snapshot(case_id, package_id) validates and returns all record references in
creation order without requiring an active current budget revision. History is
therefore readable while a reopened case has not yet rebuilt its hypotheses.
An empty store has index_ref = None and no records. Mutations require initialized
budget accounting and GENERATING_HYPOTHESES or INVESTIGATING_HYPOTHESES.
Blocked snapshots preserve their budget and evidence; new records wait for the
existing controlled resume path.
Records, content and immutable indices
The controller assigns E000001, E000002, … across the entire case lifecycle.
IDs do not reset at an analysis revision or after service restart/transport.
The model cannot supply IDs, novelty scores, duplicate markers or budget values.
analysis/evidence/0001/<hashed-operation-id>/request.json
analysis/evidence/0001/<hashed-operation-id>/result.json
analysis/evidence/0001/<hashed-operation-id>/E000001.json
analysis/evidence/0001/<hashed-operation-id>/index.json
analysis/evidence/content/<content-sha256>.json
evidence-record-v1 binds case/package identity and revision, hypothesis/action,
operation, content ref/hash, first identical record (duplicate_of), novel atom
hashes and deterministic novelty score. Its origin links the unchanged request
and reply with hashes and original reply position, or the confirmed feedback ref
and hash. Source bytes, line numbers, row JSON pointers, coverage and truncation
markers remain in the stored normalized content and original reply.
evidence-content-v1 separates actual content from query IDs, provider metadata,
pagination bookkeeping and source package generation. Source occurrences retain
location/hash/text; database rows retain location/hash/selected row values; schema
descriptors retain columns and provenance. Full contents are reused by hash.
Identical content obtained again gets a new occurrence record with a reference
to the first record; neither original record nor provenance is overwritten.
evidence-index-v1 is an immutable cumulative index of record IDs/refs/hashes,
with a parent ref/hash and next-ID counter. manifest.artifacts.evidence_index
points to the latest index. Checkpoints bind that index hash and the new record
refs. Deleting a pointer or changing counts, contents, origin hashes, duplicate
markers or novelty metadata fails replay; it cannot reset IDs or restore novelty.
All original and unknown package payloads survive unchanged. New publications
increase package_generation, leaving analysis_revision unchanged.
Novelty and budget integration
Each located content item is an atom. Its identity includes content kind, scope, coverage and the original item's data/provenance. Atoms are tracked across all stored evidence, hypotheses and analysis revisions. Sorting and deduplication are deterministic; changing request/action IDs or grouping an existing excerpt into a new result is not new evidence.
novelty_score = floor(100 * unseen_item_count / returned_item_count)
Empty findings and confirmed unable_to_test replies have score zero. A partly
overlapping excerpt can only claim its new items. For example, one new line in
a two-line excerpt caps novelty at 50; rediscovering both lines caps it at zero.
This detects exact located-content reuse, not semantic equivalence or correctness.
Different projections, genuinely different source snapshots and richer excerpts
can contain new atoms; judging their practical relevance remains the semantic
assessment's job. Coverage and truncation remain explicit, so an empty partial
snapshot is not a claim of server-wide absence.
BudgetManager.record_evidence verifies record/index/checkpoint bindings, loads
the canonical content and passes only newly observed atoms to accounting. Its
optional novelty_cap result field is a backward-compatible addition to the
budget v1 request/event schemas. The deterministic score cannot exceed either
the evidence cap or the configured new-evidence score. Empty novel payloads earn
no novelty reward even when a controller supplies semantic score 100. Existing
direct record_result callers remain supported unchanged.
Evidence publication alone does not debit, refund, settle, refill or terminate
any account. Result accounting and later semantic settlement remain separate
idempotent steps. A crash between those steps leaves the reservation recoverable,
not free to allocate again. Rewards still satisfy reward < cost and retain the
configured low-information stopping rules. No hypothesis confidence or root
cause is inferred by the store.
Robustness and limits
The store uses a per-case evidence lock, then the existing short publication
locks and generation comparison. No lock waits on an external human or model.
Read-only recomputation verifies local snapshots while publication is protected.
Repeated identical operation_id payloads reuse the existing records; changed
payloads reject. IDs are scoped by revision/operation for operation replay but
the evidence ID sequence remains case-wide. Completed publication with a failed
local index update is repaired on replay without another record or budget debit.
Replay validates schemas, index lineage, hashes, original request/reply bindings, source/feedback attachment hashes and recomputes novelty/duplicate metadata. Hashes detect inconsistency, not an adversary rewriting and rehashing an entire case history; they are not signatures.
EvidenceLimits defaults to 4 MiB per JSON artifact, 10,000 records and 64 MiB
across distinct retained evidence artifacts. Limits reject explicitly and are
configurable operational defaults, not permanent v1 format caps. Package, case,
router and budget limits remain independently applicable. Cumulative indices
and full history replay trade simplicity for overhead; optimization is later work.
CaseAssembler excludes only checkpoint-declared evidence files and their content refs from raw input identity. Thus storing investigation evidence does not invalidate the accepted case; undeclared files still count as new raw input.
Tests cover preservation, provenance, duplicate and overlapping excerpts, scoped empty results, forged/stale replies, missing reservations, idempotence, limits, pointer deletion, reopen/transport, publication crashes and confirmed test feedback. The real BUG-20 regression additionally stores its source/schema/row results and accounts them through the existing budget engine, using simulated semantic assessments and no known fix.