VianoBugAnalysis/docs/evidence-store-v1.md

EvidenceStore v1

This milestone persists validated tool replies and confirmed test feedback as portable evidence. It adds no model calls, autonomous investigation, diagnosis, patch generation, external research service, web UI or bot steering. The authoritative architecture is unchanged.

Controller API

from viano_bug_analysis.evidence import EvidenceStore

# Reserve each requested action through BudgetManager before the query.
reply = router.query(budget.snapshot(case_id, package_id).package, request)
evidence = EvidenceStore(store, router=router)
saved = evidence.store_tool_results(
    case_id, package_id, "iteration-0001", request, reply,
)
budget.record_evidence(case_id, package_id, "H1", "query-0001", saved.record_refs[0])
# A future validated controller assessment supplies the bounded semantic score.
budget.settle(case_id, package_id, "H1", "query-0001",
              semantic_score=50, reason="Assessment of the stored evidence")

The caller's router must use the same trusted descriptor adapter and operational limits as the original query. Before publication, EvidenceStore deterministically recomputes the whole batch against the current package and compares the complete reply. Invented snippets, rows, counts, provenance, changed identities or stale package generations reject the entire batch. Each result must match a still reserved budget action and operation. The full batch is published atomically; no partial evidence records or ghost IDs survive rejection.

store_feedback(case_id, package_id, operation_id, feedback_ref) imports an already checkpoint-confirmed async test reply, bound to the original hypothesis, action and pending reservation. Draft/unconfirmed feedback is not evidence. Feedback and attachment hashes are verified; no new test request is submitted.

snapshot(case_id, package_id) validates and returns all record references in creation order without requiring an active current budget revision. History is therefore readable while a reopened case has not yet rebuilt its hypotheses. An empty store has index_ref = None and no records. Mutations require initialized budget accounting and GENERATING_HYPOTHESES or INVESTIGATING_HYPOTHESES. Blocked snapshots preserve their budget and evidence; new records wait for the existing controlled resume path.

Records, content and immutable indices

The controller assigns E000001, E000002, … across the entire case lifecycle. IDs do not reset at an analysis revision or after service restart/transport. The model cannot supply IDs, novelty scores, duplicate markers or budget values.

analysis/evidence/0001/<hashed-operation-id>/request.json
analysis/evidence/0001/<hashed-operation-id>/result.json
analysis/evidence/0001/<hashed-operation-id>/E000001.json
analysis/evidence/0001/<hashed-operation-id>/index.json
analysis/evidence/content/<content-sha256>.json

evidence-record-v1 binds case/package identity and revision, hypothesis/action, operation, content ref/hash, first identical record (duplicate_of), novel atom hashes and deterministic novelty score. Its origin links the unchanged request and reply with hashes and original reply position, or the confirmed feedback ref and hash. Source bytes, line numbers, row JSON pointers, coverage and truncation markers remain in the stored normalized content and original reply.

evidence-content-v1 separates actual content from query IDs, provider metadata, pagination bookkeeping and source package generation. Source occurrences retain location/hash/text; database rows retain location/hash/selected row values; schema descriptors retain columns and provenance. Full contents are reused by hash. Identical content obtained again gets a new occurrence record with a reference to the first record; neither original record nor provenance is overwritten.

evidence-index-v1 is an immutable cumulative index of record IDs/refs/hashes, with a parent ref/hash and next-ID counter. manifest.artifacts.evidence_index points to the latest index. Checkpoints bind that index hash and the new record refs. Deleting a pointer or changing counts, contents, origin hashes, duplicate markers or novelty metadata fails replay; it cannot reset IDs or restore novelty. All original and unknown package payloads survive unchanged. New publications increase package_generation, leaving analysis_revision unchanged.

Novelty and budget integration

Each located content item is an atom. Its identity includes content kind, scope, coverage and the original item's data/provenance. Atoms are tracked across all stored evidence, hypotheses and analysis revisions. Sorting and deduplication are deterministic; changing request/action IDs or grouping an existing excerpt into a new result is not new evidence.

novelty_score = floor(100 * unseen_item_count / returned_item_count)

Empty findings and confirmed unable_to_test replies have score zero. A partly overlapping excerpt can only claim its new items. For example, one new line in a two-line excerpt caps novelty at 50; rediscovering both lines caps it at zero. This detects exact located-content reuse, not semantic equivalence or correctness. Different projections, genuinely different source snapshots and richer excerpts can contain new atoms; judging their practical relevance remains the semantic assessment's job. Coverage and truncation remain explicit, so an empty partial snapshot is not a claim of server-wide absence.

BudgetManager.record_evidence verifies record/index/checkpoint bindings, loads the canonical content and passes only newly observed atoms to accounting. Its optional novelty_cap result field is a backward-compatible addition to the budget v1 request/event schemas. The deterministic score cannot exceed either the evidence cap or the configured new-evidence score. Empty novel payloads earn no novelty reward even when a controller supplies semantic score 100. Existing direct record_result callers remain supported unchanged.

Evidence publication alone does not debit, refund, settle, refill or terminate any account. Result accounting and later semantic settlement remain separate idempotent steps. A crash between those steps leaves the reservation recoverable, not free to allocate again. Rewards still satisfy reward < cost and retain the configured low-information stopping rules. No hypothesis confidence or root cause is inferred by the store.

Robustness and limits

The store uses a per-case evidence lock, then the existing short publication locks and generation comparison. No lock waits on an external human or model. Read-only recomputation verifies local snapshots while publication is protected. Repeated identical operation_id payloads reuse the existing records; changed payloads reject. IDs are scoped by revision/operation for operation replay but the evidence ID sequence remains case-wide. Completed publication with a failed local index update is repaired on replay without another record or budget debit.

Replay validates schemas, index lineage, hashes, original request/reply bindings, source/feedback attachment hashes and recomputes novelty/duplicate metadata. Hashes detect inconsistency, not an adversary rewriting and rehashing an entire case history; they are not signatures.

EvidenceLimits defaults to 4 MiB per JSON artifact, 10,000 records and 64 MiB across distinct retained evidence artifacts. Limits reject explicitly and are configurable operational defaults, not permanent v1 format caps. Package, case, router and budget limits remain independently applicable. Cumulative indices and full history replay trade simplicity for overhead; optimization is later work.

CaseAssembler excludes only checkpoint-declared evidence files and their content refs from raw input identity. Thus storing investigation evidence does not invalidate the accepted case; undeclared files still count as new raw input.

Tests cover preservation, provenance, duplicate and overlapping excerpts, scoped empty results, forged/stale replies, missing reservations, idempotence, limits, pointer deletion, reopen/transport, publication crashes and confirmed test feedback. The real BUG-20 regression additionally stores its source/schema/row results and accounts them through the existing budget engine, using simulated semantic assessments and no known fix.