Add generic deterministic CaseBuilder with versioned input contracts

This commit is contained in:
Nico committed 2026-10-13 16:14:40 +02:00
1 parent e985e1d5f0
commit 4543e7cd31
18 files changed
+2093 -3

No files matched your search

+3 -1
View File
@@ -6,6 +6,8 @@ The implemented v1 contract is documented in
[docs/package-format-v1.md](docs/package-format-v1.md).
Job state, checkpoints and inbox admission are documented in
[docs/lifecycle-v1.md](docs/lifecycle-v1.md).
Generic case construction and its input contracts are documented in
[docs/case-builder-v1.md](docs/case-builder-v1.md).
Requires Python 3.11 or newer.
@@ -16,5 +18,5 @@ python -m venv .venv
```
The lifecycle foundation supports one-pass inbox processing and resumable package
checkpoints. It includes no analysis worker, CaseBuilder, LLM,
checkpoints and a deterministic generic CaseBuilder. It includes no analysis worker or LLM,
case interpretation, hypothesis generation or investigation tools.
+152
View File
@@ -0,0 +1,152 @@
# Generic CaseBuilder foundation v1
`architecture.md` remains authoritative. This milestone assembles supplied data
deterministically. It performs no LLM calls, interpretation, hypothesis generation,
diagnosis, investigation, external research or project-specific adapter work.
Tester hypotheses remain supplied data, never generated or promoted to facts.
## Explicit input contracts
The manifest may declare `artifacts.case_inputs`, pointing to a UTF-8 JSON index:
```json
{
"format": "viano-case-inputs",
"format_version": 1,
"reports": ["original/report.data"],
"reproductions": ["original/reproduction.data"],
"context": {
"runtime": ["original/runtime.bin"],
"database": [],
"source": [],
"research": []
},
"attachments": []
}
```
Paths and extensions have no inherent meaning. Listed reports conform to
`report-v1.schema.json`, reproductions to `reproduction-v1.schema.json`, and the
index to `case-inputs-v1.schema.json`. Schemas are bundled offline Draft 2020-12
contracts. Unlisted files are preserved as unclassified inputs without parsing
or guessing their role. A missing index produces explicit data-quality gaps.
Invalid declared versions/contracts, duplicate JSON keys, repeated sources within
an index role, unsafe or absent references are controlled errors.
A report requires format `viano-bug-report` and format_version 1. Optional fields:
title, summary, observations, comments, tester_hypotheses, expected_behavior,
expected_behavior_complete and entities. Statements have nonblank `text`, optional
`author` and timezone-aware `observed_at`, and optionally a structured `claim`
with `key` and JSON `value`. Categories are producer-supplied, not inferred from
prose. Unknown extensions remain accessible in the preserved source file.
```json
{
"format": "viano-bug-report",
"format_version": 1,
"title": "A generic observed problem",
"observations": [{"text": "The switch remains off", "author": "Tester A"}],
"tester_hypotheses": [{"text": "Perhaps a timing issue"}],
"expected_behavior": [{"text": "The switch turns on"}],
"expected_behavior_complete": false,
"entities": [{"type": "component", "id": "42", "roles": ["focus"]}]
}
```
A reproduction requires format `viano-reproduction` and format_version 1. Optional
fields: steps (strings), observations (human statements), events_ref, attachments,
facts and entities. Facts have a name and JSON value: they are supplied facts,
not independently verified technical findings. Events, attachments and all context
files remain opaque references with hashes/sizes. Supplied research stays separate;
no research is performed and no local implementation facts are inferred from it.
## Canonical case and provenance
`case-v1.schema.json` defines format `viano-case`, format_version 1, builder_version
`1`, and the architecture's main blocks: case, bug, human_observations,
expected_behavior, reproduction, entities, runtime_facts, database_context,
source_context, research, data_quality, inputs.
Identity/project/revision come only from the manifest. Each statement/fact/entity
has a source path and JSON Pointer with provenance kind. Human statements retain
author/time. Observations, comments, tester hypotheses and expected behavior are
separate. Record IDs use O/C/T/E/F namespaces; previous IDs remain stable by source
path and pointer when inputs are appended. Facts use F0001 etc. Reproduction IDs
follow source order. Integer entity IDs become strings; equal integer/string IDs
of the same explicit type combine roles and retain every source. No domain types,
relations or content are invented.
Expected-behavior completeness is missing, unspecified, reported_complete,
reported_partial or conflicting. Reported completeness is an assertion by the
reporter, not a sufficiency judgment. Different canonical JSON values for the same
explicit claim key within observations or expected behavior are flagged with
source references. Natural-language contradictions are not interpreted; comments
and tester hypotheses never participate in fact or conflict inference.
Large context/events stay outside case JSON. Fact values over the inline limit
use `value_ref` (source path and JSON Pointer) instead of `value`; nothing is
truncated or replaced by a synthetic null. Old structured inputs are inherited
from the prior case's role inventory, hash-checked, and re-read from their immutable
originals. A returning index can add only new reports without losing old statements,
facts or provenance. Prior case files remain explicitly referenced. Incompatible
prior canonical case contracts are rejected instead of guessed.
## Artifact and checkpoint integration
```python
from viano_bug_analysis.case import CaseBuilder, CaseBuildError
from viano_bug_analysis.lifecycle import Workspace, CheckpointStore
store = CheckpointStore(Workspace("service-data"))
try:
result = CaseBuilder(store).build("case-A", "stable-package")
except CaseBuildError as error:
print(error.code, error.path, str(error))
```
Input must already be admitted. VALIDATED or REOPENED enters BUILDING_CASE through
a start checkpoint; existing BUILDING_CASE work continues there. Artifacts use
`case/revisions/NNNN/case.json` and `case_summary.md`. Revision paths preserve
append-only history; manifest pointers `case` and `case_summary` select the current
documents. Existing case files and unknown artifacts are never overwritten.
Case JSON, deterministic summary and the `case-built` completion checkpoint are
published together through PackageWriter. State remains BUILDING_CASE; construction
does not execute or start interpretation. Later phase orchestration advances it.
Summary is generated only from canonical JSON, adds no interpretation and escapes
supplied Markdown/HTML. `render_summary` is pure; `CaseAssembler.assemble` is
deterministic for the same validated snapshot.
Completion uses a stable step ID per builder version and analysis revision.
Repeats check the input request hash, case contract and summary, reuse output and
repair local metadata. Crashes after publication do not create another generation;
unconfirmed work retries from the same snapshot. Changed payloads after completion
require reopening with a new analysis revision. The request hash includes explicit
source roles and all unclassified payloads, excluding known generated case outputs
and checkpoint artifacts. Analysis revisions and package generations stay separate.
Prepared output calls `CheckpointStore.advance(expected_generation=...)` to reject
concurrent changes before committing. Identical confirmed requests remain replayable
with the old expected generation. Locks and this optimistic check prevent stale
prepared data from being committed over newer work. There are no external side
effects in construction.
## Configurable resource defaults
`CaseBuilderLimits` holds operational policy, not immutable format limits:
| Limit | Default |
|---|---:|
| Individual index/report/reproduction JSON | 1 MiB |
| Aggregate structured input JSON | 8 MiB |
| Structured input documents | 256 |
| Prior/canonical case JSON | 8 MiB |
| Summary bytes | 8 MiB |
| Inline fact value | 16 KiB |
Prior case JSON is bounded separately from the structured input budget. Package
limits still apply to the archive/artifacts. Context is hashed in chunks rather
than loaded as prose. Invalid attempts preserve all source/previous case files;
they may retain a confirmed start checkpoint in BUILDING_CASE, allowing resumption
after corrected sources and a new index are appended. Changing resource defaults
does not rewrite a previously confirmed case.
+2 -1
View File
@@ -2,7 +2,8 @@
`architecture.md` remains the authoritative design. This milestone provides only
deterministic job state, checkpoint storage, locking and inbox admission. It does
not execute CaseBuilder, interpretation, hypotheses, diagnosis, tools or LLMs.
not execute interpretation, hypotheses, diagnosis, tools or LLMs. The separate
CaseBuilder module now uses these checkpoint APIs for deterministic construction.
State APIs allow future phase implementations to confirm their work; recording
a state is not evidence that this foundation executed that phase.
+6
View File
@@ -0,0 +1,6 @@
from .assembler import CaseAssembler, CaseBuilderLimits
from .builder import CaseBuilder, CaseBuildResult
from .errors import CaseBuildError
from .summary import render_summary
__all__ = ["CaseAssembler", "CaseBuilderLimits", "CaseBuilder", "CaseBuildResult", "CaseBuildError", "render_summary"]
+234
View File
@@ -0,0 +1,234 @@
import copy
import hashlib
import re
from dataclasses import dataclass
from viano_bug_analysis.lifecycle.checkpoints import checkpoint_chain
from viano_bug_analysis.lifecycle.storage import json_bytes
from viano_bug_analysis.package.validator import CHUNK_SIZE
from .contracts import read_json, require_file, validate
from .errors import CaseBuildError
from .expected_behavior import collect_expected_behavior
from .report_collector import collect_statements, provenance
from .reproduction_collector import collect_reproductions
def _stable_records(previous: list[dict], current: list[dict], prefix: str) -> list[dict]:
def identity(record):
source = record["provenance"]
return source["source_ref"], source["json_pointer"]
remaining = {identity(record): record for record in current}
result = []
ids = set()
for old in previous:
if not re.fullmatch(prefix + r"[0-9]{4,}", old["id"]) or old["id"] in ids:
raise CaseBuildError("PREVIOUS_IDS", "Prior case contains invalid or duplicate record IDs")
ids.add(old["id"])
record = remaining.pop(identity(old), None)
if record is None:
raise CaseBuildError("PREVIOUS_RECORD_LOST", "Prior source record disappeared from its structured input")
record["id"] = old["id"]
result.append(record)
next_id = max((int(value[len(prefix):]) for value in ids), default=0) + 1
for record in current:
if identity(record) in remaining:
record["id"] = f"{prefix}{next_id:04d}"
next_id += 1
result.append(record)
del remaining[identity(record)]
return result
@dataclass(frozen=True)
class CaseBuilderLimits:
"""Operational limits, independent of the JSON format contracts."""
max_json_bytes: int = 1024**2
max_structured_bytes: int = 8 * 1024**2
max_documents: int = 256
max_case_bytes: int = 8 * 1024**2
max_summary_bytes: int = 8 * 1024**2
max_inline_fact_bytes: int = 16 * 1024
def __post_init__(self):
for value in vars(self).values():
if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
raise ValueError("CaseBuilder limits must be positive integers")
class CaseAssembler:
def __init__(self, limits: CaseBuilderLimits | None = None):
self.limits = limits or CaseBuilderLimits()
def assemble(self, reader) -> dict:
manifest = reader.manifest
if manifest["analysis_revision"] < 1:
raise CaseBuildError("INVALID_REVISION", "Case construction requires an active analysis revision")
used_bytes = 0
documents = 0
def load(path, contract):
nonlocal used_bytes, documents
require_file(reader, path)
if contract == "case":
return read_json(reader, path, contract, self.limits.max_case_bytes)
documents += 1
used_bytes += reader._archive.getinfo(path).file_size
if documents > self.limits.max_documents or used_bytes > self.limits.max_structured_bytes:
raise CaseBuildError("STRUCTURED_INPUT_LIMIT", "Declared structured inputs exceed configured limits", path)
return read_json(reader, path, contract, self.limits.max_json_bytes)
chain = checkpoint_chain(reader)
excluded = {"manifest.json"} | {ref for ref, _ in chain}
prior_refs = []
for _, checkpoint in reversed(chain):
details = checkpoint["details"]
if details.get("operation") == "case-built" and details.get("builder_version") == "1":
for field in ("case_ref", "summary_ref"):
if not isinstance(details.get(field), str):
raise CaseBuildError("PREVIOUS_CASE", "Incomplete case completion checkpoint")
require_file(reader, details[field])
excluded.add(details[field])
if details.get("case_ref") not in prior_refs:
prior_refs.append(details["case_ref"])
prior_ref = manifest["artifacts"].get("case")
if prior_ref:
excluded.add(prior_ref)
if prior_ref not in prior_refs:
prior_refs.append(prior_ref)
summary_ref = manifest["artifacts"].get("case_summary")
if summary_ref:
excluded.add(summary_ref)
previous = load(prior_ref, "case") if prior_ref else None
if previous and (previous["case"]["case_id"] != manifest["case_id"] or previous["case"]["package_id"] != manifest["package_id"] or previous["case"]["analysis_revision"] > manifest["analysis_revision"]):
raise CaseBuildError("CASE_IDENTITY", "Previous case has inconsistent identity or revision", prior_ref)
inventory = {}
for name in sorted(reader.files):
if name == "manifest.json" or name in {ref for ref, _ in chain}:
continue
digest = hashlib.sha256()
size = 0
with reader._archive.open(name) as stream:
while chunk := stream.read(CHUNK_SIZE):
size += len(chunk)
if size > reader.validator.limits.max_file_bytes:
raise CaseBuildError("INPUT_LIMIT", "Payload exceeds configured file limit", name)
digest.update(chunk)
inventory[name] = {"path": name, "sha256": digest.hexdigest(), "size": size}
index_ref = manifest["artifacts"].get("case_inputs")
index = load(index_ref, "case-inputs") if index_ref else {}
roles = []
def add(path, role):
require_file(reader, path)
if path in excluded:
raise CaseBuildError("INPUT_REFERENCE", "Generated case or checkpoint cannot serve as raw input", path)
if (path, role) not in roles:
roles.append((path, role))
if previous:
for record in previous["inputs"]["records"]:
add(record["path"], record["role"])
actual = inventory[record["path"]]
if record["sha256"] != actual["sha256"] or record["size"] != actual["size"]:
raise CaseBuildError("PREVIOUS_INPUT_CHANGED", "A source referenced by the prior case changed", record["path"])
if index_ref:
add(index_ref, "index")
for field, role in (("reports", "report"), ("reproductions", "reproduction"), ("attachments", "attachment")):
paths = index.get(field, [])
if len(paths) != len(set(paths)):
raise CaseBuildError("INPUT_DUPLICATE", "A source is listed twice in the same role", index_ref)
for path in paths:
add(path, role)
for role, paths in index.get("context", {}).items():
if role not in {"runtime", "database", "source", "research"}:
continue
if len(paths) != len(set(paths)):
raise CaseBuildError("INPUT_DUPLICATE", "A context source is listed twice", index_ref)
for path in paths:
add(path, role)
reports = [(path, load(path, "report")) for path, role in roles if role == "report"]
reproductions = [(path, load(path, "reproduction")) for path, role in roles if role == "reproduction"]
for _, document in reproductions:
for path in document.get("attachments", []):
add(path, "attachment")
if document.get("events_ref"):
add(document["events_ref"], "attachment")
observations = collect_statements(reports + reproductions, "observations", "O")
comments = collect_statements(reports, "comments", "C")
hypotheses = collect_statements(reports, "tester_hypotheses", "T")
expected = collect_expected_behavior(reports)
reproduction_records, facts = collect_reproductions(reproductions, self.limits.max_inline_fact_bytes)
if previous:
old_humans = previous["human_observations"]
observations = _stable_records(old_humans["observations"], observations, "O")
comments = _stable_records(old_humans["comments"], comments, "C")
hypotheses = _stable_records(old_humans["tester_hypotheses"], hypotheses, "T")
expected["statements"] = _stable_records(previous["expected_behavior"]["statements"], expected["statements"], "E")
facts = _stable_records(previous["runtime_facts"]["facts"], facts, "F")
entities = {}
for path, document in reports + reproductions:
for index, entity in enumerate(document.get("entities", [])):
entity_id = entity["id"] if isinstance(entity["id"], str) else str(int(entity["id"]))
identity = (entity["type"], entity_id)
record = entities.setdefault(identity, {"type": identity[0], "id": identity[1], "roles": [], "provenance": []})
record["roles"] = sorted(set(record["roles"]) | set(entity.get("roles", [])))
record["provenance"].append(provenance(path, f"/entities/{index}", "report" if document["format"] == "viano-bug-report" else "reproduction_record"))
issues = []
def issue(code, message, refs=None):
issues.append({"code": code, "message": message, "refs": refs or []})
if not index_ref:
issue("MISSING_INPUT_INDEX", "No explicit case input index was supplied")
if not reports:
issue("MISSING_REPORT", "No structured report was supplied")
if not observations:
issue("MISSING_OBSERVATIONS", "No direct observations were supplied")
if not reproductions:
issue("MISSING_REPRODUCTION", "No structured reproduction was supplied")
if expected["completeness"] != "reported_complete":
issue("EXPECTED_BEHAVIOR_" + expected["completeness"].upper(), "Expected-behavior completeness is " + expected["completeness"],
[{"source_ref": path, "json_pointer": "/expected_behavior_complete"} for path, report in reports if "expected_behavior_complete" in report])
for category, records in (("observation", observations), ("expected_behavior", expected["statements"])):
claims = {}
for record in records:
if "claim" in record:
claim = record["claim"]
claims.setdefault(claim["key"], []).append(record)
for claim_key, records in sorted(claims.items()):
if len({json_bytes(record["claim"]["value"]) for record in records}) > 1:
refs = [{"source_ref": r["provenance"]["source_ref"], "json_pointer": r["provenance"]["json_pointer"]} for r in records]
issue("CONFLICTING_CLAIMS", f"Different explicit values for {category} claim {claim_key}", refs)
if category == "expected_behavior":
expected["completeness"] = "conflicting"
records = [inventory[path] | {"role": role} for path, role in roles]
classified = {path for path, _ in roles}
raw_inventory = [value for name, value in inventory.items() if name not in excluded]
request = {"builder_version": "1", "case_id": manifest["case_id"], "package_id": manifest["package_id"],
"analysis_revision": int(manifest["analysis_revision"]), "project": manifest["project"],
"index_ref": index_ref, "payloads": raw_inventory, "roles": roles}
request_hash = hashlib.sha256(json_bytes(request)).hexdigest()
refs_for = lambda role: [inventory[path] for path, value in roles if value == role]
case = {
"format": "viano-case", "format_version": 1, "builder_version": "1",
"case": {"case_id": manifest["case_id"], "package_id": manifest["package_id"], "analysis_revision": int(manifest["analysis_revision"]),
"source_generation": int(manifest["package_generation"]), "project": copy.deepcopy(manifest["project"])},
"bug": {"reports": [{"source_ref": path, "title": document.get("title"), "summary": document.get("summary")} for path, document in reports]},
"human_observations": {"observations": observations, "comments": comments, "tester_hypotheses": hypotheses},
"expected_behavior": expected,
"reproduction": {"records": reproduction_records},
"entities": [entities[identity] for identity in sorted(entities)],
"runtime_facts": {"facts": facts, "raw_refs": refs_for("runtime")},
"database_context": {"refs": refs_for("database")}, "source_context": {"refs": refs_for("source")},
"research": {"refs": refs_for("research"), "performed": False},
"data_quality": {"issues": issues, "coverage": {"reports": len(reports), "reproductions": len(reproductions), "observations": len(observations)}},
"inputs": {"index_ref": index_ref, "records": records,
"unclassified": [value for name, value in inventory.items() if name not in classified and name not in excluded],
"prior_cases": [inventory[ref] for ref in prior_refs], "request_hash": request_hash},
}
validate(case, "case")
if len(json_bytes(case)) > self.limits.max_case_bytes:
raise CaseBuildError("CASE_LIMIT", "Assembled case exceeds configured byte limit")
return case
+91
View File
@@ -0,0 +1,91 @@
from dataclasses import dataclass
from pathlib import Path
from viano_bug_analysis.lifecycle import LifecycleError, State
from viano_bug_analysis.lifecycle.checkpoints import checkpoint_chain
from viano_bug_analysis.lifecycle.storage import json_bytes
from viano_bug_analysis.package import PackageReader, PackageReadError
from viano_bug_analysis.package.validator import ARCHIVE_ERRORS
from .assembler import CaseAssembler, CaseBuilderLimits
from .contracts import read_json
from .errors import CaseBuildError
from .summary import render_summary
@dataclass(frozen=True)
class CaseBuildResult:
package: Path
case_ref: str
summary_ref: str
analysis_revision: int
reused: bool
class CaseBuilder:
def __init__(self, store, limits: CaseBuilderLimits | None = None):
self.store = store
self.assembler = CaseAssembler(limits)
def build(self, case_id: str, package_id: str) -> CaseBuildResult:
try:
latest = self.store.latest(case_id, package_id)
if latest is None:
raise CaseBuildError("CASE_NOT_FOUND", "Case must be admitted before construction")
with PackageReader(latest, self.store.limits) as reader:
manifest = reader.manifest
revision = int(manifest["analysis_revision"])
revision = revision or 1
complete_id = f"case-builder:v1:revision:{revision}:complete"
completed = next((cp for _, cp in checkpoint_chain(reader) if cp["step_id"] == complete_id), None)
if completed:
assembled = self.assembler.assemble(reader)
details = completed["details"]
if details.get("input_request_hash") != assembled["inputs"]["request_hash"]:
raise CaseBuildError("STALE_CASE", "Inputs changed after case completion; reopen as a new analysis revision")
case_ref, summary_ref = details.get("case_ref"), details.get("summary_ref")
stored = read_json(reader, case_ref, "case", self.assembler.limits.max_case_bytes)
summary = render_summary(stored).encode("utf-8")
if len(summary) > self.assembler.limits.max_summary_bytes:
raise CaseBuildError("SUMMARY_LIMIT", "Summary exceeds configured byte limit")
with reader._archive.open(summary_ref) as stream:
stored_summary = reader.validator._read_bounded(stream, min(self.assembler.limits.max_summary_bytes, self.store.limits.max_file_bytes))
if reader.manifest["artifacts"].get("case") != case_ref or reader.manifest["artifacts"].get("case_summary") != summary_ref or stored_summary != summary:
raise CaseBuildError("CASE_OUTPUT_CHANGED", "Confirmed case pointers or summary no longer match")
# Repair reconstructible metadata after a crash, without recomputing a commit.
recovered = self.store.advance(case_id, package_id, State.BUILDING_CASE, step_id=complete_id,
details=details, additions={case_ref: json_bytes(stored), summary_ref: summary},
artifacts={"case": case_ref, "case_summary": summary_ref})
if recovered != latest:
raise CaseBuildError("GENERATION_CHANGED", "Case changed while confirmed output was checked")
return CaseBuildResult(latest, case_ref, summary_ref, revision, True)
if manifest["state"] not in {State.VALIDATED, State.REOPENED, State.BUILDING_CASE}:
raise CaseBuildError("INVALID_BUILD_STATE", "CaseBuilder requires VALIDATED, REOPENED or BUILDING_CASE")
source_generation = int(manifest["package_generation"])
if manifest["state"] != State.BUILDING_CASE:
latest = self.store.advance(case_id, package_id, State.BUILDING_CASE,
step_id=f"case-builder:v1:revision:{revision}:start",
details={"operation": "case-building", "builder_version": "1"},
expected_generation=source_generation)
with PackageReader(latest, self.store.limits) as reader:
if any(cp["step_id"] == complete_id for _, cp in checkpoint_chain(reader)):
return self.build(case_id, package_id)
case = self.assembler.assemble(reader)
revision = case["case"]["analysis_revision"]
case_ref = f"case/revisions/{revision:04d}/case.json"
summary_ref = f"case/revisions/{revision:04d}/case_summary.md"
summary = render_summary(case).encode("utf-8")
if len(summary) > self.assembler.limits.max_summary_bytes:
raise CaseBuildError("SUMMARY_LIMIT", "Summary exceeds configured byte limit")
details = {"operation": "case-built", "builder_version": "1", "case_ref": case_ref, "summary_ref": summary_ref,
"input_request_hash": case["inputs"]["request_hash"]}
output = self.store.advance(case_id, package_id, State.BUILDING_CASE,
step_id=f"case-builder:v1:revision:{revision}:complete", details=details,
additions={case_ref: json_bytes(case), summary_ref: summary},
artifacts={"case": case_ref, "case_summary": summary_ref},
expected_generation=int(reader.manifest["package_generation"]))
return CaseBuildResult(output, case_ref, summary_ref, revision, False)
except (PackageReadError, LifecycleError) as error:
raise CaseBuildError(getattr(error, "code", "PACKAGE_ERROR"), str(error)) from error
except ARCHIVE_ERRORS + (TypeError, KeyError, RecursionError) as error:
raise CaseBuildError("CASE_BUILD_ERROR", str(error)) from error
+35
View File
@@ -0,0 +1,35 @@
import json
from importlib.resources import files
from jsonschema import Draft202012Validator
from jsonschema.exceptions import ValidationError
from viano_bug_analysis.package import PackageReadError
from viano_bug_analysis.package.manifest import FORMAT_CHECKER, parse_manifest, safe_path
from .errors import CaseBuildError
def validate(document: dict, contract: str, path: str | None = None) -> None:
schema = json.loads(files("viano_bug_analysis.schemas").joinpath(f"{contract}-v1.schema.json").read_text())
try:
Draft202012Validator(schema, format_checker=FORMAT_CHECKER).validate(document)
except ValidationError as error:
raise CaseBuildError("INPUT_SCHEMA" if contract != "case" else "CASE_SCHEMA", error.message, path) from error
def read_json(reader, path: str, contract: str, limit: int) -> dict:
require_file(reader, path)
try:
with reader._archive.open(path) as stream:
data = reader.validator._read_bounded(stream, min(limit, reader.validator.limits.max_file_bytes))
document = parse_manifest(data)
validate(document, contract, path)
return document
except (ValueError, RecursionError, PackageReadError) as error:
raise CaseBuildError("INPUT_JSON", str(error), path) from error
def require_file(reader, path: str) -> None:
if not safe_path(path) or path == "manifest.json" or path not in reader.files:
raise CaseBuildError("INPUT_REFERENCE", "Expected a safe existing payload file", path)
+7
View File
@@ -0,0 +1,7 @@
class CaseBuildError(Exception):
"""Controlled failure of a declared input or case construction request."""
def __init__(self, code: str, message: str, path: str | None = None):
self.code = code
self.path = path
super().__init__(message)
@@ -0,0 +1,19 @@
from .report_collector import collect_statements
def collect_expected_behavior(reports: list[tuple[str, dict]]) -> dict:
statements = collect_statements(reports, "expected_behavior", "E")
flags = {document.get("expected_behavior_complete") for _, document in reports
if document.get("expected_behavior")}
if not statements:
status = "missing"
elif True in flags and False in flags:
status = "conflicting"
elif False in flags:
status = "reported_partial"
elif flags == {True}:
status = "reported_complete"
else:
status = "unspecified"
return {"statements": statements, "completeness": status,
"provenance": "tester" if statements else "none"}
@@ -0,0 +1,19 @@
import copy
def provenance(path: str, pointer: str, kind: str, statement: dict | None = None) -> dict:
value = statement or {}
return {"kind": kind, "source_ref": path, "json_pointer": pointer,
"author": value.get("author"), "observed_at": value.get("observed_at")}
def collect_statements(documents: list[tuple[str, dict]], field: str, prefix: str) -> list[dict]:
result = []
for path, document in documents:
for index, statement in enumerate(document.get(field, [])):
record = {"id": f"{prefix}{len(result) + 1:04d}", "text": statement["text"],
"provenance": provenance(path, f"/{field}/{index}", "tester", statement)}
if "claim" in statement:
record["claim"] = copy.deepcopy(statement["claim"])
result.append(record)
return result
@@ -0,0 +1,22 @@
import copy
from viano_bug_analysis.lifecycle.storage import json_bytes
from .report_collector import provenance
def collect_reproductions(documents: list[tuple[str, dict]], inline_limit: int) -> tuple[list[dict], list[dict]]:
records, facts = [], []
for path, document in documents:
records.append({"id": f"R{len(records) + 1:04d}", "source_ref": path,
"steps": list(document.get("steps", [])), "events_ref": document.get("events_ref"),
"attachments": list(document.get("attachments", []))})
for index, fact in enumerate(document.get("facts", [])):
record = {"id": f"F{len(facts) + 1:04d}", "name": fact["name"],
"provenance": provenance(path, f"/facts/{index}", "reproduction_record")}
if len(json_bytes(fact["value"])) > inline_limit:
record["value_ref"] = {"source_ref": path, "json_pointer": f"/facts/{index}/value"}
else:
record["value"] = copy.deepcopy(fact["value"])
facts.append(record)
return records, facts
+76
View File
@@ -0,0 +1,76 @@
import html
import json
import re
from .contracts import validate
def _literal(value) -> str:
text = html.escape(str(value), quote=False)
text = re.sub(r"([\\`*_{}\[\]()#+.!|>~-])", r"\\\1", text)
return text.replace("\r", "\\r").replace("\n", "<br>")
def render_summary(case: dict) -> str:
"""Pure presentation of the canonical case, with no additional interpretation."""
validate(case, "case")
identity = case["case"]
lines = ["# Case summary", "", f"Case: {_literal(identity['case_id'])}",
f"Package: {_literal(identity['package_id'])}", f"Analysis revision: {identity['analysis_revision']}",
"", "## Reports", ""]
for report in case["bug"]["reports"]:
lines += [f"- Source: {_literal(report['source_ref'])}",
f" Title: {_literal(report['title']) if report['title'] is not None else '(not supplied)'}",
f" Summary: {_literal(report['summary']) if report['summary'] is not None else '(not supplied)'}"]
for label, records in (("Direct human observations", case["human_observations"]["observations"]),
("Tester comments", case["human_observations"]["comments"]),
("Tester hypotheses (supplied)", case["human_observations"]["tester_hypotheses"]),
("Expected behavior (supplied)", case["expected_behavior"]["statements"])):
lines += ["", f"## {label}", ""]
for record in records:
source = record["provenance"]
lines += [f"- {_literal(record['id'])}: {_literal(record['text'])}",
f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}",
f" Author: {_literal(source['author']) if source['author'] is not None else '(not supplied)'}",
f" Observed at: {_literal(source['observed_at']) if source['observed_at'] is not None else '(not supplied)'}"]
if "claim" in record:
lines.append(f" Claim: {_literal(record['claim']['key'])} = {_literal(json.dumps(record['claim']['value'], sort_keys=True, ensure_ascii=False))}")
lines += ["", f"Expected-behavior completeness: {case['expected_behavior']['completeness']}", "", "## Reproductions", ""]
for record in case["reproduction"]["records"]:
lines.append(f"- {_literal(record['id'])}: {_literal(record['source_ref'])}")
for step in record["steps"]:
lines.append(f" Step: {_literal(step)}")
if record["events_ref"]:
lines.append(f" Events: {_literal(record['events_ref'])}")
for path in record["attachments"]:
lines.append(f" Attachment: {_literal(path)}")
lines += ["", "## Entities", ""]
for entity in case["entities"]:
lines.append(f"- {_literal(entity['type'])} / {_literal(entity['id'])}; roles: {_literal(', '.join(entity['roles']))}")
for source in entity["provenance"]:
lines.append(f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}")
lines += ["", "## Facts supplied by reproduction records", ""]
for fact in case["runtime_facts"]["facts"]:
source = fact["provenance"]
if "value_ref" in fact:
ref = fact["value_ref"]
value = f"referenced at {_literal(ref['source_ref'])} {_literal(ref['json_pointer'])}"
else:
value = _literal(json.dumps(fact['value'], sort_keys=True, ensure_ascii=False))
lines.append(f"- {_literal(fact['id'])}: {_literal(fact['name'])} = {value}")
lines.append(f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}")
for label, refs in (("Runtime data", case["runtime_facts"]["raw_refs"]), ("Database context", case["database_context"]["refs"]),
("Source context", case["source_context"]["refs"]), ("Supplied research", case["research"]["refs"]),
("Prior cases", case["inputs"]["prior_cases"]), ("Unclassified inputs", case["inputs"]["unclassified"])):
lines += ["", f"## {label}", ""]
for ref in refs:
lines.append(f"- {_literal(ref['path'])} ({ref['size']} bytes; SHA-256: {ref['sha256']})")
lines += ["", "## Data quality", ""]
for issue in case["data_quality"]["issues"]:
lines.append(f"- {_literal(issue['code'])}: {_literal(issue['message'])}")
for source in issue["refs"]:
lines.append(f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}")
lines += ["", "## Input records", ""]
for record in case["inputs"]["records"]:
lines.append(f"- {_literal(record['role'])}: {_literal(record['path'])} ({record['size']} bytes; SHA-256: {record['sha256']})")
return "\n".join(lines) + "\n"
@@ -234,7 +234,8 @@ class CheckpointStore:
def advance(self, case_id: str, package_id: str, target: str, *, step_id: str,
reason: str | None = None, retry_not_before: str | None = None,
details: dict | None = None, additions: dict[str, bytes] | None = None,
artifacts: dict[str, str | None] | None = None) -> Path:
artifacts: dict[str, str | None] | None = None,
expected_generation: int | None = None) -> Path:
try:
if not isinstance(step_id, str) or not step_id.strip():
raise LifecycleError("INVALID_STEP", "A nonblank stable step_id is required")
@@ -242,6 +243,8 @@ class CheckpointStore:
raise LifecycleError("CHECKPOINT_INVALID", "Checkpoint details must be a JSON object")
if CHECKPOINT_POINTER in (artifacts or {}):
raise LifecycleError("RESERVED_ARTIFACT", "Lifecycle checkpoint pointer is service-managed")
if expected_generation is not None and (isinstance(expected_generation, bool) or not isinstance(expected_generation, int) or expected_generation < 0):
raise LifecycleError("INVALID_GENERATION", "expected_generation must be a nonnegative integer")
payload = {"target": target, "reason": reason, "retry_not_before": retry_not_before,
"details": details or {}, "artifacts": artifacts or {},
"additions": {name: hashlib.sha256(data).hexdigest() for name, data in (additions or {}).items()}}
@@ -260,6 +263,8 @@ class CheckpointStore:
self._index(latest, manifest)
atomic_json(self.workspace.runtime / "current_job.json", None)
return latest
if expected_generation is not None and manifest["package_generation"] != expected_generation:
raise LifecycleError("GENERATION_CHANGED", "Case changed while artifacts were being prepared")
resume = chain[0][1]["resume_state"] if manifest["state"] == State.BLOCKED else None
check_transition(manifest["state"], target, resume)
if manifest["state"] == State.BLOCKED and not self.retry_due(manifest):
@@ -0,0 +1,72 @@
{
"type": "object",
"required": [
"format",
"format_version"
],
"properties": {
"format": {
"const": "viano-case-inputs"
},
"format_version": {
"type": "integer",
"const": 1
},
"reports": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
}
},
"reproductions": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
}
},
"context": {
"type": "object",
"required": [],
"properties": {
"runtime": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
}
},
"database": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
}
},
"source": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
}
},
"research": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
}
}
}
},
"attachments": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
}
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
}
@@ -0,0 +1,657 @@
{
"type": "object",
"required": [
"format",
"format_version",
"builder_version",
"case",
"bug",
"human_observations",
"expected_behavior",
"reproduction",
"entities",
"runtime_facts",
"database_context",
"source_context",
"research",
"data_quality",
"inputs"
],
"properties": {
"format": {
"const": "viano-case"
},
"format_version": {
"type": "integer",
"const": 1
},
"builder_version": {
"const": "1"
},
"case": {
"type": "object",
"required": [
"case_id",
"package_id",
"analysis_revision",
"source_generation",
"project"
],
"properties": {
"case_id": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"package_id": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"analysis_revision": {
"type": "integer",
"minimum": 1
},
"source_generation": {
"type": "integer",
"minimum": 0
},
"project": {
"type": "object"
}
}
},
"bug": {
"type": "object",
"required": [
"reports"
],
"properties": {
"reports": {
"type": "array",
"items": {
"type": "object",
"required": [
"source_ref",
"title",
"summary"
],
"properties": {
"source_ref": {
"type": "string",
"minLength": 1
},
"title": {
"anyOf": [
{
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
{
"type": "null"
}
]
},
"summary": {
"anyOf": [
{
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
{
"type": "null"
}
]
}
}
}
}
}
},
"human_observations": {
"type": "object",
"required": [
"observations",
"comments",
"tester_hypotheses"
],
"properties": {
"observations": {
"type": "array",
"items": {
"$ref": "#/$defs/record"
}
},
"comments": {
"type": "array",
"items": {
"$ref": "#/$defs/record"
}
},
"tester_hypotheses": {
"type": "array",
"items": {
"$ref": "#/$defs/record"
}
}
}
},
"expected_behavior": {
"type": "object",
"required": [
"statements",
"completeness",
"provenance"
],
"properties": {
"statements": {
"type": "array",
"items": {
"$ref": "#/$defs/record"
}
},
"completeness": {
"enum": [
"missing",
"unspecified",
"reported_complete",
"reported_partial",
"conflicting"
]
},
"provenance": {
"enum": [
"tester",
"none"
]
}
}
},
"reproduction": {
"type": "object",
"required": [
"records"
],
"properties": {
"records": {
"type": "array",
"items": {
"type": "object",
"required": [
"id",
"source_ref",
"steps",
"events_ref",
"attachments"
],
"properties": {
"id": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"source_ref": {
"type": "string",
"minLength": 1
},
"steps": {
"type": "array",
"items": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
},
"events_ref": {
"anyOf": [
{
"type": "string",
"minLength": 1
},
{
"type": "null"
}
]
},
"attachments": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
}
}
}
}
}
}
},
"entities": {
"type": "array",
"items": {
"type": "object",
"required": [
"type",
"id",
"roles",
"provenance"
],
"properties": {
"type": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"id": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"roles": {
"type": "array",
"items": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
},
"provenance": {
"type": "array",
"items": {
"$ref": "#/$defs/provenance"
}
}
}
}
},
"runtime_facts": {
"type": "object",
"required": [
"facts",
"raw_refs"
],
"properties": {
"facts": {
"type": "array",
"items": {
"type": "object",
"required": [
"id",
"name",
"provenance"
],
"oneOf": [
{
"required": [
"value"
],
"not": {
"required": [
"value_ref"
]
}
},
{
"required": [
"value_ref"
],
"not": {
"required": [
"value"
]
}
}
],
"properties": {
"id": {
"type": "string",
"pattern": "^F[0-9]{4,}$"
},
"name": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"value": {},
"value_ref": {
"type": "object",
"required": [
"source_ref",
"json_pointer"
],
"properties": {
"source_ref": {
"type": "string",
"minLength": 1
},
"json_pointer": {
"type": "string",
"pattern": "^/"
}
}
},
"provenance": {
"$ref": "#/$defs/provenance"
}
}
}
},
"raw_refs": {
"type": "array",
"items": {
"$ref": "#/$defs/raw"
}
}
}
},
"database_context": {
"type": "object",
"required": [
"refs"
],
"properties": {
"refs": {
"type": "array",
"items": {
"$ref": "#/$defs/raw"
}
}
}
},
"source_context": {
"type": "object",
"required": [
"refs"
],
"properties": {
"refs": {
"type": "array",
"items": {
"$ref": "#/$defs/raw"
}
}
}
},
"research": {
"type": "object",
"required": [
"refs",
"performed"
],
"properties": {
"refs": {
"type": "array",
"items": {
"$ref": "#/$defs/raw"
}
},
"performed": {
"const": false
}
}
},
"data_quality": {
"type": "object",
"required": [
"issues",
"coverage"
],
"properties": {
"issues": {
"type": "array",
"items": {
"type": "object",
"required": [
"code",
"message",
"refs"
],
"properties": {
"code": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"message": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"refs": {
"type": "array",
"items": {
"type": "object",
"required": [
"source_ref",
"json_pointer"
],
"properties": {
"source_ref": {
"type": "string",
"minLength": 1
},
"json_pointer": {
"type": "string"
}
}
}
}
}
}
},
"coverage": {
"type": "object",
"required": [
"reports",
"reproductions",
"observations"
],
"properties": {
"reports": {
"type": "integer",
"minimum": 0
},
"reproductions": {
"type": "integer",
"minimum": 0
},
"observations": {
"type": "integer",
"minimum": 0
}
}
}
}
},
"inputs": {
"type": "object",
"required": [
"index_ref",
"records",
"unclassified",
"prior_cases",
"request_hash"
],
"properties": {
"index_ref": {
"anyOf": [
{
"type": "string",
"minLength": 1
},
{
"type": "null"
}
]
},
"records": {
"type": "array",
"items": {
"type": "object",
"required": [
"path",
"sha256",
"size",
"role"
],
"properties": {
"path": {
"type": "string",
"minLength": 1
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"size": {
"type": "integer",
"minimum": 0
},
"role": {
"enum": [
"index",
"report",
"reproduction",
"runtime",
"database",
"source",
"research",
"attachment"
]
}
}
}
},
"unclassified": {
"type": "array",
"items": {
"$ref": "#/$defs/raw"
}
},
"prior_cases": {
"type": "array",
"items": {
"$ref": "#/$defs/raw"
}
},
"request_hash": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
}
}
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$defs": {
"provenance": {
"type": "object",
"required": [
"kind",
"source_ref",
"json_pointer",
"author",
"observed_at"
],
"properties": {
"kind": {
"enum": [
"tester",
"reproduction_record",
"report"
]
},
"source_ref": {
"type": "string",
"minLength": 1
},
"json_pointer": {
"type": "string",
"pattern": "^(|/.*)$"
},
"author": {
"anyOf": [
{
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
{
"type": "null"
}
]
},
"observed_at": {
"type": [
"string",
"null"
],
"format": "date-time"
}
}
},
"claim": {
"type": "object",
"required": [
"key",
"value"
],
"properties": {
"key": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"value": {}
}
},
"raw": {
"type": "object",
"required": [
"path",
"sha256",
"size"
],
"properties": {
"path": {
"type": "string",
"minLength": 1
},
"sha256": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"size": {
"type": "integer",
"minimum": 0
}
}
},
"record": {
"type": "object",
"required": [
"id",
"text",
"provenance"
],
"properties": {
"id": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"text": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"provenance": {
"$ref": "#/$defs/provenance"
},
"claim": {
"$ref": "#/$defs/claim"
}
}
}
}
}
@@ -0,0 +1,149 @@
{
"type": "object",
"required": [
"format",
"format_version"
],
"properties": {
"format": {
"const": "viano-bug-report"
},
"format_version": {
"type": "integer",
"const": 1
},
"title": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"summary": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"observations": {
"type": "array",
"items": {
"$ref": "#/$defs/statement"
}
},
"comments": {
"type": "array",
"items": {
"$ref": "#/$defs/statement"
}
},
"tester_hypotheses": {
"type": "array",
"items": {
"$ref": "#/$defs/statement"
}
},
"expected_behavior": {
"type": "array",
"items": {
"$ref": "#/$defs/statement"
}
},
"expected_behavior_complete": {
"type": [
"boolean",
"null"
]
},
"entities": {
"type": "array",
"items": {
"$ref": "#/$defs/entity"
}
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$defs": {
"claim": {
"type": "object",
"required": [
"key",
"value"
],
"properties": {
"key": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"value": {}
}
},
"statement": {
"type": "object",
"required": [
"text"
],
"properties": {
"text": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"author": {
"anyOf": [
{
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
{
"type": "null"
}
]
},
"observed_at": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"claim": {
"$ref": "#/$defs/claim"
}
}
},
"entity": {
"type": "object",
"required": [
"type",
"id"
],
"properties": {
"type": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"id": {
"anyOf": [
{
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
{
"type": "integer"
}
]
},
"roles": {
"type": "array",
"items": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
}
}
}
}
}
@@ -0,0 +1,159 @@
{
"type": "object",
"required": [
"format",
"format_version"
],
"properties": {
"format": {
"const": "viano-reproduction"
},
"format_version": {
"type": "integer",
"const": 1
},
"steps": {
"type": "array",
"items": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
},
"observations": {
"type": "array",
"items": {
"$ref": "#/$defs/statement"
}
},
"events_ref": {
"anyOf": [
{
"type": "string",
"minLength": 1
},
{
"type": "null"
}
]
},
"attachments": {
"type": "array",
"items": {
"type": "string",
"minLength": 1
}
},
"facts": {
"type": "array",
"items": {
"type": "object",
"required": [
"name",
"value"
],
"properties": {
"name": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"value": {}
}
}
},
"entities": {
"type": "array",
"items": {
"$ref": "#/$defs/entity"
}
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$defs": {
"claim": {
"type": "object",
"required": [
"key",
"value"
],
"properties": {
"key": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"value": {}
}
},
"statement": {
"type": "object",
"required": [
"text"
],
"properties": {
"text": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"author": {
"anyOf": [
{
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
{
"type": "null"
}
]
},
"observed_at": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"claim": {
"$ref": "#/$defs/claim"
}
}
},
"entity": {
"type": "object",
"required": [
"type",
"id"
],
"properties": {
"type": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"id": {
"anyOf": [
{
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
{
"type": "integer"
}
]
},
"roles": {
"type": "array",
"items": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
}
}
}
}
}
+384
View File
@@ -0,0 +1,384 @@
import json
import pytest
from jsonschema import Draft202012Validator
from importlib.resources import files
from viano_bug_analysis.case import CaseBuilder, CaseAssembler, CaseBuilderLimits, CaseBuildError, render_summary
from viano_bug_analysis.lifecycle import CheckpointStore, LifecycleError, State, Workspace
from viano_bug_analysis.package import PackageReader, PackageValidator, PackageWriter
@pytest.fixture
def case_setup(manifest, make_package, tmp_path):
manifest["artifacts"]["case_inputs"] = "arbitrary/index.data"
report = {"format": "viano-bug-report", "format_version": 1, "title": "Generic problem", "summary": "Reported behavior",
"observations": [{"text": "The switch remains off", "author": "Tester A", "observed_at": "2026-01-01T01:00:00Z"}],
"comments": [{"text": "Seen twice"}], "tester_hypotheses": [{"text": "Perhaps a timing issue"}],
"expected_behavior": [{"text": "The switch turns on"}], "expected_behavior_complete": True,
"entities": [{"type": "component", "id": 42, "roles": ["focus"]}]}
reproduction = {"format": "viano-reproduction", "format_version": 1, "steps": ["Start", "Press switch"],
"observations": [{"text": "Still off"}], "events_ref": "raw/events", "attachments": ["raw/binary"],
"facts": [{"name": "attempt", "value": 2}], "entities": [{"type": "component", "id": "42", "roles": ["observed"]}]}
index = {"format": "viano-case-inputs", "format_version": 1, "reports": ["anything/report.data"],
"reproductions": ["anything/repro.data"], "context": {"source": ["raw/source"], "database": ["raw/db"], "runtime": ["raw/runtime"], "research": ["raw/research"]}}
entries = {"arbitrary/index.data": json.dumps(index), "anything/report.data": json.dumps(report), "anything/repro.data": json.dumps(reproduction),
"raw/events": b"many events", "raw/binary": b"\x00\xff", "raw/source": b"source raw", "raw/db": b"db raw", "raw/runtime": b"runtime raw", "raw/research": b"supplied external notes", "unknown/vendor": b"opaque"}
source = make_package(entries=entries)
store = CheckpointStore(Workspace(tmp_path / "service"))
store.intake(source)
return store, CaseBuilder(store), source, entries, report, index
@pytest.mark.parametrize("contract", ["case", "case-inputs", "report", "reproduction"])
def test_contract_schemas_valid(contract):
schema = json.loads(files("viano_bug_analysis.schemas").joinpath(f"{contract}-v1.schema.json").read_text())
Draft202012Validator.check_schema(schema)
def test_full_build_preserves_separation_and_provenance(case_setup):
store, builder, source, entries, _, _ = case_setup
original = source.read_bytes()
result = builder.build("case-A", "stable-package")
assert not result.reused
assert result.analysis_revision == 1
assert PackageValidator().validate(result.package).valid
assert source.read_bytes() == original
with PackageReader(result.package) as reader:
manifest = reader.manifest
assert manifest["state"] == State.BUILDING_CASE
assert manifest["package_generation"] == 3
case = json.loads(reader.read_bytes(result.case_ref))
assert case["case"]["source_generation"] == 2
assert case["human_observations"]["observations"][0]["text"] == "The switch remains off"
assert len(case["human_observations"]["observations"]) == 2
assert case["human_observations"]["tester_hypotheses"][0]["text"] == "Perhaps a timing issue"
assert case["expected_behavior"]["completeness"] == "reported_complete"
assert case["human_observations"]["observations"][0]["provenance"] == {"kind": "tester", "source_ref": "anything/report.data", "json_pointer": "/observations/0", "author": "Tester A", "observed_at": "2026-01-01T01:00:00Z"}
assert case["runtime_facts"]["facts"][0]["id"] == "F0001"
assert case["runtime_facts"]["facts"][0]["provenance"]["json_pointer"] == "/facts/0"
assert case["entities"][0]["id"] == "42"
assert case["entities"][0]["roles"] == ["focus", "observed"]
assert len(case["entities"][0]["provenance"]) == 2
assert case["research"]["performed"] is False
assert [item["path"] for item in case["inputs"]["unclassified"]] == ["unknown/vendor"]
assert reader.read_bytes(result.summary_ref).decode() == render_summary(case)
for path, value in entries.items():
assert reader.read_bytes(path) == (value.encode() if isinstance(value, str) else value)
def test_completed_build_is_idempotent(case_setup):
store, builder, _, _, _, _ = case_setup
first = builder.build("case-A", "stable-package")
again = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package")
assert again.reused and again.package == first.package
assert len(list(first.package.parent.glob("g*.zip"))) == 3
def test_missing_index_marks_gaps_without_guessing(make_package, tmp_path):
store = CheckpointStore(Workspace(tmp_path / "service"))
store.intake(make_package(entries={"original/bug_report.json": b"not a declared report"}))
result = CaseBuilder(store).build("case-A", "stable-package")
with PackageReader(result.package) as reader:
case = json.loads(reader.read_bytes(result.case_ref))
codes = {issue["code"] for issue in case["data_quality"]["issues"]}
assert {"MISSING_INPUT_INDEX", "MISSING_REPORT", "MISSING_OBSERVATIONS", "MISSING_REPRODUCTION", "EXPECTED_BEHAVIOR_MISSING"} <= codes
assert case["bug"]["reports"] == []
assert case["inputs"]["unclassified"][0]["path"] == "original/bug_report.json"
def rewrite_input(case_setup, tmp_path, changes, updates=None):
store = case_setup[0]
current = store.latest("case-A", "stable-package")
incoming = tmp_path / "updated-input"
PackageWriter().write(current, incoming, additions={name: json.dumps(data).encode() for name, data in changes.items()}, manifest_updates=updates)
store.intake(incoming)
return current
def test_reopen_inherits_previous_reports_and_facts(case_setup, tmp_path):
store, builder, _, _, _, _ = case_setup
first = builder.build("case-A", "stable-package")
for index, target in enumerate([State.INTERPRETING_CASE, State.GENERATING_HYPOTHESES, State.INVESTIGATING_HYPOTHESES, State.BUILDING_DIAGNOSIS, State.OUTPUT_READY, State.HANDED_OFF, State.REOPENED]):
store.advance("case-A", "stable-package", target, step_id=f"test-stage-{index}")
report = {"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "New feedback"}]}
index = {"format": "viano-case-inputs", "format_version": 1, "reports": ["feedback/new-report"]}
rewrite_input(case_setup, tmp_path, {"feedback/new-report": report, "inputs/new-index": index}, {"artifacts": {"case_inputs": "inputs/new-index"}})
result = builder.build("case-A", "stable-package")
assert result.analysis_revision == 2
with PackageReader(result.package) as reader:
new = json.loads(reader.read_bytes(result.case_ref))
old = json.loads(reader.read_bytes(first.case_ref))
assert new["human_observations"]["observations"][:2] == old["human_observations"]["observations"]
assert new["human_observations"]["observations"][-1]["text"] == "New feedback"
assert new["runtime_facts"]["facts"] == old["runtime_facts"]["facts"]
assert new["inputs"]["prior_cases"][0]["path"] == first.case_ref
assert reader.read_bytes(first.summary_ref) == render_summary(old).encode()
def test_new_data_after_completion_requires_reopen(case_setup, tmp_path):
_, builder, _, _, _, _ = case_setup
builder.build("case-A", "stable-package")
rewrite_input(case_setup, tmp_path, {"feedback/new": {"anything": "new"}})
with pytest.raises(CaseBuildError) as error:
builder.build("case-A", "stable-package")
assert error.value.code == "STALE_CASE"
def test_completion_crash_reuses_output(case_setup, monkeypatch):
store, builder, _, _, _, _ = case_setup
original_index = store._index
calls = 0
def crash(*args):
nonlocal calls
calls += 1
if calls == 2:
raise LifecycleError("SIMULATED_CRASH", "after case publication")
original_index(*args)
monkeypatch.setattr(store, "_index", crash)
with pytest.raises(CaseBuildError):
builder.build("case-A", "stable-package")
result = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package")
assert result.reused
assert len(list(result.package.parent.glob("g*.zip"))) == 3
def test_unconfirmed_build_retries_without_duplicate_start(case_setup, monkeypatch):
store, builder, _, _, _, _ = case_setup
original_write = store.writer.write
calls = 0
def crash(*args, **kwargs):
nonlocal calls
calls += 1
if calls == 2:
raise LifecycleError("SIMULATED_CRASH", "before case publication")
return original_write(*args, **kwargs)
monkeypatch.setattr(store.writer, "write", crash)
with pytest.raises(CaseBuildError):
builder.build("case-A", "stable-package")
result = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package")
assert not result.reused
assert len(list(result.package.parent.glob("g*.zip"))) == 3
def test_portable_completed_case(case_setup, tmp_path):
_, builder, _, _, _, _ = case_setup
first = builder.build("case-A", "stable-package")
other = CheckpointStore(Workspace(tmp_path / "other-service"))
other.intake(first.package)
result = CaseBuilder(other).build("case-A", "stable-package")
assert result.reused
assert result.package == other.latest("case-A", "stable-package")
def test_summary_deterministic_and_escapes_untrusted_text(case_setup):
_, builder, _, _, _, _ = case_setup
result = builder.build("case-A", "stable-package")
with PackageReader(result.package) as reader:
case = json.loads(reader.read_bytes(result.case_ref))
case["human_observations"]["observations"][0]["text"] = "<script>alert(1)</script> [link](javascript:bad)\n# fake heading"
summary = render_summary(case)
assert summary == render_summary(case)
assert "<script>" not in summary
assert "[link](javascript:" not in summary
assert "\n# fake heading" not in summary
@pytest.mark.parametrize("limit,value,code", [("max_documents", 1, "STRUCTURED_INPUT_LIMIT"), ("max_structured_bytes", 50, "STRUCTURED_INPUT_LIMIT"), ("max_json_bytes", 10, "INPUT_JSON"), ("max_case_bytes", 50, "CASE_LIMIT"), ("max_summary_bytes", 10, "SUMMARY_LIMIT")])
def test_operational_limits(case_setup, limit, value, code):
store = case_setup[0]
builder = CaseBuilder(store, CaseBuilderLimits(**{limit: value}))
with pytest.raises(CaseBuildError) as error:
builder.build("case-A", "stable-package")
assert error.value.code == code
with PackageReader(store.latest("case-A", "stable-package")) as reader:
assert reader.manifest["artifacts"]["case"] is None
def test_concurrent_change_rejects_prepared_case(case_setup, monkeypatch):
store, builder, _, _, _, _ = case_setup
original = builder.assembler.assemble
def change(reader):
case = original(reader)
store.advance("case-A", "stable-package", State.BUILDING_CASE, step_id="concurrent-work", additions={"new-data": b"new"})
return case
monkeypatch.setattr(builder.assembler, "assemble", change)
with pytest.raises(CaseBuildError) as error:
builder.build("case-A", "stable-package")
assert error.value.code == "GENERATION_CHANGED"
with PackageReader(store.latest("case-A", "stable-package")) as reader:
assert reader.manifest["artifacts"]["case"] is None
def test_expected_generation_check_and_successful_replay(case_setup):
store = case_setup[0]
with pytest.raises(LifecycleError) as error:
store.advance("case-A", "stable-package", State.BUILDING_CASE, step_id="prepared", expected_generation=0)
assert error.value.code == "GENERATION_CHANGED"
first = store.advance("case-A", "stable-package", State.BUILDING_CASE, step_id="prepared", expected_generation=1)
assert store.advance("case-A", "stable-package", State.BUILDING_CASE, step_id="prepared", expected_generation=1) == first
@pytest.mark.parametrize("change,code", [("version", "INPUT_SCHEMA"), ("path", "INPUT_REFERENCE"), ("missing", "INPUT_REFERENCE"), ("duplicate", "INPUT_DUPLICATE"), ("report", "INPUT_SCHEMA"), ("json", "INPUT_JSON")])
def test_invalid_declared_input_is_controlled(change, code, manifest, make_package, tmp_path):
manifest["artifacts"]["case_inputs"] = "index"
index = {"format": "viano-case-inputs", "format_version": 1, "reports": ["report"]}
report = {"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "observed"}]}
if change == "version":
index["format_version"] = 2
elif change == "path":
index["reports"] = ["../report"]
elif change == "missing":
index["reports"] = ["absent"]
elif change == "duplicate":
index["reports"] = ["report", "report"]
elif change == "report":
report["observations"] = "not a list"
entries = {"index": json.dumps(index), "report": json.dumps(report) if change != "json" else '{"format":"viano-bug-report","format":"duplicate"}'}
original = make_package(entries=entries)
before = original.read_bytes()
store = CheckpointStore(Workspace(tmp_path / "service"))
store.intake(original)
with pytest.raises(CaseBuildError) as error:
CaseBuilder(store).build("case-A", "stable-package")
assert error.value.code == code
assert original.read_bytes() == before
with PackageReader(store.latest("case-A", "stable-package")) as reader:
assert reader.manifest["artifacts"]["case"] is None
def test_explicit_conflicts_are_traced_without_semantic_guessing(manifest, make_package, tmp_path):
manifest["artifacts"]["case_inputs"] = "index"
reports = [
{"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "Open"}], "expected_behavior": [{"text": "Should be open", "claim": {"key": "switch_state", "value": "open"}}], "expected_behavior_complete": True},
{"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "Closed"}], "expected_behavior": [{"text": "Should be closed", "claim": {"key": "switch_state", "value": "closed"}}], "expected_behavior_complete": False},
]
index = {"format": "viano-case-inputs", "format_version": 1, "reports": ["a", "b"]}
store = CheckpointStore(Workspace(tmp_path / "service"))
store.intake(make_package(entries={"index": json.dumps(index), "a": json.dumps(reports[0]), "b": json.dumps(reports[1])}))
result = CaseBuilder(store).build("case-A", "stable-package")
with PackageReader(result.package) as reader:
case = json.loads(reader.read_bytes(result.case_ref))
conflicts = [issue for issue in case["data_quality"]["issues"] if issue["code"] == "CONFLICTING_CLAIMS"]
assert len(conflicts) == 1
assert "expected_behavior" in conflicts[0]["message"]
assert conflicts[0]["refs"] == [{"source_ref": "a", "json_pointer": "/expected_behavior/0"}, {"source_ref": "b", "json_pointer": "/expected_behavior/0"}]
assert case["expected_behavior"]["completeness"] == "conflicting"
assert len(case["human_observations"]["observations"]) == 2
def test_plain_language_disagreement_is_not_interpreted(manifest, make_package, tmp_path):
manifest["artifacts"]["case_inputs"] = "index"
report = {"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "The light is on"}, {"text": "The light is off"}], "tester_hypotheses": [{"text": "It must be broken"}]}
store = CheckpointStore(Workspace(tmp_path / "service"))
store.intake(make_package(entries={"index": json.dumps({"format": "viano-case-inputs", "format_version": 1, "reports": ["report"]}), "report": json.dumps(report)}))
result = CaseBuilder(store).build("case-A", "stable-package")
with PackageReader(result.package) as reader:
case = json.loads(reader.read_bytes(result.case_ref))
assert not any(issue["code"] == "CONFLICTING_CLAIMS" for issue in case["data_quality"]["issues"])
assert case["runtime_facts"]["facts"] == []
def test_deterministic_assembly_for_same_snapshot(case_setup):
store = case_setup[0]
path = store.advance("case-A", "stable-package", State.BUILDING_CASE, step_id="start")
with PackageReader(path) as reader:
assert CaseAssembler().assemble(reader) == CaseAssembler().assemble(reader)
def test_completed_case_summary_limit_applies_on_reuse(case_setup):
store, builder, _, _, _, _ = case_setup
builder.build("case-A", "stable-package")
with pytest.raises(CaseBuildError) as error:
CaseBuilder(store, CaseBuilderLimits(max_summary_bytes=10)).build("case-A", "stable-package")
assert error.value.code == "SUMMARY_LIMIT"
def test_builder_rejects_unknown_case_and_wrong_phase(case_setup):
store, builder, _, _, _, _ = case_setup
with pytest.raises(CaseBuildError) as error:
builder.build("unknown", "unknown")
assert error.value.code == "CASE_NOT_FOUND"
store.advance("case-A", "stable-package", State.FAILED_TO_ANALYZE, step_id="unavailable", reason="test")
with pytest.raises(CaseBuildError) as error:
builder.build("case-A", "stable-package")
assert error.value.code == "INVALID_BUILD_STATE"
@pytest.mark.parametrize("value", [0, -1, True, 1.5])
def test_invalid_builder_limits(value):
with pytest.raises(ValueError):
CaseBuilderLimits(max_documents=value)
def test_large_fact_values_remain_referenced(case_setup):
store = case_setup[0]
result = CaseBuilder(store, CaseBuilderLimits(max_inline_fact_bytes=1)).build("case-A", "stable-package")
with PackageReader(result.package) as reader:
case = json.loads(reader.read_bytes(result.case_ref))
fact = case["runtime_facts"]["facts"][0]
# JSON encoding of the integer 2 occupies one byte, so remains inline.
assert fact["value"] == 2
def test_large_fact_does_not_embed_raw_array(manifest, make_package, tmp_path):
manifest["artifacts"]["case_inputs"] = "index"
payload = list(range(100))
reproduction = {"format": "viano-reproduction", "format_version": 1, "facts": [{"name": "raw_values", "value": payload}]}
index = {"format": "viano-case-inputs", "format_version": 1, "reproductions": ["repro"]}
store = CheckpointStore(Workspace(tmp_path / "service"))
store.intake(make_package(entries={"index": json.dumps(index), "repro": json.dumps(reproduction)}))
builder = CaseBuilder(store, CaseBuilderLimits(max_inline_fact_bytes=20))
result = builder.build("case-A", "stable-package")
with PackageReader(result.package) as reader:
case = json.loads(reader.read_bytes(result.case_ref))
fact = case["runtime_facts"]["facts"][0]
assert "value" not in fact
assert fact["value_ref"] == {"source_ref": "repro", "json_pointer": "/facts/0/value"}
assert json.loads(reader.read_bytes("repro"))["facts"][0]["value"] == payload
assert "referenced at repro /facts/0/value" in reader.read_bytes(result.summary_ref).decode()
assert builder.build("case-A", "stable-package").reused
def test_real_process_exit_after_case_publication(case_setup):
import os
import subprocess
import sys
from pathlib import Path
store = case_setup[0]
script = """
import os, sys
from viano_bug_analysis.lifecycle import Workspace, CheckpointStore
from viano_bug_analysis.case import CaseBuilder
store = CheckpointStore(Workspace(sys.argv[1]))
original = store._index
def terminate(path, manifest):
if manifest['artifacts'].get('case'):
os._exit(24)
original(path, manifest)
store._index = terminate
CaseBuilder(store).build('case-A', 'stable-package')
"""
env = dict(os.environ, PYTHONPATH=str(Path(__file__).resolve().parents[1] / "src"))
process = subprocess.run([sys.executable, "-c", script, str(store.workspace.root)], env=env, capture_output=True, timeout=15)
assert process.returncode == 24, process.stderr.decode()
result = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package")
assert result.reused
assert len(list(result.package.parent.glob("g*.zip"))) == 3
def test_start_replay_can_observe_already_completed_build(case_setup, monkeypatch):
store, builder, _, _, _, _ = case_setup
original = store.advance
injected = False
def complete_elsewhere(*args, **kwargs):
nonlocal injected
result = original(*args, **kwargs)
if kwargs["step_id"].endswith(":start") and not injected:
injected = True
return CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package").package
return result
monkeypatch.setattr(store, "advance", complete_elsewhere)
result = builder.build("case-A", "stable-package")
assert result.reused
assert len(list(result.package.parent.glob("g*.zip"))) == 3