Add generic deterministic CaseBuilder with versioned input contracts
This commit is contained in:
1 parent
e985e1d5f0
commit
4543e7cd31
18 files changed
+2093
-3
No files matched your search
@@ -6,6 +6,8 @@ The implemented v1 contract is documented in
|
||||
[docs/package-format-v1.md](docs/package-format-v1.md).
|
||||
Job state, checkpoints and inbox admission are documented in
|
||||
[docs/lifecycle-v1.md](docs/lifecycle-v1.md).
|
||||
Generic case construction and its input contracts are documented in
|
||||
[docs/case-builder-v1.md](docs/case-builder-v1.md).
|
||||
|
||||
Requires Python 3.11 or newer.
|
||||
|
||||
@@ -16,5 +18,5 @@ python -m venv .venv
|
||||
```
|
||||
|
||||
The lifecycle foundation supports one-pass inbox processing and resumable package
|
||||
checkpoints. It includes no analysis worker, CaseBuilder, LLM,
|
||||
checkpoints and a deterministic generic CaseBuilder. It includes no analysis worker or LLM,
|
||||
case interpretation, hypothesis generation or investigation tools.
|
||||
@@ -0,0 +1,152 @@
|
||||
# Generic CaseBuilder foundation v1
|
||||
|
||||
`architecture.md` remains authoritative. This milestone assembles supplied data
|
||||
deterministically. It performs no LLM calls, interpretation, hypothesis generation,
|
||||
diagnosis, investigation, external research or project-specific adapter work.
|
||||
Tester hypotheses remain supplied data, never generated or promoted to facts.
|
||||
|
||||
## Explicit input contracts
|
||||
|
||||
The manifest may declare `artifacts.case_inputs`, pointing to a UTF-8 JSON index:
|
||||
|
||||
```json
|
||||
{
|
||||
"format": "viano-case-inputs",
|
||||
"format_version": 1,
|
||||
"reports": ["original/report.data"],
|
||||
"reproductions": ["original/reproduction.data"],
|
||||
"context": {
|
||||
"runtime": ["original/runtime.bin"],
|
||||
"database": [],
|
||||
"source": [],
|
||||
"research": []
|
||||
},
|
||||
"attachments": []
|
||||
}
|
||||
```
|
||||
|
||||
Paths and extensions have no inherent meaning. Listed reports conform to
|
||||
`report-v1.schema.json`, reproductions to `reproduction-v1.schema.json`, and the
|
||||
index to `case-inputs-v1.schema.json`. Schemas are bundled offline Draft 2020-12
|
||||
contracts. Unlisted files are preserved as unclassified inputs without parsing
|
||||
or guessing their role. A missing index produces explicit data-quality gaps.
|
||||
Invalid declared versions/contracts, duplicate JSON keys, repeated sources within
|
||||
an index role, unsafe or absent references are controlled errors.
|
||||
|
||||
A report requires format `viano-bug-report` and format_version 1. Optional fields:
|
||||
title, summary, observations, comments, tester_hypotheses, expected_behavior,
|
||||
expected_behavior_complete and entities. Statements have nonblank `text`, optional
|
||||
`author` and timezone-aware `observed_at`, and optionally a structured `claim`
|
||||
with `key` and JSON `value`. Categories are producer-supplied, not inferred from
|
||||
prose. Unknown extensions remain accessible in the preserved source file.
|
||||
|
||||
```json
|
||||
{
|
||||
"format": "viano-bug-report",
|
||||
"format_version": 1,
|
||||
"title": "A generic observed problem",
|
||||
"observations": [{"text": "The switch remains off", "author": "Tester A"}],
|
||||
"tester_hypotheses": [{"text": "Perhaps a timing issue"}],
|
||||
"expected_behavior": [{"text": "The switch turns on"}],
|
||||
"expected_behavior_complete": false,
|
||||
"entities": [{"type": "component", "id": "42", "roles": ["focus"]}]
|
||||
}
|
||||
```
|
||||
|
||||
A reproduction requires format `viano-reproduction` and format_version 1. Optional
|
||||
fields: steps (strings), observations (human statements), events_ref, attachments,
|
||||
facts and entities. Facts have a name and JSON value: they are supplied facts,
|
||||
not independently verified technical findings. Events, attachments and all context
|
||||
files remain opaque references with hashes/sizes. Supplied research stays separate;
|
||||
no research is performed and no local implementation facts are inferred from it.
|
||||
|
||||
## Canonical case and provenance
|
||||
|
||||
`case-v1.schema.json` defines format `viano-case`, format_version 1, builder_version
|
||||
`1`, and the architecture's main blocks: case, bug, human_observations,
|
||||
expected_behavior, reproduction, entities, runtime_facts, database_context,
|
||||
source_context, research, data_quality, inputs.
|
||||
|
||||
Identity/project/revision come only from the manifest. Each statement/fact/entity
|
||||
has a source path and JSON Pointer with provenance kind. Human statements retain
|
||||
author/time. Observations, comments, tester hypotheses and expected behavior are
|
||||
separate. Record IDs use O/C/T/E/F namespaces; previous IDs remain stable by source
|
||||
path and pointer when inputs are appended. Facts use F0001 etc. Reproduction IDs
|
||||
follow source order. Integer entity IDs become strings; equal integer/string IDs
|
||||
of the same explicit type combine roles and retain every source. No domain types,
|
||||
relations or content are invented.
|
||||
|
||||
Expected-behavior completeness is missing, unspecified, reported_complete,
|
||||
reported_partial or conflicting. Reported completeness is an assertion by the
|
||||
reporter, not a sufficiency judgment. Different canonical JSON values for the same
|
||||
explicit claim key within observations or expected behavior are flagged with
|
||||
source references. Natural-language contradictions are not interpreted; comments
|
||||
and tester hypotheses never participate in fact or conflict inference.
|
||||
|
||||
Large context/events stay outside case JSON. Fact values over the inline limit
|
||||
use `value_ref` (source path and JSON Pointer) instead of `value`; nothing is
|
||||
truncated or replaced by a synthetic null. Old structured inputs are inherited
|
||||
from the prior case's role inventory, hash-checked, and re-read from their immutable
|
||||
originals. A returning index can add only new reports without losing old statements,
|
||||
facts or provenance. Prior case files remain explicitly referenced. Incompatible
|
||||
prior canonical case contracts are rejected instead of guessed.
|
||||
|
||||
## Artifact and checkpoint integration
|
||||
|
||||
```python
|
||||
from viano_bug_analysis.case import CaseBuilder, CaseBuildError
|
||||
from viano_bug_analysis.lifecycle import Workspace, CheckpointStore
|
||||
|
||||
store = CheckpointStore(Workspace("service-data"))
|
||||
try:
|
||||
result = CaseBuilder(store).build("case-A", "stable-package")
|
||||
except CaseBuildError as error:
|
||||
print(error.code, error.path, str(error))
|
||||
```
|
||||
|
||||
Input must already be admitted. VALIDATED or REOPENED enters BUILDING_CASE through
|
||||
a start checkpoint; existing BUILDING_CASE work continues there. Artifacts use
|
||||
`case/revisions/NNNN/case.json` and `case_summary.md`. Revision paths preserve
|
||||
append-only history; manifest pointers `case` and `case_summary` select the current
|
||||
documents. Existing case files and unknown artifacts are never overwritten.
|
||||
|
||||
Case JSON, deterministic summary and the `case-built` completion checkpoint are
|
||||
published together through PackageWriter. State remains BUILDING_CASE; construction
|
||||
does not execute or start interpretation. Later phase orchestration advances it.
|
||||
Summary is generated only from canonical JSON, adds no interpretation and escapes
|
||||
supplied Markdown/HTML. `render_summary` is pure; `CaseAssembler.assemble` is
|
||||
deterministic for the same validated snapshot.
|
||||
|
||||
Completion uses a stable step ID per builder version and analysis revision.
|
||||
Repeats check the input request hash, case contract and summary, reuse output and
|
||||
repair local metadata. Crashes after publication do not create another generation;
|
||||
unconfirmed work retries from the same snapshot. Changed payloads after completion
|
||||
require reopening with a new analysis revision. The request hash includes explicit
|
||||
source roles and all unclassified payloads, excluding known generated case outputs
|
||||
and checkpoint artifacts. Analysis revisions and package generations stay separate.
|
||||
|
||||
Prepared output calls `CheckpointStore.advance(expected_generation=...)` to reject
|
||||
concurrent changes before committing. Identical confirmed requests remain replayable
|
||||
with the old expected generation. Locks and this optimistic check prevent stale
|
||||
prepared data from being committed over newer work. There are no external side
|
||||
effects in construction.
|
||||
|
||||
## Configurable resource defaults
|
||||
|
||||
`CaseBuilderLimits` holds operational policy, not immutable format limits:
|
||||
|
||||
| Limit | Default |
|
||||
|---|---:|
|
||||
| Individual index/report/reproduction JSON | 1 MiB |
|
||||
| Aggregate structured input JSON | 8 MiB |
|
||||
| Structured input documents | 256 |
|
||||
| Prior/canonical case JSON | 8 MiB |
|
||||
| Summary bytes | 8 MiB |
|
||||
| Inline fact value | 16 KiB |
|
||||
|
||||
Prior case JSON is bounded separately from the structured input budget. Package
|
||||
limits still apply to the archive/artifacts. Context is hashed in chunks rather
|
||||
than loaded as prose. Invalid attempts preserve all source/previous case files;
|
||||
they may retain a confirmed start checkpoint in BUILDING_CASE, allowing resumption
|
||||
after corrected sources and a new index are appended. Changing resource defaults
|
||||
does not rewrite a previously confirmed case.
|
||||
@@ -2,7 +2,8 @@
|
||||
|
||||
`architecture.md` remains the authoritative design. This milestone provides only
|
||||
deterministic job state, checkpoint storage, locking and inbox admission. It does
|
||||
not execute CaseBuilder, interpretation, hypotheses, diagnosis, tools or LLMs.
|
||||
not execute interpretation, hypotheses, diagnosis, tools or LLMs. The separate
|
||||
CaseBuilder module now uses these checkpoint APIs for deterministic construction.
|
||||
State APIs allow future phase implementations to confirm their work; recording
|
||||
a state is not evidence that this foundation executed that phase.
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
from .assembler import CaseAssembler, CaseBuilderLimits
|
||||
from .builder import CaseBuilder, CaseBuildResult
|
||||
from .errors import CaseBuildError
|
||||
from .summary import render_summary
|
||||
|
||||
__all__ = ["CaseAssembler", "CaseBuilderLimits", "CaseBuilder", "CaseBuildResult", "CaseBuildError", "render_summary"]
|
||||
@@ -0,0 +1,234 @@
|
||||
import copy
|
||||
import hashlib
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from viano_bug_analysis.lifecycle.checkpoints import checkpoint_chain
|
||||
from viano_bug_analysis.lifecycle.storage import json_bytes
|
||||
from viano_bug_analysis.package.validator import CHUNK_SIZE
|
||||
|
||||
from .contracts import read_json, require_file, validate
|
||||
from .errors import CaseBuildError
|
||||
from .expected_behavior import collect_expected_behavior
|
||||
from .report_collector import collect_statements, provenance
|
||||
from .reproduction_collector import collect_reproductions
|
||||
|
||||
|
||||
def _stable_records(previous: list[dict], current: list[dict], prefix: str) -> list[dict]:
|
||||
def identity(record):
|
||||
source = record["provenance"]
|
||||
return source["source_ref"], source["json_pointer"]
|
||||
remaining = {identity(record): record for record in current}
|
||||
result = []
|
||||
ids = set()
|
||||
for old in previous:
|
||||
if not re.fullmatch(prefix + r"[0-9]{4,}", old["id"]) or old["id"] in ids:
|
||||
raise CaseBuildError("PREVIOUS_IDS", "Prior case contains invalid or duplicate record IDs")
|
||||
ids.add(old["id"])
|
||||
record = remaining.pop(identity(old), None)
|
||||
if record is None:
|
||||
raise CaseBuildError("PREVIOUS_RECORD_LOST", "Prior source record disappeared from its structured input")
|
||||
record["id"] = old["id"]
|
||||
result.append(record)
|
||||
next_id = max((int(value[len(prefix):]) for value in ids), default=0) + 1
|
||||
for record in current:
|
||||
if identity(record) in remaining:
|
||||
record["id"] = f"{prefix}{next_id:04d}"
|
||||
next_id += 1
|
||||
result.append(record)
|
||||
del remaining[identity(record)]
|
||||
return result
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CaseBuilderLimits:
|
||||
"""Operational limits, independent of the JSON format contracts."""
|
||||
|
||||
max_json_bytes: int = 1024**2
|
||||
max_structured_bytes: int = 8 * 1024**2
|
||||
max_documents: int = 256
|
||||
max_case_bytes: int = 8 * 1024**2
|
||||
max_summary_bytes: int = 8 * 1024**2
|
||||
max_inline_fact_bytes: int = 16 * 1024
|
||||
|
||||
def __post_init__(self):
|
||||
for value in vars(self).values():
|
||||
if isinstance(value, bool) or not isinstance(value, int) or value <= 0:
|
||||
raise ValueError("CaseBuilder limits must be positive integers")
|
||||
|
||||
|
||||
class CaseAssembler:
|
||||
def __init__(self, limits: CaseBuilderLimits | None = None):
|
||||
self.limits = limits or CaseBuilderLimits()
|
||||
|
||||
def assemble(self, reader) -> dict:
|
||||
manifest = reader.manifest
|
||||
if manifest["analysis_revision"] < 1:
|
||||
raise CaseBuildError("INVALID_REVISION", "Case construction requires an active analysis revision")
|
||||
used_bytes = 0
|
||||
documents = 0
|
||||
|
||||
def load(path, contract):
|
||||
nonlocal used_bytes, documents
|
||||
require_file(reader, path)
|
||||
if contract == "case":
|
||||
return read_json(reader, path, contract, self.limits.max_case_bytes)
|
||||
documents += 1
|
||||
used_bytes += reader._archive.getinfo(path).file_size
|
||||
if documents > self.limits.max_documents or used_bytes > self.limits.max_structured_bytes:
|
||||
raise CaseBuildError("STRUCTURED_INPUT_LIMIT", "Declared structured inputs exceed configured limits", path)
|
||||
return read_json(reader, path, contract, self.limits.max_json_bytes)
|
||||
|
||||
chain = checkpoint_chain(reader)
|
||||
excluded = {"manifest.json"} | {ref for ref, _ in chain}
|
||||
prior_refs = []
|
||||
for _, checkpoint in reversed(chain):
|
||||
details = checkpoint["details"]
|
||||
if details.get("operation") == "case-built" and details.get("builder_version") == "1":
|
||||
for field in ("case_ref", "summary_ref"):
|
||||
if not isinstance(details.get(field), str):
|
||||
raise CaseBuildError("PREVIOUS_CASE", "Incomplete case completion checkpoint")
|
||||
require_file(reader, details[field])
|
||||
excluded.add(details[field])
|
||||
if details.get("case_ref") not in prior_refs:
|
||||
prior_refs.append(details["case_ref"])
|
||||
prior_ref = manifest["artifacts"].get("case")
|
||||
if prior_ref:
|
||||
excluded.add(prior_ref)
|
||||
if prior_ref not in prior_refs:
|
||||
prior_refs.append(prior_ref)
|
||||
summary_ref = manifest["artifacts"].get("case_summary")
|
||||
if summary_ref:
|
||||
excluded.add(summary_ref)
|
||||
previous = load(prior_ref, "case") if prior_ref else None
|
||||
if previous and (previous["case"]["case_id"] != manifest["case_id"] or previous["case"]["package_id"] != manifest["package_id"] or previous["case"]["analysis_revision"] > manifest["analysis_revision"]):
|
||||
raise CaseBuildError("CASE_IDENTITY", "Previous case has inconsistent identity or revision", prior_ref)
|
||||
inventory = {}
|
||||
for name in sorted(reader.files):
|
||||
if name == "manifest.json" or name in {ref for ref, _ in chain}:
|
||||
continue
|
||||
digest = hashlib.sha256()
|
||||
size = 0
|
||||
with reader._archive.open(name) as stream:
|
||||
while chunk := stream.read(CHUNK_SIZE):
|
||||
size += len(chunk)
|
||||
if size > reader.validator.limits.max_file_bytes:
|
||||
raise CaseBuildError("INPUT_LIMIT", "Payload exceeds configured file limit", name)
|
||||
digest.update(chunk)
|
||||
inventory[name] = {"path": name, "sha256": digest.hexdigest(), "size": size}
|
||||
index_ref = manifest["artifacts"].get("case_inputs")
|
||||
index = load(index_ref, "case-inputs") if index_ref else {}
|
||||
roles = []
|
||||
|
||||
def add(path, role):
|
||||
require_file(reader, path)
|
||||
if path in excluded:
|
||||
raise CaseBuildError("INPUT_REFERENCE", "Generated case or checkpoint cannot serve as raw input", path)
|
||||
if (path, role) not in roles:
|
||||
roles.append((path, role))
|
||||
|
||||
if previous:
|
||||
for record in previous["inputs"]["records"]:
|
||||
add(record["path"], record["role"])
|
||||
actual = inventory[record["path"]]
|
||||
if record["sha256"] != actual["sha256"] or record["size"] != actual["size"]:
|
||||
raise CaseBuildError("PREVIOUS_INPUT_CHANGED", "A source referenced by the prior case changed", record["path"])
|
||||
if index_ref:
|
||||
add(index_ref, "index")
|
||||
for field, role in (("reports", "report"), ("reproductions", "reproduction"), ("attachments", "attachment")):
|
||||
paths = index.get(field, [])
|
||||
if len(paths) != len(set(paths)):
|
||||
raise CaseBuildError("INPUT_DUPLICATE", "A source is listed twice in the same role", index_ref)
|
||||
for path in paths:
|
||||
add(path, role)
|
||||
for role, paths in index.get("context", {}).items():
|
||||
if role not in {"runtime", "database", "source", "research"}:
|
||||
continue
|
||||
if len(paths) != len(set(paths)):
|
||||
raise CaseBuildError("INPUT_DUPLICATE", "A context source is listed twice", index_ref)
|
||||
for path in paths:
|
||||
add(path, role)
|
||||
reports = [(path, load(path, "report")) for path, role in roles if role == "report"]
|
||||
reproductions = [(path, load(path, "reproduction")) for path, role in roles if role == "reproduction"]
|
||||
for _, document in reproductions:
|
||||
for path in document.get("attachments", []):
|
||||
add(path, "attachment")
|
||||
if document.get("events_ref"):
|
||||
add(document["events_ref"], "attachment")
|
||||
observations = collect_statements(reports + reproductions, "observations", "O")
|
||||
comments = collect_statements(reports, "comments", "C")
|
||||
hypotheses = collect_statements(reports, "tester_hypotheses", "T")
|
||||
expected = collect_expected_behavior(reports)
|
||||
reproduction_records, facts = collect_reproductions(reproductions, self.limits.max_inline_fact_bytes)
|
||||
if previous:
|
||||
old_humans = previous["human_observations"]
|
||||
observations = _stable_records(old_humans["observations"], observations, "O")
|
||||
comments = _stable_records(old_humans["comments"], comments, "C")
|
||||
hypotheses = _stable_records(old_humans["tester_hypotheses"], hypotheses, "T")
|
||||
expected["statements"] = _stable_records(previous["expected_behavior"]["statements"], expected["statements"], "E")
|
||||
facts = _stable_records(previous["runtime_facts"]["facts"], facts, "F")
|
||||
entities = {}
|
||||
for path, document in reports + reproductions:
|
||||
for index, entity in enumerate(document.get("entities", [])):
|
||||
entity_id = entity["id"] if isinstance(entity["id"], str) else str(int(entity["id"]))
|
||||
identity = (entity["type"], entity_id)
|
||||
record = entities.setdefault(identity, {"type": identity[0], "id": identity[1], "roles": [], "provenance": []})
|
||||
record["roles"] = sorted(set(record["roles"]) | set(entity.get("roles", [])))
|
||||
record["provenance"].append(provenance(path, f"/entities/{index}", "report" if document["format"] == "viano-bug-report" else "reproduction_record"))
|
||||
issues = []
|
||||
|
||||
def issue(code, message, refs=None):
|
||||
issues.append({"code": code, "message": message, "refs": refs or []})
|
||||
|
||||
if not index_ref:
|
||||
issue("MISSING_INPUT_INDEX", "No explicit case input index was supplied")
|
||||
if not reports:
|
||||
issue("MISSING_REPORT", "No structured report was supplied")
|
||||
if not observations:
|
||||
issue("MISSING_OBSERVATIONS", "No direct observations were supplied")
|
||||
if not reproductions:
|
||||
issue("MISSING_REPRODUCTION", "No structured reproduction was supplied")
|
||||
if expected["completeness"] != "reported_complete":
|
||||
issue("EXPECTED_BEHAVIOR_" + expected["completeness"].upper(), "Expected-behavior completeness is " + expected["completeness"],
|
||||
[{"source_ref": path, "json_pointer": "/expected_behavior_complete"} for path, report in reports if "expected_behavior_complete" in report])
|
||||
for category, records in (("observation", observations), ("expected_behavior", expected["statements"])):
|
||||
claims = {}
|
||||
for record in records:
|
||||
if "claim" in record:
|
||||
claim = record["claim"]
|
||||
claims.setdefault(claim["key"], []).append(record)
|
||||
for claim_key, records in sorted(claims.items()):
|
||||
if len({json_bytes(record["claim"]["value"]) for record in records}) > 1:
|
||||
refs = [{"source_ref": r["provenance"]["source_ref"], "json_pointer": r["provenance"]["json_pointer"]} for r in records]
|
||||
issue("CONFLICTING_CLAIMS", f"Different explicit values for {category} claim {claim_key}", refs)
|
||||
if category == "expected_behavior":
|
||||
expected["completeness"] = "conflicting"
|
||||
records = [inventory[path] | {"role": role} for path, role in roles]
|
||||
classified = {path for path, _ in roles}
|
||||
raw_inventory = [value for name, value in inventory.items() if name not in excluded]
|
||||
request = {"builder_version": "1", "case_id": manifest["case_id"], "package_id": manifest["package_id"],
|
||||
"analysis_revision": int(manifest["analysis_revision"]), "project": manifest["project"],
|
||||
"index_ref": index_ref, "payloads": raw_inventory, "roles": roles}
|
||||
request_hash = hashlib.sha256(json_bytes(request)).hexdigest()
|
||||
refs_for = lambda role: [inventory[path] for path, value in roles if value == role]
|
||||
case = {
|
||||
"format": "viano-case", "format_version": 1, "builder_version": "1",
|
||||
"case": {"case_id": manifest["case_id"], "package_id": manifest["package_id"], "analysis_revision": int(manifest["analysis_revision"]),
|
||||
"source_generation": int(manifest["package_generation"]), "project": copy.deepcopy(manifest["project"])},
|
||||
"bug": {"reports": [{"source_ref": path, "title": document.get("title"), "summary": document.get("summary")} for path, document in reports]},
|
||||
"human_observations": {"observations": observations, "comments": comments, "tester_hypotheses": hypotheses},
|
||||
"expected_behavior": expected,
|
||||
"reproduction": {"records": reproduction_records},
|
||||
"entities": [entities[identity] for identity in sorted(entities)],
|
||||
"runtime_facts": {"facts": facts, "raw_refs": refs_for("runtime")},
|
||||
"database_context": {"refs": refs_for("database")}, "source_context": {"refs": refs_for("source")},
|
||||
"research": {"refs": refs_for("research"), "performed": False},
|
||||
"data_quality": {"issues": issues, "coverage": {"reports": len(reports), "reproductions": len(reproductions), "observations": len(observations)}},
|
||||
"inputs": {"index_ref": index_ref, "records": records,
|
||||
"unclassified": [value for name, value in inventory.items() if name not in classified and name not in excluded],
|
||||
"prior_cases": [inventory[ref] for ref in prior_refs], "request_hash": request_hash},
|
||||
}
|
||||
validate(case, "case")
|
||||
if len(json_bytes(case)) > self.limits.max_case_bytes:
|
||||
raise CaseBuildError("CASE_LIMIT", "Assembled case exceeds configured byte limit")
|
||||
return case
|
||||
@@ -0,0 +1,91 @@
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from viano_bug_analysis.lifecycle import LifecycleError, State
|
||||
from viano_bug_analysis.lifecycle.checkpoints import checkpoint_chain
|
||||
from viano_bug_analysis.lifecycle.storage import json_bytes
|
||||
from viano_bug_analysis.package import PackageReader, PackageReadError
|
||||
from viano_bug_analysis.package.validator import ARCHIVE_ERRORS
|
||||
|
||||
from .assembler import CaseAssembler, CaseBuilderLimits
|
||||
from .contracts import read_json
|
||||
from .errors import CaseBuildError
|
||||
from .summary import render_summary
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CaseBuildResult:
|
||||
package: Path
|
||||
case_ref: str
|
||||
summary_ref: str
|
||||
analysis_revision: int
|
||||
reused: bool
|
||||
|
||||
|
||||
class CaseBuilder:
|
||||
def __init__(self, store, limits: CaseBuilderLimits | None = None):
|
||||
self.store = store
|
||||
self.assembler = CaseAssembler(limits)
|
||||
|
||||
def build(self, case_id: str, package_id: str) -> CaseBuildResult:
|
||||
try:
|
||||
latest = self.store.latest(case_id, package_id)
|
||||
if latest is None:
|
||||
raise CaseBuildError("CASE_NOT_FOUND", "Case must be admitted before construction")
|
||||
with PackageReader(latest, self.store.limits) as reader:
|
||||
manifest = reader.manifest
|
||||
revision = int(manifest["analysis_revision"])
|
||||
revision = revision or 1
|
||||
complete_id = f"case-builder:v1:revision:{revision}:complete"
|
||||
completed = next((cp for _, cp in checkpoint_chain(reader) if cp["step_id"] == complete_id), None)
|
||||
if completed:
|
||||
assembled = self.assembler.assemble(reader)
|
||||
details = completed["details"]
|
||||
if details.get("input_request_hash") != assembled["inputs"]["request_hash"]:
|
||||
raise CaseBuildError("STALE_CASE", "Inputs changed after case completion; reopen as a new analysis revision")
|
||||
case_ref, summary_ref = details.get("case_ref"), details.get("summary_ref")
|
||||
stored = read_json(reader, case_ref, "case", self.assembler.limits.max_case_bytes)
|
||||
summary = render_summary(stored).encode("utf-8")
|
||||
if len(summary) > self.assembler.limits.max_summary_bytes:
|
||||
raise CaseBuildError("SUMMARY_LIMIT", "Summary exceeds configured byte limit")
|
||||
with reader._archive.open(summary_ref) as stream:
|
||||
stored_summary = reader.validator._read_bounded(stream, min(self.assembler.limits.max_summary_bytes, self.store.limits.max_file_bytes))
|
||||
if reader.manifest["artifacts"].get("case") != case_ref or reader.manifest["artifacts"].get("case_summary") != summary_ref or stored_summary != summary:
|
||||
raise CaseBuildError("CASE_OUTPUT_CHANGED", "Confirmed case pointers or summary no longer match")
|
||||
# Repair reconstructible metadata after a crash, without recomputing a commit.
|
||||
recovered = self.store.advance(case_id, package_id, State.BUILDING_CASE, step_id=complete_id,
|
||||
details=details, additions={case_ref: json_bytes(stored), summary_ref: summary},
|
||||
artifacts={"case": case_ref, "case_summary": summary_ref})
|
||||
if recovered != latest:
|
||||
raise CaseBuildError("GENERATION_CHANGED", "Case changed while confirmed output was checked")
|
||||
return CaseBuildResult(latest, case_ref, summary_ref, revision, True)
|
||||
if manifest["state"] not in {State.VALIDATED, State.REOPENED, State.BUILDING_CASE}:
|
||||
raise CaseBuildError("INVALID_BUILD_STATE", "CaseBuilder requires VALIDATED, REOPENED or BUILDING_CASE")
|
||||
source_generation = int(manifest["package_generation"])
|
||||
if manifest["state"] != State.BUILDING_CASE:
|
||||
latest = self.store.advance(case_id, package_id, State.BUILDING_CASE,
|
||||
step_id=f"case-builder:v1:revision:{revision}:start",
|
||||
details={"operation": "case-building", "builder_version": "1"},
|
||||
expected_generation=source_generation)
|
||||
with PackageReader(latest, self.store.limits) as reader:
|
||||
if any(cp["step_id"] == complete_id for _, cp in checkpoint_chain(reader)):
|
||||
return self.build(case_id, package_id)
|
||||
case = self.assembler.assemble(reader)
|
||||
revision = case["case"]["analysis_revision"]
|
||||
case_ref = f"case/revisions/{revision:04d}/case.json"
|
||||
summary_ref = f"case/revisions/{revision:04d}/case_summary.md"
|
||||
summary = render_summary(case).encode("utf-8")
|
||||
if len(summary) > self.assembler.limits.max_summary_bytes:
|
||||
raise CaseBuildError("SUMMARY_LIMIT", "Summary exceeds configured byte limit")
|
||||
details = {"operation": "case-built", "builder_version": "1", "case_ref": case_ref, "summary_ref": summary_ref,
|
||||
"input_request_hash": case["inputs"]["request_hash"]}
|
||||
output = self.store.advance(case_id, package_id, State.BUILDING_CASE,
|
||||
step_id=f"case-builder:v1:revision:{revision}:complete", details=details,
|
||||
additions={case_ref: json_bytes(case), summary_ref: summary},
|
||||
artifacts={"case": case_ref, "case_summary": summary_ref},
|
||||
expected_generation=int(reader.manifest["package_generation"]))
|
||||
return CaseBuildResult(output, case_ref, summary_ref, revision, False)
|
||||
except (PackageReadError, LifecycleError) as error:
|
||||
raise CaseBuildError(getattr(error, "code", "PACKAGE_ERROR"), str(error)) from error
|
||||
except ARCHIVE_ERRORS + (TypeError, KeyError, RecursionError) as error:
|
||||
raise CaseBuildError("CASE_BUILD_ERROR", str(error)) from error
|
||||
@@ -0,0 +1,35 @@
|
||||
import json
|
||||
from importlib.resources import files
|
||||
|
||||
from jsonschema import Draft202012Validator
|
||||
from jsonschema.exceptions import ValidationError
|
||||
|
||||
from viano_bug_analysis.package import PackageReadError
|
||||
from viano_bug_analysis.package.manifest import FORMAT_CHECKER, parse_manifest, safe_path
|
||||
|
||||
from .errors import CaseBuildError
|
||||
|
||||
|
||||
def validate(document: dict, contract: str, path: str | None = None) -> None:
|
||||
schema = json.loads(files("viano_bug_analysis.schemas").joinpath(f"{contract}-v1.schema.json").read_text())
|
||||
try:
|
||||
Draft202012Validator(schema, format_checker=FORMAT_CHECKER).validate(document)
|
||||
except ValidationError as error:
|
||||
raise CaseBuildError("INPUT_SCHEMA" if contract != "case" else "CASE_SCHEMA", error.message, path) from error
|
||||
|
||||
|
||||
def read_json(reader, path: str, contract: str, limit: int) -> dict:
|
||||
require_file(reader, path)
|
||||
try:
|
||||
with reader._archive.open(path) as stream:
|
||||
data = reader.validator._read_bounded(stream, min(limit, reader.validator.limits.max_file_bytes))
|
||||
document = parse_manifest(data)
|
||||
validate(document, contract, path)
|
||||
return document
|
||||
except (ValueError, RecursionError, PackageReadError) as error:
|
||||
raise CaseBuildError("INPUT_JSON", str(error), path) from error
|
||||
|
||||
|
||||
def require_file(reader, path: str) -> None:
|
||||
if not safe_path(path) or path == "manifest.json" or path not in reader.files:
|
||||
raise CaseBuildError("INPUT_REFERENCE", "Expected a safe existing payload file", path)
|
||||
@@ -0,0 +1,7 @@
|
||||
class CaseBuildError(Exception):
|
||||
"""Controlled failure of a declared input or case construction request."""
|
||||
|
||||
def __init__(self, code: str, message: str, path: str | None = None):
|
||||
self.code = code
|
||||
self.path = path
|
||||
super().__init__(message)
|
||||
@@ -0,0 +1,19 @@
|
||||
from .report_collector import collect_statements
|
||||
|
||||
|
||||
def collect_expected_behavior(reports: list[tuple[str, dict]]) -> dict:
|
||||
statements = collect_statements(reports, "expected_behavior", "E")
|
||||
flags = {document.get("expected_behavior_complete") for _, document in reports
|
||||
if document.get("expected_behavior")}
|
||||
if not statements:
|
||||
status = "missing"
|
||||
elif True in flags and False in flags:
|
||||
status = "conflicting"
|
||||
elif False in flags:
|
||||
status = "reported_partial"
|
||||
elif flags == {True}:
|
||||
status = "reported_complete"
|
||||
else:
|
||||
status = "unspecified"
|
||||
return {"statements": statements, "completeness": status,
|
||||
"provenance": "tester" if statements else "none"}
|
||||
@@ -0,0 +1,19 @@
|
||||
import copy
|
||||
|
||||
|
||||
def provenance(path: str, pointer: str, kind: str, statement: dict | None = None) -> dict:
|
||||
value = statement or {}
|
||||
return {"kind": kind, "source_ref": path, "json_pointer": pointer,
|
||||
"author": value.get("author"), "observed_at": value.get("observed_at")}
|
||||
|
||||
|
||||
def collect_statements(documents: list[tuple[str, dict]], field: str, prefix: str) -> list[dict]:
|
||||
result = []
|
||||
for path, document in documents:
|
||||
for index, statement in enumerate(document.get(field, [])):
|
||||
record = {"id": f"{prefix}{len(result) + 1:04d}", "text": statement["text"],
|
||||
"provenance": provenance(path, f"/{field}/{index}", "tester", statement)}
|
||||
if "claim" in statement:
|
||||
record["claim"] = copy.deepcopy(statement["claim"])
|
||||
result.append(record)
|
||||
return result
|
||||
@@ -0,0 +1,22 @@
|
||||
import copy
|
||||
|
||||
from viano_bug_analysis.lifecycle.storage import json_bytes
|
||||
|
||||
from .report_collector import provenance
|
||||
|
||||
|
||||
def collect_reproductions(documents: list[tuple[str, dict]], inline_limit: int) -> tuple[list[dict], list[dict]]:
|
||||
records, facts = [], []
|
||||
for path, document in documents:
|
||||
records.append({"id": f"R{len(records) + 1:04d}", "source_ref": path,
|
||||
"steps": list(document.get("steps", [])), "events_ref": document.get("events_ref"),
|
||||
"attachments": list(document.get("attachments", []))})
|
||||
for index, fact in enumerate(document.get("facts", [])):
|
||||
record = {"id": f"F{len(facts) + 1:04d}", "name": fact["name"],
|
||||
"provenance": provenance(path, f"/facts/{index}", "reproduction_record")}
|
||||
if len(json_bytes(fact["value"])) > inline_limit:
|
||||
record["value_ref"] = {"source_ref": path, "json_pointer": f"/facts/{index}/value"}
|
||||
else:
|
||||
record["value"] = copy.deepcopy(fact["value"])
|
||||
facts.append(record)
|
||||
return records, facts
|
||||
@@ -0,0 +1,76 @@
|
||||
import html
|
||||
import json
|
||||
import re
|
||||
|
||||
from .contracts import validate
|
||||
|
||||
|
||||
def _literal(value) -> str:
|
||||
text = html.escape(str(value), quote=False)
|
||||
text = re.sub(r"([\\`*_{}\[\]()#+.!|>~-])", r"\\\1", text)
|
||||
return text.replace("\r", "\\r").replace("\n", "<br>")
|
||||
|
||||
|
||||
def render_summary(case: dict) -> str:
|
||||
"""Pure presentation of the canonical case, with no additional interpretation."""
|
||||
validate(case, "case")
|
||||
identity = case["case"]
|
||||
lines = ["# Case summary", "", f"Case: {_literal(identity['case_id'])}",
|
||||
f"Package: {_literal(identity['package_id'])}", f"Analysis revision: {identity['analysis_revision']}",
|
||||
"", "## Reports", ""]
|
||||
for report in case["bug"]["reports"]:
|
||||
lines += [f"- Source: {_literal(report['source_ref'])}",
|
||||
f" Title: {_literal(report['title']) if report['title'] is not None else '(not supplied)'}",
|
||||
f" Summary: {_literal(report['summary']) if report['summary'] is not None else '(not supplied)'}"]
|
||||
for label, records in (("Direct human observations", case["human_observations"]["observations"]),
|
||||
("Tester comments", case["human_observations"]["comments"]),
|
||||
("Tester hypotheses (supplied)", case["human_observations"]["tester_hypotheses"]),
|
||||
("Expected behavior (supplied)", case["expected_behavior"]["statements"])):
|
||||
lines += ["", f"## {label}", ""]
|
||||
for record in records:
|
||||
source = record["provenance"]
|
||||
lines += [f"- {_literal(record['id'])}: {_literal(record['text'])}",
|
||||
f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}",
|
||||
f" Author: {_literal(source['author']) if source['author'] is not None else '(not supplied)'}",
|
||||
f" Observed at: {_literal(source['observed_at']) if source['observed_at'] is not None else '(not supplied)'}"]
|
||||
if "claim" in record:
|
||||
lines.append(f" Claim: {_literal(record['claim']['key'])} = {_literal(json.dumps(record['claim']['value'], sort_keys=True, ensure_ascii=False))}")
|
||||
lines += ["", f"Expected-behavior completeness: {case['expected_behavior']['completeness']}", "", "## Reproductions", ""]
|
||||
for record in case["reproduction"]["records"]:
|
||||
lines.append(f"- {_literal(record['id'])}: {_literal(record['source_ref'])}")
|
||||
for step in record["steps"]:
|
||||
lines.append(f" Step: {_literal(step)}")
|
||||
if record["events_ref"]:
|
||||
lines.append(f" Events: {_literal(record['events_ref'])}")
|
||||
for path in record["attachments"]:
|
||||
lines.append(f" Attachment: {_literal(path)}")
|
||||
lines += ["", "## Entities", ""]
|
||||
for entity in case["entities"]:
|
||||
lines.append(f"- {_literal(entity['type'])} / {_literal(entity['id'])}; roles: {_literal(', '.join(entity['roles']))}")
|
||||
for source in entity["provenance"]:
|
||||
lines.append(f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}")
|
||||
lines += ["", "## Facts supplied by reproduction records", ""]
|
||||
for fact in case["runtime_facts"]["facts"]:
|
||||
source = fact["provenance"]
|
||||
if "value_ref" in fact:
|
||||
ref = fact["value_ref"]
|
||||
value = f"referenced at {_literal(ref['source_ref'])} {_literal(ref['json_pointer'])}"
|
||||
else:
|
||||
value = _literal(json.dumps(fact['value'], sort_keys=True, ensure_ascii=False))
|
||||
lines.append(f"- {_literal(fact['id'])}: {_literal(fact['name'])} = {value}")
|
||||
lines.append(f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}")
|
||||
for label, refs in (("Runtime data", case["runtime_facts"]["raw_refs"]), ("Database context", case["database_context"]["refs"]),
|
||||
("Source context", case["source_context"]["refs"]), ("Supplied research", case["research"]["refs"]),
|
||||
("Prior cases", case["inputs"]["prior_cases"]), ("Unclassified inputs", case["inputs"]["unclassified"])):
|
||||
lines += ["", f"## {label}", ""]
|
||||
for ref in refs:
|
||||
lines.append(f"- {_literal(ref['path'])} ({ref['size']} bytes; SHA-256: {ref['sha256']})")
|
||||
lines += ["", "## Data quality", ""]
|
||||
for issue in case["data_quality"]["issues"]:
|
||||
lines.append(f"- {_literal(issue['code'])}: {_literal(issue['message'])}")
|
||||
for source in issue["refs"]:
|
||||
lines.append(f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}")
|
||||
lines += ["", "## Input records", ""]
|
||||
for record in case["inputs"]["records"]:
|
||||
lines.append(f"- {_literal(record['role'])}: {_literal(record['path'])} ({record['size']} bytes; SHA-256: {record['sha256']})")
|
||||
return "\n".join(lines) + "\n"
|
||||
@@ -234,7 +234,8 @@ class CheckpointStore:
|
||||
def advance(self, case_id: str, package_id: str, target: str, *, step_id: str,
|
||||
reason: str | None = None, retry_not_before: str | None = None,
|
||||
details: dict | None = None, additions: dict[str, bytes] | None = None,
|
||||
artifacts: dict[str, str | None] | None = None) -> Path:
|
||||
artifacts: dict[str, str | None] | None = None,
|
||||
expected_generation: int | None = None) -> Path:
|
||||
try:
|
||||
if not isinstance(step_id, str) or not step_id.strip():
|
||||
raise LifecycleError("INVALID_STEP", "A nonblank stable step_id is required")
|
||||
@@ -242,6 +243,8 @@ class CheckpointStore:
|
||||
raise LifecycleError("CHECKPOINT_INVALID", "Checkpoint details must be a JSON object")
|
||||
if CHECKPOINT_POINTER in (artifacts or {}):
|
||||
raise LifecycleError("RESERVED_ARTIFACT", "Lifecycle checkpoint pointer is service-managed")
|
||||
if expected_generation is not None and (isinstance(expected_generation, bool) or not isinstance(expected_generation, int) or expected_generation < 0):
|
||||
raise LifecycleError("INVALID_GENERATION", "expected_generation must be a nonnegative integer")
|
||||
payload = {"target": target, "reason": reason, "retry_not_before": retry_not_before,
|
||||
"details": details or {}, "artifacts": artifacts or {},
|
||||
"additions": {name: hashlib.sha256(data).hexdigest() for name, data in (additions or {}).items()}}
|
||||
@@ -260,6 +263,8 @@ class CheckpointStore:
|
||||
self._index(latest, manifest)
|
||||
atomic_json(self.workspace.runtime / "current_job.json", None)
|
||||
return latest
|
||||
if expected_generation is not None and manifest["package_generation"] != expected_generation:
|
||||
raise LifecycleError("GENERATION_CHANGED", "Case changed while artifacts were being prepared")
|
||||
resume = chain[0][1]["resume_state"] if manifest["state"] == State.BLOCKED else None
|
||||
check_transition(manifest["state"], target, resume)
|
||||
if manifest["state"] == State.BLOCKED and not self.retry_due(manifest):
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
{
|
||||
"type": "object",
|
||||
"required": [
|
||||
"format",
|
||||
"format_version"
|
||||
],
|
||||
"properties": {
|
||||
"format": {
|
||||
"const": "viano-case-inputs"
|
||||
},
|
||||
"format_version": {
|
||||
"type": "integer",
|
||||
"const": 1
|
||||
},
|
||||
"reports": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"reproductions": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"context": {
|
||||
"type": "object",
|
||||
"required": [],
|
||||
"properties": {
|
||||
"runtime": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"database": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"source": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"research": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"attachments": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
}
|
||||
},
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema"
|
||||
}
|
||||
@@ -0,0 +1,657 @@
|
||||
{
|
||||
"type": "object",
|
||||
"required": [
|
||||
"format",
|
||||
"format_version",
|
||||
"builder_version",
|
||||
"case",
|
||||
"bug",
|
||||
"human_observations",
|
||||
"expected_behavior",
|
||||
"reproduction",
|
||||
"entities",
|
||||
"runtime_facts",
|
||||
"database_context",
|
||||
"source_context",
|
||||
"research",
|
||||
"data_quality",
|
||||
"inputs"
|
||||
],
|
||||
"properties": {
|
||||
"format": {
|
||||
"const": "viano-case"
|
||||
},
|
||||
"format_version": {
|
||||
"type": "integer",
|
||||
"const": 1
|
||||
},
|
||||
"builder_version": {
|
||||
"const": "1"
|
||||
},
|
||||
"case": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"case_id",
|
||||
"package_id",
|
||||
"analysis_revision",
|
||||
"source_generation",
|
||||
"project"
|
||||
],
|
||||
"properties": {
|
||||
"case_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"package_id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"analysis_revision": {
|
||||
"type": "integer",
|
||||
"minimum": 1
|
||||
},
|
||||
"source_generation": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"project": {
|
||||
"type": "object"
|
||||
}
|
||||
}
|
||||
},
|
||||
"bug": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"reports"
|
||||
],
|
||||
"properties": {
|
||||
"reports": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"source_ref",
|
||||
"title",
|
||||
"summary"
|
||||
],
|
||||
"properties": {
|
||||
"source_ref": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"title": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"summary": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"human_observations": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"observations",
|
||||
"comments",
|
||||
"tester_hypotheses"
|
||||
],
|
||||
"properties": {
|
||||
"observations": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/record"
|
||||
}
|
||||
},
|
||||
"comments": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/record"
|
||||
}
|
||||
},
|
||||
"tester_hypotheses": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/record"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"expected_behavior": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"statements",
|
||||
"completeness",
|
||||
"provenance"
|
||||
],
|
||||
"properties": {
|
||||
"statements": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/record"
|
||||
}
|
||||
},
|
||||
"completeness": {
|
||||
"enum": [
|
||||
"missing",
|
||||
"unspecified",
|
||||
"reported_complete",
|
||||
"reported_partial",
|
||||
"conflicting"
|
||||
]
|
||||
},
|
||||
"provenance": {
|
||||
"enum": [
|
||||
"tester",
|
||||
"none"
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"reproduction": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"records"
|
||||
],
|
||||
"properties": {
|
||||
"records": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"source_ref",
|
||||
"steps",
|
||||
"events_ref",
|
||||
"attachments"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"source_ref": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
}
|
||||
},
|
||||
"events_ref": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"attachments": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"entities": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"type",
|
||||
"id",
|
||||
"roles",
|
||||
"provenance"
|
||||
],
|
||||
"properties": {
|
||||
"type": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"roles": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
}
|
||||
},
|
||||
"provenance": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/provenance"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"runtime_facts": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"facts",
|
||||
"raw_refs"
|
||||
],
|
||||
"properties": {
|
||||
"facts": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"name",
|
||||
"provenance"
|
||||
],
|
||||
"oneOf": [
|
||||
{
|
||||
"required": [
|
||||
"value"
|
||||
],
|
||||
"not": {
|
||||
"required": [
|
||||
"value_ref"
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"required": [
|
||||
"value_ref"
|
||||
],
|
||||
"not": {
|
||||
"required": [
|
||||
"value"
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"pattern": "^F[0-9]{4,}$"
|
||||
},
|
||||
"name": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"value": {},
|
||||
"value_ref": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"source_ref",
|
||||
"json_pointer"
|
||||
],
|
||||
"properties": {
|
||||
"source_ref": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"json_pointer": {
|
||||
"type": "string",
|
||||
"pattern": "^/"
|
||||
}
|
||||
}
|
||||
},
|
||||
"provenance": {
|
||||
"$ref": "#/$defs/provenance"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"raw_refs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/raw"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"database_context": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"refs"
|
||||
],
|
||||
"properties": {
|
||||
"refs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/raw"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"source_context": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"refs"
|
||||
],
|
||||
"properties": {
|
||||
"refs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/raw"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"research": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"refs",
|
||||
"performed"
|
||||
],
|
||||
"properties": {
|
||||
"refs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/raw"
|
||||
}
|
||||
},
|
||||
"performed": {
|
||||
"const": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"data_quality": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"issues",
|
||||
"coverage"
|
||||
],
|
||||
"properties": {
|
||||
"issues": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"code",
|
||||
"message",
|
||||
"refs"
|
||||
],
|
||||
"properties": {
|
||||
"code": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"message": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"refs": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"source_ref",
|
||||
"json_pointer"
|
||||
],
|
||||
"properties": {
|
||||
"source_ref": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"json_pointer": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"coverage": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"reports",
|
||||
"reproductions",
|
||||
"observations"
|
||||
],
|
||||
"properties": {
|
||||
"reports": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"reproductions": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"observations": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"index_ref",
|
||||
"records",
|
||||
"unclassified",
|
||||
"prior_cases",
|
||||
"request_hash"
|
||||
],
|
||||
"properties": {
|
||||
"index_ref": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"records": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"path",
|
||||
"sha256",
|
||||
"size",
|
||||
"role"
|
||||
],
|
||||
"properties": {
|
||||
"path": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"sha256": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{64}$"
|
||||
},
|
||||
"size": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
},
|
||||
"role": {
|
||||
"enum": [
|
||||
"index",
|
||||
"report",
|
||||
"reproduction",
|
||||
"runtime",
|
||||
"database",
|
||||
"source",
|
||||
"research",
|
||||
"attachment"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"unclassified": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/raw"
|
||||
}
|
||||
},
|
||||
"prior_cases": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/raw"
|
||||
}
|
||||
},
|
||||
"request_hash": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{64}$"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$defs": {
|
||||
"provenance": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"kind",
|
||||
"source_ref",
|
||||
"json_pointer",
|
||||
"author",
|
||||
"observed_at"
|
||||
],
|
||||
"properties": {
|
||||
"kind": {
|
||||
"enum": [
|
||||
"tester",
|
||||
"reproduction_record",
|
||||
"report"
|
||||
]
|
||||
},
|
||||
"source_ref": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"json_pointer": {
|
||||
"type": "string",
|
||||
"pattern": "^(|/.*)$"
|
||||
},
|
||||
"author": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"observed_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"format": "date-time"
|
||||
}
|
||||
}
|
||||
},
|
||||
"claim": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"key",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"key": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"value": {}
|
||||
}
|
||||
},
|
||||
"raw": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"path",
|
||||
"sha256",
|
||||
"size"
|
||||
],
|
||||
"properties": {
|
||||
"path": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"sha256": {
|
||||
"type": "string",
|
||||
"pattern": "^[0-9a-f]{64}$"
|
||||
},
|
||||
"size": {
|
||||
"type": "integer",
|
||||
"minimum": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
"record": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"id",
|
||||
"text",
|
||||
"provenance"
|
||||
],
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"text": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"provenance": {
|
||||
"$ref": "#/$defs/provenance"
|
||||
},
|
||||
"claim": {
|
||||
"$ref": "#/$defs/claim"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
{
|
||||
"type": "object",
|
||||
"required": [
|
||||
"format",
|
||||
"format_version"
|
||||
],
|
||||
"properties": {
|
||||
"format": {
|
||||
"const": "viano-bug-report"
|
||||
},
|
||||
"format_version": {
|
||||
"type": "integer",
|
||||
"const": 1
|
||||
},
|
||||
"title": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"summary": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"observations": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/statement"
|
||||
}
|
||||
},
|
||||
"comments": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/statement"
|
||||
}
|
||||
},
|
||||
"tester_hypotheses": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/statement"
|
||||
}
|
||||
},
|
||||
"expected_behavior": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/statement"
|
||||
}
|
||||
},
|
||||
"expected_behavior_complete": {
|
||||
"type": [
|
||||
"boolean",
|
||||
"null"
|
||||
]
|
||||
},
|
||||
"entities": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/entity"
|
||||
}
|
||||
}
|
||||
},
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$defs": {
|
||||
"claim": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"key",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"key": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"value": {}
|
||||
}
|
||||
},
|
||||
"statement": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"text"
|
||||
],
|
||||
"properties": {
|
||||
"text": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"author": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"observed_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"format": "date-time"
|
||||
},
|
||||
"claim": {
|
||||
"$ref": "#/$defs/claim"
|
||||
}
|
||||
}
|
||||
},
|
||||
"entity": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"type",
|
||||
"id"
|
||||
],
|
||||
"properties": {
|
||||
"type": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"id": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
{
|
||||
"type": "integer"
|
||||
}
|
||||
]
|
||||
},
|
||||
"roles": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,159 @@
|
||||
{
|
||||
"type": "object",
|
||||
"required": [
|
||||
"format",
|
||||
"format_version"
|
||||
],
|
||||
"properties": {
|
||||
"format": {
|
||||
"const": "viano-reproduction"
|
||||
},
|
||||
"format_version": {
|
||||
"type": "integer",
|
||||
"const": 1
|
||||
},
|
||||
"steps": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
}
|
||||
},
|
||||
"observations": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/statement"
|
||||
}
|
||||
},
|
||||
"events_ref": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"attachments": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"facts": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"name",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"value": {}
|
||||
}
|
||||
}
|
||||
},
|
||||
"entities": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/$defs/entity"
|
||||
}
|
||||
}
|
||||
},
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$defs": {
|
||||
"claim": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"key",
|
||||
"value"
|
||||
],
|
||||
"properties": {
|
||||
"key": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"value": {}
|
||||
}
|
||||
},
|
||||
"statement": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"text"
|
||||
],
|
||||
"properties": {
|
||||
"text": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"author": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
"observed_at": {
|
||||
"type": [
|
||||
"string",
|
||||
"null"
|
||||
],
|
||||
"format": "date-time"
|
||||
},
|
||||
"claim": {
|
||||
"$ref": "#/$defs/claim"
|
||||
}
|
||||
}
|
||||
},
|
||||
"entity": {
|
||||
"type": "object",
|
||||
"required": [
|
||||
"type",
|
||||
"id"
|
||||
],
|
||||
"properties": {
|
||||
"type": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
"id": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
},
|
||||
{
|
||||
"type": "integer"
|
||||
}
|
||||
]
|
||||
},
|
||||
"roles": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string",
|
||||
"minLength": 1,
|
||||
"pattern": "\\S"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,384 @@
|
||||
import json
|
||||
|
||||
import pytest
|
||||
from jsonschema import Draft202012Validator
|
||||
from importlib.resources import files
|
||||
|
||||
from viano_bug_analysis.case import CaseBuilder, CaseAssembler, CaseBuilderLimits, CaseBuildError, render_summary
|
||||
from viano_bug_analysis.lifecycle import CheckpointStore, LifecycleError, State, Workspace
|
||||
from viano_bug_analysis.package import PackageReader, PackageValidator, PackageWriter
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def case_setup(manifest, make_package, tmp_path):
|
||||
manifest["artifacts"]["case_inputs"] = "arbitrary/index.data"
|
||||
report = {"format": "viano-bug-report", "format_version": 1, "title": "Generic problem", "summary": "Reported behavior",
|
||||
"observations": [{"text": "The switch remains off", "author": "Tester A", "observed_at": "2026-01-01T01:00:00Z"}],
|
||||
"comments": [{"text": "Seen twice"}], "tester_hypotheses": [{"text": "Perhaps a timing issue"}],
|
||||
"expected_behavior": [{"text": "The switch turns on"}], "expected_behavior_complete": True,
|
||||
"entities": [{"type": "component", "id": 42, "roles": ["focus"]}]}
|
||||
reproduction = {"format": "viano-reproduction", "format_version": 1, "steps": ["Start", "Press switch"],
|
||||
"observations": [{"text": "Still off"}], "events_ref": "raw/events", "attachments": ["raw/binary"],
|
||||
"facts": [{"name": "attempt", "value": 2}], "entities": [{"type": "component", "id": "42", "roles": ["observed"]}]}
|
||||
index = {"format": "viano-case-inputs", "format_version": 1, "reports": ["anything/report.data"],
|
||||
"reproductions": ["anything/repro.data"], "context": {"source": ["raw/source"], "database": ["raw/db"], "runtime": ["raw/runtime"], "research": ["raw/research"]}}
|
||||
entries = {"arbitrary/index.data": json.dumps(index), "anything/report.data": json.dumps(report), "anything/repro.data": json.dumps(reproduction),
|
||||
"raw/events": b"many events", "raw/binary": b"\x00\xff", "raw/source": b"source raw", "raw/db": b"db raw", "raw/runtime": b"runtime raw", "raw/research": b"supplied external notes", "unknown/vendor": b"opaque"}
|
||||
source = make_package(entries=entries)
|
||||
store = CheckpointStore(Workspace(tmp_path / "service"))
|
||||
store.intake(source)
|
||||
return store, CaseBuilder(store), source, entries, report, index
|
||||
|
||||
|
||||
@pytest.mark.parametrize("contract", ["case", "case-inputs", "report", "reproduction"])
|
||||
def test_contract_schemas_valid(contract):
|
||||
schema = json.loads(files("viano_bug_analysis.schemas").joinpath(f"{contract}-v1.schema.json").read_text())
|
||||
Draft202012Validator.check_schema(schema)
|
||||
|
||||
|
||||
def test_full_build_preserves_separation_and_provenance(case_setup):
|
||||
store, builder, source, entries, _, _ = case_setup
|
||||
original = source.read_bytes()
|
||||
result = builder.build("case-A", "stable-package")
|
||||
assert not result.reused
|
||||
assert result.analysis_revision == 1
|
||||
assert PackageValidator().validate(result.package).valid
|
||||
assert source.read_bytes() == original
|
||||
with PackageReader(result.package) as reader:
|
||||
manifest = reader.manifest
|
||||
assert manifest["state"] == State.BUILDING_CASE
|
||||
assert manifest["package_generation"] == 3
|
||||
case = json.loads(reader.read_bytes(result.case_ref))
|
||||
assert case["case"]["source_generation"] == 2
|
||||
assert case["human_observations"]["observations"][0]["text"] == "The switch remains off"
|
||||
assert len(case["human_observations"]["observations"]) == 2
|
||||
assert case["human_observations"]["tester_hypotheses"][0]["text"] == "Perhaps a timing issue"
|
||||
assert case["expected_behavior"]["completeness"] == "reported_complete"
|
||||
assert case["human_observations"]["observations"][0]["provenance"] == {"kind": "tester", "source_ref": "anything/report.data", "json_pointer": "/observations/0", "author": "Tester A", "observed_at": "2026-01-01T01:00:00Z"}
|
||||
assert case["runtime_facts"]["facts"][0]["id"] == "F0001"
|
||||
assert case["runtime_facts"]["facts"][0]["provenance"]["json_pointer"] == "/facts/0"
|
||||
assert case["entities"][0]["id"] == "42"
|
||||
assert case["entities"][0]["roles"] == ["focus", "observed"]
|
||||
assert len(case["entities"][0]["provenance"]) == 2
|
||||
assert case["research"]["performed"] is False
|
||||
assert [item["path"] for item in case["inputs"]["unclassified"]] == ["unknown/vendor"]
|
||||
assert reader.read_bytes(result.summary_ref).decode() == render_summary(case)
|
||||
for path, value in entries.items():
|
||||
assert reader.read_bytes(path) == (value.encode() if isinstance(value, str) else value)
|
||||
|
||||
|
||||
def test_completed_build_is_idempotent(case_setup):
|
||||
store, builder, _, _, _, _ = case_setup
|
||||
first = builder.build("case-A", "stable-package")
|
||||
again = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package")
|
||||
assert again.reused and again.package == first.package
|
||||
assert len(list(first.package.parent.glob("g*.zip"))) == 3
|
||||
|
||||
|
||||
def test_missing_index_marks_gaps_without_guessing(make_package, tmp_path):
|
||||
store = CheckpointStore(Workspace(tmp_path / "service"))
|
||||
store.intake(make_package(entries={"original/bug_report.json": b"not a declared report"}))
|
||||
result = CaseBuilder(store).build("case-A", "stable-package")
|
||||
with PackageReader(result.package) as reader:
|
||||
case = json.loads(reader.read_bytes(result.case_ref))
|
||||
codes = {issue["code"] for issue in case["data_quality"]["issues"]}
|
||||
assert {"MISSING_INPUT_INDEX", "MISSING_REPORT", "MISSING_OBSERVATIONS", "MISSING_REPRODUCTION", "EXPECTED_BEHAVIOR_MISSING"} <= codes
|
||||
assert case["bug"]["reports"] == []
|
||||
assert case["inputs"]["unclassified"][0]["path"] == "original/bug_report.json"
|
||||
|
||||
|
||||
def rewrite_input(case_setup, tmp_path, changes, updates=None):
|
||||
store = case_setup[0]
|
||||
current = store.latest("case-A", "stable-package")
|
||||
incoming = tmp_path / "updated-input"
|
||||
PackageWriter().write(current, incoming, additions={name: json.dumps(data).encode() for name, data in changes.items()}, manifest_updates=updates)
|
||||
store.intake(incoming)
|
||||
return current
|
||||
|
||||
|
||||
def test_reopen_inherits_previous_reports_and_facts(case_setup, tmp_path):
|
||||
store, builder, _, _, _, _ = case_setup
|
||||
first = builder.build("case-A", "stable-package")
|
||||
for index, target in enumerate([State.INTERPRETING_CASE, State.GENERATING_HYPOTHESES, State.INVESTIGATING_HYPOTHESES, State.BUILDING_DIAGNOSIS, State.OUTPUT_READY, State.HANDED_OFF, State.REOPENED]):
|
||||
store.advance("case-A", "stable-package", target, step_id=f"test-stage-{index}")
|
||||
report = {"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "New feedback"}]}
|
||||
index = {"format": "viano-case-inputs", "format_version": 1, "reports": ["feedback/new-report"]}
|
||||
rewrite_input(case_setup, tmp_path, {"feedback/new-report": report, "inputs/new-index": index}, {"artifacts": {"case_inputs": "inputs/new-index"}})
|
||||
result = builder.build("case-A", "stable-package")
|
||||
assert result.analysis_revision == 2
|
||||
with PackageReader(result.package) as reader:
|
||||
new = json.loads(reader.read_bytes(result.case_ref))
|
||||
old = json.loads(reader.read_bytes(first.case_ref))
|
||||
assert new["human_observations"]["observations"][:2] == old["human_observations"]["observations"]
|
||||
assert new["human_observations"]["observations"][-1]["text"] == "New feedback"
|
||||
assert new["runtime_facts"]["facts"] == old["runtime_facts"]["facts"]
|
||||
assert new["inputs"]["prior_cases"][0]["path"] == first.case_ref
|
||||
assert reader.read_bytes(first.summary_ref) == render_summary(old).encode()
|
||||
|
||||
|
||||
def test_new_data_after_completion_requires_reopen(case_setup, tmp_path):
|
||||
_, builder, _, _, _, _ = case_setup
|
||||
builder.build("case-A", "stable-package")
|
||||
rewrite_input(case_setup, tmp_path, {"feedback/new": {"anything": "new"}})
|
||||
with pytest.raises(CaseBuildError) as error:
|
||||
builder.build("case-A", "stable-package")
|
||||
assert error.value.code == "STALE_CASE"
|
||||
|
||||
|
||||
def test_completion_crash_reuses_output(case_setup, monkeypatch):
|
||||
store, builder, _, _, _, _ = case_setup
|
||||
original_index = store._index
|
||||
calls = 0
|
||||
def crash(*args):
|
||||
nonlocal calls
|
||||
calls += 1
|
||||
if calls == 2:
|
||||
raise LifecycleError("SIMULATED_CRASH", "after case publication")
|
||||
original_index(*args)
|
||||
monkeypatch.setattr(store, "_index", crash)
|
||||
with pytest.raises(CaseBuildError):
|
||||
builder.build("case-A", "stable-package")
|
||||
result = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package")
|
||||
assert result.reused
|
||||
assert len(list(result.package.parent.glob("g*.zip"))) == 3
|
||||
|
||||
|
||||
def test_unconfirmed_build_retries_without_duplicate_start(case_setup, monkeypatch):
|
||||
store, builder, _, _, _, _ = case_setup
|
||||
original_write = store.writer.write
|
||||
calls = 0
|
||||
def crash(*args, **kwargs):
|
||||
nonlocal calls
|
||||
calls += 1
|
||||
if calls == 2:
|
||||
raise LifecycleError("SIMULATED_CRASH", "before case publication")
|
||||
return original_write(*args, **kwargs)
|
||||
monkeypatch.setattr(store.writer, "write", crash)
|
||||
with pytest.raises(CaseBuildError):
|
||||
builder.build("case-A", "stable-package")
|
||||
result = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package")
|
||||
assert not result.reused
|
||||
assert len(list(result.package.parent.glob("g*.zip"))) == 3
|
||||
|
||||
|
||||
def test_portable_completed_case(case_setup, tmp_path):
|
||||
_, builder, _, _, _, _ = case_setup
|
||||
first = builder.build("case-A", "stable-package")
|
||||
other = CheckpointStore(Workspace(tmp_path / "other-service"))
|
||||
other.intake(first.package)
|
||||
result = CaseBuilder(other).build("case-A", "stable-package")
|
||||
assert result.reused
|
||||
assert result.package == other.latest("case-A", "stable-package")
|
||||
|
||||
|
||||
def test_summary_deterministic_and_escapes_untrusted_text(case_setup):
|
||||
_, builder, _, _, _, _ = case_setup
|
||||
result = builder.build("case-A", "stable-package")
|
||||
with PackageReader(result.package) as reader:
|
||||
case = json.loads(reader.read_bytes(result.case_ref))
|
||||
case["human_observations"]["observations"][0]["text"] = "<script>alert(1)</script> [link](javascript:bad)\n# fake heading"
|
||||
summary = render_summary(case)
|
||||
assert summary == render_summary(case)
|
||||
assert "<script>" not in summary
|
||||
assert "[link](javascript:" not in summary
|
||||
assert "\n# fake heading" not in summary
|
||||
|
||||
|
||||
@pytest.mark.parametrize("limit,value,code", [("max_documents", 1, "STRUCTURED_INPUT_LIMIT"), ("max_structured_bytes", 50, "STRUCTURED_INPUT_LIMIT"), ("max_json_bytes", 10, "INPUT_JSON"), ("max_case_bytes", 50, "CASE_LIMIT"), ("max_summary_bytes", 10, "SUMMARY_LIMIT")])
|
||||
def test_operational_limits(case_setup, limit, value, code):
|
||||
store = case_setup[0]
|
||||
builder = CaseBuilder(store, CaseBuilderLimits(**{limit: value}))
|
||||
with pytest.raises(CaseBuildError) as error:
|
||||
builder.build("case-A", "stable-package")
|
||||
assert error.value.code == code
|
||||
with PackageReader(store.latest("case-A", "stable-package")) as reader:
|
||||
assert reader.manifest["artifacts"]["case"] is None
|
||||
|
||||
|
||||
def test_concurrent_change_rejects_prepared_case(case_setup, monkeypatch):
|
||||
store, builder, _, _, _, _ = case_setup
|
||||
original = builder.assembler.assemble
|
||||
def change(reader):
|
||||
case = original(reader)
|
||||
store.advance("case-A", "stable-package", State.BUILDING_CASE, step_id="concurrent-work", additions={"new-data": b"new"})
|
||||
return case
|
||||
monkeypatch.setattr(builder.assembler, "assemble", change)
|
||||
with pytest.raises(CaseBuildError) as error:
|
||||
builder.build("case-A", "stable-package")
|
||||
assert error.value.code == "GENERATION_CHANGED"
|
||||
with PackageReader(store.latest("case-A", "stable-package")) as reader:
|
||||
assert reader.manifest["artifacts"]["case"] is None
|
||||
|
||||
|
||||
def test_expected_generation_check_and_successful_replay(case_setup):
|
||||
store = case_setup[0]
|
||||
with pytest.raises(LifecycleError) as error:
|
||||
store.advance("case-A", "stable-package", State.BUILDING_CASE, step_id="prepared", expected_generation=0)
|
||||
assert error.value.code == "GENERATION_CHANGED"
|
||||
first = store.advance("case-A", "stable-package", State.BUILDING_CASE, step_id="prepared", expected_generation=1)
|
||||
assert store.advance("case-A", "stable-package", State.BUILDING_CASE, step_id="prepared", expected_generation=1) == first
|
||||
|
||||
|
||||
@pytest.mark.parametrize("change,code", [("version", "INPUT_SCHEMA"), ("path", "INPUT_REFERENCE"), ("missing", "INPUT_REFERENCE"), ("duplicate", "INPUT_DUPLICATE"), ("report", "INPUT_SCHEMA"), ("json", "INPUT_JSON")])
|
||||
def test_invalid_declared_input_is_controlled(change, code, manifest, make_package, tmp_path):
|
||||
manifest["artifacts"]["case_inputs"] = "index"
|
||||
index = {"format": "viano-case-inputs", "format_version": 1, "reports": ["report"]}
|
||||
report = {"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "observed"}]}
|
||||
if change == "version":
|
||||
index["format_version"] = 2
|
||||
elif change == "path":
|
||||
index["reports"] = ["../report"]
|
||||
elif change == "missing":
|
||||
index["reports"] = ["absent"]
|
||||
elif change == "duplicate":
|
||||
index["reports"] = ["report", "report"]
|
||||
elif change == "report":
|
||||
report["observations"] = "not a list"
|
||||
entries = {"index": json.dumps(index), "report": json.dumps(report) if change != "json" else '{"format":"viano-bug-report","format":"duplicate"}'}
|
||||
original = make_package(entries=entries)
|
||||
before = original.read_bytes()
|
||||
store = CheckpointStore(Workspace(tmp_path / "service"))
|
||||
store.intake(original)
|
||||
with pytest.raises(CaseBuildError) as error:
|
||||
CaseBuilder(store).build("case-A", "stable-package")
|
||||
assert error.value.code == code
|
||||
assert original.read_bytes() == before
|
||||
with PackageReader(store.latest("case-A", "stable-package")) as reader:
|
||||
assert reader.manifest["artifacts"]["case"] is None
|
||||
|
||||
|
||||
def test_explicit_conflicts_are_traced_without_semantic_guessing(manifest, make_package, tmp_path):
|
||||
manifest["artifacts"]["case_inputs"] = "index"
|
||||
reports = [
|
||||
{"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "Open"}], "expected_behavior": [{"text": "Should be open", "claim": {"key": "switch_state", "value": "open"}}], "expected_behavior_complete": True},
|
||||
{"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "Closed"}], "expected_behavior": [{"text": "Should be closed", "claim": {"key": "switch_state", "value": "closed"}}], "expected_behavior_complete": False},
|
||||
]
|
||||
index = {"format": "viano-case-inputs", "format_version": 1, "reports": ["a", "b"]}
|
||||
store = CheckpointStore(Workspace(tmp_path / "service"))
|
||||
store.intake(make_package(entries={"index": json.dumps(index), "a": json.dumps(reports[0]), "b": json.dumps(reports[1])}))
|
||||
result = CaseBuilder(store).build("case-A", "stable-package")
|
||||
with PackageReader(result.package) as reader:
|
||||
case = json.loads(reader.read_bytes(result.case_ref))
|
||||
conflicts = [issue for issue in case["data_quality"]["issues"] if issue["code"] == "CONFLICTING_CLAIMS"]
|
||||
assert len(conflicts) == 1
|
||||
assert "expected_behavior" in conflicts[0]["message"]
|
||||
assert conflicts[0]["refs"] == [{"source_ref": "a", "json_pointer": "/expected_behavior/0"}, {"source_ref": "b", "json_pointer": "/expected_behavior/0"}]
|
||||
assert case["expected_behavior"]["completeness"] == "conflicting"
|
||||
assert len(case["human_observations"]["observations"]) == 2
|
||||
|
||||
|
||||
def test_plain_language_disagreement_is_not_interpreted(manifest, make_package, tmp_path):
|
||||
manifest["artifacts"]["case_inputs"] = "index"
|
||||
report = {"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "The light is on"}, {"text": "The light is off"}], "tester_hypotheses": [{"text": "It must be broken"}]}
|
||||
store = CheckpointStore(Workspace(tmp_path / "service"))
|
||||
store.intake(make_package(entries={"index": json.dumps({"format": "viano-case-inputs", "format_version": 1, "reports": ["report"]}), "report": json.dumps(report)}))
|
||||
result = CaseBuilder(store).build("case-A", "stable-package")
|
||||
with PackageReader(result.package) as reader:
|
||||
case = json.loads(reader.read_bytes(result.case_ref))
|
||||
assert not any(issue["code"] == "CONFLICTING_CLAIMS" for issue in case["data_quality"]["issues"])
|
||||
assert case["runtime_facts"]["facts"] == []
|
||||
|
||||
|
||||
def test_deterministic_assembly_for_same_snapshot(case_setup):
|
||||
store = case_setup[0]
|
||||
path = store.advance("case-A", "stable-package", State.BUILDING_CASE, step_id="start")
|
||||
with PackageReader(path) as reader:
|
||||
assert CaseAssembler().assemble(reader) == CaseAssembler().assemble(reader)
|
||||
|
||||
|
||||
def test_completed_case_summary_limit_applies_on_reuse(case_setup):
|
||||
store, builder, _, _, _, _ = case_setup
|
||||
builder.build("case-A", "stable-package")
|
||||
with pytest.raises(CaseBuildError) as error:
|
||||
CaseBuilder(store, CaseBuilderLimits(max_summary_bytes=10)).build("case-A", "stable-package")
|
||||
assert error.value.code == "SUMMARY_LIMIT"
|
||||
|
||||
|
||||
def test_builder_rejects_unknown_case_and_wrong_phase(case_setup):
|
||||
store, builder, _, _, _, _ = case_setup
|
||||
with pytest.raises(CaseBuildError) as error:
|
||||
builder.build("unknown", "unknown")
|
||||
assert error.value.code == "CASE_NOT_FOUND"
|
||||
store.advance("case-A", "stable-package", State.FAILED_TO_ANALYZE, step_id="unavailable", reason="test")
|
||||
with pytest.raises(CaseBuildError) as error:
|
||||
builder.build("case-A", "stable-package")
|
||||
assert error.value.code == "INVALID_BUILD_STATE"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [0, -1, True, 1.5])
|
||||
def test_invalid_builder_limits(value):
|
||||
with pytest.raises(ValueError):
|
||||
CaseBuilderLimits(max_documents=value)
|
||||
|
||||
|
||||
def test_large_fact_values_remain_referenced(case_setup):
|
||||
store = case_setup[0]
|
||||
result = CaseBuilder(store, CaseBuilderLimits(max_inline_fact_bytes=1)).build("case-A", "stable-package")
|
||||
with PackageReader(result.package) as reader:
|
||||
case = json.loads(reader.read_bytes(result.case_ref))
|
||||
fact = case["runtime_facts"]["facts"][0]
|
||||
# JSON encoding of the integer 2 occupies one byte, so remains inline.
|
||||
assert fact["value"] == 2
|
||||
|
||||
|
||||
def test_large_fact_does_not_embed_raw_array(manifest, make_package, tmp_path):
|
||||
manifest["artifacts"]["case_inputs"] = "index"
|
||||
payload = list(range(100))
|
||||
reproduction = {"format": "viano-reproduction", "format_version": 1, "facts": [{"name": "raw_values", "value": payload}]}
|
||||
index = {"format": "viano-case-inputs", "format_version": 1, "reproductions": ["repro"]}
|
||||
store = CheckpointStore(Workspace(tmp_path / "service"))
|
||||
store.intake(make_package(entries={"index": json.dumps(index), "repro": json.dumps(reproduction)}))
|
||||
builder = CaseBuilder(store, CaseBuilderLimits(max_inline_fact_bytes=20))
|
||||
result = builder.build("case-A", "stable-package")
|
||||
with PackageReader(result.package) as reader:
|
||||
case = json.loads(reader.read_bytes(result.case_ref))
|
||||
fact = case["runtime_facts"]["facts"][0]
|
||||
assert "value" not in fact
|
||||
assert fact["value_ref"] == {"source_ref": "repro", "json_pointer": "/facts/0/value"}
|
||||
assert json.loads(reader.read_bytes("repro"))["facts"][0]["value"] == payload
|
||||
assert "referenced at repro /facts/0/value" in reader.read_bytes(result.summary_ref).decode()
|
||||
assert builder.build("case-A", "stable-package").reused
|
||||
|
||||
|
||||
def test_real_process_exit_after_case_publication(case_setup):
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
store = case_setup[0]
|
||||
script = """
|
||||
import os, sys
|
||||
from viano_bug_analysis.lifecycle import Workspace, CheckpointStore
|
||||
from viano_bug_analysis.case import CaseBuilder
|
||||
store = CheckpointStore(Workspace(sys.argv[1]))
|
||||
original = store._index
|
||||
def terminate(path, manifest):
|
||||
if manifest['artifacts'].get('case'):
|
||||
os._exit(24)
|
||||
original(path, manifest)
|
||||
store._index = terminate
|
||||
CaseBuilder(store).build('case-A', 'stable-package')
|
||||
"""
|
||||
env = dict(os.environ, PYTHONPATH=str(Path(__file__).resolve().parents[1] / "src"))
|
||||
process = subprocess.run([sys.executable, "-c", script, str(store.workspace.root)], env=env, capture_output=True, timeout=15)
|
||||
assert process.returncode == 24, process.stderr.decode()
|
||||
result = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package")
|
||||
assert result.reused
|
||||
assert len(list(result.package.parent.glob("g*.zip"))) == 3
|
||||
|
||||
|
||||
def test_start_replay_can_observe_already_completed_build(case_setup, monkeypatch):
|
||||
store, builder, _, _, _, _ = case_setup
|
||||
original = store.advance
|
||||
injected = False
|
||||
def complete_elsewhere(*args, **kwargs):
|
||||
nonlocal injected
|
||||
result = original(*args, **kwargs)
|
||||
if kwargs["step_id"].endswith(":start") and not injected:
|
||||
injected = True
|
||||
return CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package").package
|
||||
return result
|
||||
monkeypatch.setattr(store, "advance", complete_elsewhere)
|
||||
result = builder.build("case-A", "stable-package")
|
||||
assert result.reused
|
||||
assert len(list(result.package.parent.glob("g*.zip"))) == 3
|
||||
Reference in new issue
Block a user