diff --git a/README.md b/README.md index ce24131..f31c459 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,8 @@ The implemented v1 contract is documented in [docs/package-format-v1.md](docs/package-format-v1.md). Job state, checkpoints and inbox admission are documented in [docs/lifecycle-v1.md](docs/lifecycle-v1.md). +Generic case construction and its input contracts are documented in +[docs/case-builder-v1.md](docs/case-builder-v1.md). Requires Python 3.11 or newer. @@ -16,5 +18,5 @@ python -m venv .venv ``` The lifecycle foundation supports one-pass inbox processing and resumable package -checkpoints. It includes no analysis worker, CaseBuilder, LLM, +checkpoints and a deterministic generic CaseBuilder. It includes no analysis worker or LLM, case interpretation, hypothesis generation or investigation tools. diff --git a/docs/case-builder-v1.md b/docs/case-builder-v1.md new file mode 100644 index 0000000..31afc17 --- /dev/null +++ b/docs/case-builder-v1.md @@ -0,0 +1,152 @@ +# Generic CaseBuilder foundation v1 + +`architecture.md` remains authoritative. This milestone assembles supplied data +deterministically. It performs no LLM calls, interpretation, hypothesis generation, +diagnosis, investigation, external research or project-specific adapter work. +Tester hypotheses remain supplied data, never generated or promoted to facts. + +## Explicit input contracts + +The manifest may declare `artifacts.case_inputs`, pointing to a UTF-8 JSON index: + +```json +{ + "format": "viano-case-inputs", + "format_version": 1, + "reports": ["original/report.data"], + "reproductions": ["original/reproduction.data"], + "context": { + "runtime": ["original/runtime.bin"], + "database": [], + "source": [], + "research": [] + }, + "attachments": [] +} +``` + +Paths and extensions have no inherent meaning. Listed reports conform to +`report-v1.schema.json`, reproductions to `reproduction-v1.schema.json`, and the +index to `case-inputs-v1.schema.json`. Schemas are bundled offline Draft 2020-12 +contracts. Unlisted files are preserved as unclassified inputs without parsing +or guessing their role. A missing index produces explicit data-quality gaps. +Invalid declared versions/contracts, duplicate JSON keys, repeated sources within +an index role, unsafe or absent references are controlled errors. + +A report requires format `viano-bug-report` and format_version 1. Optional fields: +title, summary, observations, comments, tester_hypotheses, expected_behavior, +expected_behavior_complete and entities. Statements have nonblank `text`, optional +`author` and timezone-aware `observed_at`, and optionally a structured `claim` +with `key` and JSON `value`. Categories are producer-supplied, not inferred from +prose. Unknown extensions remain accessible in the preserved source file. + +```json +{ + "format": "viano-bug-report", + "format_version": 1, + "title": "A generic observed problem", + "observations": [{"text": "The switch remains off", "author": "Tester A"}], + "tester_hypotheses": [{"text": "Perhaps a timing issue"}], + "expected_behavior": [{"text": "The switch turns on"}], + "expected_behavior_complete": false, + "entities": [{"type": "component", "id": "42", "roles": ["focus"]}] +} +``` + +A reproduction requires format `viano-reproduction` and format_version 1. Optional +fields: steps (strings), observations (human statements), events_ref, attachments, +facts and entities. Facts have a name and JSON value: they are supplied facts, +not independently verified technical findings. Events, attachments and all context +files remain opaque references with hashes/sizes. Supplied research stays separate; +no research is performed and no local implementation facts are inferred from it. + +## Canonical case and provenance + +`case-v1.schema.json` defines format `viano-case`, format_version 1, builder_version +`1`, and the architecture's main blocks: case, bug, human_observations, +expected_behavior, reproduction, entities, runtime_facts, database_context, +source_context, research, data_quality, inputs. + +Identity/project/revision come only from the manifest. Each statement/fact/entity +has a source path and JSON Pointer with provenance kind. Human statements retain +author/time. Observations, comments, tester hypotheses and expected behavior are +separate. Record IDs use O/C/T/E/F namespaces; previous IDs remain stable by source +path and pointer when inputs are appended. Facts use F0001 etc. Reproduction IDs +follow source order. Integer entity IDs become strings; equal integer/string IDs +of the same explicit type combine roles and retain every source. No domain types, +relations or content are invented. + +Expected-behavior completeness is missing, unspecified, reported_complete, +reported_partial or conflicting. Reported completeness is an assertion by the +reporter, not a sufficiency judgment. Different canonical JSON values for the same +explicit claim key within observations or expected behavior are flagged with +source references. Natural-language contradictions are not interpreted; comments +and tester hypotheses never participate in fact or conflict inference. + +Large context/events stay outside case JSON. Fact values over the inline limit +use `value_ref` (source path and JSON Pointer) instead of `value`; nothing is +truncated or replaced by a synthetic null. Old structured inputs are inherited +from the prior case's role inventory, hash-checked, and re-read from their immutable +originals. A returning index can add only new reports without losing old statements, +facts or provenance. Prior case files remain explicitly referenced. Incompatible +prior canonical case contracts are rejected instead of guessed. + +## Artifact and checkpoint integration + +```python +from viano_bug_analysis.case import CaseBuilder, CaseBuildError +from viano_bug_analysis.lifecycle import Workspace, CheckpointStore + +store = CheckpointStore(Workspace("service-data")) +try: + result = CaseBuilder(store).build("case-A", "stable-package") +except CaseBuildError as error: + print(error.code, error.path, str(error)) +``` + +Input must already be admitted. VALIDATED or REOPENED enters BUILDING_CASE through +a start checkpoint; existing BUILDING_CASE work continues there. Artifacts use +`case/revisions/NNNN/case.json` and `case_summary.md`. Revision paths preserve +append-only history; manifest pointers `case` and `case_summary` select the current +documents. Existing case files and unknown artifacts are never overwritten. + +Case JSON, deterministic summary and the `case-built` completion checkpoint are +published together through PackageWriter. State remains BUILDING_CASE; construction +does not execute or start interpretation. Later phase orchestration advances it. +Summary is generated only from canonical JSON, adds no interpretation and escapes +supplied Markdown/HTML. `render_summary` is pure; `CaseAssembler.assemble` is +deterministic for the same validated snapshot. + +Completion uses a stable step ID per builder version and analysis revision. +Repeats check the input request hash, case contract and summary, reuse output and +repair local metadata. Crashes after publication do not create another generation; +unconfirmed work retries from the same snapshot. Changed payloads after completion +require reopening with a new analysis revision. The request hash includes explicit +source roles and all unclassified payloads, excluding known generated case outputs +and checkpoint artifacts. Analysis revisions and package generations stay separate. + +Prepared output calls `CheckpointStore.advance(expected_generation=...)` to reject +concurrent changes before committing. Identical confirmed requests remain replayable +with the old expected generation. Locks and this optimistic check prevent stale +prepared data from being committed over newer work. There are no external side +effects in construction. + +## Configurable resource defaults + +`CaseBuilderLimits` holds operational policy, not immutable format limits: + +| Limit | Default | +|---|---:| +| Individual index/report/reproduction JSON | 1 MiB | +| Aggregate structured input JSON | 8 MiB | +| Structured input documents | 256 | +| Prior/canonical case JSON | 8 MiB | +| Summary bytes | 8 MiB | +| Inline fact value | 16 KiB | + +Prior case JSON is bounded separately from the structured input budget. Package +limits still apply to the archive/artifacts. Context is hashed in chunks rather +than loaded as prose. Invalid attempts preserve all source/previous case files; +they may retain a confirmed start checkpoint in BUILDING_CASE, allowing resumption +after corrected sources and a new index are appended. Changing resource defaults +does not rewrite a previously confirmed case. diff --git a/docs/lifecycle-v1.md b/docs/lifecycle-v1.md index e0c00a0..3b9cebf 100644 --- a/docs/lifecycle-v1.md +++ b/docs/lifecycle-v1.md @@ -2,7 +2,8 @@ `architecture.md` remains the authoritative design. This milestone provides only deterministic job state, checkpoint storage, locking and inbox admission. It does -not execute CaseBuilder, interpretation, hypotheses, diagnosis, tools or LLMs. +not execute interpretation, hypotheses, diagnosis, tools or LLMs. The separate +CaseBuilder module now uses these checkpoint APIs for deterministic construction. State APIs allow future phase implementations to confirm their work; recording a state is not evidence that this foundation executed that phase. diff --git a/src/viano_bug_analysis/case/__init__.py b/src/viano_bug_analysis/case/__init__.py new file mode 100644 index 0000000..37123f9 --- /dev/null +++ b/src/viano_bug_analysis/case/__init__.py @@ -0,0 +1,6 @@ +from .assembler import CaseAssembler, CaseBuilderLimits +from .builder import CaseBuilder, CaseBuildResult +from .errors import CaseBuildError +from .summary import render_summary + +__all__ = ["CaseAssembler", "CaseBuilderLimits", "CaseBuilder", "CaseBuildResult", "CaseBuildError", "render_summary"] diff --git a/src/viano_bug_analysis/case/assembler.py b/src/viano_bug_analysis/case/assembler.py new file mode 100644 index 0000000..4997749 --- /dev/null +++ b/src/viano_bug_analysis/case/assembler.py @@ -0,0 +1,234 @@ +import copy +import hashlib +import re +from dataclasses import dataclass + +from viano_bug_analysis.lifecycle.checkpoints import checkpoint_chain +from viano_bug_analysis.lifecycle.storage import json_bytes +from viano_bug_analysis.package.validator import CHUNK_SIZE + +from .contracts import read_json, require_file, validate +from .errors import CaseBuildError +from .expected_behavior import collect_expected_behavior +from .report_collector import collect_statements, provenance +from .reproduction_collector import collect_reproductions + + +def _stable_records(previous: list[dict], current: list[dict], prefix: str) -> list[dict]: + def identity(record): + source = record["provenance"] + return source["source_ref"], source["json_pointer"] + remaining = {identity(record): record for record in current} + result = [] + ids = set() + for old in previous: + if not re.fullmatch(prefix + r"[0-9]{4,}", old["id"]) or old["id"] in ids: + raise CaseBuildError("PREVIOUS_IDS", "Prior case contains invalid or duplicate record IDs") + ids.add(old["id"]) + record = remaining.pop(identity(old), None) + if record is None: + raise CaseBuildError("PREVIOUS_RECORD_LOST", "Prior source record disappeared from its structured input") + record["id"] = old["id"] + result.append(record) + next_id = max((int(value[len(prefix):]) for value in ids), default=0) + 1 + for record in current: + if identity(record) in remaining: + record["id"] = f"{prefix}{next_id:04d}" + next_id += 1 + result.append(record) + del remaining[identity(record)] + return result + + +@dataclass(frozen=True) +class CaseBuilderLimits: + """Operational limits, independent of the JSON format contracts.""" + + max_json_bytes: int = 1024**2 + max_structured_bytes: int = 8 * 1024**2 + max_documents: int = 256 + max_case_bytes: int = 8 * 1024**2 + max_summary_bytes: int = 8 * 1024**2 + max_inline_fact_bytes: int = 16 * 1024 + + def __post_init__(self): + for value in vars(self).values(): + if isinstance(value, bool) or not isinstance(value, int) or value <= 0: + raise ValueError("CaseBuilder limits must be positive integers") + + +class CaseAssembler: + def __init__(self, limits: CaseBuilderLimits | None = None): + self.limits = limits or CaseBuilderLimits() + + def assemble(self, reader) -> dict: + manifest = reader.manifest + if manifest["analysis_revision"] < 1: + raise CaseBuildError("INVALID_REVISION", "Case construction requires an active analysis revision") + used_bytes = 0 + documents = 0 + + def load(path, contract): + nonlocal used_bytes, documents + require_file(reader, path) + if contract == "case": + return read_json(reader, path, contract, self.limits.max_case_bytes) + documents += 1 + used_bytes += reader._archive.getinfo(path).file_size + if documents > self.limits.max_documents or used_bytes > self.limits.max_structured_bytes: + raise CaseBuildError("STRUCTURED_INPUT_LIMIT", "Declared structured inputs exceed configured limits", path) + return read_json(reader, path, contract, self.limits.max_json_bytes) + + chain = checkpoint_chain(reader) + excluded = {"manifest.json"} | {ref for ref, _ in chain} + prior_refs = [] + for _, checkpoint in reversed(chain): + details = checkpoint["details"] + if details.get("operation") == "case-built" and details.get("builder_version") == "1": + for field in ("case_ref", "summary_ref"): + if not isinstance(details.get(field), str): + raise CaseBuildError("PREVIOUS_CASE", "Incomplete case completion checkpoint") + require_file(reader, details[field]) + excluded.add(details[field]) + if details.get("case_ref") not in prior_refs: + prior_refs.append(details["case_ref"]) + prior_ref = manifest["artifacts"].get("case") + if prior_ref: + excluded.add(prior_ref) + if prior_ref not in prior_refs: + prior_refs.append(prior_ref) + summary_ref = manifest["artifacts"].get("case_summary") + if summary_ref: + excluded.add(summary_ref) + previous = load(prior_ref, "case") if prior_ref else None + if previous and (previous["case"]["case_id"] != manifest["case_id"] or previous["case"]["package_id"] != manifest["package_id"] or previous["case"]["analysis_revision"] > manifest["analysis_revision"]): + raise CaseBuildError("CASE_IDENTITY", "Previous case has inconsistent identity or revision", prior_ref) + inventory = {} + for name in sorted(reader.files): + if name == "manifest.json" or name in {ref for ref, _ in chain}: + continue + digest = hashlib.sha256() + size = 0 + with reader._archive.open(name) as stream: + while chunk := stream.read(CHUNK_SIZE): + size += len(chunk) + if size > reader.validator.limits.max_file_bytes: + raise CaseBuildError("INPUT_LIMIT", "Payload exceeds configured file limit", name) + digest.update(chunk) + inventory[name] = {"path": name, "sha256": digest.hexdigest(), "size": size} + index_ref = manifest["artifacts"].get("case_inputs") + index = load(index_ref, "case-inputs") if index_ref else {} + roles = [] + + def add(path, role): + require_file(reader, path) + if path in excluded: + raise CaseBuildError("INPUT_REFERENCE", "Generated case or checkpoint cannot serve as raw input", path) + if (path, role) not in roles: + roles.append((path, role)) + + if previous: + for record in previous["inputs"]["records"]: + add(record["path"], record["role"]) + actual = inventory[record["path"]] + if record["sha256"] != actual["sha256"] or record["size"] != actual["size"]: + raise CaseBuildError("PREVIOUS_INPUT_CHANGED", "A source referenced by the prior case changed", record["path"]) + if index_ref: + add(index_ref, "index") + for field, role in (("reports", "report"), ("reproductions", "reproduction"), ("attachments", "attachment")): + paths = index.get(field, []) + if len(paths) != len(set(paths)): + raise CaseBuildError("INPUT_DUPLICATE", "A source is listed twice in the same role", index_ref) + for path in paths: + add(path, role) + for role, paths in index.get("context", {}).items(): + if role not in {"runtime", "database", "source", "research"}: + continue + if len(paths) != len(set(paths)): + raise CaseBuildError("INPUT_DUPLICATE", "A context source is listed twice", index_ref) + for path in paths: + add(path, role) + reports = [(path, load(path, "report")) for path, role in roles if role == "report"] + reproductions = [(path, load(path, "reproduction")) for path, role in roles if role == "reproduction"] + for _, document in reproductions: + for path in document.get("attachments", []): + add(path, "attachment") + if document.get("events_ref"): + add(document["events_ref"], "attachment") + observations = collect_statements(reports + reproductions, "observations", "O") + comments = collect_statements(reports, "comments", "C") + hypotheses = collect_statements(reports, "tester_hypotheses", "T") + expected = collect_expected_behavior(reports) + reproduction_records, facts = collect_reproductions(reproductions, self.limits.max_inline_fact_bytes) + if previous: + old_humans = previous["human_observations"] + observations = _stable_records(old_humans["observations"], observations, "O") + comments = _stable_records(old_humans["comments"], comments, "C") + hypotheses = _stable_records(old_humans["tester_hypotheses"], hypotheses, "T") + expected["statements"] = _stable_records(previous["expected_behavior"]["statements"], expected["statements"], "E") + facts = _stable_records(previous["runtime_facts"]["facts"], facts, "F") + entities = {} + for path, document in reports + reproductions: + for index, entity in enumerate(document.get("entities", [])): + entity_id = entity["id"] if isinstance(entity["id"], str) else str(int(entity["id"])) + identity = (entity["type"], entity_id) + record = entities.setdefault(identity, {"type": identity[0], "id": identity[1], "roles": [], "provenance": []}) + record["roles"] = sorted(set(record["roles"]) | set(entity.get("roles", []))) + record["provenance"].append(provenance(path, f"/entities/{index}", "report" if document["format"] == "viano-bug-report" else "reproduction_record")) + issues = [] + + def issue(code, message, refs=None): + issues.append({"code": code, "message": message, "refs": refs or []}) + + if not index_ref: + issue("MISSING_INPUT_INDEX", "No explicit case input index was supplied") + if not reports: + issue("MISSING_REPORT", "No structured report was supplied") + if not observations: + issue("MISSING_OBSERVATIONS", "No direct observations were supplied") + if not reproductions: + issue("MISSING_REPRODUCTION", "No structured reproduction was supplied") + if expected["completeness"] != "reported_complete": + issue("EXPECTED_BEHAVIOR_" + expected["completeness"].upper(), "Expected-behavior completeness is " + expected["completeness"], + [{"source_ref": path, "json_pointer": "/expected_behavior_complete"} for path, report in reports if "expected_behavior_complete" in report]) + for category, records in (("observation", observations), ("expected_behavior", expected["statements"])): + claims = {} + for record in records: + if "claim" in record: + claim = record["claim"] + claims.setdefault(claim["key"], []).append(record) + for claim_key, records in sorted(claims.items()): + if len({json_bytes(record["claim"]["value"]) for record in records}) > 1: + refs = [{"source_ref": r["provenance"]["source_ref"], "json_pointer": r["provenance"]["json_pointer"]} for r in records] + issue("CONFLICTING_CLAIMS", f"Different explicit values for {category} claim {claim_key}", refs) + if category == "expected_behavior": + expected["completeness"] = "conflicting" + records = [inventory[path] | {"role": role} for path, role in roles] + classified = {path for path, _ in roles} + raw_inventory = [value for name, value in inventory.items() if name not in excluded] + request = {"builder_version": "1", "case_id": manifest["case_id"], "package_id": manifest["package_id"], + "analysis_revision": int(manifest["analysis_revision"]), "project": manifest["project"], + "index_ref": index_ref, "payloads": raw_inventory, "roles": roles} + request_hash = hashlib.sha256(json_bytes(request)).hexdigest() + refs_for = lambda role: [inventory[path] for path, value in roles if value == role] + case = { + "format": "viano-case", "format_version": 1, "builder_version": "1", + "case": {"case_id": manifest["case_id"], "package_id": manifest["package_id"], "analysis_revision": int(manifest["analysis_revision"]), + "source_generation": int(manifest["package_generation"]), "project": copy.deepcopy(manifest["project"])}, + "bug": {"reports": [{"source_ref": path, "title": document.get("title"), "summary": document.get("summary")} for path, document in reports]}, + "human_observations": {"observations": observations, "comments": comments, "tester_hypotheses": hypotheses}, + "expected_behavior": expected, + "reproduction": {"records": reproduction_records}, + "entities": [entities[identity] for identity in sorted(entities)], + "runtime_facts": {"facts": facts, "raw_refs": refs_for("runtime")}, + "database_context": {"refs": refs_for("database")}, "source_context": {"refs": refs_for("source")}, + "research": {"refs": refs_for("research"), "performed": False}, + "data_quality": {"issues": issues, "coverage": {"reports": len(reports), "reproductions": len(reproductions), "observations": len(observations)}}, + "inputs": {"index_ref": index_ref, "records": records, + "unclassified": [value for name, value in inventory.items() if name not in classified and name not in excluded], + "prior_cases": [inventory[ref] for ref in prior_refs], "request_hash": request_hash}, + } + validate(case, "case") + if len(json_bytes(case)) > self.limits.max_case_bytes: + raise CaseBuildError("CASE_LIMIT", "Assembled case exceeds configured byte limit") + return case diff --git a/src/viano_bug_analysis/case/builder.py b/src/viano_bug_analysis/case/builder.py new file mode 100644 index 0000000..7d92431 --- /dev/null +++ b/src/viano_bug_analysis/case/builder.py @@ -0,0 +1,91 @@ +from dataclasses import dataclass +from pathlib import Path + +from viano_bug_analysis.lifecycle import LifecycleError, State +from viano_bug_analysis.lifecycle.checkpoints import checkpoint_chain +from viano_bug_analysis.lifecycle.storage import json_bytes +from viano_bug_analysis.package import PackageReader, PackageReadError +from viano_bug_analysis.package.validator import ARCHIVE_ERRORS + +from .assembler import CaseAssembler, CaseBuilderLimits +from .contracts import read_json +from .errors import CaseBuildError +from .summary import render_summary + + +@dataclass(frozen=True) +class CaseBuildResult: + package: Path + case_ref: str + summary_ref: str + analysis_revision: int + reused: bool + + +class CaseBuilder: + def __init__(self, store, limits: CaseBuilderLimits | None = None): + self.store = store + self.assembler = CaseAssembler(limits) + + def build(self, case_id: str, package_id: str) -> CaseBuildResult: + try: + latest = self.store.latest(case_id, package_id) + if latest is None: + raise CaseBuildError("CASE_NOT_FOUND", "Case must be admitted before construction") + with PackageReader(latest, self.store.limits) as reader: + manifest = reader.manifest + revision = int(manifest["analysis_revision"]) + revision = revision or 1 + complete_id = f"case-builder:v1:revision:{revision}:complete" + completed = next((cp for _, cp in checkpoint_chain(reader) if cp["step_id"] == complete_id), None) + if completed: + assembled = self.assembler.assemble(reader) + details = completed["details"] + if details.get("input_request_hash") != assembled["inputs"]["request_hash"]: + raise CaseBuildError("STALE_CASE", "Inputs changed after case completion; reopen as a new analysis revision") + case_ref, summary_ref = details.get("case_ref"), details.get("summary_ref") + stored = read_json(reader, case_ref, "case", self.assembler.limits.max_case_bytes) + summary = render_summary(stored).encode("utf-8") + if len(summary) > self.assembler.limits.max_summary_bytes: + raise CaseBuildError("SUMMARY_LIMIT", "Summary exceeds configured byte limit") + with reader._archive.open(summary_ref) as stream: + stored_summary = reader.validator._read_bounded(stream, min(self.assembler.limits.max_summary_bytes, self.store.limits.max_file_bytes)) + if reader.manifest["artifacts"].get("case") != case_ref or reader.manifest["artifacts"].get("case_summary") != summary_ref or stored_summary != summary: + raise CaseBuildError("CASE_OUTPUT_CHANGED", "Confirmed case pointers or summary no longer match") + # Repair reconstructible metadata after a crash, without recomputing a commit. + recovered = self.store.advance(case_id, package_id, State.BUILDING_CASE, step_id=complete_id, + details=details, additions={case_ref: json_bytes(stored), summary_ref: summary}, + artifacts={"case": case_ref, "case_summary": summary_ref}) + if recovered != latest: + raise CaseBuildError("GENERATION_CHANGED", "Case changed while confirmed output was checked") + return CaseBuildResult(latest, case_ref, summary_ref, revision, True) + if manifest["state"] not in {State.VALIDATED, State.REOPENED, State.BUILDING_CASE}: + raise CaseBuildError("INVALID_BUILD_STATE", "CaseBuilder requires VALIDATED, REOPENED or BUILDING_CASE") + source_generation = int(manifest["package_generation"]) + if manifest["state"] != State.BUILDING_CASE: + latest = self.store.advance(case_id, package_id, State.BUILDING_CASE, + step_id=f"case-builder:v1:revision:{revision}:start", + details={"operation": "case-building", "builder_version": "1"}, + expected_generation=source_generation) + with PackageReader(latest, self.store.limits) as reader: + if any(cp["step_id"] == complete_id for _, cp in checkpoint_chain(reader)): + return self.build(case_id, package_id) + case = self.assembler.assemble(reader) + revision = case["case"]["analysis_revision"] + case_ref = f"case/revisions/{revision:04d}/case.json" + summary_ref = f"case/revisions/{revision:04d}/case_summary.md" + summary = render_summary(case).encode("utf-8") + if len(summary) > self.assembler.limits.max_summary_bytes: + raise CaseBuildError("SUMMARY_LIMIT", "Summary exceeds configured byte limit") + details = {"operation": "case-built", "builder_version": "1", "case_ref": case_ref, "summary_ref": summary_ref, + "input_request_hash": case["inputs"]["request_hash"]} + output = self.store.advance(case_id, package_id, State.BUILDING_CASE, + step_id=f"case-builder:v1:revision:{revision}:complete", details=details, + additions={case_ref: json_bytes(case), summary_ref: summary}, + artifacts={"case": case_ref, "case_summary": summary_ref}, + expected_generation=int(reader.manifest["package_generation"])) + return CaseBuildResult(output, case_ref, summary_ref, revision, False) + except (PackageReadError, LifecycleError) as error: + raise CaseBuildError(getattr(error, "code", "PACKAGE_ERROR"), str(error)) from error + except ARCHIVE_ERRORS + (TypeError, KeyError, RecursionError) as error: + raise CaseBuildError("CASE_BUILD_ERROR", str(error)) from error diff --git a/src/viano_bug_analysis/case/contracts.py b/src/viano_bug_analysis/case/contracts.py new file mode 100644 index 0000000..1c8f067 --- /dev/null +++ b/src/viano_bug_analysis/case/contracts.py @@ -0,0 +1,35 @@ +import json +from importlib.resources import files + +from jsonschema import Draft202012Validator +from jsonschema.exceptions import ValidationError + +from viano_bug_analysis.package import PackageReadError +from viano_bug_analysis.package.manifest import FORMAT_CHECKER, parse_manifest, safe_path + +from .errors import CaseBuildError + + +def validate(document: dict, contract: str, path: str | None = None) -> None: + schema = json.loads(files("viano_bug_analysis.schemas").joinpath(f"{contract}-v1.schema.json").read_text()) + try: + Draft202012Validator(schema, format_checker=FORMAT_CHECKER).validate(document) + except ValidationError as error: + raise CaseBuildError("INPUT_SCHEMA" if contract != "case" else "CASE_SCHEMA", error.message, path) from error + + +def read_json(reader, path: str, contract: str, limit: int) -> dict: + require_file(reader, path) + try: + with reader._archive.open(path) as stream: + data = reader.validator._read_bounded(stream, min(limit, reader.validator.limits.max_file_bytes)) + document = parse_manifest(data) + validate(document, contract, path) + return document + except (ValueError, RecursionError, PackageReadError) as error: + raise CaseBuildError("INPUT_JSON", str(error), path) from error + + +def require_file(reader, path: str) -> None: + if not safe_path(path) or path == "manifest.json" or path not in reader.files: + raise CaseBuildError("INPUT_REFERENCE", "Expected a safe existing payload file", path) diff --git a/src/viano_bug_analysis/case/errors.py b/src/viano_bug_analysis/case/errors.py new file mode 100644 index 0000000..25f1c1e --- /dev/null +++ b/src/viano_bug_analysis/case/errors.py @@ -0,0 +1,7 @@ +class CaseBuildError(Exception): + """Controlled failure of a declared input or case construction request.""" + + def __init__(self, code: str, message: str, path: str | None = None): + self.code = code + self.path = path + super().__init__(message) diff --git a/src/viano_bug_analysis/case/expected_behavior.py b/src/viano_bug_analysis/case/expected_behavior.py new file mode 100644 index 0000000..f54e78f --- /dev/null +++ b/src/viano_bug_analysis/case/expected_behavior.py @@ -0,0 +1,19 @@ +from .report_collector import collect_statements + + +def collect_expected_behavior(reports: list[tuple[str, dict]]) -> dict: + statements = collect_statements(reports, "expected_behavior", "E") + flags = {document.get("expected_behavior_complete") for _, document in reports + if document.get("expected_behavior")} + if not statements: + status = "missing" + elif True in flags and False in flags: + status = "conflicting" + elif False in flags: + status = "reported_partial" + elif flags == {True}: + status = "reported_complete" + else: + status = "unspecified" + return {"statements": statements, "completeness": status, + "provenance": "tester" if statements else "none"} diff --git a/src/viano_bug_analysis/case/report_collector.py b/src/viano_bug_analysis/case/report_collector.py new file mode 100644 index 0000000..6585259 --- /dev/null +++ b/src/viano_bug_analysis/case/report_collector.py @@ -0,0 +1,19 @@ +import copy + + +def provenance(path: str, pointer: str, kind: str, statement: dict | None = None) -> dict: + value = statement or {} + return {"kind": kind, "source_ref": path, "json_pointer": pointer, + "author": value.get("author"), "observed_at": value.get("observed_at")} + + +def collect_statements(documents: list[tuple[str, dict]], field: str, prefix: str) -> list[dict]: + result = [] + for path, document in documents: + for index, statement in enumerate(document.get(field, [])): + record = {"id": f"{prefix}{len(result) + 1:04d}", "text": statement["text"], + "provenance": provenance(path, f"/{field}/{index}", "tester", statement)} + if "claim" in statement: + record["claim"] = copy.deepcopy(statement["claim"]) + result.append(record) + return result diff --git a/src/viano_bug_analysis/case/reproduction_collector.py b/src/viano_bug_analysis/case/reproduction_collector.py new file mode 100644 index 0000000..62e1a52 --- /dev/null +++ b/src/viano_bug_analysis/case/reproduction_collector.py @@ -0,0 +1,22 @@ +import copy + +from viano_bug_analysis.lifecycle.storage import json_bytes + +from .report_collector import provenance + + +def collect_reproductions(documents: list[tuple[str, dict]], inline_limit: int) -> tuple[list[dict], list[dict]]: + records, facts = [], [] + for path, document in documents: + records.append({"id": f"R{len(records) + 1:04d}", "source_ref": path, + "steps": list(document.get("steps", [])), "events_ref": document.get("events_ref"), + "attachments": list(document.get("attachments", []))}) + for index, fact in enumerate(document.get("facts", [])): + record = {"id": f"F{len(facts) + 1:04d}", "name": fact["name"], + "provenance": provenance(path, f"/facts/{index}", "reproduction_record")} + if len(json_bytes(fact["value"])) > inline_limit: + record["value_ref"] = {"source_ref": path, "json_pointer": f"/facts/{index}/value"} + else: + record["value"] = copy.deepcopy(fact["value"]) + facts.append(record) + return records, facts diff --git a/src/viano_bug_analysis/case/summary.py b/src/viano_bug_analysis/case/summary.py new file mode 100644 index 0000000..fc89676 --- /dev/null +++ b/src/viano_bug_analysis/case/summary.py @@ -0,0 +1,76 @@ +import html +import json +import re + +from .contracts import validate + + +def _literal(value) -> str: + text = html.escape(str(value), quote=False) + text = re.sub(r"([\\`*_{}\[\]()#+.!|>~-])", r"\\\1", text) + return text.replace("\r", "\\r").replace("\n", "
") + + +def render_summary(case: dict) -> str: + """Pure presentation of the canonical case, with no additional interpretation.""" + validate(case, "case") + identity = case["case"] + lines = ["# Case summary", "", f"Case: {_literal(identity['case_id'])}", + f"Package: {_literal(identity['package_id'])}", f"Analysis revision: {identity['analysis_revision']}", + "", "## Reports", ""] + for report in case["bug"]["reports"]: + lines += [f"- Source: {_literal(report['source_ref'])}", + f" Title: {_literal(report['title']) if report['title'] is not None else '(not supplied)'}", + f" Summary: {_literal(report['summary']) if report['summary'] is not None else '(not supplied)'}"] + for label, records in (("Direct human observations", case["human_observations"]["observations"]), + ("Tester comments", case["human_observations"]["comments"]), + ("Tester hypotheses (supplied)", case["human_observations"]["tester_hypotheses"]), + ("Expected behavior (supplied)", case["expected_behavior"]["statements"])): + lines += ["", f"## {label}", ""] + for record in records: + source = record["provenance"] + lines += [f"- {_literal(record['id'])}: {_literal(record['text'])}", + f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}", + f" Author: {_literal(source['author']) if source['author'] is not None else '(not supplied)'}", + f" Observed at: {_literal(source['observed_at']) if source['observed_at'] is not None else '(not supplied)'}"] + if "claim" in record: + lines.append(f" Claim: {_literal(record['claim']['key'])} = {_literal(json.dumps(record['claim']['value'], sort_keys=True, ensure_ascii=False))}") + lines += ["", f"Expected-behavior completeness: {case['expected_behavior']['completeness']}", "", "## Reproductions", ""] + for record in case["reproduction"]["records"]: + lines.append(f"- {_literal(record['id'])}: {_literal(record['source_ref'])}") + for step in record["steps"]: + lines.append(f" Step: {_literal(step)}") + if record["events_ref"]: + lines.append(f" Events: {_literal(record['events_ref'])}") + for path in record["attachments"]: + lines.append(f" Attachment: {_literal(path)}") + lines += ["", "## Entities", ""] + for entity in case["entities"]: + lines.append(f"- {_literal(entity['type'])} / {_literal(entity['id'])}; roles: {_literal(', '.join(entity['roles']))}") + for source in entity["provenance"]: + lines.append(f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}") + lines += ["", "## Facts supplied by reproduction records", ""] + for fact in case["runtime_facts"]["facts"]: + source = fact["provenance"] + if "value_ref" in fact: + ref = fact["value_ref"] + value = f"referenced at {_literal(ref['source_ref'])} {_literal(ref['json_pointer'])}" + else: + value = _literal(json.dumps(fact['value'], sort_keys=True, ensure_ascii=False)) + lines.append(f"- {_literal(fact['id'])}: {_literal(fact['name'])} = {value}") + lines.append(f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}") + for label, refs in (("Runtime data", case["runtime_facts"]["raw_refs"]), ("Database context", case["database_context"]["refs"]), + ("Source context", case["source_context"]["refs"]), ("Supplied research", case["research"]["refs"]), + ("Prior cases", case["inputs"]["prior_cases"]), ("Unclassified inputs", case["inputs"]["unclassified"])): + lines += ["", f"## {label}", ""] + for ref in refs: + lines.append(f"- {_literal(ref['path'])} ({ref['size']} bytes; SHA-256: {ref['sha256']})") + lines += ["", "## Data quality", ""] + for issue in case["data_quality"]["issues"]: + lines.append(f"- {_literal(issue['code'])}: {_literal(issue['message'])}") + for source in issue["refs"]: + lines.append(f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}") + lines += ["", "## Input records", ""] + for record in case["inputs"]["records"]: + lines.append(f"- {_literal(record['role'])}: {_literal(record['path'])} ({record['size']} bytes; SHA-256: {record['sha256']})") + return "\n".join(lines) + "\n" diff --git a/src/viano_bug_analysis/lifecycle/checkpoints.py b/src/viano_bug_analysis/lifecycle/checkpoints.py index f4f98b7..68012ff 100644 --- a/src/viano_bug_analysis/lifecycle/checkpoints.py +++ b/src/viano_bug_analysis/lifecycle/checkpoints.py @@ -234,7 +234,8 @@ class CheckpointStore: def advance(self, case_id: str, package_id: str, target: str, *, step_id: str, reason: str | None = None, retry_not_before: str | None = None, details: dict | None = None, additions: dict[str, bytes] | None = None, - artifacts: dict[str, str | None] | None = None) -> Path: + artifacts: dict[str, str | None] | None = None, + expected_generation: int | None = None) -> Path: try: if not isinstance(step_id, str) or not step_id.strip(): raise LifecycleError("INVALID_STEP", "A nonblank stable step_id is required") @@ -242,6 +243,8 @@ class CheckpointStore: raise LifecycleError("CHECKPOINT_INVALID", "Checkpoint details must be a JSON object") if CHECKPOINT_POINTER in (artifacts or {}): raise LifecycleError("RESERVED_ARTIFACT", "Lifecycle checkpoint pointer is service-managed") + if expected_generation is not None and (isinstance(expected_generation, bool) or not isinstance(expected_generation, int) or expected_generation < 0): + raise LifecycleError("INVALID_GENERATION", "expected_generation must be a nonnegative integer") payload = {"target": target, "reason": reason, "retry_not_before": retry_not_before, "details": details or {}, "artifacts": artifacts or {}, "additions": {name: hashlib.sha256(data).hexdigest() for name, data in (additions or {}).items()}} @@ -260,6 +263,8 @@ class CheckpointStore: self._index(latest, manifest) atomic_json(self.workspace.runtime / "current_job.json", None) return latest + if expected_generation is not None and manifest["package_generation"] != expected_generation: + raise LifecycleError("GENERATION_CHANGED", "Case changed while artifacts were being prepared") resume = chain[0][1]["resume_state"] if manifest["state"] == State.BLOCKED else None check_transition(manifest["state"], target, resume) if manifest["state"] == State.BLOCKED and not self.retry_due(manifest): diff --git a/src/viano_bug_analysis/schemas/case-inputs-v1.schema.json b/src/viano_bug_analysis/schemas/case-inputs-v1.schema.json new file mode 100644 index 0000000..cfec405 --- /dev/null +++ b/src/viano_bug_analysis/schemas/case-inputs-v1.schema.json @@ -0,0 +1,72 @@ +{ + "type": "object", + "required": [ + "format", + "format_version" + ], + "properties": { + "format": { + "const": "viano-case-inputs" + }, + "format_version": { + "type": "integer", + "const": 1 + }, + "reports": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + }, + "reproductions": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + }, + "context": { + "type": "object", + "required": [], + "properties": { + "runtime": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + }, + "database": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + }, + "source": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + }, + "research": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + } + } + }, + "attachments": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + } + }, + "$schema": "https://json-schema.org/draft/2020-12/schema" +} diff --git a/src/viano_bug_analysis/schemas/case-v1.schema.json b/src/viano_bug_analysis/schemas/case-v1.schema.json new file mode 100644 index 0000000..8c2cb98 --- /dev/null +++ b/src/viano_bug_analysis/schemas/case-v1.schema.json @@ -0,0 +1,657 @@ +{ + "type": "object", + "required": [ + "format", + "format_version", + "builder_version", + "case", + "bug", + "human_observations", + "expected_behavior", + "reproduction", + "entities", + "runtime_facts", + "database_context", + "source_context", + "research", + "data_quality", + "inputs" + ], + "properties": { + "format": { + "const": "viano-case" + }, + "format_version": { + "type": "integer", + "const": 1 + }, + "builder_version": { + "const": "1" + }, + "case": { + "type": "object", + "required": [ + "case_id", + "package_id", + "analysis_revision", + "source_generation", + "project" + ], + "properties": { + "case_id": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "package_id": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "analysis_revision": { + "type": "integer", + "minimum": 1 + }, + "source_generation": { + "type": "integer", + "minimum": 0 + }, + "project": { + "type": "object" + } + } + }, + "bug": { + "type": "object", + "required": [ + "reports" + ], + "properties": { + "reports": { + "type": "array", + "items": { + "type": "object", + "required": [ + "source_ref", + "title", + "summary" + ], + "properties": { + "source_ref": { + "type": "string", + "minLength": 1 + }, + "title": { + "anyOf": [ + { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + { + "type": "null" + } + ] + }, + "summary": { + "anyOf": [ + { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + { + "type": "null" + } + ] + } + } + } + } + } + }, + "human_observations": { + "type": "object", + "required": [ + "observations", + "comments", + "tester_hypotheses" + ], + "properties": { + "observations": { + "type": "array", + "items": { + "$ref": "#/$defs/record" + } + }, + "comments": { + "type": "array", + "items": { + "$ref": "#/$defs/record" + } + }, + "tester_hypotheses": { + "type": "array", + "items": { + "$ref": "#/$defs/record" + } + } + } + }, + "expected_behavior": { + "type": "object", + "required": [ + "statements", + "completeness", + "provenance" + ], + "properties": { + "statements": { + "type": "array", + "items": { + "$ref": "#/$defs/record" + } + }, + "completeness": { + "enum": [ + "missing", + "unspecified", + "reported_complete", + "reported_partial", + "conflicting" + ] + }, + "provenance": { + "enum": [ + "tester", + "none" + ] + } + } + }, + "reproduction": { + "type": "object", + "required": [ + "records" + ], + "properties": { + "records": { + "type": "array", + "items": { + "type": "object", + "required": [ + "id", + "source_ref", + "steps", + "events_ref", + "attachments" + ], + "properties": { + "id": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "source_ref": { + "type": "string", + "minLength": 1 + }, + "steps": { + "type": "array", + "items": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + } + }, + "events_ref": { + "anyOf": [ + { + "type": "string", + "minLength": 1 + }, + { + "type": "null" + } + ] + }, + "attachments": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + } + } + } + } + } + }, + "entities": { + "type": "array", + "items": { + "type": "object", + "required": [ + "type", + "id", + "roles", + "provenance" + ], + "properties": { + "type": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "id": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "roles": { + "type": "array", + "items": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + } + }, + "provenance": { + "type": "array", + "items": { + "$ref": "#/$defs/provenance" + } + } + } + } + }, + "runtime_facts": { + "type": "object", + "required": [ + "facts", + "raw_refs" + ], + "properties": { + "facts": { + "type": "array", + "items": { + "type": "object", + "required": [ + "id", + "name", + "provenance" + ], + "oneOf": [ + { + "required": [ + "value" + ], + "not": { + "required": [ + "value_ref" + ] + } + }, + { + "required": [ + "value_ref" + ], + "not": { + "required": [ + "value" + ] + } + } + ], + "properties": { + "id": { + "type": "string", + "pattern": "^F[0-9]{4,}$" + }, + "name": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "value": {}, + "value_ref": { + "type": "object", + "required": [ + "source_ref", + "json_pointer" + ], + "properties": { + "source_ref": { + "type": "string", + "minLength": 1 + }, + "json_pointer": { + "type": "string", + "pattern": "^/" + } + } + }, + "provenance": { + "$ref": "#/$defs/provenance" + } + } + } + }, + "raw_refs": { + "type": "array", + "items": { + "$ref": "#/$defs/raw" + } + } + } + }, + "database_context": { + "type": "object", + "required": [ + "refs" + ], + "properties": { + "refs": { + "type": "array", + "items": { + "$ref": "#/$defs/raw" + } + } + } + }, + "source_context": { + "type": "object", + "required": [ + "refs" + ], + "properties": { + "refs": { + "type": "array", + "items": { + "$ref": "#/$defs/raw" + } + } + } + }, + "research": { + "type": "object", + "required": [ + "refs", + "performed" + ], + "properties": { + "refs": { + "type": "array", + "items": { + "$ref": "#/$defs/raw" + } + }, + "performed": { + "const": false + } + } + }, + "data_quality": { + "type": "object", + "required": [ + "issues", + "coverage" + ], + "properties": { + "issues": { + "type": "array", + "items": { + "type": "object", + "required": [ + "code", + "message", + "refs" + ], + "properties": { + "code": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "message": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "refs": { + "type": "array", + "items": { + "type": "object", + "required": [ + "source_ref", + "json_pointer" + ], + "properties": { + "source_ref": { + "type": "string", + "minLength": 1 + }, + "json_pointer": { + "type": "string" + } + } + } + } + } + } + }, + "coverage": { + "type": "object", + "required": [ + "reports", + "reproductions", + "observations" + ], + "properties": { + "reports": { + "type": "integer", + "minimum": 0 + }, + "reproductions": { + "type": "integer", + "minimum": 0 + }, + "observations": { + "type": "integer", + "minimum": 0 + } + } + } + } + }, + "inputs": { + "type": "object", + "required": [ + "index_ref", + "records", + "unclassified", + "prior_cases", + "request_hash" + ], + "properties": { + "index_ref": { + "anyOf": [ + { + "type": "string", + "minLength": 1 + }, + { + "type": "null" + } + ] + }, + "records": { + "type": "array", + "items": { + "type": "object", + "required": [ + "path", + "sha256", + "size", + "role" + ], + "properties": { + "path": { + "type": "string", + "minLength": 1 + }, + "sha256": { + "type": "string", + "pattern": "^[0-9a-f]{64}$" + }, + "size": { + "type": "integer", + "minimum": 0 + }, + "role": { + "enum": [ + "index", + "report", + "reproduction", + "runtime", + "database", + "source", + "research", + "attachment" + ] + } + } + } + }, + "unclassified": { + "type": "array", + "items": { + "$ref": "#/$defs/raw" + } + }, + "prior_cases": { + "type": "array", + "items": { + "$ref": "#/$defs/raw" + } + }, + "request_hash": { + "type": "string", + "pattern": "^[0-9a-f]{64}$" + } + } + } + }, + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$defs": { + "provenance": { + "type": "object", + "required": [ + "kind", + "source_ref", + "json_pointer", + "author", + "observed_at" + ], + "properties": { + "kind": { + "enum": [ + "tester", + "reproduction_record", + "report" + ] + }, + "source_ref": { + "type": "string", + "minLength": 1 + }, + "json_pointer": { + "type": "string", + "pattern": "^(|/.*)$" + }, + "author": { + "anyOf": [ + { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + { + "type": "null" + } + ] + }, + "observed_at": { + "type": [ + "string", + "null" + ], + "format": "date-time" + } + } + }, + "claim": { + "type": "object", + "required": [ + "key", + "value" + ], + "properties": { + "key": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "value": {} + } + }, + "raw": { + "type": "object", + "required": [ + "path", + "sha256", + "size" + ], + "properties": { + "path": { + "type": "string", + "minLength": 1 + }, + "sha256": { + "type": "string", + "pattern": "^[0-9a-f]{64}$" + }, + "size": { + "type": "integer", + "minimum": 0 + } + } + }, + "record": { + "type": "object", + "required": [ + "id", + "text", + "provenance" + ], + "properties": { + "id": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "text": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "provenance": { + "$ref": "#/$defs/provenance" + }, + "claim": { + "$ref": "#/$defs/claim" + } + } + } + } +} diff --git a/src/viano_bug_analysis/schemas/report-v1.schema.json b/src/viano_bug_analysis/schemas/report-v1.schema.json new file mode 100644 index 0000000..f775c78 --- /dev/null +++ b/src/viano_bug_analysis/schemas/report-v1.schema.json @@ -0,0 +1,149 @@ +{ + "type": "object", + "required": [ + "format", + "format_version" + ], + "properties": { + "format": { + "const": "viano-bug-report" + }, + "format_version": { + "type": "integer", + "const": 1 + }, + "title": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "summary": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "observations": { + "type": "array", + "items": { + "$ref": "#/$defs/statement" + } + }, + "comments": { + "type": "array", + "items": { + "$ref": "#/$defs/statement" + } + }, + "tester_hypotheses": { + "type": "array", + "items": { + "$ref": "#/$defs/statement" + } + }, + "expected_behavior": { + "type": "array", + "items": { + "$ref": "#/$defs/statement" + } + }, + "expected_behavior_complete": { + "type": [ + "boolean", + "null" + ] + }, + "entities": { + "type": "array", + "items": { + "$ref": "#/$defs/entity" + } + } + }, + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$defs": { + "claim": { + "type": "object", + "required": [ + "key", + "value" + ], + "properties": { + "key": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "value": {} + } + }, + "statement": { + "type": "object", + "required": [ + "text" + ], + "properties": { + "text": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "author": { + "anyOf": [ + { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + { + "type": "null" + } + ] + }, + "observed_at": { + "type": [ + "string", + "null" + ], + "format": "date-time" + }, + "claim": { + "$ref": "#/$defs/claim" + } + } + }, + "entity": { + "type": "object", + "required": [ + "type", + "id" + ], + "properties": { + "type": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "id": { + "anyOf": [ + { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + { + "type": "integer" + } + ] + }, + "roles": { + "type": "array", + "items": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + } + } + } + } + } +} diff --git a/src/viano_bug_analysis/schemas/reproduction-v1.schema.json b/src/viano_bug_analysis/schemas/reproduction-v1.schema.json new file mode 100644 index 0000000..f131555 --- /dev/null +++ b/src/viano_bug_analysis/schemas/reproduction-v1.schema.json @@ -0,0 +1,159 @@ +{ + "type": "object", + "required": [ + "format", + "format_version" + ], + "properties": { + "format": { + "const": "viano-reproduction" + }, + "format_version": { + "type": "integer", + "const": 1 + }, + "steps": { + "type": "array", + "items": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + } + }, + "observations": { + "type": "array", + "items": { + "$ref": "#/$defs/statement" + } + }, + "events_ref": { + "anyOf": [ + { + "type": "string", + "minLength": 1 + }, + { + "type": "null" + } + ] + }, + "attachments": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + }, + "facts": { + "type": "array", + "items": { + "type": "object", + "required": [ + "name", + "value" + ], + "properties": { + "name": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "value": {} + } + } + }, + "entities": { + "type": "array", + "items": { + "$ref": "#/$defs/entity" + } + } + }, + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$defs": { + "claim": { + "type": "object", + "required": [ + "key", + "value" + ], + "properties": { + "key": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "value": {} + } + }, + "statement": { + "type": "object", + "required": [ + "text" + ], + "properties": { + "text": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "author": { + "anyOf": [ + { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + { + "type": "null" + } + ] + }, + "observed_at": { + "type": [ + "string", + "null" + ], + "format": "date-time" + }, + "claim": { + "$ref": "#/$defs/claim" + } + } + }, + "entity": { + "type": "object", + "required": [ + "type", + "id" + ], + "properties": { + "type": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + "id": { + "anyOf": [ + { + "type": "string", + "minLength": 1, + "pattern": "\\S" + }, + { + "type": "integer" + } + ] + }, + "roles": { + "type": "array", + "items": { + "type": "string", + "minLength": 1, + "pattern": "\\S" + } + } + } + } + } +} diff --git a/tests/test_case_builder.py b/tests/test_case_builder.py new file mode 100644 index 0000000..c2465c6 --- /dev/null +++ b/tests/test_case_builder.py @@ -0,0 +1,384 @@ +import json + +import pytest +from jsonschema import Draft202012Validator +from importlib.resources import files + +from viano_bug_analysis.case import CaseBuilder, CaseAssembler, CaseBuilderLimits, CaseBuildError, render_summary +from viano_bug_analysis.lifecycle import CheckpointStore, LifecycleError, State, Workspace +from viano_bug_analysis.package import PackageReader, PackageValidator, PackageWriter + + +@pytest.fixture +def case_setup(manifest, make_package, tmp_path): + manifest["artifacts"]["case_inputs"] = "arbitrary/index.data" + report = {"format": "viano-bug-report", "format_version": 1, "title": "Generic problem", "summary": "Reported behavior", + "observations": [{"text": "The switch remains off", "author": "Tester A", "observed_at": "2026-01-01T01:00:00Z"}], + "comments": [{"text": "Seen twice"}], "tester_hypotheses": [{"text": "Perhaps a timing issue"}], + "expected_behavior": [{"text": "The switch turns on"}], "expected_behavior_complete": True, + "entities": [{"type": "component", "id": 42, "roles": ["focus"]}]} + reproduction = {"format": "viano-reproduction", "format_version": 1, "steps": ["Start", "Press switch"], + "observations": [{"text": "Still off"}], "events_ref": "raw/events", "attachments": ["raw/binary"], + "facts": [{"name": "attempt", "value": 2}], "entities": [{"type": "component", "id": "42", "roles": ["observed"]}]} + index = {"format": "viano-case-inputs", "format_version": 1, "reports": ["anything/report.data"], + "reproductions": ["anything/repro.data"], "context": {"source": ["raw/source"], "database": ["raw/db"], "runtime": ["raw/runtime"], "research": ["raw/research"]}} + entries = {"arbitrary/index.data": json.dumps(index), "anything/report.data": json.dumps(report), "anything/repro.data": json.dumps(reproduction), + "raw/events": b"many events", "raw/binary": b"\x00\xff", "raw/source": b"source raw", "raw/db": b"db raw", "raw/runtime": b"runtime raw", "raw/research": b"supplied external notes", "unknown/vendor": b"opaque"} + source = make_package(entries=entries) + store = CheckpointStore(Workspace(tmp_path / "service")) + store.intake(source) + return store, CaseBuilder(store), source, entries, report, index + + +@pytest.mark.parametrize("contract", ["case", "case-inputs", "report", "reproduction"]) +def test_contract_schemas_valid(contract): + schema = json.loads(files("viano_bug_analysis.schemas").joinpath(f"{contract}-v1.schema.json").read_text()) + Draft202012Validator.check_schema(schema) + + +def test_full_build_preserves_separation_and_provenance(case_setup): + store, builder, source, entries, _, _ = case_setup + original = source.read_bytes() + result = builder.build("case-A", "stable-package") + assert not result.reused + assert result.analysis_revision == 1 + assert PackageValidator().validate(result.package).valid + assert source.read_bytes() == original + with PackageReader(result.package) as reader: + manifest = reader.manifest + assert manifest["state"] == State.BUILDING_CASE + assert manifest["package_generation"] == 3 + case = json.loads(reader.read_bytes(result.case_ref)) + assert case["case"]["source_generation"] == 2 + assert case["human_observations"]["observations"][0]["text"] == "The switch remains off" + assert len(case["human_observations"]["observations"]) == 2 + assert case["human_observations"]["tester_hypotheses"][0]["text"] == "Perhaps a timing issue" + assert case["expected_behavior"]["completeness"] == "reported_complete" + assert case["human_observations"]["observations"][0]["provenance"] == {"kind": "tester", "source_ref": "anything/report.data", "json_pointer": "/observations/0", "author": "Tester A", "observed_at": "2026-01-01T01:00:00Z"} + assert case["runtime_facts"]["facts"][0]["id"] == "F0001" + assert case["runtime_facts"]["facts"][0]["provenance"]["json_pointer"] == "/facts/0" + assert case["entities"][0]["id"] == "42" + assert case["entities"][0]["roles"] == ["focus", "observed"] + assert len(case["entities"][0]["provenance"]) == 2 + assert case["research"]["performed"] is False + assert [item["path"] for item in case["inputs"]["unclassified"]] == ["unknown/vendor"] + assert reader.read_bytes(result.summary_ref).decode() == render_summary(case) + for path, value in entries.items(): + assert reader.read_bytes(path) == (value.encode() if isinstance(value, str) else value) + + +def test_completed_build_is_idempotent(case_setup): + store, builder, _, _, _, _ = case_setup + first = builder.build("case-A", "stable-package") + again = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package") + assert again.reused and again.package == first.package + assert len(list(first.package.parent.glob("g*.zip"))) == 3 + + +def test_missing_index_marks_gaps_without_guessing(make_package, tmp_path): + store = CheckpointStore(Workspace(tmp_path / "service")) + store.intake(make_package(entries={"original/bug_report.json": b"not a declared report"})) + result = CaseBuilder(store).build("case-A", "stable-package") + with PackageReader(result.package) as reader: + case = json.loads(reader.read_bytes(result.case_ref)) + codes = {issue["code"] for issue in case["data_quality"]["issues"]} + assert {"MISSING_INPUT_INDEX", "MISSING_REPORT", "MISSING_OBSERVATIONS", "MISSING_REPRODUCTION", "EXPECTED_BEHAVIOR_MISSING"} <= codes + assert case["bug"]["reports"] == [] + assert case["inputs"]["unclassified"][0]["path"] == "original/bug_report.json" + + +def rewrite_input(case_setup, tmp_path, changes, updates=None): + store = case_setup[0] + current = store.latest("case-A", "stable-package") + incoming = tmp_path / "updated-input" + PackageWriter().write(current, incoming, additions={name: json.dumps(data).encode() for name, data in changes.items()}, manifest_updates=updates) + store.intake(incoming) + return current + + +def test_reopen_inherits_previous_reports_and_facts(case_setup, tmp_path): + store, builder, _, _, _, _ = case_setup + first = builder.build("case-A", "stable-package") + for index, target in enumerate([State.INTERPRETING_CASE, State.GENERATING_HYPOTHESES, State.INVESTIGATING_HYPOTHESES, State.BUILDING_DIAGNOSIS, State.OUTPUT_READY, State.HANDED_OFF, State.REOPENED]): + store.advance("case-A", "stable-package", target, step_id=f"test-stage-{index}") + report = {"format": "viano-bug-report", "format_version": 1, "observations": [{"text": "New feedback"}]} + index = {"format": "viano-case-inputs", "format_version": 1, "reports": ["feedback/new-report"]} + rewrite_input(case_setup, tmp_path, {"feedback/new-report": report, "inputs/new-index": index}, {"artifacts": {"case_inputs": "inputs/new-index"}}) + result = builder.build("case-A", "stable-package") + assert result.analysis_revision == 2 + with PackageReader(result.package) as reader: + new = json.loads(reader.read_bytes(result.case_ref)) + old = json.loads(reader.read_bytes(first.case_ref)) + assert new["human_observations"]["observations"][:2] == old["human_observations"]["observations"] + assert new["human_observations"]["observations"][-1]["text"] == "New feedback" + assert new["runtime_facts"]["facts"] == old["runtime_facts"]["facts"] + assert new["inputs"]["prior_cases"][0]["path"] == first.case_ref + assert reader.read_bytes(first.summary_ref) == render_summary(old).encode() + + +def test_new_data_after_completion_requires_reopen(case_setup, tmp_path): + _, builder, _, _, _, _ = case_setup + builder.build("case-A", "stable-package") + rewrite_input(case_setup, tmp_path, {"feedback/new": {"anything": "new"}}) + with pytest.raises(CaseBuildError) as error: + builder.build("case-A", "stable-package") + assert error.value.code == "STALE_CASE" + + +def test_completion_crash_reuses_output(case_setup, monkeypatch): + store, builder, _, _, _, _ = case_setup + original_index = store._index + calls = 0 + def crash(*args): + nonlocal calls + calls += 1 + if calls == 2: + raise LifecycleError("SIMULATED_CRASH", "after case publication") + original_index(*args) + monkeypatch.setattr(store, "_index", crash) + with pytest.raises(CaseBuildError): + builder.build("case-A", "stable-package") + result = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package") + assert result.reused + assert len(list(result.package.parent.glob("g*.zip"))) == 3 + + +def test_unconfirmed_build_retries_without_duplicate_start(case_setup, monkeypatch): + store, builder, _, _, _, _ = case_setup + original_write = store.writer.write + calls = 0 + def crash(*args, **kwargs): + nonlocal calls + calls += 1 + if calls == 2: + raise LifecycleError("SIMULATED_CRASH", "before case publication") + return original_write(*args, **kwargs) + monkeypatch.setattr(store.writer, "write", crash) + with pytest.raises(CaseBuildError): + builder.build("case-A", "stable-package") + result = CaseBuilder(CheckpointStore(store.workspace)).build("case-A", "stable-package") + assert not result.reused + assert len(list(result.package.parent.glob("g*.zip"))) == 3 + + +def test_portable_completed_case(case_setup, tmp_path): + _, builder, _, _, _, _ = case_setup + first = builder.build("case-A", "stable-package") + other = CheckpointStore(Workspace(tmp_path / "other-service")) + other.intake(first.package) + result = CaseBuilder(other).build("case-A", "stable-package") + assert result.reused + assert result.package == other.latest("case-A", "stable-package") + + +def test_summary_deterministic_and_escapes_untrusted_text(case_setup): + _, builder, _, _, _, _ = case_setup + result = builder.build("case-A", "stable-package") + with PackageReader(result.package) as reader: + case = json.loads(reader.read_bytes(result.case_ref)) + case["human_observations"]["observations"][0]["text"] = " [link](javascript:bad)\n# fake heading" + summary = render_summary(case) + assert summary == render_summary(case) + assert "