Add expected behavior resolver and read-only research contracts

This commit is contained in:
Nico committed 2026-10-13 20:26:10 +02:00
1 parent 741c1ca4a5
commit 50b9d6e021
12 files changed
+640 -9

No files matched your search

+2
View File
@@ -22,3 +22,5 @@ python -m venv .venv
The lifecycle foundation supports one-pass inbox processing and resumable package
checkpoints and a deterministic generic CaseBuilder. It includes no analysis worker or LLM,
case interpretation, hypothesis generation or investigation tools.
Expected-behavior resolution and the read-only provider contract: [expected-behavior-v1.md](docs/expected-behavior-v1.md).
+23
View File
@@ -0,0 +1,23 @@
# Expected behavior and research v1
The resolver is part of CaseBuilder, project independent and deterministic. Architecture sections 10 and 27 remain authoritative. No LLM, online provider, interpretation or investigation is implemented here.
Tester declarations are retained verbatim with stable E IDs and source provenance. Only explicit `expected_behavior_complete: true` is sufficient; partial, unspecified, missing or contradictory declarations require research. Natural-language completeness and contradictions are not inferred. Explicit claim conflicts remain visible and cannot be cleared by research. Tester statements are never replaced by external statements.
`research.request` in the canonical case contains a versioned request or null for complete tester declarations. Requests contain identities, revision, project metadata, report titles/summaries, entity types/IDs and tester expectations. Local source, database and runtime payloads are excluded. Report text remains untrusted data; a future provider must enforce the intended-behavior policy independently. External sources establish intended behavior, never the state of a local implementation.
## Provider boundary
`ExternalResearchProvider.lookup(request, timeout_seconds=...)` is read-only. Providers must implement their transport deadline. This synchronous interface cannot forcibly terminate arbitrary Python provider code. `ResearchService` invokes the provider once, validates the entire response, verifies the request hash and citation indices, applies configurable result limits and returns deterministic append-only request/result artifacts. There is no hidden retry loop. Timeout/network errors become `RESEARCH_UNAVAILABLE`; other provider failures are controlled errors. A future worker can use the existing CheckpointStore BLOCKED/routing/retry mechanism; the standalone service does not change lifecycle state on failure.
Research-result v1 requires provider identity, request hash, intended-behavior scope, complete/partial/not_found status and source URL/title/retrieval timestamp/excerpt for every cited statement. Citation indices must exist. Completeness is the provider's explicit declaration, not a verification of external truth. No search, network fetching or source credibility assessment occurs in this milestone.
## Package lifecycle
Before CaseBuilder completion, advance the admitted case into BUILDING_CASE, then call `ResearchService(provider).enrich(active_package, destination)`. It builds the bounded request, stores immutable request/result files under `research/<content-hash>/`, adds a new input index and updates only the manifest's case_inputs pointer through PackageWriter. Intake that new generation into CheckpointStore and run CaseBuilder. Existing files remain byte-identical, package_id and analysis_revision remain stable and package_generation increases. A completed revision must be reopened before enrichment. Complete tester expectations bypass automatic research.
The explicit `case-inputs.expected_research` list designates structured research-result documents. Arbitrary context.research files remain opaque and cannot silently become expected behavior. Each result has a sibling request.json; its hash, identity and nonfuture revision are checked. Historical research remains in the case across revisions; it is traceable historical input, not silently re-fetched or asserted current. Applicability to changed project versions will require provider/policy decisions in a later milestone.
CaseBuilder includes external statements in expected_behavior, with tester/external_research/combined provenance and complete/partial status, source details and stable IDs. Facts and implementation context are not modified by research. The deterministic summary presents citations and excerpts without additional interpretation. Completed CaseBuilder retries reuse confirmed output.
ResearchLimits defaults (1 MiB result, 30 seconds provider deadline) are configurable operational safeguards, not format limits. Generic CaseBuilder and ZIP limits still apply. A test provider is exercised in tests; no production provider or dependencies on external accounts are introduced.
+21 -4
View File
@@ -9,7 +9,7 @@ from viano_bug_analysis.package.validator import CHUNK_SIZE
from .contracts import read_json, require_file, validate
from .errors import CaseBuildError
from .expected_behavior import collect_expected_behavior
from .expected_behavior import ExpectedBehaviorResolver, collect_expected_behavior
from .report_collector import collect_statements, provenance
from .reproduction_collector import collect_reproductions
@@ -135,7 +135,7 @@ class CaseAssembler:
raise CaseBuildError("PREVIOUS_INPUT_CHANGED", "A source referenced by the prior case changed", record["path"])
if index_ref:
add(index_ref, "index")
for field, role in (("reports", "report"), ("reproductions", "reproduction"), ("attachments", "attachment")):
for field, role in (("reports", "report"), ("reproductions", "reproduction"), ("attachments", "attachment"), ("expected_research", "expected_research")):
paths = index.get(field, [])
if len(paths) != len(set(paths)):
raise CaseBuildError("INPUT_DUPLICATE", "A source is listed twice in the same role", index_ref)
@@ -167,6 +167,8 @@ class CaseAssembler:
hypotheses = collect_statements(reports, "tester_hypotheses", "T")
expected = collect_expected_behavior(reports)
reproduction_records, facts = collect_reproductions(reproductions, self.limits.max_inline_fact_bytes)
research_documents = [(path, load(path, "research-result")) for path, role in roles if role == "expected_research"]
expected = ExpectedBehaviorResolver().resolve(expected, research_documents)
if previous:
old_humans = previous["human_observations"]
observations = _stable_records(old_humans["observations"], observations, "O")
@@ -182,6 +184,15 @@ class CaseAssembler:
record = entities.setdefault(identity, {"type": identity[0], "id": identity[1], "roles": [], "provenance": []})
record["roles"] = sorted(set(record["roles"]) | set(entity.get("roles", [])))
record["provenance"].append(provenance(path, f"/entities/{index}", "report" if document["format"] == "viano-bug-report" else "reproduction_record", entity))
research_request = ExpectedBehaviorResolver().request(manifest, [d for _, d in reports], [entities[k] for k in sorted(entities)], collect_expected_behavior(reports))
for path, result in research_documents:
request_ref = path.rsplit("/", 1)[0] + "/request.json"
request = load(request_ref, "research-request")
add(request_ref, "research")
if hashlib.sha256(json_bytes(request)).hexdigest() != result["request_hash"]:
raise CaseBuildError("RESEARCH_REQUEST_MISMATCH", "Stored research request hash does not match", path)
if request["case_id"] != manifest["case_id"] or request["package_id"] != manifest["package_id"] or request["analysis_revision"] > manifest["analysis_revision"]:
raise CaseBuildError("RESEARCH_IDENTITY", "Research belongs to another case or future revision", path)
issues = []
def issue(code, message, refs=None):
@@ -199,7 +210,7 @@ class CaseAssembler:
issue("MISSING_OBSERVATIONS", "No direct observations were supplied")
if not reproductions:
issue("MISSING_REPRODUCTION", "No structured reproduction was supplied")
if expected["completeness"] != "reported_complete":
if expected["completeness"] not in {"reported_complete", "researched_complete"}:
issue("EXPECTED_BEHAVIOR_" + expected["completeness"].upper(), "Expected-behavior completeness is " + expected["completeness"],
[{"source_ref": path, "json_pointer": "/expected_behavior_complete"} for path, report in reports if "expected_behavior_complete" in report])
for category, records in (("observation", observations), ("expected_behavior", expected["statements"])):
@@ -214,6 +225,12 @@ class CaseAssembler:
issue("CONFLICTING_CLAIMS", f"Different explicit values for {category} claim {claim_key}", refs)
if category == "expected_behavior":
expected["completeness"] = "conflicting"
if expected["completeness"] == "researched_complete":
research_request = None
if expected["completeness"] == "conflicting":
tester_expected = collect_expected_behavior(reports)
tester_expected["completeness"] = "conflicting"
research_request = ExpectedBehaviorResolver().request(manifest, [d for _, d in reports], [entities[k] for k in sorted(entities)], tester_expected)
records = [inventory[path] | {"role": role} for path, role in roles]
classified = {path for path, _ in roles}
raw_inventory = [value for name, value in inventory.items() if name not in excluded]
@@ -234,7 +251,7 @@ class CaseAssembler:
"entities": [entities[identity] for identity in sorted(entities)],
"runtime_facts": {"facts": facts, "raw_refs": refs_for("runtime")},
"database_context": {"refs": refs_for("database")}, "source_context": {"refs": refs_for("source")},
"research": {"refs": refs_for("research"), "performed": False},
"research": {"refs": refs_for("research") + refs_for("expected_research"), "performed": bool(research_documents), "request": research_request},
"data_quality": {"issues": issues, "coverage": {"reports": len(reports), "reproductions": len(reproductions), "observations": len(observations)}},
"inputs": {"index_ref": index_ref, "records": records,
"unclassified": [value for name, value in inventory.items() if name not in classified and name not in excluded],
@@ -17,3 +17,41 @@ def collect_expected_behavior(reports: list[tuple[str, dict]]) -> dict:
status = "unspecified"
return {"statements": statements, "completeness": status,
"provenance": "tester" if statements else "none"}
class ExpectedBehaviorResolver:
"""Resolve explicit declarations without interpreting natural language."""
def request(self, identity: dict, reports: list[dict], entities: list[dict], expected: dict) -> dict | None:
if expected["completeness"] in {"reported_complete", "researched_complete"}:
return None
return {"format": "viano-research-request", "format_version": 1,
"case_id": identity["case_id"], "package_id": identity["package_id"],
"analysis_revision": identity["analysis_revision"],
"purpose": "expected_behavior", "scope": "intended_behavior_only",
"project": identity["project"],
"reports": [{"title": r.get("title"), "summary": r.get("summary")} for r in reports],
"entities": [{"type": e["type"], "id": e["id"]} for e in entities],
"tester_expectations": [r["text"] for r in expected["statements"] if r["provenance"]["kind"] == "tester"]}
def resolve(self, expected: dict, research: list[tuple[str, dict]]) -> dict:
import copy
from .errors import CaseBuildError
result = copy.deepcopy(expected)
statuses = []
for path, document in research:
for index, statement in enumerate(document["statements"]):
if any(i >= len(document["sources"]) for i in statement["source_indices"]):
raise CaseBuildError("RESEARCH_SOURCE", "Research statement references a missing source", path)
record = {"id": f"E{len(result['statements']) + 1:04d}", "text": statement["text"],
"provenance": {"kind": "external_research", "source_ref": path,
"json_pointer": f"/statements/{index}", "author": None, "observed_at": None},
"external_sources": [copy.deepcopy(document["sources"][i]) for i in statement["source_indices"]]}
result["statements"].append(record)
if document["statements"]:
statuses.append(document["status"])
if statuses:
result["provenance"] = "combined" if expected["statements"] else "external_research"
if result["completeness"] not in {"reported_complete", "conflicting"}:
result["completeness"] = "researched_complete" if set(statuses) == {"complete"} else "researched_partial"
return result
+6
View File
@@ -46,6 +46,9 @@ def render_summary(case: dict) -> str:
f" Author: {_literal(source['author']) if source['author'] is not None else '(not supplied)'}",
f" Observed at: {_literal(source['observed_at']) if source['observed_at'] is not None else '(not supplied)'}"]
lines.extend(_origin_lines(source.get("origin")))
for external in record.get("external_sources", []):
lines.append(f" External source: {_literal(external['title'])} {_literal(external['url'])}; retrieved: {_literal(external['retrieved_at'])}")
lines.append(f" Source excerpt: {_literal(external['excerpt'])}")
if "claim" in record:
lines.append(f" Claim: {_literal(record['claim']['key'])} = {_literal(json.dumps(record['claim']['value'], sort_keys=True, ensure_ascii=False))}")
lines += ["", f"Expected-behavior completeness: {case['expected_behavior']['completeness']}", "", "## Reproductions", ""]
@@ -64,6 +67,9 @@ def render_summary(case: dict) -> str:
for source in entity["provenance"]:
lines.append(f" Source: {_literal(source['source_ref'])} {_literal(source['json_pointer'])}")
lines.extend(_origin_lines(source.get("origin")))
for external in record.get("external_sources", []):
lines.append(f" External source: {_literal(external['title'])} {_literal(external['url'])}; retrieved: {_literal(external['retrieved_at'])}")
lines.append(f" Source excerpt: {_literal(external['excerpt'])}")
lines += ["", "## Facts supplied by reproduction records", ""]
for fact in case["runtime_facts"]["facts"]:
source = fact["provenance"]
@@ -0,0 +1,3 @@
from .provider import ExternalResearchProvider, ResearchError, ResearchLimits, ResearchService
__all__ = ["ExternalResearchProvider", "ResearchError", "ResearchLimits", "ResearchService"]
@@ -0,0 +1,95 @@
"""Read-only provider boundary; no network implementation or shell access."""
import copy
import hashlib
from dataclasses import dataclass
from typing import Protocol
from viano_bug_analysis.case.contracts import validate
from viano_bug_analysis.case.errors import CaseBuildError
from viano_bug_analysis.lifecycle.storage import json_bytes
class ResearchError(CaseBuildError):
pass
class ExternalResearchProvider(Protocol):
def lookup(self, request: dict, *, timeout_seconds: float) -> dict:
"""Return a research-result v1 document; respect the supplied deadline."""
...
@dataclass(frozen=True)
class ResearchLimits:
max_result_bytes: int = 1024 * 1024
timeout_seconds: float = 30.0
def __post_init__(self):
if isinstance(self.max_result_bytes, bool) or not isinstance(self.max_result_bytes, int) or self.max_result_bytes <= 0:
raise ValueError("Result limit must be a positive integer")
if isinstance(self.timeout_seconds, bool) or not isinstance(self.timeout_seconds, (float, int)) or not 0 < self.timeout_seconds < float("inf"):
raise ValueError("Timeout must be positive and finite")
class ResearchService:
def __init__(self, provider: ExternalResearchProvider, limits: ResearchLimits | None = None):
self.provider = provider
self.limits = limits or ResearchLimits()
def collect(self, request: dict) -> tuple[str, dict[str, bytes]]:
validate(request, "research-request")
expected_hash = hashlib.sha256(json_bytes(request)).hexdigest()
try:
result = copy.deepcopy(self.provider.lookup(copy.deepcopy(request), timeout_seconds=self.limits.timeout_seconds))
except (TimeoutError, ConnectionError, OSError) as error:
raise ResearchError("RESEARCH_UNAVAILABLE", "Research provider temporarily unavailable") from error
except Exception as error:
raise ResearchError("RESEARCH_PROVIDER", "Research provider failed") from error
try:
data = json_bytes(result)
except (ValueError, TypeError, RecursionError) as error:
raise ResearchError("RESEARCH_JSON", "Provider returned unserializable data") from error
if len(data) > self.limits.max_result_bytes:
raise ResearchError("RESEARCH_LIMIT", "Result exceeds configured operational limit")
validate(result, "research-result")
if result["request_hash"] != expected_hash:
raise ResearchError("RESEARCH_REQUEST_MISMATCH", "Provider result belongs to another request")
for statement in result["statements"]:
if any(i >= len(result["sources"]) for i in statement["source_indices"]):
raise ResearchError("RESEARCH_SOURCE", "Provider cited a missing source")
base = "research/" + hashlib.sha256(json_bytes(request) + data).hexdigest()
return base + "/result.json", {base + "/request.json": json_bytes(request), base + "/result.json": data}
def enrich(self, source, destination):
"""Publish research inputs before CaseBuilder completion; source stays intact.
Intake the resulting generation into CheckpointStore, then run CaseBuilder.
A completed revision must be reopened before enrichment.
"""
from viano_bug_analysis.case.assembler import CaseAssembler
from viano_bug_analysis.case.contracts import read_json
from viano_bug_analysis.lifecycle.checkpoints import checkpoint_chain
from viano_bug_analysis.package import PackageReader, PackageWriter
with PackageReader(source) as reader:
manifest = reader.manifest
if manifest["state"] != "BUILDING_CASE":
raise ResearchError("RESEARCH_STATE", "Research enrichment requires BUILDING_CASE")
if any(cp["details"].get("operation") == "case-built" and cp["analysis_revision"] == manifest["analysis_revision"] for _, cp in checkpoint_chain(reader)):
raise ResearchError("STALE_CASE", "Reopen the completed revision before research enrichment")
assembler = CaseAssembler()
case = assembler.assemble(reader)
request = case["research"]["request"]
if request is None:
raise ResearchError("RESEARCH_NOT_REQUIRED", "Complete tester expectations need no automatic research")
index_ref = manifest["artifacts"].get("case_inputs")
index = read_json(reader, index_ref, "case-inputs", assembler.limits.max_json_bytes) if index_ref else {"format": "viano-case-inputs", "format_version": 1}
result_ref, additions = self.collect(request)
index.setdefault("expected_research", [])
if result_ref in index["expected_research"]:
raise ResearchError("RESEARCH_EXISTS", "Identical research already recorded")
index["expected_research"].append(result_ref)
new_index = result_ref.rsplit("/", 1)[0] + "/case-inputs.json"
additions[new_index] = json_bytes(index)
return PackageWriter().write(source, destination, additions=additions,
manifest_updates={"artifacts": {**manifest["artifacts"], "case_inputs": new_index}})
@@ -66,6 +66,15 @@
"type": "string",
"minLength": 1
}
},
"expected_research": {
"type": "array",
"uniqueItems": true,
"items": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema"
@@ -161,13 +161,17 @@
"unspecified",
"reported_complete",
"reported_partial",
"conflicting"
"conflicting",
"researched_complete",
"researched_partial"
]
},
"provenance": {
"enum": [
"tester",
"none"
"none",
"external_research",
"combined"
]
}
}
@@ -393,7 +397,7 @@
}
},
"performed": {
"const": false
"type": "boolean"
}
}
},
@@ -523,7 +527,8 @@
"database",
"source",
"research",
"attachment"
"attachment",
"expected_research"
]
}
}
@@ -564,7 +569,8 @@
"enum": [
"tester",
"reproduction_record",
"report"
"report",
"external_research"
]
},
"source_ref": {
@@ -659,6 +665,40 @@
},
"claim": {
"$ref": "#/$defs/claim"
},
"external_sources": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"required": [
"url",
"title",
"retrieved_at",
"excerpt"
],
"properties": {
"url": {
"type": "string",
"pattern": "^https?://[^\\s/]+(?:/[^\\s]*)?$"
},
"title": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"retrieved_at": {
"type": "string",
"format": "date-time"
},
"excerpt": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
},
"additionalProperties": false
}
}
}
},
@@ -0,0 +1,105 @@
{
"type": "object",
"required": [
"format",
"format_version",
"case_id",
"package_id",
"analysis_revision",
"purpose",
"scope",
"project",
"reports",
"entities",
"tester_expectations"
],
"properties": {
"format": {
"const": "viano-research-request"
},
"format_version": {
"const": 1
},
"case_id": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"package_id": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"analysis_revision": {
"type": "integer",
"minimum": 1
},
"purpose": {
"const": "expected_behavior"
},
"scope": {
"const": "intended_behavior_only"
},
"project": {
"type": "object"
},
"reports": {
"type": "array",
"items": {
"type": "object",
"required": [
"title",
"summary"
],
"properties": {
"title": {
"type": [
"string",
"null"
]
},
"summary": {
"type": [
"string",
"null"
]
}
},
"additionalProperties": false
}
},
"entities": {
"type": "array",
"items": {
"type": "object",
"required": [
"type",
"id"
],
"properties": {
"type": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"id": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
},
"additionalProperties": false
}
},
"tester_expectations": {
"type": "array",
"items": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
}
},
"additionalProperties": false,
"$schema": "https://json-schema.org/draft/2020-12/schema"
}
@@ -0,0 +1,127 @@
{
"type": "object",
"required": [
"format",
"format_version",
"request_hash",
"provider",
"scope",
"status",
"sources",
"statements"
],
"properties": {
"format": {
"const": "viano-research-result"
},
"format_version": {
"const": 1
},
"request_hash": {
"type": "string",
"pattern": "^[0-9a-f]{64}$"
},
"provider": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"scope": {
"const": "intended_behavior_only"
},
"status": {
"enum": [
"complete",
"partial",
"not_found"
]
},
"sources": {
"type": "array",
"items": {
"type": "object",
"required": [
"url",
"title",
"retrieved_at",
"excerpt"
],
"properties": {
"url": {
"type": "string",
"pattern": "^https?://[^\\s/]+(?:/[^\\s]*)?$"
},
"title": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"retrieved_at": {
"type": "string",
"format": "date-time"
},
"excerpt": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
}
},
"additionalProperties": false
}
},
"statements": {
"type": "array",
"items": {
"type": "object",
"required": [
"text",
"source_indices"
],
"properties": {
"text": {
"type": "string",
"minLength": 1,
"pattern": "\\S"
},
"source_indices": {
"type": "array",
"minItems": 1,
"uniqueItems": true,
"items": {
"type": "integer",
"minimum": 0
}
}
},
"additionalProperties": false
}
}
},
"additionalProperties": false,
"allOf": [
{
"if": {
"properties": {
"status": {
"const": "not_found"
}
}
},
"then": {
"properties": {
"statements": {
"maxItems": 0
}
}
},
"else": {
"properties": {
"statements": {
"minItems": 1
}
}
}
}
],
"$schema": "https://json-schema.org/draft/2020-12/schema"
}
+166
View File
@@ -0,0 +1,166 @@
import hashlib
import json
import pytest
from viano_bug_analysis.case.expected_behavior import ExpectedBehaviorResolver, collect_expected_behavior
from viano_bug_analysis.case import CaseAssembler, CaseBuildError
from viano_bug_analysis.lifecycle import State
from viano_bug_analysis.lifecycle.storage import json_bytes
from viano_bug_analysis.package import PackageReader, PackageWriter, PackageValidator
from viano_bug_analysis.research import ResearchError, ResearchLimits, ResearchService
from test_case_builder import case_setup
def request():
return ExpectedBehaviorResolver().request(
{"case_id": "case-A", "package_id": "stable-package", "analysis_revision": 1, "project": {}}, [], [],
{"statements": [], "completeness": "missing", "provenance": "none"})
def result(req, status="complete"):
return {"format": "viano-research-result", "format_version": 1, "request_hash": hashlib.sha256(json_bytes(req)).hexdigest(),
"provider": "fixture", "scope": "intended_behavior_only", "status": status,
"sources": [{"url": "https://example.org/manual", "title": "Manual", "retrieved_at": "2026-01-01T00:00:00Z", "excerpt": "The switch turns on."}],
"statements": [] if status == "not_found" else [{"text": "The switch turns on.", "source_indices": [0]}]}
class FixtureProvider:
def __init__(self, response):
self.response = response
def lookup(self, request, *, timeout_seconds):
assert timeout_seconds > 0
return self.response
@pytest.mark.parametrize("flag,status", [(True,"reported_complete"),(False,"reported_partial"),(None,"unspecified")])
def test_explicit_completeness(flag, status):
expected = collect_expected_behavior([("report", {"expected_behavior": [{"text":"Expected"}], "expected_behavior_complete": flag})])
assert expected["completeness"] == status
req = ExpectedBehaviorResolver().request({"case_id":"c","package_id":"p","analysis_revision":1,"project":{}}, [], [], expected)
assert (req is None) == (flag is True)
@pytest.mark.parametrize("status,completeness", [("complete","researched_complete"),("partial","researched_partial"),("not_found","missing")])
def test_resolution(status, completeness):
expected = {"statements":[],"completeness":"missing","provenance":"none"}
resolved = ExpectedBehaviorResolver().resolve(expected, [("research/result.json",result(request(), status))])
assert resolved["completeness"] == completeness
assert expected["statements"] == []
if resolved["statements"]:
assert resolved["statements"][0]["external_sources"][0]["url"].startswith("https:")
def test_combined_does_not_override_tester_or_conflict():
expected = collect_expected_behavior([("r", {"expected_behavior":[{"text":"Tester statement"}],"expected_behavior_complete":False})])
resolved = ExpectedBehaviorResolver().resolve(expected, [("research/result.json",result(request()))])
assert resolved["provenance"] == "combined"
assert resolved["statements"][0] == expected["statements"][0]
expected["completeness"] = "conflicting"
assert ExpectedBehaviorResolver().resolve(expected, [("r",result(request()))])["completeness"] == "conflicting"
@pytest.mark.parametrize("mutation", ["identity","source","scope","time"])
def test_invalid_provider_is_rejected(mutation):
response = result(request())
if mutation == "identity": response["request_hash"] = "0" * 64
if mutation == "source": response["statements"][0]["source_indices"] = [2]
if mutation == "scope": response["scope"] = "local_implementation"
if mutation == "time": response["sources"][0]["retrieved_at"] = "2026-01-01"
with pytest.raises(CaseBuildError): ResearchService(FixtureProvider(response)).collect(request())
def test_provider_failure_controlled():
class Offline:
def lookup(self, request, *, timeout_seconds): raise TimeoutError()
with pytest.raises(ResearchError, match="temporarily unavailable"):
ResearchService(Offline()).collect(request())
with pytest.raises(ResearchError):
ResearchService(FixtureProvider(result(request())), ResearchLimits(max_result_bytes=1)).collect(request())
def test_collection_deterministic():
service = ResearchService(FixtureProvider(result(request())))
assert service.collect(request()) == service.collect(request())
def test_package_roundtrip_and_restart(case_setup, tmp_path):
store, builder, _, entries, _, index = case_setup
active = store.advance("case-A","stable-package",State.BUILDING_CASE,step_id="prepare-research")
with PackageReader(active) as reader:
case = CaseAssembler().assemble(reader)
# Research can also be explicitly supplied to corroborate a complete tester report.
req = request()
req["reports"] = [{"title":"Generic problem","summary":"Reported behavior"}]
req["entities"] = [{"type":"component","id":"42"}]
req["tester_expectations"] = ["The switch turns on"]
req["project"] = reader.manifest["project"]
artifacts = reader.manifest["artifacts"]
ref, additions = ResearchService(FixtureProvider(result(req))).collect(req)
index = dict(index, expected_research=[ref])
additions["research/new-index.json"] = json_bytes(index)
enriched = tmp_path / "enriched.zip"
PackageWriter().write(active,enriched,additions=additions,manifest_updates={"artifacts": {**artifacts,"case_inputs":"research/new-index.json"}})
store.intake(enriched)
built = builder.build("case-A","stable-package")
assert PackageValidator().validate(built.package).valid
with PackageReader(built.package) as reader:
canonical = json.loads(reader.read_bytes(built.case_ref))
assert canonical["expected_behavior"]["provenance"] == "combined"
assert canonical["research"]["performed"] is True
for path, value in entries.items():
assert reader.read_bytes(path) == (value.encode() if isinstance(value,str) else value)
assert builder.build("case-A","stable-package").reused
def test_service_enrichment_missing_expectations(manifest, make_package, tmp_path):
from viano_bug_analysis.lifecycle import CheckpointStore, Workspace
from viano_bug_analysis.case import CaseBuilder
manifest["artifacts"]["case_inputs"] = "input.json"
source = make_package(entries={"input.json": json_bytes({"format":"viano-case-inputs","format_version":1,"reports":["report.json"]}),
"report.json": json_bytes({"format":"viano-bug-report","format_version":1,"title":"Switch remains off"}),
"unknown.bin": b"preserve"})
store = CheckpointStore(Workspace(tmp_path / "workspace"))
store.intake(source)
active = store.advance("case-A","stable-package",State.BUILDING_CASE,step_id="research-start")
class DynamicProvider:
calls = 0
def lookup(self, request, *, timeout_seconds):
self.calls += 1
assert "source_context" not in request and "database_context" not in request
return result(request)
provider = DynamicProvider()
service = ResearchService(provider)
destination = tmp_path / "researched.zip"
service.enrich(active, destination)
assert provider.calls == 1
assert PackageValidator().validate(destination).valid
with PackageReader(active) as before, PackageReader(destination) as after:
assert after.manifest["package_generation"] == before.manifest["package_generation"] + 1
assert after.manifest["package_id"] == before.manifest["package_id"]
for path in before.files:
if path != "manifest.json": assert before.read_bytes(path) == after.read_bytes(path)
store.intake(destination)
built = CaseBuilder(store).build("case-A","stable-package")
with PackageReader(built.package) as reader:
case = json.loads(reader.read_bytes(built.case_ref))
assert case["expected_behavior"]["completeness"] == "researched_complete"
assert case["expected_behavior"]["provenance"] == "external_research"
assert case["runtime_facts"]["facts"] == []
with pytest.raises(ResearchError, match="Reopen"):
service.enrich(built.package, tmp_path / "invalid.zip")
assert provider.calls == 1
@pytest.mark.parametrize("value", [0, -1, True, float("inf"), float("nan")])
def test_invalid_timeout(value):
with pytest.raises(ValueError): ResearchLimits(timeout_seconds=value)
@pytest.mark.parametrize("contract", ["research-request", "research-result"])
def test_research_schemas(contract):
from importlib.resources import files
from jsonschema import Draft202012Validator
schema = json.loads(files("viano_bug_analysis.schemas").joinpath(contract + "-v1.schema.json").read_text())
Draft202012Validator.check_schema(schema)