Case package format v1

The authoritative design is architecture.md. This document specifies only the project-independent package foundation. A package is a long-lived, self-contained case record in a ZIP archive. Its filename and extension carry no program meaning. Exactly one root manifest.json is the authoritative entry point.

Identity, state and routing

format is viano-bug-case-package; format_version is integer 1. package_id is an opaque nonblank string identifying the case package throughout its lifetime. Enriching and emitting it again preserves this identity and increases package_generation (a nonnegative integer). case_id identifies the case. analysis_revision is a separate nonnegative integer, initially zero; it is independent of package generation. The reader does not derive or increment any of these values.

project requires generic type and profile strings and accepts display_name. created_at and updated_at are RFC-3339 timestamps with timezone; updated time must not precede creation time. Timestamp validation rejects leap seconds because Python's datetime model cannot represent them; lowercase t/z and fractional seconds are supported.

state uses the lifecycle states defined in the architecture: NEW, VALIDATED, BUILDING_CASE, INTERPRETING_CASE, GENERATING_HYPOTHESES, INVESTIGATING_HYPOTHESES, BUILDING_DIAGNOSIS, OUTPUT_READY, HANDED_OFF, BLOCKED, FAILED_TO_ANALYZE, REOPENED.

routing is separate and requires:

destination: null or inbox, quarantine, cases, outbox, review; a logical route, never a filesystem path. Cache and runtime are service storage, not routes.
reason: null or nonblank explanatory string.
retry_count: nonnegative integer, initially zero.
retry_not_before: null or RFC-3339 timestamp.

No state/route combination triggers execution. This milestone checks the contract only, without implementing routing, retry, locking or lifecycle transitions.

References and integrity

artifacts requires case, diagnosis, change_request, each null or an internal file path. Additional artifact pointers follow the same rule. history requires nonnegative counters analysis_revisions, reproductions, evidence, handoffs, feedback, and nullable events_ref. Counters are supplied summaries; the reader does not infer them from directory names or impose diagnostic semantics.

integrity maps internal file paths to { "sha256": "<64 lowercase hex digits>", "size": <uncompressed byte count> }. All supplied records are checked. A fresh package may have an empty map. Producers should cover important immutable artifacts. The format foundation does not infer which project artifacts are important. Hashes detect changes, not authenticity. The manifest cannot reference or hash itself. Every non-null reference must resolve to an actual ZIP file.

The bundled manifest-v1.schema.json uses JSON Schema Draft 2020-12 and only local schema references; validation requires no network. Unknown extension fields are accepted and retained. Unsupported format versions, duplicate JSON keys, non-UTF-8 manifests and nonstandard numeric constants are rejected.

Preservation

original/ and completed analysis/revisions/NNNN/ are immutable. New evidence, feedback and revisions are appended. Only explicitly designated index/pointer files (manifest.json, analysis/current.json) may later be updated. Unknown files are preserved unchanged. Validator and reader never write, extract, delete or rewrite anything. The writer checks append-only and output-superset preservation against its input. Historical preservation cannot be proven from one standalone incoming archive.

ZIP acceptance and configurable security policy

This implementation accepts stored and deflated ZIP entries, including ZIP64 within configured limits. It rejects encryption, symlinks and other special filesystem entries, unsafe paths, duplicate entries, case-insensitive collisions, and file/directory conflicts. Paths must be relative POSIX paths without empty, dot or parent segments, backslashes, colons or control characters. Explicit empty directory entries are accepted. Paths are rejected, never silently repaired. The reader does not extract files; future extraction requires its own checks.

PackageLimits is a configurable operational security policy. These defaults are not immutable limits of the v1 package format and do not appear in the manifest schema:

Limit	Default
Archive bytes	512 MiB
Expanded total bytes	2 GiB
Expanded file bytes	256 MiB
Manifest bytes	1 MiB
ZIP entries (including directories)	10,000
Per-entry compression ratio	200:1

Declared sizes are checked before reading. Actual reads are bounded and total bytes are counted while streaming every file for CRC verification; listed SHA-256 hashes are checked during the same pass. Manifest parsing is bounded separately. Malformed archive, I/O, JSON, schema and integrity failures produce structured validation issues. Programmer errors and process-level failures are not hidden. Input files must not be modified concurrently while open; the reader retains the same file handle and archive used for validation.

Public API

from viano_bug_analysis.package import (
    PackageLimits, PackageValidator, PackageReader, PackageReadError,
)

result = PackageValidator(PackageLimits()).validate("renamed-package.bin")
if not result.valid:
    for issue in result.issues:
        print(issue.code, issue.path, issue.message)

try:
    with PackageReader("renamed-package.bin") as package:
        manifest = package.manifest  # defensive copy, including extensions
        inventory = package.files   # all file paths, including unknown files
        data = package.read_bytes("original/bug_report.json")
except PackageReadError as error:
    issues = error.result.issues

Read access outside the context is a programming error (RuntimeError). Invalid packages and missing member requests raise PackageReadError with structured details. read_bytes is bounded by policy but materializes a single file in memory; validation itself streams. No artifact interpretation is performed.

Writer API and publication contract

PackageFactory.create(destination, manifest, artifacts) creates a generation-zero package from caller-supplied metadata and byte artifacts. It adds integrity records, applies PackageLimits, validates the staged ZIP, flushes/fsyncs and publishes atomically without replacing an existing target. It is project-independent; subsequent enrichment still uses PackageWriter.

from viano_bug_analysis.package import PackageWriter, PackageWriteError

try:
    result = PackageWriter().write(
        "input.any-name",
        "next-generation.any-name",
        additions={"case/revisions/0001/case.json": b'{"example": true}'},
        index_updates={"analysis/current.json": b'{"revision": 1}'},
        manifest_updates={
            "state": "VALIDATED",
            "routing": {"destination": "cases"},
            "artifacts": {"case": "case/revisions/0001/case.json"},
        },
    )
except PackageWriteError as error:
    issues = error.result.issues

additions accepts new file paths and bytes only. Existing files cannot be overwritten, even with identical bytes. Unknown input files and explicit directory entries are carried forward. ZIP comments and retained entry metadata are copied; compressed representations may change, but retained file bytes must be identical. Existing analysis/revisions/<revision>/ namespaces are frozen, including empty directories: this conservative foundation does not infer whether a revision is still active. New revisions use new namespaces. New reproductions and feedback use new paths. There is no deletion operation.

index_updates can create or replace only analysis/current.json. manifest.json is managed exclusively by the writer. Index bytes are opaque at this milestone; no interpretation or current-index schema is implemented.

manifest_updates accepts only state, routing, analysis_revision, artifacts and history. Object updates merge recursively, preserving nested extension fields. Identity, format, project and creation time remain unchanged. The writer sets generation to input generation plus one, sets updated time to the later of the current UTC time and the previous updated time, and maintains SHA-256/size records for every output file except the manifest. Existing integrity-record extensions are retained. Analysis revision and standard history counters cannot decrease; they are otherwise supplied explicitly, never inferred from generation or artifact names. No lifecycle transition policy or automatic history event is introduced. Existing history/events.jsonl cannot be replaced: a future event writer needs an explicit byte-prefix append contract; for now use a new event artifact and update events_ref if needed.

The source is fully validated before staging. Output uses ZIP_STORED so highly compressible new data does not create archives rejected by compression-ratio defaults. The same configurable PackageLimits apply to input and output. This may require a larger archive-byte policy for a previously compressed input. Copying and preservation comparison stream in bounded chunks; additions are caller-provided bytes. The source must remain unchanged throughout the call.

Staging occurs in an exclusively created temporary file in the destination's existing parent directory. The ZIP is closed, flushed and fsynced, then fully validated. Its manifest must equal the planned manifest; every input entry must survive, and every protected file is compared byte for byte. The target is only published after these checks. Normal failures remove staging files and leave the input and existing target unchanged. A process crash may leave an unpublished staging file; this milestone does not implement recovery jobs. An exceptional cleanup failure before publication raises a structured error; after publication it emits a ResourceWarning and still returns success, since the valid target has already been committed. A staging file may remain in that case and can be removed by the caller.

By default an existing target is rejected. Atomic no-clobber publication uses a same-directory hard link, preventing concurrent writers from silently replacing each other; the destination filesystem must support hard links. overwrite=True explicitly permits atomic replacement using os.replace. Input aliases (including hard links), symbolic-link targets and in-place output are rejected. Temporary files are cleaned up after publication. This is atomic visibility with a flushed file, not a claim of portable directory-metadata durability across power loss. Destination directories and input files must be controlled by the caller; this milestone does not implement case locking or protection from concurrent external filesystem mutation. Every successful write, including one without additions, produces exactly one new generation. Job-level retry idempotence is deferred to the later lifecycle layer.

11 KiB Raw Permalink Blame History