211 lines
11 KiB
Markdown
211 lines
11 KiB
Markdown
# Case package format v1
|
|
|
|
The authoritative design is `architecture.md`. This document specifies only the
|
|
project-independent package foundation. A package is a long-lived, self-contained
|
|
case record in a ZIP archive. Its filename and extension carry no program meaning.
|
|
Exactly one root `manifest.json` is the authoritative entry point.
|
|
|
|
## Identity, state and routing
|
|
|
|
`format` is `viano-bug-case-package`; `format_version` is integer `1`.
|
|
`package_id` is an opaque nonblank string identifying the case package throughout
|
|
its lifetime. Enriching and emitting it again preserves this identity and
|
|
increases `package_generation` (a nonnegative integer).
|
|
`case_id` identifies the case. `analysis_revision` is a separate nonnegative
|
|
integer, initially zero; it is independent of package generation.
|
|
The reader does not derive or increment any of these values.
|
|
|
|
`project` requires generic `type` and `profile` strings and accepts `display_name`.
|
|
`created_at` and `updated_at` are RFC-3339 timestamps with timezone; updated time
|
|
must not precede creation time.
|
|
Timestamp validation rejects leap seconds because Python's datetime model cannot
|
|
represent them; lowercase `t`/`z` and fractional seconds are supported.
|
|
|
|
`state` uses the lifecycle states defined in the architecture: NEW, VALIDATED,
|
|
BUILDING_CASE, INTERPRETING_CASE, GENERATING_HYPOTHESES,
|
|
INVESTIGATING_HYPOTHESES, BUILDING_DIAGNOSIS, OUTPUT_READY, HANDED_OFF,
|
|
BLOCKED, FAILED_TO_ANALYZE, REOPENED.
|
|
|
|
`routing` is separate and requires:
|
|
|
|
- `destination`: null or inbox, quarantine, cases, outbox, review; a logical route,
|
|
never a filesystem path. Cache and runtime are service storage, not routes.
|
|
- `reason`: null or nonblank explanatory string.
|
|
- `retry_count`: nonnegative integer, initially zero.
|
|
- `retry_not_before`: null or RFC-3339 timestamp.
|
|
|
|
No state/route combination triggers execution. This milestone checks the contract
|
|
only, without implementing routing, retry, locking or lifecycle transitions.
|
|
|
|
## References and integrity
|
|
|
|
`artifacts` requires `case`, `diagnosis`, `change_request`, each null or an internal
|
|
file path. Additional artifact pointers follow the same rule. `history` requires
|
|
nonnegative counters `analysis_revisions`, `reproductions`, `evidence`, `handoffs`,
|
|
`feedback`, and nullable `events_ref`. Counters are supplied summaries; the reader
|
|
does not infer them from directory names or impose diagnostic semantics.
|
|
|
|
`integrity` maps internal file paths to `{ "sha256": "<64 lowercase hex digits>",
|
|
"size": <uncompressed byte count> }`. All supplied records are checked. A fresh
|
|
package may have an empty map. Producers should cover important immutable
|
|
artifacts. The format foundation does not infer which project artifacts are
|
|
important. Hashes detect changes, not authenticity. The manifest cannot reference
|
|
or hash itself. Every non-null reference must resolve to an actual ZIP file.
|
|
|
|
The bundled `manifest-v1.schema.json` uses JSON Schema Draft 2020-12 and only local
|
|
schema references; validation requires no network. Unknown extension fields are
|
|
accepted and retained. Unsupported format versions, duplicate JSON keys,
|
|
non-UTF-8 manifests and nonstandard numeric constants are rejected.
|
|
|
|
## Preservation
|
|
|
|
`original/` and completed `analysis/revisions/NNNN/` are immutable. New evidence,
|
|
feedback and revisions are appended. Only explicitly designated index/pointer
|
|
files (`manifest.json`, `analysis/current.json`) may later be updated. Unknown
|
|
files are preserved unchanged. Validator and reader never write, extract, delete
|
|
or rewrite anything. The writer checks append-only and output-superset preservation
|
|
against its input. Historical preservation cannot be proven from one standalone
|
|
incoming archive.
|
|
|
|
## ZIP acceptance and configurable security policy
|
|
|
|
This implementation accepts stored and deflated ZIP entries, including ZIP64
|
|
within configured limits. It rejects encryption, symlinks and other special
|
|
filesystem entries, unsafe paths, duplicate entries, case-insensitive collisions,
|
|
and file/directory conflicts. Paths must be relative POSIX paths without empty,
|
|
dot or parent segments, backslashes, colons or control characters. Explicit empty
|
|
directory entries are accepted. Paths are rejected, never silently repaired.
|
|
The reader does not extract files; future extraction requires its own checks.
|
|
|
|
`PackageLimits` is a configurable operational security policy. These defaults
|
|
are **not immutable limits of the v1 package format** and do not appear in the
|
|
manifest schema:
|
|
|
|
| Limit | Default |
|
|
|---|---:|
|
|
| Archive bytes | 512 MiB |
|
|
| Expanded total bytes | 2 GiB |
|
|
| Expanded file bytes | 256 MiB |
|
|
| Manifest bytes | 1 MiB |
|
|
| ZIP entries (including directories) | 10,000 |
|
|
| Per-entry compression ratio | 200:1 |
|
|
|
|
Declared sizes are checked before reading. Actual reads are bounded and total
|
|
bytes are counted while streaming every file for CRC verification; listed SHA-256
|
|
hashes are checked during the same pass. Manifest parsing is bounded separately.
|
|
Malformed archive, I/O, JSON, schema and integrity failures produce structured
|
|
validation issues. Programmer errors and process-level failures are not hidden.
|
|
Input files must not be modified concurrently while open; the reader retains the
|
|
same file handle and archive used for validation.
|
|
|
|
## Public API
|
|
|
|
```python
|
|
from viano_bug_analysis.package import (
|
|
PackageLimits, PackageValidator, PackageReader, PackageReadError,
|
|
)
|
|
|
|
result = PackageValidator(PackageLimits()).validate("renamed-package.bin")
|
|
if not result.valid:
|
|
for issue in result.issues:
|
|
print(issue.code, issue.path, issue.message)
|
|
|
|
try:
|
|
with PackageReader("renamed-package.bin") as package:
|
|
manifest = package.manifest # defensive copy, including extensions
|
|
inventory = package.files # all file paths, including unknown files
|
|
data = package.read_bytes("original/bug_report.json")
|
|
except PackageReadError as error:
|
|
issues = error.result.issues
|
|
```
|
|
|
|
Read access outside the context is a programming error (`RuntimeError`). Invalid
|
|
packages and missing member requests raise `PackageReadError` with structured
|
|
details. `read_bytes` is bounded by policy but materializes a single file in memory;
|
|
validation itself streams. No artifact interpretation is performed.
|
|
|
|
## Writer API and publication contract
|
|
|
|
`PackageFactory.create(destination, manifest, artifacts)` creates a generation-zero
|
|
package from caller-supplied metadata and byte artifacts. It adds integrity records,
|
|
applies PackageLimits, validates the staged ZIP, flushes/fsyncs and publishes
|
|
atomically without replacing an existing target. It is project-independent;
|
|
subsequent enrichment still uses PackageWriter.
|
|
|
|
```python
|
|
from viano_bug_analysis.package import PackageWriter, PackageWriteError
|
|
|
|
try:
|
|
result = PackageWriter().write(
|
|
"input.any-name",
|
|
"next-generation.any-name",
|
|
additions={"case/revisions/0001/case.json": b'{"example": true}'},
|
|
index_updates={"analysis/current.json": b'{"revision": 1}'},
|
|
manifest_updates={
|
|
"state": "VALIDATED",
|
|
"routing": {"destination": "cases"},
|
|
"artifacts": {"case": "case/revisions/0001/case.json"},
|
|
},
|
|
)
|
|
except PackageWriteError as error:
|
|
issues = error.result.issues
|
|
```
|
|
|
|
`additions` accepts new file paths and bytes only. Existing files cannot be
|
|
overwritten, even with identical bytes. Unknown input files and explicit directory
|
|
entries are carried forward. ZIP comments and retained entry metadata are copied;
|
|
compressed representations may change, but retained file bytes must be identical.
|
|
Existing `analysis/revisions/<revision>/` namespaces are frozen, including empty
|
|
directories: this conservative foundation does not infer whether a revision is
|
|
still active. New revisions use new namespaces. New reproductions and feedback
|
|
use new paths. There is no deletion operation.
|
|
|
|
`index_updates` can create or replace only `analysis/current.json`. `manifest.json`
|
|
is managed exclusively by the writer. Index bytes are opaque at this milestone;
|
|
no interpretation or current-index schema is implemented.
|
|
|
|
`manifest_updates` accepts only `state`, `routing`, `analysis_revision`, `artifacts`
|
|
and `history`. Object updates merge recursively, preserving nested extension
|
|
fields. Identity, format, project and creation time remain unchanged. The writer
|
|
sets generation to input generation plus one, sets updated time to the later of
|
|
the current UTC time and the previous updated time, and maintains SHA-256/size
|
|
records for every output file except the manifest. Existing integrity-record
|
|
extensions are retained. Analysis revision and standard history counters cannot
|
|
decrease; they are otherwise supplied explicitly, never inferred from generation
|
|
or artifact names. No lifecycle transition policy or automatic history event is
|
|
introduced. Existing `history/events.jsonl` cannot be replaced: a future event
|
|
writer needs an explicit byte-prefix append contract; for now use a new event
|
|
artifact and update `events_ref` if needed.
|
|
|
|
The source is fully validated before staging. Output uses ZIP_STORED so highly
|
|
compressible new data does not create archives rejected by compression-ratio
|
|
defaults. The same configurable `PackageLimits` apply to input and output. This
|
|
may require a larger archive-byte policy for a previously compressed input.
|
|
Copying and preservation comparison stream in bounded chunks; additions are
|
|
caller-provided bytes. The source must remain unchanged throughout the call.
|
|
|
|
Staging occurs in an exclusively created temporary file in the destination's
|
|
existing parent directory. The ZIP is closed, flushed and fsynced, then fully
|
|
validated. Its manifest must equal the planned manifest; every input entry must
|
|
survive, and every protected file is compared byte for byte. The target is only
|
|
published after these checks. Normal failures remove staging files and leave
|
|
the input and existing target unchanged. A process crash may leave an unpublished
|
|
staging file; this milestone does not implement recovery jobs.
|
|
An exceptional cleanup failure before publication raises a structured error;
|
|
after publication it emits a ResourceWarning and still returns success, since
|
|
the valid target has already been committed. A staging file may remain in that
|
|
case and can be removed by the caller.
|
|
|
|
By default an existing target is rejected. Atomic no-clobber publication uses a
|
|
same-directory hard link, preventing concurrent writers from silently replacing
|
|
each other; the destination filesystem must support hard links. `overwrite=True`
|
|
explicitly permits atomic replacement using `os.replace`. Input aliases (including
|
|
hard links), symbolic-link targets and in-place output are rejected. Temporary
|
|
files are cleaned up after publication. This is atomic visibility with a flushed
|
|
file, not a claim of portable directory-metadata durability across power loss.
|
|
Destination directories and input files must be controlled by the caller; this
|
|
milestone does not implement case locking or protection from concurrent external
|
|
filesystem mutation. Every successful write, including one without additions,
|
|
produces exactly one new generation. Job-level retry idempotence is deferred to
|
|
the later lifecycle layer.
|