Add append-only package writer with atomic validated publication

This commit is contained in:
Nico committed 2026-10-13 12:17:10 +02:00
1 parent 112487242a
commit f832b038a5
7 files changed
+584 -10

No files matched your search

+84 -5
View File
@@ -9,8 +9,8 @@ Exactly one root `manifest.json` is the authoritative entry point.
`format` is `viano-bug-case-package`; `format_version` is integer `1`.
`package_id` is an opaque nonblank string identifying the case package throughout
its lifetime. Enriching and emitting it again preserves this identity and, in a
future writer, increases `package_generation` (a nonnegative integer).
its lifetime. Enriching and emitting it again preserves this identity and
increases `package_generation` (a nonnegative integer).
`case_id` identifies the case. `analysis_revision` is a separate nonnegative
integer, initially zero; it is independent of package generation.
The reader does not derive or increment any of these values.
@@ -63,9 +63,9 @@ non-UTF-8 manifests and nonstandard numeric constants are rejected.
feedback and revisions are appended. Only explicitly designated index/pointer
files (`manifest.json`, `analysis/current.json`) may later be updated. Unknown
files are preserved unchanged. Validator and reader never write, extract, delete
or rewrite anything. Historical append-only and output-superset comparisons,
atomic generation and generation increments belong to a future writer; they
cannot be proven from one standalone incoming archive.
or rewrite anything. The writer checks append-only and output-superset preservation
against its input. Historical preservation cannot be proven from one standalone
incoming archive.
## ZIP acceptance and configurable security policy
@@ -123,3 +123,82 @@ Read access outside the context is a programming error (`RuntimeError`). Invalid
packages and missing member requests raise `PackageReadError` with structured
details. `read_bytes` is bounded by policy but materializes a single file in memory;
validation itself streams. No artifact interpretation is performed.
## Writer API and publication contract
```python
from viano_bug_analysis.package import PackageWriter, PackageWriteError
try:
result = PackageWriter().write(
"input.any-name",
"next-generation.any-name",
additions={"case/revisions/0001/case.json": b'{"example": true}'},
index_updates={"analysis/current.json": b'{"revision": 1}'},
manifest_updates={
"state": "VALIDATED",
"routing": {"destination": "cases"},
"artifacts": {"case": "case/revisions/0001/case.json"},
},
)
except PackageWriteError as error:
issues = error.result.issues
```
`additions` accepts new file paths and bytes only. Existing files cannot be
overwritten, even with identical bytes. Unknown input files and explicit directory
entries are carried forward. ZIP comments and retained entry metadata are copied;
compressed representations may change, but retained file bytes must be identical.
Existing `analysis/revisions/<revision>/` namespaces are frozen, including empty
directories: this conservative foundation does not infer whether a revision is
still active. New revisions use new namespaces. New reproductions and feedback
use new paths. There is no deletion operation.
`index_updates` can create or replace only `analysis/current.json`. `manifest.json`
is managed exclusively by the writer. Index bytes are opaque at this milestone;
no interpretation or current-index schema is implemented.
`manifest_updates` accepts only `state`, `routing`, `analysis_revision`, `artifacts`
and `history`. Object updates merge recursively, preserving nested extension
fields. Identity, format, project and creation time remain unchanged. The writer
sets generation to input generation plus one, sets updated time to the later of
the current UTC time and the previous updated time, and maintains SHA-256/size
records for every output file except the manifest. Existing integrity-record
extensions are retained. Analysis revision and standard history counters cannot
decrease; they are otherwise supplied explicitly, never inferred from generation
or artifact names. No lifecycle transition policy or automatic history event is
introduced. Existing `history/events.jsonl` cannot be replaced: a future event
writer needs an explicit byte-prefix append contract; for now use a new event
artifact and update `events_ref` if needed.
The source is fully validated before staging. Output uses ZIP_STORED so highly
compressible new data does not create archives rejected by compression-ratio
defaults. The same configurable `PackageLimits` apply to input and output. This
may require a larger archive-byte policy for a previously compressed input.
Copying and preservation comparison stream in bounded chunks; additions are
caller-provided bytes. The source must remain unchanged throughout the call.
Staging occurs in an exclusively created temporary file in the destination's
existing parent directory. The ZIP is closed, flushed and fsynced, then fully
validated. Its manifest must equal the planned manifest; every input entry must
survive, and every protected file is compared byte for byte. The target is only
published after these checks. Normal failures remove staging files and leave
the input and existing target unchanged. A process crash may leave an unpublished
staging file; this milestone does not implement recovery jobs.
An exceptional cleanup failure before publication raises a structured error;
after publication it emits a ResourceWarning and still returns success, since
the valid target has already been committed. A staging file may remain in that
case and can be removed by the caller.
By default an existing target is rejected. Atomic no-clobber publication uses a
same-directory hard link, preventing concurrent writers from silently replacing
each other; the destination filesystem must support hard links. `overwrite=True`
explicitly permits atomic replacement using `os.replace`. Input aliases (including
hard links), symbolic-link targets and in-place output are rejected. Temporary
files are cleaned up after publication. This is atomic visibility with a flushed
file, not a claim of portable directory-metadata durability across power loss.
Destination directories and input files must be controlled by the caller; this
milestone does not implement case locking or protection from concurrent external
filesystem mutation. Every successful write, including one without additions,
produces exactly one new generation. Job-level retry idempotence is deferred to
the later lifecycle layer.