Add append-only package writer with atomic validated publication
This commit is contained in:
1 parent
112487242a
commit
f832b038a5
7 files changed
+584
-10
No files matched your search
@@ -9,8 +9,8 @@ Exactly one root `manifest.json` is the authoritative entry point.
|
||||
|
||||
`format` is `viano-bug-case-package`; `format_version` is integer `1`.
|
||||
`package_id` is an opaque nonblank string identifying the case package throughout
|
||||
its lifetime. Enriching and emitting it again preserves this identity and, in a
|
||||
future writer, increases `package_generation` (a nonnegative integer).
|
||||
its lifetime. Enriching and emitting it again preserves this identity and
|
||||
increases `package_generation` (a nonnegative integer).
|
||||
`case_id` identifies the case. `analysis_revision` is a separate nonnegative
|
||||
integer, initially zero; it is independent of package generation.
|
||||
The reader does not derive or increment any of these values.
|
||||
@@ -63,9 +63,9 @@ non-UTF-8 manifests and nonstandard numeric constants are rejected.
|
||||
feedback and revisions are appended. Only explicitly designated index/pointer
|
||||
files (`manifest.json`, `analysis/current.json`) may later be updated. Unknown
|
||||
files are preserved unchanged. Validator and reader never write, extract, delete
|
||||
or rewrite anything. Historical append-only and output-superset comparisons,
|
||||
atomic generation and generation increments belong to a future writer; they
|
||||
cannot be proven from one standalone incoming archive.
|
||||
or rewrite anything. The writer checks append-only and output-superset preservation
|
||||
against its input. Historical preservation cannot be proven from one standalone
|
||||
incoming archive.
|
||||
|
||||
## ZIP acceptance and configurable security policy
|
||||
|
||||
@@ -123,3 +123,82 @@ Read access outside the context is a programming error (`RuntimeError`). Invalid
|
||||
packages and missing member requests raise `PackageReadError` with structured
|
||||
details. `read_bytes` is bounded by policy but materializes a single file in memory;
|
||||
validation itself streams. No artifact interpretation is performed.
|
||||
|
||||
## Writer API and publication contract
|
||||
|
||||
```python
|
||||
from viano_bug_analysis.package import PackageWriter, PackageWriteError
|
||||
|
||||
try:
|
||||
result = PackageWriter().write(
|
||||
"input.any-name",
|
||||
"next-generation.any-name",
|
||||
additions={"case/revisions/0001/case.json": b'{"example": true}'},
|
||||
index_updates={"analysis/current.json": b'{"revision": 1}'},
|
||||
manifest_updates={
|
||||
"state": "VALIDATED",
|
||||
"routing": {"destination": "cases"},
|
||||
"artifacts": {"case": "case/revisions/0001/case.json"},
|
||||
},
|
||||
)
|
||||
except PackageWriteError as error:
|
||||
issues = error.result.issues
|
||||
```
|
||||
|
||||
`additions` accepts new file paths and bytes only. Existing files cannot be
|
||||
overwritten, even with identical bytes. Unknown input files and explicit directory
|
||||
entries are carried forward. ZIP comments and retained entry metadata are copied;
|
||||
compressed representations may change, but retained file bytes must be identical.
|
||||
Existing `analysis/revisions/<revision>/` namespaces are frozen, including empty
|
||||
directories: this conservative foundation does not infer whether a revision is
|
||||
still active. New revisions use new namespaces. New reproductions and feedback
|
||||
use new paths. There is no deletion operation.
|
||||
|
||||
`index_updates` can create or replace only `analysis/current.json`. `manifest.json`
|
||||
is managed exclusively by the writer. Index bytes are opaque at this milestone;
|
||||
no interpretation or current-index schema is implemented.
|
||||
|
||||
`manifest_updates` accepts only `state`, `routing`, `analysis_revision`, `artifacts`
|
||||
and `history`. Object updates merge recursively, preserving nested extension
|
||||
fields. Identity, format, project and creation time remain unchanged. The writer
|
||||
sets generation to input generation plus one, sets updated time to the later of
|
||||
the current UTC time and the previous updated time, and maintains SHA-256/size
|
||||
records for every output file except the manifest. Existing integrity-record
|
||||
extensions are retained. Analysis revision and standard history counters cannot
|
||||
decrease; they are otherwise supplied explicitly, never inferred from generation
|
||||
or artifact names. No lifecycle transition policy or automatic history event is
|
||||
introduced. Existing `history/events.jsonl` cannot be replaced: a future event
|
||||
writer needs an explicit byte-prefix append contract; for now use a new event
|
||||
artifact and update `events_ref` if needed.
|
||||
|
||||
The source is fully validated before staging. Output uses ZIP_STORED so highly
|
||||
compressible new data does not create archives rejected by compression-ratio
|
||||
defaults. The same configurable `PackageLimits` apply to input and output. This
|
||||
may require a larger archive-byte policy for a previously compressed input.
|
||||
Copying and preservation comparison stream in bounded chunks; additions are
|
||||
caller-provided bytes. The source must remain unchanged throughout the call.
|
||||
|
||||
Staging occurs in an exclusively created temporary file in the destination's
|
||||
existing parent directory. The ZIP is closed, flushed and fsynced, then fully
|
||||
validated. Its manifest must equal the planned manifest; every input entry must
|
||||
survive, and every protected file is compared byte for byte. The target is only
|
||||
published after these checks. Normal failures remove staging files and leave
|
||||
the input and existing target unchanged. A process crash may leave an unpublished
|
||||
staging file; this milestone does not implement recovery jobs.
|
||||
An exceptional cleanup failure before publication raises a structured error;
|
||||
after publication it emits a ResourceWarning and still returns success, since
|
||||
the valid target has already been committed. A staging file may remain in that
|
||||
case and can be removed by the caller.
|
||||
|
||||
By default an existing target is rejected. Atomic no-clobber publication uses a
|
||||
same-directory hard link, preventing concurrent writers from silently replacing
|
||||
each other; the destination filesystem must support hard links. `overwrite=True`
|
||||
explicitly permits atomic replacement using `os.replace`. Input aliases (including
|
||||
hard links), symbolic-link targets and in-place output are rejected. Temporary
|
||||
files are cleaned up after publication. This is atomic visibility with a flushed
|
||||
file, not a claim of portable directory-metadata durability across power loss.
|
||||
Destination directories and input files must be controlled by the caller; this
|
||||
milestone does not implement case locking or protection from concurrent external
|
||||
filesystem mutation. Every successful write, including one without additions,
|
||||
produces exactly one new generation. Job-level retry idempotence is deferred to
|
||||
the later lifecycle layer.
|
||||
Reference in new issue
Block a user