Add scoped operator controls and durable worker release queue

config/api.example.toml

@@ -19,3 +19,10 @@ planning_mode = "joint"
 id = "wow-capture-worker"
 token_env = "VBA_INTAKE_TOKEN"
 projects = ["viano-core"]
+
+# Separate operator identity/token. Submission clients remain non-operators.
+# [[clients]]
+# id = "human-operator"
+# token_env = "VBA_OPERATOR_TOKEN"
+# projects = ["viano-core"]
+# operator = true

docs/analysis-worker.md

@@ -148,6 +148,13 @@ löschen oder Untersuchungsbudgets zu verbrauchen. Die konfigurierbare
 20-GB-Vorgabe für Entwicklungs-/Testdaten bleibt davon unabhängig. Der Worker
 löscht keine Paketgenerationen und keine Inhalte erhaltener Akten.
 
+Die [Web-Betriebssteuerung](web-console.md#betriebssteuerung) kann neue Fälle je
+Projekt pausieren, während begonnene Analysen abschließen. Freigabeaufträge werden
+dauerhaft gespeichert und zwischen Dispatches unter derselben Fall-Lease geprüft.
+Damit ist für Web-Freigaben kein Worker-Stopp nötig. CLI-Freigaben benötigen
+weiterhin den gestoppten Dienst. Alle fachlichen Retry- und Budgetregeln gelten
+für beide Wege; ein veralteter Webauftrag entsperrt keinen neueren Paketstand.
+
 Ctrl+C bzw. SIGTERM fordert einen geordneten Stopp an. Der laufende Dispatch darf
 noch abschließen; erst danach wird die Prozess-Lease freigegeben. Ein Dispatch
 kann mehrere interne Schritte/Modellanfragen enthalten und entsprechend dauern.

docs/test-results/operator-control-20261022.md

@@ -0,0 +1,28 @@
+# Betriebssteuerung – 2026-10-22
+
+Umfang: projektbezogene Pause neuer Analysen mit Weiterlauf begonnener Fälle,
+dauerhafte Fortsetzungsaufträge zwischen Worker-Dispatches, getrennte Betreiberrechte,
+Aktionsprotokoll sowie Workerstatus im Web. Keine Prompt-/Budgetänderungen und keine
+Änderung der maßgeblichen Designspezifikation.
+
+Eingaben: künstliche CaseBuilder-/Worker-/API-Fixtures und synthetischer UI-Server.
+Keine kostenpflichtigen Modellanfragen und keine externen DB-/Repositoryzugriffe.
+
+Prüfungen:
+
+- Worker-, Dashboard-, vcore-Intake- und API-Tests: 50 bestanden.
+- Persistente Pause nach Worker-Neustart, Weiterlauf begonnener Fälle,
+  aktuelle und veraltete Freigaben, Ablehnung von Modellvalidierungssperren.
+- API-Berechtigungen, Projektscope, Protokollsichtbarkeit, ungültige/zu große
+  Steuerungsnachrichten und unbekannte Zusatzfelder.
+- Echter Headless-Edge: Pause/Wiederzulassung, Aktionsprotokoll, Desktop/Mobil,
+  bisherige Diagramme/Falldetails/Abmeldung: PASS, keine Browserfehler/Überläufe.
+  Nach zwei Fehlern beim Test-Abschluss wurde geordnetes Browserbeenden ergänzt;
+  der abschließende Test endet erfolgreich einschließlich temporärer Bereinigung.
+- Test-Workspaces vor Testbeginn: 16.939.129.428 Bytes; zusammen mit kleinen
+  Fixtures und temporären lokalen Tests unter 20 GB. Keine alten Akten gelöscht.
+
+Grenzen: HTTP-Annahme bleibt während einer Pause verfügbar; Pause stoppt neue
+Analysen an Worker-Grenzen, keine laufenden Modellaufrufe. Freigaben sind asynchron
+und können bei erneuter Prüfung abgelehnt werden. Ein gestoppter Worker wird durch
+die Weboberfläche nicht als Betriebssystemdienst gestartet.

docs/web-console.md

@@ -70,8 +70,8 @@ Die Oberfläche bietet ein einfaches Formular für neue wowdiag-/wowsrc-Aufträg
 für abgeschlossene ZIP-Akten mit Testerfeedback. Es verwendet dieselbe Multipart-
 API und denselben Idempotenzmechanismus wie externe Programme. Bei unveränderten
 Eingaben verwendet eine Wiederholung denselben Schlüssel. Anhänge und Fallakten
-werden nicht clientseitig bearbeitet. Administrative Worker-Eingriffe bleiben ein
+werden nicht clientseitig bearbeitet. Betreiberaktionen stehen separat berechtigten
-weiterer Ausbauschritt.
+Clients zur Verfügung.
 
 ## Fallakte und Ergebnisse
 
@@ -88,8 +88,43 @@ Die Revisionsauswahl liest akzeptierte Artefakte aus der Checkpointhistorie der
 aktuellen vollständigen Akte. Ergebnisse neuerer Revisionen erscheinen nicht in
 älteren Ansichten. Über die Übergabe-API erfasste Patch-Rückmeldungen werden bis
 zur gewählten Revision dargestellt. Bei gesperrten Fällen erscheinen Sperrgrund
-und passende Betreiberhinweise; die Ansicht setzt keine Budgets zurück und hebt
+und passende Betreiberhinweise. Für berechtigte Betreiber kann eine zulässige
-keine Sperren auf.
+Fortsetzung angefordert werden; Budgets werden dabei nicht zurückgesetzt.
+
+## Betriebssteuerung
+
+Clients erhalten standardmäßig keine Betreiberrechte. Ein eigener Client mit
+eigenem Token und `operator = true` in der API-Konfiguration erhält Aktionen für
+seine freigegebenen Projekte (Beispiel in `config/api.example.toml`). API nach einer
+Rechteänderung neu starten. Der Gemini-Schlüssel ist dafür nicht geeignet.
+
+- **Neue Fälle pausieren / wieder zulassen:** gilt je Projekt und bleibt nach
+  Neustart erhalten. HTTP-Aufträge werden weiterhin dauerhaft angenommen. Der
+  Worker lässt sie im Eingang bzw. wartet bei noch nicht begonnenen Fällen.
+  Begonnene Analysen laufen weiter. Eine Pause unterbricht keinen Modellaufruf.
+- **Fortsetzung nach Fehlerbehebung anfordern:** in der aktuellen Falldetailansicht
+  bei passenden Schedulerfehlern bzw. `RESEARCH_UNAVAILABLE`. Die API bestätigt
+  nur `QUEUED`, nicht bereits eine Freigabe. Der lebende Worker prüft den Auftrag
+  zwischen Dispatches erneut; ein gestoppter Worker bearbeitet ihn nach Neustart.
+- Veraltete Paketgeneration, geänderte Konfigurationsbindung, noch nicht fällige
+  Research-Sperre und unzulässige fachliche Sperren werden abgewiesen. Keine
+  Zurücksetzung von Hypothesenbudgets, Versuchszählern oder LLM-Operationen.
+- Das Aktionsprotokoll dokumentiert Clientkennung, UTC-Zeit, Projekt, Aktion und
+  Ergebnis. Fortsetzungen haben getrennte Eingangs- und Ausführungseinträge mit
+  derselben Auftragskennung. Die Ansicht zeigt die letzten 100 berechtigten
+  Einträge; das vollständige Protokoll bleibt unter `runtime/worker/control/`.
+  Es enthält keine Tokens oder Konfigurationsinhalte. Erfolgreiche lokale
+  CLI-Freigaben erscheinen als `local-cli`.
+- Workerdetails zeigen Erreichbarkeit, letzten Zustand, Aktivität und Fehlercode.
+  Ein alter Snapshot gilt nicht als lebender Worker. Globale Workerdetails sind
+  nur Betreibern mit Zugriff auf alle konfigurierten Projekte zugänglich.
+
+`GET /v1/operations` liefert Rechte, projektbezogenen Pausenzustand und für Betreiber
+Protokoll/Workerzustand. `POST /v1/operations` benötigt Betreiberrechte und JSON:
+`{"action":"pause","project_id":"viano-core"}` bzw. `resume`, oder
+`{"action":"retry","key":"<Schlüssel aus der Übersicht>","package_generation":42}`.
+Bei gleichzeitigem Zugriff kann `API_CONTROL_BUSY` auftreten; nach Aktualisierung
+erneut versuchen. Der Worker muss nach diesem Softwareupdate neu gestartet werden.
 
 `GET /v1/cases/{key}?revision=N` liefert diese projektgebundene Ansicht.
 `GET /v1/cases/{key}/package` lädt die vollständige aktuelle Akte einschließlich

src/viano_bug_analysis/api/app.py

@@ -1,6 +1,7 @@
 import asyncio
 import functools
 import hmac
+import json
 import shutil
 import tempfile
 from pathlib import Path
@@ -19,6 +20,8 @@ from .contracts import schema
 from .service import IntakeService
 from .dashboard import Dashboard
 from .cases import Cases
+from .operations import Operations
+from viano_bug_analysis.lifecycle.errors import LockBusy
 from viano_bug_analysis.lifecycle.errors import LifecycleError
 from viano_bug_analysis.package import PackageReadError
 from importlib.resources import files
@@ -28,6 +31,7 @@ def create_app(config):
     service = IntakeService(config)
     dashboard = Dashboard(service)
     cases = Cases(service)
+    operations = Operations(service)
     tokens = config.tokens()
     service.recover()
     active = 0
@@ -84,14 +88,45 @@ def create_app(config):
         response.headers['Cache-Control'] = 'no-store'
         return response
 
+    async def operation(request):
+        client = authenticate(request)
+        if request.method == 'POST':
+            if not client.operator:
+                raise APIError('API_FORBIDDEN',403)
+            if request.headers.get('content-type','').split(';')[0] != 'application/json':
+                raise APIError('API_CONTENT_TYPE',415)
+            body=bytearray()
+            async for chunk in request.stream():
+                body.extend(chunk)
+                if len(body)>4096:
+                    raise APIError('API_CONTROL_INVALID',413)
+            try:
+                data=json.loads(body)
+            except ValueError:
+                raise APIError('API_CONTROL_INVALID',422) from None
+            try:
+                result=await run_in_threadpool(operations.change,client,data)
+            except LockBusy:
+                raise APIError('API_CONTROL_BUSY',409) from None
+            except (LifecycleError,PackageReadError):
+                raise APIError('API_CASE_UNAVAILABLE',409) from None
+            except (ValueError,KeyError,TypeError):
+                raise APIError('API_CONTROL_UNAVAILABLE',503) from None
+        else:
+            try:
+                result=await run_in_threadpool(operations.view,client)
+            except (ValueError,KeyError,TypeError):
+                raise APIError('API_CONTROL_UNAVAILABLE',503) from None
+        return JSONResponse(result,headers={'Cache-Control':'no-store'})
+
     async def web(request):
         from starlette.responses import Response
         name = request.path_params.get('asset', 'index.html')
-        if name not in {'index.html', 'dashboard.js', 'dashboard.css', 'case-detail.js'}:
+        if name not in {'index.html', 'dashboard.js', 'dashboard.css', 'case-detail.js','operations.js'}:
             raise APIError('API_NOT_FOUND', 404)
         data = files('viano_bug_analysis.web').joinpath(name).read_bytes()
         response = Response(data, media_type={'index.html': 'text/html', 'dashboard.js': 'text/javascript',
-            'dashboard.css': 'text/css', 'case-detail.js':'text/javascript'}[name])
+            'dashboard.css': 'text/css', 'case-detail.js':'text/javascript','operations.js':'text/javascript'}[name])
         response.headers.update({'Content-Security-Policy': "default-src 'self'; script-src 'self'; style-src 'self'; connect-src 'self'; object-src 'none'; base-uri 'none'; frame-ancestors 'none'",
             'X-Content-Type-Options': 'nosniff', 'Referrer-Policy': 'no-referrer', 'Cache-Control': 'no-store'})
         return response
@@ -165,6 +200,7 @@ def create_app(config):
 
     app = Starlette(routes=[Route('/', web), Route('/assets/{asset}', web),
         Route('/v1/overview', overview), Route('/healthz', health), Route('/v1/profiles', profiles),
+        Route('/v1/operations',operation,methods=['GET','POST']),
         Route('/v1/cases/{handle}', case_detail), Route('/v1/cases/{handle}/package', case_package),
         Route('/v1/schema', contract), Route('/v1/submissions', submit, methods=['POST']),
         Route('/v1/submissions/{submission}', status), Route('/v1/submissions/{submission}/package', package)],

src/viano_bug_analysis/api/config.py

@@ -66,6 +66,7 @@ class Client:
     id: str
     token_env: str
     projects: tuple[str, ...]
+    operator: bool = False
 
 
 @dataclass(frozen=True)
@@ -85,7 +86,7 @@ class APIConfig:
                 or not isinstance(self.host, str) or not self.host):
             raise APIError('API_CONFIG', 500)
         for c in self.clients:
-            if (not isinstance(c.id, str) or not IDENTIFIER.fullmatch(c.id)
+            if (type(c.operator) is not bool or not isinstance(c.id, str) or not IDENTIFIER.fullmatch(c.id)
                     or not isinstance(c.token_env, str)
                     or not re.fullmatch(r'[A-Za-z_][A-Za-z0-9_]*', c.token_env)
                     or not c.projects or not set(c.projects) <= ids):

src/viano_bug_analysis/api/operations.py

@@ -0,0 +1,72 @@
+"""Scoped operator endpoints; mutations are durable intent, never model calls."""
+import json
+from datetime import datetime, timezone
+
+from .config import APIError
+from .cases import Cases
+from viano_bug_analysis.worker.control import Control
+from viano_bug_analysis.package import PackageReader
+from viano_bug_analysis.telemetry import read_status
+
+
+class Operations:
+    def __init__(self, service):
+        self.service = service
+        self.control = Control(service.workspace)
+
+    def view(self, client):
+        value = self.control.read()
+        path = self.service.workspace.runtime / 'worker/status.json'
+        try:
+            status = json.loads(path.read_bytes()) if path.exists() else {}
+            if not isinstance(status,dict):
+                status={}
+        except (OSError,ValueError):
+            status={}
+        try:
+            telemetry = read_status(self.service.workspace.root)
+            live = (telemetry.get('observer') == 'LIVE' and bool(status.get('service_id'))
+                and telemetry.get('service_id') == status.get('service_id') and status.get('status') != 'STOPPED')
+        except Exception:
+            live = False
+        # A shared worker's case/error context may belong to another project.
+        # Non-operators see only scoped pause state; global health is operator-only.
+        full_scope = set(client.projects) == {p.id for p in self.service.config.projects}
+        worker = ({k:status.get(k) for k in ('status','updated_at','code')} | {'live':live}
+            if client.operator and full_scope else None)
+        return {'operator':client.operator, 'worker':worker,
+            'projects':[{'project_id':p,'paused':value['paused'].get(p,False)} for p in client.projects],
+            'events':[e for e in value['events'] if e['project_id'] in client.projects][-100:] if client.operator else [],
+            'pending':sum(c['project_id'] in client.projects for c in value['commands']) if client.operator else 0}
+
+    def change(self, client, data):
+        if not client.operator:
+            raise APIError('API_FORBIDDEN',403)
+        if not isinstance(data,dict):
+            raise APIError('API_CONTROL_INVALID',422)
+        action=data.get('action')
+        if not isinstance(action,str):
+            raise APIError('API_CONTROL_INVALID',422)
+        if action in {'pause','resume'} and set(data)=={'action','project_id'}:
+            project=data['project_id']
+            if project not in client.projects:
+                raise APIError('API_FORBIDDEN',403)
+            self.control.pause(client.id,project,action=='pause')
+            return {'status':'APPLIED'}
+        if action=='retry' and set(data)=={'action','key','package_generation'}:
+            if type(data['package_generation']) is not int or not isinstance(data['key'],str):
+                raise APIError('API_CONTROL_INVALID',422)
+            path=Cases(self.service).head(data['key'],client)
+            with PackageReader(path) as reader:
+                manifest=reader.manifest
+            if manifest['package_generation'] != data['package_generation']:
+                raise APIError('API_CONTROL_STALE',409)
+            if (manifest['state']=='BLOCKED' and (manifest['routing']['reason']!='RESEARCH_UNAVAILABLE'
+                    or (manifest['routing']['retry_not_before'] and datetime.fromisoformat(
+                        manifest['routing']['retry_not_before'].replace('Z','+00:00')) > datetime.now(timezone.utc)))):
+                raise APIError('API_CONTROL_RETRY_STATE',409)
+            if manifest['state'] in {'OUTPUT_READY','HANDED_OFF','FAILED_TO_ANALYZE'}:
+                raise APIError('API_CONTROL_RETRY_STATE',409)
+            command=self.control.enqueue(client.id,manifest['project']['id'],manifest)
+            return {'status':'QUEUED','command_id':command}
+        raise APIError('API_CONTROL_INVALID',422)

src/viano_bug_analysis/web/case-detail.js

@@ -62,7 +62,9 @@ function renderDetail(data){
   const actions=text('div','','next-actions');
   if(data.reason)actions.append(text('p',`Sperr-/Reviewgrund: ${data.reason}`));
   if(data.worker.code)actions.append(text('p',`Workerhinweis: ${data.worker.code}`));
-  data.actions.forEach(a=>actions.append(text('p',a.text)));root.append(panel('Zustand und nächste Schritte',actions));
+  data.actions.forEach(a=>actions.append(text('p',a.text)));
+  if(typeof addRetryControl==='function')addRetryControl(data,actions);
+  root.append(panel('Zustand und nächste Schritte',actions));
   root.append(panel('Testerbeobachtungen',semantic({observations:data.observations,tester_assumptions:data.tester_assumptions,expected_behavior:data.expected_behavior})));
   root.append(panel('Caseinterpretation',Object.keys(data.interpretation).length?semantic(data.interpretation):text('p','Für diese Revision liegt noch keine akzeptierte Interpretation vor.','hint'),false));
   const hypotheses=text('div','','hypothesis-list');

src/viano_bug_analysis/web/dashboard.js

@@ -26,6 +26,7 @@ function logout() {
   $('submit-form').reset();$('receipt').textContent='';$('new-files').hidden=false;$('return-files').hidden=true;
   ['workers','case-rows','bars','legend','project','project-filter','result-project','result-rows','case-detail'].forEach(id=>$(id).replaceChildren());
   if(typeof clearDetails==='function')clearDetails();
+  if(typeof clearOperations==='function')clearOperations();
   ['active-count','token-count','correction-count','finished-count'].forEach(id=>$(id).textContent='—');
   $('pie').getContext('2d').clearRect(0,0,360,360);
 }
@@ -106,6 +107,7 @@ async function refresh() {
   clearTimeout(timer); if(!token)return;const generation=session;
   try {const response=await api('/v1/overview');if(generation!==session)return;overview=response;render();$('updated').textContent=`Aktualisiert ${new Date().toLocaleTimeString('de-DE')}`;
     error(overview.read_errors?'Einige Akten konnten nicht gelesen werden. Die Verbrauchsdaten können unvollständig sein.':'');
+    if(typeof refreshOperations==='function')await refreshOperations();
   } catch(e) {if(generation===session)error(`Aktualisierung fehlgeschlagen: ${e.message}`);}
   if(token && generation===session)timer=setTimeout(refresh,4000);
 }

src/viano_bug_analysis/web/index.html

@@ -3,7 +3,7 @@
 <head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
 <title>BugAnalysis · Übersicht</title><link rel="stylesheet" href="/assets/dashboard.css">
 <script src="/assets/dashboard.js" defer></script>
-<script src="/assets/case-detail.js" defer></script></head>
+<script src="/assets/case-detail.js" defer></script><script src="/assets/operations.js" defer></script></head>
 <body>
 <aside class="rail"><a class="brand" href="/">V<span>BA</span><small>BugAnalysis</small></a>
 <nav aria-label="Navigation"><a href="#overview" class="selected">Übersicht</a><a href="#cases">Laufende Fälle</a><a href="#results">Ergebnisse</a><a href="#submit">Auftrag übergeben</a></nav>
@@ -15,7 +15,7 @@
 <p id="error" class="error" role="alert" hidden></p>
 <div id="content" hidden>
 <div id="overview-page">
-<section class="stats" aria-label="Kennzahlen">
+<section class="panel" id="operations"><h2>Betriebssteuerung</h2><div id="operation-status"></div><div id="operation-projects"></div><details id="operation-audit" hidden><summary>Aktionsprotokoll</summary><div id="operation-events"></div></details></section><section class="stats" aria-label="Kennzahlen">
 <article><span>Aktive Fälle</span><strong id="active-count">—</strong><small>Noch nicht abgeschlossen</small></article>
 <article><span>Erfasste Tokens</span><strong id="token-count">—</strong><small>Eingabe + Ausgabe inkl. Denken</small></article>
 <article><span>Korrekturanteil</span><strong id="correction-count">—</strong><small>Zusätzliche Korrekturen und Wiederholungen</small></article>

src/viano_bug_analysis/web/operations.js

@@ -0,0 +1,34 @@
+'use strict';
+let operationState=null, operationSending=false;
+function clearOperations(){operationState=null;['operation-status','operation-projects','operation-events'].forEach(id=>$(id).replaceChildren());$('operation-audit').hidden=true;}
+async function changeOperation(data,button){
+  if(operationSending)return;const generation=session;operationSending=true;button.disabled=true;
+  try{const result=await api('/v1/operations',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(data)});
+    if(generation!==session)return;
+    error(result.status==='QUEUED'?'Fortsetzung vorgemerkt. Der Worker prüft die Freigabe vor der Ausführung.':'');
+    await refreshOperations();
+  }catch(e){if(generation===session)error(`Betriebsaktion fehlgeschlagen: ${e.message}`);}
+  finally{operationSending=false;if(generation===session)button.disabled=false;}
+}
+function addRetryControl(data,root){
+  if(!operationState?.operator||data.revision!==data.current_revision)return;
+  const retry=data.state==='BLOCKED'?data.reason==='RESEARCH_UNAVAILABLE':!finished.has(data.state)&&['ERROR','ATTENTION','BUSY','DEFERRED','RETRY_PENDING'].includes(data.worker.status);
+  if(!retry)return;
+  const button=text('button','Fortsetzung nach Fehlerbehebung anfordern');button.type='button';
+  button.addEventListener('click',()=>changeOperation({action:'retry',key:data.key,package_generation:data.package_generation},button));root.append(button);
+}
+async function refreshOperations(){
+  const generation=session;const result=await api('/v1/operations');if(generation!==session)return;
+  const changed=operationState?.operator!==result.operator;operationState=result;
+  const status=$('operation-status');status.replaceChildren();
+  if(result.worker){const w=result.worker;status.append(text('p',`${w.live?'Worker erreichbar':'Kein lebender Worker bestätigt'} · Letzter Zustand: ${w.status||'Unbekannt'}${w.code?' · '+w.code:''}`));
+    if(w.updated_at)status.append(text('p',`Letzte Aktivität: ${new Date(w.updated_at).toLocaleString('de-DE')}`,'hint'));}
+  else status.append(text('p',result.operator?'Workerdetails erfordern Zugriff auf alle Projekte.':'Lesender Betriebszugriff. Betreiberaktionen sind nicht freigegeben.','hint'));
+  if(result.pending)status.append(text('p',`${result.pending} Fortsetzungsaufträge warten auf den Worker.`));
+  const projectsRoot=$('operation-projects');projectsRoot.replaceChildren();
+  result.projects.forEach(p=>{const row=text('div','','section-head');row.append(text('p',`${p.project_id} · ${p.paused?'Neue Fälle pausiert':'Neue Fälle freigegeben'}`));
+    if(result.operator){const button=text('button',p.paused?'Neue Fälle wieder zulassen':'Neue Fälle pausieren');button.type='button';button.addEventListener('click',()=>changeOperation({action:p.paused?'resume':'pause',project_id:p.project_id},button));row.append(button);}projectsRoot.append(row);});
+  $('operation-audit').hidden=!result.operator;const log=$('operation-events');log.replaceChildren();
+  result.events.slice().reverse().forEach(e=>log.append(text('p',`${new Date(e.at).toLocaleString('de-DE')} · ${e.actor} · ${e.project_id}${e.case_id?' · '+e.case_id:''} · ${e.action} · ${e.status}${e.code?' · '+e.code:''}`)));
+  if(changed&&selectedCase&&location.hash.startsWith('#case/'))renderDetail(selectedCase);
+}

src/viano_bug_analysis/worker/config.py

@@ -9,7 +9,7 @@ from viano_bug_analysis.telemetry.safety import safe_code
 
 WORKER_CODES = frozenset({'WORKER_CONFIG', 'WORKER_LLM_CONFIG_REQUIRED', 'WORKER_PROJECT_BINDING',
     'WORKER_CONFIG_CHANGED', 'WORKER_STORAGE_IDENTITY', 'WORKER_CASE_ERROR', 'WORKER_PROJECT_UNKNOWN',
-    'WORKER_STORAGE_LOW', 'WORKER_CYCLE_ERROR', 'WORKER_BUSY', 'WORKER_RETRY_STATE', 'WORKER_NOT_FOUND'})
+    'WORKER_STORAGE_LOW', 'WORKER_CYCLE_ERROR', 'WORKER_BUSY', 'WORKER_RETRY_STATE', 'WORKER_RETRY_STALE', 'WORKER_NOT_FOUND'})
 
 
 def code(value):

src/viano_bug_analysis/worker/control.py

@@ -0,0 +1,57 @@
+"""Durable operator intent and audit log, shared by HTTP and worker processes."""
+import json
+import uuid
+from contextlib import contextmanager
+
+from viano_bug_analysis.api.config import directory
+from viano_bug_analysis.lifecycle.locking import FileLock
+from viano_bug_analysis.lifecycle.job import timestamp
+from viano_bug_analysis.lifecycle.storage import atomic_json
+
+
+class Control:
+    def __init__(self, workspace):
+        self.root = directory(workspace.runtime / 'worker/control')
+        self.lock = workspace.runtime / 'locks/worker-control.lock'
+
+    def read(self):
+        path = self.root / 'state.json'
+        return json.loads(path.read_bytes()) if path.exists() else {'paused':{}, 'commands':[], 'events':[]}
+
+    @contextmanager
+    def transaction(self):
+        with FileLock(self.lock):
+            value = self.read()
+            yield value
+            atomic_json(self.root / 'state.json', value)
+
+    @staticmethod
+    def event(value, actor, action, project, **fields):
+        value['events'].append({'id':uuid.uuid4().hex, 'at':timestamp(),
+            'actor':actor, 'action':action, 'project_id':project, **fields})
+
+    def pause(self, actor, project, paused):
+        with self.transaction() as value:
+            value['paused'][project] = paused
+            self.event(value, actor, 'pause' if paused else 'resume', project, status='APPLIED')
+
+    def enqueue(self, actor, project, manifest):
+        with self.transaction() as value:
+            if len(value['commands']) >= 100:
+                from viano_bug_analysis.api.config import APIError
+                raise APIError('API_CONTROL_BUSY', 409)
+            identity = {k:manifest[k] for k in ('case_id','package_id','package_generation')}
+            for command in value['commands']:
+                if all(command[k] == v for k,v in identity.items()):
+                    return command['id']
+            command = {'id':uuid.uuid4().hex,'actor':actor,'project_id':project,**identity}
+            value['commands'].append(command)
+            self.event(value, actor, 'retry', project, command_id=command['id'], status='QUEUED', **identity)
+            return command['id']
+
+    def complete(self, command, code=None):
+        with self.transaction() as value:
+            value['commands'] = [c for c in value['commands'] if c['id'] != command['id']]
+            self.event(value, command['actor'], 'retry', command['project_id'],
+                command_id=command['id'], case_id=command['case_id'], package_id=command['package_id'],
+                status='REJECTED' if code else 'APPLIED', code=code)

src/viano_bug_analysis/worker/service.py

@@ -15,6 +15,7 @@ from viano_bug_analysis.lifecycle.storage import atomic_json, key
 from viano_bug_analysis.package import PackageReader
 from viano_bug_analysis.workflow import drive
 from .config import WorkerError, binding, load_project, code as safe_code
+from .control import Control
 
 class AnalysisWorker:
     """One analysis step per eligible case/pass; case files remain authoritative."""
@@ -33,6 +34,7 @@ class AnalysisWorker:
         self.stop_requested = False
         self.last_case = None
         self.last_inbox = None
+        self.control = Control(self.workspace)
 
     def request_stop(self):
         self.stop_requested = True
@@ -113,14 +115,20 @@ class AnalysisWorker:
             self.runtimes[project.id] = runtime
         return runtime
 
-    def retry(self, case_id, package_id):
+    def retry(self, case_id, package_id, *, expected_generation=None, locked=False):
         """Release only a scheduler error; never bypass case/operation budgets."""
-        with FileLock(self.workspace.runtime / 'locks/analysis-service.lock'):
+        if not locked:
+            with FileLock(self.workspace.runtime / 'locks/analysis-service.lock'):
+                with FileLock(self.workspace.runtime / 'locks' / ('trial-' + key(case_id) + '.lock')):
+                    return self.retry(case_id, package_id, expected_generation=expected_generation, locked=True)
+        if locked:
             latest = self.store.latest(case_id, package_id)
             if not latest:
                 raise WorkerError('WORKER_NOT_FOUND')
             with PackageReader(latest, self.store.limits) as reader:
                 manifest = reader.manifest
+            if expected_generation is not None and manifest['package_generation'] != expected_generation:
+                raise WorkerError('WORKER_RETRY_STALE')
             record = self._record(manifest)
             project = self.projects.get(manifest['project'].get('id'))
             if not project:
@@ -141,7 +149,28 @@ class AnalysisWorker:
                     expected_generation=manifest['package_generation'])
                 with PackageReader(latest, self.store.limits) as reader:
                     manifest = reader.manifest
+            elif manifest['state'] in {State.OUTPUT_READY, State.HANDED_OFF, State.FAILED_TO_ANALYZE}:
+                raise WorkerError('WORKER_RETRY_STATE')
+            elif record.get('status') not in {'ERROR', 'ATTENTION', 'BUSY', 'DEFERRED', 'RETRY_PENDING'}:
+                raise WorkerError('WORKER_RETRY_STATE')
             self._save(manifest, 'RELEASED')
+            if expected_generation is None:
+                with self.control.transaction() as value:
+                    self.control.event(value,'local-cli','retry',project.id,status='APPLIED',
+                        case_id=case_id,package_id=package_id)
+
+    def _commands(self):
+        for command in self.control.read()['commands']:
+            try:
+                with FileLock(self.workspace.runtime / 'locks' / ('trial-' + key(command['case_id']) + '.lock')):
+                    self.retry(command['case_id'], command['package_id'],
+                        expected_generation=command['package_generation'], locked=True)
+            except LockBusy:
+                continue
+            except Exception as error:
+                self.control.complete(command, safe_code(getattr(error, 'code', 'WORKER_CASE_ERROR')))
+            else:
+                self.control.complete(command)
 
     def _admit(self):
         sources = sorted(self.workspace.inbox.glob('*.zip'))
@@ -156,9 +185,12 @@ class AnalysisWorker:
                 try:
                     with PackageReader(source, self.store.limits) as reader:
                         case_id = reader.manifest['case_id']
+                        project_id = reader.manifest['project'].get('id')
                 except Exception:
                     self.queue.process(source)  # normal validator/quarantine behavior
                     continue
+                if self.control.read()['paused'].get(project_id, False):
+                    continue
                 with FileLock(self.workspace.runtime / 'locks' / ('trial-' + key(case_id) + '.lock')):
                     result = self.queue.process(source)
                     if result.status in {'ERROR', 'BUSY'}:
@@ -168,6 +200,7 @@ class AnalysisWorker:
 
     async def tick(self):
         self._status('RUNNING')
+        self._commands()
         if shutil.disk_usage(self.workspace.root).free < self.config.min_free_bytes:
             self._status('STORAGE_LOW', code='WORKER_STORAGE_LOW')
             return 0
@@ -198,6 +231,9 @@ class AnalysisWorker:
                     if same_generation and not self._due(record.get('retry_not_before')):
                         continue
                     project = self.projects.get(manifest['project'].get('id'))
+                    if (manifest['state'] in {State.NEW, State.VALIDATED, State.REOPENED}
+                            and self.control.read()['paused'].get(manifest['project'].get('id'), False)):
+                        continue
                     if not project or (project.type, project.profile) != (
                             manifest['project'].get('type'), manifest['project'].get('profile')):
                         self._pause(manifest, 'WORKER_PROJECT_UNKNOWN', status='ATTENTION')

tests/dashboard-smoke.mjs

@@ -3,7 +3,9 @@ import {spawn} from 'node:child_process';
 import {mkdtemp, mkdir, writeFile, rm} from 'node:fs/promises';
 import {tmpdir} from 'node:os';
 import path from 'node:path';
-const port=18765, debugPort=18766;
+import {createServer} from 'node:net';
+async function freePort(){const server=createServer();await new Promise(resolve=>server.listen(0,'127.0.0.1',resolve));const port=server.address().port;await new Promise(resolve=>server.close(resolve));return port;}
+const port=await freePort(), debugPort=await freePort();
 const pause=ms=>new Promise(r=>setTimeout(r,ms));
 const output=path.resolve('service-data/ui-preview');await mkdir(output,{recursive:true});
 const profile=await mkdtemp(path.join(tmpdir(),'vba-browser-'));
@@ -20,7 +22,7 @@ try {
   ws=new WebSocket(tabs.find(t=>t.type==='page').webSocketDebuggerUrl);
   await new Promise((resolve,reject)=>{ws.onopen=resolve;ws.onerror=reject;});
   let next=0;const pending=new Map();
-  ws.onmessage=e=>{const m=JSON.parse(e.data);if(m.id){const p=pending.get(m.id);pending.delete(m.id);m.error?p.reject(new Error(m.error.message)):p.resolve(m.result);}
+  ws.onmessage=e=>{const m=JSON.parse(e.data);if(m.id){const p=pending.get(m.id);pending.delete(m.id);if(!p)return;m.error?p.reject(new Error(m.error.message)):p.resolve(m.result);}
     if(m.method==='Runtime.exceptionThrown')errors.push(m.params.exceptionDetails.text);
     if(m.method==='Log.entryAdded'&&m.params.entry.level==='error')errors.push(m.params.entry.text);};
   const send=(method,params={})=>new Promise((resolve,reject)=>{const id=++next;pending.set(id,{resolve,reject});ws.send(JSON.stringify({id,method,params}));});
@@ -31,6 +33,12 @@ try {
   await until(()=>evaluate("document.readyState === 'complete' && typeof render === 'function'"));
   await evaluate("document.getElementById('token').value='ui_fixture_"+'x'.repeat(40)+"';document.getElementById('login-form').dispatchEvent(new Event('submit',{bubbles:true,cancelable:true}));");
   await until(()=>evaluate("document.getElementById('token-count').textContent==='35.300'"));
+  await until(()=>evaluate("document.querySelector('#operation-projects button')"));
+  await evaluate("document.querySelector('#operation-projects button').click()");
+  await until(()=>evaluate("document.getElementById('operation-projects').textContent.includes('Neue Fälle pausiert')"));
+  if(!await evaluate("document.getElementById('operation-events').textContent.includes('preview')"))throw new Error('Operator audit missing');
+  await evaluate("document.querySelector('#operation-projects button').click()");
+  await until(()=>evaluate("document.getElementById('operation-projects').textContent.includes('Neue Fälle freigegeben')"));
   const state=await evaluate("({workers:document.querySelectorAll('.worker').length,cases:document.querySelectorAll('#case-rows tr').length,active:document.getElementById('active-count').textContent,correction:document.getElementById('correction-count').textContent,overflow:document.documentElement.scrollWidth>innerWidth,stored:localStorage.length,progress:document.querySelector('.worker [role=progressbar]').getAttribute('aria-valuenow')})");
   if(state.workers!==1||state.cases!==1||state.active!=='1'||state.stored!==0||state.overflow||state.progress!=='57')throw new Error(JSON.stringify(state));
   const categories=await evaluate("Array.from(document.querySelectorAll('#legend li')).map(e=>e.textContent)");
@@ -58,13 +66,16 @@ try {
   await evaluate("document.getElementById('logout').click()");
   if(!await evaluate("document.getElementById('content').hidden&&!document.getElementById('login').hidden&&document.getElementById('token-count').textContent==='—'"))throw new Error('Logout retained visible case data');
   if(!await evaluate("document.getElementById('case-detail').childElementCount===0&&document.getElementById('result-rows').childElementCount===0"))throw new Error('Logout retained detail data');
+  if(!await evaluate("document.getElementById('operation-events').childElementCount===0"))throw new Error('Logout retained audit');
   if(errors.filter(e=>!e.includes('404')).length)throw new Error(errors.join('\n'));
   console.log(JSON.stringify({status:'PASS',...state,console_errors:errors.filter(e=>!e.includes('404'))}));
 } finally {
-  ws?.close();browser?.kill();server?.kill();await pause(400);
+  // Let Chromium close its database handles before removing its own profile.
+  if(ws?.readyState===WebSocket.OPEN)ws.send(JSON.stringify({id:999999,method:'Browser.close'}));
+  await pause(1500);ws?.close();browser?.kill();server?.kill();await pause(400);
   for(const [folder,prefix] of [[profile,'vba-browser-'],[fixtureRoot,'vba-ui-data-']]) {
     const target=path.resolve(folder),relative=path.relative(path.resolve(tmpdir()),target);
     if(!relative||relative.startsWith('..')||path.isAbsolute(relative)||!path.basename(target).startsWith(prefix))throw new Error('Unsafe temporary cleanup');
-    await rm(target,{recursive:true,force:true,maxRetries:5,retryDelay:200});
+    await rm(target,{recursive:true,force:true,maxRetries:15,retryDelay:300});
   }
 }

tests/test_dashboard.py

@@ -198,3 +198,41 @@ def test_historical_artifacts_are_not_replaced_by_latest_revision():
         ('oldcp',{'analysis_revision':1,'details':{'operation':'diagnosis-complete','diagnosis_ref':'old'}})]
     assert references(reader,chain,1)['diagnosis']=='old'
     assert references(reader,chain,2)['diagnosis']=='new'
+
+
+def test_operator_permissions_scope_and_audit(case_setup,monkeypatch):
+    store,builder,*_=case_setup
+    builder.build('case-A','stable-package')
+    monkeypatch.setenv('CAPTURE_TOKEN','capture_'+'a'*40)
+    monkeypatch.setenv('OPERATOR_TOKEN','operator_'+'b'*40)
+    config=APIConfig(store.workspace.root,(Project('p','wow','vcore'),Project('other','wow','vcore')),
+        (Client('capture','CAPTURE_TOKEN',('p',)),Client('operator','OPERATOR_TOKEN',('p',),operator=True)))
+    app=create_app(config);client=TestClient(app)
+    capture={'Authorization':'Bearer capture_'+'a'*40}
+    operator={'Authorization':'Bearer operator_'+'b'*40}
+    assert client.get('/v1/operations').status_code==401
+    assert client.post('/v1/operations',headers=capture,json={'action':'pause','project_id':'p'}).status_code==403
+    assert client.post('/v1/operations',headers=operator,json={'action':'pause','project_id':'other'}).status_code==403
+    assert client.post('/v1/operations',headers=operator,json={'action':'pause','project_id':'p'}).status_code==200
+    view=client.get('/v1/operations',headers=capture).json()
+    assert view['projects']==[{'project_id':'p','paused':True}] and view['events']==[]
+    view=client.get('/v1/operations',headers=operator).json()
+    assert view['events'][-1]['actor']=='operator' and view['worker'] is None
+    assert client.post('/v1/operations',headers=operator,json={'action':'resume','project_id':'p','reset_budget':True}).status_code==422
+    assert client.post('/v1/operations',headers=operator,content='x'*4097).status_code==415
+    assert client.post('/v1/operations',headers=operator|{'Content-Type':'application/json'},content='x'*4097).status_code==413
+    from viano_bug_analysis.api.cases import case_key
+    from viano_bug_analysis.package import PackageReader
+    with PackageReader(store.latest('case-A','stable-package')) as reader:
+        generation=reader.manifest['package_generation']
+    command={'action':'retry','key':case_key('case-A','stable-package'),'package_generation':generation}
+    first=client.post('/v1/operations',headers=operator,json=command)
+    assert first.status_code==200 and first.json()['status']=='QUEUED'
+    assert client.post('/v1/operations',headers=operator,json=command).json()==first.json()
+    assert client.get('/v1/operations',headers=operator).json()['pending']==1
+    assert client.post('/v1/operations',headers=operator,json=command|{'package_generation':0}).status_code==409
+    store.advance('case-A','stable-package','BLOCKED',step_id='control:invalid',reason='INVALID_LLM_RESPONSES',
+        retry_not_before='2020-01-01T00:00:00Z')
+    with PackageReader(store.latest('case-A','stable-package')) as reader:
+        generation=reader.manifest['package_generation']
+    assert client.post('/v1/operations',headers=operator,json=command|{'package_generation':generation}).status_code==409

tests/test_worker.py

@@ -281,3 +281,41 @@ def test_exception_after_checkpoint_holds_actual_latest_generation(setup):
     with PackageReader(store.latest(*CASE)) as reader:
         assert record['package_generation'] == reader.manifest['package_generation']
     assert record['status'] == 'ATTENTION'
+
+
+def test_operator_pause_survives_restart_and_drains_started_cases(setup):
+    config,loader,store=setup
+    worker=AnalysisWorker(config,loader=loader)
+    worker.control.pause('operator','test',True)
+    assert asyncio.run(worker.tick())==0
+    worker=AnalysisWorker(config,loader=loader)
+    assert asyncio.run(worker.tick())==0
+    worker.control.pause('operator','test',False)
+    assert asyncio.run(worker.tick())==1
+    worker.control.pause('operator','test',True)
+    assert asyncio.run(worker.tick())==1 # already started case continues
+
+
+def test_operator_commands_recheck_generation_and_forbid_validation_reset(setup):
+    config,loader,store=setup
+    worker=AnalysisWorker(config,loader=loader)
+    store.advance(*CASE,State.BUILDING_CASE,step_id='operator:start')
+    store.advance(*CASE,State.BLOCKED,step_id='operator:block',reason='RESEARCH_UNAVAILABLE',
+        retry_not_before='2020-01-01T00:00:00Z',details={'automatic_retry':False})
+    with PackageReader(store.latest(*CASE)) as reader:
+        manifest=reader.manifest
+    worker.control.enqueue('operator','test',manifest)
+    worker._commands()
+    assert not worker.control.read()['commands']
+    assert worker.control.read()['events'][-1]['status']=='APPLIED'
+    worker.control.enqueue('operator','test',manifest) # stale UI cannot release new head
+    worker._commands()
+    assert worker.control.read()['events'][-1]['status']=='REJECTED'
+    store.advance(*CASE,State.BLOCKED,step_id='operator:invalid',reason='INVALID_LLM_RESPONSES',
+        retry_not_before='2020-01-01T00:00:00Z')
+    path=store.latest(*CASE)
+    with PackageReader(path) as reader:
+        worker.control.enqueue('operator','test',reader.manifest)
+    worker._commands()
+    assert store.latest(*CASE)==path
+    assert worker.control.read()['events'][-1]['status']=='REJECTED'

tests/ui_dashboard_server.py

@@ -13,6 +13,7 @@ from viano_bug_analysis.lifecycle.storage import atomic_json, json_bytes
 from viano_bug_analysis.package.factory import PackageFactory
 from viano_bug_analysis.case import CaseBuilder
 import viano_bug_analysis.api.dashboard as dashboard
+import viano_bug_analysis.api.operations as operations
 
 TOKEN = 'ui_fixture_' + 'x' * 40
 
@@ -90,10 +91,11 @@ if __name__ == '__main__':
         identity=seed(root,'BUG-000020','Wolpertinger fehlen in Dun Morogh')
         seed(root,'BUG-000017','Leuchtbrille – Kontrolllauf',completed=True)
         config=APIConfig(root,(Project('viano-core','wow','vcore'),),
-            (Client('preview','VBA_UI_TEST_TOKEN',('viano-core',)),))
+            (Client('preview','VBA_UI_TEST_TOKEN',('viano-core',),operator=True),))
         os.environ['VBA_UI_TEST_TOKEN']=TOKEN
         dashboard.read_status=lambda _: {'observer':'LIVE','service_id':'fixture-worker'}
+        operations.read_status=dashboard.read_status
         folder=root/'runtime/worker';folder.mkdir(exist_ok=True)
-        atomic_json(folder/'status.json',{'status':'ANALYZING','service_id':'fixture-worker',
+        atomic_json(folder/'status.json',{'status':'ANALYZING','service_id':'fixture-worker','updated_at':timestamp(),
             'case':{'case_id':identity[0],'package_id':identity[1]}})
         uvicorn.run(create_app(config),host='127.0.0.1',port=int(os.environ.get('VBA_UI_TEST_PORT','8765')),access_log=False)