Add human test request inbox and authenticated feedback resume
This commit is contained in:
1 parent
ac179883c6
commit
9fc5c1805c
18 files changed
+377
-13
No files matched your search
@@ -14,6 +14,7 @@ planning_mode = "joint"
|
||||
# project_config = "project.local.toml"
|
||||
# llm_config = "llm.early-review.local.toml"
|
||||
# investigation_config = "investigation.bounded.toml"
|
||||
# budget_config = "budget.human-tests.toml"
|
||||
|
||||
[[clients]]
|
||||
id = "wow-capture-worker"
|
||||
@@ -26,3 +27,4 @@ projects = ["viano-core"]
|
||||
# token_env = "VBA_OPERATOR_TOKEN"
|
||||
# projects = ["viano-core"]
|
||||
# operator = true
|
||||
# tester = true
|
||||
@@ -0,0 +1,5 @@
|
||||
# Human requests are available when an authenticated web feedback adapter exists.
|
||||
# Frozen per new analysis revision; existing revisions retain their policy.
|
||||
[budget]
|
||||
[budget.action_costs]
|
||||
"testing.request_human" = 40
|
||||
@@ -1,8 +1,9 @@
|
||||
# Asynchronous test handoff v1
|
||||
|
||||
This addition prepares the existing project for externally performed human and
|
||||
automated tests. No web UI, HTTP feedback endpoint, notifications, bot steering,
|
||||
client automation, Tool Router or Investigation Loop is implemented here.
|
||||
This contract supports externally performed human and automated tests. The human
|
||||
HTTP adapter and dedicated web page are now documented in
|
||||
[Web console](web-console.md#menschliche-testanfragen). Automated execution,
|
||||
notifications, bot steering and client automation are not provided by that page.
|
||||
`docs/architecture.md` remains unchanged and authoritative: VianoBugAnalysis
|
||||
does not steer labs. It requests observations through a future external adapter.
|
||||
|
||||
@@ -10,8 +11,8 @@ does not steer labs. It requests observations through a future external adapter.
|
||||
|
||||
`AsyncTestCoordinator` is imported from `viano_bug_analysis.lifecycle.async_tests`.
|
||||
It is a trusted controller service, not a callable operation exposed directly to
|
||||
an LLM. The future router must validate requests before calling it. The future
|
||||
web adapter must authenticate/authorize users and confirm feedback before calling
|
||||
an LLM. The investigation controller validates requests before calling it. The
|
||||
web adapter authenticates/authorizes users and confirms feedback before calling
|
||||
`confirm_feedback`; a payload claiming `confirmed = true` is not authentication.
|
||||
User-supplied observations are untrusted data, not executable instructions.
|
||||
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
# Menschliche Rückfragen im Web – 2026-10-22
|
||||
|
||||
Ziel: Die vorhandene menschliche Testanforderung bedienbar und für neue
|
||||
Dienstanalysen freigegeben machen. Eigene Seite mit offenen Anfragen, Projekt,
|
||||
Bug, Zeit, Projektfilter und Sortierungen; Anzeige des Testauftrags und bestätigtes
|
||||
Textfeedback. Separate Testerrechte bzw. Betreiberrechte erforderlich.
|
||||
|
||||
Neue Budgetdatei `config/budget.human-tests.toml`: menschliche Tests 40 fachliche
|
||||
Einheiten, keine Freigabe automatisierter Tests. Die ignorierte lokale
|
||||
API-Konfiguration verweist darauf. Vorhandene eingefrorene Revisionen unverändert.
|
||||
Antworten benutzen den bestehenden AsyncTestCoordinator und dessen Budget-/Identitäts-
|
||||
und Resume-Prüfungen, keine eigene Zustandsabkürzung. Serverseitige Bestätigung
|
||||
unter der authentifizierten Clientkennung. Keine kostenpflichtigen LLM-Aufrufe.
|
||||
|
||||
Prüfung mit synthetischen Daten:
|
||||
|
||||
- API-/Coordinator-/Dashboard-/Worker-Tests: beide Antwortausgänge,
|
||||
Projektberechtigung, fehlende Rechte, falsche Anfrage, fehlende Bestätigung,
|
||||
identische bzw. widersprüchliche Wiederholung, unveränderte Budgetkonten und
|
||||
Fortsetzung in derselben Revision.
|
||||
Zusammen 51 Tests bestanden.
|
||||
- Schemavertrag: ohne Budgetfreigabe kein Testzweig, bei menschlicher Freigabe
|
||||
ausschließlich `human` als Testexecutor.
|
||||
- Echter Headless-Edge: eigene Seite, Fall-/Projektanzeige, Sortierauswahl,
|
||||
Auftragsanzeige und Antwortformular, Mobilansicht ohne horizontalen Überlauf,
|
||||
Datenentfernung bei Abmeldung. Feedbackwirkung separat mit echter Coordinator-
|
||||
und Budget-Fixture getestet, UI-Anfrage ist ausdrücklich eine Anzeige-Fixture.
|
||||
- Vor Tests Test-Workspaces 16.939.129.428 Bytes; kleine Fixtures und temporäre
|
||||
pytest-/Browserdaten bleiben mit Reserve innerhalb des 20-GB-Budgets.
|
||||
- Breitere Untersuchungstests: 72 bestanden, 26 alte Tests in
|
||||
`test_investigator.py` scheitern am vorhandenen `INVESTIGATION_PROMPT_LIMIT`.
|
||||
Vollständige Gegenprobe dieses Moduls mit deaktivierter neuer Schemafilterung:
|
||||
dieselben 26 Fehler, 6 bestanden. Keine Erweiterung der Zeichengrenze vorgenommen.
|
||||
|
||||
Grenzen: Textfeedback; Dateianhänge und automatisierte Testausführung sind keine
|
||||
Bestandteile dieser Oberfläche. Alte Revisionen benötigen ihre ursprüngliche Policy.
|
||||
@@ -126,6 +126,54 @@ Protokoll/Workerzustand. `POST /v1/operations` benötigt Betreiberrechte und JSO
|
||||
Bei gleichzeitigem Zugriff kann `API_CONTROL_BUSY` auftreten; nach Aktualisierung
|
||||
erneut versuchen. Der Worker muss nach diesem Softwareupdate neu gestartet werden.
|
||||
|
||||
## Menschliche Testanfragen
|
||||
|
||||
Die eigene Seite **Testanfragen** listet offene menschliche Rückfragen mit
|
||||
Eingangszeit, Projekt, Bug-ID, Falltitel und Frage. Projektfilter sowie Sortierung
|
||||
„Älteste zuerst“, „Neueste zuerst“ und „Projekt, dann älteste zuerst“ ermöglichen
|
||||
chronologisches oder projektweises Abarbeiten. Die Liste aktualisiert sich mit der
|
||||
Übersicht; eine noch offene Eingabe wird dabei nicht überschrieben.
|
||||
|
||||
„Ansehen / beantworten“ zeigt Umgebung, Voraussetzungen, Testschritte, benötigte
|
||||
Beobachtungen und angefragten Termin. Die Antwort erfasst die tatsächliche Umgebung,
|
||||
Beobachtungen und „Test durchgeführt“ oder „Nicht durchführbar“. Die ausdrückliche
|
||||
Bestätigung ist erforderlich. Textdaten bleiben untrusted observations, keine
|
||||
auszuführenden Befehle. Diese erste Oberfläche nimmt Text, keine Dateianhänge an.
|
||||
|
||||
Clients brauchen `tester = true` oder Betreiberrechte und Projektzugriff, um eine
|
||||
Antwort zu bestätigen. Ein reiner Uploadclient erhält diese Rechte nicht automatisch.
|
||||
Beispiel eines separaten menschlichen Clients in der API-Konfiguration:
|
||||
|
||||
```toml
|
||||
[[clients]]
|
||||
id = "human-dev"
|
||||
token_env = "VBA_HUMAN_TOKEN"
|
||||
projects = ["viano-core"]
|
||||
tester = true
|
||||
```
|
||||
|
||||
Eigenen API-Token über diese Umgebungsvariable bereitstellen und API neu starten.
|
||||
Die bestätigende Identität und Zeit stammen vom Server, nicht aus frei editierbaren
|
||||
Formularfeldern. `GET /v1/test-requests` liefert ausschließlich freigegebene Projekte;
|
||||
`POST /v1/test-requests/{case-key}` bestätigt die konkrete Anfrage/Revision.
|
||||
Kollisionen, veraltete Anfragen und widersprüchliche Wiederholungen werden abgewiesen.
|
||||
Eine identische erneute Übermittlung desselben Clients bleibt ohne doppelte Wirkung.
|
||||
|
||||
Eine bestätigte Antwort setzt `AWAITING_TEST_FEEDBACK` über den vorhandenen
|
||||
Coordinator auf `INVESTIGATING_HYPOTHESES` zurück. Revision, Budget und ursprüngliche
|
||||
Belege bleiben erhalten. Der laufende Worker verarbeitet die Rückmeldung regulär;
|
||||
eine besondere Betreiberfreigabe ist dafür nicht erforderlich. Der Termin führt
|
||||
auch nach Ablauf zu keiner automatischen Fortsetzung und verwirft die Anfrage nicht.
|
||||
|
||||
Die Werkzeugfreigabe erfolgt über `budget_config = "budget.human-tests.toml"` in
|
||||
den vertrauenswürdigen Projekteinstellungen. Die lokale Dienstkonfiguration wurde
|
||||
entsprechend ergänzt. Diese Policy erlaubt `testing.request_human` für 40 fachliche
|
||||
Einheiten, automatisierte Tests bleiben ohne Eintrag gesperrt. Bei Trial-Läufen
|
||||
die Datei ausdrücklich mit `--budget-config config/budget.human-tests.toml` angeben.
|
||||
Bestehende Revisionen behalten ihre eingefrorene Policy; sie werden nicht migriert.
|
||||
Nach Konfigurationsänderung API und Worker neu starten. Keine Modellanfrage wird
|
||||
allein durch Öffnen oder Beantworten der Seite gestartet; der Worker läuft separat.
|
||||
|
||||
`GET /v1/cases/{key}?revision=N` liefert diese projektgebundene Ansicht.
|
||||
`GET /v1/cases/{key}/package` lädt die vollständige aktuelle Akte einschließlich
|
||||
aller erhaltenen Revisionen herunter, auch als Zwischenstand eines laufenden Falls.
|
||||
|
||||
@@ -7,6 +7,29 @@ PROMPT_VERSION="investigation_v1"
|
||||
|
||||
|
||||
def output_schema(context=None, *, strict=False):
|
||||
schema=_output_schema(context,strict=strict)
|
||||
if context and 'budget_policy' in context:
|
||||
costs=context['budget_policy']['action_costs']
|
||||
allowed=[e for e in ('human','automated') if 'testing.request_'+e in costs]
|
||||
def restrict(value):
|
||||
if isinstance(value,list):
|
||||
return [restrict(v) for v in value if not (isinstance(v,dict)
|
||||
and v.get('properties',{}).get('kind',{}).get('const')=='test' and not allowed)]
|
||||
if not isinstance(value,dict):return value
|
||||
value={k:restrict(v) for k,v in value.items()}
|
||||
properties=value.get('properties',{})
|
||||
if properties.get('kind',{}).get('const')=='test':
|
||||
properties['executor']['enum']=allowed
|
||||
if not allowed:
|
||||
for field in ('kind','intent'):
|
||||
if 'enum' in properties.get(field,{}):
|
||||
properties[field]['enum']=[v for v in properties[field]['enum'] if v!='test']
|
||||
return value
|
||||
schema=restrict(schema)
|
||||
return schema
|
||||
|
||||
|
||||
def _output_schema(context=None, *, strict=False):
|
||||
if context and context.get("startup_explore"):
|
||||
from .investigation_startup import explore_schema
|
||||
return explore_schema(context)
|
||||
|
||||
@@ -21,6 +21,7 @@ from .service import IntakeService
|
||||
from .dashboard import Dashboard
|
||||
from .cases import Cases
|
||||
from .operations import Operations
|
||||
from .test_requests import TestRequests
|
||||
from viano_bug_analysis.lifecycle.errors import LockBusy
|
||||
from viano_bug_analysis.lifecycle.errors import LifecycleError
|
||||
from viano_bug_analysis.package import PackageReadError
|
||||
@@ -32,6 +33,7 @@ def create_app(config):
|
||||
dashboard = Dashboard(service)
|
||||
cases = Cases(service)
|
||||
operations = Operations(service)
|
||||
test_requests = TestRequests(service)
|
||||
tokens = config.tokens()
|
||||
service.recover()
|
||||
active = 0
|
||||
@@ -119,14 +121,40 @@ def create_app(config):
|
||||
raise APIError('API_CONTROL_UNAVAILABLE',503) from None
|
||||
return JSONResponse(result,headers={'Cache-Control':'no-store'})
|
||||
|
||||
async def tests(request):
|
||||
client=authenticate(request)
|
||||
try:
|
||||
if request.method=='GET':
|
||||
result=await run_in_threadpool(test_requests.listing,client)
|
||||
else:
|
||||
if not (client.operator or client.tester):
|
||||
raise APIError('API_FORBIDDEN',403)
|
||||
if request.headers.get('content-type','').split(';')[0]!='application/json':
|
||||
raise APIError('API_CONTENT_TYPE',415)
|
||||
body=bytearray()
|
||||
async for chunk in request.stream():
|
||||
body.extend(chunk)
|
||||
if len(body)>64*1024:
|
||||
raise APIError('API_TEST_FEEDBACK_INVALID',413)
|
||||
try:
|
||||
data=json.loads(body)
|
||||
except ValueError:
|
||||
raise APIError('API_TEST_FEEDBACK_INVALID',422) from None
|
||||
result=await run_in_threadpool(test_requests.answer,client,request.path_params['handle'],data)
|
||||
except LockBusy:
|
||||
raise APIError('API_TEST_BUSY',409) from None
|
||||
except (LifecycleError,PackageReadError) as exc:
|
||||
raise APIError(getattr(exc,'code','API_CASE_UNAVAILABLE'),409) from None
|
||||
return JSONResponse(result,headers={'Cache-Control':'no-store'})
|
||||
|
||||
async def web(request):
|
||||
from starlette.responses import Response
|
||||
name = request.path_params.get('asset', 'index.html')
|
||||
if name not in {'index.html', 'dashboard.js', 'dashboard.css', 'case-detail.js','operations.js'}:
|
||||
if name not in {'index.html', 'dashboard.js', 'dashboard.css', 'case-detail.js','operations.js','test-requests.js'}:
|
||||
raise APIError('API_NOT_FOUND', 404)
|
||||
data = files('viano_bug_analysis.web').joinpath(name).read_bytes()
|
||||
response = Response(data, media_type={'index.html': 'text/html', 'dashboard.js': 'text/javascript',
|
||||
'dashboard.css': 'text/css', 'case-detail.js':'text/javascript','operations.js':'text/javascript'}[name])
|
||||
'dashboard.css': 'text/css', 'case-detail.js':'text/javascript','operations.js':'text/javascript','test-requests.js':'text/javascript'}[name])
|
||||
response.headers.update({'Content-Security-Policy': "default-src 'self'; script-src 'self'; style-src 'self'; connect-src 'self'; object-src 'none'; base-uri 'none'; frame-ancestors 'none'",
|
||||
'X-Content-Type-Options': 'nosniff', 'Referrer-Policy': 'no-referrer', 'Cache-Control': 'no-store'})
|
||||
return response
|
||||
@@ -201,6 +229,7 @@ def create_app(config):
|
||||
app = Starlette(routes=[Route('/', web), Route('/assets/{asset}', web),
|
||||
Route('/v1/overview', overview), Route('/healthz', health), Route('/v1/profiles', profiles),
|
||||
Route('/v1/operations',operation,methods=['GET','POST']),
|
||||
Route('/v1/test-requests',tests),Route('/v1/test-requests/{handle}',tests,methods=['POST']),
|
||||
Route('/v1/cases/{handle}', case_detail), Route('/v1/cases/{handle}/package', case_package),
|
||||
Route('/v1/schema', contract), Route('/v1/submissions', submit, methods=['POST']),
|
||||
Route('/v1/submissions/{submission}', status), Route('/v1/submissions/{submission}/package', package)],
|
||||
|
||||
@@ -112,7 +112,7 @@ def next_actions(state, reason, worker):
|
||||
return [{'kind':'download','text':'Die Analyse ist abgeschlossen. Aufträge und Fallakte an die nächste Instanz übergeben.'},
|
||||
{'kind':'return','text':'Nach gescheitertem Patch oder neuen Beobachtungen die unveränderte Akte mit Rückmeldung übergeben.'}]
|
||||
help_ = {
|
||||
'AWAITING_TEST_FEEDBACK': 'Die angeforderte Testerantwort muss über den vorgesehenen Feedbackablauf bestätigt werden. Keine neue Analyse-Revision eröffnen.',
|
||||
'AWAITING_TEST_FEEDBACK': 'Die angeforderte Testerantwort auf der Seite Testanfragen bestätigen. Die Untersuchung wird in derselben Revision fortgesetzt.',
|
||||
'RESEARCH_UNAVAILABLE': 'Datenbank-/Repository-Dienst prüfen. Nach Behebung ist eine gezielte Betreiberfreigabe über den Worker möglich.',
|
||||
'INVALID_LLM_RESPONSES': 'Die Antworten wurden fachlich oder technisch zurückgewiesen. Fehlerdiagnostik prüfen; keine Budgets zurücksetzen.',
|
||||
'LLM_CONFIGURATION': 'Providerkonfiguration und Zugangsdaten durch den Betreiber prüfen.',
|
||||
|
||||
@@ -67,6 +67,7 @@ class Client:
|
||||
token_env: str
|
||||
projects: tuple[str, ...]
|
||||
operator: bool = False
|
||||
tester: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -86,7 +87,7 @@ class APIConfig:
|
||||
or not isinstance(self.host, str) or not self.host):
|
||||
raise APIError('API_CONFIG', 500)
|
||||
for c in self.clients:
|
||||
if (type(c.operator) is not bool or not isinstance(c.id, str) or not IDENTIFIER.fullmatch(c.id)
|
||||
if (type(c.operator) is not bool or type(c.tester) is not bool or not isinstance(c.id, str) or not IDENTIFIER.fullmatch(c.id)
|
||||
or not isinstance(c.token_env, str)
|
||||
or not re.fullmatch(r'[A-Za-z_][A-Za-z0-9_]*', c.token_env)
|
||||
or not c.projects or not set(c.projects) <= ids):
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
"""Authenticated human observations resume the existing case-local test contract."""
|
||||
from .config import APIError
|
||||
from .cases import Cases, case_key
|
||||
from .dashboard import read_json
|
||||
from viano_bug_analysis.lifecycle.async_tests import AsyncTestCoordinator
|
||||
from viano_bug_analysis.lifecycle.checkpoints import checkpoint_chain
|
||||
from viano_bug_analysis.lifecycle.locking import FileLock
|
||||
from viano_bug_analysis.lifecycle.storage import key
|
||||
from viano_bug_analysis.lifecycle.job import timestamp
|
||||
from viano_bug_analysis.package import PackageReader
|
||||
|
||||
|
||||
class TestRequests:
|
||||
def __init__(self,service):
|
||||
self.service=service
|
||||
self.cases=Cases(service)
|
||||
|
||||
def request(self,reader):
|
||||
m=reader.manifest
|
||||
value=read_json(reader,m['artifacts'].get('async_test_request'))
|
||||
if (value.get('executor')!='human' or any(value.get(k)!=m[k]
|
||||
for k in ('case_id','package_id','analysis_revision'))):
|
||||
return None
|
||||
return value
|
||||
|
||||
def listing(self,client):
|
||||
rows=[];errors=0
|
||||
for folder in self.service.workspace.cases.glob('*/*'):
|
||||
try:
|
||||
handle=folder.parent.name+'.'+folder.name
|
||||
path=self.cases.head(handle,client)
|
||||
with PackageReader(path) as reader:
|
||||
m=reader.manifest
|
||||
if m['state']!='BLOCKED' or m['routing']['reason']!='AWAITING_TEST_FEEDBACK':
|
||||
continue
|
||||
request=self.request(reader)
|
||||
if not request:
|
||||
continue
|
||||
ref=m['artifacts']['async_test_request']
|
||||
cp=next(cp for _,cp in checkpoint_chain(reader) if cp['details'].get('operation')=='async-test-wait'
|
||||
and cp['details'].get('request_ref')==ref)
|
||||
case=read_json(reader,m['artifacts'].get('case'))
|
||||
title=next((r.get('title') for r in case.get('bug',{}).get('reports',[]) if r.get('title')),m['case_id'])
|
||||
rows.append({'key':handle,'case_id':m['case_id'],'project_id':m['project']['id'],
|
||||
'title':title,'received_at':cp['created_at'],'request':request})
|
||||
except APIError as error:
|
||||
if error.status!=404:
|
||||
errors+=1
|
||||
except Exception:
|
||||
errors+=1
|
||||
return {'can_answer':client.operator or client.tester,'requests':sorted(rows,key=lambda r:(r['received_at'],r['key'])),
|
||||
'read_errors':errors}
|
||||
|
||||
def answer(self,client,handle,data):
|
||||
if not (client.operator or client.tester):
|
||||
raise APIError('API_FORBIDDEN',403)
|
||||
fields={'request_id','analysis_revision','outcome','observations','environment','confirmed'}
|
||||
if (not isinstance(data,dict) or set(data)!=fields or data['confirmed'] is not True
|
||||
or type(data['analysis_revision']) is not int
|
||||
or data['outcome'] not in ('completed','unable_to_test')
|
||||
or not isinstance(data['environment'],str) or not data['environment'].strip()
|
||||
or len(data['environment'])>4000 or not isinstance(data['observations'],list)
|
||||
or not 1<=len(data['observations'])<=100
|
||||
or any(not isinstance(s,str) or not s.strip() or len(s)>12000 for s in data['observations'])):
|
||||
raise APIError('API_TEST_FEEDBACK_INVALID',422)
|
||||
path=self.cases.head(handle,client)
|
||||
with PackageReader(path) as reader:
|
||||
case_id=reader.manifest['case_id']
|
||||
with FileLock(self.service.workspace.runtime/'locks'/('trial-'+key(case_id)+'.lock')):
|
||||
path=self.cases.head(handle,client)
|
||||
with PackageReader(path) as reader:
|
||||
request=self.request(reader)
|
||||
if not request or (request['request_id'],request['analysis_revision'])!=(data['request_id'],data['analysis_revision']):
|
||||
raise APIError('API_TEST_REQUEST_STALE',409)
|
||||
feedback={k:request[k] for k in ('case_id','package_id','analysis_revision','request_id','hypothesis_id','action_id','executor','format_version')}
|
||||
feedback.update(format='viano-async-test-feedback',outcome=data['outcome'],observations=data['observations'],
|
||||
environment=data['environment'],tester_id=client.id,attachments=[],
|
||||
confirmation={'confirmed':True,'confirmed_by':client.id,'confirmed_at':timestamp()})
|
||||
saved=read_json(reader,reader.manifest['artifacts'].get('async_test_feedback'))
|
||||
# Network retries preserve the original server confirmation timestamp.
|
||||
if saved and all(saved.get(k)==v for k,v in feedback.items() if k!='confirmation') and saved.get('confirmation',{}).get('confirmed_by')==client.id:
|
||||
return {'status':'RESUMED','reused':True}
|
||||
result=AsyncTestCoordinator(self.service.store).confirm_feedback(feedback)
|
||||
return {'status':'RESUMED','reused':result.reused}
|
||||
@@ -29,7 +29,7 @@ class AsyncTestResult:
|
||||
class AsyncTestCoordinator:
|
||||
"""Trusted controller API, never a public HTTP endpoint or direct LLM tool.
|
||||
|
||||
A future authenticated web adapter calls confirm_feedback after approval.
|
||||
The authenticated web adapter calls confirm_feedback after confirmation.
|
||||
No messages are sent and no clients/bots/labs are controlled here.
|
||||
"""
|
||||
|
||||
|
||||
@@ -63,6 +63,7 @@ function renderDetail(data){
|
||||
if(data.reason)actions.append(text('p',`Sperr-/Reviewgrund: ${data.reason}`));
|
||||
if(data.worker.code)actions.append(text('p',`Workerhinweis: ${data.worker.code}`));
|
||||
data.actions.forEach(a=>actions.append(text('p',a.text)));
|
||||
if(data.reason==='AWAITING_TEST_FEEDBACK'){const link=text('a','Offene Testanfragen öffnen','case-link');link.href='#tests';actions.append(link);}
|
||||
if(typeof addRetryControl==='function')addRetryControl(data,actions);
|
||||
root.append(panel('Zustand und nächste Schritte',actions));
|
||||
root.append(panel('Testerbeobachtungen',semantic({observations:data.observations,tester_assumptions:data.tester_assumptions,expected_behavior:data.expected_behavior})));
|
||||
@@ -106,6 +107,8 @@ function syncDetail(){
|
||||
if(!detailLoading&&(!selectedCase||detailHash!==location.hash||(!location.hash.includes('/revision/')&¤t?.package_generation!==selectedCase.package_generation)))loadDetail(location.hash);
|
||||
}
|
||||
function route(){
|
||||
if(typeof testRoute==='function'&&testRoute())return;
|
||||
if($('test-page'))$('test-page').hidden=true;
|
||||
const detail=location.hash.startsWith('#case/'),results=location.hash==='#results',submissionPage=location.hash==='#submit';
|
||||
$('overview-page').hidden=detail||results||submissionPage;$('results').hidden=!results;$('case-detail').hidden=!detail;$('submit').hidden=detail||results;
|
||||
document.querySelector('header h1').textContent=detail?'Die Fallakte.':results?'Abgeschlossene Fälle.':submissionPage?'Auftrag übergeben.':'Analyse im Blick.';
|
||||
|
||||
@@ -27,6 +27,7 @@ function logout() {
|
||||
['workers','case-rows','bars','legend','project','project-filter','result-project','result-rows','case-detail'].forEach(id=>$(id).replaceChildren());
|
||||
if(typeof clearDetails==='function')clearDetails();
|
||||
if(typeof clearOperations==='function')clearOperations();
|
||||
if(typeof clearTests==='function')clearTests();
|
||||
['active-count','token-count','correction-count','finished-count'].forEach(id=>$(id).textContent='—');
|
||||
$('pie').getContext('2d').clearRect(0,0,360,360);
|
||||
}
|
||||
@@ -108,6 +109,7 @@ async function refresh() {
|
||||
try {const response=await api('/v1/overview');if(generation!==session)return;overview=response;render();$('updated').textContent=`Aktualisiert ${new Date().toLocaleTimeString('de-DE')}`;
|
||||
error(overview.read_errors?'Einige Akten konnten nicht gelesen werden. Die Verbrauchsdaten können unvollständig sein.':'');
|
||||
if(typeof refreshOperations==='function')await refreshOperations();
|
||||
if(typeof refreshTests==='function')await refreshTests();
|
||||
} catch(e) {if(generation===session)error(`Aktualisierung fehlgeschlagen: ${e.message}`);}
|
||||
if(token && generation===session)timer=setTimeout(refresh,4000);
|
||||
}
|
||||
|
||||
@@ -3,10 +3,10 @@
|
||||
<head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>BugAnalysis · Übersicht</title><link rel="stylesheet" href="/assets/dashboard.css">
|
||||
<script src="/assets/dashboard.js" defer></script>
|
||||
<script src="/assets/case-detail.js" defer></script><script src="/assets/operations.js" defer></script></head>
|
||||
<script src="/assets/case-detail.js" defer></script><script src="/assets/operations.js" defer></script><script src="/assets/test-requests.js" defer></script></head>
|
||||
<body>
|
||||
<aside class="rail"><a class="brand" href="/">V<span>BA</span><small>BugAnalysis</small></a>
|
||||
<nav aria-label="Navigation"><a href="#overview" class="selected">Übersicht</a><a href="#cases">Laufende Fälle</a><a href="#results">Ergebnisse</a><a href="#submit">Auftrag übergeben</a></nav>
|
||||
<nav aria-label="Navigation"><a href="#overview" class="selected">Übersicht</a><a href="#cases">Laufende Fälle</a><a href="#results">Ergebnisse</a><a href="#tests">Testanfragen</a><a href="#submit">Auftrag übergeben</a></nav>
|
||||
<div class="rail-bottom">ANALYSEBETRIEB<br><span id="connection">Nicht verbunden</span><button id="logout" type="button" hidden>Abmelden</button></div></aside>
|
||||
<main id="overview">
|
||||
<header><div><div class="eyebrow">BETRIEBSÜBERSICHT</div><h1>Analyse im Blick.</h1><p>Aktuelle Arbeit und messbarer Verbrauch.</p></div><div class="live"><span class="dot"></span><span id="updated">Warte auf Verbindung</span></div></header>
|
||||
@@ -14,7 +14,7 @@
|
||||
<form id="login-form"><label for="token">API-Zugangsschlüssel</label><div class="inline"><input id="token" type="password" autocomplete="off" required placeholder="Zugangsschlüssel"><button type="submit">Verbinden</button></div></form></section>
|
||||
<p id="error" class="error" role="alert" hidden></p>
|
||||
<div id="content" hidden>
|
||||
<div id="overview-page">
|
||||
<section class="panel" id="test-page" hidden><h2>Offene Testanfragen</h2><p class="hint">Beobachtungen ergänzen und die laufende Untersuchung fortsetzen.</p><div class="detail-controls"><label>Projekt<select id="test-project"></select></label><label>Sortierung<select id="test-sort"><option value="oldest">Älteste zuerst</option><option value="newest">Neueste zuerst</option><option value="project">Projekt, dann älteste zuerst</option></select></label></div><div id="test-list"></div><div id="test-answer"></div></section><div id="overview-page">
|
||||
<section class="panel" id="operations"><h2>Betriebssteuerung</h2><div id="operation-status"></div><div id="operation-projects"></div><details id="operation-audit" hidden><summary>Aktionsprotokoll</summary><div id="operation-events"></div></details></section><section class="stats" aria-label="Kennzahlen">
|
||||
<article><span>Aktive Fälle</span><strong id="active-count">—</strong><small>Noch nicht abgeschlossen</small></article>
|
||||
<article><span>Erfasste Tokens</span><strong id="token-count">—</strong><small>Eingabe + Ausgabe inkl. Denken</small></article>
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
'use strict';
|
||||
let testData=null,answerKey=null,answerBusy=false;
|
||||
function clearTests(){testData=null;answerKey=null;['test-list','test-answer','test-project'].forEach(id=>$(id).replaceChildren());}
|
||||
function testRoute(){
|
||||
if(location.hash!=='#tests')return false;
|
||||
['overview-page','results','case-detail','submit'].forEach(id=>$(id).hidden=true);$('test-page').hidden=false;
|
||||
document.querySelector('header h1').textContent='Rückfragen beantworten.';
|
||||
document.querySelectorAll('nav a').forEach(a=>a.classList.toggle('selected',a.hash==='#tests'));
|
||||
if(token)refreshTests().catch(e=>error(e.message));return true;
|
||||
}
|
||||
async function refreshTests(){
|
||||
const generation=session;const result=await api('/v1/test-requests');if(generation!==session)return;
|
||||
testData=result;const filter=$('test-project'),selected=filter.value;
|
||||
filter.replaceChildren(new Option('Alle Projekte',''));projects.forEach(p=>filter.add(new Option(p.project_id,p.project_id)));filter.value=selected;
|
||||
renderTests();
|
||||
}
|
||||
function renderTests(){
|
||||
if(!testData)return;
|
||||
const project=$('test-project').value,sort=$('test-sort').value,root=$('test-list');root.replaceChildren();
|
||||
if(testData.read_errors)root.append(text('p','Einige Anfragen konnten nicht gelesen werden.','error'));
|
||||
const rows=testData.requests.filter(r=>!project||r.project_id===project).slice().sort((a,b)=>
|
||||
(sort==='project'?a.project_id.localeCompare(b.project_id):0)||
|
||||
(sort==='newest'?b.received_at.localeCompare(a.received_at):a.received_at.localeCompare(b.received_at))||a.key.localeCompare(b.key));
|
||||
const wrap=text('div','','table-wrap'),table=document.createElement('table'),head=document.createElement('tr');
|
||||
['Eingang','Projekt','Bug','Anfrage',''].forEach(v=>head.append(text('th',v)));const thead=document.createElement('thead');thead.append(head);table.append(thead);const body=document.createElement('tbody');
|
||||
rows.forEach(row=>{const tr=document.createElement('tr');tr.append(text('td',new Date(row.received_at).toLocaleString('de-DE')),text('td',row.project_id));
|
||||
const cell=document.createElement('td'),link=text('a',row.case_id,'case-link');link.href='#case/'+row.key;cell.append(link,text('small',row.title));tr.append(cell,text('td',row.request.question.slice(0,160)+(row.request.question.length>160?' …':'')));
|
||||
const action=document.createElement('td'),button=text('button','Ansehen / beantworten');button.type='button';button.addEventListener('click',()=>openTest(row));action.append(button);tr.append(action);body.append(tr);});
|
||||
table.append(body);wrap.append(table);root.append(wrap);if(!rows.length)root.append(text('p','Keine offenen Testanfragen für diese Auswahl.','hint'));
|
||||
if(answerKey&&!testData.requests.some(r=>r.key===answerKey)){$('test-answer').replaceChildren(text('p','Diese Anfrage ist nicht mehr offen. Bitte die Liste aktualisieren.','hint'));answerKey=null;}
|
||||
}
|
||||
function openTest(row){
|
||||
if(answerBusy)return;
|
||||
answerKey=row.key;const root=$('test-answer');root.replaceChildren();const request=row.request;
|
||||
root.append(text('h3',`${row.project_id} · ${row.case_id}`),text('p',request.question));
|
||||
for(const [field,label] of [['environment','Angeforderte Umgebung'],['prerequisites','Voraussetzungen'],['steps','Testschritte'],['requested_observations','Benötigte Beobachtungen']])root.append(panel(label,semantic(request[field])));
|
||||
root.append(text('p',`Angefragter Termin: ${new Date(request.deadline).toLocaleString('de-DE')}. Ein abgelaufener Termin ersetzt keine Antwort.`,'hint'));
|
||||
if(!testData.can_answer){root.append(text('p','Zum Beantworten werden Tester- oder Betreiberrechte benötigt.','hint'));return;}
|
||||
const form=document.createElement('form');form.className='semantic-list';
|
||||
function field(label,element){const l=text('label',label);l.append(element);form.append(l);return element;}
|
||||
const outcome=document.createElement('select');outcome.add(new Option('Test durchgeführt','completed'));outcome.add(new Option('Nicht durchführbar','unable_to_test'));field('Ergebnis',outcome);
|
||||
const environment=field('Tatsächlich getestete Umgebung / Build',document.createElement('input'));environment.required=true;environment.maxLength=4000;
|
||||
const observations=field('Beobachtungen bzw. Grund, warum der Test nicht möglich ist',document.createElement('textarea'));observations.required=true;observations.rows=7;observations.maxLength=12000;
|
||||
const confirm=document.createElement('input');confirm.type='checkbox';confirm.required=true;field('Ich bestätige diese Angaben als meine Test-Rückmeldung.',confirm);
|
||||
const button=text('button','Antwort bestätigen und Untersuchung fortsetzen');button.type='submit';form.append(button);root.append(form);
|
||||
form.addEventListener('submit',async event=>{event.preventDefault();if(answerBusy)return;const generation=session;answerBusy=true;button.disabled=true;
|
||||
try{await api('/v1/test-requests/'+row.key,{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({request_id:request.request_id,analysis_revision:request.analysis_revision,outcome:outcome.value,environment:environment.value,observations:[observations.value],confirmed:confirm.checked})});
|
||||
if(generation!==session)return;root.replaceChildren(text('p','Antwort gespeichert. Der Worker kann die Untersuchung fortsetzen.'));answerKey=null;await refreshTests();
|
||||
}catch(e){if(generation===session)error(`Antwort nicht gespeichert: ${e.message}`);}finally{answerBusy=false;if(generation===session)button.disabled=false;}});
|
||||
root.scrollIntoView({behavior:'smooth',block:'start'});
|
||||
}
|
||||
$('test-project').addEventListener('change',()=>{answerKey=null;$('test-answer').replaceChildren();renderTests();});$('test-sort').addEventListener('change',renderTests);
|
||||
if(location.hash==='#tests')testRoute();
|
||||
@@ -49,6 +49,16 @@ try {
|
||||
await send('Emulation.setDeviceMetricsOverride',{width:390,height:1200,deviceScaleFactor:1,mobile:true});await pause(200);
|
||||
if(await evaluate('document.documentElement.scrollWidth>innerWidth'))throw new Error('Mobile horizontal overflow');
|
||||
await writeFile(path.join(output,'mobile.png'),Buffer.from((await send('Page.captureScreenshot',{format:'png'})).data,'base64'));
|
||||
await evaluate("location.hash='#tests'");
|
||||
await until(()=>evaluate("!document.getElementById('test-page').hidden&&document.querySelector('#test-list button')"));
|
||||
if(!await evaluate("document.getElementById('test-list').textContent.includes('viano-core')&&document.getElementById('test-list').textContent.includes('BUG-000020')"))throw new Error('Test request project/case missing');
|
||||
const sorting=await evaluate("(()=>{const original=testData.requests;const row=original[0];testData.requests=[{...row,case_id:'NEW',received_at:'2098-01-01T00:00:00Z'},{...row,project_id:'aaa',case_id:'OLD',received_at:'2000-01-01T00:00:00Z'},row];const first=()=>document.querySelector('#test-list tbody tr').textContent;document.getElementById('test-sort').value='oldest';renderTests();const oldest=first().includes('OLD');document.getElementById('test-sort').value='newest';renderTests();const newest=first().includes('NEW');document.getElementById('test-sort').value='project';renderTests();const project=first().includes('aaa');document.getElementById('test-project').value='viano-core';renderTests();const filtered=document.querySelectorAll('#test-list tbody tr').length===2;testData.requests=original;document.getElementById('test-project').value='';renderTests();return oldest&&newest&&project&&filtered;})()");
|
||||
if(!sorting)throw new Error('Test request sorting/filtering failed');
|
||||
await evaluate("document.getElementById('test-sort').value='project';document.getElementById('test-sort').dispatchEvent(new Event('change'));document.querySelector('#test-list button').click()");
|
||||
if(!await evaluate("document.querySelector('#test-answer textarea')&&document.getElementById('test-answer').textContent.includes('Sichtbarkeit und Build')"))throw new Error('Human test answer form missing');
|
||||
if(await evaluate('document.documentElement.scrollWidth>innerWidth'))throw new Error('Test page mobile overflow');
|
||||
await writeFile(path.join(output,'test-requests-mobile.png'),Buffer.from((await send('Page.captureScreenshot',{format:'png'})).data,'base64'));
|
||||
await evaluate("location.hash='#overview'");await pause(100);
|
||||
await evaluate("document.getElementById('project-filter').value='viano-core';document.getElementById('project-filter').dispatchEvent(new Event('change')); ");
|
||||
if(await evaluate("document.getElementById('token-count').textContent")!=='35.300')throw new Error('Project aggregation differs');
|
||||
await evaluate("location.hash='#results'");
|
||||
@@ -67,6 +77,7 @@ try {
|
||||
if(!await evaluate("document.getElementById('content').hidden&&!document.getElementById('login').hidden&&document.getElementById('token-count').textContent==='—'"))throw new Error('Logout retained visible case data');
|
||||
if(!await evaluate("document.getElementById('case-detail').childElementCount===0&&document.getElementById('result-rows').childElementCount===0"))throw new Error('Logout retained detail data');
|
||||
if(!await evaluate("document.getElementById('operation-events').childElementCount===0"))throw new Error('Logout retained audit');
|
||||
if(!await evaluate("document.getElementById('test-answer').childElementCount===0&&document.getElementById('test-list').childElementCount===0"))throw new Error('Logout retained test request data');
|
||||
if(errors.filter(e=>!e.includes('404')).length)throw new Error(errors.join('\n'));
|
||||
console.log(JSON.stringify({status:'PASS',...state,console_errors:errors.filter(e=>!e.includes('404'))}));
|
||||
} finally {
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
import pytest
|
||||
from starlette.testclient import TestClient
|
||||
from viano_bug_analysis.api import APIConfig,Client,Project
|
||||
from viano_bug_analysis.api.app import create_app
|
||||
from viano_bug_analysis.api.cases import case_key
|
||||
from viano_bug_analysis.package import PackageReader
|
||||
from test_async_tests import setup, hypotheses, case_setup, built_case, interpreted_case
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def project_manifest(manifest):
|
||||
manifest['project']={'id':'p','type':'wow','profile':'vcore'}
|
||||
|
||||
|
||||
@pytest.mark.parametrize('outcome',['completed','unable_to_test'])
|
||||
def test_web_human_feedback_auth_scope_replay_and_resume(hypotheses,monkeypatch,outcome):
|
||||
store,budget,coordinator,before,request,_=setup(hypotheses)
|
||||
coordinator.submit(request)
|
||||
monkeypatch.setenv('TESTER_TOKEN','tester_'+'x'*40)
|
||||
monkeypatch.setenv('CAPTURE_TOKEN','capture_'+'x'*40)
|
||||
monkeypatch.setenv('OTHER_TOKEN','other_'+'x'*40)
|
||||
config=APIConfig(store.workspace.root,(Project('p','wow','vcore'),Project('other','wow','vcore')),
|
||||
(Client('tester','TESTER_TOKEN',('p',),tester=True),Client('capture','CAPTURE_TOKEN',('p',)),Client('other','OTHER_TOKEN',('other',),tester=True)))
|
||||
client=TestClient(create_app(config));auth={'Authorization':'Bearer tester_'+'x'*40}
|
||||
url='/v1/test-requests';handle=case_key(*('case-A','stable-package'))
|
||||
assert client.get(url).status_code==401
|
||||
rows=client.get(url,headers=auth).json()
|
||||
assert rows['can_answer'] and rows['requests'][0]['project_id']=='p'
|
||||
answer={'request_id':request['request_id'],'analysis_revision':1,'outcome':outcome,
|
||||
'observations':['Transition absent'],'environment':'build-A','confirmed':True}
|
||||
assert client.post(url+'/'+handle,headers={'Authorization':'Bearer capture_'+'x'*40},json=answer).status_code==403
|
||||
other={'Authorization':'Bearer other_'+'x'*40}
|
||||
assert client.get(url,headers=other).json()['requests']==[]
|
||||
assert client.post(url+'/'+handle,headers=other,json=answer).status_code==404
|
||||
assert client.post(url+'/'+handle,headers=auth,json=answer|{'request_id':'wrong'}).status_code==409
|
||||
assert client.post(url+'/'+handle,headers=auth,json=answer|{'confirmed':False}).status_code==422
|
||||
response=client.post(url+'/'+handle,headers=auth,json=answer)
|
||||
assert response.status_code==200 and response.json()['status']=='RESUMED'
|
||||
assert client.post(url+'/'+handle,headers=auth,json=answer).json()['reused']
|
||||
assert client.post(url+'/'+handle,headers=auth,json=answer|{'observations':['different']}).status_code==409
|
||||
assert client.get(url,headers=auth).json()['requests']==[]
|
||||
assert budget.snapshot('case-A','stable-package').state==before.state
|
||||
with PackageReader(store.latest('case-A','stable-package')) as reader:
|
||||
assert reader.manifest['state']=='INVESTIGATING_HYPOTHESES' and reader.manifest['analysis_revision']==1
|
||||
|
||||
|
||||
def test_unavailable_test_executors_are_not_advertised():
|
||||
from viano_bug_analysis.analysis.investigation_prompt import output_schema
|
||||
def context(costs):
|
||||
return {'tool_schema':{},'budget_policy':{'action_costs':costs}}
|
||||
schema=output_schema(context({}),strict=True)
|
||||
assert not any(b.get('properties',{}).get('kind',{}).get('const')=='test' for b in schema['properties']['next']['oneOf'])
|
||||
schema=output_schema(context({'testing.request_human':40}),strict=True)
|
||||
branch=next(b for b in schema['properties']['next']['oneOf'] if b['properties']['kind'].get('const')=='test')
|
||||
assert branch['properties']['executor']['enum']==['human']
|
||||
@@ -90,6 +90,17 @@ if __name__ == '__main__':
|
||||
root=Path(temporary)
|
||||
identity=seed(root,'BUG-000020','Wolpertinger fehlen in Dun Morogh')
|
||||
seed(root,'BUG-000017','Leuchtbrille – Kontrolllauf',completed=True)
|
||||
# Display-only test request fixture; real feedback/budget binding is tested
|
||||
# independently through the actual coordinator in test_human_web.py.
|
||||
store=CheckpointStore(Workspace(root));ref='ui-test-request.json'
|
||||
request={'case_id':identity[0],'package_id':identity[1],'analysis_revision':1,'request_id':'ui-test',
|
||||
'executor':'human','hypothesis_id':'H1','question':'Ist die Kreatur nach Questannahme sichtbar?',
|
||||
'environment':'Lokaler Testserver','prerequisites':['Quest noch nicht angenommen'],
|
||||
'steps':['Quest annehmen','Umgebung beobachten'],'requested_observations':['Sichtbarkeit und Build notieren'],
|
||||
'deadline':'2099-01-01T00:00:00Z'}
|
||||
store.advance(*identity,'BLOCKED',step_id='ui-test-wait',reason='AWAITING_TEST_FEEDBACK',
|
||||
retry_not_before=request['deadline'],artifacts={'async_test_request':ref},
|
||||
details={'operation':'async-test-wait','request_ref':ref},additions={ref:json_bytes(request)})
|
||||
config=APIConfig(root,(Project('viano-core','wow','vcore'),),
|
||||
(Client('preview','VBA_UI_TEST_TOKEN',('viano-core',),operator=True),))
|
||||
os.environ['VBA_UI_TEST_TOKEN']=TOKEN
|
||||
|
||||
Reference in new issue
Block a user