Add human test request inbox and authenticated feedback resume
This commit is contained in:
1 parent
ac179883c6
commit
9fc5c1805c
18 files changed
+377
-13
No files matched your search
@@ -14,6 +14,7 @@ planning_mode = "joint"
|
|||||||
# project_config = "project.local.toml"
|
# project_config = "project.local.toml"
|
||||||
# llm_config = "llm.early-review.local.toml"
|
# llm_config = "llm.early-review.local.toml"
|
||||||
# investigation_config = "investigation.bounded.toml"
|
# investigation_config = "investigation.bounded.toml"
|
||||||
|
# budget_config = "budget.human-tests.toml"
|
||||||
|
|
||||||
[[clients]]
|
[[clients]]
|
||||||
id = "wow-capture-worker"
|
id = "wow-capture-worker"
|
||||||
@@ -26,3 +27,4 @@ projects = ["viano-core"]
|
|||||||
# token_env = "VBA_OPERATOR_TOKEN"
|
# token_env = "VBA_OPERATOR_TOKEN"
|
||||||
# projects = ["viano-core"]
|
# projects = ["viano-core"]
|
||||||
# operator = true
|
# operator = true
|
||||||
|
# tester = true
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
# Human requests are available when an authenticated web feedback adapter exists.
|
||||||
|
# Frozen per new analysis revision; existing revisions retain their policy.
|
||||||
|
[budget]
|
||||||
|
[budget.action_costs]
|
||||||
|
"testing.request_human" = 40
|
||||||
@@ -1,8 +1,9 @@
|
|||||||
# Asynchronous test handoff v1
|
# Asynchronous test handoff v1
|
||||||
|
|
||||||
This addition prepares the existing project for externally performed human and
|
This contract supports externally performed human and automated tests. The human
|
||||||
automated tests. No web UI, HTTP feedback endpoint, notifications, bot steering,
|
HTTP adapter and dedicated web page are now documented in
|
||||||
client automation, Tool Router or Investigation Loop is implemented here.
|
[Web console](web-console.md#menschliche-testanfragen). Automated execution,
|
||||||
|
notifications, bot steering and client automation are not provided by that page.
|
||||||
`docs/architecture.md` remains unchanged and authoritative: VianoBugAnalysis
|
`docs/architecture.md` remains unchanged and authoritative: VianoBugAnalysis
|
||||||
does not steer labs. It requests observations through a future external adapter.
|
does not steer labs. It requests observations through a future external adapter.
|
||||||
|
|
||||||
@@ -10,8 +11,8 @@ does not steer labs. It requests observations through a future external adapter.
|
|||||||
|
|
||||||
`AsyncTestCoordinator` is imported from `viano_bug_analysis.lifecycle.async_tests`.
|
`AsyncTestCoordinator` is imported from `viano_bug_analysis.lifecycle.async_tests`.
|
||||||
It is a trusted controller service, not a callable operation exposed directly to
|
It is a trusted controller service, not a callable operation exposed directly to
|
||||||
an LLM. The future router must validate requests before calling it. The future
|
an LLM. The investigation controller validates requests before calling it. The
|
||||||
web adapter must authenticate/authorize users and confirm feedback before calling
|
web adapter authenticates/authorizes users and confirms feedback before calling
|
||||||
`confirm_feedback`; a payload claiming `confirmed = true` is not authentication.
|
`confirm_feedback`; a payload claiming `confirmed = true` is not authentication.
|
||||||
User-supplied observations are untrusted data, not executable instructions.
|
User-supplied observations are untrusted data, not executable instructions.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# Menschliche Rückfragen im Web – 2026-10-22
|
||||||
|
|
||||||
|
Ziel: Die vorhandene menschliche Testanforderung bedienbar und für neue
|
||||||
|
Dienstanalysen freigegeben machen. Eigene Seite mit offenen Anfragen, Projekt,
|
||||||
|
Bug, Zeit, Projektfilter und Sortierungen; Anzeige des Testauftrags und bestätigtes
|
||||||
|
Textfeedback. Separate Testerrechte bzw. Betreiberrechte erforderlich.
|
||||||
|
|
||||||
|
Neue Budgetdatei `config/budget.human-tests.toml`: menschliche Tests 40 fachliche
|
||||||
|
Einheiten, keine Freigabe automatisierter Tests. Die ignorierte lokale
|
||||||
|
API-Konfiguration verweist darauf. Vorhandene eingefrorene Revisionen unverändert.
|
||||||
|
Antworten benutzen den bestehenden AsyncTestCoordinator und dessen Budget-/Identitäts-
|
||||||
|
und Resume-Prüfungen, keine eigene Zustandsabkürzung. Serverseitige Bestätigung
|
||||||
|
unter der authentifizierten Clientkennung. Keine kostenpflichtigen LLM-Aufrufe.
|
||||||
|
|
||||||
|
Prüfung mit synthetischen Daten:
|
||||||
|
|
||||||
|
- API-/Coordinator-/Dashboard-/Worker-Tests: beide Antwortausgänge,
|
||||||
|
Projektberechtigung, fehlende Rechte, falsche Anfrage, fehlende Bestätigung,
|
||||||
|
identische bzw. widersprüchliche Wiederholung, unveränderte Budgetkonten und
|
||||||
|
Fortsetzung in derselben Revision.
|
||||||
|
Zusammen 51 Tests bestanden.
|
||||||
|
- Schemavertrag: ohne Budgetfreigabe kein Testzweig, bei menschlicher Freigabe
|
||||||
|
ausschließlich `human` als Testexecutor.
|
||||||
|
- Echter Headless-Edge: eigene Seite, Fall-/Projektanzeige, Sortierauswahl,
|
||||||
|
Auftragsanzeige und Antwortformular, Mobilansicht ohne horizontalen Überlauf,
|
||||||
|
Datenentfernung bei Abmeldung. Feedbackwirkung separat mit echter Coordinator-
|
||||||
|
und Budget-Fixture getestet, UI-Anfrage ist ausdrücklich eine Anzeige-Fixture.
|
||||||
|
- Vor Tests Test-Workspaces 16.939.129.428 Bytes; kleine Fixtures und temporäre
|
||||||
|
pytest-/Browserdaten bleiben mit Reserve innerhalb des 20-GB-Budgets.
|
||||||
|
- Breitere Untersuchungstests: 72 bestanden, 26 alte Tests in
|
||||||
|
`test_investigator.py` scheitern am vorhandenen `INVESTIGATION_PROMPT_LIMIT`.
|
||||||
|
Vollständige Gegenprobe dieses Moduls mit deaktivierter neuer Schemafilterung:
|
||||||
|
dieselben 26 Fehler, 6 bestanden. Keine Erweiterung der Zeichengrenze vorgenommen.
|
||||||
|
|
||||||
|
Grenzen: Textfeedback; Dateianhänge und automatisierte Testausführung sind keine
|
||||||
|
Bestandteile dieser Oberfläche. Alte Revisionen benötigen ihre ursprüngliche Policy.
|
||||||
@@ -126,6 +126,54 @@ Protokoll/Workerzustand. `POST /v1/operations` benötigt Betreiberrechte und JSO
|
|||||||
Bei gleichzeitigem Zugriff kann `API_CONTROL_BUSY` auftreten; nach Aktualisierung
|
Bei gleichzeitigem Zugriff kann `API_CONTROL_BUSY` auftreten; nach Aktualisierung
|
||||||
erneut versuchen. Der Worker muss nach diesem Softwareupdate neu gestartet werden.
|
erneut versuchen. Der Worker muss nach diesem Softwareupdate neu gestartet werden.
|
||||||
|
|
||||||
|
## Menschliche Testanfragen
|
||||||
|
|
||||||
|
Die eigene Seite **Testanfragen** listet offene menschliche Rückfragen mit
|
||||||
|
Eingangszeit, Projekt, Bug-ID, Falltitel und Frage. Projektfilter sowie Sortierung
|
||||||
|
„Älteste zuerst“, „Neueste zuerst“ und „Projekt, dann älteste zuerst“ ermöglichen
|
||||||
|
chronologisches oder projektweises Abarbeiten. Die Liste aktualisiert sich mit der
|
||||||
|
Übersicht; eine noch offene Eingabe wird dabei nicht überschrieben.
|
||||||
|
|
||||||
|
„Ansehen / beantworten“ zeigt Umgebung, Voraussetzungen, Testschritte, benötigte
|
||||||
|
Beobachtungen und angefragten Termin. Die Antwort erfasst die tatsächliche Umgebung,
|
||||||
|
Beobachtungen und „Test durchgeführt“ oder „Nicht durchführbar“. Die ausdrückliche
|
||||||
|
Bestätigung ist erforderlich. Textdaten bleiben untrusted observations, keine
|
||||||
|
auszuführenden Befehle. Diese erste Oberfläche nimmt Text, keine Dateianhänge an.
|
||||||
|
|
||||||
|
Clients brauchen `tester = true` oder Betreiberrechte und Projektzugriff, um eine
|
||||||
|
Antwort zu bestätigen. Ein reiner Uploadclient erhält diese Rechte nicht automatisch.
|
||||||
|
Beispiel eines separaten menschlichen Clients in der API-Konfiguration:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[[clients]]
|
||||||
|
id = "human-dev"
|
||||||
|
token_env = "VBA_HUMAN_TOKEN"
|
||||||
|
projects = ["viano-core"]
|
||||||
|
tester = true
|
||||||
|
```
|
||||||
|
|
||||||
|
Eigenen API-Token über diese Umgebungsvariable bereitstellen und API neu starten.
|
||||||
|
Die bestätigende Identität und Zeit stammen vom Server, nicht aus frei editierbaren
|
||||||
|
Formularfeldern. `GET /v1/test-requests` liefert ausschließlich freigegebene Projekte;
|
||||||
|
`POST /v1/test-requests/{case-key}` bestätigt die konkrete Anfrage/Revision.
|
||||||
|
Kollisionen, veraltete Anfragen und widersprüchliche Wiederholungen werden abgewiesen.
|
||||||
|
Eine identische erneute Übermittlung desselben Clients bleibt ohne doppelte Wirkung.
|
||||||
|
|
||||||
|
Eine bestätigte Antwort setzt `AWAITING_TEST_FEEDBACK` über den vorhandenen
|
||||||
|
Coordinator auf `INVESTIGATING_HYPOTHESES` zurück. Revision, Budget und ursprüngliche
|
||||||
|
Belege bleiben erhalten. Der laufende Worker verarbeitet die Rückmeldung regulär;
|
||||||
|
eine besondere Betreiberfreigabe ist dafür nicht erforderlich. Der Termin führt
|
||||||
|
auch nach Ablauf zu keiner automatischen Fortsetzung und verwirft die Anfrage nicht.
|
||||||
|
|
||||||
|
Die Werkzeugfreigabe erfolgt über `budget_config = "budget.human-tests.toml"` in
|
||||||
|
den vertrauenswürdigen Projekteinstellungen. Die lokale Dienstkonfiguration wurde
|
||||||
|
entsprechend ergänzt. Diese Policy erlaubt `testing.request_human` für 40 fachliche
|
||||||
|
Einheiten, automatisierte Tests bleiben ohne Eintrag gesperrt. Bei Trial-Läufen
|
||||||
|
die Datei ausdrücklich mit `--budget-config config/budget.human-tests.toml` angeben.
|
||||||
|
Bestehende Revisionen behalten ihre eingefrorene Policy; sie werden nicht migriert.
|
||||||
|
Nach Konfigurationsänderung API und Worker neu starten. Keine Modellanfrage wird
|
||||||
|
allein durch Öffnen oder Beantworten der Seite gestartet; der Worker läuft separat.
|
||||||
|
|
||||||
`GET /v1/cases/{key}?revision=N` liefert diese projektgebundene Ansicht.
|
`GET /v1/cases/{key}?revision=N` liefert diese projektgebundene Ansicht.
|
||||||
`GET /v1/cases/{key}/package` lädt die vollständige aktuelle Akte einschließlich
|
`GET /v1/cases/{key}/package` lädt die vollständige aktuelle Akte einschließlich
|
||||||
aller erhaltenen Revisionen herunter, auch als Zwischenstand eines laufenden Falls.
|
aller erhaltenen Revisionen herunter, auch als Zwischenstand eines laufenden Falls.
|
||||||
|
|||||||
@@ -7,6 +7,29 @@ PROMPT_VERSION="investigation_v1"
|
|||||||
|
|
||||||
|
|
||||||
def output_schema(context=None, *, strict=False):
|
def output_schema(context=None, *, strict=False):
|
||||||
|
schema=_output_schema(context,strict=strict)
|
||||||
|
if context and 'budget_policy' in context:
|
||||||
|
costs=context['budget_policy']['action_costs']
|
||||||
|
allowed=[e for e in ('human','automated') if 'testing.request_'+e in costs]
|
||||||
|
def restrict(value):
|
||||||
|
if isinstance(value,list):
|
||||||
|
return [restrict(v) for v in value if not (isinstance(v,dict)
|
||||||
|
and v.get('properties',{}).get('kind',{}).get('const')=='test' and not allowed)]
|
||||||
|
if not isinstance(value,dict):return value
|
||||||
|
value={k:restrict(v) for k,v in value.items()}
|
||||||
|
properties=value.get('properties',{})
|
||||||
|
if properties.get('kind',{}).get('const')=='test':
|
||||||
|
properties['executor']['enum']=allowed
|
||||||
|
if not allowed:
|
||||||
|
for field in ('kind','intent'):
|
||||||
|
if 'enum' in properties.get(field,{}):
|
||||||
|
properties[field]['enum']=[v for v in properties[field]['enum'] if v!='test']
|
||||||
|
return value
|
||||||
|
schema=restrict(schema)
|
||||||
|
return schema
|
||||||
|
|
||||||
|
|
||||||
|
def _output_schema(context=None, *, strict=False):
|
||||||
if context and context.get("startup_explore"):
|
if context and context.get("startup_explore"):
|
||||||
from .investigation_startup import explore_schema
|
from .investigation_startup import explore_schema
|
||||||
return explore_schema(context)
|
return explore_schema(context)
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ from .service import IntakeService
|
|||||||
from .dashboard import Dashboard
|
from .dashboard import Dashboard
|
||||||
from .cases import Cases
|
from .cases import Cases
|
||||||
from .operations import Operations
|
from .operations import Operations
|
||||||
|
from .test_requests import TestRequests
|
||||||
from viano_bug_analysis.lifecycle.errors import LockBusy
|
from viano_bug_analysis.lifecycle.errors import LockBusy
|
||||||
from viano_bug_analysis.lifecycle.errors import LifecycleError
|
from viano_bug_analysis.lifecycle.errors import LifecycleError
|
||||||
from viano_bug_analysis.package import PackageReadError
|
from viano_bug_analysis.package import PackageReadError
|
||||||
@@ -32,6 +33,7 @@ def create_app(config):
|
|||||||
dashboard = Dashboard(service)
|
dashboard = Dashboard(service)
|
||||||
cases = Cases(service)
|
cases = Cases(service)
|
||||||
operations = Operations(service)
|
operations = Operations(service)
|
||||||
|
test_requests = TestRequests(service)
|
||||||
tokens = config.tokens()
|
tokens = config.tokens()
|
||||||
service.recover()
|
service.recover()
|
||||||
active = 0
|
active = 0
|
||||||
@@ -119,14 +121,40 @@ def create_app(config):
|
|||||||
raise APIError('API_CONTROL_UNAVAILABLE',503) from None
|
raise APIError('API_CONTROL_UNAVAILABLE',503) from None
|
||||||
return JSONResponse(result,headers={'Cache-Control':'no-store'})
|
return JSONResponse(result,headers={'Cache-Control':'no-store'})
|
||||||
|
|
||||||
|
async def tests(request):
|
||||||
|
client=authenticate(request)
|
||||||
|
try:
|
||||||
|
if request.method=='GET':
|
||||||
|
result=await run_in_threadpool(test_requests.listing,client)
|
||||||
|
else:
|
||||||
|
if not (client.operator or client.tester):
|
||||||
|
raise APIError('API_FORBIDDEN',403)
|
||||||
|
if request.headers.get('content-type','').split(';')[0]!='application/json':
|
||||||
|
raise APIError('API_CONTENT_TYPE',415)
|
||||||
|
body=bytearray()
|
||||||
|
async for chunk in request.stream():
|
||||||
|
body.extend(chunk)
|
||||||
|
if len(body)>64*1024:
|
||||||
|
raise APIError('API_TEST_FEEDBACK_INVALID',413)
|
||||||
|
try:
|
||||||
|
data=json.loads(body)
|
||||||
|
except ValueError:
|
||||||
|
raise APIError('API_TEST_FEEDBACK_INVALID',422) from None
|
||||||
|
result=await run_in_threadpool(test_requests.answer,client,request.path_params['handle'],data)
|
||||||
|
except LockBusy:
|
||||||
|
raise APIError('API_TEST_BUSY',409) from None
|
||||||
|
except (LifecycleError,PackageReadError) as exc:
|
||||||
|
raise APIError(getattr(exc,'code','API_CASE_UNAVAILABLE'),409) from None
|
||||||
|
return JSONResponse(result,headers={'Cache-Control':'no-store'})
|
||||||
|
|
||||||
async def web(request):
|
async def web(request):
|
||||||
from starlette.responses import Response
|
from starlette.responses import Response
|
||||||
name = request.path_params.get('asset', 'index.html')
|
name = request.path_params.get('asset', 'index.html')
|
||||||
if name not in {'index.html', 'dashboard.js', 'dashboard.css', 'case-detail.js','operations.js'}:
|
if name not in {'index.html', 'dashboard.js', 'dashboard.css', 'case-detail.js','operations.js','test-requests.js'}:
|
||||||
raise APIError('API_NOT_FOUND', 404)
|
raise APIError('API_NOT_FOUND', 404)
|
||||||
data = files('viano_bug_analysis.web').joinpath(name).read_bytes()
|
data = files('viano_bug_analysis.web').joinpath(name).read_bytes()
|
||||||
response = Response(data, media_type={'index.html': 'text/html', 'dashboard.js': 'text/javascript',
|
response = Response(data, media_type={'index.html': 'text/html', 'dashboard.js': 'text/javascript',
|
||||||
'dashboard.css': 'text/css', 'case-detail.js':'text/javascript','operations.js':'text/javascript'}[name])
|
'dashboard.css': 'text/css', 'case-detail.js':'text/javascript','operations.js':'text/javascript','test-requests.js':'text/javascript'}[name])
|
||||||
response.headers.update({'Content-Security-Policy': "default-src 'self'; script-src 'self'; style-src 'self'; connect-src 'self'; object-src 'none'; base-uri 'none'; frame-ancestors 'none'",
|
response.headers.update({'Content-Security-Policy': "default-src 'self'; script-src 'self'; style-src 'self'; connect-src 'self'; object-src 'none'; base-uri 'none'; frame-ancestors 'none'",
|
||||||
'X-Content-Type-Options': 'nosniff', 'Referrer-Policy': 'no-referrer', 'Cache-Control': 'no-store'})
|
'X-Content-Type-Options': 'nosniff', 'Referrer-Policy': 'no-referrer', 'Cache-Control': 'no-store'})
|
||||||
return response
|
return response
|
||||||
@@ -201,6 +229,7 @@ def create_app(config):
|
|||||||
app = Starlette(routes=[Route('/', web), Route('/assets/{asset}', web),
|
app = Starlette(routes=[Route('/', web), Route('/assets/{asset}', web),
|
||||||
Route('/v1/overview', overview), Route('/healthz', health), Route('/v1/profiles', profiles),
|
Route('/v1/overview', overview), Route('/healthz', health), Route('/v1/profiles', profiles),
|
||||||
Route('/v1/operations',operation,methods=['GET','POST']),
|
Route('/v1/operations',operation,methods=['GET','POST']),
|
||||||
|
Route('/v1/test-requests',tests),Route('/v1/test-requests/{handle}',tests,methods=['POST']),
|
||||||
Route('/v1/cases/{handle}', case_detail), Route('/v1/cases/{handle}/package', case_package),
|
Route('/v1/cases/{handle}', case_detail), Route('/v1/cases/{handle}/package', case_package),
|
||||||
Route('/v1/schema', contract), Route('/v1/submissions', submit, methods=['POST']),
|
Route('/v1/schema', contract), Route('/v1/submissions', submit, methods=['POST']),
|
||||||
Route('/v1/submissions/{submission}', status), Route('/v1/submissions/{submission}/package', package)],
|
Route('/v1/submissions/{submission}', status), Route('/v1/submissions/{submission}/package', package)],
|
||||||
|
|||||||
@@ -112,7 +112,7 @@ def next_actions(state, reason, worker):
|
|||||||
return [{'kind':'download','text':'Die Analyse ist abgeschlossen. Aufträge und Fallakte an die nächste Instanz übergeben.'},
|
return [{'kind':'download','text':'Die Analyse ist abgeschlossen. Aufträge und Fallakte an die nächste Instanz übergeben.'},
|
||||||
{'kind':'return','text':'Nach gescheitertem Patch oder neuen Beobachtungen die unveränderte Akte mit Rückmeldung übergeben.'}]
|
{'kind':'return','text':'Nach gescheitertem Patch oder neuen Beobachtungen die unveränderte Akte mit Rückmeldung übergeben.'}]
|
||||||
help_ = {
|
help_ = {
|
||||||
'AWAITING_TEST_FEEDBACK': 'Die angeforderte Testerantwort muss über den vorgesehenen Feedbackablauf bestätigt werden. Keine neue Analyse-Revision eröffnen.',
|
'AWAITING_TEST_FEEDBACK': 'Die angeforderte Testerantwort auf der Seite Testanfragen bestätigen. Die Untersuchung wird in derselben Revision fortgesetzt.',
|
||||||
'RESEARCH_UNAVAILABLE': 'Datenbank-/Repository-Dienst prüfen. Nach Behebung ist eine gezielte Betreiberfreigabe über den Worker möglich.',
|
'RESEARCH_UNAVAILABLE': 'Datenbank-/Repository-Dienst prüfen. Nach Behebung ist eine gezielte Betreiberfreigabe über den Worker möglich.',
|
||||||
'INVALID_LLM_RESPONSES': 'Die Antworten wurden fachlich oder technisch zurückgewiesen. Fehlerdiagnostik prüfen; keine Budgets zurücksetzen.',
|
'INVALID_LLM_RESPONSES': 'Die Antworten wurden fachlich oder technisch zurückgewiesen. Fehlerdiagnostik prüfen; keine Budgets zurücksetzen.',
|
||||||
'LLM_CONFIGURATION': 'Providerkonfiguration und Zugangsdaten durch den Betreiber prüfen.',
|
'LLM_CONFIGURATION': 'Providerkonfiguration und Zugangsdaten durch den Betreiber prüfen.',
|
||||||
|
|||||||
@@ -67,6 +67,7 @@ class Client:
|
|||||||
token_env: str
|
token_env: str
|
||||||
projects: tuple[str, ...]
|
projects: tuple[str, ...]
|
||||||
operator: bool = False
|
operator: bool = False
|
||||||
|
tester: bool = False
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -86,7 +87,7 @@ class APIConfig:
|
|||||||
or not isinstance(self.host, str) or not self.host):
|
or not isinstance(self.host, str) or not self.host):
|
||||||
raise APIError('API_CONFIG', 500)
|
raise APIError('API_CONFIG', 500)
|
||||||
for c in self.clients:
|
for c in self.clients:
|
||||||
if (type(c.operator) is not bool or not isinstance(c.id, str) or not IDENTIFIER.fullmatch(c.id)
|
if (type(c.operator) is not bool or type(c.tester) is not bool or not isinstance(c.id, str) or not IDENTIFIER.fullmatch(c.id)
|
||||||
or not isinstance(c.token_env, str)
|
or not isinstance(c.token_env, str)
|
||||||
or not re.fullmatch(r'[A-Za-z_][A-Za-z0-9_]*', c.token_env)
|
or not re.fullmatch(r'[A-Za-z_][A-Za-z0-9_]*', c.token_env)
|
||||||
or not c.projects or not set(c.projects) <= ids):
|
or not c.projects or not set(c.projects) <= ids):
|
||||||
|
|||||||
@@ -0,0 +1,84 @@
|
|||||||
|
"""Authenticated human observations resume the existing case-local test contract."""
|
||||||
|
from .config import APIError
|
||||||
|
from .cases import Cases, case_key
|
||||||
|
from .dashboard import read_json
|
||||||
|
from viano_bug_analysis.lifecycle.async_tests import AsyncTestCoordinator
|
||||||
|
from viano_bug_analysis.lifecycle.checkpoints import checkpoint_chain
|
||||||
|
from viano_bug_analysis.lifecycle.locking import FileLock
|
||||||
|
from viano_bug_analysis.lifecycle.storage import key
|
||||||
|
from viano_bug_analysis.lifecycle.job import timestamp
|
||||||
|
from viano_bug_analysis.package import PackageReader
|
||||||
|
|
||||||
|
|
||||||
|
class TestRequests:
|
||||||
|
def __init__(self,service):
|
||||||
|
self.service=service
|
||||||
|
self.cases=Cases(service)
|
||||||
|
|
||||||
|
def request(self,reader):
|
||||||
|
m=reader.manifest
|
||||||
|
value=read_json(reader,m['artifacts'].get('async_test_request'))
|
||||||
|
if (value.get('executor')!='human' or any(value.get(k)!=m[k]
|
||||||
|
for k in ('case_id','package_id','analysis_revision'))):
|
||||||
|
return None
|
||||||
|
return value
|
||||||
|
|
||||||
|
def listing(self,client):
|
||||||
|
rows=[];errors=0
|
||||||
|
for folder in self.service.workspace.cases.glob('*/*'):
|
||||||
|
try:
|
||||||
|
handle=folder.parent.name+'.'+folder.name
|
||||||
|
path=self.cases.head(handle,client)
|
||||||
|
with PackageReader(path) as reader:
|
||||||
|
m=reader.manifest
|
||||||
|
if m['state']!='BLOCKED' or m['routing']['reason']!='AWAITING_TEST_FEEDBACK':
|
||||||
|
continue
|
||||||
|
request=self.request(reader)
|
||||||
|
if not request:
|
||||||
|
continue
|
||||||
|
ref=m['artifacts']['async_test_request']
|
||||||
|
cp=next(cp for _,cp in checkpoint_chain(reader) if cp['details'].get('operation')=='async-test-wait'
|
||||||
|
and cp['details'].get('request_ref')==ref)
|
||||||
|
case=read_json(reader,m['artifacts'].get('case'))
|
||||||
|
title=next((r.get('title') for r in case.get('bug',{}).get('reports',[]) if r.get('title')),m['case_id'])
|
||||||
|
rows.append({'key':handle,'case_id':m['case_id'],'project_id':m['project']['id'],
|
||||||
|
'title':title,'received_at':cp['created_at'],'request':request})
|
||||||
|
except APIError as error:
|
||||||
|
if error.status!=404:
|
||||||
|
errors+=1
|
||||||
|
except Exception:
|
||||||
|
errors+=1
|
||||||
|
return {'can_answer':client.operator or client.tester,'requests':sorted(rows,key=lambda r:(r['received_at'],r['key'])),
|
||||||
|
'read_errors':errors}
|
||||||
|
|
||||||
|
def answer(self,client,handle,data):
|
||||||
|
if not (client.operator or client.tester):
|
||||||
|
raise APIError('API_FORBIDDEN',403)
|
||||||
|
fields={'request_id','analysis_revision','outcome','observations','environment','confirmed'}
|
||||||
|
if (not isinstance(data,dict) or set(data)!=fields or data['confirmed'] is not True
|
||||||
|
or type(data['analysis_revision']) is not int
|
||||||
|
or data['outcome'] not in ('completed','unable_to_test')
|
||||||
|
or not isinstance(data['environment'],str) or not data['environment'].strip()
|
||||||
|
or len(data['environment'])>4000 or not isinstance(data['observations'],list)
|
||||||
|
or not 1<=len(data['observations'])<=100
|
||||||
|
or any(not isinstance(s,str) or not s.strip() or len(s)>12000 for s in data['observations'])):
|
||||||
|
raise APIError('API_TEST_FEEDBACK_INVALID',422)
|
||||||
|
path=self.cases.head(handle,client)
|
||||||
|
with PackageReader(path) as reader:
|
||||||
|
case_id=reader.manifest['case_id']
|
||||||
|
with FileLock(self.service.workspace.runtime/'locks'/('trial-'+key(case_id)+'.lock')):
|
||||||
|
path=self.cases.head(handle,client)
|
||||||
|
with PackageReader(path) as reader:
|
||||||
|
request=self.request(reader)
|
||||||
|
if not request or (request['request_id'],request['analysis_revision'])!=(data['request_id'],data['analysis_revision']):
|
||||||
|
raise APIError('API_TEST_REQUEST_STALE',409)
|
||||||
|
feedback={k:request[k] for k in ('case_id','package_id','analysis_revision','request_id','hypothesis_id','action_id','executor','format_version')}
|
||||||
|
feedback.update(format='viano-async-test-feedback',outcome=data['outcome'],observations=data['observations'],
|
||||||
|
environment=data['environment'],tester_id=client.id,attachments=[],
|
||||||
|
confirmation={'confirmed':True,'confirmed_by':client.id,'confirmed_at':timestamp()})
|
||||||
|
saved=read_json(reader,reader.manifest['artifacts'].get('async_test_feedback'))
|
||||||
|
# Network retries preserve the original server confirmation timestamp.
|
||||||
|
if saved and all(saved.get(k)==v for k,v in feedback.items() if k!='confirmation') and saved.get('confirmation',{}).get('confirmed_by')==client.id:
|
||||||
|
return {'status':'RESUMED','reused':True}
|
||||||
|
result=AsyncTestCoordinator(self.service.store).confirm_feedback(feedback)
|
||||||
|
return {'status':'RESUMED','reused':result.reused}
|
||||||
@@ -29,7 +29,7 @@ class AsyncTestResult:
|
|||||||
class AsyncTestCoordinator:
|
class AsyncTestCoordinator:
|
||||||
"""Trusted controller API, never a public HTTP endpoint or direct LLM tool.
|
"""Trusted controller API, never a public HTTP endpoint or direct LLM tool.
|
||||||
|
|
||||||
A future authenticated web adapter calls confirm_feedback after approval.
|
The authenticated web adapter calls confirm_feedback after confirmation.
|
||||||
No messages are sent and no clients/bots/labs are controlled here.
|
No messages are sent and no clients/bots/labs are controlled here.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|||||||
@@ -63,6 +63,7 @@ function renderDetail(data){
|
|||||||
if(data.reason)actions.append(text('p',`Sperr-/Reviewgrund: ${data.reason}`));
|
if(data.reason)actions.append(text('p',`Sperr-/Reviewgrund: ${data.reason}`));
|
||||||
if(data.worker.code)actions.append(text('p',`Workerhinweis: ${data.worker.code}`));
|
if(data.worker.code)actions.append(text('p',`Workerhinweis: ${data.worker.code}`));
|
||||||
data.actions.forEach(a=>actions.append(text('p',a.text)));
|
data.actions.forEach(a=>actions.append(text('p',a.text)));
|
||||||
|
if(data.reason==='AWAITING_TEST_FEEDBACK'){const link=text('a','Offene Testanfragen öffnen','case-link');link.href='#tests';actions.append(link);}
|
||||||
if(typeof addRetryControl==='function')addRetryControl(data,actions);
|
if(typeof addRetryControl==='function')addRetryControl(data,actions);
|
||||||
root.append(panel('Zustand und nächste Schritte',actions));
|
root.append(panel('Zustand und nächste Schritte',actions));
|
||||||
root.append(panel('Testerbeobachtungen',semantic({observations:data.observations,tester_assumptions:data.tester_assumptions,expected_behavior:data.expected_behavior})));
|
root.append(panel('Testerbeobachtungen',semantic({observations:data.observations,tester_assumptions:data.tester_assumptions,expected_behavior:data.expected_behavior})));
|
||||||
@@ -106,6 +107,8 @@ function syncDetail(){
|
|||||||
if(!detailLoading&&(!selectedCase||detailHash!==location.hash||(!location.hash.includes('/revision/')&¤t?.package_generation!==selectedCase.package_generation)))loadDetail(location.hash);
|
if(!detailLoading&&(!selectedCase||detailHash!==location.hash||(!location.hash.includes('/revision/')&¤t?.package_generation!==selectedCase.package_generation)))loadDetail(location.hash);
|
||||||
}
|
}
|
||||||
function route(){
|
function route(){
|
||||||
|
if(typeof testRoute==='function'&&testRoute())return;
|
||||||
|
if($('test-page'))$('test-page').hidden=true;
|
||||||
const detail=location.hash.startsWith('#case/'),results=location.hash==='#results',submissionPage=location.hash==='#submit';
|
const detail=location.hash.startsWith('#case/'),results=location.hash==='#results',submissionPage=location.hash==='#submit';
|
||||||
$('overview-page').hidden=detail||results||submissionPage;$('results').hidden=!results;$('case-detail').hidden=!detail;$('submit').hidden=detail||results;
|
$('overview-page').hidden=detail||results||submissionPage;$('results').hidden=!results;$('case-detail').hidden=!detail;$('submit').hidden=detail||results;
|
||||||
document.querySelector('header h1').textContent=detail?'Die Fallakte.':results?'Abgeschlossene Fälle.':submissionPage?'Auftrag übergeben.':'Analyse im Blick.';
|
document.querySelector('header h1').textContent=detail?'Die Fallakte.':results?'Abgeschlossene Fälle.':submissionPage?'Auftrag übergeben.':'Analyse im Blick.';
|
||||||
|
|||||||
@@ -27,6 +27,7 @@ function logout() {
|
|||||||
['workers','case-rows','bars','legend','project','project-filter','result-project','result-rows','case-detail'].forEach(id=>$(id).replaceChildren());
|
['workers','case-rows','bars','legend','project','project-filter','result-project','result-rows','case-detail'].forEach(id=>$(id).replaceChildren());
|
||||||
if(typeof clearDetails==='function')clearDetails();
|
if(typeof clearDetails==='function')clearDetails();
|
||||||
if(typeof clearOperations==='function')clearOperations();
|
if(typeof clearOperations==='function')clearOperations();
|
||||||
|
if(typeof clearTests==='function')clearTests();
|
||||||
['active-count','token-count','correction-count','finished-count'].forEach(id=>$(id).textContent='—');
|
['active-count','token-count','correction-count','finished-count'].forEach(id=>$(id).textContent='—');
|
||||||
$('pie').getContext('2d').clearRect(0,0,360,360);
|
$('pie').getContext('2d').clearRect(0,0,360,360);
|
||||||
}
|
}
|
||||||
@@ -108,6 +109,7 @@ async function refresh() {
|
|||||||
try {const response=await api('/v1/overview');if(generation!==session)return;overview=response;render();$('updated').textContent=`Aktualisiert ${new Date().toLocaleTimeString('de-DE')}`;
|
try {const response=await api('/v1/overview');if(generation!==session)return;overview=response;render();$('updated').textContent=`Aktualisiert ${new Date().toLocaleTimeString('de-DE')}`;
|
||||||
error(overview.read_errors?'Einige Akten konnten nicht gelesen werden. Die Verbrauchsdaten können unvollständig sein.':'');
|
error(overview.read_errors?'Einige Akten konnten nicht gelesen werden. Die Verbrauchsdaten können unvollständig sein.':'');
|
||||||
if(typeof refreshOperations==='function')await refreshOperations();
|
if(typeof refreshOperations==='function')await refreshOperations();
|
||||||
|
if(typeof refreshTests==='function')await refreshTests();
|
||||||
} catch(e) {if(generation===session)error(`Aktualisierung fehlgeschlagen: ${e.message}`);}
|
} catch(e) {if(generation===session)error(`Aktualisierung fehlgeschlagen: ${e.message}`);}
|
||||||
if(token && generation===session)timer=setTimeout(refresh,4000);
|
if(token && generation===session)timer=setTimeout(refresh,4000);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,10 +3,10 @@
|
|||||||
<head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
|
<head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
<title>BugAnalysis · Übersicht</title><link rel="stylesheet" href="/assets/dashboard.css">
|
<title>BugAnalysis · Übersicht</title><link rel="stylesheet" href="/assets/dashboard.css">
|
||||||
<script src="/assets/dashboard.js" defer></script>
|
<script src="/assets/dashboard.js" defer></script>
|
||||||
<script src="/assets/case-detail.js" defer></script><script src="/assets/operations.js" defer></script></head>
|
<script src="/assets/case-detail.js" defer></script><script src="/assets/operations.js" defer></script><script src="/assets/test-requests.js" defer></script></head>
|
||||||
<body>
|
<body>
|
||||||
<aside class="rail"><a class="brand" href="/">V<span>BA</span><small>BugAnalysis</small></a>
|
<aside class="rail"><a class="brand" href="/">V<span>BA</span><small>BugAnalysis</small></a>
|
||||||
<nav aria-label="Navigation"><a href="#overview" class="selected">Übersicht</a><a href="#cases">Laufende Fälle</a><a href="#results">Ergebnisse</a><a href="#submit">Auftrag übergeben</a></nav>
|
<nav aria-label="Navigation"><a href="#overview" class="selected">Übersicht</a><a href="#cases">Laufende Fälle</a><a href="#results">Ergebnisse</a><a href="#tests">Testanfragen</a><a href="#submit">Auftrag übergeben</a></nav>
|
||||||
<div class="rail-bottom">ANALYSEBETRIEB<br><span id="connection">Nicht verbunden</span><button id="logout" type="button" hidden>Abmelden</button></div></aside>
|
<div class="rail-bottom">ANALYSEBETRIEB<br><span id="connection">Nicht verbunden</span><button id="logout" type="button" hidden>Abmelden</button></div></aside>
|
||||||
<main id="overview">
|
<main id="overview">
|
||||||
<header><div><div class="eyebrow">BETRIEBSÜBERSICHT</div><h1>Analyse im Blick.</h1><p>Aktuelle Arbeit und messbarer Verbrauch.</p></div><div class="live"><span class="dot"></span><span id="updated">Warte auf Verbindung</span></div></header>
|
<header><div><div class="eyebrow">BETRIEBSÜBERSICHT</div><h1>Analyse im Blick.</h1><p>Aktuelle Arbeit und messbarer Verbrauch.</p></div><div class="live"><span class="dot"></span><span id="updated">Warte auf Verbindung</span></div></header>
|
||||||
@@ -14,7 +14,7 @@
|
|||||||
<form id="login-form"><label for="token">API-Zugangsschlüssel</label><div class="inline"><input id="token" type="password" autocomplete="off" required placeholder="Zugangsschlüssel"><button type="submit">Verbinden</button></div></form></section>
|
<form id="login-form"><label for="token">API-Zugangsschlüssel</label><div class="inline"><input id="token" type="password" autocomplete="off" required placeholder="Zugangsschlüssel"><button type="submit">Verbinden</button></div></form></section>
|
||||||
<p id="error" class="error" role="alert" hidden></p>
|
<p id="error" class="error" role="alert" hidden></p>
|
||||||
<div id="content" hidden>
|
<div id="content" hidden>
|
||||||
<div id="overview-page">
|
<section class="panel" id="test-page" hidden><h2>Offene Testanfragen</h2><p class="hint">Beobachtungen ergänzen und die laufende Untersuchung fortsetzen.</p><div class="detail-controls"><label>Projekt<select id="test-project"></select></label><label>Sortierung<select id="test-sort"><option value="oldest">Älteste zuerst</option><option value="newest">Neueste zuerst</option><option value="project">Projekt, dann älteste zuerst</option></select></label></div><div id="test-list"></div><div id="test-answer"></div></section><div id="overview-page">
|
||||||
<section class="panel" id="operations"><h2>Betriebssteuerung</h2><div id="operation-status"></div><div id="operation-projects"></div><details id="operation-audit" hidden><summary>Aktionsprotokoll</summary><div id="operation-events"></div></details></section><section class="stats" aria-label="Kennzahlen">
|
<section class="panel" id="operations"><h2>Betriebssteuerung</h2><div id="operation-status"></div><div id="operation-projects"></div><details id="operation-audit" hidden><summary>Aktionsprotokoll</summary><div id="operation-events"></div></details></section><section class="stats" aria-label="Kennzahlen">
|
||||||
<article><span>Aktive Fälle</span><strong id="active-count">—</strong><small>Noch nicht abgeschlossen</small></article>
|
<article><span>Aktive Fälle</span><strong id="active-count">—</strong><small>Noch nicht abgeschlossen</small></article>
|
||||||
<article><span>Erfasste Tokens</span><strong id="token-count">—</strong><small>Eingabe + Ausgabe inkl. Denken</small></article>
|
<article><span>Erfasste Tokens</span><strong id="token-count">—</strong><small>Eingabe + Ausgabe inkl. Denken</small></article>
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
'use strict';
|
||||||
|
let testData=null,answerKey=null,answerBusy=false;
|
||||||
|
function clearTests(){testData=null;answerKey=null;['test-list','test-answer','test-project'].forEach(id=>$(id).replaceChildren());}
|
||||||
|
function testRoute(){
|
||||||
|
if(location.hash!=='#tests')return false;
|
||||||
|
['overview-page','results','case-detail','submit'].forEach(id=>$(id).hidden=true);$('test-page').hidden=false;
|
||||||
|
document.querySelector('header h1').textContent='Rückfragen beantworten.';
|
||||||
|
document.querySelectorAll('nav a').forEach(a=>a.classList.toggle('selected',a.hash==='#tests'));
|
||||||
|
if(token)refreshTests().catch(e=>error(e.message));return true;
|
||||||
|
}
|
||||||
|
async function refreshTests(){
|
||||||
|
const generation=session;const result=await api('/v1/test-requests');if(generation!==session)return;
|
||||||
|
testData=result;const filter=$('test-project'),selected=filter.value;
|
||||||
|
filter.replaceChildren(new Option('Alle Projekte',''));projects.forEach(p=>filter.add(new Option(p.project_id,p.project_id)));filter.value=selected;
|
||||||
|
renderTests();
|
||||||
|
}
|
||||||
|
function renderTests(){
|
||||||
|
if(!testData)return;
|
||||||
|
const project=$('test-project').value,sort=$('test-sort').value,root=$('test-list');root.replaceChildren();
|
||||||
|
if(testData.read_errors)root.append(text('p','Einige Anfragen konnten nicht gelesen werden.','error'));
|
||||||
|
const rows=testData.requests.filter(r=>!project||r.project_id===project).slice().sort((a,b)=>
|
||||||
|
(sort==='project'?a.project_id.localeCompare(b.project_id):0)||
|
||||||
|
(sort==='newest'?b.received_at.localeCompare(a.received_at):a.received_at.localeCompare(b.received_at))||a.key.localeCompare(b.key));
|
||||||
|
const wrap=text('div','','table-wrap'),table=document.createElement('table'),head=document.createElement('tr');
|
||||||
|
['Eingang','Projekt','Bug','Anfrage',''].forEach(v=>head.append(text('th',v)));const thead=document.createElement('thead');thead.append(head);table.append(thead);const body=document.createElement('tbody');
|
||||||
|
rows.forEach(row=>{const tr=document.createElement('tr');tr.append(text('td',new Date(row.received_at).toLocaleString('de-DE')),text('td',row.project_id));
|
||||||
|
const cell=document.createElement('td'),link=text('a',row.case_id,'case-link');link.href='#case/'+row.key;cell.append(link,text('small',row.title));tr.append(cell,text('td',row.request.question.slice(0,160)+(row.request.question.length>160?' …':'')));
|
||||||
|
const action=document.createElement('td'),button=text('button','Ansehen / beantworten');button.type='button';button.addEventListener('click',()=>openTest(row));action.append(button);tr.append(action);body.append(tr);});
|
||||||
|
table.append(body);wrap.append(table);root.append(wrap);if(!rows.length)root.append(text('p','Keine offenen Testanfragen für diese Auswahl.','hint'));
|
||||||
|
if(answerKey&&!testData.requests.some(r=>r.key===answerKey)){$('test-answer').replaceChildren(text('p','Diese Anfrage ist nicht mehr offen. Bitte die Liste aktualisieren.','hint'));answerKey=null;}
|
||||||
|
}
|
||||||
|
function openTest(row){
|
||||||
|
if(answerBusy)return;
|
||||||
|
answerKey=row.key;const root=$('test-answer');root.replaceChildren();const request=row.request;
|
||||||
|
root.append(text('h3',`${row.project_id} · ${row.case_id}`),text('p',request.question));
|
||||||
|
for(const [field,label] of [['environment','Angeforderte Umgebung'],['prerequisites','Voraussetzungen'],['steps','Testschritte'],['requested_observations','Benötigte Beobachtungen']])root.append(panel(label,semantic(request[field])));
|
||||||
|
root.append(text('p',`Angefragter Termin: ${new Date(request.deadline).toLocaleString('de-DE')}. Ein abgelaufener Termin ersetzt keine Antwort.`,'hint'));
|
||||||
|
if(!testData.can_answer){root.append(text('p','Zum Beantworten werden Tester- oder Betreiberrechte benötigt.','hint'));return;}
|
||||||
|
const form=document.createElement('form');form.className='semantic-list';
|
||||||
|
function field(label,element){const l=text('label',label);l.append(element);form.append(l);return element;}
|
||||||
|
const outcome=document.createElement('select');outcome.add(new Option('Test durchgeführt','completed'));outcome.add(new Option('Nicht durchführbar','unable_to_test'));field('Ergebnis',outcome);
|
||||||
|
const environment=field('Tatsächlich getestete Umgebung / Build',document.createElement('input'));environment.required=true;environment.maxLength=4000;
|
||||||
|
const observations=field('Beobachtungen bzw. Grund, warum der Test nicht möglich ist',document.createElement('textarea'));observations.required=true;observations.rows=7;observations.maxLength=12000;
|
||||||
|
const confirm=document.createElement('input');confirm.type='checkbox';confirm.required=true;field('Ich bestätige diese Angaben als meine Test-Rückmeldung.',confirm);
|
||||||
|
const button=text('button','Antwort bestätigen und Untersuchung fortsetzen');button.type='submit';form.append(button);root.append(form);
|
||||||
|
form.addEventListener('submit',async event=>{event.preventDefault();if(answerBusy)return;const generation=session;answerBusy=true;button.disabled=true;
|
||||||
|
try{await api('/v1/test-requests/'+row.key,{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({request_id:request.request_id,analysis_revision:request.analysis_revision,outcome:outcome.value,environment:environment.value,observations:[observations.value],confirmed:confirm.checked})});
|
||||||
|
if(generation!==session)return;root.replaceChildren(text('p','Antwort gespeichert. Der Worker kann die Untersuchung fortsetzen.'));answerKey=null;await refreshTests();
|
||||||
|
}catch(e){if(generation===session)error(`Antwort nicht gespeichert: ${e.message}`);}finally{answerBusy=false;if(generation===session)button.disabled=false;}});
|
||||||
|
root.scrollIntoView({behavior:'smooth',block:'start'});
|
||||||
|
}
|
||||||
|
$('test-project').addEventListener('change',()=>{answerKey=null;$('test-answer').replaceChildren();renderTests();});$('test-sort').addEventListener('change',renderTests);
|
||||||
|
if(location.hash==='#tests')testRoute();
|
||||||
@@ -49,6 +49,16 @@ try {
|
|||||||
await send('Emulation.setDeviceMetricsOverride',{width:390,height:1200,deviceScaleFactor:1,mobile:true});await pause(200);
|
await send('Emulation.setDeviceMetricsOverride',{width:390,height:1200,deviceScaleFactor:1,mobile:true});await pause(200);
|
||||||
if(await evaluate('document.documentElement.scrollWidth>innerWidth'))throw new Error('Mobile horizontal overflow');
|
if(await evaluate('document.documentElement.scrollWidth>innerWidth'))throw new Error('Mobile horizontal overflow');
|
||||||
await writeFile(path.join(output,'mobile.png'),Buffer.from((await send('Page.captureScreenshot',{format:'png'})).data,'base64'));
|
await writeFile(path.join(output,'mobile.png'),Buffer.from((await send('Page.captureScreenshot',{format:'png'})).data,'base64'));
|
||||||
|
await evaluate("location.hash='#tests'");
|
||||||
|
await until(()=>evaluate("!document.getElementById('test-page').hidden&&document.querySelector('#test-list button')"));
|
||||||
|
if(!await evaluate("document.getElementById('test-list').textContent.includes('viano-core')&&document.getElementById('test-list').textContent.includes('BUG-000020')"))throw new Error('Test request project/case missing');
|
||||||
|
const sorting=await evaluate("(()=>{const original=testData.requests;const row=original[0];testData.requests=[{...row,case_id:'NEW',received_at:'2098-01-01T00:00:00Z'},{...row,project_id:'aaa',case_id:'OLD',received_at:'2000-01-01T00:00:00Z'},row];const first=()=>document.querySelector('#test-list tbody tr').textContent;document.getElementById('test-sort').value='oldest';renderTests();const oldest=first().includes('OLD');document.getElementById('test-sort').value='newest';renderTests();const newest=first().includes('NEW');document.getElementById('test-sort').value='project';renderTests();const project=first().includes('aaa');document.getElementById('test-project').value='viano-core';renderTests();const filtered=document.querySelectorAll('#test-list tbody tr').length===2;testData.requests=original;document.getElementById('test-project').value='';renderTests();return oldest&&newest&&project&&filtered;})()");
|
||||||
|
if(!sorting)throw new Error('Test request sorting/filtering failed');
|
||||||
|
await evaluate("document.getElementById('test-sort').value='project';document.getElementById('test-sort').dispatchEvent(new Event('change'));document.querySelector('#test-list button').click()");
|
||||||
|
if(!await evaluate("document.querySelector('#test-answer textarea')&&document.getElementById('test-answer').textContent.includes('Sichtbarkeit und Build')"))throw new Error('Human test answer form missing');
|
||||||
|
if(await evaluate('document.documentElement.scrollWidth>innerWidth'))throw new Error('Test page mobile overflow');
|
||||||
|
await writeFile(path.join(output,'test-requests-mobile.png'),Buffer.from((await send('Page.captureScreenshot',{format:'png'})).data,'base64'));
|
||||||
|
await evaluate("location.hash='#overview'");await pause(100);
|
||||||
await evaluate("document.getElementById('project-filter').value='viano-core';document.getElementById('project-filter').dispatchEvent(new Event('change')); ");
|
await evaluate("document.getElementById('project-filter').value='viano-core';document.getElementById('project-filter').dispatchEvent(new Event('change')); ");
|
||||||
if(await evaluate("document.getElementById('token-count').textContent")!=='35.300')throw new Error('Project aggregation differs');
|
if(await evaluate("document.getElementById('token-count').textContent")!=='35.300')throw new Error('Project aggregation differs');
|
||||||
await evaluate("location.hash='#results'");
|
await evaluate("location.hash='#results'");
|
||||||
@@ -67,6 +77,7 @@ try {
|
|||||||
if(!await evaluate("document.getElementById('content').hidden&&!document.getElementById('login').hidden&&document.getElementById('token-count').textContent==='—'"))throw new Error('Logout retained visible case data');
|
if(!await evaluate("document.getElementById('content').hidden&&!document.getElementById('login').hidden&&document.getElementById('token-count').textContent==='—'"))throw new Error('Logout retained visible case data');
|
||||||
if(!await evaluate("document.getElementById('case-detail').childElementCount===0&&document.getElementById('result-rows').childElementCount===0"))throw new Error('Logout retained detail data');
|
if(!await evaluate("document.getElementById('case-detail').childElementCount===0&&document.getElementById('result-rows').childElementCount===0"))throw new Error('Logout retained detail data');
|
||||||
if(!await evaluate("document.getElementById('operation-events').childElementCount===0"))throw new Error('Logout retained audit');
|
if(!await evaluate("document.getElementById('operation-events').childElementCount===0"))throw new Error('Logout retained audit');
|
||||||
|
if(!await evaluate("document.getElementById('test-answer').childElementCount===0&&document.getElementById('test-list').childElementCount===0"))throw new Error('Logout retained test request data');
|
||||||
if(errors.filter(e=>!e.includes('404')).length)throw new Error(errors.join('\n'));
|
if(errors.filter(e=>!e.includes('404')).length)throw new Error(errors.join('\n'));
|
||||||
console.log(JSON.stringify({status:'PASS',...state,console_errors:errors.filter(e=>!e.includes('404'))}));
|
console.log(JSON.stringify({status:'PASS',...state,console_errors:errors.filter(e=>!e.includes('404'))}));
|
||||||
} finally {
|
} finally {
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
import pytest
|
||||||
|
from starlette.testclient import TestClient
|
||||||
|
from viano_bug_analysis.api import APIConfig,Client,Project
|
||||||
|
from viano_bug_analysis.api.app import create_app
|
||||||
|
from viano_bug_analysis.api.cases import case_key
|
||||||
|
from viano_bug_analysis.package import PackageReader
|
||||||
|
from test_async_tests import setup, hypotheses, case_setup, built_case, interpreted_case
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def project_manifest(manifest):
|
||||||
|
manifest['project']={'id':'p','type':'wow','profile':'vcore'}
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize('outcome',['completed','unable_to_test'])
|
||||||
|
def test_web_human_feedback_auth_scope_replay_and_resume(hypotheses,monkeypatch,outcome):
|
||||||
|
store,budget,coordinator,before,request,_=setup(hypotheses)
|
||||||
|
coordinator.submit(request)
|
||||||
|
monkeypatch.setenv('TESTER_TOKEN','tester_'+'x'*40)
|
||||||
|
monkeypatch.setenv('CAPTURE_TOKEN','capture_'+'x'*40)
|
||||||
|
monkeypatch.setenv('OTHER_TOKEN','other_'+'x'*40)
|
||||||
|
config=APIConfig(store.workspace.root,(Project('p','wow','vcore'),Project('other','wow','vcore')),
|
||||||
|
(Client('tester','TESTER_TOKEN',('p',),tester=True),Client('capture','CAPTURE_TOKEN',('p',)),Client('other','OTHER_TOKEN',('other',),tester=True)))
|
||||||
|
client=TestClient(create_app(config));auth={'Authorization':'Bearer tester_'+'x'*40}
|
||||||
|
url='/v1/test-requests';handle=case_key(*('case-A','stable-package'))
|
||||||
|
assert client.get(url).status_code==401
|
||||||
|
rows=client.get(url,headers=auth).json()
|
||||||
|
assert rows['can_answer'] and rows['requests'][0]['project_id']=='p'
|
||||||
|
answer={'request_id':request['request_id'],'analysis_revision':1,'outcome':outcome,
|
||||||
|
'observations':['Transition absent'],'environment':'build-A','confirmed':True}
|
||||||
|
assert client.post(url+'/'+handle,headers={'Authorization':'Bearer capture_'+'x'*40},json=answer).status_code==403
|
||||||
|
other={'Authorization':'Bearer other_'+'x'*40}
|
||||||
|
assert client.get(url,headers=other).json()['requests']==[]
|
||||||
|
assert client.post(url+'/'+handle,headers=other,json=answer).status_code==404
|
||||||
|
assert client.post(url+'/'+handle,headers=auth,json=answer|{'request_id':'wrong'}).status_code==409
|
||||||
|
assert client.post(url+'/'+handle,headers=auth,json=answer|{'confirmed':False}).status_code==422
|
||||||
|
response=client.post(url+'/'+handle,headers=auth,json=answer)
|
||||||
|
assert response.status_code==200 and response.json()['status']=='RESUMED'
|
||||||
|
assert client.post(url+'/'+handle,headers=auth,json=answer).json()['reused']
|
||||||
|
assert client.post(url+'/'+handle,headers=auth,json=answer|{'observations':['different']}).status_code==409
|
||||||
|
assert client.get(url,headers=auth).json()['requests']==[]
|
||||||
|
assert budget.snapshot('case-A','stable-package').state==before.state
|
||||||
|
with PackageReader(store.latest('case-A','stable-package')) as reader:
|
||||||
|
assert reader.manifest['state']=='INVESTIGATING_HYPOTHESES' and reader.manifest['analysis_revision']==1
|
||||||
|
|
||||||
|
|
||||||
|
def test_unavailable_test_executors_are_not_advertised():
|
||||||
|
from viano_bug_analysis.analysis.investigation_prompt import output_schema
|
||||||
|
def context(costs):
|
||||||
|
return {'tool_schema':{},'budget_policy':{'action_costs':costs}}
|
||||||
|
schema=output_schema(context({}),strict=True)
|
||||||
|
assert not any(b.get('properties',{}).get('kind',{}).get('const')=='test' for b in schema['properties']['next']['oneOf'])
|
||||||
|
schema=output_schema(context({'testing.request_human':40}),strict=True)
|
||||||
|
branch=next(b for b in schema['properties']['next']['oneOf'] if b['properties']['kind'].get('const')=='test')
|
||||||
|
assert branch['properties']['executor']['enum']==['human']
|
||||||
@@ -90,6 +90,17 @@ if __name__ == '__main__':
|
|||||||
root=Path(temporary)
|
root=Path(temporary)
|
||||||
identity=seed(root,'BUG-000020','Wolpertinger fehlen in Dun Morogh')
|
identity=seed(root,'BUG-000020','Wolpertinger fehlen in Dun Morogh')
|
||||||
seed(root,'BUG-000017','Leuchtbrille – Kontrolllauf',completed=True)
|
seed(root,'BUG-000017','Leuchtbrille – Kontrolllauf',completed=True)
|
||||||
|
# Display-only test request fixture; real feedback/budget binding is tested
|
||||||
|
# independently through the actual coordinator in test_human_web.py.
|
||||||
|
store=CheckpointStore(Workspace(root));ref='ui-test-request.json'
|
||||||
|
request={'case_id':identity[0],'package_id':identity[1],'analysis_revision':1,'request_id':'ui-test',
|
||||||
|
'executor':'human','hypothesis_id':'H1','question':'Ist die Kreatur nach Questannahme sichtbar?',
|
||||||
|
'environment':'Lokaler Testserver','prerequisites':['Quest noch nicht angenommen'],
|
||||||
|
'steps':['Quest annehmen','Umgebung beobachten'],'requested_observations':['Sichtbarkeit und Build notieren'],
|
||||||
|
'deadline':'2099-01-01T00:00:00Z'}
|
||||||
|
store.advance(*identity,'BLOCKED',step_id='ui-test-wait',reason='AWAITING_TEST_FEEDBACK',
|
||||||
|
retry_not_before=request['deadline'],artifacts={'async_test_request':ref},
|
||||||
|
details={'operation':'async-test-wait','request_ref':ref},additions={ref:json_bytes(request)})
|
||||||
config=APIConfig(root,(Project('viano-core','wow','vcore'),),
|
config=APIConfig(root,(Project('viano-core','wow','vcore'),),
|
||||||
(Client('preview','VBA_UI_TEST_TOKEN',('viano-core',),operator=True),))
|
(Client('preview','VBA_UI_TEST_TOKEN',('viano-core',),operator=True),))
|
||||||
os.environ['VBA_UI_TEST_TOKEN']=TOKEN
|
os.environ['VBA_UI_TEST_TOKEN']=TOKEN
|
||||||
|
|||||||
Reference in new issue
Block a user