Use compact investigation working sets with durable local evidence recall
This commit is contained in:
Nico committed
1 parent
e8d0602ed3
commit
d81f2f67f3
21 files changed
+1614
-29
No files matched your search
@@ -0,0 +1,125 @@
|
||||
# Kompakte Hypothesenuntersuchung v15
|
||||
|
||||
## Scope
|
||||
|
||||
Nur die Projekt-Investigation verwendet neu investigation_project_v15. CaseBuilder,
|
||||
Interpretation, Hypothesengenerierung und Diagnose/Ãœbergaben bleiben fachlich
|
||||
unverändert. Der kanonische Kontext und sämtliche alten Recherchepayloads bleiben
|
||||
in der transportablen Case-Akte erhalten. Es gibt keine zusätzliche LLM-
|
||||
Zusammenfassung, kein Euro-Abbruchlimit und keine projektspezifische Bugheuristik.
|
||||
|
||||
## Arbeitskontext
|
||||
|
||||
Die LLM erhält die originale Interpretation mit Beobachtungen, Sollbeschreibung,
|
||||
technischen Fakten und Einschränkungen; die ausgewählte Hypothese mit offenen
|
||||
Fragen und geschlossenen Frage-IDs; alternative Mechanismen; den aktuellen
|
||||
kumulativen Arbeitsstand und die tatsächlichen Befunde anderer Hypothesen.
|
||||
Deren ursprüngliche Planungsfragen und Verwaltungsdaten werden nicht wiederholt.
|
||||
Der Arbeitsstand ist ausdrücklich Modellbewertung, kein neuer unabhängiger Beleg.
|
||||
Er wird in derselben Entscheidung fortgeschrieben. Text, Scope, Richtung und
|
||||
Beleg-IDs bleiben nachvollziehbar; Entfernen/Umwerten erfordert weiterhin Gründe.
|
||||
|
||||
Das neue eigene Ergebnis und noch abzurechnende eigene Ergebnisse werden vollständig
|
||||
übertragen. Eine konfigurierte kleine Anzahl letzter eigener Rohresultate bleibt
|
||||
sichtbar. Andere Hypothesen bewerten ihre noch offenen Ergebnisse auf ihrem eigenen
|
||||
Turn; deren Queryinventar bleibt sichtbar und kann lokal gelesen werden.
|
||||
Gegenbefunde bleiben im eigenen und fremden Arbeitsstand ausdrücklich vorhanden;
|
||||
ihre alten Rohdaten werden nicht endlos automatisch erneut angehängt. Solange für
|
||||
einen alten Stand kein Arbeitsbrief existiert, wird einmal vollständig gebootstrapped;
|
||||
nicht in einem Brief repräsentierte alte Gegenbelege bleiben dabei direkt lesbar.
|
||||
|
||||
Bekannte Schemas bleiben als Spaltennamen-/Typkatalog erhalten. Frische oder gezielt
|
||||
gelesene Schemas werden vollständig übergeben, einschließlich Schlüsseln und
|
||||
Nullbarkeit. Für weggelassene Metadaten ist die originale Schemaantwort lokal
|
||||
lesbar. Schemaherkunft, Abdeckung und Kürzung bleiben sichtbar. Unbekannte
|
||||
Providerschemaformen werden vollständig beibehalten statt falsch umgedeutet.
|
||||
|
||||
Neue Datenresultate werden nur strukturell komprimiert: gemeinsame Werte und
|
||||
Feldnamen einmal, sämtliche Zeilen, skalare Typen und Duplikate bleiben erhalten.
|
||||
Source-Ergebnisse behalten Pfad, Revision, Zeilen und Text sowie Abdeckungsgrenzen.
|
||||
Operator-Tabellenallowlists werden nicht in jedem Resultat erneut als Provenienz
|
||||
mitgeschickt. Duplikatbelege werden weiterhin als nicht unabhängige Beobachtungen
|
||||
markiert. Paketquellen verwenden deterministische PS-/PD-Aliase; der Controller
|
||||
bindet diese an den unveränderten kanonischen Kontext.
|
||||
|
||||
## Lokales Nachlesen
|
||||
|
||||
recall liest bis zu vier bekannte Evidence-IDs aus der Case-Akte, ohne eine neue
|
||||
DB-/Git-Abfrage oder einen neuen Reward zu erzeugen. Gelesene Einträge bleiben
|
||||
über folgende Turns gepinnt; die letzten vier explizit gewählten Einträge werden
|
||||
behalten. Bereits vollständig sichtbare Belege erneut anzufordern wird kontrolliert
|
||||
abgelehnt. Ein bislang ungelesener Archivbeleg darf nicht allein aufgrund seiner
|
||||
Inventarnennung als neuer Beleg eingeführt werden. Bestehende Arbeitsbefunde und
|
||||
sichtbare Schemakataloge bleiben referenzierbar. Unbekannte IDs bleiben ungültig.
|
||||
|
||||
Der neue Aufruf besitzt einen kleinen JSON-Drahtvertrag plus dokumentierte
|
||||
Argumentformen. Vor jeglicher Budget-/Toolwirkung wird erneut der vollständige
|
||||
bisherige Vertrag geprüft: Pflichtfelder, Argumente, Textlängen, Referenzen und
|
||||
Grenzen. Dies gilt auch für eine nach Absturz gespeicherte ausführbare Antwort.
|
||||
Rohantwort und abgeleiteter Arbeitsstand bleiben getrennte unveränderliche Artefakte.
|
||||
Gespeicherte v14-Anfragen behalten ihre eingefrorene Darstellung und Normalisierung.
|
||||
|
||||
Finish, Reject, asynchrone Tests, Defer und RequestDiagnosis bleiben erhalten.
|
||||
Der Frühabschluss benötigt weiterhin einen Gegencheck und eine unabhängige finale
|
||||
Diagnoseentscheidung; verbleibendes Budget erzwingt keine weitere Forschung.
|
||||
Lokale Leseschritte zählen weiterhin zu den begrenzten Iterationen.
|
||||
|
||||
## Konfiguration und Messung
|
||||
|
||||
Zusätzliche optionale [investigation]-Werte:
|
||||
|
||||
```toml
|
||||
prompt_target_characters = 13125
|
||||
recent_evidence_count = 1
|
||||
```
|
||||
|
||||
Das Zeichenziel ist ausschließlich ein Richtwert, kein Paketformatlimit und keine
|
||||
Abschneide-/Abbruchgrenze. Pflichtresultate, Bootstrap, viele Gegenbefunde oder
|
||||
bekannte Schemas dürfen darüber liegen. Die geplante logische Nachrichtengröße und
|
||||
above_target werden im bestehenden Service-Log erfasst. Die tatsächlichen
|
||||
Provider-Tokens bleiben die maßgebliche Messung.
|
||||
|
||||
Die neue Ansicht bleibt mit dem stabilen Fallcache kompatibel. Für v15 darf die
|
||||
Cacheerzeugung trotz alter Zeichenschwelle versucht werden; tatsächliche
|
||||
Anbietergrenzen entscheiden. Google nennt für Gemini 3.8 Flash mindestens 4096
|
||||
Tokens: https://ai.google.dev/gemini-api/docs/caching . Es werden keine Fülltokens
|
||||
hinzugefügt. Explizites cache_required bleibt strikt. Das alte lokale Pflichtprofil
|
||||
wurde nicht verändert. Das neue lokale Profil config/llm.compact-investigation.local.toml
|
||||
übernimmt dessen Modell-/Denkeinstellungen und aktiviert weiter Caching, setzt
|
||||
aber cache_required=false und entfernt den Pflichtpräfix. Ein vom Anbieter
|
||||
abgelehnter Cache verhindert so keine kleine Untersuchung. Keine Secrets im Report.
|
||||
|
||||
## Offline-Auswertung
|
||||
|
||||
Die sechs erfolgreichen manuellen Proben verbrauchten zusammen 29394 Eingabe-,
|
||||
1274 Antwort- und 16181 Denk-Tokens, bei 127,360 Sekunden API-Laufzeit. Sie zeigen
|
||||
plausible einzelne Entscheidungen, aber noch keine vollständige v15-Regression.
|
||||
Korrigierte Kontofolge: 8,34 vor 8,35, anschließend 8,38 EUR. Verzögerte Buchungen
|
||||
werden nicht willkürlich auf einzelne Aufrufe verteilt.
|
||||
|
||||
Projection gespeicherter Arbeitskontexte, ohne API-Aufrufe oder umgeschriebene
|
||||
alte Modellbefunde, gemessen als tatsächliche Gemini-Nachrichtentextzeichen:
|
||||
|
||||
| Archiv | Kontexte | Median Reduktion | Median v15 |
|
||||
|---|---:|---:|---:|
|
||||
| Bug 17 Arbeitsbrief | 18 | ca. 73.5 % | ca. 21824 Zeichen |
|
||||
| Bug 20 v14 | 10 | ca. 72.2 % | ca. 21410 Zeichen |
|
||||
|
||||
Der Richtwert ist in diesen alten Kontextprojektionen noch nicht erreicht. Die
|
||||
Altanfragen enthalten insbesondere große vollständige Ergebnisabfragen und lange
|
||||
bereits formulierte Arbeitsbefunde. Sie werden nicht zur Erfüllung einer Zahl
|
||||
abgeschnitten. Eingabe-Token-/Kostenreduktion und Qualität müssen mit einem neuen
|
||||
Gesamtlauf geprüft werden; die Offline-Zahlen sind keine gemessene Tokenersparnis.
|
||||
Das Zehntel-Ziel bleibt ausdrücklich offen. Vorbereitung startet keine LLM-Aufrufe.
|
||||
|
||||
Reproduzierbares Messwerkzeug: scripts/measure_compact_investigation.py.
|
||||
Messdateien: measurements/investigation-v15-offline-20261017.json und
|
||||
measurements/compact-investigation-six-probes-20261017.json.
|
||||
|
||||
Validierung: 375 relevante Tests bestanden, breite Regression plus gezielte
|
||||
Nachläufe der angepassten v14-Testfixtures und neuen v15-Frühabschlussfälle.
|
||||
Case-/Budget-/Replay-Prüfungen, lokale Beleglesung, vollständige frische Payloads
|
||||
und die unveränderte unabhängige Diagnose sind abgedeckt. Keine Live-API-Abfragen
|
||||
bei der Implementierung. Das Speichergesamtbudget einschließlich temporärer
|
||||
pytest-Kopien bleibt nach Archivierung des älteren Stable-Cache-Vergleichslaufs
|
||||
unter 20 GB. Neuere Bug-17-/Bug-20-Vergleiche bleiben erhalten.
|
||||
@@ -0,0 +1,104 @@
|
||||
{
|
||||
"measurement": "compact_investigation_six_probes_20261017",
|
||||
"calls": [
|
||||
{
|
||||
"file": "result-jsonmode-1.json",
|
||||
"id": "compact_investigation_probe_v1",
|
||||
"status": "SUCCESS",
|
||||
"characters": 10925,
|
||||
"usage": {
|
||||
"input_tokens": 2872,
|
||||
"candidate_output_tokens": 82,
|
||||
"reasoning_tokens": 989,
|
||||
"cached_input_tokens": null,
|
||||
"total_tokens": 3943
|
||||
},
|
||||
"elapsed_seconds": 9.219
|
||||
},
|
||||
{
|
||||
"file": "result-step2.json",
|
||||
"id": "compact_investigation_probe_step2_v1",
|
||||
"status": "SUCCESS",
|
||||
"characters": 13289,
|
||||
"usage": {
|
||||
"input_tokens": 3649,
|
||||
"candidate_output_tokens": 142,
|
||||
"reasoning_tokens": 750,
|
||||
"cached_input_tokens": null,
|
||||
"total_tokens": 4541
|
||||
},
|
||||
"elapsed_seconds": 7.078
|
||||
},
|
||||
{
|
||||
"file": "result-step3.json",
|
||||
"id": "compact_investigation_probe_step3_v1",
|
||||
"status": "SUCCESS",
|
||||
"characters": 15895,
|
||||
"usage": {
|
||||
"input_tokens": 4874,
|
||||
"candidate_output_tokens": 145,
|
||||
"reasoning_tokens": 1987,
|
||||
"cached_input_tokens": null,
|
||||
"total_tokens": 7006
|
||||
},
|
||||
"elapsed_seconds": 16.125
|
||||
},
|
||||
{
|
||||
"file": "result-step4.json",
|
||||
"id": "compact_investigation_probe_step4_v1",
|
||||
"status": "SUCCESS",
|
||||
"characters": 17289,
|
||||
"usage": {
|
||||
"input_tokens": 5316,
|
||||
"candidate_output_tokens": 398,
|
||||
"reasoning_tokens": 2510,
|
||||
"cached_input_tokens": null,
|
||||
"total_tokens": 8224
|
||||
},
|
||||
"elapsed_seconds": 20.813
|
||||
},
|
||||
{
|
||||
"file": "result-step5.json",
|
||||
"id": "compact_investigation_probe_step5_v1",
|
||||
"status": "SUCCESS",
|
||||
"characters": 18782,
|
||||
"usage": {
|
||||
"input_tokens": 6171,
|
||||
"candidate_output_tokens": 300,
|
||||
"reasoning_tokens": 3728,
|
||||
"cached_input_tokens": null,
|
||||
"total_tokens": 10199
|
||||
},
|
||||
"elapsed_seconds": 28.187
|
||||
},
|
||||
{
|
||||
"file": "result-step6.json",
|
||||
"id": "compact_investigation_probe_step6_v1",
|
||||
"status": "SUCCESS",
|
||||
"characters": 19835,
|
||||
"usage": {
|
||||
"input_tokens": 6512,
|
||||
"candidate_output_tokens": 207,
|
||||
"reasoning_tokens": 6217,
|
||||
"cached_input_tokens": null,
|
||||
"total_tokens": 12936
|
||||
},
|
||||
"elapsed_seconds": 45.938
|
||||
}
|
||||
],
|
||||
"totals": {
|
||||
"input_tokens": 29394,
|
||||
"candidate_output_tokens": 1274,
|
||||
"reasoning_tokens": 16181,
|
||||
"total_tokens": 46849
|
||||
},
|
||||
"total_elapsed_seconds": 127.36,
|
||||
"account_history_eur": [
|
||||
"8.30",
|
||||
"8.31",
|
||||
"8.34",
|
||||
"8.35",
|
||||
"8.38"
|
||||
],
|
||||
"account_note": "User correction: 8.34 preceded 8.35; latest account total 8.38. Posting is delayed; deltas are not allocated to individual calls. Failed HTTP attempts have no token counters and are excluded from successful-call totals."
|
||||
}
|
||||
@@ -0,0 +1,503 @@
|
||||
{
|
||||
"measurement": "investigation_v15_offline_projection_v1",
|
||||
"notes": "Actual Gemini wire text characters, not measured tokens, cost, model output or proof of quality. Old stored model findings are reused without rewriting.",
|
||||
"results": [
|
||||
{
|
||||
"package": "C:\\Users\\User\\Documents\\Projekte\\VianoBugAnalysis\\VianoBugAnalysis\\service-data\\trials\\bug17-working-brief-20261016\\cases\\ee545dc4fba1e7d80fe28afa53898e02427d7114493f62d3eb771d670253282d\\64852e6db3f62a047bf987787fac2e4ef02c0534dc2d4da325ec63b999ad387f\\g00000114.zip",
|
||||
"samples": 18,
|
||||
"median_reduction_percent": 73.50999999999999,
|
||||
"median_v15_characters": 21824.0,
|
||||
"min_v15_characters": 15896,
|
||||
"max_v15_characters": 24952,
|
||||
"above_target_samples": 18,
|
||||
"rows": [
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 62737,
|
||||
"v15_characters": 15896,
|
||||
"reduction_percent": 74.66,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 0,
|
||||
"visible_raw_ids": [],
|
||||
"schema_ids": [],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 66087,
|
||||
"v15_characters": 19270,
|
||||
"reduction_percent": 70.84,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 1,
|
||||
"visible_raw_ids": [
|
||||
"E000001"
|
||||
],
|
||||
"schema_ids": [],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H3-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 69798,
|
||||
"v15_characters": 21575,
|
||||
"reduction_percent": 69.09,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 2,
|
||||
"visible_raw_ids": [
|
||||
"E000001",
|
||||
"E000002"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H3-0001-attempt-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 69841,
|
||||
"v15_characters": 21618,
|
||||
"reduction_percent": 69.05,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 2,
|
||||
"visible_raw_ids": [
|
||||
"E000001",
|
||||
"E000002"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0002.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 73940,
|
||||
"v15_characters": 21639,
|
||||
"reduction_percent": 70.73,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 3,
|
||||
"visible_raw_ids": [
|
||||
"E000001"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0003.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 77892,
|
||||
"v15_characters": 22460,
|
||||
"reduction_percent": 71.17,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 4,
|
||||
"visible_raw_ids": [
|
||||
"E000004"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0003-attempt-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 77944,
|
||||
"v15_characters": 22512,
|
||||
"reduction_percent": 71.12,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 4,
|
||||
"visible_raw_ids": [
|
||||
"E000004"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0004.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 80802,
|
||||
"v15_characters": 22221,
|
||||
"reduction_percent": 72.5,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 5,
|
||||
"visible_raw_ids": [
|
||||
"E000005"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0005.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 84432,
|
||||
"v15_characters": 23183,
|
||||
"reduction_percent": 72.54,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 6,
|
||||
"visible_raw_ids": [
|
||||
"E000006"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0002.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 87184,
|
||||
"v15_characters": 21192,
|
||||
"reduction_percent": 75.69,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 7,
|
||||
"visible_raw_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0002-attempt-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 87227,
|
||||
"v15_characters": 21235,
|
||||
"reduction_percent": 75.66,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 7,
|
||||
"visible_raw_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0002-attempt-0002.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 87239,
|
||||
"v15_characters": 21247,
|
||||
"reduction_percent": 75.65,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 7,
|
||||
"visible_raw_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0003.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 88697,
|
||||
"v15_characters": 21798,
|
||||
"reduction_percent": 75.42,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 8,
|
||||
"visible_raw_ids": [
|
||||
"E000008"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0003-attempt-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 88749,
|
||||
"v15_characters": 21850,
|
||||
"reduction_percent": 75.38,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 8,
|
||||
"visible_raw_ids": [
|
||||
"E000008"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0003-attempt-0002.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 88750,
|
||||
"v15_characters": 21851,
|
||||
"reduction_percent": 75.38,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 8,
|
||||
"visible_raw_ids": [
|
||||
"E000008"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0004.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 91020,
|
||||
"v15_characters": 23207,
|
||||
"reduction_percent": 74.5,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 9,
|
||||
"visible_raw_ids": [
|
||||
"E000009"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0005.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 94055,
|
||||
"v15_characters": 24897,
|
||||
"reduction_percent": 73.53,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 10,
|
||||
"visible_raw_ids": [
|
||||
"E000010"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0005-attempt-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v13",
|
||||
"baseline_characters": 94110,
|
||||
"v15_characters": 24952,
|
||||
"reduction_percent": 73.49,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 10,
|
||||
"visible_raw_ids": [
|
||||
"E000010"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"package": "C:\\Users\\User\\Documents\\Projekte\\VianoBugAnalysis\\VianoBugAnalysis\\service-data\\trials\\bug20-lean-reply-20261017\\cases\\897a914b67a02e6109a6b567e377e43c72b90ab216bfb9d2dc24485a5a55db8d\\a4c26dd6d602c9c6d0ac7964d029953916d7a47784cc23e56bcc62bd79325067\\g00000103.zip",
|
||||
"samples": 10,
|
||||
"median_reduction_percent": 72.22999999999999,
|
||||
"median_v15_characters": 21410.5,
|
||||
"min_v15_characters": 16460,
|
||||
"max_v15_characters": 25694,
|
||||
"above_target_samples": 10,
|
||||
"rows": [
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v14",
|
||||
"baseline_characters": 66329,
|
||||
"v15_characters": 16460,
|
||||
"reduction_percent": 75.18,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 0,
|
||||
"visible_raw_ids": [],
|
||||
"schema_ids": [],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v14",
|
||||
"baseline_characters": 70086,
|
||||
"v15_characters": 18498,
|
||||
"reduction_percent": 73.61,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 1,
|
||||
"visible_raw_ids": [
|
||||
"E000001"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000001"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H3-0001.json",
|
||||
"baseline_prompt_version": "investigation_project_v14",
|
||||
"baseline_characters": 72752,
|
||||
"v15_characters": 20641,
|
||||
"reduction_percent": 71.63,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 2,
|
||||
"visible_raw_ids": [
|
||||
"E000001",
|
||||
"E000002"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000001"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0002.json",
|
||||
"baseline_prompt_version": "investigation_project_v14",
|
||||
"baseline_characters": 76060,
|
||||
"v15_characters": 20983,
|
||||
"reduction_percent": 72.41,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 3,
|
||||
"visible_raw_ids": [
|
||||
"E000002"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000001"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0002.json",
|
||||
"baseline_prompt_version": "investigation_project_v14",
|
||||
"baseline_characters": 77025,
|
||||
"v15_characters": 21072,
|
||||
"reduction_percent": 72.64,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 4,
|
||||
"visible_raw_ids": [
|
||||
"E000001"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000001",
|
||||
"E000004"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0003.json",
|
||||
"baseline_prompt_version": "investigation_project_v14",
|
||||
"baseline_characters": 78758,
|
||||
"v15_characters": 21749,
|
||||
"reduction_percent": 72.39,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 5,
|
||||
"visible_raw_ids": [
|
||||
"E000005"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000001",
|
||||
"E000004"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0004.json",
|
||||
"baseline_prompt_version": "investigation_project_v14",
|
||||
"baseline_characters": 80164,
|
||||
"v15_characters": 22392,
|
||||
"reduction_percent": 72.07,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 6,
|
||||
"visible_raw_ids": [
|
||||
"E000006"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000001",
|
||||
"E000004"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H1-0005.json",
|
||||
"baseline_prompt_version": "investigation_project_v14",
|
||||
"baseline_characters": 81606,
|
||||
"v15_characters": 23085,
|
||||
"reduction_percent": 71.71,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 7,
|
||||
"visible_raw_ids": [
|
||||
"E000007"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000001",
|
||||
"E000004"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H2-0003.json",
|
||||
"baseline_prompt_version": "investigation_project_v14",
|
||||
"baseline_characters": 81828,
|
||||
"v15_characters": 24308,
|
||||
"reduction_percent": 70.29,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 7,
|
||||
"visible_raw_ids": [
|
||||
"E000004"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000001",
|
||||
"E000004"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
},
|
||||
{
|
||||
"context": "analysis/investigation/0001/contexts/H3-0002.json",
|
||||
"baseline_prompt_version": "investigation_project_v14",
|
||||
"baseline_characters": 81478,
|
||||
"v15_characters": 25694,
|
||||
"reduction_percent": 68.47,
|
||||
"target_characters": 13125,
|
||||
"above_target": true,
|
||||
"evidence_count": 7,
|
||||
"visible_raw_ids": [
|
||||
"E000003"
|
||||
],
|
||||
"schema_ids": [
|
||||
"E000001",
|
||||
"E000004"
|
||||
],
|
||||
"archive_ids_count": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in new issue
Block a user