# LLM failure diagnostics

Failed response processing now retains optional `outcome.metadata` and
`outcome.diagnostics` in the existing v1 `llm/operations/.../round-NNNN.json`
records. Old records without these fields remain valid and replay without API
calls. Previously discarded details cannot be recovered retrospectively.

For incomplete replies, diagnostics preserve:

- The API finish reason, restricted to known values; absent/unrecognized values
  are labeled `missing`/`unknown` rather than copying arbitrary provider text.
- Boolean refusal/tool-call indicators, not their text or arguments.
- Input/output tokens and reasoning tokens when supplied by the API.
- Configured output-token limit, received response bytes and output character count.

Metadata records the configured provider/model, prompt version, elapsed seconds,
known usage, previous unavailable-provider attempts, and estimated cost only when
usage and operator-configured prices are available. Missing/invalid token values
remain unknown, not zero. API output token usage may include reasoning, depending
on the provider; do not infer reasoning usage when the API omits that field.

JSON parsing errors retain available envelope details. Schema rejection retains
usage and a safe validation keyword, without storing response contents or the
validation exception message. Diagnostic fields have their own allowlisted JSON
schema (`llm-failure-diagnostics-v1.schema.json`). No headers, API keys, provider
error bodies, refusal text, tool arguments or partial completion text are added.

The same safe details are visible in structured service `llm_round` log events.
Known usage/cost from failed replies now contributes to existing telemetry
counters. The connection-test CLI includes diagnostic/metadata fields on errors.
Existing retry, failover, validation limits, lifecycle and hypothesis budgets are
unchanged. Successful reply metadata and response contracts remain unchanged.

A `finish_reason = length` indicates an output-budget stop reported by the API;
compare the output token count with `max_output_tokens`, and inspect reasoning
usage if available. `content_filter`, refusal/tool-call indicators, or missing
finish reasons describe different failure conditions. An incomplete response
alone does not prove a transport timeout. Transport/availability failures still
use their existing codes and HTTP attempt records; usage may be unavailable.

Tests use local HTTP fixtures and model fixtures. They cover truncation,
refusal/tool calls, malformed usage, unknown finish reasons, JSON/schema failures,
usage persistence, replay without network calls, telemetry counters/logs and
removal of free provider text. No paid model request is needed.