Contextual entity resolution

Entity resolution belongs to Case Interpretation, the first semantic phase. The deterministic CaseBuilder and its completed case.json remain unchanged. This step identifies entities needed to understand a report; it does not inspect spawn availability, generate hypotheses, diagnose a cause or propose changes.

Sequence

A versioned model prompt proposes grounded name searches, with contextual roles and references to readable case sources. Search terms must occur in the cited text; invented names, unknown catalogs and unread references fail validation. The model cannot choose tables, columns, credentials or endpoints.
The controller validates the entire plan. The existing read-only project research service executes parameterized, literal contains name queries against operator-configured catalogs. Each result is captured in the package, including its exact query, database snapshot binding and coverage limits.
A separate model assessment compares the returned candidates with the original context. It returns selected, ambiguous or no_match for each search. A selection must refer to an actual candidate from that search and cite both its capture and the original mention. Truncated searches cannot produce a unique selection. Similar names alone are insufficient.
The controller adds captured lookup facts and selected entities to the frozen interpretation context. The final interpretation uses the resolved prompt. Selected entities must be retained with mention citations; unresolved identities must be retained as unknowns. These constraints participate in the existing interpretation correction loop.

Normally this adds two model calls before the final interpretation. It is a quality feature, not a token reduction. Invalid replies and provider retries can add calls, within the persisted policies. No API request is needed to replay completed resolution and interpretation artifacts.

Configuration

config/project.example.toml demonstrates the optional [entity_resolution] block. It is disabled when absent. The current Vcore example enables creature name and localized creature name catalogs; the core mechanism is independent of WoW and supports other entity types and database tables.

max_searches: maximum proposed searches (default 4).
max_candidates: maximum rows per search (default 8).
max_invalid_responses: maximum invalid replies per semantic stage (default 3).
catalogs: operator-selected entity type, table, identity column, name column and returned data columns. Tables must also be in the project DB allowlist.

These are configurable operational safety defaults, not package format limits. No arbitrary SQL, regex, joins, model-supplied table names, network browsing, runtime spawn counts or unlimited research loop is exposed here. Additional catalogs can cover localized names or other entity domains. A miss is scoped to the chosen catalog, name filter and configured snapshot; it never proves that an entity is absent from the runtime world.

Provenance and lifecycle

Artifacts live below analysis/interpretations/NNNN/entity-resolution/: frozen input and policy, accepted plan, content-addressed query captures, candidate index, assessment, and attempt-specific rejected response records. The usual LLM artifacts retain provider, model, token counts, timing and validated requests/responses for both semantic stages. Every accepted stage is checkpointed.

Existing canonical source aliases stay unchanged. New F aliases expose captured technical lookup results; new N aliases expose selected entities with exact capture row provenance, mention references and an assessment reference. New D aliases expose unresolved identities. Their /entity_resolution/ pointers refer to these separate artifacts, not invented fields inside the canonical case.

A DB-backed ID/name is a technical fact. Its assigned role is explicitly labeled model_contextual_selection_not_causation: semantic association is not a proof of runtime behavior or of a defect. An existing quest giver remains relevant and can still be a contributor; it is not equated with the missing target creature. Not seeing a creature is not automatically proof of missing spawn records.

Restart reuses accepted plans, captures and assessments. A database outage blocks the case as RESEARCH_UNAVAILABLE without losing the plan. Invalid response exhaustion blocks as ENTITY_RESOLUTION_INVALID_RESPONSES. Neither changes hypothesis budgets or creates hypotheses. Blocked cases do not hold a worker; resumption is explicit, after the recorded retry timestamp (60-second safety delay). Exhausted response policies require review and a new analysis revision. Changing catalogs or snapshot bindings during an unfinished stage is rejected. Completed results can be verified in a different workspace without a live DB.

Existing completed/started interpretations are not silently enriched or rewritten. They retain their original prompts and contexts; use a fresh case or a new analysis revision to test this feature. --stop-after interpretation includes resolution and still prevents hypothesis generation and investigation.

Tests

Offline tests cover unobserved affected subjects, competing candidates, unknowns, grounding, limits, invalid response corrections/exhaustion, scoped empty results, DB outages, provider retries, changed bindings, append-only preservation, portable replay and the trial phase stop.

Opt-in tests use VBA_TEST_PROJECT_CONFIG for the local SELECT-only snapshot. With VBA_TEST_CASE_PACKAGE pointing to a built but not yet interpreted package, an additional end-to-end test uses the real case and database, with fixture model replies. It proves query/provenance/lifecycle integration, not the quality of a live model's semantic decisions. For the known Wolpertinger example, the fixture proposes the name found in the quest text; the real DB supplies the ID. The implementation contains no case-specific Creature-ID or diagnosis.

Prepared development trial

A fresh copy of BUG-000020 is prepared in service-data/trials/entity-resolution-20261015, package ID trial-entity-resolution-BUG-000020-20261015. Only import, admission and deterministic case construction have run (generation 4, no model calls).

Run from the repository root in the PowerShell session containing the API key:

.\.venv\Scripts\python.exe -m viano_bug_analysis.trial run --workspace service-data/trials/entity-resolution-20261015 --llm-config config/llm.timeout300.local.toml --project-config config/project.local.toml --case-id BUG-000020 --package-id trial-entity-resolution-BUG-000020-20261015 --stop-after interpretation --max-dispatches 6

This makes billable model requests but stops before hypotheses/investigation. If a provider retry is pending, repeat the same command after its timestamp. After completion, inspect the resolution plan, captured candidates, assessment and final interpretation together. Success means the model itself proposes a context-grounded search, distinguishes Goldark from the affected creature, and retains the selected identity and its provenance. A fixture test alone does not establish this live-model quality.

Schema visibility fix (prompt protocol v2)

New resolution runs freeze protocol_version = 2 in their input artifact. Both planning and assessment include the output schema in the user message, so providers configured with response_format = "json_object" can see the required fields. That provider mode only requests JSON; it does not transmit the internal request schema. Protocol v1 omitted that message field. Existing v1 artifacts retain their original request construction and prompts for immutable replay.

The first live entity-resolution trial exhausted planning validation: two LLM_OUTPUT_SCHEMA results and one LLM_INCOMPLETE_OUTPUT. No lookup was executed. The error records do not retain rejected response bodies or the exact incomplete finish reason, so they cannot establish each response's detailed cause. Do not retry that exhausted case unchanged; use the fresh trial below.

A corrected trial is prepared at service-data/trials/entity-resolution-schema-20261015 with package ID trial-entity-resolution-schema-BUG-000020-20261015:

.\.venv\Scripts\python.exe -m viano_bug_analysis.trial run --workspace service-data/trials/entity-resolution-schema-20261015 --llm-config config/llm.timeout300.local.toml --project-config config/project.local.toml --case-id BUG-000020 --package-id trial-entity-resolution-schema-BUG-000020-20261015 --stop-after interpretation --max-dispatches 6

8.6 KiB Raw Blame History