VianoBugAnalysis/docs/entity-resolution-v1.md

Contextual entity resolution

Entity resolution belongs to Case Interpretation, the first semantic phase. The deterministic CaseBuilder and its completed case.json remain unchanged. This step identifies entities needed to understand a report; it does not inspect spawn availability, generate hypotheses, diagnose a cause or propose changes.

Sequence

1. A versioned model prompt proposes grounded name searches, with contextual roles and references to readable case sources. Search terms must occur in the cited text; invented names, unknown catalogs and unread references fail validation. The model cannot choose tables, columns, credentials or endpoints.
2. The controller validates the entire plan. The existing read-only project research service executes parameterized, literal contains name queries against operator-configured catalogs. Each result is captured in the package, including its exact query, database snapshot binding and coverage limits.
3. A separate model assessment compares the returned candidates with the original context. It returns selected, ambiguous or no_match for each search. A selection must refer to an actual candidate from that search and cite both its capture and the original mention. Truncated searches cannot produce a unique selection. Similar names alone are insufficient.
4. The controller adds captured lookup facts and selected entities to the frozen interpretation context. The final interpretation uses the resolved prompt. Selected entities must be retained with mention citations; unresolved identities must be retained as unknowns. These constraints participate in the existing interpretation correction loop.

Normally this adds two model calls before the final interpretation. It is a quality feature, not a token reduction. Invalid replies and provider retries can add calls, within the persisted policies. No API request is needed to replay completed resolution and interpretation artifacts.

Configuration

config/project.example.toml demonstrates the optional [entity_resolution] block. It is disabled when absent. The current Vcore example enables creature name and localized creature name catalogs; the core mechanism is independent of WoW and supports other entity types and database tables.

• max_searches: maximum proposed searches (default 4).
• max_candidates: maximum rows per search (default 8).
• max_invalid_responses: maximum invalid replies per semantic stage (default 3).
• catalogs: operator-selected entity type, table, identity column, name column and returned data columns. Tables must also be in the project DB allowlist.

These are configurable operational safety defaults, not package format limits. No arbitrary SQL, regex, joins, model-supplied table names, network browsing, runtime spawn counts or unlimited research loop is exposed here. Additional catalogs can cover localized names or other entity domains. A miss is scoped to the chosen catalog, name filter and configured snapshot; it never proves that an entity is absent from the runtime world.

Provenance and lifecycle

Artifacts live below analysis/interpretations/NNNN/entity-resolution/: frozen input and policy, accepted plan, content-addressed query captures, candidate index, assessment, and attempt-specific rejected response records. The usual LLM artifacts retain provider, model, token counts, timing and validated requests/responses for both semantic stages. Every accepted stage is checkpointed.

Existing canonical source aliases stay unchanged. New F aliases expose captured technical lookup results; new N aliases expose selected entities with exact capture row provenance, mention references and an assessment reference. New D aliases expose unresolved identities. Their /entity_resolution/ pointers refer to these separate artifacts, not invented fields inside the canonical case.

A DB-backed ID/name is a technical fact. Its assigned role is explicitly labeled model_contextual_selection_not_causation: semantic association is not a proof of runtime behavior or of a defect. An existing quest giver remains relevant and can still be a contributor; it is not equated with the missing target creature. Not seeing a creature is not automatically proof of missing spawn records.

Restart reuses accepted plans, captures and assessments. A database outage blocks the case as RESEARCH_UNAVAILABLE without losing the plan. Invalid response exhaustion blocks as ENTITY_RESOLUTION_INVALID_RESPONSES. Neither changes hypothesis budgets or creates hypotheses. Blocked cases do not hold a worker; resumption is explicit, after the recorded retry timestamp (60-second safety delay). Exhausted response policies require review and a new analysis revision. Changing catalogs or snapshot bindings during an unfinished stage is rejected. Completed results can be verified in a different workspace without a live DB.

Existing completed/started interpretations are not silently enriched or rewritten. They retain their original prompts and contexts; use a fresh case or a new analysis revision to test this feature. --stop-after interpretation includes resolution and still prevents hypothesis generation and investigation.

Tests

Offline tests cover unobserved affected subjects, competing candidates, unknowns, grounding, limits, invalid response corrections/exhaustion, scoped empty results, DB outages, provider retries, changed bindings, append-only preservation, portable replay and the trial phase stop.

Opt-in tests use VBA_TEST_PROJECT_CONFIG for the local SELECT-only snapshot. With VBA_TEST_CASE_PACKAGE pointing to a built but not yet interpreted package, an additional end-to-end test uses the real case and database, with fixture model replies. It proves query/provenance/lifecycle integration, not the quality of a live model's semantic decisions. For the known Wolpertinger example, the fixture proposes the name found in the quest text; the real DB supplies the ID. The implementation contains no case-specific Creature-ID or diagnosis.

Prepared development trial

A fresh copy of BUG-000020 is prepared in service-data/trials/entity-resolution-20261015, package ID trial-entity-resolution-BUG-000020-20261015. Only import, admission and deterministic case construction have run (generation 4, no model calls).

Run from the repository root in the PowerShell session containing the API key:

.\.venv\Scripts\python.exe -m viano_bug_analysis.trial run --workspace service-data/trials/entity-resolution-20261015 --llm-config config/llm.timeout300.local.toml --project-config config/project.local.toml --case-id BUG-000020 --package-id trial-entity-resolution-BUG-000020-20261015 --stop-after interpretation --max-dispatches 6

This makes billable model requests but stops before hypotheses/investigation. If a provider retry is pending, repeat the same command after its timestamp. After completion, inspect the resolution plan, captured candidates, assessment and final interpretation together. Success means the model itself proposes a context-grounded search, distinguishes Goldark from the affected creature, and retains the selected identity and its provenance. A fixture test alone does not establish this live-model quality.

Schema visibility fix (prompt protocol v2)

New resolution runs freeze protocol_version = 2 in their input artifact. Both planning and assessment include the output schema in the user message, so providers configured with response_format = "json_object" can see the required fields. That provider mode only requests JSON; it does not transmit the internal request schema. Protocol v1 omitted that message field. Existing v1 artifacts retain their original request construction and prompts for immutable replay.

The first live entity-resolution trial exhausted planning validation: two LLM_OUTPUT_SCHEMA results and one LLM_INCOMPLETE_OUTPUT. No lookup was executed. The error records do not retain rejected response bodies or the exact incomplete finish reason, so they cannot establish each response's detailed cause. Do not retry that exhausted case unchanged; use the fresh trial below.

A corrected trial is prepared at service-data/trials/entity-resolution-schema-20261015 with package ID trial-entity-resolution-schema-BUG-000020-20261015:

.\.venv\Scripts\python.exe -m viano_bug_analysis.trial run --workspace service-data/trials/entity-resolution-schema-20261015 --llm-config config/llm.timeout300.local.toml --project-config config/project.local.toml --case-id BUG-000020 --package-id trial-entity-resolution-schema-BUG-000020-20261015 --stop-after interpretation --max-dispatches 6

Interpretation citation policy v2

New resolution input artifacts additionally freeze interpretation_protocol_version = 2. Their context uses linked_capture_v2 citation markers and case_interpretation_resolved_v2. Existing artifacts keep the original marker-free context, prompt, validation and feedback behavior.

The selected entity_id must still be present in relevant_entities. Its source_ids must include an original mention and either the entity alias or an inline entity_name_lookup fact backed by the same capture as that selected entity. A matching DB fact is sufficient identity evidence; a redundant N alias is not required. An unrelated fact cannot satisfy this requirement. Missing entities and missing provenance receive distinct error codes.

Correction feedback for the new prompt lists missing entities, the exact field, and permitted original-mention/capture aliases. It reports these even alongside source-kind errors, rather than waiting for multiple correction rounds to expose each issue. Tester observations and technical facts remain strictly separated. The prompt also distinguishes a scoped broad/localized lookup ambiguity from an independently selected, specifically named subject; the former does not by itself invalidate the latter.

The second real Gemini reply from the schema trial is an offline regression fixture. It is accepted with its original response and citations under the new policy; the old rule continues to reproduce the recorded historical rejection. This does not overwrite or accept artifacts in the exhausted REVIEW case.

Grouped identities and descriptive behavior (interpretation protocol v3)

Fresh resolution inputs freeze interpretation_protocol_version = 3. Historical v1/v2 inputs keep their original contexts, aliases, validation and prompts. The new final prompt is case_interpretation_resolved_v3; the two lookup prompts remain protocol v2. No additional model calls are introduced.

Selected entities are grouped by exact (type, id) within the configured project and lookup snapshot. A known canonical entity keeps its alias and original provenance. Each group keeps every selected name, optional locale, role, reason, mention citation and exact captured row in lookup_associations. Technical lookup facts and raw captures are not removed. Different IDs or entity types are never merged merely because names are similar. Any matching group capture can establish identity in the final interpretation's citations.

The new response contract case-interpretation-quality-output-v1.schema.json adds two explicitly labeled fields:

• described_behavior: statements derived from readable project_description facts, with basis = local_description and verification = not_independently_verified.
• provisional_behavior_gap: optional observation/description comparison, with basis = local_description, status = provisional, and citations to both an observation and a description actually included above.

These are separate from expected_behavior: that field continues to require formal expectation sources, whose provenance distinguishes tester declarations from external research. The original completeness/provenance fields and Q0001 are not promoted by a local description. behavior_gap and first_known_divergence still require formal expectation sources. Descriptions of prerequisites or a capture flow do not prove spawn behavior, correct runtime implementation, a defect, or an independently verified specification.

The artifact schema binds the new response shape to the new prompt version. Markdown summaries show the descriptive flow and provisional gap with explicit verification labels. Hypothesis contexts retain these fields and their provenance as separate interpretation source kinds. Empty descriptions and a null provisional gap are valid when no applicable description or comparison is supported.

Tests cover grouping/locale/provenance retention, canonical aliases, distinct IDs/types, valid alternative capture citations, a complete multi-query pipeline, formal/local source separation, two-sided provisional gaps, downstream context propagation, summaries, actual DB integration with fixture model replies, and historical protocol replay. No paid API call is required for these checks.

Capture citation correction feedback

For ENTITY_RESOLUTION_CAPTURE_SOURCE, the next assessment attempt receives an explicit list of missing V capture IDs, zero-based search indices, exact /resolutions/N/source_ids paths and original mention source IDs. A mention in reason is not a structured citation. The model must return a complete corrected assessment; validation remains unchanged and citations are never auto-inserted.

Feedback is frozen in an append-only feedback-assessment-NNNN.json checkpoint before the new call. Existing requests and error artifacts retain their original hashes. This also supports resuming older failed assessments without repeating accepted plans or database queries. Retry limits and budgets are unchanged.

Interpretation protocol 4: unbound entity labels

Fresh resolutions freeze interpretation_protocol_version=4. A separate context policy selects case_interpretation_resolved_v4; historical protocols remain byte-reconstructible and completed interpretations are never overwritten.

For an entity without a nonempty explicit name or named lookup_associations, the output schema fixes its relevance reason to a neutral type/ID description. The semantic validator enforces the same rule and correction feedback supplies the exact required reason. Descriptive nouns, numeric quest fields and empty script lookups cannot establish an ID/name binding. Numeric field relationships remain available as technical facts; human observations and descriptions remain unchanged. Named entities backed by resolution remain freely explainable.

This is deliberately conservative: even if another technical fact contains a name, it is not automatically associated with an unnamed entity merely by a matching number across potentially different identifier domains. A subsequent explicit typed name binding can remove this restriction.

The prompt applies the prohibition on inferred name/functional mappings to every output field. Deterministic enforcement is limited to relevant_entities.reason; it is not a general semantic verifier of all free text. No project-specific item IDs or presumed correct item names are hardcoded.

Interpretation protocol 5: typed names already present in project captures

An enrichment entity rule may declare name_column alongside its entity_type, table, ID column and selected columns. Both ID and name columns must be selected. A nonempty name in a returned row produces a project_entity_name fact with type, ID, exact name, capture reference and separate JSON pointers for ID and name. No additional database query is introduced. Unconfigured tables, empty results/names, and nearby prose produce no name binding.

Fresh interpretation protocol 5 joins these facts to entities by both type and ID, recording name_bindings with the fact alias and exact row provenance. It does not overwrite canonical names/roles or pick a winner among conflicting names. The schema and semantic validator require citing a bound name fact when including the associated entity. New correction feedback identifies permitted name evidence. A name binding establishes a display name, not a functional role, spawn presence or causal responsibility. Entities without bindings remain neutral. Protocols 1–4 reconstruct their historical contexts and prompts unchanged.

The example project maps creature names from creature_template.name and quest titles from quest_template.LogTitle. These project-specific choices live in configuration, not in core analysis code. Item name bindings remain absent until an explicit suitable data source and typed mapping are configured.

Entity resolution protocol 4: contextual identity guidance

Fresh inputs use plan/assessment prompt protocol 4; interpretation protocol 6 and its compact presentation remain unchanged. Prior protocols reconstruct their original requests. No hypothesis generation or investigation is introduced here.

Planning prioritizes specific affected-subject names in descriptive titles and body text, before broad/localized terms or outcome phrases. Existing typed project_entity_name bindings are listed as known names; contextual participants do not require another name query simply because their role may remain relevant. Exact-substring grounding, configured search limits and read-only query rules stay unchanged. No language-specific singularization or case-specific IDs are added.

Assessment receives literal candidate-name/description overlaps with source IDs. For a generic name contained in a longer matching name of another typed identity, the more specific candidate is listed separately. These are contextual hints, not automatic identity assignments. The prompt assesses all captures together, correlates locales by typed ID and preserves genuine ambiguity. Empty plural or outcome-name searches cannot negate a specific successful name lookup.

The per-index assessment schema requires its exact V capture citation AND an original mention in source_ids. Existing semantic validation still independently checks candidates, truncation and grounding. The historical capture-citation correction path remains tested for old packages. No extra LLM call or search is added. Live stability still requires a fresh real-model test.