CS Paper Radar

Beyond Symmetry: Asymmetric Matching with Transformers for Collective Entity Resolution

Andrei Olar

cs.DB ·submitted 2026-11-11 08:34 UTC · v1
benchmark

Abstract

A core assumption of entity resolution (ER) is its symmetric nature, which rests on a principle so self-evident it borders on tautology: if a refers to the same object as b, then b must also refer to the same object as a. However, this global property of symmetry does not necessarily extend to each of the complex decision processes within ER systems. In cases involving vague or ambiguous information, enforcing symmetry can be detrimental to performance, an issue exemplified by the ``bad triplet problem''. This paper investigates the effects of input-order asymmetry within a collective ER framework that integrates distinct matching and clustering stages. The input order-aware matching stage utilizes pre-trained transformers to explicitly model and exploit asymmetry, making it possible to evaluate its effect on clustering performance across multiple benchmark datasets. Our results demonstrate that asymmetry is not only measurable but also introduces a significant skew in the final entity clusters, highlighting the need to re-evaluate supervised training methodologies for matchers, particularly the development of direction-aware training paradigms that can preserve, rather than neutralize, the inherent asymmetry in matching tasks.

Closest papers

Nearest neighbours by meaning across the whole index.

Entity Matching using Decision Models

cs.CL cs.AI ·2026-10-25 ·Aaron Steiner, Ralph Peeters, Christian Bizer
benchmarkcode/data similarity 0.60
Abstract · 15 pages, 2 figures, 15 tables. Code, prompts, and results: https://github.com/wbsg-uni-mannheim/decision-models-4-entity-matching

Entity matching is the task of deciding whether two records describe the same real-world entity. Current entity matching methods rely on pre-trained language models (PLMs) which require task specific fine-tuning or generative large language models (LLMs) which do not require task-specific fine-tuning but are slow and expensive at matching large record sets. Decision models are a new class of LLM-based models that return a probability for each predefined answer option rather than generate text. This paper investigates decision models as an alternative to generative LLMs and fine-tuned PLMs for entity matching. We evaluate six decision models in a zero-shot setting across eight benchmarks and compare their performance with that of the generative LLM GPT-6 Luna and the PLM-based matcher Ditto. Our experiments show that, without task-specific training data, the four strongest decision models achieve mean F1 scores close to that of GPT-6 Luna, with the best model slightly surpassing it. This model also outperforms Ditto on six of the eight benchmarks. All decision models are faster than GPT-6 Luna, and most are less expensive. For three of the four strongest decision models, the predicted match probabilities are well calibrated without requiring any task-specific recalibration. A model cascade that routes the least confident record pairs to an ensemble of generative LLMs improves the mean F1 scores of the four strongest decision models by 1.9-4.0 percentage points. Using the best decision model and its best-performing prompt for each benchmark, the cascade surpasses the highest published F1 scores from literature on four of the eight benchmarks.

0.25

Better Retrieval, Limited Clustering Gains: A Controlled Study of Multilingual Company Entity Resolution

cs.AI ·2026-10-04 ·Yijiashun Qi, Yuxuan Li, Hanzhe Guo
published similarity 0.59
Abstract · accepted by ICEKIM 2026

Company name retrieval and final entity clustering can respond differently to encoder adaptation. We study this dependency through a controlled pipeline: train multilingual E5 encoders with matched positive examples and random or hard negatives, select checkpoints on development data, retrieve within equal country-specific edge budgets, and apply unchanged pair classifiers and guarded Union-Find clus- tering. Evaluation uses a subsequently collected GLEIF snapshot; a stratified, assisted human audit and a post-hoc temperature sweep examine label and parameter sensitivity. On 3,633 names with source-silver labels, adapted retrieval raises direct pair recall from 53.74% to 76.98%, whereas cluster recall moves from 32.54% to 33.33%. Most newly retrieved positives fail the existing classifier threshold. The study contributes a controlled comparison and pair-level diagnosis of this loss between retrieval and confirma-tion. It shows why evaluating a replacement retriever requires measuring the resulting entity partition, while leaving joint matcher retraining and independent gold-label evaluation for future work.

0.29

Reliability of LLM Judges for Evaluating Entity Alignment

cs.AI ·2026-10-07 ·Vaibhava Lakshmi Ravideshik, Mayank Kejriwal
top venuebenchmarkcode/data similarity 0.56
Abstract

Entity Alignment (EA) identifies equivalent entities across knowledge graphs and is critical for knowledge base integration and ontology merging. Evaluating EA systems at scale requires expensive expert annotation, making systematic assessment across diverse domains practically infeasible. LLM-as-judge evaluation offers a potentially scalable alternative, yet its reliability for structured prediction tasks like EA remains unstudied. We present the first systematic benchmarking study across three frontier models, three datasets, and four EA systems, using perturbation bias diagnostics, meta-evaluation across all dataset--judge--prompt combinations, and counterfactual label-flip tests. We identify anchor bias, a failure mode in which judges invert discrimination when the system's decision label is visible. Judge discrimination collapses under label-exposing prompts (J-ROC-AUC 0.12--0.87), while a label-free protocol recovers near-ceiling capability on distinctive-name datasets (0.98--1.00 for NeuSymEA) and significant recovery on biomedical pairs (0.93--0.95). Counterfactual label flips show that the visible label causally shifts scores by more than 2 points in 53--99% of pairs (FSR), with sensitivity growing with model capability, a pattern we call the frontier model paradox: stronger judges are more label-sensitive, not less, yet also better label-free predictors. The direction of the shifts is mixed, so we report this as an observed pattern. A blinded two-annotator human evaluation (102 pairs, Cohen's $κ$ = 0.902) provides a gold-label reference. We release the first biomedical EA benchmark (MeSH--SNOMED CT, 15K pairs) and a reproducible auditing framework for LLM judge reliability in EA. Code and data are available at https://github.com/vaibhavalakshmiravideshik/llm-as-a-judge-entity-alignment.

0.45

CaRL-EM: Cost-Aware Reinforcement Learning for Entity Matching with LLMs

cs.CL ·2026-09-01 ·Chaohui Guo, Michel Klein, Zhisheng Huang
top venue similarity 0.55
Abstract · Accepted to ACL 2026 Main Conference

Entity matching (EM) requires fine-grained contextual understanding and domain knowledge. Recent work shows that large language models (LLMs) can serve as strong matchers across domains, but most methods either make independent pairwise decisions or rely on manually designed composite pipelines, thus lacking flexibility in realistic multi-candidate settings. At the same time, they typically ignore inference cost at scale. We formulate LLM-based EM with candidates as a cost-aware sequential decision problem and propose CaRL-EM, a reinforcement learning controller that manages LLM operations. Given the state of an anchor record, its candidate set, and the cost, CaRL-EM adaptively chooses among different operators (Match/Compare/Select/Decide) and model capacities to maximize a quality-cost objective. The policy interacts with abstract operators, allowing the same controller to be reused with different underlying LLM backends at inference time without retraining. Experiments on 7 benchmarks show that CaRL-EM (i) learns to dynamically plan the usage of inexpensive and expensive operators based on task complexity, (ii) achieves robust zero-shot transfer across diverse datasets and domains, and (iii) consistently achieves a better quality-cost trade-off than strong LLM-based baselines and manually designed pipelines, yielding a lower inference cost at comparable or higher quality.

0.69

MERGED: Multimodal Entity Resolution via Generated Expert Reasoning Distillation

cs.IR ·2026-09-01 ·You-Lin Chen, Kyoungjun Park, Bin Xu, Prithviraj Sen, Abhishek Prasad Ram Tripathi +1 more
published similarity 0.55
Abstract · Accepted at the NeurIPS 2026 Workshop on Grounded and Faithful Vision-Language Models for Real-World Deployment (VLM4RWD). 11 pages, 3 figur…

In product entity resolution, relationship definitions constantly evolve with business needs, yet adapting to each change traditionally requires slow, costly human annotation that is often noisy and carries no reasoning. Large vision-language models (VLMs) prompted zero-shot can adapt to a new definition immediately and supply the reasoning that human labels lack, but their cost and latency are prohibitive at production scale. We present MERGED, a distillation framework that transfers not just labels but structured reasoning from large teacher VLMs into a compact 7B-parameter student, requiring no human annotation for training. Multiple teachers label each product pair and articulate the reasoning behind their decision: agreement pairs supply supervised fine-tuning, while disagreements are resolved by a meta-judge into preference pairs for Direct Preference Optimization. Evaluated against human-labeled ground truth on a multilingual e-commerce dataset, the resulting student improves PR-AUC by 13.79% over the same backbone trained on human labels and surpasses the larger Qwen2.5-32B-VL baseline by 6.32% at 6x lower cost, while also yielding tighter label-reasoning consistency (over 10% above Qwen2.5-32B-VL). Moreover, re-applying MERGED from an existing checkpoint adapts to a new relationship definition with only 10K samples, improving PR-AUC by 6.97% over zero-shot and outperforming from-scratch training. MERGED enables rapid adaptation to evolving relationship definitions, supporting a new one in days rather than months, at a cost and latency suitable for large-scale industrial deployment.

0.42

Curate Before You Connect: Identity and Ontology Tagging in a Production Knowledge Graph

cs.AI ·2026-08-11 ·Vaibhav Dangaich, Kevin Lewis, Kundeshwar Pundalik
benchmarkposition similarity 0.55
Abstract

Extraction produces candidate entities and relationships; writing them into a graph is where identity is decided, and identity decisions are destructive in a way extraction errors are not. A wrong type can be corrected later, but two records merged under one identity cannot be separated once their properties have been combined, and the merge leaves no error behind. This paper describes the ingestion and ontology-tagging layer that turns a validated extraction stream into a knowledge graph of 537,157 entities and 2,198,567 relationships drawn from 98,795 government documents. We describe a record-identity ladder that decides sameness from identifier columns, name columns, display names and type-scoped position rather than from name similarity. The ladder governs de-duplication within parsed tables, while the graph write applies a coarser canonical-name key, so records sharing a canonical name merge automatically on exact equality. We argue rather than demonstrate that this is where the automation line belongs: no identity benchmark is reported, and the over-merges the key permits are undetectable by construction. That policy, under which entity resolution only ever flags candidates, followed an incident in which two surface forms of one name were merged, corrupting a correct record and deleting eight entities from an unrelated document. We then describe multi-class ontology tagging and an evidence asymmetry we did not anticipate: an entity name is an instance label rather than a type assertion, so matching name fragments against a class index invents classifications. Requiring anchored evidence cut role assignments on an enriched sample from 36 to 4, all confirmed correct. We quantify the graph's conformance debt, show secondary classifications compensating for a mis-parented primary class, and describe a curation queue grown to 48,403 pending proposals against 775 human decisions.

0.04

Iris: Climbing to the Search Frontier

cs.AI ·2026-09-03 ·Ziyuan Liu, Hengqi Liu, Zichuan Wang, Yang Qin, Jiachen Liang +6 more
benchmarkcode/data similarity 0.55
Abstract · 12 pages, 2 figures

We present Iris-mini and Iris-pro, two search agents trained at the 35B-A3B and 397B-A17B scales, together with the data pipeline and training recipe behind them. Tasks are reverse-constructed from the hyperlink structure of a web corpus: we author multi-hop chains over an entity graph distilled from a seed page and its out-links, rewrite every non-answer entity into a descriptive reference so that no clue can be resolved by string matching, and admit only questions that a reference model fails closed-book yet solves once the supporting evidence is supplied. These questions are then turned into trajectories, which are filtered at both the trajectory and the turn level before SFT. The policy is then optimized by RL against live search, with the reward judge and the observation summarizer served inside the training cluster, and with over-long rollouts interrupted at the request level and resumed from their committed prefix at the next step. We alternate the two stages in a procedure we call SFT-RL climbing, returning the hardest solved and most efficient rollouts of each RL round to the next supervised pass. Because inference-time context management is worth more on these benchmarks than most reported differences between systems, we evaluate every benchmark both with and without it, holding the tool set, the context limit, and the judge fixed. All results come from a single ReAct agent, with no sub-agents and no test-time verification. With management enabled, on BrowseComp, BrowseComp-ZH, DeepSearchQA, and HLE the two models reach $82.2/84.8/86.9/52.3$ and $88.6/85.1/92.9/56.4$, the strongest overall results among open-source search agents in their respective parameter ranges. We plan to release the model weights together with the complete recipe for data construction, training, and evaluation.

0.52

MERIT: Scaling KG Foundation Models with Relation Transformers

cs.LG ·2026-10-11 ·Cosimo Gregucci, Daniel Hernandez, Antonio Vergari, Steffen Staab
similarity 0.54
Abstract

Knowledge graph (KG) foundation models (KGFMs) predict links on unseen graphs zero-shot. KGFMs have been pioneered by ULTRA which builds a relation graph from co-occurrences that match motif templates, and passes messages over it to obtain relation representations. More recent KGFMs such as MOTIF and TRIX increase the size of this relation graph, and FLOCK replaces message-passing with probabilistic random-walk ensembles. However, larger relation graphs mean more messages, and more walks mean more sequences to encode, so they improve performance at higher training and inference cost. This cost hinders their use in more complex reasoning tasks, such as complex query answering. To reverse this trend, we design MERIT, the first KGFM with a transformer relation encoder: every relation name attends to every other, and the motifs enter as a soft prior rather than as edges. Over 56 benchmarks, MERIT matches the zero-shot MRR of SoTA KGFMs while being 6x faster than TRIX and over 100x faster than FLOCK at inference. Fine-tuned for complex query answering like UltraQuery, MERITQuery improves over it at matched cost.

0.23