CS Paper Radar

Corporate-Family Resolution Is Not a String-Matching Problem: A Public Benchmark Stratified by Name Visibility

Harshit Gupta

cs.DB cs.IRcs.LGcs.LO ·submitted 2026-09-02 19:16 UTC · v1
benchmarkcode/data

Abstract

Deciding whether two supplier records belong to the same corporate family is a prerequisite for spend consolidation, credit exposure aggregation and sanctions screening. It is usually treated as entity matching, but the tasks differ: a family link connects records that are deliberately different entities, and the evidence often appears in neither record. We introduce CorpFam, a public benchmark of 54,864 candidate pairs over 10,307 corporate families, derived from 6,638,350 US federal award records in which every supplier self-reports its ultimate parent to a government registry. Pairs are stratified by name visibility: whether the names are identical after normalisation, share a distinctive token, or share none. Because strata have positive rates from 10.2% to 97.3%, we report per-stratum recall, base-rate invariant, rather than F1, which is not. The strongest of 5 matchers recovers 100.0% of identical pairs and 4.2% of invisible ones; no method exceeds 4.7% on the latter. The failure begins before matching. Blocking decides which pairs a matcher sees, and we evaluate 7 schemes spanning phonetic keys, attribute keys that ignore the name, and semantic nearest neighbours. None reaches three percent on invisible pairs, and their union recovers 6.8%. 93.2% of these links never enter the candidate set, so no matching-stage improvement can reach them. The links are real: against SEC Exhibit 21 subsidiary schedules, which share no provenance with procurement registration, 64.2% of invisible links are corroborated, against 0.16% under permuted parents and 0.41% against the same parent's wrong exhibit: two unrelated nulls agreeing to within 0.25 points. Corporate-family resolution is a retrieval problem misfiled as a matching problem; the intervention point is candidate generation, not ranking. The benchmark, adjudication log, and code reproducing every number are released.

Comment
21 pages, 8 tables. Benchmark, full adjudication log, and code reproducing every number: https://github.com/harshitg93/corpfam-benchmark

Closest papers

Nearest neighbours by meaning across the whole index.

Near-Duplicate Families Break Exact-Record Membership Inference

cs.CR ·2026-09-27 ·Yiyong Liu, Jiayang Liu, Yixin Tan, Lu Sun, Rui Wen
similarity 0.54
Abstract

Membership inference (MI) asks whether a specific record appeared in a model's training set and is increasingly used as evidence for data provenance and copyright auditing. These applications require determining whether the exact queried record was used for training, rather than merely whether the model was exposed to similar content. Making this distinction is challenging because web-scale datasets naturally contain near-duplicates, including syndicated articles, mirrored pages, and lightly modified images. We show that this creates a fundamental confound for standard MI. A clean-reference audit typically calibrates membership against a null in which neither the queried record nor its near-duplicate family is present. In deployment, however, the queried record may be absent while a non-identical family member was used for training. We introduce a four-world audit that independently varies exact-record inclusion and family presence to separate these cases. Natural near-duplicate families cause severe false attribution. On CC-News, a clean-reference LiRA auditor labels 99.70% of family-present exact non-members as members at 1.00% false-positive rate. This failure persists across alternative scores, model architectures, and executed deduplication and retraining. Controlled interventions further reveal that the effect depends on the learning objective. In classification, faithful families largely substitute for the exact record, reducing exact-given-family inference to near chance. In autoregressive language modeling, the exact sequence retains a detectable residual, while family presence still confounds clean-reference decisions. These results show that positive model-only membership evidence may establish family-level exposure without establishing exact-record provenance.

0.37

Better Retrieval, Limited Clustering Gains: A Controlled Study of Multilingual Company Entity Resolution

cs.AI ·2026-10-04 ·Yijiashun Qi, Yuxuan Li, Hanzhe Guo
published similarity 0.53
Abstract · accepted by ICEKIM 2026

Company name retrieval and final entity clustering can respond differently to encoder adaptation. We study this dependency through a controlled pipeline: train multilingual E5 encoders with matched positive examples and random or hard negatives, select checkpoints on development data, retrieve within equal country-specific edge budgets, and apply unchanged pair classifiers and guarded Union-Find clus- tering. Evaluation uses a subsequently collected GLEIF snapshot; a stratified, assisted human audit and a post-hoc temperature sweep examine label and parameter sensitivity. On 3,633 names with source-silver labels, adapted retrieval raises direct pair recall from 53.74% to 76.98%, whereas cluster recall moves from 32.54% to 33.33%. Most newly retrieved positives fail the existing classifier threshold. The study contributes a controlled comparison and pair-level diagnosis of this loss between retrieval and confirma-tion. It shows why evaluating a replacement retriever requires measuring the resulting entity partition, while leaving joint matcher retraining and independent gold-label evaluation for future work.

0.29

Growing Clusters Lower the Bar: Profile Accretion in LLM-Based Incremental Entity Resolution

cs.CL cs.DB ·2026-11-17 ·Huixue Han, Shangzhi Xu, Anqi Xiong, Yu-Chu Tian
unusual similarity 0.53
Abstract

Incremental entity resolution pipelines ask a large language model (LLM) whether a new record belongs to an existing cluster and show it the records already linked. Analyses of matching oracles assume that the answer depends only on the records compared. For the LLM matchers we test, it depends on cluster size, and the model lowers its bar for joining a cluster as the cluster grows. On French procurement suppliers from FOPPA, with SIREN identifiers as ground truth, we hold the new record fixed and vary only the number of verified members shown. Sixteen members instead of one raise the false match rate on similarly named companies by \dfmrMin--\dfmrMax{} percentage points in six open-weight models, while their area under the ROC curve rises by only \aucGainMin--\aucGainMax. In Claude Sonnet~5 and Haiku~5.5 the rise is \clSonD{} and \clHaiD{} points. The shift persists with street addresses, prompt variants and the BatchER and ComEM formats, and one false merge raises acceptance of the intruder's other records in every open model. It does not arise on Czech and Bulgarian TED notices, where one record with its full address already identifies the company. In online linking on confusable blocks, a single threshold set on labelled data raises recall by a factor of \fxRRmin--\fxRRmax{} for five models with no detectable loss of precision and no extra LLM calls.

0.26

When Names Cross Scripts: A Source-Grounded Benchmark for Historical Entity Reconciliation in the Mongol World

cs.CL cs.AI ·2026-08-24 ·Xiang Chen, Zeyu Zhang
benchmark similarity 0.51
Abstract · 38 pages, 4 figures, 7 tables. arXiv preprint

Historical people may appear under different languages, scripts, and transcription traditions, while distinct individuals may share highly similar or even identical names. This makes historical identity reconciliation more than a problem of string matching or transliteration. We introduce MHER, a provenance-controlled benchmark for pairwise reconciliation of person-name attestations from the Mongol world. MHER contains a balanced 396-pair Name-only core over 84 primary historical persons and a stricter 160-pair Source-grounded subset constructed from mention-by-source evidence, with entity-disjoint development and test splits. Across five generative systems, correctly Source-grounded evidence improves paired TEST accuracy by 12.96 to 94.44 percentage points relative to Name-only input. On five identical-surface different-person cases, all models fail under names alone (0/25 model-item decisions), whereas Source-grounded evidence yields 24/25 correct resolutions, with the remaining output an abstention. Context-only ablations show that historical descriptions often carry substantial identity information, while explicitly signaled misgrounding controls produce substantially lower performance. We also find that names are not uniformly beneficial: for Qwen3-8B, restoring surface forms converts ten otherwise correct Context-only distinctions into false identity merges. These results show that historical entity reconciliation depends not only on surface correspondence, but on whether identity judgments respond appropriately to provenance-controlled historical evidence. MHER therefore provides a controlled framework for studying evidence use, abstention, and failure modes in historical NLP.

0.20

Measuring Brand and Source Discovery under Repeated LLM Queries: A Finite-Sample Audit

cs.IR cs.CL ·2026-09-04 ·Dmitrij Żatuchin
similarity 0.50
Abstract · 14 pages, 4 figures. Substantially revised finite-sample audit of repeated LLM queries with extraction sensitivity, a matched five-call comp…

Repeated-query audits must distinguish recovery of a collected set from completeness of possible outputs. We apply sample-based rarefaction to 4,500 responses from 50 buying questions, six configurations and 15 calls per cell. Historical-dictionary median ten-call recovery of the observed 15-call set ranges from 92.6% to 95.2%; re-adjudicating all 45,683 candidate strings changes this range to 89.5%-94.7%. Two blinded Gemini 3.1 Pro annotation roles assessed 600 complete answers, yielding micro F1 of 0.908 for canonical-name agreement and 0.975 for span-overlap agreement. This is AI-based evidence, without a human reference study. A separate matched roster analysis of 3,750 records per wave gives median single-call recovery of the observed five-call set of 80.0%-92.5% in February and 90.0%-100.0% in September, with question-subset dependence. Source accumulation also changes when API-returned hosts are restricted to those referenced by answer citation markers. These findings show that recovery percentages depend on extraction, question selection and the finite reference collection. They support explicit measurement definitions and sensitivity analyses, without establishing exhaustive repertoires, causal retrieval effects or a universal stopping rule.

0.50

Cleaning or Recalling? Knowledge Dependence in LLM-based Data Cleaning [Experiment, Analysis & Benchmark]

cs.DB ·2026-11-05 ·Jingru Yang, Honglu Zhang, Jiayi Mi, Ju Fan, Zheng Chen +1 more
benchmark similarity 0.49
Abstract

High scores on public data-cleaning benchmarks can reflect knowledge of familiar entities, making it difficult to assess how well an LLM repairs unfamiliar data. We present CoRE (Cleaning or Recalling), a protocol comparing public benchmarks with fresh real records and paired name surrogates built from English words or syllables. The surrogates retain error positions and non-identity values while transferring the benchmark's error mechanisms. Across 21 controlled tables and eight LLMs, repair performance varies substantially with entity names on Hospital and, for commercial models, Movies. Renaming removes some information needed for repair, such as a film's identifier mapping. It also affects repairs whose candidates remain available: replacing entity names with word-based surrogates lowers DeepSeek-V4-Pro's year accuracy from 91% to 18%. On Hospital and Movies, commercial models score higher on fresh records than on word-based surrogates. A screened continuation probe finds no positive evidence of benchmark-row memory in the tested models. Familiar facts can conflict with the task. Even with explicit table-evidence instructions, commercial models rewrite assigned directors at rates of 0.000-0.404 on known films. Reference scoring can also disagree with external records. On Rayyan, an ISSN correction matching a title-matched PubMed record receives no credit, whereas 94% of TabClean's credited repairs reproduce the reference date transformation. We provide controlled tables, scoring code and materials for offline replay.

0.24

What Unlinked Aggregates Still Prove: Ranker Comparisons from Exact and Differentially Private Item Counts

cs.IR ·2026-11-16 ·Chang Xu, Hao Wang, Gedi Zhou
publishedcode/dataunusual similarity 0.49
Abstract · 10 pages, 4 figures. Accepted for publication in the Privacy and Security of Big Data special session at IEEE BigData 2026. Code available a…

Privacy controls can prevent an evaluator from linking purchases to the recommendations users received, while a merchant can still release complete item-level purchase counts for the evaluation cohort. We ask when these counts identify which recommender performs better. The counts constrain who could have bought each item. We bound the performance difference over all assignments consistent with the counts. A reduced transportation network handles full catalogs and can certify attainable endpoints. If the interval excludes zero, it identifies the better recommender for that cohort. With differentially private counts, one confidence guarantee applies jointly to all comparisons from the same release. In two e-commerce cohorts of about 20,000 and 100,000 users, exact counts narrowed every interval compared with bounds that ignored the counts. Most decisions involved a recommender that favors popular items. For NDCG@10, which rewards ranking purchased items higher, each setting combined one of three personalized pairs with a pattern and rate of missing links. Exact counts settled 14 of 168 settings in the smaller cohort and none of 81 in the larger cohort. In the smaller cohort, exact counts broken down by user group settled two personalized pairs when 90% of users remained linked, at the cost of revealing more about individual purchases. Whole-cohort private releases with one count per item retained much of the narrowing but settled no personalized pair in either cohort. Grouping less popular items into bins let private counts settle one personalized pair in the smaller cohort.

0.53

Beyond Symmetry: Asymmetric Matching with Transformers for Collective Entity Resolution

cs.DB ·2026-11-11 ·Andrei Olar
benchmark similarity 0.49
Abstract

A core assumption of entity resolution (ER) is its symmetric nature, which rests on a principle so self-evident it borders on tautology: if a refers to the same object as b, then b must also refer to the same object as a. However, this global property of symmetry does not necessarily extend to each of the complex decision processes within ER systems. In cases involving vague or ambiguous information, enforcing symmetry can be detrimental to performance, an issue exemplified by the ``bad triplet problem''. This paper investigates the effects of input-order asymmetry within a collective ER framework that integrates distinct matching and clustering stages. The input order-aware matching stage utilizes pre-trained transformers to explicitly model and exploit asymmetry, making it possible to evaluate its effect on clustering performance across multiple benchmark datasets. Our results demonstrate that asymmetry is not only measurable but also introduces a significant skew in the final entity clusters, highlighting the need to re-evaluate supervised training methodologies for matchers, particularly the development of direction-aware training paradigms that can preserve, rather than neutralize, the inherent asymmetry in matching tasks.

0.22