CS Paper Radar

SQuIREL: A Context-Relative Framework for Measuring Entity Richness in Wikidata

Michelle Nwachukwu, Albert Meroño-Peñuela, Martim Brandão

cs.AI cs.DL ·submitted 2026-10-17 14:16 UTC · v1

Abstract

Wikidata is a large, heterogeneous knowledge graph and applying data quality metrics requires context-specific approaches. The "richness" quality metric is currently under-researched and typically restricted to the ontology level, which fails to capture entity-specific structural depth. To address this gap, we introduce SQuIREL, a method that measures richness directly at the entity level using six dimensions: Semantic Specificity, Qualifier Density, Internal Connectivity, Reference Provenance, External Network Connectivity, and Linguistic Accessibility. These dimensions are kept disaggregated to preserve actionability for targeted editorial intervention. We apply this metric to two case studies of oil paintings (Brazilian vs. French) and writers (grouped by citizenship and gender) to demonstrate how it surfaces representational inequalities that may indicate systemic bias. Our metric successfully distinguishes between highly tailored and basic entity profiles, revealing nuances that raw counts and existing aggregate scores obscure. Specifically, it identifies structural imbalances, such as 48% of Brazilian paintings clustering as low-richness stubs against 0% in the French cohort, as well as intersectional patterns, including contrasting gender gaps between UK and Nigerian writers.

Comment
34 pages, 16 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Cross-lingual Biography Enrichment via Claim Extraction and Alignment

cs.CL cs.AI ·2026-08-24 ·Yifei Song, Ziyang Chen, Emil Sayilov, Claire Gardent
top venuebenchmark similarity 0.61
Abstract · Accepted by EMNLP 2026 main conference

English Wikipedia is often treated as the default encyclopedic source, yet non-English Wikipedia editions can contain richer locally grounded information for long-tail figures. We study cross-lingual biography enrichment: enriching an existing English biography with facts supported by a non-English biography about the same person. Focusing on women from non-English-speaking contexts, we introduce \textsc{CLAW-4L}, a benchmark consisting of 300 Wikipedia biography pairs linking an English biography with its French, Chinese or Azerbaijani counterpart, along with claim annotations and a fine-grained claim-pair relation corpus. We propose a claim-based enrichment framework that extracts English claims from both biographies, aligns them to identify enrichment evidence from the non-English biography, and rewrites the English biography using the selected claims. Our results show that non-English Wikipedia biographies provide valuable evidence for improving English biography coverage, while lower-resource settings remain challenging.

0.39

A Computational Framework for Modelling Organisation-Level Semantic Identity from Longitudinal Textual Data

cs.LG cs.CL ·2026-08-27 ·Brinda Murali Krishna, Oktay Karakuş, Can Eyüpoğlu
similarity 0.61
Abstract

Can longitudinal texts produced by organisationally affiliated contributors reveal collective semantic differentiation and change without obscuring differences among those contributors? We develop a computational social science framework that combines established semantic and graph measures into interpretable organisational fingerprints and temporal profiles. We use \emph{organisation-level semantic identity} to describe a corpus-dependent interpretation of recurring and changing features of affiliated textual output, rather than corporate culture or managerial intention. Drawing on an inherited semantic landscape, we examine 1,289 songs by 126 artists affiliated with four K-pop entertainment companies between 1997 and 2023. The evidence for differentiation is selective. After correction, all three novelty-related measures show artist-level differences, whereas the diversity and connectivity measures tested do not; only two company--community enrichments remain significant. Temporal trajectories reveal further distinctions between portfolios. Artist-grouped balanced accuracy reaches 0.334 against a chance baseline of 0.25, providing modest supporting evidence of affiliation-related signal rather than independent validation of identity. Robustness analyses support the presence of continuous semantic structure more strongly than precise rankings. Together, these findings show how collective regularities can coexist with shared content, contributor heterogeneity and change over time, while leaving cross-domain generality and broader organisational construct validity unresolved.

0.23

Preserving What Matters: Semantic Scaffolds Beyond Saturation in Summarization Evaluation

cs.CL cs.AI ·2026-09-18 ·Nikhil Reddy Pottanigari, Ramin Fahimi, Noah Bolger, Sepideh Kharaghani, Ying Zhang
similarity 0.59
Abstract · Accepted at the AIMS Workshop at COLM 2026

Summarization ships in countless production systems, making model selection a routine decision that depends on measuring summary quality. Existing metrics struggle to support this: ROUGE captures only surface overlap, while LLM-as-judge scores saturate to near-identical values that fail to rank models effectively. We observe this saturation across three public datasets, two proprietary datasets, and multilingual settings. Motivated by this, we introduce Semantic Scaffold, an evaluation framework that extracts a hierarchical representation of facts, questions, and entity attributes from a source text, labeling each as a main point or supporting detail, and reusing this structure as a fixed reference for scoring summaries. From this representation, we derive three diagnostic metrics: Fact Preservation Score (FPS), Question Preservation Score (QPS), and Entity Preservation Score (EPS), designed to reward the preservation of essential information while penalizing detail overload, and position them as interpretable diagnostics that remain informative where holistic axes collapse. Finally, we analyze four recurring failure modes of ROUGE and LLM-as-judge scores, demonstrating that scaffold-based evaluation remains informative where conventional metrics collapse.

0.24

Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

cs.CL ·2026-09-09 ·Parinthapat Pengpun, Simran Khanuja, Graham Neubig
top venuebenchmark similarity 0.58
Abstract · Accepted to EMNLP 2026 Main Conference

Multimodal entity linking grounds entity mentions in text and images to knowledge-base entries. These systems degrade on rare entities, but prior work measures rarity primarily through popularity-based metrics such as pageviews. We broaden this view using knowledge-graph structural metrics that capture how well an entity is documented and connected. These metrics identify many rare entities that popularity metrics miss. Across the resulting rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, showing that different rarity definitions expose different failure modes. To address these failures, we introduce a simple, training-free framework in which a reasoning-capable vision-language model iteratively searches and reasons over Wikipedia, gathering evidence dynamically. Controlled experiments show that reasoning and retrieval are complementary. Reasoning alone does not significantly improve accuracy on rare entities. Retrieval without reasoning improves rare-entity accuracy but can hurt overall accuracy. Their combination performs best. On MERLIN, a multilingual multimodal entity linking benchmark over five languages (Hindi, Indonesian, Japanese, Tamil, Vietnamese), our best system improves over the state of the art by 6.9% overall and by up to 23.3% on rare-entity slices. We release MERLIN-Rare, rare-entity test slices for targeted evaluation, with our framework.

0.39

Initial Evaluation of Potential Bias in Coverage of Humans in Wikidata

cs.CY cs.AIcs.CL ·2026-09-17 ·Clair Kronk
code/data similarity 0.58
Abstract · 11 pages, 3498 words, 1 table, 5 figures. Fixed some typos (Figure X changed to Figure 2, Figure Y changed to Figure 3, Figure G changed to …

Introduction. Open collaborative knowledge graphs such as Wikidata increasingly ground agentic artificial intelligence, information retrieval, and language modeling systems, making systematic auditing of their demographic representation and overall equity a research imperative. Methods. Herein, we present an open-source auditing platform that ingests over 10 million statement bindings representing over 6 million humans on Wikidata via QLever, and evaluates representation of gender, sexual orientation, geography, birthplace urbanicity, ethnicity, multilingual coverage of labels, descriptions, and aliases, occupation, and select intersectional pairs of these entities. It does so by making use of Chi-square goodness-of-fit tests, 95% Wilson-score confidence intervals, and disparity ratios, in light of Rubin's missingness taxonomy. Results. Women accounted for 28.71% (CI +/-0.04) of all humans in Wikidata with a stated gender. 38.26% of humans had a citizenship statement, with Western Europe and North America (WENA) representing approximately 53% of such statements. Among 1.8 million birthplaces that could be classified, rural birthplaces were observed in 2.48% of cases (in comparison to 27.4% global baseline). Fewer than 1.2% of entities carried an ethnicity statement, and non-English Wikidata descriptions covered 18.2% of items. Discussion. Our findings reveal significant missingness across the evaluated axes. Ethnicity and sexual orientation were the most critically under-documented (missing statements) while rural birthplaces and non-WENA citizenship were the most underrepresented.

0.45

Preserving contextual information in cultural heritage metadata through multidimensional knowledge graphs

cs.DB ·2026-09-07 ·Lyndon Nixon, Valentina Presutti, Celian Ringwald, Andrea Schimmenti
published similarity 0.58
Abstract · ACCEPTED AT swodch-2026 workshop - ISWC2026

Using Knowledge Graphs (KGs) to describe Cultural Heritage Objects (CHOs) supports semantic richness and interoperability. However, standard KGs fail to capture the context-dependent validity of statements. This limitation is critical for cultural heritage metadata, which must often accommodate evolving or conflicting viewpoints, such as colonial versus post-colonial perspectives or shifting scientific consensus. While current knowledge representation methods address basic contextualization via provenance, qualifiers or reification, they lack a unified framework to simultaneously model and query data across multiple social, cultural, and political dimensions. To bridge this gap, we introduce the conceptual foundations of Multi-dimensional Knowledge Graphs (MKGs) and discuss how they preserve complex, multi-layered contexts in CHO metadata.

0.39

W-RAG: Source-Aware Retrieval for Enterprise Document Generation from Heterogeneous Knowledge Bases

cs.SE cs.CLcs.IR ·2026-08-22 ·Hridya Dhulipala, Rajesh Ombase, Michael Wang, Tien N. Nguyen
dataset similarity 0.58
Abstract · 18 pages

Retrieval-Augmented Generation (RAG) enables large language models to incorporate external knowledge during generation, improving factual grounding and domain adaptability. However, existing RAG pipelines assume that evidence retrieved from multiple repositories can be ranked globally using a single similarity function. While suitable for open-domain retrieval, this assumption breaks down in enterprise document generation, where heterogeneous knowledge bases (such as policies, regulations, technical documentation, and departmental guidelines) serve distinct roles and must be jointly represented in the generated document. As a result, global ranking often produces unbalanced context dominated by a subset of sources, leading to incomplete enterprise drafts. To address this limitation, we propose W-RAG, a source-aware retrieval framework that performs ontology-guided retrieval, local ranking within each knowledge base, and source-level weighting to regulate evidence composition. We further introduce a new dataset for retrieval-grounded enterprise document generation spanning multiple document types and industry domains. Experiments show that standard RAG pipelines struggle on this task, while W-RAG significantly improves document coverage and generation quality.

0.25

VEX-Bench: Benchmarking Verification Complexity of LLM-Generated Misinformation

cs.LG cs.AIcs.CLcs.CY+1 ·2026-09-28 ·Hanxun Huang, Yutao Wu, Qizhou Wang, Silvia Montaña-Niño, Yige Li +7 more
publishedbenchmarkcode/data similarity 0.57
Abstract · NeurIPS 2026

Large language models (LLMs) have made misinformation inexpensive to produce but not to verify, creating a growing asymmetry in the information ecosystem. Under tight time, labor, and budget constraints, media organizations, platforms, and fact-checkers rely on screening to prioritize which content to verify. We introduce VEX-Bench, a unified benchmark for evaluating the verification complexity of LLM-generated misinformation, as perceived during screening, across models and generation methods. Verification complexity is assessed along multiple dimensions derived from journalistic and fact-checking practices, capturing checkability, harm potential, source credibility signals, imposter legitimacy, and expected verification effort. We define the VEX score as an integrated measure combining elicitation yield and verification complexity to quantify how generated content consumes limited verification capacity. We construct a benchmark spanning two misinformation categories, 6 high-stakes domains, and 60 real-world topics, and evaluate 7 frontier LLMs and 7 generation methods, yielding 5{,}880 articles. We employ an LLM-as-judge for scalable evaluation and validate it using content-analysis methodology, including ordinal Krippendorff $α$ for inter-annotator reliability, complemented by fact-checking agents for verification. Our findings show that no single method dominates all dimensions, underscoring the need for multi-dimensional evaluation. LLMs can generate high-VEX misinformation at 3$\times$ to 169$\times$ lower cost than agent-based verification. Such content is often prioritized during screening, consuming scarce verification resources and introducing a systematic risk of misallocation in resource-constrained verification systems. The code is publicly available in our \href{https://github.com/HanxunH/VEX-Bench}{GitHub repository}.

0.59