CS Paper Radar

Do General NLP Embeddings Capture Ontological Reasoning?

Hamed Babaei Giglou, Jennifer D'Souza, Sören Auer

cs.CL cs.AI ·submitted 2026-08-31 18:03 UTC · v1
top venuebenchmark

Abstract

General-purpose NLP embedding models perform well on linguistic tasks, but their ability to capture symbolic ontological structure remains unclear. We introduce AVA, a systematic framework for evaluating whether embeddings distinguish logic-sensitive relational semantics in ontologies and knowledge graphs. AVA comprises 171,007 contrastive triplets derived from 163 heterogeneous ontologies using hierarchy inversion, relation substitution, and disjointness injection. Each triplet contains an ontology statement, a semantically equivalent paraphrase, and a logic-sensitive hard negative with contradictory relational meaning. We evaluate more than 25 state-of-the-art embedding models and find substantial limitations: the best model achieves only 0.739 triplet accuracy, while hard negative accuracy falls to 0.135. Fine-tuning improves discrimination by a large margin but transfers poorly to downstream Semantic Web tasks, including taxonomy discovery and ontology alignment. Further analysis suggests that improvements stem partly from perturbation-specific pattern recognition rather than robust ontological understanding. These findings reveal a persistent gap between linguistic representation learning and ontology-level discrimination, challenging the assumption that strong NLP benchmark performance translates to Semantic Web competence.

Comment
7 pages, 3 figures. Accepted as a short paper at CIKM 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Semantic Abstraction for Natural Language Inference: a Methodological Framework for Discovering and Compensating Semantic Knowledge and Reasoning Gaps in Large Language Models

cs.CL ·2026-09-22 ·David Torres-Moreno, Jorge Hermosillo-Valadez
published similarity 0.68
Abstract · 59 pages, 13 figures. Preprint of the article published in Knowledge-Based Systems, https://doi.org/10.1016/j.knosys.2025.114825

Despite their outstanding performance on many NLP tasks, LLMs face serious challenges related to semantic abstraction. In this study, we are interested in understanding how LLMs leverage abstract semantic knowledge in natural language inference (NLI), which requires sophisticated linguistic capabilities to interpret implicit meanings, contextual conceptual relationships, and semantic connections between words and phrases. To this end, we propose a methodological framework for constructing new semantic knowledge at a higher level of abstraction, which we define under the notions of semantic compatibility and incompatibility for NLI. In this framework, the meaning of the lexical-semantic relations between the premise and the hypothesis is reconfigured to achieve a more flexible semantic network that induces different reasoning paths in LLMs. These new pathways show a consistent pattern of responses that allows agreement on a single response. The results demonstrate that our proposal allows to discover and compensate for LLMs' semantic knowledge gaps in NLI, achieving significant improvements in accuracy, exceeding 10% for some models, and in particular for the non-entailment class. It is essential to note that LLMs need structured knowledge and not just more data to bridge reasoning gaps. Our hybrid approach directs attention to overlooked word relationships, allowing models to synthesize missing information. We believe that the future lies not in increasing model size, but in creating a semantic scafolding that mimics the flexibility of human thinking. Hopefully, our proposal will enable the development of more robust agents and interpretable reasoning, guiding AI toward reliable language understanding.

0.63

Inductive Learning of Concept Embeddings from Cross-Linguistic Colexification Patterns with Textual Attributes

cs.CL ·2026-10-15 ·Arne Rubehn
similarity 0.66
Abstract

The advent of distributional semantics as a framework for computational semantics marked a paradigm shift in the nature of representations, by which meanings are represented as dense vectors commonly known as word embeddings. Computational approaches to historical or typological language comparison, on the other hand, still mostly rely on discrete representations, while the potential of embedding-based methods remains largely untapped in this context. Recent studies have proposed concept embeddings as a language-independent, onomasiological counterpart to word embeddings, based on networks of cross-linguistic colexification patterns. While the resulting concept embeddings reflect meaningful semantic properties and relations, they suffer from a lack of scalability, since they rely on transductive embedding approaches which only learn embeddings for nodes present in the training data. In this study, we present an inductive framework for learning concept embeddings, combining the structural properties of full and partial colexification networks with semantic information from short concept descriptions. This framework allows the embedding of unseen concepts based on a short definition. We show that the resulting concept embeddings correlate with various cross-linguistic measures of similarity or relatedness, while generalizing well to new concepts that were absent from the training data.

0.16

Peeling the Matryoshka: Extracting Ontological Structure from Retrieval Embeddings

cs.CL cs.IRcs.LG ·2026-10-30 ·Eunchan Kim, Inwhee Joe
top venuecode/data similarity 0.66
Abstract · Accepted to EMNLP 2026 (Main Conference). 13 pages, 6 figures, 7 tables. Code: https://github.com/Eunchan24/PEEL

Taxonomy construction is essential for knowledge-intensive NLP yet remains costly: clustering-based methods (i) need careful per-corpus tuning and yield unstable hierarchies, while LLM-based pipelines (ii) depend on expensive task-specific training that does not transfer across domains. We report an empirical finding that bears on both: in Matryoshka Representation Learning (MRL) embeddings, where each leading prefix is itself a usable, coarser vector, prefix length aligns with the coarse-to-fine granularity of gold taxonomies; shuffled- and reverse-dimension controls isolate the alignment to the learned early prefix dimensions. The pattern holds across three architecturally distinct MRL models. To test whether this regularity is usable, we introduce PEEL (Peeling Embeddings to Extract Latent ontologies): a minimal probe over these frozen embeddings, trained once with an order-embedding objective that represents hierarchy as vector containment; a minimum spanning arborescence then decodes taxonomic DAGs. Using only a concept vocabulary (no documents), PEEL recovers taxonomic DAGs with high structural fidelity on two OLLM benchmarks, transferring to a new domain with no target-edge supervision. The end-to-end OLLM pipeline, which jointly discovers concepts and induces structure from document corpora, is reported as a cross-task reference. Pre-trained retrieval embeddings encode a decodable granularity gradient; a minimal probe renders it as explicit parent-child (is-a) structure.

0.55

AutoOntic: Automatic Ontology Induction from Text for Knowledge Graph Construction

cs.CL cs.AIcs.LG ·2026-11-11 ·Alla Chepurova, Aydar Bulatov, Mikhail Burtsev, Yuri Kuratov
code/data similarity 0.65
Abstract

Graph-based retrieval-augmented generation depends on the quality of the knowledge graph (KG) it retrieves over, yet schema-free LLM-based extraction pipelines typically produce flat graphs: deduplicated entities and relations with no type hierarchy and no domain/range constraints. We present AutoOntic, a pipeline that automatically constructs an ontology from text by canonicalizing open-vocabulary entity types and relations, inducing a hierarchy over the canonicalized types, and mining relation-level domain and range constraints. Its hierarchy induction procedure prioritizes types using corpus-derived relation-argument profiles and places them through local, LLM-adjudicated parent/child checks, allowing newly inserted types to trigger reassignment of existing parent-child links. The resulting hierarchy constrains entity deduplication and supports the induction of relation-level domain/range constraints. For intrinsic evaluation, we introduce a reference-based protocol that classifies induced ontology types as exact, broader, narrower, or unrelated to reference domain and range classes. On the Wikidata-TekGen and DBpedia-WebNLG subsets of Text2KGBench, over 90% of AutoOntic's induced endpoint types are semantically aligned with the reference classes across two LLM backbones: about half match exactly, while most remaining differences concern granularity. On MINE, knowledge graphs organized using an ontology built by AutoOntic achieve 88% retrieval accuracy. AutoOntic is available at https://github.com/screemix/AutoOntic.

0.28

Reflective Compositional Grounding for Natural-Language-to-SPARQL

cs.DB ·2026-10-15 ·Isabelle Mohr, John Dujany, Andre Freitas
benchmark similarity 0.65
Abstract

Natural-language query generation over large knowledge graphs embodies the classical vocabulary problem in human-systems communication. Questions express concepts through wording and compositions that can differ substantially from the target schema, while limited model context requires selective access to schema information. We present an NL2SPARQL framework that places conceptual compositionality at the center of query interpretation and connects it to the relational composition required for binding and query construction. A fixed concept encoder represents compound expressions jointly, and schema candidates are ranked using semantic similarity, type compatibility, and hierarchy proximity. Reflective refinement critically examines how the grounded meanings are realized through argument bindings, joins, and operator scope. Diagnosed semantic discrepancies guide revisions to the responsible stage. We formalise a monotone refinement discipline that accumulates scoped interpretation requirements and establish conditions under which changes to the composed process preserve previously validated interpretations. Across five benchmark/knowledge-graph targets, mean schema recall@10 increases from 0.712 to 0.837. With retrieval held fixed, reflective prompt rewriting adds 2.27-3.33 answer-F1 points. These experiments evaluate the grounding and refinement mechanisms, while the formal analysis specifies the stronger requirements for conservative adaptation across queries.

0.07

LLM-Assisted Discovery of Typed Semantic Links for Ontology Network Construction

cs.CL cs.AI ·2026-10-01 ·Nouha Hayouni, Sheeba Samuel, Alsayed Algergawy
similarity 0.65
Abstract

Constructing typed, justified semantic links between ontologies is essential for enabling interoperability across heterogeneous and interdisciplinary knowledge domains. However, manually curating such links is difficult to scale. To address this challenge, we propose an end-to-end framework for ontology network construction that automates the discovery and generation of both intra-domain and inter-domain relationships. Our approach combines domain-adapted DistilBERT embeddings for dense contextual representation, clustering-based pre-filtering to reduce the candidate search space, and GPT-4o-driven relationship generation via iterative prompt engineering to produce semantically rich, interpretable links. Applied to ReproduceMeON - a network of 33 ontologies spanning machine learning, microscopy, computational science, and experimental workflow - the pipeline reduces approximately 800k raw concept pairs to 95k high-quality candidates. Human expert validation of 429 generated relationships by two independent annotators yields an overall precision of 80.19% (91.49% on high-certainty annotations) and an F1 of 0.890, with substantial inter-annotator agreement. Comparative experiments against five similarity-based baselines, including Sentence-BERT, show a substantial performance gap (best baseline F1 = 0.581), while an ablation study demonstrates that similarity-based methods alone fail to discriminate valid from invalid relationships (AUC approx 0.5) on the filtered candidate set. These findings highlight the necessity of LLM-based reasoning over concept roles and domain semantics for accurate relationship construction.

0.18

Explore-over-Graph: Hybrid Embedding-LLM Reasoning for Knowledge Graph Question Answering under Incompleteness

cs.CL ·2026-09-30 ·Ola El Khatib, Djellel Difallah
publishedbenchmark similarity 0.64
Abstract

Large language models (LLMs) are increasingly combined with knowledge graphs (KGs) to ground reasoning in structured evidence. However, most LLM-based KGQA methods rely on traversing existing graph edges and become unreliable when reasoning paths are broken by missing facts. Alternatives that ask LLMs to generate missing knowledge risk introducing hallucinated evidence. We introduce XoG (eXplore-over-Graph), a framework for multi-hop question answering over incomplete KGs that recovers missing reasoning paths from learned graph structure rather than LLM parametric knowledge. XoG combines type-level entity-relation statistics to identify candidate relations with KG embeddings to retrieve plausible missing entities, using the LLM as a semantic selector and reasoner. These mechanisms are integrated into an iterative planning-exploration-reasoning process. Experiments on WebQSP, CWQ, and the Wikidata-based BRINK benchmark show that XoG remains competitive on complete KGs and consistently outperforms comparable methods without task-specific KGQA training under KG incompleteness. These gains persist across multiple LLM backbones, indicating that stronger LLMs alone do not resolve missing graph evidence. XoG also reduces LLM token consumption by up to 33% compared with a closely related planning-based approach.

0.22

FedV-KGQA in Practice: Design Lessons and an Interactive Prototype

cs.AI cs.CLcs.IR ·2026-09-12 ·Md Saikat Islam Khan Bappy, Oshani Seneviratne
published similarity 0.64
Abstract · Accepted at the ISWC 2026 Posters and Demos Track; to appear in the ISWC 2026 Companion Volume (CEUR-WS)

Knowledge graph question answering usually assumes that one system can reach the whole graph. In practice, facts are often held by organizations that share entity identifiers but own disjoint relation types, so no single party sees a complete reasoning chain. This poster presents the empirical findings of FedV-KGQA on multi-hop question answering over such vertically partitioned graphs. Each silo enriches its local graph and trains a knowledge graph embedding on its own triples. A server then concatenates the silo-specific entity views, anchors the projected question at the topic entity, and ranks candidates by similarity. Raw triples and relation embeddings never leave a silo. Comparing the FedV-KGQA experiments with one another yields three results. First, federated fusion recovers most of the centralized accuracy, while a single silo recovers little. Second, anchoring and enrichment matter more than the choice of embedding model. Third, the cheapest encoder depends on the target accuracy rather than on parameter count. This poster paper contributes that cross-experiment comparison, four design lessons drawn from it, and an interactive prototype that runs real inference and traces the full pipeline, per question, on released checkpoints.

0.16