CS Paper Radar

CID: Measuring Cognitive Work Through Semantic State Change

Ron Fulbright

cs.AI ·submitted 2026-11-06 14:52 UTC · v1

Abstract

Traditional information theory provides powerful tools for analyzing statistical properties of information but offers limited insight into meaning and semantic transformation. It has never been possible to quantify the meaning of an arbitrary piece of information. Large language models represent information as vectors embedded within high-dimensional semantic spaces, enabling meaning to be analyzed geometrically. This paper defines semantic state as the vector representation of information within a model's semantic space and defines cognitive work as the transformation between semantic states, movement through semantic space. To quantify this transformation, we introduce the Contextual Information Differential (CID), a covariance-aware metric based on Mahalanobis distance measuring semantic displacement relative to the structure of the embedding manifold. By treating meaning as a geometric property of semantic space, CID provides a practical method for measuring semantic change, cognitive transformation, and information processing. This framework establishes a foundation for analyzing cognitive processing, learning, reasoning, knowledge generation, synthetic expertise, and human/AI collaboration through implementation-independent measurement of semantic state change in both human and artificial systems.

Comment
12 pages, 1 figure, 12 equations

Closest papers

Nearest neighbours by meaning across the whole index.

Beyond Linear Concepts: Discovering and Aligning Non-Linear Concept Manifolds in Large Language Models

cs.LG cs.CL ·2026-10-01 ·Tido Specht, Elias Benedict Krey, Nils Neukirch, Nils Strodthoff
similarity 0.58
Abstract · 24 pages, 13 figures. Code: https://anonymous.4open.science/r/NLMCD-NLP-C5E7

Understanding information processing in large language models (LLMs) requires dissecting the geometric organization of their internal token representations. While existing mechanistic interpretability (MI) methods seek to extract concepts, they are constrained by a strong linearity assumption challenged by evidence of non-linear feature manifolds. We move beyond linear concepts by adapting Non-Linear Multi-Dimensional Concept Discovery (NLMCD) from computer vision to token-level LLM activations, modeling concepts as low-dimensional manifolds. To compare concept manifolds across layers and models, we introduce a concept-based alignment (CBA) score, a generalized Rand index that measures geometric proximity without explicit feature matching. Our analysis yields six key findings: (i) a neighboring-layer sanity check shows CBA is more sensitive than PCA- or CKA-based linear baselines; (ii) layer-by-layer alignment matrices reveal two block structures in intermediate and late layers, consistent across models and obscured by linear metrics; (iii) concept composition remains syntax-dominated through most of the network before giving way to increasingly mixed syntactic-semantic concepts in later layers, with increasing output-orientation toward the final layers; (iv) multilingual concept sharing between English and Mandarin is training-dependent rather than universal, strongest in Qwen, weaker in Llama, and absent in GPT-2; (v) inter-model alignment mirrors this structure, with strong correspondence between same-family Qwen models of different scale but weak alignment across model families; and (vi) across Tulu-3 training stages, alignment is highest between adjacent stages, with the largest shift between the base model and SFT, while subsequent preference-alignment stages (DPO, RLVR) leave early layers largely unchanged and RLVR mostly preserves DPO's concepts in late layers.

0.39

What Converges in the Platonic Representation Hypothesis? Structure over Geometry

cs.LG cs.CVmath.AT ·2026-09-23 ·Junwon You, Mihyun Jang, Sangwoo Mo, Jae-Hun Jung
similarity 0.57
Abstract · 33 pages, 12 figures, 6 tables

The Platonic Representation Hypothesis suggests that increasingly capable models converge toward shared representations. Recent work narrows this claim to shared local neighborhood relationships, finding that capacity-dependent trends in several global similarity measures largely disappear after calibration. We challenge this interpretation by showing that prior local-global comparisons confound structural scale (local versus global) with what is compared: relational structure, defined by which samples are related, versus metric geometry, characterized by quantitative relations such as distances, similarities, or correlations. To disentangle these factors, we construct a controlled $2\times2$ framework that evaluates both relational structure and metric geometry at local and global scales. We introduce $H_0$ skeleton overlap as a global counterpart to mutual $k$-nearest neighbors, together with matched distance-aware variants. Across vision-language models, relational structure exhibits robust representational convergence at both scales after calibration, whereas increasingly stringent distance agreement substantially weakens alignment and progressively flattens the capacity-dependent trend. We further extend the analysis beyond ambient Euclidean geometry by evaluating distance agreement under a Riemannian metric approximation and recover the same structure-geometry pattern. The pattern is also reproduced in video-text representations. Together, these results show that relational convergence extends beyond local neighborhoods to global spanning structure, whereas metric geometry exhibits substantially weaker convergence.

0.39

Concept Representations in Vector Space: Toward an Interdisciplinary Study of Meaning Variation

cs.CL cs.CY ·2026-10-18 ·Simon Josef Durstewitz
code/data similarity 0.56
Abstract · 20 pages, 5 figures. Companion paper to arXiv:2610.15496. Code: https://github.com/simon-josef/conrep

An increasing number of disciplines recognize the explanatory potential of meaning-bearing mental structures through which individuals interpret the world and guide their behavior. Concept representations constitute a core class of these structures, emerging from individuals' experiences with concept referents and encoding and organizing their understanding of them. Recognizing a private level of meaning, where concepts are not objectively defined by rules in the world but constitute subjective structures in individuals' minds, raises a fundamental question: do different individuals actually mean the same thing when they invoke the same concept? As latent structures, concept representations are difficult to access and model empirically. The present study introduces a measurement framework that makes such representations empirically accessible through textual traces elicited under standardized conditions. Using transformer-based sentence embeddings, the framework maps standardized responses elicited for concepts into a high-dimensional shared vector space, opening concept representations to the systematic analysis of interindividual variation. It develops three analytical avenues: (1) comparing representational sharedness across concepts, (2) identifying variables that correlate with representational variation, and (3) examining how focal actors or subpopulations deviate from a reference population. Each avenue is illustrated through an empirical proof-of-concept drawing on a large-scale free-association dataset. Together, the demonstrations show how researchers from across disciplines can move from a research question and data source to analysis and interpretation. An accompanying GitHub repository provides a ready-to-use implementation of the framework for existing and custom datasets.

0.19

Coupled Usage-Sense Processes: Temporal and Attributable Lexical Semantic Change

cs.CL stat.ML ·2026-09-25 ·Haruka Ezoe, Ryohei Hisano
theory similarity 0.56
Abstract

Lexical semantic change is usually summarized by a scalar distance between independently sampled period distributions. This measures how much a word changed, but does not reveal when it changed, which mechanisms and component movements carried the change, or which usages support the attribution. We introduce Coupled Usage--Sense Processes (CUSP), which derives these answers from a single marginal preserving temporal process. A hierarchical coupling relates contextual distributions through latent usage components, while Markov composition makes adjacent and longer span correspondences compatible. Displacement operators quantify change magnitude and timing, split variation exactly between movement of component centers and reorganization within components, and attribute it to transported component pairs. Word-local modes resolve distinct directions of change and their activity over time, while representative passages from attributed components ground the analysis in text. Under a Gaussian mixture specialization, we prove parametric recovery of the operators and squared distances. Synthetic experiments support the predicted rate. CUSP remains competitive on English and German DWUG and recovers controlled Janus profiles while maintaining compositionally coherent transport. A large corpus of US court opinions demonstrates transition, mode, and passage attribution in unlabeled natural text. CUSP thus makes magnitude, timing, mechanism, movement, modes, and textual evidence compatible views of one lexical history.

0.19

Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking

cs.CV cs.AI ·2026-10-02 ·Haoyang Luo, Linwei Tao, Jie Gui, Xinghao Chen, Chang Xu +2 more
top venue similarity 0.56
Abstract · Accepted by NeurIPS 2026

Multimodal Large Language Models (MLLMs) suffer from hallucinations, creating a critical need for Uncertainty Quantification (UQ) to ensure reliable deployment. However, existing approaches struggle to detect uncertainty caused by superficial associations, especially when the query-relevant signal is weak. We mainly attribute this issue to their bias toward aleatoric uncertainty arising from data ambiguity, overlooking epistemic uncertainty stemming from model limitations. To further decompose uncertainty types for a comprehensive UQ, we propose Causal-Invariant Masking (CIM), which measures the semantic shift between the original predictions and those conditioned on a causally-focused view. Based on this framework, we introduce Semantic Divergence as our core metric for UQ and provide theoretical evidence that it converges to the variance of model's sensitivity to non-causal correlations, establishing its ability to capture MLLM's limitation. To accelerate UQ in MLLMs, we further propose Expected Embedding Drift (EED), a fast geometric proxy metric that estimates semantic shift directly within the hyperspherical embedding space. Experiments show that our method achieves state-of-the-art performance on various benchmarks, while the proposed EED accelerates by nearly 50% with comparable performance.

0.44

SynFlow: A Multidimensional Diachronic Semantic Analysis Toolkit

cs.CL ·2026-08-19 ·Bach Phan-Tat, Kris Heylen, Dirk Geeraerts, Stefano De Pascale, Dirk Speelman
code/data similarity 0.56
Abstract

Lexical semantic change (LSC) is commonly modelled through vector-space representations, but these approaches often provide limited insight into which aspects of usage are changing. Diachronic corpus research instead examines interpretable dimensions such as syntactic behaviour, morphology, and constructional patterns, but typically through separate analytical workflows. We present SynFlow, an open-source toolkit for multidimensional diachronic analysis of linguistic usage. SynFlow converts linguistic observations into period-specific distributions and applies a shared workflow across dependency-based co-occurrences, morphological features, constructional configurations, and externally derived representations such as Frame Semantics. It supports different distance measures, together with value-level decomposition, statistical testing, and incremental clustering of lexical fillers. We demonstrate SynFlow through a qualitative case study of the German adjective viral, showing how a single semantic development is reflected across syntactic, lexical, constructional, and morphological dimensions. We further report previously published results on SemEval-2020 Task 1 to situate the performance of these representations relative to existing lexical semantic change detection systems.

0.44

ECOKV: Geometry-Aware KV Cache Eviction via Complementary Diversity Metrics

cs.CL cs.AIcs.CV ·2026-09-06 ·Chin Ting Hsu, Yu-Syuan Xu, Ling Zou, Hsien-Kai Kuo, Wen-Huang Cheng
similarity 0.55
Abstract · 18 pages, 8 figures

Although multimodal Large Language Models (MLLMs) excel in diverse tasks, their scalability remains limited by the memory and computational overhead of KV cache storage. Recent KV cache eviction approaches incorporate a cosine similarity-based diversity metric with importance metrics to selectively retain critical key-value pairs. However, cosine similarity involves normalization that discards magnitude information, and it often yields uniformly high similarity values across layers due to the anisotropy property of hidden representations. In our study ECOKV, we rigorously deconstruct the capabilities of existing diversity metrics. Moving beyond simple measurement, we propose a geometry-aware composite metric that jointly leverages Euclidean distance and cosine similarity to capture token diversity from complementary perspectives. Furthermore, we use these two metrics to estimate the redundancy level of each attention head, allowing adaptive weighting between diversity and importance scores during token selection. Finally, we demonstrate that the observation window commonly employed to preserve recent tokens can be substantially reduced, thereby allocating more cache capacity to informative tokens and yielding consistent improvements. Extensive experiments demonstrate that ECOKV achieves state-of-the-art performance under various compression ratios and can be seamlessly integrated with existing KV cache eviction methods. We further analyze the relationship between importance and diversity, and examine redundancy patterns across layers and attention heads.

0.20

Embedding Models Measure in Peculiar Ways

cs.CL cs.LG ·2026-09-17 ·Juri Opitz, Andrianos Michail
similarity 0.55
Abstract

Embedding spaces define notions of semantic similarity and distance. We study whether those embeddings reflect physical measurements of mass, distance, time and volume, which admit a unique, objective notion of semantic equivalence and distance. We find that physical measurement is only weakly modeled in the embedding space, and that instead quite peculiar measurement patterns can be observed. Further analysis indicates that embedding representations of physical measurements are strongly influenced by superficial string similarity, and recalibration of similarity does not substantially improve the alignment.

0.29