CS Paper Radar

Contrastive Neural Embeddings Reveal Individual Traits Beyond Conversational Role

Hubert Huang, Michelle McCleod, Brendan Ames, Evie Malaia

cs.LG q-bio.NC ·submitted 2026-10-02 15:00 UTC · v1
unusual

Abstract

Contrastive representation learning is increasingly used to recover low-dimensional structure from neural recordings, but its output is typically validated by decoding accuracy rather than by the geometry of the manifold it produces. We apply CEBRA to EEG recorded from dyads in conversation, and analyze the resulting embedding, which training constrains to the 2D sphere. Labels describing the dyads, including the absolute difference between partners' autism-quotient scores, decode well above chance (0.77 against a 0.55 majority baseline for binary AQ magnitude; 0.44 against 0.25 for the six-class $|Δ$AQ$|$ partition). However, the two permutation controls have notable differences in results: permuting labels over a frozen embedding yields p = 0.001, whereas retraining the encoder under each permutation yields p = 0.50. Only the latter tests the label rather than the geometry. Consistent with this, spherical mixture structure and per-class dispersion track identity rather than autism trait differences in dyads; frequency-band and non-oscillatory activity ablation controls do not change the results. However, participant-level model does separate from its identity-aware null (p = 0.0099) while speaker-versus-listener role analysis performs at chance in the same embedding, indicating a manifold organized by individual -- and, in contrast with current neurolinguistics models, almost invariant to speaking vs. listening. Based on these results, we suggest that retraining-based nulls should be the default for grouped-data contrastive embeddings.

Closest papers

Nearest neighbours by meaning across the whole index.

Alignment De-Identifies Without Aligning: A Congruence-Invariant Relational Gap in Cross-Subject Neural Representations

cs.HC q-bio.NC ·2026-11-11 ·Anna Notaro, Letizia Rossato, Sadasivan Puthusserypady
published similarity 0.59
Abstract · 23 pages (12 pages main text and references, 11 pages appendix), 2 figures. Accepted at NeurIPS 2026, NeurReps Workshop

In brain-computer interfaces, inter-subject variability remains a major challenge that limits decoding performance across users. Subject-wise alignment is widely used to improve cross-subject transfer of neural decoders, yet it is typically evaluated only through decoding accuracy, which cannot determine whether aligned representations share greater task-related structure. We therefore distinguish two questions that accuracy conflates: whether alignment removes subject-specific information and whether it makes the relational organization of task conditions more similar across individuals. Using three motor imagery-EEG cohorts, trials are represented as covariance matrices and aligned by recentering each subject's mean covariance to a common reference. Subject-specific information is quantified with a subject-identification probe, whereas relational structure is assessed using representational dissimilarity matrices (RDMs) computed from condition means with the affine-invariant Riemannian distance. We show that recentering leaves this distance, and therefore the resulting RDMs, unchanged. Consistent with this result, recentering reduces subject identification accuracy from 0.96 to chance level across all cohorts, while altering RDMs only at numerical precision. Cross-subject agreement in relational structure is significantly above zero but below the estimator's lower noise ceiling. Introducing an explicit similarity regularizer during training also fails to increase held-out agreement: the apparent reduction in between-subject differences is explained by a decline in the reliability ceiling rather than improved alignment. These findings show that eliminating subject identity is not equivalent to aligning relational geometry. More broadly, alignment claims must be interpreted relative to the invariances of both the distance and the statistic used to evaluate them.

0.33

Semantic Alignment of EEG and Text: A Contrastive Learning Framework for Decoding Imagined Speech

cs.LG q-bio.NC ·2026-10-14 ·Shahryar G. Namdari, Francesco Iacomi, Riccardo Barbieri, Luca Mainardi
similarity 0.59
Abstract

Imagined speech decoding from EEG offers a promising route toward naturalistic brain-computer interfaces. Traditional closed-set classifiers treat semantic categories as independent labels and do not explicitly exploit relationships among linguistic intents, limiting output flexibility. We adapt a multimodal contrastive learning framework to imagined speech, aligning EEG trials with language representations in the embedding space of a frozen pretrained text encoder. An EEG encoder is trained with a composite objective combining semantically reweighted EEG-sentence alignment and text-prototype-based class discrimination, encouraging semantic structure while preserving separability among 39 intent categories. We evaluate the approach on the Chisco imagined speech dataset across five subjects, reporting Top-k accuracy, macro-F1, and balanced pairwise matching accuracy (2v2). We also investigate maximum EEG-text cosine similarity (MaxCosine) as an intrinsic indicator of decoding reliability and probe semantic structure beyond category labels using five evaluation-time semantic axes and a qualitative Semantic Gallery Probe (SGP) with previously unseen candidate sentences. The contrastive framework achieved decoding performance comparable to a task-optimized closed-set classifier while mapping EEG into a frozen language embedding space. MaxCosine identified subsets of held-out trials with higher Top-1 and Top-5 accuracy across subjects. Evaluation-time semantic axes showed mean agreement above the 50% chance reference across all five dimensions, and the SGP qualitatively illustrated direct comparison with unseen candidate sentences. Together, these findings suggest that EEG-language alignment can retain comparable fixed-intent decoding while supporting similarity-based retrieval, intrinsic reliability ranking, and evaluation of broader semantic structure.

0.09

NormMoX: A Normative Mixture of Experts for Explainable Autism Classification from Structural and Functional MRI

cs.LG ·2026-11-02 ·Muhammad Hasan Masrur, Fani Deligianni
code/data similarity 0.58
Abstract · 5 pages, 3 figures, 2 tables

While structural and functional magnetic resonance imaging (sMRI, fMRI) offer profound potential for researching autism spectrum disorder (ASD) and understanding individual neurodivergence, current classifiers lack the explanatory power needed for scientific study, clinical use and personal-level insights. Most models learn from imaging features that also vary with age, sex, scanner site and head motion. They fuse the two modalities without a common reference and explain decisions against arbitrary reference inputs rather than a typical brain. We present NormMoX, a normative mixture of experts that makes the typical brain the shared reference for prediction, fusion and explanation. Normative models trained on large healthy cohorts give the value each brain measure is expected to take for a person's age and sex. NormMoX learns from each person's deviations from these expected values. For each modality, a neural network expert turns the deviations into a score of evidence for ASD. A fusion head, MoX, compares each score with those of typical controls before combining them. It therefore needs no imputation when a modality is missing and shows in which modality each person is atypical. On 1,035 participants of ABIDE I, NormMoX reached an AUROC of $0.728 \pm 0.004$ in repeated cross-validation, above linear, single-modality and deep fusion baselines. NormMoX thus shows not only whether a person is likely to have ASD, but where, and in which modality, their brain departs from typical. The implementation is available at https://github.com/Novelex/NormMox.

0.36

Multi modal perceived conversation success prediction for spontaneous autistic and non-autistic dyadic interactions

cs.CL cs.LG ·2026-10-02 ·Thanushi Withanage, Desi Jones, Noah Sasson, Elizabeth Redcay, Carol Espy-Wilson
similarity 0.58
Abstract · Submitted to ICASSP 2027

Conversation success is important for communication, social bonds, and well-being and is often reduced in autistic-neurotypical conversations. Previous studies have predicted perceived conversation success using acoustic and lexical features in pediatric autism-enriched samples, or using nonverbal behavior in non-autistic group interactions. Other work has characterized prosodic, facial, motor, and turn-taking synchrony in autistic interactions, but has primarily targeted diagnostic classification or group differences. The present study instead predicts participant perceived conversation success from complementary articulatory, facial, acoustic-prosodic, and turn-taking measures across three spontaneous dyadic-interaction datasets including autistic and non autistic individuals. By leveraging multimodal features, we demonstrate that conversational success is inherently multidimensional and requires a multi-faceted analytical framework. Combining these complementary modalities offers the potential to improve prediction of perceived conversation success by capturing information unavailable from any single modality.

0.30

Quantifying Trait-State Hierarchy in EEG Foundation Model and Signal Processing Embeddings

cs.AI cs.LG ·2026-11-03 ·Anusha Madapura, Aditya Kommineni, Shrikanth Narayanan
similarity 0.58
Abstract

Electroencephalography (EEG) measures brain activity through recording electrical potentials of the scalp. EEG recordings capture two distinct sets of characteristics in the signal: (1) \emph{Traits} -- information that remains relatively constant over the course of a recording session, such as subject identity, age, and sex; (2) \emph{States} -- signatures that encode responses to external stimuli, including visual, auditory, and motor information. There is limited understanding of how different EEG modeling features (signal processing and foundation models) encode state and trait characteristics. In this work, we quantify state- and trait-level features captured by EEG embeddings using a hierarchy metric and kNN purity on Brain-Computer Interface dataset trials, and find that a majority of EEG features form a macro-trait hierarchy (embeddings form trait-level clusters with micro-states within). Our linear probe results indicate that this hierarchy structure leads to suboptimal state distinguishability, negatively impacting BCI performance of EEG foundation models. To mitigate this, we explore normalization techniques that reduce trait-specific information in embeddings and provide a $\sim$ 10\% improvement in balanced accuracy for state classification. Additionally, we observe that when fine-tuning models for downstream state decoding, embedding spaces transform from macro-trait to macro-state hierarchies, which in turn lead to better state decoding.

0.13

Articulatory Entrainment and Coordination Complexity in Spontaneous Autistic and Non-autistic Dialogue

cs.LG ·2026-10-02 ·Thanushi Withanage, Carol Espy-Wilson, Elizabeth Redcay, Desi Jones, Noah Sasson
top venue similarity 0.56
Abstract · Accepted to be published in Interspeech 2026

Articulatory entrainment, the adaptation of vocal tract coordination to facilitate interaction remains underexplored in spontaneous dialogue, particularly among autistic speakers. Many prior studies have utilized task-based, phoneme-level analyses with invasive measurement techniques. Here, we introduce a speaker-independent framework to quantify articulatory entrainment in spontaneous dyadic conversations among autistic and non-autistic adults using acoustic-to-articulatory inversion and coordination complexity metrics. We examine temporal changes in articulatory coordination across interaction. Non-autistic dyads exhibit increasing coordination complexity and stronger entrainment over time, autistic dyads show moderate effects, and mixed dyads demonstrate the least alignment. Greater articulatory entrainment correlates with higher self-reported conversational success, indicating increase in coordination complexity as a marker of effective social interaction.

0.68

Language processing in the autistic brain: A meta-analysis of neuroimaging studies across modalities and age

cs.CL q-bio.NC ·2026-11-09 ·Hanlin Wu, Bei Xiao, Qingxi He
similarity 0.56
Abstract · 51 pages, 10 figures

Language is central to social communication, which is altered in autism, and language abilities vary widely among autistic individuals. How the autistic brain processes language is central to understanding autistic communication. Yet neuroimaging studies report inconsistent findings on how autistic brain activity is altered during language processing. We synthesized 45 fMRI and PET studies (753 autistic and 777 non-autistic participants) with seed-based d mapping with permutation of subject images (SDM-PSI). We estimated autistic and non-autistic activation and their difference with meta-regressions adjusted for each study's language modality and mean age. Autistic and non-autistic brains activated a similar left-lateralized language network. Autistic brains showed reduced activity throughout the left perisylvian cortex, from the inferior frontal gyrus to the mid superior temporal gyrus and the posterior superior temporal sulcus, and increased activity in the right temporal pole. Spoken language showed a larger autistic reduction than written language in the left mid superior temporal gyrus. Adults, but not children and adolescents, showed increased autistic activity in the right superior temporal gyrus and cuneus. Autistic language processing thus relies on the typical left-lateralized network but shows reduced leftward lateralization.

0.26

Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology

cs.AI ·2026-09-29 ·Junbo Shen, Jinying Gao, Bo Lei
code/data similarity 0.56
Abstract · Code: https://github.com/ai4biomedicine/LangPatch

Integrating transcriptomic and electrophysiological data is essential for building multimodal foundation models for neuroscience. Patch-seq provides paired measurements of gene expression and intrinsic electrophysiology from the same neuron, establishing a basis for training cross-modal models. Here we introduce LangPatch, a foundation-model-based contrastive learning framework that uses paired Patch-seq data to align pretrained GenePT representations with electrophysiological phenotypes through a language-based interface. Gene descriptions and verbalized electrophysiological profiles are embedded by the same frozen text encoder. A context adapter and projection modules connect the modalities through paired contrastive learning. Across mouse visual, mouse motor, and human cortical cohorts, LangPatch achieves the highest mean transcriptome-to-electrophysiology prediction correlation among the evaluated foundation-model and representation-learning methods. It also improves held-out cross-modal alignment in the two mouse cohorts (FOSCTTM 0.107/0.135 vs. 0.208/0.222 for JAMIE, an existing cross-modal Patch-seq imputation method). It predicts transcriptomic family, type, cortical layer, and marker-gene expression from electrophysiology, exceeding other baselines on most endpoints. More importantly, the method transfers across brain areas and species: a model trained on mouse visual cortex predicts electrophysiology in motor cortex with approximately 70% correlation retention and in human cortex with 47% (58% on acute-slice recordings). Together, these results demonstrate alignment between molecular and functional representations of neurons, providing a building block for multimodal foundation models in neuroscience.

0.38