CS Paper Radar

A Hybrid Gaze-Motor Imagery BCI Framework for Effective Decision Communication

Gowtham Reddy N, KongFatt Wong-Lin, Yogesh Kumar Meena

cs.HC cs.AIcs.ET ·submitted 2026-08-22 05:34 UTC · v1
published

Abstract

Non-invasive brain-computer interfaces (BCIs) and eye-tracking technologies offer promising communication pathways; however, motor imagery (MI)-based BCIs often suffer from low discriminability and high inter-subject variability. To mitigate these issues, this study investigates the impact of visual fixation on neural response stability in both standalone MI and hybrid MI-eye tracking systems. We then propose a novel asynchronous hybrid paradigm that streamlines user intent by utilising eye-tracking for direct selection, followed by MI-based confirmation, significantly reducing the operational steps required by conventional systems. The paradigm was evaluated with 15 healthy participants using a 16-channel EEG system. Results show that MI-related information is predominantly localised within motor cortex regions, with limited-channel configurations (SVM: 0.58) achieving performance comparable to full-montage setups (SVM: 0.54). The hybrid MI paradigm further outperforms conventional MI, achieving up to 100% accuracy with greater robustness across all channel configurations. Our findings indicate that visual fixation enhances neural response stability, while integrating eye-tracking with MI enables the development of reliable, scalable multi-command BCI systems suitable for real-world applications.

Comment
6 pages, 5 figures, selected to be presented at Brain-Machine Interface (BMI) Systems Session, IEEE International Conference on Systems, Man, and Cybernetics (IEEE SMC 2026)

Closest papers

Nearest neighbours by meaning across the whole index.

MAESTRO: a Multimodal Auditory-attention Egocentric Speech-TRacking Open corpus

eess.AS cs.HCcs.LGcs.SD+1 ·2026-09-25 ·K M Naimul Hassan, Ali Alavi, Donald S. Williamson
benchmarkcode/datahot direction similarity 0.62
Abstract · 11 pages, 8 figures. Submitted to IEEE Transactions on Audio, Speech, and Language Processing

Humans rely on gaze, head movements, and visual cues to attend to speakers in noisy environments, yet auditory attention decoding (AAD) has been studied primarily using electroencephalography (EEG). We introduce the Multimodal Auditory-attention Egocentric Speech-TRacking Open (MAESTRO) corpus, the first AAD dataset to simultaneously record EEG, eye gaze, pupillometry, egocentric video, and head inertial measurement unit (IMU) data. MAESTRO includes four competing speakers and background noise across multiple signal-to-noise ratio (SNR) conditions, enabling attention decoding under realistic listening scenarios. Through a four-speaker attention decoding benchmark, we show that combining behavioral and physiological signals improves decoding performance over EEG-only approaches, enabling future advances in multimodal auditory attention decoding. These findings open the door to new applications, analyses, and methodological advances in multimodal AAD. The complete dataset is publicly available at https://huggingface.co/datasets/aspire-osu/maestro-eeg-dataset . The official code repository is available at https://github.com/ASPIRE-OSU/MAESTRO .

0.72

Motor Imagery Classification Using Hybrid Cross-Subject Class Mixup and Selective Attention

cs.LG ·2026-11-16 ·Jiwon Kim, Chae-Won Lee, Jong-Seok Lee
similarity 0.61
Abstract

Motor imagery (MI) enables a brain-computer interface (BCI) to decode movement intentions without overt movement. Hybrid electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) classification combines complementary signals, but inter-subject variability and the design of information exchange between modalities affect generalization to new subjects. We propose a framework that combines Cross-Subject Class Mixup (CSCM) with Selective Attention. CSCM aims to construct new training data to make the model robust against inter-subject variability. It independently selects EEG and fNIRS donor trials from other training subjects within the anchor class and mixes each modality while preserving the original label. Selective Attention provides an effective way to simultaneously model the temporal signal characteristics and fuse the two modalities. It retains causal temporal self-attention for EEG and uses a classification token (CLS) to integrate both modalities, without fNIRS attention updates or direct cross-modal temporal-token attention. Leave-one-subject-out (LOSO) evaluation across 28 subjects on a public EEG-fNIRS dataset yields a mean balanced accuracy (BA) of 76.85% and a macro-averaged F1 score (Macro-F1) of 76.58%. The proposed model achieves the highest mean performance among five reimplemented baselines. We also examine the contributions of CSCM and Selective Attention through modality-specific mixing and donor-policy comparisons, temporal-policy comparisons, interventions on direct cross-modal paths, and component ablations.

0.11

GALA: Gallery-Level Alignment for Real-Life Vision Decoding from EEG

cs.AI cs.LG ·2026-11-18 ·Philippe Forest, Giulia Lioi, Mai Pham Quyen, Alix Lamouroux, Javen Shi +1 more
benchmark similarity 0.61
Abstract

Neural decoding of visual experience from electroencephalography (EEG) has achieved strong performance in intra-subject settings, but current methods fail to generalize across individuals: on the standard 200-way benchmark THINGS-EEG-2, top-1 accuracy drops from above 80% within subjects to about 32% for an unseen subject. Adapting a pre-trained encoder to new subjects and sessions typically requires labeled data, a bottleneck whose collection time severely limits practical device deployment. In a practical Brain-Computer Interface (BCI), however, the possible outcomes are known in advance and form a gallery of candidates, and the unlabeled EEG of the user accumulates as soon as the device is used. We propose a new setting, incremental transductive calibration, in which the decoder is calibrated regularly on all the unlabeled EEG received so far and on the known gallery, without any labels. This setting is closer to the conditions of a deployed BCI than the zero-shot and batch evaluations in current use. We evaluate it with a protocol in which queries arrive one at a time, with unknown and possibly unequal frequencies, and each query is decoded before it is used for calibration. In this setting, we introduce GALA : GAllery-Level Alignment, a label-free calibration method which registers the unlabeled EEG embeddings onto the gallery through shrinkage whitening, hubness correction and a Sinkhorn--Procrustes alignment with query frequencies estimated from the buffer, updated after every query. On THINGS-EEG-2, with a gallery of 200 images, it raises the top-1 accuracy on a new subject from 28% to 67% over 53 minutes of use. Across 60 evaluation settings that vary the gallery size, the query frequencies and the number of repetitions averaged per query, GALA is the most accurate label-free method in every setting, 6.6 points above the best competing method on average at the end of the stream.

0.26

Augmenting Human Performance with an XR Agent Learning from Online Behavior and BCI Evidence

cs.AI cs.HC ·2026-08-31 ·Ziheng Li, Xichen He, Haoyan Chen, Charlie Zou, Sheng Bai +8 more
top venue similarity 0.61
Abstract · To appear in ACM UIST 2026. 30 pages, 23 figures

We present OLIVE, a framework for adapting a foundation model to provide real-time assistance in temporally demanding, high-stakes, and dynamic tasks. We show that passive EEG, fused online with behavioral evidence, can meaningfully extend the number of targets users detect and engage beyond their unaided action bandwidth. OLIVE learns from both explicit behavioral signals (the targets the user shoots down in an XR first-person shooter game) and implicit physiological signals (fixation-locked EEG) to provide timely guidance, continuously adapting a frozen vision-language model's inference on which items are task-relevant by jointly estimating per-source reliability without manual labels or offline training. Through three user studies, including two live deployments of an assistive agent driven by OLIVE in XR, we show that OLIVE Pareto-dominates prior test-time adaptation frameworks, achieving the highest convergence rate at comparable convergence speed. Combining implicit physiological and explicit behavioral signals, the OLIVE agent produces the largest and most reliable within-session improvement to a user's ability to detect and engage targets, largely independent of the individual's skill. When the target switches silently, the agent that uses both behavioral and physiological signals reconverges significantly faster than the behavior-only agent (1.27 times faster on average, p = .008), restoring trustworthy guidance at the moment the task changes, precisely when reliable assistance matters most.

0.48

Bio-MF: Low-Latency and High-Fidelity EEG-to-fNIRS Cross-Modal Generation for Hybrid Motor-Imagery Brain--Computer Interfaces

cs.LG cs.AI ·2026-09-17 ·Boyuan Zhao, Sifan Zhang, Luping Chen
code/data similarity 0.60
Abstract · 10pages

Hybrid motor-imagery brain-computer interfaces (MI-BCIs) combining EEG and fNIRS can outperform EEG-only systems by exploiting complementary electrophysiological and hemodynamic information. To obtain such hybrid information when paired EEG-fNIRS acquisition is unavailable or inconvenient, recent studies have focused on EEG-to-fNIRS cross-modal generation. However, existing methods still suffer from slow generation and often require pretraining, limiting their use in real-time MI-BCI scenarios. Although one-step generative models offer an attractive route to low-latency synthesis, removing the iterative refinement process can reduce generation fidelity and introduce non-physiological artifacts. To address these problems, this paper proposes Bio-MF, a latent-free one-step MeanFlow framework for EEG-conditioned fNIRS generation. Bio-MF performs direct signal-space x-prediction, converts this signal-space output into MeanFlow velocity supervision, and completes inference with one network evaluation. To preserve task-relevant hemodynamic structure under heterogeneous sensor layouts, Bio-MF integrates Spatial-Temporal Interactive 4D Encoding, cross-modal classifier-free guidance, and noise-level-gated FFT regularization. On Dataset 1, EEG + synthetic fNIRS improves ACC over EEG-only by 3.37 and 4.15 percentage points for HbR and HbO, respectively. On Dataset 2, the corresponding gains remain 2.98 and 2.50 percentage points under the unseen 64-channel EEG montage. On an RTX PRO 6000 GPU, Bio-MF generates one fNIRS trial in 7.0 ms, corresponding to an 857x speedup over the 1000-step SCDM latency. These results show that Bio-MF enables fast EEG-to-fNIRS synthesis while preserving task-relevant generation quality for downstream hybrid MI decoding. Our code is available at https://github.com/psychosiwa/Bio-MF.

0.36

Representational and Functional Robustness to Electrode Montages in EEG Foundation Models

cs.LG ·2026-09-28 ·Jakob Steglich, Justus Meyer zu Bexten, Shakiba Moradi, Laure Ciernik, Simon M. Hofmann +1 more
similarity 0.59
Abstract · 17 pages, 9 figures

EEG foundation models (EEG-FMs) are intended to generalize across different datasets by learning representations that, ideally, are invariant to dataset-specific EEG configurations such as electrode montages. However, EEG-FMs that accept different montages as input do not guarantee that representations and predictions remain stable across different electrode configurations, especially outside the training setting. In this work, we investigate the effects of different electrode montages through a joint functional and representational analysis of four EEG foundation models selected to span distinct montage-handling designs. We evaluate embeddings on cross-subject resting-state eyes-open/closed and within-subject motor-imagery classification under spatially informed channel reduction. Functional robustness is tested through the generalizability of linear probes across channel counts, while representational robustness is assessed through within-subject similarity and preservation of between-subject geometry. The four models show distinct robustness profiles, and the two axes dissociate: large changes in embedding similarity need not come with comparable probe degradation, and stable embeddings can still lose downstream performance. Comparing two readouts of the same encoder further shows that aggregation, not the encoder alone, determines functional robustness: pooling into anatomically aligned regions degrades less than a learned global readout, despite being montage-invariant by construction. Montage robustness is therefore a joint property of the encoder and its aggregation, and characterizing it requires both a representational and a functional axis. Input compatibility alone is evidence for neither.

0.44

Toward Neuroadaptive Communication Interfaces: A Framework for Personalized Speech and Language Support

cs.HC cs.CL ·2026-10-25 ·Carlos Bruno, Luis Ricardo Santos
similarity 0.59
Abstract

Speech and language technologies have expanded access to communication, but the support they provide is often fixed. The same linguistic or visual aid may help one user and hinder another, or become less useful as task demands change. This project proposes a framework for adapting language presentation to individual abilities and ongoing interaction. It separates access to a message from the estimation of user needs and from the selection of support, including sentence segmentation, pictograms, and optional spoken output. Behavioral feedback provides the primary evidence for adaptation; electroencephalography is considered an additional, uncertainty-aware source of information rather than a direct measure of comprehension. We specify a staged evaluation comparing fixed, behavior-adaptive, and neuroadaptive interfaces. The central question is whether neural information improves comprehension and communication efficiency beyond a strong behavioral baseline.

0.12

Lightweight Adaptation of EEG Foundation Models for Stroke Motor Imagery Decoding: Domain Shift and Subject-Level Robustness

cs.CE cs.LG ·2026-08-31 ·Anh T. Nguyen, Zihua Sun, Michelle J. Johnson
benchmark similarity 0.58
Abstract

Motor imagery (MI) electroencephalography (EEG) decoding could support post-stroke rehabilitation, but models developed on healthy cohorts may not transfer reliably to pathological EEG. We evaluated whether Low-Rank Adaptation (LoRA) can efficiently adapt three pretrained EEG foundation models (i.e., LaBraM-base, REVE-base, and REVE-large) for binary left- versus right-hand MI decoding. Frozen-backbone head-only baselines and LoRA adaptation were evaluated using subject-wise five-fold cross-validation on the PhysioNet EEG Motor Movement/Imagery Dataset and a binary subset of the UET175 dataset comprising 30 stroke participants. On EEGMMIDB, LoRA increased accuracy to 0.822 for LaBraM-base and 0.957 for REVE-base. On UET175, all head-only models performed near chance. With LoRA, LaBraM-base remained near chance (0.499$\pm$0.009), whereas REVE-base reached 0.847$\pm$0.194 and outperformed REVE-large (0.806$\pm$0.178), indicating that increased model capacity alone did not improve stroke-domain adaptation. The strongest stroke configuration, REVE-base LoRA, was further evaluated using within-cohort leave-one-subject-out cross-validation (LOOCV), showing 0.952 mean accuracy, but subject-wise accuracy ranged from 0.586 to 1.000, revealing a small low-performing tail. Zero-shot transfer from EEGMMIDB to UET175 remained near chance (0.464$\pm$0.072). These findings show that healthy-benchmark performance does not ensure transfer to stroke EEG. Translation of EEG foundation models to pseudo-online or real-time rehabilitation BCIs should therefore include target-domain adaptation and subject-level assessment of temporal informativeness, spatial sensitivity, and physiological discriminability.

0.28