CS Paper Radar

Soft Active Electromyography Interface for Machine Learning-Enabled Silent Speech Recognition

Yuta Kurotaki, Shusuke Yamakoshi, Reitaro Yoshida, Yutaka Isoda, Tamami Takano, Yuji Isano, Yusuke Miyake, Kentaro Kuribayashi, Hiroki Ota

cs.LG cs.HCcs.SD ·submitted 2026-08-27 12:36 UTC · v1
publishedunusual

Abstract

Silent speech recognition (SSR) provides an alternative communication pathway in the absence of audible speech. However, conventional approaches are limited by the need for constant facial attachment, privacy concerns, and unstable signal acquisition. Here, we propose a soft, active electromyography (EMG) interface that enables word-level SSR using machine learning. Worn on the hand, the device uses a fingertip electrode that can be positioned near the lips to acquire EMG signals only when needed. The interface integrates liquid metal (LM) interconnects, transparent flexible printed circuit (FPC) electrodes, and elastomer encapsulation to ensure high mechanical stability during finger motion. A deep neural network trained on these stable signals achieved a mean accuracy of 97.2 $\pm$ 1.3% across three subjects in classifying a 30-word vocabulary, demonstrating robust linguistic discrimination. Furthermore, real-time drone control validates the practicality of this approach in noisy and privacy-sensitive environments where conventional voice recognition fails. This study highlights the potential of soft, wearable EMG systems as secure and intuitive human-machine interfaces.

Comment
17 pages, 5 figures, supplementary information
Journal ref
Advanced Intelligent Systems 2026, 0, e70440
DOI
10.1002/aisy.70440

Closest papers

Nearest neighbours by meaning across the whole index.

Data-Centric Neuromotor Interfaces for Portable Human-Machine Interaction

cs.RO ·2026-08-31 ·Jiaxuan Li, Di Wu, Jianhua Liu, Yuxin Zhao, Jinnuo Li +5 more
publishedbenchmark similarity 0.64
Abstract

Dexterous human-machine interaction requires intuitive and expressive interfaces that can be efficiently deployed on constrained edge devices. Flexible material-based neuromotor interfaces hold considerable promise, as they decode human movement intention into natural control. Although emerging flexible electronic skins enable wearable high-fidelity data acquisition, practical deployment inevitably involves trade-offs between computational resources and portability. We present a data-centric paradigm where physiological features yield fundamental separability, providing sufficient discriminative cues for recognition. A wireless, high-bandwidth system developed for collecting various electrophysiological signals, when integrated with muscle-specific electrodes, forms a surface electromyography-based interface. Exploiting highly separable data, a 2,210-parameter model achieves 94.36% accuracy across 34 gestures and can be rapidly deployed on edge devices, establishing a new thousand-parameter benchmark for dexterous decoding. The underlying data-algorithm interactions in the data-centric paradigm are further clarified, demonstrating its feasibility in real-world scenarios. This study provides a principled and validated pathway for practical deployment of reliable neuromotor interfaces.

0.22

Tactile Perception through Fluid-Solid Interaction

cs.RO cond-mat.soft ·2026-09-29 ·Arman Goshtasbi, Minke Berghuis, Aida Parvaresh, Saravana Prashanth Murali Babu, Robert W. Style +1 more
top venue similarity 0.57
Abstract

Soft tactile sensors elevate robotic touch through enhanced flexibility and adaptability, yet most existing designs depend on embedded electronics that are susceptible to interference and environmental limitations. In this work, we leverage fluid-solid interactions to develop a class of soft tactile sensors that operate entirely without electronics at the sensing site. The sensor comprises a fluid-filled elastomeric channel connected to only two external pressure sensors. Touching different regions of the elastomeric surface displaces the viscous fluid, producing distinct pressure patterns that encode both touch position and force. These signals are decoded through a machine learning framework that integrates feature extraction, soft clustering, and adaptive neuro-fuzzy inference to achieve accurate localization and force estimation. We validate this concept through single-point touch localization and force estimation in a linear (1D) sensor and extend the same sensing principle to 2D tactile mapping by routing the channel across the surface using space-filling curves, while maintaining the same minimal hardware setup. This simple approach remains effective in environments where conventional electronic sensors often fail, such as underwater or in the presence of magnetic interference.

0.56

Feature extraction for LSTM-based finger-joint angle estimation from sEMG

cs.LG ·2026-11-02 ·Gerda de Jager, Marelie H Davel, Ian Thomson
similarity 0.57
Abstract · 20 pages, 8 figures

Continuous estimation of finger-joint angles from surface electromyography (sEMG) is a promising approach for controlling dexterous myoelectric prostheses. Deep time series models have been applied successfully in this context, typically relying on feature extraction rather than raw signals, given limited subject-specific data. Various time-domain and frequency-domain features can be used, extracted for a given window and step size. This study systematically investigates these three design choices for Long Short-Term Memory (LSTM) regression on the Ninapro DB2 dataset. Window sizes from 300 to 1200 ms and step sizes from 10 to 100 ms were evaluated in combination with six time- and frequency-domain features. Models were evaluated using regression performance metrics, inference time, and memory footprint to assess feasibility on embedded microcontroller hardware suitable for a prosthetic device. A 300 ms window with a 100 ms step, paired with either the Mean Absolute Value (MAV) or Root Mean Squared (RMS) feature, yielded the best performance (test PCC of 0.779 +/- 0.080 / 0.779 +/- 0.085). Window size, step size, and feature choice were found to interact non-trivially: jointly tuning them can improve both accuracy and computational feasibility for real-time, embedded myoelectric control.

0.28

MotorSense: A Video-EMG Dataset of Motoric Representations for Action Understanding

cs.CV cs.RO ·2026-10-30 ·Eadom Dessalene, Michael Maynord, Amir-Hossein Shahidzadeh, Botao He, Yasamin Tabatabaee +4 more
publishedcode/data similarity 0.57
Abstract · NeurIPS 2026. 27 pages, 9 figures, 5 tables. Project page: https://www.cs.umd.edu/~edessale/motorsense/

Computer vision models learn hand action only from its visual consequences, without access to the underlying embodied motor signals. We address this absence with MotorSense, the first large-scale paired egocentric video and surface electromyography (EMG) dataset. MotorSense pairs bimanual EMG bands with egocentric video and yields synchronized video, bimanual EMG and IMU, and audio across 100 hours of unscripted manipulation. EMG offers a powerful scalable alternative to manual annotation: it captures finger movement, hand configuration, and the forces exerted between hand and environment as a dense signal free of occlusion and labeler bias, whereas manual annotation is costly, subjective, and unreliable under occlusion. We use MotorSense to learn sensorimotor alignment, producing an organization of action that is shared across the visual and motoric spaces, through cross-modal pre-training between video and EMG. The resultant vision models are then transferred to downstream tasks that benefit from sensorimotor alignment: in this case, action recognition, 3D hand reconstruction, and 3D contact reconstruction. We further demonstrate a generative application in which EMG motor signals are used to condition a video diffusion model, yielding videos of fine-grained hand manipulation. Across four action-centric computer vision tasks and eight popular video datasets, we demonstrate that sensorimotor alignment provides a stronger foundation for visual representation learning than conventional self-supervised learning methods over video, outperforming every baseline across all tasks and datasets evaluated.

0.28

LiteEMG-FM: An Efficient and Deployable Foundation Model for Robust EMG Sensing

cs.LG ·2026-10-01 ·Tianhao Wu, Xu Wu, Amirmohammad Radmehr, Jiawei Yu, Yi Wu +2 more
similarity 0.56
Abstract · The authors are withdrawing this manuscript because the current experimental evaluation and validation are insufficient to adequately suppor…

Electromyography (EMG) signals vary substantially across individuals, body regions, recording sessions, and sensing hardware, limiting the generalization of models for assistive devices and human-computer interaction. Existing time-series foundation models are also computationally expensive for real-time wearable deployment and often fail to capture EMG-specific time-frequency characteristics. We present LiteEMG-FM, an efficient hybrid CNN-Transformer foundation model for practical EMG sensing. Pretrained on 16 diverse upper- and lower-limb EMG datasets, LiteEMG-FM learns representations that generalize across users and datasets. For resource-constrained deployment, we implement a hierarchical wake-up architecture in which a lightweight, always-on 1D-CNN filters rest and non-target activity and activates LiteEMG-FM only for valid gestures. We evaluate full inference offloading, split inference, and full on-device processing, characterizing their trade-offs in latency, power consumption, and memory footprint. Across diverse evaluation settings, LiteEMG-FM outperforms state-of-the-art time-series foundation models and supervised baselines, particularly under zero-calibration cross-participant and data-scarce conditions. These results demonstrate that LiteEMG-FM is an effective, efficient, and deployable foundation model for EMG applications.

0.11

Wearable Multimodal Human-Machine Interface for Integrated Hand Intentions Decoding in Dynamic Teleoperation

cs.RO ·2026-09-07 ·Jiaxuan Li, Yinshi Wu, Xiao Zhang, Hongyu Wang, Renzhen Le +2 more
similarity 0.56
Abstract

Under ubiquitous teleoperation environments with optically challenging conditions, an interface for tele-operated grasping that combines wearability with precise decoding of hand intentions (hand pose, gestures, and grasping force) is essential. Yet, existing interfaces often fall short in meeting these demands, compromising either the diversity of multiple intentions decoding or wearability. To address this, we developed a novel Multiple Intentions Decoding Human-Machine Interface (MI-DHMI) that integrates high-throughput surface electromyography (sEMG) sensors with hand-mounted and forearm-mounted inertial measurement units (IMUs). The developed interface is supported by a unified framework for simultaneous multiple intentions decoding. By employing multimodal deep learning and hardware design with a low noise floor, the decoding framework selectively focuses on the sEMG components that are genuinely associated with finger movements. This effectively reduces decoding errors caused by sEMG variability during unconstrained upper-limb motions, thereby significantly enhancing robustness. Even under unconstrained wrist and forearm motion, the interface achieves a gesture recognition accuracy exceeding 97%, grasping force estimation with $R^2 = 0.95$, and hand pose decoding consistent with the actual hand pose, outperforming baseline devices and algorithms. Ablation studies further validate the effectiveness of the proposed decoding framework. Finally, two online experiments were conducted to validate the device, demonstrating its superior performance in high-stability tasks, including a pouring task and object grasping. The developed interface provides a new solution of a fully wearable, multiple intentions decoding system, offering effective support for ubiquitous teleoperation and contributing to the advancement of human-machine interaction research.

0.27

A spinal-inspired spiking neural network for continuous finger-joint kinematics estimation from high-density surface EMG

cs.HC ·2026-10-25 ·Cristian David Guerrero-Mendez, Leonardo Abdala Elias
similarity 0.54
Abstract · 32 pages (25 pages of main text and 7 pages of supplementary material), 9 figures, 1 table

Decoding simultaneous and proportional finger kinematics from high-density surface electromyography (HD sEMG) remains challenging. Although data-driven methods have shown promising performance for finger movement decoding, their limited physiological interpretability remains a relevant limitation. This study proposes and evaluates a spiking neural network (SNN) that incorporates spinal-inspired structures and mechanisms to predict multi-joint finger kinematics from HD sEMG signals. The spinal-inspired SNN includes a spike-encoding computational model with input-noise variability, a phenomenological representation of Henneman's size principle, and excitation/inhibition (E/I) balance. HD sEMG and kinematic data were collected from 23 healthy participants performing grasping tasks under different elbow and wrist postures. Performance was assessed using six prediction metrics and compared against six baseline machine learning and deep learning methods. The proposed method showed a large and significant ($p<0.05$, rank-biserial correlation $r_{rb}>0.7$) improvement over all baseline methods across all accuracy metrics. Performance improved significantly with increasing spike encoder firing rates up to approximately 83 Hz, and with more regular input spike trains compared with a Poisson-like encoder. In contrast, the E/I balance did not significantly affect the decoding performance of the algorithm. The firing rate and inter-spike interval statistics of the SNN differed significantly across motor tasks. These results demonstrate that incorporating neurophysiological principles into SNNs improves prediction performance while providing additional physiological interpretability for simultaneous and proportional finger-kinematics decoding. This approach represents a promising step toward physiology-informed neural networks, with potential applications in motor rehabilitation.

0.23

A Co-Located MMG-FES Wearable Interface for Action-Updated State-Gated Stimulation

cs.HC ·2026-10-11 ·Cong Liu, Xiao Jin, Zhenhua Yu
similarity 0.53
Abstract · 9 pages, 8 figures. Submitted to IEEE Robotics and Automation Letters

Functional electrical stimulation (FES) assists movement, but fixed schedules cannot track how a repeatedly activated muscle changes, and surface electromyography (EMG) near the stimulation electrodes is often obscured by artifacts. Prior work recorded mechanomyography (MMG) passively during closed-loop FES cycling and estimated FES-induced fatigue or predicted fatigue for FES control, without gating stimulation with stimulated-site MMG. In this study, we present VG-FES, a wearable module co-locating microphone-based MMG, programmable charge-balanced biphasic stimulation, and embedded state-gated control at one on-skin thigh site. After each action, we compare rectus femoris (RF) mean power frequency (MPF) with an early-session baseline of the same direction, giving a relative state. A threshold on its window means enables fixed-duration bursts separated by rest periods. During onboard stimulation, MMG remained recordable, whereas near-site EMG was artifact-dominated. Under prolonged external stimulation, cycle-wise MMG tracked an independent local-pressure reference (Spearman rho = 0.968). In leave-one-session-out evaluation on 2284 squat actions (11 sessions, eight participants), MMG-based action segmentation reached an event-detection F1 of 0.923 and direction classification a balanced accuracy of 0.925. Online, over 15.69 min of self-paced squats, one participant received four bursts while MMG acquisition and action monitoring continued. Mechanical feedback from the stimulated site offers wearable FES and assistive robotics a per-action basis for state-gated stimulation.

0.40