CS Paper Radar

Pod-wise Multimodal Federated Learning for Heterogeneous IoT

Joy Okolo, Zihao Ding, Jun Huang

cs.NI ·submitted 2026-11-02 15:15 UTC · v1

Abstract

Federated learning (FL) is well suited to Internet of Things (IoT) sensing. However, devices carry different sensors and come online in periodic patterns, and as such, the sensors among the participants change from round to round. Under naive averaging, devices without a sensor overwrite its parameters, and a single-sensor device never learns how its sensor relates to the others. Existing methods either redesign local training for mismatched sensors (assuming always-available devices) or select clients without regard to their sensors. We propose Tier-aware Intra-pod Decoupled Encoders (TIDE). Specifically, each sensor has its own encoder and auxiliary head, whose parameters are merged only within the "pod" (i.e., the devices that hold the sensor), while the fusion layer and the task head remain shared by all devices. On three multimodal activity recognition datasets against eight baselines, TIDE reaches 78.06% on ActionSense, 7.63 points above the next method, is the most accurate on all three datasets at half its own communication, and at equal communication leads on ActionSense and is ahead within seed variation on the other two.

Closest papers

Nearest neighbours by meaning across the whole index.

FedMHAR: Federated Multimodal Human Activity Recognition using Multi-Agent Reinforcement Learning

cs.AI ·2026-09-27 ·Debasmita Dey, Tanmay Sen, Himel Mallick
similarity 0.67
Abstract

Human Activity Recognition (HAR) from heterogeneous wearable sensors is fundamental to the Internet of Health Things (IoHT), supporting rehabilitation, elderly care, and smart healthcare. Existing multimodal fusion methods often assign fixed equal weights to sensor streams, overlooking differences in modality importance, acquisition cost, and sensor quality, which can vary due to movement, incorrect placement, or temporary blockage. We propose an adaptive and cost-aware multimodal HAR framework based on multi-agent reinforcement learning for centralized HAR and extend it to federated learning as FedMHAR. In the centralized setting, multimodal fusion is formulated as a cooperative Multi-Agent Reinforcement Learning (MARL) problem, where each sensing modality is assigned a PPO-based agent that learns per-sample fusion weights, enabling the model to emphasize informative modalities while down-weighting costly sensors when cheaper alternatives provide sufficient information. In the federated setting, we introduce BiFL-PPO, a bidirectional federated optimization strategy in which a server-side PPO policy learns client-specific trust weights and feeds them back to adapt local learning rates and proximal regularization. Unlike round-level optimization, BiFL-PPO uses dense batch-level rewards for more frequent feedback and stable training under heterogeneous client data. Evaluation on the MEx Rehabilitation and UTD Multimodal Human Action datasets shows that the centralized framework achieves 87.30% and 94.98% accuracy, respectively, outperforming conventional fusion methods and state-of-the-art HAR models. FedMHAR achieves 79.74% and 77.49% in the federated setting, consistently surpassing FedAvg, FedProx, FedBN, FedNova, and AdaFedProx, while providing more stable performance and reducing sensor acquisition cost.

0.28

EdgeHAR: An Edge-Native Compact Sensor Foundation Model for Human Activity Recognition

cs.LG cs.AI ·2026-09-13 ·He Zhang, Siyu Yuan, Siyu Liu, Sizhen Bian, Bin Guo
published similarity 0.63
Abstract

Sensor-based human activity recognition (HAR) is fundamental to ubiquitous and wearable computing, yet existing foundation models are largely designed for cloud-scale deployment and struggle with real-world sensing shifts, including unseen users, devices, sampling rates, and sensor placements. We present \textbf{EdgeHAR}, an edge-native compact sensor foundation model designed for wearable intelligence. Unlike conventional models that entangle activity knowledge with acquisition variations, EdgeHAR learns transferable representations by factorizing sensor signals into three latent codes: an \textbf{(i)Activity-Semantic Code} capturing reusable activity knowledge, a \textbf{(ii)Motion-Dynamics Code} modeling temporal patterns, and an \textbf{(iii)Acquisition-Context Code} representing sensor-specific variations. This disentangled design enables efficient adaptation to new users, devices, placements, and activity classes with limited target-domain data. By incorporating lightweight adaptation modules, EdgeHAR achieves foundation-model-level transferability while satisfying edge constraints in computation, memory, latency, and privacy. Experiments across heterogeneous HAR datasets demonstrate that EdgeHAR maintains competitive recognition performance under distribution shifts with substantially reduced deployment cost. EdgeHAR establishes a practical paradigm for compact, edge-first foundation models for ubiquitous sensing systems.

0.33

Learning OTA: A Unified Framework for Edge Sensing, Computation, and Communication

cs.IT cs.NIeess.SP ·2026-10-03 ·Mehdi Karbalayghareh, David J. Love, Christopher G. Brinton
code/data similarity 0.61
Abstract

Most federated learning (FL) studies implicitly assume that training data is readily available at participating edge devices. In practice, this data must first be acquired through heterogeneous sensors with potentially different measurement modalities, and the quality and cost of this acquisition process can directly affect the learning performance. This dependence creates a coupled relationship among sensing, computation, and communication, necessitating a system model that captures their joint effect on resource efficiency and learning accuracy. This paper develops such a unified treatment for an over-the-air (OTA) FL system in which each device configures its sensing modality, power, resolution, and sample size, with aggregations conducted OTA at the server. We derive models for sensing noise, energy consumption, latency, and OTA aggregation distortion, and establish a non-convex convergence bound quantifying how these factors influence the global model trajectory. Guided by this analysis, we formulate a per-round optimization that balances energy, latency, and learning quality under physical device constraints. The resulting mixed optimization is addressed through an alternating procedure with closed-form and one-dimensional updates for continuous variables and a lightweight discrete search. Numerical experiments corroborate the advantage of jointly optimizing sensing, computation, and communication decisions.

0.21

FedSwitch: Federated Region Classification From Wireless Channel Measurements

eess.SP ·2026-10-02 ·Mattia Piana, Stefano Rini, Stefano Tomasin
similarity 0.59
Abstract

Wireless Internet-of-Things (IoT) networks can leverage locally observed channel state information (CSI) at base-stations (BSs) to perform region classification, i.e., infer the region of origin of a transmitter (TX), thereby enabling channel-based authentication. However, centralizing these measurements incurs substantial communication overhead and requires sharing location-dependent radio fingerprints. To address these difficulties, we propose an edge-native federated learning (FL) framework where BSs collaboratively train a shared classifier via a parameter server (PS) without exposing raw data, thereby supporting physical-layer security tasks like spoofing detection. To overcome the statistical heterogeneity arising from diverse BS locations and propagation conditions, we introduce FedSwitch, a personalized FL algorithm that automatically transitions edge nodes from collaborative training to local fine-tuning upon loss stagnation. We provide theoretical convergence guarantees for the proposed collaborative training phase and derive analytical bounds linking classification errors to statistical discrepancies. Extensive evaluations on synthetic channel measurements demonstrate that FedSwitch significantly improves region-classification accuracy over standard FL baselines.

0.17

Contribution-Aware Bandwidth Allocation for Multimodal Split Learning

cs.LG cs.DCcs.NI ·2026-09-01 ·Iason Ofeidis, Leandros Tassiulas
similarity 0.59
Abstract · 10 pages, 4 figures

Multimodal models are increasingly the default option for perception at the network edge, yet they are trained almost entirely in the datacenter, because a client holding several sensor streams cannot host an encoder per modality. Split Learning makes such training feasible by keeping only the first layers on the device, at the cost of an uplink that must carry smashed activations for every modality at every step. Existing compression schemes give each modality the same keep-ratio, so the shared budget is divided in proportion to smashed-activation dimension, a quantity unrelated to how much each modality contributes to the fused prediction. We make that division an explicit decision and call it inter-modality allocation: under a fixed uplink budget, every policy transmits the same expected payload and differs only in how that payload is split across modalities. Our allocator, ModalShare, sets each modality's keep-ratio from a Shapley contribution score that the server computes over coalitions of activations it has already received. Measuring this score adds no uplink traffic and no client-side computation, and needs no prior knowledge of which stream is which. ModalShare improves accuracy over equal keep-ratios by 15.4 and 12.4 percentage points on CREMA-D and MVSA at matched payload in 5x compression, with strong performance across three compressors, three datasets, and four budgets. We show that existing compressors underperform in multimodal settings, with ModalShare recovering what gains are left behind.

0.27

From Classifier Heads to Encoder Prototypes: Personalized Federated Learning with Head-Derived Clustering

cs.LG ·2026-10-21 ·Amirardalan Dehghanpour, Arian Noori, Ehsan Tanghatari, Naimeh Omidvar, Hamid R. Rabiee
similarity 0.59
Abstract

Personalized federated learning must decide which clients should share representations. We introduce HDC-FL (Head-Derived Clustering for Personalized Federated Learning), which reads that decision off a signal shared-encoder training already produces: the client-specific classifi- er heads. At periodic refresh rounds the server projects the cached heads into a normalized low-rank descriptor space, groups the descriptors, builds one encoder prototype per group from the cached client encoders, and returns to each client a soft mixture of those prototypes paired with its own unchanged head. Client-specific heads are never averaged or distributed to oth- er clients; the shared-encoder trajectory is left intact; and no second optimization trajectory, additional local objective, or post-training fine-tuning stage is required. Across 11 CIFAR-10 and CIFAR-100 configurations covering label-subset, Dirichlet and rotation heterogeneity with ResNet-18 and MobileNetV2 encoders, HDC-FL is competitive with the strongest personalized and clustered baselines in macro-averaged mean accuracy and attains the numerically highest macro-averaged worst-client and bottom-20% accuracy among the evaluated methods. The tail results distinguish methods that look similar when only their means are compared. A controlled donor-swap diagnostic further finds positive donor-ranking associations between head similarity and encoder compatibility in all three settings tested, most clearly under covariate shift.

0.14

Learning Multimodal Sensor Fusion with JEPA Representations for Breathing Activity and Pose Recognition

cs.CV cs.AI ·2026-10-11 ·Yunxiang Luo, Constantino Álvarez Casado, Xiaoting Wu, Nhi Nguyen, Le Nguyen +2 more
hot direction similarity 0.59
Abstract · 6 pages, 2 figures, 3 equations, 4 tables, conference paper

Predictive video representations do not by themselves specify how a multimodal system should handle missing sensors. We transfer frozen V-JEPA2 features to a set of heterogeneous sensors comprised by RGB video, depth video, and radar time series. A shared fusion model uses modality queries, fixed sensor slots, and explicit availability to classify breathing activity and pose from all seven non-empty sensor subsets. The contribution is a controlled evaluation of subset supervision, pooling capacity, and representation transfer on 39 OMuSense-23 subjects with five subject-group folds and three seeds. With matched downstream parameter counts, the final model reaches recording-level activity macro-F1 of $75.23%\pm0.60$ averaged across seven subsets. Supervising all subsets in each batch improves the matched query-pooling control by 1.923 percentage points (conditional 95% subject-bootstrap interval [0.647, 3.209]), while query pooling shows no clear advantage over matched mean pooling under balanced supervision. Radar probes support transfer beyond pooled wavelet pixels, 64-token inference approximately preserves mean activity performance, and the tested Consensus-JEPA loss adds no measured benefit against its exact control.

0.36

Adaptive Unequal Error Protection for Semantic Split Learning over Wireless Channels

cs.IT eess.SP ·2026-08-17 ·Vukan Ninkovic, Dejan Vukobratovic, Dragisa Miskovic, Chao Wang
published similarity 0.59
Abstract · Accepted for publication at IEEE Communications Letters

We propose a task-aware semantic split learning (SL) framework for wireless edge-cloud inference, in which the reliability of transmitted latent representations is dynamically adapted to their relevance for the downstream task. An autoencoder (AE)-based physical (PHY) layer enables end-to-end learning of the communication interface, while unequal error protection (UEP) is realized via mutual information (MI)-driven prioritization of latent components during training. The gradient of the estimated MI with respect to each latent component serves as a sensitivity-based proxy for task relevance, providing a fully learning-driven prioritization that adapts to both the data distribution and the downstream task. We further show that this prioritization translates into measurable physical-layer effects: MI-guided UEP assigns significantly higher transmit power to the most task-critical latent components compared to the equal error protection (EEP) baseline. Experiments on real-world IoT sensing data demonstrate consistent gains over equal and fixed-UEP baselines across SNR regimes. Additional analysis confirms ranking stability, estimator robustness and generalization across datasets and task types, indicating broad applicability of the proposed framework.

0.31