CS Paper Radar

Logit-Aware MIMO AirComp for Distributed Mixture-of-Experts LLM Inference over Wireless Edge Networks

Lyutianyang Zhang, Yunjian Jia, Liu Cao, Dengke Wang, Jinke Ren, Shuguang Cui

cs.IT cs.AIcs.LG ·submitted 2026-09-22 15:55 UTC · v1

Abstract

Distributed mixture-of-experts (MoE) inference is a promising architecture for deploying large language models (LLMs) at wireless edge networks because sparse experts can be placed across coordinated base stations (BSs), while the anchor node and user equipment (UE) can offload LLM inference tasks to BSs. The communication bottleneck is the MoE aggregation, where the anchor BS must recover a weighted sum of selected expert outputs before each decoding step. Over-the-air computation (AirComp) is well matched to this operation because the wireless multiple-access channel naturally superposes simultaneous transmissions. However, conventional AirComp minimizes communication distortion, whereas MoE aggregation errors have unequal impact on LLM outputs. We propose a logit-aware MIMO AirComp framework that estimates local logit sensitivity as block-level weights and jointly optimizes receive combiners and BS precoders under per-BS power constraints. We also develop an alternating algorithm that protects decoding decisions from aggregation perturbations. Using OpenCompass, we evaluate Qwen3-30B-A3B-Instruct-2507-FP8 on GSM8K and ARC-Challenge. At 30 dB aggregation SNR, perturbed Qwen3 retains 99.3% of clean GSM8K accuracy and 94.8% of clean ARC-Challenge accuracy. In a direct ARC-Challenge closed-loop audit over 295 examples, SW-AirComp achieves 43.73% accuracy at 30 dB and 26.78% at 20 dB, outperforming unweighted, matched-filter, and zero-forcing AirComp. In the wireless simulator, SW-AirComp reduces decision-relevant aggregation distortion; at 20 dB, its final weighted-sum mean-squared error is 53.5% lower than unweighted AirComp under the same channels, samples, and sensitivity weights. Logit-RMSE and logit-gap diagnostics further show that the gain comes from reducing output-logit perturbation and lowering the risk of top-token changes.

Comment
13 pages, 9 figures

Closest papers

Nearest neighbours by meaning across the whole index.

AirMoE: Realizing Over-the-Air Distributed Mixture-of-Experts Inference at the Wireless Edge

eess.SP ·2026-08-24 ·Huiling Yang, Zhanwei Wang, Kaibin Huang
similarity 0.82
Abstract

Mixture-of-experts (MoE) architectures enable efficient large language model (LLM) inference at the wireless edge through sparse activation. The wireless distributed MoE (WIDE) architecture addresses edge-resource constraints by distributing experts across devices coordinated by an edge server. However, WIDE suffers from repeated uplink transmissions of high-dimensional expert outputs via orthogonal multiple access. To overcome this bottleneck, we propose AirMoE, an over-the-air computing (AirComp)-enabled framework for simultaneous expert-output aggregation via wireless waveform superposition. Integrating AirComp into MoE inference introduces three challenges: fast-varying aggregation weights, layer-dependent error sensitivity, and channel-aware expert placement. To address these challenges, we construct an inference-aware AirMoE error metric to quantify aggregation distortion effects on end-to-end (E2E) inference accuracy via perturbation-based layer-sensitivity calibration. We then formulate a joint optimization problem to minimize this error and decompose it, without loss of optimality, into a two-timescale framework. At the fast timescale, we derive a globally optimal threshold-based power-control policy that separates devices into coefficient-aligned and full-power groups. At the slow timescale, we develop an activation- and channel-aware expert placement strategy that assigns more important experts to devices with lower channel-power cost. Extensive experiments demonstrate that AirMoE outperforms representative baselines in E2E inference accuracy, especially under strong device heterogeneity.

0.24

MatMoE: Matryoshka Mixture-of-Experts with Dynamic Mixed-Precision Quantization for Efficient Inference

cs.LG cs.AI ·2026-11-01 ·Renjie Wei, Haochen Huang, Chuyu Qiu, Jinqi Wen, Dehao Xu +1 more
published similarity 0.64
Abstract · Accepeted by ICCAD 2026

Mixture-of-Experts (MoE) architecture has been widely adopted in large language models (LLMs) since it reduces the computational cost through sparse expert activation. However, the massive parameter scale of MoE models hinders their deployment on resource-constrained edge devices. Although existing mixed-precision quantization reduces the model size, they are inherently static, fixing the bit-width of each expert across decoding steps. Such static schemes yield suboptimal performance as they overlook the dynamic variation in expert importance during decoding. Furthermore, they fail to adapt to the dynamic latency constraint driven by the fluctuating system utilization. To address these challenges, we propose MatMoE, a dynamic mixed-precision quantization framework for efficient MoE inference. MatMoE features: (1) a Matryoshka MoE PTQ algorithm that enables experts to operate flexibly at multiple precisions within a stringent memory budget; (2) a floor-and-compensation quantization method that eliminates redundant memory accesses in the Matryoshka representation; and (3) a dynamic bit-width allocation strategy that determines the precision for each expert based on their routing scores to satisfy the dynamic latency requirement. We implement our MatMoE system atop llama.cpp for both CPU and GPU platforms. Extensive evaluations across three modern MoE models and challenging benchmarks demonstrate that MatMoE achieves $4.0\% \sim 18.3\%$ higher accuracy than state-of-the-art static mixed-precision method. Compared to llama.cpp, MatMoE delivers $1.3\times \sim 3.7\times$ speedup on Raspberry Pi 5, and $1.2\times \sim 1.3\times$ speedup on NVIDIA RTX 4090 and Jetson AGX Orin.

0.20

ProQuant: Preserving Expert Quality over Quantity in Ultra-Low-Bit Quantization of Mixture-of-Experts Models

cs.AI ·2026-10-22 ·Jiaqi Chen, Xianglong Yan, Chengzhu Bao, Xie Chen, Yulun Zhang
code/data similarity 0.64
Abstract

Mixture-of-Experts (MoE) large language models reduce computation through sparse activation but still incur substantial storage costs. Mixed-precision post-training quantization (PTQ) reduces these costs by quantizing different model components at different bit widths, but existing methods often struggle to preserve generation quality under tight bit budgets. We identify two key challenges: (1) nonlinear quantization errors at extremely low precision are underestimated when cross-layer propagation is ignored; (2) direct router fine-tuning may cause routing drift, and calibration gains may not generalize to downstream tasks. To address these issues, we propose ProQuant, a weight-only PTQ framework for MoE models. Prune-or-Prioritize Mixed-Precision Allocation (PoP) jointly optimizes expert pruning and bit-width assignment. It allows higher-precision experts to substitute for severely degraded ones while improving bit-budget utilization. Routing-Consistent RMSNorm Finetune (RoCo) adapts expert input representations through mathematically equivalent transformations. It compensates for quantization errors to reduce routing drift. Extensive experiments demonstrate that ProQuant outperforms existing methods across diverse generative tasks. Code is available at https://github.com/ThrowBug/ProQuant.git.

0.27

AIR-LLM: Broadcasting AI Weights over Radio for Memory-Free Edge LLM Inference via RF Computing

cs.IT cs.ETcs.LGeess.SP+1 ·2026-09-30 ·Zhihui Gao, Tingjun Chen, Dirk Englund
similarity 0.64
Abstract · 14 pages, 12 figures, 6 tables. Appendix: 12 pages, 7 figures, 11 tables

Next-generation large language models (LLMs) are expanding from the cloud to ubiquitous edge devices. However, edge devices typically either lack the memory to store increasingly large LLM weights or, even with enough memory, spend unaffordable energy on loading the weights. This raises our question: can an edge device run an LLM without storing or loading its weights, but receive them over the air and consume them on the fly? Inspired by wireless broadcasting, we present AIR-LLM, an LLM inference architecture for edge devices, which is composed of: (i) a central radio (e.g., 5G base stations) that broadcasts the LLM weights into the air, and (ii) the edge user that receives the weights and completes the general matrix-vector multiplication (GEMV) of LLM inference directly in the radio frequency (RF) domain using RF mixers. To further shorten the airtime, AIR-LLM exploits MIMO spatial multiplexing and proposes an energy-efficient precoder-postcoder pair on the edge to calibrate its own wireless channel. Since the central radio stays user-unaware, AIR-LLM is user-scalable so that one broadcast serves unlimited users within its coverage. We implement AIR-LLM on the NVIDIA Sionna ray-traced channels of two real-world urban scenes and the profiling of a real RF mixer. With a WikiText-2 perplexity degradation of 4.0% on LLaMA-3.1-8B, AIR-LLM saves the energy by 157.7x/40.4x against the FP16 and weight-only quantization baselines; with 20 users, its airtime is 104.1x/26.0x shorter, respectively.

0.27

WiSDoM: Wireless Sparse Decision Transformer with Mixture-of-Experts for Multi-Task Mobile Network Optimization

cs.NI cs.AIcs.LG ·2026-08-31 ·Fatih Temiz, Shavbo Salehi, Melike Erol-Kantarci
similarity 0.62
Abstract · 13 pages, 11 figures, submitted to IEEE for possible publication

Emerging 6G wireless networks are expected to operate across diverse deployment scenarios, where variations in network topology, user mobility, traffic demand, and radio conditions challenge the scalability of conventional radio resource management (RRM). While offline reinforcement learning (RL) methods have demonstrated strong decision-making capabilities, learning a single policy that performs consistently across heterogeneous wireless environments remains difficult due to conflicting optimization objectives and limited model specialization. These challenges become particularly pronounced in coordinated multipoint (CoMP) transmission, where selecting the optimal serving-cell combination requires sequential decision-making under evolving network conditions. This paper presents the Wireless Sparse Decision Transformer with Mixture of Experts (WiSDoM), a sparse multi-task offline RL framework for adaptive multi-cell selection. WiSDoM combines Decision Transformers (DTs) with a Mixture-of-Experts (MoE) architecture that dynamically activates specialized experts according to task characteristics. This MoE mechanism improves model capacity without proportionally increasing inference cost, mitigates negative transfer, and enables expert specialization across tasks. WiSDoM is trained jointly on diverse network configurations spanning multiple base station and user equipment densities, mobility levels, and scheduler policies. Experimental results show that WiSDoM consistently outperforms heuristic methods, single-task models, and conventional multi-task DTs, improving quality of experience (QoE) by up to 55% while activating approximately one-third of the parameters of its dense counterpart during inference. Furthermore, WiSDoM exhibits strong task generalization and efficiently adapts to unseen wireless scenarios through few-shot prompting without retraining or fine-tuning.

0.34

OMP-MoE: Efficient Expert Pruning for Mixture-of-Experts LLMs via Orthogonal Matching Pursuit

cs.LG ·2026-09-09 ·Dezhi Li, Lujun Li, Qiyuan Zhu, Hao Gu, Bei Liu +2 more
similarity 0.61
Abstract · Work in progress, revisions ongoing

Mixture-of-Experts (MoE) models enable efficient scaling of large language models but face critical deployment challenges due to massive memory requirements. Existing pruning methods either incur prohibitive search costs or neglect the dynamic interdependencies between experts. To address these challenges, we present OMP-MoE, a novel training-free compression framework for reducing expert redundancy in MoE-based LLMs. Based on observations of expert contribution patterns, we reformulate the pruning problem as a sparse signal reconstruction task solved through Orthogonal Matching Pursuit. Specifically, our method first treats individual expert contributions as dictionary atoms and selects experts that greedily minimize reconstruction error with linear computational complexity. Then, we optimize cross-layer expert allocation through a water-filling strategy that accounts for both reconstruction quality and routing stability. Finally, we introduce OMP-MoE†, an adaptive inference mechanism that dynamically adjusts expert activation based on energy prediction. Comprehensive experiments on Qwen, DeepSeek-V2, GPT-OSS, and Mixtral MoE demonstrate consistent improvements over existing methods at 25-50% pruning ratios. For Qwen3-30B-A3B at 50% compression, we retain 93.3% of original performance, achieving 33$\times$ faster search and 1.55$\times$ inference speedup. Codes will be available after acceptance.

0.21

Multi-view Semantic-aware Resource Allocation for Edge AI

cs.IT cs.LGeess.SP ·2026-10-25 ·Jiawei Luo, Hussein Saleh, Mohamad Assaad
similarity 0.61
Abstract · Submitted to IEEE Wireless Communications Letters. Jiawei Luo and Hussein Saleh contributed equally to this work

Semantic communication is emerging as a pivotal paradigm to overcome wireless resource scarcity in 6G networks. This paper proposes a novel end-to-end framework for semantics-aware resource allocation in multi-user Edge AI. We extend the Opportunistic Information Bottleneck (OIB) to a multi-view setting, termed the Multi-view Gaussian Information Bottleneck (MGIB), by fusing complementary source views without variational approximation or network retraining. A resource allocation framework is then formulated for joint compression-level selection, transmission mode selection (i.e., the number of active sources per group), and PRB assignment. The problem is formulated as a Mixed-Integer Linear Program (MILP), solved efficiently via a proposed Lagrangian Relaxation (LR) algorithm, and benchmarked against the NP-hard optimal solution. Simulations demonstrate that the proposed LR scheme achieves near-optimal utility, and doubles PRB efficiency versus average allocation.

0.29

S2-MoE: Enabling Efficient Self-Speculative Decoding for Mixture-of-Experts on Edge Devices

cs.AI ·2026-08-15 ·Haochen Huang, Shengxuan Qiu, Meng Li
code/data similarity 0.61
Abstract · 13 pages, 10 figures

Deploying large language models (LLMs) for inference on edge devices is challenging due to severe memory and bandwidth constraints. While speculative decoding and Mixture-of-Experts (MoE) have been proposed to improve inference efficiency, naively combining them often incurs excessive verification overhead and poor expert reuse, limiting their effectiveness in memory-bound edge settings. In this work, we propose S2-MoE, an efficient self-speculative decoding framework for MoE inference on edge devices. S2-MoE reduces redundant verification through routing-aware adaptive speculative expansion, improves verification efficiency with reuse-aware expert gating, and aligns draft and target execution via shared context. Implemented in llama$.$cpp, S2-MoE achieves up to $5.3\times$ speedup (about $2.0\times$ on average) over standard autoregressive decoding across diverse MoE models and datasets on edge devices. Code is available at https://github.com/angerybob/S2-MoE.

0.67