CS Paper Radar

CoFi-CLM: A Coarse-to-Fine Channel Language Model for Finite-Bit CSI Feedback

Chao Zhang, Cheng Luo, Luping Xiang, Kun Yang

eess.SP ·submitted 2026-09-14 16:56 UTC · v1
code/data

Abstract

In frequency-division duplex massive multiple-input multiple-output (MIMO) systems, the user equipment (UE) must convey high-dimensional downlink channel state information (CSI) under a stringent finite-bit feedback budget. Deep learning (DL) has emerged as a powerful tool for CSI compression due to its ability to capture complex channel correlations and learn compact CSI representations. However, recovering fine-scale channel structure from limited feedback remains challenging. To address this challenge, we propose the Coarse-to-Fine Channel Language Model (CoFi-CLM), a large AI model deployed at the base station (BS) that explicitly learns dependencies between coarse and fine CSI representations. Specifically, the UE reports coarse-token indices from a learned codebook, while CoFi-CLM predicts distributions over unreported fine-token indices conditioned on this feedback in a single Transformer forward pass. A dual-path decoder fuses the direct coarse reconstruction with the generated fine-scale reconstruction. Concentrating fine-token prediction and fusion at the BS allows the CLM capacity to scale without increasing the computational or storage cost at the UE. Across 64 to 160 feedback bits, CoFi-CLM consistently outperforms the compared methods on both seen and unseen scenarios. At 128 bits, it improves the normalized mean square error (NMSE) over the recent large AI model baseline by approximately 1 dB on both sets. The performance gap between seen and unseen scenarios further supports its generalization under the considered channel model. The source code is publicly available at https://github.com/doovvv/CoFi-CLM.

Closest papers

Nearest neighbours by meaning across the whole index.

Model-Driven Deep Learning with Rank-One Sensing for Efficient CSI Feedback

eess.SP ·2026-10-08 ·Shunpu Tang, Qianqian Yang, Seung-Woo Ko, Jihong Park, Kaibin Huang
code/data similarity 0.64
Abstract

Downlink channel state information (CSI) feedback is essential for beamforming optimization in frequency-division duplex (FDD) massive multiple-input multiple-output (MIMO) systems. However, the feedback overhead increases with the number of antennas and subcarriers, posing a major challenge to practical deployment. Although recent deep learning (DL)-based methods reduce this overhead by compressing CSI at the user equipment (UE) and reconstructing it at the base station (BS), most of them treat CSI as a generic image and rely on convolutional or Transformer architectures. As a result, the low-rank multipath prior is insufficiently exploited, while non-negligible computation is introduced at both the UE and the BS. To address this issue, we propose a low-rank prior (LRP)-guided CSI feedback framework that performs structured sensing and reconstruction directly over rank-one channel components. Specifically, the LRP encoder conducts learnable rank-one sensing to obtain path-aware measurements at the UE, while the LRP decoder reconstructs CSI through rank-one synthesis at the BS, avoiding the iterative recovery required by classical solvers. To compensate for finite-rank approximation residuals, we extend our previous DCRNet into DCRNetV2 by incorporating the proposed LRP backbone and gated dilated-convolutional residual branches. Experiments on multiple datasets and scenarios show that the proposed methods achieve a better accuracy-complexity tradeoff than existing model-based and DL-based baselines. In particular, standalone LRP achieves more than $10$~dB NMSE improvement over model-based baselines with much lower complexity, while DCRNetV2 achieves comparable accuracy to state-of-the-art Transformer-based methods with only about one-fifth of the complexity. The source code is available at https://github.com/tangshunpu/DCRNetV2.

0.22

Low-Rank Prior-Guided Rank-One Sensing for Efficient CSI Feedback

eess.SP ·2026-09-23 ·Shunpu Tang, Qianqian Yang, Seung-Woo Ko, Jihong Park
similarity 0.63
Abstract

Downlink channel state information (CSI) feedback is essential for frequency-division duplex massive MIMO, yet the feedback overhead grows rapidly with the numbers of antennas and subcarriers. To reduce this overhead, most deep learning approaches compress CSI by treating it as a generic image, leaving its low-rank multipath structure unexploited. In contrast, model-driven alternatives explicitly embed this structure in iterative recovery, but at the cost of high computational complexity and latency. To overcome these limitations, we propose a low-rank prior-guided (LRP) framework that performs learnable rank-one sensing at the user equipment to compress CSI into low-dimensional codewords, and reconstructs CSI by direct rank-one synthesis at the base station. Both compression and reconstruction are jointly optimized, and fully exploit the low-rank structure of CSI. We further develop DCRNetV2, which preserves LRP as its backbone and uses gated dilated-convolutional residual paths to compensate for finite-rank errors. Experimental results show that LRP outperforms iterative model-based methods with lower complexity, and DCRNetV2 achieves a better accuracy-complexity tradeoff than existing learning-based methods.

0.15

WiFo-R: Wireless Foundation Model with Recurrent Channel Representation

eess.SP ·2026-11-08 ·Liu Xuanyu, Cao Keyi, Cheng Xiang
similarity 0.62
Abstract

Wireless foundation models learn transferable channel state information (CSI) representations across heterogeneous wireless systems. However, conventional grid-aligned representations couple feature sequence length to CSI dimensions, restricting the allocation of representation capacity to different propagation conditions and task requirements while increasing inference overhead. This paper proposes WiFo-R, a wireless foundation model that learns progressive channel representations with selectable capacity. A channel-query cross-attention encoder aggregates spatial, temporal, and spectral CSI information, while parameter-shared recurrent updates refine existing channel features and introduce additional latent tokens. Multi-mask, multi-depth self-supervised pretraining equips intermediate states with strong reconstruction capabilities, followed by task-aligned post-training for channel estimation and CSI feedback. WiFo-R supports temporal and frequency-domain channel prediction, channel estimation, and variable-rate CSI feedback, and transfers to localization and LoS/NLoS classification. Experiments across in-distribution and out-of-distribution channel configurations demonstrate accurate reconstruction, progressive performance improvements, and strong transferability. A single feedback encoder--decoder supports six payload lengths at fixed quantization precision, while cross-attention and selectable recurrent depth improve the accuracy--complexity trade-off for large-scale CSI observations.

0.23

Learning-to-Transition for Large-scale and High-Order MIMO Detection

cs.IT cs.AI ·2026-08-14 ·Yubo Zhang, Yiyao Liu, Xiaodong Wang
similarity 0.62
Abstract

High-order multiple-input multiple-output (MIMO) detection requires efficient search over a large discrete symbol space while producing reliable soft information for channel decoding. This paper develops a learning-to-transition (L2T) framework that formulates MIMO detection as a stochastic sequence of complete-vector transitions. At each transition, a channel-coupled Transformer updates both the instance embedding and the sampling policy, while a blockwise autoregressive factorization captures inter-stream dependence with moderate sequential complexity. For hard-output detection, a transition network is applied recursively and trained through a residual-to-BER curriculum, which first learns the MIMO search geometry from the exact residual metric and then aligns the policy with transmitted-bit accuracy. For soft-output reception, the well-trained hard policy is cloned at the parameter level into every layer of an untied soft-input soft-output iterative detection and decoding (IDD) receiver. This tied-to-untied transfer preserves the learned zero-prior search dynamics while enabling layer- and round-specific specialization under decoder feedback. Within each IDD round, decoder priors tilt candidate generation according to Bayes' rule, and likelihood-weighted terminal hypotheses produce posterior and extrinsic log-likelihood ratios for LDPC decoding. A multi-stage training strategy further stabilizes the hard-to-soft transfer by progressively exposing the receiver to synthetic and in-loop decoder-generated priors.

0.04

Pretraining Latent Information Feedback Transformers with Teacher Supervision

cs.CL ·2026-09-29 ·Dor Tirosh, Ido Amos, Mor Geva
similarity 0.61
Abstract

Transformer language models (LMs) are feed-forward: deep-layer representations are never fed back to shallower layers, and the only pathway for information to flow downward across generation steps is the decoded token. This narrow channel forces models to recompute intermediate results and to discard alternative continuations. In this work, we remove this bottleneck during pretraining, introducing the LIFT (Latent Information Feedback Transformer) architecture and training method which enable LMs to propagate state across generation. We achieve this by turning recurrent-state learning into a teacher-forced prediction problem: each input token is paired with an information-dense state, derived from the next-token distribution of an off-the-shelf pretrained LM. The model, extended with a small number of additional parameters, is then trained to predict both the next token and the next state. As the input states are precomputed, pretraining remains fully parallel across positions. At inference, the model's own predicted states are fed back, with a minor computational overhead that decreases with model size. Experiments with pretrained models ranging from 135M to 1B parameters show that LIFT consistently outperforms standard Transformers and baselines on language modeling, downstream reasoning tasks, and procedural tasks under token-matched budget, while being on par with or ahead of compute-matched Transformers. Moreover, a controlled study on a state-tracking task shows that a tiny LIFT outperforms same-size Transformers trained on 8x more data, even when trained with the states of a Transformer that fails the task. Overall, we show that LMs can learn to exploit deep-to-shallow feedback during pretraining via scalable teacher supervision.

0.26

Asymmetric Looped Transformers: Fixed Global Representations for an Iterative Decoder

cs.CL ·2026-10-20 ·Cheng Liang, Tengfei Zhang, Wenjie DU, Jianyu Wu, Changlian Ma +4 more
highly cited similarity 0.61
Abstract

Looped Transformers increase effective depth by repeatedly applying shared layers, offering a way to scale computation without increasing parameter count. However, full-stack looping places context construction and iterative state refinement within the same recurrent computation, without explicitly distinguishing their roles. We introduce the Asymmetric Looped Transformer (AsymLoop), which separates single-pass context encoding from recurrent state refinement. A single-pass causal encoder produces layer-specific Global key-value (KV) caches that remain fixed across iterations, while a weight-tied recurrent decoder generates Local KV from its evolving state. Joint attention over both sources under a single softmax allows each update to integrate persistent contextual information with the current intermediate state. For training, we apply a heavy-tail-guided layerwise learning-rate method to the encoder and decoder. Experiments on FineWeb-Edu across four model scales, from 60M to 1B, show consistently lower validation perplexity and higher average downstream accuracy than Full Loop under matched effective depth and training token budgets. At 1B, AsymLoop improves average zero-shot accuracy across seven commonsense reasoning benchmarks from 49.58% to 50.85%. Further experiments extend these benefits to longer training budgets, an OLMo-style backbone, and sparse mixture-of-experts models. These results support asymmetric context encoding and state refinement as an effective design for recurrent language models.

0.31

The Von-Neumann State-Space Transformer for neural decoding

cs.LG cs.AIq-bio.NC ·2026-08-25 ·Morteza Sarafyazd
benchmark similarity 0.61
Abstract · 14 pages, 5 figures, 5 tables

Cortical computation is strikingly low-dimensional: a handful of latent variables, carried in a neural population's activity, steer the higher-dimensional responses of individual neurons. Our aim is sample efficiency-models that decode well from limited data and at small parameter budgets. In a standard Transformer layer, the feed-forward block applies the same operator to every token. We suggest a von-Neumann inspired hypothesis of efficient computation as an alternative for neural decoding: a controller decodes an instruction and then executes a token-specific operator; the usual realization-a soft mixture of experts-only blends their outputs, not operators. We introduce a von-Neumann State-Space Transformer (VN-SST), a memory-augmented Transformer whose feed-forward block is a low-rank instruction bank: a shared base operator plus a small set of learned low-rank instructions, from which a per-token code synthesizes the weight matrix actually used at that token. The code is read from a low- dimensional projection of a carried state-space memory, so a slow latent trajectory acts as an instruction pointer-mirroring how low-dimensional dynamics may route cortical computation. On three motor-cortex neural-decoding benchmarks, VN-SST is far more data-efficient than a modern Transformer, each jointly predicting spikes and decoding behavior. This model wins by a wide margin on the scarcest benchmark, leads on the other two, and turns longer context into rising rather than falling accuracy. We evaluated that the network compresses a large instruction bank to a few bits per token, so program capacity acts as a control channel, not an accuracy lever. The same model is also more parameter-efficient on two small text benchmarks used for language modeling (LLMs), suggesting a generic mechanism.

0.19

Generative Deep Learning for CSI Feedback in Massive MIMO via Channel Distribution Prediction

eess.SP cs.IT ·2026-10-12 ·Liujia Yao, Changsheng You, Chao Zhou, Yi Gong, Chengwen Xing
similarity 0.60
Abstract

Existing deep-learning-based channel state information (CSI) feedback methods reduce feedback overhead by compressing high-dimensional CSI into low-dimensional representations and/or exploiting historical CSI to predict future CSI. However, these methods may become unreliable when user equipment (UE) suddenly changes its motion. To address this issue, we propose a prediction-aided coding (PAC) framework for generative CSI feedback, which adaptively encodes CSI based on predicted channel/CSI distributions and the observed CSI in real time. Each CSI frame is represented by multiple CSI tokens obtained without using historical CSI, making token generation robust to abrupt UE motion. In the first stage, the conditional distribution, instead of a deterministic value, of the next-frame CSI tokens is predicted based on historical tokens. In the second stage, once the next frame arrives, the predicted distribution is used for variable-length entropy coding of its CSI tokens. To further reduce feedback overhead and inference latency, we propose efficient designs for both stages. For the first stage, a customized causal Transformer with a new positional embedding strategy accurately predicts next-frame token probabilities. For the second stage, a parallel causal decoding strategy reduces the inference latency caused by sequential token decoding. Extensive experiments show that, compared with existing history-aware baselines, PAC reduces feedback overhead by approximately 72% to 84% while achieving higher reconstruction accuracy. Additionally, PAC maintains CSI reconstruction accuracy under random-walk trajectories, whereas existing history-aware baselines suffer sharp degradation in reconstruction accuracy.

0.20