CS Paper Radar

MaskCode: Mask Transformer for Feedback-Assisted Coding With Linear Block Codes

Jonggyu Jang, Hongjae Nam, Vishrant Tripathi, David J. Love, Hyun Jong Yang

cs.IT cs.LGeess.SP ·submitted 2026-09-01 04:47 UTC · v1

Abstract

Feedback-based coding schemes have demonstrated substantial performance gains over today's open-loop coding schemes. Unfortunately, these gains are usually achieved in idealized settings with perfect feedback. Over the last few years, machine learning-based schemes have been shown to be promising solutions for implementing feedback-based codes, particularly when combined with short-block-length open-loop error correcting codes (ECCs) in a concatenated coding structure. However, existing ML-based feedback schemes remain agnostic to the outer code's structure, potentially misallocating feedback resources on error patterns already correctable by the outer ECC. To address this, we propose MaskCode, a Transformer-based inner feedback code for concatenated coding systems, which explicitly incorporates structural knowledge of the outer linear block code into the inner feedback encoder design via two synergistic mechanisms: 1) a soft syndrome-based input that informs the encoder about potential parity constraint violations, and 2) a code-aware attention mask derived from the Tanner graph. We further show that end-to-end training with a differentiable belief propagation (BP) decoder offers no additional gain, as MaskCode's structure-aware design already internalizes the structural knowledge of the outer code; in fact, backpropagation through the iterative BP decoder introduces gradient explosion, which degrades rather than improves performance. Extensive evaluations on BCH and LDPC outer codes demonstrate that MaskCode consistently outperforms all baselines, achieving up to 1.5 dB SNR gain.

Comment
15 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Do Language Models Need a Trainable Input Embedding Table? Fixed Minimal Token Codes at 1.7B-Class Scale

cs.CL ·2026-10-02 ·A. Bochkov
similarity 0.57
Abstract

A trainable input embedding table assigns each vocabulary item an independently adjustable vector. We investigate whether this token-specific parameterization is required for substantial language-modeling capability, or whether a shared Transformer can learn from fixed token identities. We compare three decoder-only language models trained from scratch with the same tokenizer, contextual backbone, untied output-head architecture, and training recipe, with a target budget of 100 billion prediction tokens per model. Their input interfaces are a learned table, canonical 16-bit token-ID codes, and one fixed invertible recoding over GF(2). The fixed codes are repeated to model width without an additional trainable input projection. Both fixed-code models acquire substantial capabilities: canonical codes achieve 52.40\% HellaSwag normalized accuracy, 70.51\% PIQA accuracy, and 42.75\% LAMBADA accuracy. The learned-input control performs better on several evaluations, including HellaSwag and LAMBADA, so these results establish viability rather than performance parity. The fixed interfaces remove 100.7 million trainable parameters, yielding 1.711B-parameter models, but parameter reduction is not the central result. These single-run experiments distinguish architectural necessity from empirical utility: independently trainable token-specific input vectors are not required for the observed capabilities. A fixed identity interface also provides a controlled setting for studying representation learning downstream of an immutable input, without establishing where particular capabilities are localized.

0.41

HiER-BLS: A Hierarchy-Guided and Error-Correcting Robust Incremental Broad Learning System

cs.LG ·2026-10-05 ·Gongli Zhang, C. L. Philip Chen, Zhulin Liu
similarity 0.57
Abstract

Broad Learning System (BLS) supports analytical training and incremental expansion, but its growth needs guidance on which inputs new blocks should learn from. Weight errors pose a further challenge by displacing learned outputs across class boundaries. We propose HiER-BLS to couple hierarchy-guided representation growth with error-correcting learning. Successive blocks focus on inputs selected by feature importance and correlation while preserving earlier representations. The evolving branch guides encoded learners through subspace size and sample confidence, so its learning experience informs both their feature views and supervision. For finite broad readouts, we show how codeword correlations transform fitted class scores. Prediction preservation depends on the distance from the actual output to the nearest decoding boundary relative to the model's sensitivity to weight errors. Experiments on five image and five tabular datasets demonstrate improved classification performance over representative BLS variants. Component studies show that hierarchy guidance benefits the encoded branch even when the guiding branch has lower standalone accuracy, with further gains from combining their scores. Longer codes continue to improve accuracy under stronger Gaussian weight errors after clean accuracy has largely saturated.

0.27

Neural Classifiers as Error-Correcting Codes

cs.LG cs.IT ·2026-11-13 ·Sergey Salishev, Anton Makarov, Oleg Granichin, Renata Avros, Zeev Volkovich
publishedhot direction similarity 0.57
Abstract · 43 pages, 6 figures. v2: author accepted manuscript; includes published-version DOI link and CC BY-NC-ND 4.0 notice. Published in Informatio…

A unified framework is proposed that characterizes supervised deep classification as the decoding of error-correcting codes in the Shannon sense. Cross-entropy optimization is shown to align softmax classifiers with maximum-likelihood decoding, where logits map to Hamming distances to class-specific codewords. This correspondence explains the emergence of standard divergence objectives (e.g., Kullback-Leibler, Jeffreys, Bregman) and provides a geometric basis for the stability of decision boundaries. Furthermore, the geometries of Neural Collapse and LeJEPA are identified as optimal constellations for decoding under isotropic noise. Unlike classical Error-Correcting Output Codes, deep networks are shown to learn effective codebooks in the feature space implicitly. This insight is leveraged to derive novel criteria for lossless quantization, improved knowledge distillation, and architectural regularization based on code redundancy. Finally, this perspective is extended to self-supervised encoders, offering a cohesive explanation for the geometry of latent representations in modern neural models.

0.40

Nonlinear Posterior-Mean Feedback Codes for AWGN Channels

cs.IT ·2026-10-04 ·Yingyao Zhou, Natasha Devroye, Milos Zefran, Gyorgy Turan
similarity 0.57
Abstract

Feedback can improve the reliability of communication over additive white Gaussian noise (AWGN) channels. Classical feedback codes are interpretable, or easy to understand, but often rely on linear estimation, while deep-learned feedback codes can achieve strong performance but require many learned parameters and are often black boxes that are difficult to interpret. In this work, we propose a new posterior-mean feedback coding framework for AWGN channels with feedback. The proposed scheme uses posterior-mean refinement to construct nonlinear feedback codes under both noiseless passive feedback and noisy active feedback, with maximum a posteriori (MAP) decoding at the receiver. This scheme is analytically described, but a few learned parameters related to power allocation, and hence we consider this an ``interpretable,'' or human-understandable code. We further develop a projection-based design to improve robustness under noisy feedback and support larger message sizes. Numerical results show that the proposed schemes achieve strong finite-blocklength performance and outperform several analytical and learned feedback coding baselines, while using only a small number of learned design parameters.

0.21

Learning to Decode Linear Codes with Generalized Belief Propagation Network

cs.AI ·2026-11-04 ·Samira Malek
similarity 0.55
Abstract

Error correcting codes are a fundamental component of communication systems, enabling reliable data transmission over noisy channels. Recently, neural decoders have demonstrated notable performance gains over classical iterative algorithms. In this paper, we propose a Generalized Belief Propagation Network (GBPNet) that preserves message-passing symmetry while enhancing the decoder's learning capability. The architecture uses cascaded variable-node updates and learned element-wise message scaling while retaining the Tanner graph structure. We also introduce two training strategies, progressive training and high-SNR training, that improve decoding accuracy. Experimental results on several linear codes show that GBPNet achieves lower bit error rates than conventional belief propagation and the evaluated state-of-the-art neural decoders.

0.25

Learning-Theoretic Foundation for General Coded Computing: The Straggler Setting

cs.LG ·2026-08-28 ·Parsa Moradi, Behrooz Tahmasebi, Mohammad Ali Maddah-Ali
theory similarity 0.55
Abstract

Coded computing has emerged as a powerful paradigm for mitigating the impact of straggling workers in distributed computing systems. However, existing coded-computing schemes are predominantly designed for the exact recovery of highly structured computations, such as polynomial evaluation and matrix multiplication, and typically rely on strict recovery thresholds. These assumptions significantly limit their applicability to modern machine-learning workloads, particularly deep neural networks (DNNs), whose computations generally lack rigid algebraic structure and, in many applications, require only accurate approximations rather than exact recovery. To address this gap, we revisit coded computing from a learning-theoretic perspective and introduce General Coded Computing (GCC). Rather than adopting existing algebraic tools, GCC formulates coded computing through a natural end-to-end mean-squared error loss that directly measures the discrepancy between the desired computations and their recovered estimates. By deriving suitable upper bounds and restricting the encoder and decoder to a reproducing kernel Hilbert space (RKHS) with mild smoothness constraints, we show that both the encoder and decoder admit specific representations as linear combinations of RKHS kernel functions. This representation allows the corresponding coefficients to be computed efficiently. Moreover, this framework enables us to establish theoretical performance guarantees for GCC under two complementary straggler regimes. In the worst-case setting with $N$ worker nodes, and at most $S$ stragglers, we show that the end-to-end loss decays at least at rate $O(S^3N^{-3})$ for standard configurations. We then study a probabilistic setting in which each worker independently straggles with probability $p$. We prove that the expected loss can still converge at rate $O(\log_{1/p}^3(N)N^{-3})$.

0.44

On the Multi-Weight Shift-Sum Iterative Decoding of Short BCH Codes

cs.IT eess.SP ·2026-10-12 ·Yinglei Yang, Di Zhang, Aimin Li, Yifei Shen, Alexey Frolov +1 more
publishedtheoryhot direction similarity 0.55
Abstract

This paper investigates a shift-sum decoding algorithm that employs dual codewords. Existing dual codeword shift-sum decoding schemes mainly rely on minimum-weight dual codewords. However, the number of such codewords is often limited, which restricts the amount of reliability information available to the decoder. To address this issue, we extend the component counting mechanism from minimum-weight dual codewords to dual codewords whose Hamming weights lie within a prescribed upper bound. We further introduce a normalized component counting rule to suppress the overcounting bias caused by the much larger populations of higher-weight classes. Based on this framework, we develop a hard-decision iterative error reduction decoder and a soft information-aided iterative error reduction decoder. Numerical results show that the proposed method consistently outperforms existing minimum-weight shift-sum baselines especially for short codes where additional dual codeword information is particularly valuable.

0.51

PIC: Revisiting INR for Image Coding with Fast Encoding and Sub-Millisecond Decoding

cs.CV ·2026-09-08 ·Xiang Liu, Jinxiang Wang, Bin Chen, Zimo Liu, Mingyao Hong +3 more
top venuecode/data similarity 0.54
Abstract · Accepted at ECCV 2026. Code is available at https://github.com/actcwlf/PIC

Implicit neural representation (INR) has achieved remarkable progress in novel view synthesis and image/video coding in recent years.Compared to conventional end-to-end image codecs, INR-based compressors demonstrate significant advantages in decoding complexity. However, their practical application has been hindered by the inferior encoding speed and underutilized decoding efficiency.In this work, we propose a feedforward INR image coding architecture, Practical INR Image Codec (PIC), that computes all the necessary information for INR network in a single forward pass, achieving an encoding speed of 20 FPS. Additionally, we implement a highly optimized decoder that reaches 2000 FPS decoding speed, significantly surpassing JPEG's performance at comparable rate-distortion (RD) performance. To the best of our knowledge, this work presents the first learning-based image codec that simultaneously outperforms or is comparable with JPEG in both RD performance and decoding speed while maintaining practical encoding speed. Code is available at https://github.com/actcwlf/PIC.

0.39