CS Paper Radar

Spatial Token Interleaving and Two-Pass BP Decoding for Semantic Image Transmission

Yifan Feng, Chentao Yue, Haiyao Yu, Gaoyang Pang, Xinyu Chen, Hao Chang, Branka Vucetic, Yonghui Li

cs.IT eess.SP ·submitted 2026-10-31 06:47 UTC · v1

Abstract

Semantic error control coding (SECC) can correct the residual errors of a standard channel code with a foundation model that supplies a prior on the source content, but existing SECC schemes are designed for text. In this paper, we develop SECC for image transmission with spatial token interleaving and a two-pass belief propagation (BP) schedule. At the transmitter, a vector-quantized (VQ) tokenizer converts an image into tokens, which are transmitted in multiple low-density parity-check (LDPC) code blocks of fifth-generation New Radio (5G NR). In raster order, a block that BP fails to decode corrupts consecutive token rows, so the interleaver reorders the tokens such that adjacent tokens are carried by different blocks, without additional transmitted bits. At the receiver, after the first BP pass, token recovery fuses the token probabilities given by BP with the predictions of the autoregressive model Emu3-Gen, and the second BP pass decodes each block again with the fused distributions as priors on the information bits. On Kodak24 at 0.5 channel uses per pixel and a signal-to-noise ratio (SNR) of 0.15 dB, the second pass removes 92% of the token errors left by token recovery, and interleaving reduces the final token error rate by 88%. At medium to high SNR, the proposed receiver exceeds deep joint source--channel coding baselines with the same channel uses by at least 1.2 dB in peak signal-to-noise ratio.

Closest papers

Nearest neighbours by meaning across the whole index.

Thinking in Tokens, Talking in Bits: A Practical Interface for Token Communication

eess.SP cs.ITeess.IV ·2026-09-14 ·Chanho Park, Bumsu Park, Soonhee Kwon, Sangrim Lee, Namyoon Lee
similarity 0.66
Abstract · 7 pages, 4 figures

Advanced artificial intelligence models think in tokens; contemporary communication systems carry bits. The direct way to bridge this gap is to transmit tokens, but that makes a model-specific representation part of the air interface, coupling the endpoints through a shared tokenizer, codebook, and often a neural transceiver. We take a different route: keep bits in the payload and let tokens control how those bits are generated and protected. The resulting token-bit interface transition aligns task-side tokens with source- and channel-coding units, translates token relevance into codec controls, and preserves the induced priority order across the coding chain. We instantiate it for image classification, where a vision transformer scores the task relevance of each image region from its attention maps: those scores steer block-wise JPEG rate allocation, then group the compressed bits for protection at different polar-code rates. The payload remains an explicit, reconstructable bitstream recovered by a correspondingly configured decoder. Over-the-air experiments on a software-defined radio testbed show improved accuracy--latency tradeoffs over separate source-channel coding, performance competitive with far more memory-intensive neural joint source-channel coding, and graceful degradation under channel mismatch. Token communication, then, need not transmit tokens explicitly; what it needs is an interface through which tokens determine how bits are communicated.

0.14

Beyond Token Accuracy: Prioritizing What Matters for Visual Reconstruction

cs.CV ·2026-10-02 ·Zhicheng Liu, Zhouxiang Zhao, Chenliang Wu, Zhaohui Yang, Zhaoyang Zhang
similarity 0.65
Abstract

Tokens have become a unified interface for multimodal foundation models, making visual-token communication a natural paradigm for efficient image delivery. However, existing methods typically rely on static policies that cannot jointly adapt to image content and channel conditions. Moreover, their token-level utility objectives do not necessarily translate into improved image reconstruction quality. In this paper, we propose AdapToC, an adaptive, reconstruction-oriented visual-token communication framework. At the transmitter, an adaptive selector jointly models image content, channel state, and communication budget to perform instance-wise resource allocation. Rather than using a fixed token rate and protection policy, it dynamically determines how many tokens should be transmitted and assigns different protection levels according to token importance and current channel conditions. At the receiver, an adaptive MaskGIT receiver incorporates channel reliability into contextual token modeling. It distinguishes tokens with different reliability levels, preserves high-confidence observations, corrects potentially corrupted tokens, and iteratively reconstructs missing content from the received evidence and global visual context. By co-designing token quantity, unequal protection, and reliability-aware recovery for image-level reconstruction quality, AdapToC achieves a peak mean PSNR gain of 4.20 dB over the strongest static baseline under matched communication costs and state-of-the-art performance among the evaluated visual-token communication methods.

0.24

Image CL-SEC: Cross-Layer Semantic Error Correction for Image Communication

eess.IV ·2026-10-19 ·Feifan Zhang, Xuelin Bai, Yirun Wang, Yuyang Du, Soung Chang Liew
similarity 0.65
Abstract

Semantic error correction (SEC) uses the semantic knowledge of pretrained models to complement conventional channel decoding. By combining physical-layer decoding information with application-layer semantic predictions, cross-layer SEC (CL-SEC) has demonstrated substantial performance gains in text transmission; however, its applicability to image transmission remains largely unexplored. This paper proposes Image CL-SEC, which, to the best of our knowledge, is the first extension of the CL-SEC framework to image transmission. Image CL-SEC uses channel-decoding information together with contextual predictions from a pretrained visual model to resolve residual uncertainty after forward error correction (FEC) decoding. In our implementation, images are represented as discrete visual tokens. For each token position selected for re-evaluation, the probability distribution produced by the channel decoder is combined with the distribution predicted by the visual model to identify the most likely transmitted token. The resulting decision therefore reflects both channel evidence and visual context. Experiments across multiple channel conditions show that, at low and moderate signal-to-noise ratios (SNRs), Image CL-SEC improves pixel-level fidelity, perceptual similarity, and token-recovery accuracy relative to either channel decoding or visual-model inference alone. Supplementary experiments using different pretrained visual models further demonstrate that these gains are not specific to a single model.

0.15

Baseline-Relative Counterfactual Refinement for Bit-Aware Visual Token Communication

cs.AI ·2026-08-17 ·Jia Guo, Xiaohan Zhao, Changwang Liu, Shuqing He, Chenyang Zhang +2 more
similarity 0.60
Abstract

Generative visual-token communication reduces transmission load by sending only selected discrete tokens and reconstructing missing content at the receiver. However, existing token-selection criteria based on local uncertainty, importance, or diversity do not directly determine whether changing the current selection improves the final reconstruction under the same packet budget. To address this problem, we propose Gated Counterfactual Refinement for Communication (GCR-C), a rollout-style correction layer over Local-MDL. GCR-C constructs a compact diversified candidate set, evaluates each candidate through matched full-budget Local-MDL continuation, and replaces the baseline action only when a positive baseline-relative reconstruction gain is obtained. Experiments on CIFAR-10, STL-10, a coded 5G-LDPC link, and a limited high-resolution Kodak transfer show that GCR-C consistently improves reconstruction quality at active low- and medium-rate operating points without increasing the realized packet rate, while remaining effective across changes in dataset, channel condition, resolution, token grid, and tokenizer. The results also reveal a clear quality--computation tradeoff due to the additional encoder-side counterfactual evaluation.

0.38

Joint Semantic-Channel Coding and Modulation for Generative Image Transmission

cs.CV ·2026-10-21 ·Hao Cao, Jingkai Ying, Yinhuan Huang, Zhijin Qin, Yuan Shen
hot direction similarity 0.60
Abstract · 23 pages, 10 figures

Digital image transmission typically optimizes source coding, channel coding, and modulation separately, limiting how source representations adapt to channel impairments. We propose SemQAM, a joint semantic-channel coding and modulation scheme for generative image transmission converting source to modulated symbols, such as standard QAM symbols. Channel impairments create two challenges: they can corrupt the latent representation at the receiver, and this corruption can compound through autoregressive coding, degrading reconstruction quality. First, SemQAM represents latent features with vector quantization (VQ), maps each VQ index to a QAM symbol, and updates the codeword corresponding to the detected symbol so that channel impairments directly affect codebook learning. Moreover, SemQAM uses the conditional expectation of received codewords to approximate the receiver reconstruction and reduce error propagation during autoregressive coding. SemQAM supports multiple SNRs and CBRs in a unified model, while also supporting conventional source coding through SemQAM-C. Experiments show that (i) under the same LDPC and QAM settings, SemQAM-C outperforms OneDC in LPIPS and DISTS with substantially lower complexity, showing that the unified model retains competitive capability for source coding. (ii) Compared with this separated counterpart, SemQAM reduces channel uses by 18.57% at matched LPIPS and outperforms the compared JSCC methods, demonstrating the benefit of joint coding.

0.34

Disentangling Channel Semantics in Vision Transformers via Token Decorrelation and Composition-Aware Modulation

cs.CV ·2026-11-07 ·Daeun Kim, Hyesong Choi, Hyejin Park, Dongbo Min
top venuecode/data similarity 0.59
Abstract · Accepted at NeurIPS 2026. 32 pages, 12 figures

Multi-channel images encode heterogeneous channel semantics aligned over a shared spatial structure, which requires models to jointly capture globally shared structure and channel-specific variations. Recent multi-channel Vision Transformers attempt to address this challenge by augmenting the [CLS] token with memory tokens to organize multi-channel representations under varying channel compositions. However, we find that these context tokens collapse onto a few dominant channels under high inter-channel redundancy, producing redundant representations and a biased global summary that fails to reliably support composition-aware channel interpretation. We propose MuCa-ViT (Multi-Channel Composition-Aware Vision Transformer), a unified framework that addresses these limitations through two coupled mechanisms. Factorial Token Learning (FTL) enforces token-level decorrelation among context tokens, encouraging the [CLS] token to capture globally shared semantics while memory tokens preserve complementary channel-specific cues. Channel Composition-Aware Modulation (CAM) uses the FTL-refined [CLS] as a semantic anchor to generate sample-wise modulation signals, which enable composition-aware interpretation of channels within a unified backbone rather than relying on static channel embeddings. Experiments on JUMP-CP, CHAMMI, and So2Sat show that MuCa-ViT outperforms prior multi-channel ViTs across all three benchmarks, and the magnitude of improvement aligns with the inter-channel redundancy of each dataset ($ρ\in [0.19, 0.77]$), validating the diagnostic analysis that motivates our design. Our code is available at https://github.com/kimdanni/MuCa-ViT .

0.55

Semantically Steerable Image Tokenization

cs.CV ·2026-10-11 ·Danilo de Goede, Mohammad Mahdi Derakhshani, Yuki M. Asano, Cees G. M. Snoek
similarity 0.59
Abstract

Adaptive image tokenizers compress an image into a variable number of tokens matched to its complexity, and recent work makes this allocation controllable: conditioned on a target reconstruction error, the tokenizer itself decides how many tokens an image needs. However, the fidelity measure guiding this allocation shapes which aspects of the image are preserved under compression. The pixel-wise error used in prior work is blind to the spatial structure that carries an image's meaning, and under heavy compression such tokenizers are prone to abruptly discarding tokens that encode essential content. We introduce GIST (Guided Image Semantic Tokenizer), an adaptive tokenizer that can be steered toward a requested semantic fidelity. Its token allocation is jointly controlled by pixel-level and semantic target errors, with semantic error measured in the feature space of a frozen DINOv3 encoder. This joint control enables semantic protection: with the semantic target fixed at zero, the reconstruction target can be relaxed to discard pixel detail while preserving what the image depicts. In a controlled comparison on ImageNet-100, GIST outperforms its pixel-controlled counterpart KARL at every token count, with up to $30\%$ lower LPIPS and $37\%$ lower DreamSim. On average, lowering its semantic target reduces LPIPS $2\times$ faster per token than lowering the reconstruction target, and up to $5\times$ in the low-token regime. Semantic fidelity is thus an effective control signal for adaptive tokenization: it lets a tokenizer see, and so preserve, what an image depicts.

0.20

Personalized Digital Semantic Communication for Image Transmission with Vision-Language Models

eess.IV cs.MM ·2026-08-14 ·Nan Li, Li Zhou, Haijun Wang, Jun Xiong, Haitao Zhao +1 more
published similarity 0.59
Abstract · Accepted by IEEE GLOBECOM 2026

Semantic communication (SC) enables bandwidth-efficient wireless image transmission, but most existing SC schemes are user-agnostic and ignore receiver-dependent semantics. To address this issue, we propose a personalized digital semantic communication (PDSC) framework that integrates a vision-language model (VLM)-based semantic encoder with a latent diffusion model (LDM)-based semantic decoder. Specifically, the semantic encoder extracts source-aware personalized semantic tokens from both the source image and the receiver's historical interactions. These tokens are vector-quantized into discrete semantic indices and further encoded into a compact fixed-length bitstream, enabling compatibility with digital transmission. At the receiver, the semantic decoder reconstructs a personalized image conditioned on the recovered semantic tokens. Furthermore, we formulate a capacity-constrained personalized semantic rate-distortion problem and introduce a semantic distortion metric that jointly characterizes source-semantic fidelity and user-preference alignment. Experiments show that PDSC achieves superior source-semantic consistency and personalization over state-of-the-art SC baselines, including CDDM and MoS, under bandwidth-limited wireless transmission.

0.22