CS Paper Radar

Image CL-SEC: Cross-Layer Semantic Error Correction for Image Communication

Feifan Zhang, Xuelin Bai, Yirun Wang, Yuyang Du, Soung Chang Liew

eess.IV ·submitted 2026-10-19 05:15 UTC · v2

Abstract

Semantic error correction (SEC) uses the semantic knowledge of pretrained models to complement conventional channel decoding. By combining physical-layer decoding information with application-layer semantic predictions, cross-layer SEC (CL-SEC) has demonstrated substantial performance gains in text transmission; however, its applicability to image transmission remains largely unexplored. This paper proposes Image CL-SEC, which, to the best of our knowledge, is the first extension of the CL-SEC framework to image transmission. Image CL-SEC uses channel-decoding information together with contextual predictions from a pretrained visual model to resolve residual uncertainty after forward error correction (FEC) decoding. In our implementation, images are represented as discrete visual tokens. For each token position selected for re-evaluation, the probability distribution produced by the channel decoder is combined with the distribution predicted by the visual model to identify the most likely transmitted token. The resulting decision therefore reflects both channel evidence and visual context. Experiments across multiple channel conditions show that, at low and moderate signal-to-noise ratios (SNRs), Image CL-SEC improves pixel-level fidelity, perceptual similarity, and token-recovery accuracy relative to either channel decoding or visual-model inference alone. Supplementary experiments using different pretrained visual models further demonstrate that these gains are not specific to a single model.

Closest papers

Nearest neighbours by meaning across the whole index.

Beyond Token Accuracy: Prioritizing What Matters for Visual Reconstruction

cs.CV ·2026-10-02 ·Zhicheng Liu, Zhouxiang Zhao, Chenliang Wu, Zhaohui Yang, Zhaoyang Zhang
similarity 0.68
Abstract

Tokens have become a unified interface for multimodal foundation models, making visual-token communication a natural paradigm for efficient image delivery. However, existing methods typically rely on static policies that cannot jointly adapt to image content and channel conditions. Moreover, their token-level utility objectives do not necessarily translate into improved image reconstruction quality. In this paper, we propose AdapToC, an adaptive, reconstruction-oriented visual-token communication framework. At the transmitter, an adaptive selector jointly models image content, channel state, and communication budget to perform instance-wise resource allocation. Rather than using a fixed token rate and protection policy, it dynamically determines how many tokens should be transmitted and assigns different protection levels according to token importance and current channel conditions. At the receiver, an adaptive MaskGIT receiver incorporates channel reliability into contextual token modeling. It distinguishes tokens with different reliability levels, preserves high-confidence observations, corrects potentially corrupted tokens, and iteratively reconstructs missing content from the received evidence and global visual context. By co-designing token quantity, unequal protection, and reliability-aware recovery for image-level reconstruction quality, AdapToC achieves a peak mean PSNR gain of 4.20 dB over the strongest static baseline under matched communication costs and state-of-the-art performance among the evaluated visual-token communication methods.

0.24

Flow Matching-Based Semantic Communication With Time-Aware Structural Residual Injection

eess.IV cs.CV ·2026-10-06 ·Chengyang Liang, Dong Li
similarity 0.67
Abstract

Generative semantic communication (SemCom) enables low-bandwidth image transmission by reconstructing images from semantic descriptions. However, text-only schemes often suffer from poor structural fidelity due to the stochastic nature of generative models, while still requiring high transmission payload and inference latency. This letter proposes a dual-information-flow SemCom framework based on Flow Matching. At the transmitter, the most semantically matched prompt is selected from multiple candidate descriptions, while an ultra-low-dimensional structural residual velocity vector is extracted from the inversion trajectory of the original image. At the receiver, a Time-Aware Recursive Residual Injection (TARRI) network progressively releases structural corrections along the denoising trajectory using the transmitted residual vector as structural memory. The proposed framework transmits only prompt embedding and a compact residual vector, achieving robust high-fidelity image reconstruction under extremely low payload. Experimental results demonstrate that the proposed framework significantly improves both structural fidelity and perceptual quality over existing text-driven semantic communication schemes.

0.21

Semantic Error Control Coding with Foundation Models for Future Communications

cs.IT ·2026-08-12 ·Chentao Yue, Gaoyang Pang, Branka Vucetic, Yonghui Li
similarity 0.66
Abstract · This work has been submitted to the IEEE for possible publication

Classical channel decoding typically treats all information sequences as equally likely and relies primarily on the channel observations and code structure, without exploiting statistical or semantic structure in the source data. Although source compression is designed to remove redundancy, practical source coding can leave substantial residual structure that conventional channel decoders do not exploit. Modern multimodal data sources, including text, speech, and images, exhibit rich statistical and semantic dependencies that foundation models can learn and exploit to improve channel decoding. This article introduces semantic error control coding (SECC), which seamlessly integrates the semantic structure of the source into encoding and decoding through a foundation model. The semantic source prior, represented by the model's a priori probability of the source content, directs code redundancy toward semantically important content at the encoder, and improves reliability estimation, candidate search, and error detection/correction at the decoder. The channel code keeps its algebraic structure, and its constraints ensure that the semantic suggestions from the foundation model comply with this structure. We describe the SECC framework, classify its design methods into three approaches, and demonstrate representative schemes on text sources. The demonstrated schemes show several decibels of coding gain over conventional decoding on AWGN channels, and reach error rates below the normal approximation bound. Finally, we identify several open challenges.

0.43

Cross-Modal Semantic Successive Refinement for Generative Image Communication over Underwater Acoustic Channels

cs.IT cs.CV ·2026-11-04 ·Xu Pei, Bin Wang, Guangxu Zhu, Yi Zhang, Haiyan Fan
similarity 0.65
Abstract

Underwater acoustic communication faces challenges such as limited available bandwidth and complex channel environments, which impose demanding requirements on efficient image transmission and reliable reconstruction. To address these challenges, this paper proposes a generative image communication framework based on cross-modal semantic successive refinement. At the transmitter, text descriptions, compact bottleneck features, and visual side information compressed by residual vector quantization are transmitted. At the receiver, text and bottleneck features are first fused to predict a base estimate of the intermediate visual representation. The received RVQ indices are then used to recover supplementary visual information and perform residual correction, followed by image reconstruction using a one-step Pixel Mean Flow model. During training, a CLIP-based text-image semantic consistency constraint is introduced to preserve the semantic correspondence between generated results and the original text descriptions. During transmission, protection resources are allocated according to the importance of different information branches and RVQ quantization stages. Simulations using COCO 2017 images and measured underwater acoustic channel records from Watermark show that text and bottleneck features provide complementary semantic and structural conditions, RVQ visual side information further improves reconstruction quality, and layered unequal error protection helps improve recovery at low SNRs. Code-rate allocation experiments reveal a tradeoff between the reliability of fundamental conditions and visual-detail reconstruction under a limited visual-feature coding budget, providing guidance for allocating protection resources in underwater acoustic image transmission.

0.21

Spatial Token Interleaving and Two-Pass BP Decoding for Semantic Image Transmission

cs.IT eess.SP ·2026-10-31 ·Yifan Feng, Chentao Yue, Haiyao Yu, Gaoyang Pang, Xinyu Chen +3 more
similarity 0.65
Abstract

Semantic error control coding (SECC) can correct the residual errors of a standard channel code with a foundation model that supplies a prior on the source content, but existing SECC schemes are designed for text. In this paper, we develop SECC for image transmission with spatial token interleaving and a two-pass belief propagation (BP) schedule. At the transmitter, a vector-quantized (VQ) tokenizer converts an image into tokens, which are transmitted in multiple low-density parity-check (LDPC) code blocks of fifth-generation New Radio (5G NR). In raster order, a block that BP fails to decode corrupts consecutive token rows, so the interleaver reorders the tokens such that adjacent tokens are carried by different blocks, without additional transmitted bits. At the receiver, after the first BP pass, token recovery fuses the token probabilities given by BP with the predictions of the autoregressive model Emu3-Gen, and the second BP pass decodes each block again with the fused distributions as priors on the information bits. On Kodak24 at 0.5 channel uses per pixel and a signal-to-noise ratio (SNR) of 0.15 dB, the second pass removes 92% of the token errors left by token recovery, and interleaving reduces the final token error rate by 88%. At medium to high SNR, the proposed receiver exceeds deep joint source--channel coding baselines with the same channel uses by at least 1.2 dB in peak signal-to-noise ratio.

0.20

Joint Semantic-Channel Coding and Modulation for Generative Image Transmission

cs.CV ·2026-10-21 ·Hao Cao, Jingkai Ying, Yinhuan Huang, Zhijin Qin, Yuan Shen
hot direction similarity 0.63
Abstract · 23 pages, 10 figures

Digital image transmission typically optimizes source coding, channel coding, and modulation separately, limiting how source representations adapt to channel impairments. We propose SemQAM, a joint semantic-channel coding and modulation scheme for generative image transmission converting source to modulated symbols, such as standard QAM symbols. Channel impairments create two challenges: they can corrupt the latent representation at the receiver, and this corruption can compound through autoregressive coding, degrading reconstruction quality. First, SemQAM represents latent features with vector quantization (VQ), maps each VQ index to a QAM symbol, and updates the codeword corresponding to the detected symbol so that channel impairments directly affect codebook learning. Moreover, SemQAM uses the conditional expectation of received codewords to approximate the receiver reconstruction and reduce error propagation during autoregressive coding. SemQAM supports multiple SNRs and CBRs in a unified model, while also supporting conventional source coding through SemQAM-C. Experiments show that (i) under the same LDPC and QAM settings, SemQAM-C outperforms OneDC in LPIPS and DISTS with substantially lower complexity, showing that the unified model retains competitive capability for source coding. (ii) Compared with this separated counterpart, SemQAM reduces channel uses by 18.57% at matched LPIPS and outperforms the compared JSCC methods, demonstrating the benefit of joint coding.

0.34

ODC: Orthogonal Drift Correction for Improved Text-to-Image Semantic Alignment at Inference

cs.CV cs.CLcs.LG ·2026-10-31 ·Kasra Kamali, Basim Azam, Tuan Duong Trinh, Jean Honorio, Naveed Akhtar
top venuecode/data similarity 0.62
Abstract · Accepted to Transactions on Machine Learning Research (TMLR) 2026

Text-to-image models have achieved remarkable success in generating high-quality images from textual descriptions. However, they often struggle with "semantic drift," where the generated output fails to precisely align with complex or nuanced text prompts. While recent approaches have attempted to address semantic errors regarding attribute binding or object presence, there remains a gap for a more holistic method that addresses these issues by directly refining the text embeddings of the initial user prompt. In this work, we introduce Orthogonal Drift Correction (ODC), an inference-time guidance method designed to mitigate semantic drift without requiring model retraining or additional user inputs. ODC guides image generation through a two-stage process. In the first stage, it identifies the semantic drift by evaluating the initially generated image against the user prompt in a shared vision-language embedding space. It then isolates the component of this drift vector that is orthogonal to the prompt's direction and translates it back into text via a vocabulary-based surrogate mechanism. In the second stage, it produces refined text conditioning for a second generation pass by feeding both the initial text embedding and the re-embedded drift representation into an adaptive rank-reduced concept removal module. Our experiments demonstrate that ODC consistently improves prompt-image alignment, increasing CLIP scores by up to 5.4% over the baseline on the COALBench-300 and PartiPrompts benchmarks. Furthermore, on the T2I-CompBench complex subset, ODC improves the average 3-in-1 score by 4.5% over the competing methods. As a plug-and-play module, ODC offers a practical and efficient method for improving the reliability of text-to-image models. Code is available at https://github.com/kasrakamali/ODC.

0.44

Baseline-Relative Counterfactual Refinement for Bit-Aware Visual Token Communication

cs.AI ·2026-08-17 ·Jia Guo, Xiaohan Zhao, Changwang Liu, Shuqing He, Chenyang Zhang +2 more
similarity 0.61
Abstract

Generative visual-token communication reduces transmission load by sending only selected discrete tokens and reconstructing missing content at the receiver. However, existing token-selection criteria based on local uncertainty, importance, or diversity do not directly determine whether changing the current selection improves the final reconstruction under the same packet budget. To address this problem, we propose Gated Counterfactual Refinement for Communication (GCR-C), a rollout-style correction layer over Local-MDL. GCR-C constructs a compact diversified candidate set, evaluates each candidate through matched full-budget Local-MDL continuation, and replaces the baseline action only when a positive baseline-relative reconstruction gain is obtained. Experiments on CIFAR-10, STL-10, a coded 5G-LDPC link, and a limited high-resolution Kodak transfer show that GCR-C consistently improves reconstruction quality at active low- and medium-rate operating points without increasing the realized packet rate, while remaining effective across changes in dataset, channel condition, resolution, token grid, and tokenizer. The results also reveal a clear quality--computation tradeoff due to the additional encoder-side counterfactual evaluation.

0.38