CS Paper Radar

Channel-Adaptive Region Adjacency Graph Carriers for Semantic Image Communication

Karim Abdallah, Maria Slim, Mariette Awad, Hadi Sarieddeen

eess.SP cs.LG ·submitted 2026-09-13 15:46 UTC · v1
published

Abstract

Semantic image communication seeks to preserve task-relevant scene structure under limited channel resources, but carriers are often dense latent tensors or grid-aligned semantic layouts that do not explicitly encode region-level relations. This work introduces a segmentation-derived region adjacency graph (RAG) carrier, termed channel-adaptive RAG (CA-RAG), for joint source-channel coding-style image communication. Nodes store interpretable region attributes, edges preserve adjacency, channel-adaptive graph simplification (CGS) controls the node budget, and semantic belief propagation refines noisy graph embeddings before diffusion-based reconstruction. On Cityscapes, pre-channel RAG payloads are several times smaller than compressed class-index layouts in a 2,000-image study. Under additive white Gaussian noise at signal-to-noise ratios from 0 to 15 dB, CA-RAG reports higher semantic consistency than deep joint source-channel coding and a same-decoder layout baseline, with comparable perceptual quality. At 10 dB, the full-budget rate-sweep point reaches mean intersection over union (mIoU) = 0.329 at approximately 3.3 x 10^3 channel uses, while the default adaptive-CGS setting reports mIoU = 0.294 at approximately 2.6 x 10^3 channel uses.

Comment
6 pages, 5 figures, 7 tables. Accepted for presentation at IEEE GLOBECOM 2026, Macau. This is the author's accepted version; the final published version will be available via IEEE Xplore

Closest papers

Nearest neighbours by meaning across the whole index.

FlowSem: Flow Matching for Adaptive Wireless Image Transmission in Semantic Communication

eess.IV ·2026-08-21 ·Arnab Emon, Eslam Eldeeb
similarity 0.62
Abstract

Wireless image transmission becomes challenging under poor channel conditions and stringent bandwidth constraints, as the receiver needs to preserve both pixel-level fidelity and meaningful visual structure. Classical separation-based systems, such as better portable graphics with low-density parity-check coding (BPG+LDPC), may suffer from cliff-effect behavior. While deep joint source-channel coding (DeepJSCC) provides graceful degradation as channel conditions worsen, its reconstructions may lose fine details under strong compression and severe channel distortion. To address this limitation, this paper proposes a two-stage flow matching-based semantic communication framework, termed FlowSem. In the first stage, a signal-to-noise ratio (SNR)-adaptive DeepJSCC model maps the source image into channel symbols and produces a coarse reconstruction at the receiver. In the second stage, a conditional flow matching model generates the final image from Gaussian noise conditioned on the DeepJSCC reconstruction and channel SNR. The proposed framework is evaluated on the Cityscapes dataset under additive white Gaussian noise (AWGN) and Rayleigh fading channels using a fixed channel-symbol budget. The baselines include a rate-matched BPG+LDPC system, DeepJSCC, a denoising diffusion probabilistic model (DDPM), and a denoising diffusion implicit model (DDIM). Results show that FlowSem achieves competitive pixel-level fidelity and improved structural and perceptual reconstruction quality over the considered generative baselines across different channel conditions. FlowSem provides up to 60% lower Fréchet Inception Distance (FID) than the diffusion baselines at low SNRs. Moreover, FlowSem reaches high reconstruction quality using only a few ODE integration steps, providing a favorable quality-latency tradeoff compared with standard DDPM and accelerated DDIM sampling.

0.36

Image CL-SEC: Cross-Layer Semantic Error Correction for Image Communication

eess.IV ·2026-10-19 ·Feifan Zhang, Xuelin Bai, Yirun Wang, Yuyang Du, Soung Chang Liew
similarity 0.61
Abstract

Semantic error correction (SEC) uses the semantic knowledge of pretrained models to complement conventional channel decoding. By combining physical-layer decoding information with application-layer semantic predictions, cross-layer SEC (CL-SEC) has demonstrated substantial performance gains in text transmission; however, its applicability to image transmission remains largely unexplored. This paper proposes Image CL-SEC, which, to the best of our knowledge, is the first extension of the CL-SEC framework to image transmission. Image CL-SEC uses channel-decoding information together with contextual predictions from a pretrained visual model to resolve residual uncertainty after forward error correction (FEC) decoding. In our implementation, images are represented as discrete visual tokens. For each token position selected for re-evaluation, the probability distribution produced by the channel decoder is combined with the distribution predicted by the visual model to identify the most likely transmitted token. The resulting decision therefore reflects both channel evidence and visual context. Experiments across multiple channel conditions show that, at low and moderate signal-to-noise ratios (SNRs), Image CL-SEC improves pixel-level fidelity, perceptual similarity, and token-recovery accuracy relative to either channel decoding or visual-model inference alone. Supplementary experiments using different pretrained visual models further demonstrate that these gains are not specific to a single model.

0.15

Token-Oriented Semantic Communication with Pretrained Vision Transformers

eess.SP cs.AIcs.CVcs.LG ·2026-08-26 ·Jiwoong Im, Minwoo Kim, Jaeho Lee, Yo-Seb Jeon, Yongjune Kim
similarity 0.60
Abstract

Token communications realize the semantic communication principle at the granularity of transformer tokens, providing a promising direction for client--server collaborative inference in resource-constrained edge systems. However, directly transmitting token embeddings presents two practical challenges: substantial communication cost and limited interoperability across model-specific token embedding spaces. To address these challenges, we propose a \emph{token-oriented} semantic communication framework. In this framework, token-level task relevance determines which compressed image latents are transmitted, enabling token-granular transmission without directly transmitting token embeddings. The framework is modular, coordinating three pretrained components---a lightweight client-side vision transformer (ViT), a learned image compression (LIC) model, and a large server-side ViT---without end-to-end training. The key enabler is the one-to-one spatial alignment between ViT patch tokens and the LIC latent vectors, which allows token-level task relevance to directly determine which latent vectors are transmitted. Building on this alignment, token-aligned LIC selectively transmits task-relevant latents, layer-selective attention rollout estimates token relevance from a selected range of attention layers in a single forward pass, and surrogate token substitution adapts the frozen server model by optimizing a single learnable token. Experiments on ImageNet show that the proposed framework achieves a more favorable rate--accuracy trade-off than recent semantic communication schemes, hand-crafted codecs, and task-agnostic LIC models.

0.23

Lightweight Generative Image Semantic Communication over Packet Erasure Channels

eess.SP ·2026-09-07 ·Yufei Bo, Haoshuo Zhang, Meixia Tao, Jing He, Xinming Huang
similarity 0.59
Abstract

This paper addresses packet loss in semantic communication caused by network congestion or channel fluctuations. We propose LGSemCom, a lightweight generative packet-level joint source-channel coding (JSCC) framework for efficient and robust image transmission over packet erasure channels. Unlike conventional distortion-oriented recovery methods that yield blurry averages over erased regions, LGSemCom reformulates image recovery under packet erasures as a conditional generative task. By leveraging adversarial learning, the proposed decoder synthesizes plausible details without increasing complexity during inference. A key innovation of our framework is an erasure-aware weighting (EAW) strategy, which prioritizes generation in erased regions while preserving pixel fidelity in correctly received areas. To ensure computational efficiency, LGSemCom employs a fully convolutional codec based on efficient long-range attention blocks (ELABs) that capture global semantic dependencies with low complexity. Extensive experiments show that LGSemCom achieves superior perceptual reconstruction quality compared with existing benchmarks under severe packet loss, while achieving an order of magnitude faster inference. These attributes make LGSemCom highly suitable for latency-sensitive applications on resource-constrained edge devices.

0.05

BudgetPix: Compute-Adaptive Tokenization for Pixel-Space Image Diffusion

cs.CV ·2026-10-08 ·Ozgur Kara, Yujia Chen, Daniel Watson, David Forsyth, James Matthew Rehg +2 more
code/data similarity 0.59
Abstract · More details are available at our project page: https://karaozgur.com/BudgetPix

Most image generation models rely on uniform tokenization, allocating the exact same computational budget to equally-sized image patches. This static paradigm cannot adapt to different resource constraints at inference time, and yields suboptimal quality-cost tradeoff by devoting the same effort to both plain backgrounds and intricate details. We propose BudgetPix, an adaptive tokenization framework that dynamically allocates compute based on visual complexity and spatial layout, enabling flexible computational budgeting at inference time. BudgetPix comprises three key components: (1) an adaptive encoder that maps a fixed-size image to a variable-length token sequence using an entropy-guided quadtree alongside a multi-scale patch embedder; (2) a scale-aware decoder reconstructs fixed-resolution images from multi-scale token sets; and (3) a flexible training and sampling schedule that enables pixel-space denoisers to operate across variable token counts. BudgetPix seamlessly integrates with existing pixel-space diffusion architectures, enabling a single checkpoint to be operated at a wide range of compute budgets. Evaluated on text-to-image generation, BudgetPix matches the fidelity of MiniT2I-L at $512^2$ and PixelDiT at $1024^2$ using just 25% of the original compute budget. In class-conditional generation using a MeanFlow backbone, BudgetPix requires merely 60% of the full compute budget to produce images with near-zero quality degradation, observing a marginal 0.8-point increase in FID. Comprehensive assessments by human and VLM judges confirm that BudgetPix establishes a significantly improved quality-efficiency tradeoff over prior budget-adaptive baselines. More details are available at our project page: https://karaozgur.com/BudgetPix

0.49

Tree-VQ: Progressive Image Compression from Pretrained Vector Quantizers

cs.CV ·2026-09-03 ·Mingming Ma, Xinkun Wang, Tianyi Xu, Qingyu Luo, Fu Li +1 more
similarity 0.59
Abstract

Progressive image compression requires a single embedded representation whose received prefixes can be decoded without re-encoding the source. Modern vector-quantized (VQ) image models provide strong discrete endpoint representations, but conventional flat codeword indices do not define meaningful intermediate states for a neural decoder. We present Tree-VQ, a post-hoc conversion of a pretrained flat VQ tokenizer into a fine-grained, arbitrary-prefix progressive representation while preserving its encoder assignments and every learned leaf vector. The key idea is to organize the original codebook into a balanced binary hierarchy, associate explicit representations with internal nodes, and transmit branch decisions in depth-major order. Consequently, once the image header is available, every payload prefix uniquely specifies a valid latent state: each additional branch bit refines exactly one token, and transmission can therefore be truncated at essentially any payload position rather than only at a small number of stage boundaries. We further adapt one shared decoder on the complete-depth and mixed-depth latent states encountered under such arbitrary truncation, making these densely spaced prefixes useful for reconstruction rather than merely syntactically decodable. On Kodak, Tree-VQ achieves a DISTS-based BD-rate saving of 52.1% relative to ProGIC, while exposing thousands of valid arbitrary-prefix operating points from a single embedded bitstream.

0.14

DecompJSCC: Joint Source-Channel Coding With Decoupled SNR and Rate Adaptation

eess.IV eess.SP ·2026-11-13 ·Yunhao Liu, Jiancun Fan, Haojun Sun, Jie Luo
similarity 0.58
Abstract · 5 pages,5 figures,1 table. Submitted to IEEE Transactions on Vehicular Technology

Deep joint source-channel coding (DeepJSCC) for image transmission must adapt to changes in the signal-to-noise ratio (SNR) and available transmission resources. However, some existing joint-adaptive methods condition the encoder on the requested rate, while others apply static masks after encoding. The former couples latent formation to the rate request, whereas the latter cannot adapt channel selection to image content. To address these limitations, we propose DecompJSCC, which decomposes joint adaptation into SNR-conditioned latent formation and post-encoding rate control within a single model. The encoder applies SNR-conditioned block attention residuals across all stages, aggregating intra-stage and cross-stage semantic representations. At the receiver, an adapter performs SNR-conditioned affine correction. We introduce a post-encoding rate-adaptive channel semantic budgeting (RCSB) module whose shared gate selects channels of the formed latent by image content, adjusts their gains, and keeps exactly the requested count. This keeps the encoder output independent of the rate request for a fixed image and SNR. Experimental results show that DecompJSCC outperforms SwinJSCC and SA-RA-JSCC in peak signal-to-noise ratio (PSNR) across a range of SNR and rate settings, by 0.64 dB and 1.00 dB over SwinJSCC and by 0.97 dB and 0.96 dB over SA-RA-JSCC on Kodak and CLIC2021, while using only 7.5% of SwinJSCC's Adaptive ModNet parameters.

0.17

OSD-JSCC: Neural Joint Source-Channel Coding with One-Step Diffusion

eess.SP cs.IT ·2026-11-04 ·Shunpu Tang, Qianqian Yang
code/data similarity 0.57
Abstract · 6 pages, 5 figures. Code: https://github.com/tangshunpu/OSD-JSCC

Neural joint source-channel coding (JSCC) enables efficient image transmission over noisy channels, yet distortion-optimized approaches often produce blurry reconstructions. While diffusion priors can enhance visual fidelity, their iterative sampling incurs substantial decoding latency. To address this limitation, we propose OSD-JSCC, an SNR-adaptive JSCC framework with a single diffusion step. Specifically, OSD-JSCC transmits pretrained variational autoencoder (VAE) latents through an SNR-conditioned JSCC codec and incorporates two key designs. First, we introduce a stochastic diffusion-state interface that enables the JSCC decoder to predict a Gaussian diffusion state rather than a clean latent, while extracting image-specific semantic tokens. The sampled diffusion state preserves spatial structure and provides a good initialization for one-step refinement, whereas the semantic tokens, together with an SNR token, guide the reconstruction process. Second, we develop SNR-conditioned feature modulation and adaptive timestep selection to jointly adapt latent transmission and diffusion refinement to varying channel conditions. Furthermore, we adopt a three-stage training strategy with low-rank adaptation (LoRA) to efficiently adapt the pretrained diffusion prior to the JSCC task. Extensive experiments on Kodak and CLIC2020 demonstrate that OSD-JSCC achieves superior perceptual quality and decoding efficiency. Compared with existing diffusion-based JSCC baselines, OSD-JSCC reduces LPIPS by approximately 43% on Kodak and 53% on CLIC2020, while achieving up to a 50.6$\times$ reduction in decoding latency.

0.41