CS Paper Radar

DecompJSCC: Joint Source-Channel Coding With Decoupled SNR and Rate Adaptation

Yunhao Liu, Jiancun Fan, Haojun Sun, Jie Luo

eess.IV eess.SP ·submitted 2026-11-13 16:23 UTC · v2

Abstract

Deep joint source-channel coding (DeepJSCC) for image transmission must adapt to changes in the signal-to-noise ratio (SNR) and available transmission resources. However, some existing joint-adaptive methods condition the encoder on the requested rate, while others apply static masks after encoding. The former couples latent formation to the rate request, whereas the latter cannot adapt channel selection to image content. To address these limitations, we propose DecompJSCC, which decomposes joint adaptation into SNR-conditioned latent formation and post-encoding rate control within a single model. The encoder applies SNR-conditioned block attention residuals across all stages, aggregating intra-stage and cross-stage semantic representations. At the receiver, an adapter performs SNR-conditioned affine correction. We introduce a post-encoding rate-adaptive channel semantic budgeting (RCSB) module whose shared gate selects channels of the formed latent by image content, adjusts their gains, and keeps exactly the requested count. This keeps the encoder output independent of the rate request for a fixed image and SNR. Experimental results show that DecompJSCC outperforms SwinJSCC and SA-RA-JSCC in peak signal-to-noise ratio (PSNR) across a range of SNR and rate settings, by 0.64 dB and 1.00 dB over SwinJSCC and by 0.97 dB and 0.96 dB over SA-RA-JSCC on Kodak and CLIC2021, while using only 7.5% of SwinJSCC's Adaptive ModNet parameters.

Comment
5 pages,5 figures,1 table. Submitted to IEEE Transactions on Vehicular Technology

Closest papers

Nearest neighbours by meaning across the whole index.

OSD-JSCC: Neural Joint Source-Channel Coding with One-Step Diffusion

eess.SP cs.IT ·2026-11-04 ·Shunpu Tang, Qianqian Yang
code/data similarity 0.66
Abstract · 6 pages, 5 figures. Code: https://github.com/tangshunpu/OSD-JSCC

Neural joint source-channel coding (JSCC) enables efficient image transmission over noisy channels, yet distortion-optimized approaches often produce blurry reconstructions. While diffusion priors can enhance visual fidelity, their iterative sampling incurs substantial decoding latency. To address this limitation, we propose OSD-JSCC, an SNR-adaptive JSCC framework with a single diffusion step. Specifically, OSD-JSCC transmits pretrained variational autoencoder (VAE) latents through an SNR-conditioned JSCC codec and incorporates two key designs. First, we introduce a stochastic diffusion-state interface that enables the JSCC decoder to predict a Gaussian diffusion state rather than a clean latent, while extracting image-specific semantic tokens. The sampled diffusion state preserves spatial structure and provides a good initialization for one-step refinement, whereas the semantic tokens, together with an SNR token, guide the reconstruction process. Second, we develop SNR-conditioned feature modulation and adaptive timestep selection to jointly adapt latent transmission and diffusion refinement to varying channel conditions. Furthermore, we adopt a three-stage training strategy with low-rank adaptation (LoRA) to efficiently adapt the pretrained diffusion prior to the JSCC task. Extensive experiments on Kodak and CLIC2020 demonstrate that OSD-JSCC achieves superior perceptual quality and decoding efficiency. Compared with existing diffusion-based JSCC baselines, OSD-JSCC reduces LPIPS by approximately 43% on Kodak and 53% on CLIC2020, while achieving up to a 50.6$\times$ reduction in decoding latency.

0.41

DARTS: Decoder-Aware Representation Tuning via Surgery for Model Merging

cs.LG ·2026-08-28 ·Aaryan Ajay Sharma, Sai Nishanth Padala, Seganrasan Subramanian
top venue similarity 0.64
Abstract · Accepted to EMNLP 2026 Main Conference

Model merging combines multiple task-specific fine-tuned LLMs into a single multi-task model without additional training. However, merged models are known to suffer from representation bias: systematic drift between the merged model's hidden states and those of each individual source model. Prior work (Yang et al., 2024a) study and mitigate this bias for encoder-based vision models using a lightweight correction module trained with L1 loss. However, such bias is not studied for decoder models due to their autoregressive nature. We analyze the problem of representation bias in decoder models, and show two challenges absent in encoders: (1) the causal attention mask causes bias to accumulate across token positions, requiring position-dependent correction; and (2) not all token positions are equally important, i.e., high-entropy (decision-critical) positions matter far more than low-entropy ones. To address these challenges, we propose Decoder-Aware Representation Tuning via Surgery (DARTS). DARTS employs a novel entropy-weighted L1 loss to upweight correction at high-entropy positions where errors most affect generation quality, and a per-position additive bias that captures position-dependent error without overparameterization. We perform extensive evaluation on three domains: code generation (HumanEval), mathematical reasoning (GSM8K), and instruction following (AlpacaEval) on Llama-2-7B models, and show DARTS achieves significant improvement over the standard surgery approach while adding negligible parameters ($0.1\%$ of total parameters).

0.39

Single-Model Adaptive Wireless Image Transmission via Feature Sparsity Regularization

eess.IV cs.IT ·2026-08-22 ·Xianghao Cui, Li Lan, Qi He, Bo Che, Chenyuan Feng +2 more
similarity 0.64
Abstract · 16 pages, 13 figures, including supplementary material. v2: Incorporated the supplementary material into the PDF and refined cross-reference…

Learned joint source-channel coding (JSCC) enables robust wireless image transmission by jointly optimizing the transmitter and receiver over differentiable channel models. For bandwidth-limited and time-varying visual links, a single model should support user-adjustable transmission rate and adapt to changing wireless channel conditions, while also dynamically allocating resources according to spatial content. Existing content-adaptive or dynamic allocation schemes often rely on entropy coding, context/probability prediction, explicit rate maps or masks, or auxiliary allocation networks, complicating the encoder-decoder pipeline and increasing side-information overhead. We propose TS-JSCC, a single-model adaptive JSCC framework with tail-structured sparsification. First, an L1-based tail-structured sparsification objective encourages each token to retain an active feature-channel prefix while suppressing trailing ones. This enables content-adaptive feature-channel allocation with compact side information through active-prefix transmission. Second, lightweight stage-wise neural regulating modules use a normalized sparsity-control coefficient and the channel signal-to-noise ratio (SNR) to rescale intermediate features for single-model transmission rate and SNR adaptation. Experiments on CIFAR-10, Kodak, and CLIC2021 under additive white Gaussian noise (AWGN) and Rayleigh fading show that TS-JSCC achieves strong rate-distortion performance against the latest learned-JSCC baselines and remains competitive with the considered idealized separation baselines, while retaining a simple one-shot encoder-decoder without extra structures or computations.

0.21

Task-Oriented Semantic Feature Transmission for Multi-Task Satellite Remote Sensing over Low-SNR Channels

cs.CV cs.LG ·2026-09-17 ·Shuoyuan Sun, Hongyu Wang, Mugen Peng, Wenjia Xu
published similarity 0.62
Abstract

Conventional satellite remote sensing transmission follows a reconstruct-then-infer paradigm that optimizes pixel-level fidelity, creating an objective mismatch with downstream tasks such as classification and detection, especially at low SNR. This paper investigates a task-oriented framework that bypasses image reconstruction and directly transmits semantic features extracted by a multitask-pretrained backbone. A lightweight channel adaptation module (CAM) compresses feature dimensionality for bandwidth reduction, and a feature restorer recovers task-relevant structure after channel corruption. With the backbone frozen, the CAM and task-specific downstream heads are jointly optimized with task and feature-level supervision under random-SNR training. Under the adopted AWGN setting, experiments on scene classification and object detection show consistent gains over reconstruction-oriented JSCC baselines across different SNR conditions, with the largest improvements in the low-SNR regime.

0.38

Semantic-Aware Joint Source-Channel Optimization for Encoder-Agnostic Digital Video Communication

cs.LG ·2026-09-30 ·Xiangben Zhu, Caili Guo, Yang Yang, Chuanhong Liu, Meiyi Zhu
similarity 0.62
Abstract

Video semantic communication has attracted increasing attention as a promising approach to improving video transmission efficiency. However, most existing approaches rely on computationally intensive deep learning-based video encoders and decoders, which hinders their deployment in resource-constrained scenarios. To address this issue, we propose a lightweight semantic-aware joint source-channel optimization (SAJSCO) scheme that can be integrated into existing digital video communication systems as a plug-in module. Specifically, we develop a video communication system model in which the transmitter jointly optimizes source and channel coding parameters based on the inter-frame semantic importance of the input video and estimated channel state information. On this basis, we formulate an optimization problem that maximizes semantic importance weighted video reconstruction quality under a maximum bitrate constraint. To solve it, we first quantify inter-frame semantic importance using a cosine similarity-based metric with a shifted window mechanism. We then develop a multi-actor proximal policy optimization (MPPO) algorithm to solve the formulated problem by jointly adapting the source compression rate and channel coding rate. The learned policy can be directly applied to different video encoders without encoder-specific retraining or fine-tuning. SAJSCO achieves Bjøntegaard Delta rate reductions of 34.86\% and 18.01\% when integrated with H.265, a conventional video encoder, and DCVC-RT, a deep learning-based video encoder. Over-the-air experiments on a hardware testbed further demonstrate a PSNR gain of up to 1.448 dB with H.265 and an LPIPS reduction of up to 0.033 with DCVC-RT compared with the respective best-performing fixed-parameter baselines.

0.23

Energy-Aware Compression-Computation Co-Adaptation for Latency Minimization in Multi-User Semantic Communication

cs.IT cs.ETcs.NI ·2026-08-13 ·Loc X. Nguyen, Yumin Park, Avi Deb Raha, Huy Q. Le, Zhu Han +2 more
similarity 0.62
Abstract · 13 pages, 7 figures, 5 tables

Deep joint source-channel coding-enabled (DeepJSCC) semantic communication (SemCom) has excelled at delivering high perceptual quality at low channel-bandwidth ratios, which positions it as a pillar for next-generation wireless networks. However, the existing works have difficulty accommodating user heterogeneity in terms of communication channel quality, expected quality-of-service (QoS) targets, and the available local energy. Therefore, in this paper, we explicitly reflect the heterogeneity of user devices in terms of the differences in expected QoS, channel condition, and local energy, and then mathematically formulate the problem. Next, we propose an energy-aware compression-computation co-adaptation (CoCo) framework, in which the base station can meet the expected user QoS by transmitting a longer signal or offloading the task to a local device. The user has to dedicate energy to denoising the signal to recover higher-fidelity latent features before feeding it to the semantic decoder. To solve the formulated problem, we first decompose it into two sub-problems: parameter optimization and resource allocation problems. Specifically, we propose a robust codec that effectively works under a diversity of compression rates and channel noise without re-training, while the greedy sub-carrier allocation lowers the communication time. Finally, we present simulation results on standard image datasets over additive white Gaussian noise to demonstrate the effectiveness of CoCo, which reduces total latency relative to rate-only adaptive DeepJSCC or denoising-only, thereby ensuring the demands of each individual user are met.

0.10

FlowSem: Flow Matching for Adaptive Wireless Image Transmission in Semantic Communication

eess.IV ·2026-08-21 ·Arnab Emon, Eslam Eldeeb
similarity 0.62
Abstract

Wireless image transmission becomes challenging under poor channel conditions and stringent bandwidth constraints, as the receiver needs to preserve both pixel-level fidelity and meaningful visual structure. Classical separation-based systems, such as better portable graphics with low-density parity-check coding (BPG+LDPC), may suffer from cliff-effect behavior. While deep joint source-channel coding (DeepJSCC) provides graceful degradation as channel conditions worsen, its reconstructions may lose fine details under strong compression and severe channel distortion. To address this limitation, this paper proposes a two-stage flow matching-based semantic communication framework, termed FlowSem. In the first stage, a signal-to-noise ratio (SNR)-adaptive DeepJSCC model maps the source image into channel symbols and produces a coarse reconstruction at the receiver. In the second stage, a conditional flow matching model generates the final image from Gaussian noise conditioned on the DeepJSCC reconstruction and channel SNR. The proposed framework is evaluated on the Cityscapes dataset under additive white Gaussian noise (AWGN) and Rayleigh fading channels using a fixed channel-symbol budget. The baselines include a rate-matched BPG+LDPC system, DeepJSCC, a denoising diffusion probabilistic model (DDPM), and a denoising diffusion implicit model (DDIM). Results show that FlowSem achieves competitive pixel-level fidelity and improved structural and perceptual reconstruction quality over the considered generative baselines across different channel conditions. FlowSem provides up to 60% lower Fréchet Inception Distance (FID) than the diffusion baselines at low SNRs. Moreover, FlowSem reaches high reconstruction quality using only a few ODE integration steps, providing a favorable quality-latency tradeoff compared with standard DDPM and accelerated DDIM sampling.

0.36

SPIDER: Multi-Layer Semantic Token Pruning and Adaptive Sub-Layer Skipping in Multimodal Large Language Models

cs.CV cs.AI ·2026-09-28 ·Tianxiang Chen, Zhentao Tan, Zi Ye, Yue Wu, Xiaobing Tu +7 more
top venue similarity 0.61
Abstract

Multimodal Large Language Models face significant efficiency challenges that stem from two distinct yet coupled sources: data redundancy and computational redundancy. While most methods focus on data redundancy by pruning visual tokens from the output of the visual encoder or computing redundancy in LLM decoders using blockwise importance, the finer-grained inter-layer representation shifts and the distribution differences within the layers themselves have not been fully explored. In this work, we comprehensively investigate this dual-level inefficiency. We posit that intermediate layer tokens from vision encoders should be considered for effective visual token pruning, as semantic focus shifts across layers, with middle-layer tokens capturing more detailed object-centric information that deeper layers may abstract away. Furthermore, we reveal the differential contributions of Attention and FFNs across distinct LLM decoder layers. Building upon these discoveries, we propose \textbf{SPIDER}, a training-free framework that integrates multi-layer \underline{\textbf{S}}emantic visual token \underline{\textbf{P}}run\underline{\textbf{I}}ng with an a\underline{\textbf{D}}aptive sub-lay\underline{\textbf{ER}} skipping mechanism. Experimental evaluations demonstrate that SPIDER consistently maintains strong performance across various MLLM architectures and reduction ratios. For instance, on LLaVA-NeXT-7B, SPIDER reduces FLOPs by $79\%$ while maintaining 96$\%$ of the baseline performance.

0.32