CS Paper Radar

Flow Matching-Based Semantic Communication With Time-Aware Structural Residual Injection

Chengyang Liang, Dong Li

eess.IV cs.CV ·submitted 2026-10-06 02:49 UTC · v1

Abstract

Generative semantic communication (SemCom) enables low-bandwidth image transmission by reconstructing images from semantic descriptions. However, text-only schemes often suffer from poor structural fidelity due to the stochastic nature of generative models, while still requiring high transmission payload and inference latency. This letter proposes a dual-information-flow SemCom framework based on Flow Matching. At the transmitter, the most semantically matched prompt is selected from multiple candidate descriptions, while an ultra-low-dimensional structural residual velocity vector is extracted from the inversion trajectory of the original image. At the receiver, a Time-Aware Recursive Residual Injection (TARRI) network progressively releases structural corrections along the denoising trajectory using the transmitted residual vector as structural memory. The proposed framework transmits only prompt embedding and a compact residual vector, achieving robust high-fidelity image reconstruction under extremely low payload. Experimental results demonstrate that the proposed framework significantly improves both structural fidelity and perceptual quality over existing text-driven semantic communication schemes.

Closest papers

Nearest neighbours by meaning across the whole index.

TISC: A Text-Driven Image Semantic Communication System for Faithful Reconstruction

cs.CV cs.NI ·2026-08-17 ·Feifan Zhang, Yuyang Du, Xiaoyan Liu, Soung Chang Liew
similarity 0.67
Abstract

Generative image semantic communication converts an image into a text description and then performs text-to-image reconstruction at the receiver via diffusion-based generative models. This paradigm has attracted broad attention due to its extremely low bandwidth cost. However, existing methods still face two critical bottlenecks across image-to-text (I2T) semantic extraction at the transmitter and text-to-image (T2I) semantic reconstruction at the receiver: (i) semantic loss and distortion in I2T, where holistic image descriptions may omit fine-grained object attributes and spatial-position information, causing the generated text to deviate from the original image semantics; and (ii) insufficient semantic faithfulness in T2I, where even with the same semantically faithful text description, different initial noise settings may lead diffusion-based reconstruction to produce images with different levels of semantic consistency with the original image. These issues jointly limit the semantic faithfulness of image reconstruction. To address them, we propose TISC, a text-driven image semantic communication framework tailored for faithful reconstruction. TISC incorporates two key designs: (1) Tree-Structured Attribute Semantic Extraction (TSASE), which decomposes semantic extraction into global scene, background, and object-level attribute descriptions, covering spatial position, shape/pose, color, material, and other physical attributes for each detected object; and (2) an Initial Noise Optimization (INO) mechanism, which selects an initial noise seed at the transmitter according to a comprehensive similarity score that jointly considers visual and semantic consistency. Experiments on multiple datasets show that TSASE improves object-position recovery and semantic description faithfulness, while the INO parameter study supports the adopted configuration for noise selection.

0.10

Image CL-SEC: Cross-Layer Semantic Error Correction for Image Communication

eess.IV ·2026-10-19 ·Feifan Zhang, Xuelin Bai, Yirun Wang, Yuyang Du, Soung Chang Liew
similarity 0.67
Abstract

Semantic error correction (SEC) uses the semantic knowledge of pretrained models to complement conventional channel decoding. By combining physical-layer decoding information with application-layer semantic predictions, cross-layer SEC (CL-SEC) has demonstrated substantial performance gains in text transmission; however, its applicability to image transmission remains largely unexplored. This paper proposes Image CL-SEC, which, to the best of our knowledge, is the first extension of the CL-SEC framework to image transmission. Image CL-SEC uses channel-decoding information together with contextual predictions from a pretrained visual model to resolve residual uncertainty after forward error correction (FEC) decoding. In our implementation, images are represented as discrete visual tokens. For each token position selected for re-evaluation, the probability distribution produced by the channel decoder is combined with the distribution predicted by the visual model to identify the most likely transmitted token. The resulting decision therefore reflects both channel evidence and visual context. Experiments across multiple channel conditions show that, at low and moderate signal-to-noise ratios (SNRs), Image CL-SEC improves pixel-level fidelity, perceptual similarity, and token-recovery accuracy relative to either channel decoding or visual-model inference alone. Supplementary experiments using different pretrained visual models further demonstrate that these gains are not specific to a single model.

0.15

Lightweight Generative Image Semantic Communication over Packet Erasure Channels

eess.SP ·2026-09-07 ·Yufei Bo, Haoshuo Zhang, Meixia Tao, Jing He, Xinming Huang
similarity 0.65
Abstract

This paper addresses packet loss in semantic communication caused by network congestion or channel fluctuations. We propose LGSemCom, a lightweight generative packet-level joint source-channel coding (JSCC) framework for efficient and robust image transmission over packet erasure channels. Unlike conventional distortion-oriented recovery methods that yield blurry averages over erased regions, LGSemCom reformulates image recovery under packet erasures as a conditional generative task. By leveraging adversarial learning, the proposed decoder synthesizes plausible details without increasing complexity during inference. A key innovation of our framework is an erasure-aware weighting (EAW) strategy, which prioritizes generation in erased regions while preserving pixel fidelity in correctly received areas. To ensure computational efficiency, LGSemCom employs a fully convolutional codec based on efficient long-range attention blocks (ELABs) that capture global semantic dependencies with low complexity. Extensive experiments show that LGSemCom achieves superior perceptual reconstruction quality compared with existing benchmarks under severe packet loss, while achieving an order of magnitude faster inference. These attributes make LGSemCom highly suitable for latency-sensitive applications on resource-constrained edge devices.

0.05

Nexus: Structured Synergy for Efficient Text-to-Image Generation using Rectified Flow Model

cs.CV ·2026-08-17 ·Yizhao Wang
similarity 0.65
Abstract · 12 pages, 4 figures

Diffusion and flow matching models have made significant progress in text-to-image generation, yet high computation, quadratic complexity, and large memory footprint hinder high-resolution synthesis and edge deployment. We propose Nexus, which integrates sparse architecture, linear complexity, and low-bit quantization. It combines MoE feed-forward layers, gated DeltaNet attention, and per-expert low-bit training to reduce computation and memory. Their joint optimization allows Nexus to achieve generation quality comparable to mainstream models such as SDXL and SD3 while delivering markedly higher inference efficiency. Experiments on COCO and LAION validate its effectiveness.

0.16

Restoring Semantic Fidelity for Training-Free High-Resolution Image Generation in Diffusion Transformers

cs.CV ·2026-11-01 ·Junjie Li, Zengbin Wang, Gaoxiang Li, Muyi Sun, Yixin Zhang +1 more
code/data similarity 0.65
Abstract · accept by acm mm2026

Diffusion Transformer (DiT)-based text-to-image models have recently achieved remarkable success in image synthesis. However, when directly applied beyond their training resolution, these models often suffer from severe semantic degradation, where meaningful semantic content disappears. Although existing training-free high-resolution generation methods can mitigate this issue, the inherent semantic degradation at high resolutions remains largely unresolved, leading to degraded fine-grained details. Furthermore, widely used structural guidance methods lack modeling of the structure and details in the intermediate trajectory, which degrades the quality of the generated results. In this paper, we propose a training-free framework for high-resolution image generation to address these challenges. First, we propose a text-visual scale-decoupled attention (TVSDA) module, which restores the model's ability to generate semantically meaningful content at high resolutions by preserving the global receptive field of text tokens while aligning the receptive field of image tokens with the training resolution. Second, we propose a structure-detail guidance (SDG) strategy that explicitly models structural information and fine details through scale-wise and temporal-wise guidance, overcoming the limitations of conventional low-resolution guidance. Extensive experiments demonstrate that our method significantly improves both structural coherence and detail fidelity over existing training-free DiT-based high-resolution generation approaches. The code is available at https://github.com/ljjcoder/Restoring-Semantic-Fidelity.

0.17

Cross-Modal Semantic Successive Refinement for Generative Image Communication over Underwater Acoustic Channels

cs.IT cs.CV ·2026-11-04 ·Xu Pei, Bin Wang, Guangxu Zhu, Yi Zhang, Haiyan Fan
similarity 0.64
Abstract

Underwater acoustic communication faces challenges such as limited available bandwidth and complex channel environments, which impose demanding requirements on efficient image transmission and reliable reconstruction. To address these challenges, this paper proposes a generative image communication framework based on cross-modal semantic successive refinement. At the transmitter, text descriptions, compact bottleneck features, and visual side information compressed by residual vector quantization are transmitted. At the receiver, text and bottleneck features are first fused to predict a base estimate of the intermediate visual representation. The received RVQ indices are then used to recover supplementary visual information and perform residual correction, followed by image reconstruction using a one-step Pixel Mean Flow model. During training, a CLIP-based text-image semantic consistency constraint is introduced to preserve the semantic correspondence between generated results and the original text descriptions. During transmission, protection resources are allocated according to the importance of different information branches and RVQ quantization stages. Simulations using COCO 2017 images and measured underwater acoustic channel records from Watermark show that text and bottleneck features provide complementary semantic and structural conditions, RVQ visual side information further improves reconstruction quality, and layered unequal error protection helps improve recovery at low SNRs. Code-rate allocation experiments reveal a tradeoff between the reliability of fundamental conditions and visual-detail reconstruction under a limited visual-feature coding budget, providing guidance for allocating protection resources in underwater acoustic image transmission.

0.21

Unlocking Spatial Grounding in Flow-Matching Models via SNR Tuning

cs.CV cs.LG ·2026-10-21 ·Martin Thißen, Elke Hergenröther
publishedbenchmark similarity 0.63
Abstract · Accepted at BMVC 2026 (Oral)

Despite the success of foundation models in image synthesis, achieving fine-grained spatial control remains a significant challenge, typically requiring auxiliary structural adapters or complex inference-time steering. While modern vision-language (VLM) text encoders possess the latent capacity to represent spatial coordinates, generative models often fail to leverage this information. We demonstrate that the signal-to-noise ratio (SNR) distribution is a critical determinant of spatial grounding in flow-matching models. Using a novel synthetic benchmark of geometric primitives, we isolate structural mechanics from textural complexity inherent in natural imagery and introduce a rigorous color-aware layout metric to quantify spatial fidelity. We show that training on a centered SNR distribution results in poor layout adherence (25.9%). Instead, we identify high-noise regimes as a "critical learning phase" for spatial grounding. Shifting the training focus toward these stages increases layout adherence to 80.8%, effectively reaching the 83.6% reconstruction ceiling of the underlying variational autoencoder. We find that the necessity for this shift scales with resolution and aspect ratio variance, and that the conservative noise schedules employed by models like FLUX.1 are insufficient for precise layout adherence. Finally, we show that these findings translate to real-world imagery via the COCO dataset (1.0 vs. 57.3 Average Precision), where state-of-the-art layout adherence is unlocked through distribution tuning alone, without architectural modifications.

0.24

Flow Straight to Reality: Perceptually Consistent Flow Matching for Efficient Image Restoration

cs.CV cs.AIcs.LG ·2026-08-11 ·Sangwoo Jo, Donggeun Ko, Jayeon Kang, Youngsang Kwak, Jaehwa Kwak +1 more
top venuecode/data similarity 0.63
Abstract · Accepted to ECCV 2026. Code is available at https://github.com/aiimaginglab/PCFlow

Image restoration is fundamentally constrained by the tradeoff between distortion and perception: minimizing pixel-wise error yields over-smoothed results, whereas optimizing for perceptual realism often introduces structural deviations. Recent approaches attempt to balance this tradeoff via posterior sampling or multi-stage generative pipelines, yet remain computationally expensive and architecturally complex. To overcome these limitations, we propose PCFlow (Perceptually Consistent Flow Matching), a unified framework that directly parameterizes a continuous transport from degraded observations to clean targets, jointly optimizing distortion and perceptual quality. While its latent consistency flow objective drives stable and efficient few-step inference, a Latent Consistency Perceptual Loss (LCPL) imposes semantic constraints directly on the guiding velocity field, steering the dynamics toward visually sharp data manifolds. Furthermore, recognizing the inherent conflict between structural and perceptual consistencies, we integrate a conflict-free gradient projection strategy to stabilize the multi-objective optimization landscape. Combined with lightweight, convolution-only backbone, PCFlow achieves competitive performance across diverse restoration tasks at a fraction of traditional computational costs.

0.89