CS Paper Radar

Learning Nonlinear Waveform Fusion Across Scales for Integrated Sensing and Communication

Jiayu Liu, Kai Wan, Shuo Shao, Giuseppe Caire

cs.AI ·submitted 2026-10-23 03:02 UTC · v1

Abstract

Fixed source superposition and frequency partitioning constrain how communication content interacts with a probing reference in integrated sensing and communication (ISAC). We learn nonlinear waveform fusion across scales for image transmission and localization of multiple targets in range and angle. Reception uses the information available at each end. Adaptive temporal feature weighting and spectral updates combine image features and a deterministic linearly frequency-modulated reference in a shared complex state, generating a waveform with fixed total energy over the array frame. At the user equipment (UE), the reference and channel state information condition a neural demixer for image reconstruction. At the base station, the realized transmit pulse defines a coherent dictionary for sequential localization with learned local correction, physical refinement, and residual updates. We compare the proposed waveforms with superposition in the time domain and learned spectral linear fusion on 512 independent test images, using a shared fixed UE receiver and a baseline sensing receiver using coherent peak search. Averaged across eleven task weights, the proposed waveforms reduce normalized localization error by 29.0\% and 27.0\%, respectively, while increasing mean peak signal-to-noise ratio by 1.40 and 0.82~dB. With the proposed waveforms fixed, learned local correction reduces mean normalized localization error by 12.2\% relative to the same sequential receiver without learned correction.

Closest papers

Nearest neighbours by meaning across the whole index.

Radio Echo Representation Learning for Semantic Sensing in Multi-Modal ISAC Systems

eess.SP ·2026-10-11 ·Guanping Shang, Anzheng Tang, Kwok Kit Chan, Shenghui Song
similarity 0.63
Abstract · This paper has been submitted to the IEEE trans journals

Semantic sensing extends integrated sensing and communication (ISAC) beyond physical parameter estimation by exploiting task-oriented representations from radio echoes. Nevertheless, existing multimodal sensing remains largely limited to vision-assisted radio physical parameter estimation or typically processes the visual and radio modalities separately, leaving radio representations underutilized for visual task enhancement. Feature-level fusion enables closer cross-modal interaction, while echo interference complicates the learning of user-specific radio representations and its semantic consistency with the visual entities. To address these challenges, this paper proposes a multimodal semantic sensing framework for orthogonal frequency-division multiplexing (OFDM)-ISAC systems. We propose a radio encoder to learn radio representations oriented toward visual localization from superimposed pilot echoes, together with Variational Radio-Visual Grounding (VRVG) to promote semantic consistency between the learned radio representations and visual spatial features. The radio representations are then formulated as semantic probes to highlight the corresponding entities in the visual scene through cross-modal fusion. Experiments using real-world RCooper dataset show that the resulting synergistic radio-visual embeddings enable accurate radio-visual user matching and significantly improve both user-specific visual localization and physical parameter estimation performance. The results substantiate the benefits of radio representation learning and semantically consistent fusion for multimodal semantic sensing.

0.30

VISTA: View-Independent micro-Doppler Reconstruction with Multistatic ISAC

cs.AR ·2026-11-11 ·Jacopo Pegoraro, Marco Canil, Aurora Peloso, Jesus O. Lacruz, Xitao Gong +3 more
published similarity 0.60
Abstract · Accepted to be presented in the IEEE ISAC 2026 conference

The micro-Doppler (uD) spectrum provides detailed characteristics of target movement, making it highly valuable for applications such as target classification and human activity recognition. However, traditional Integrated Sensing And Communication (ISAC) systems are highly view-dependent and limited by blind spots and angle dependencies. Existing approaches rely on either configuration-specific solutions or data-driven models. The former results in poor generalization when device deployments or target orientations change, while the latter requires acquiring extensive multi-view datasets, which is time-consuming and not always feasible. To address these limitations, in this work we propose VISTA, a novel method for reconstructing the uD spectrogram in an absolute coordinate frame, yielding view-independent Doppler features that are invariant to the spatial distribution of ISAC nodes. We formulate the uD estimation as a sparse reconstruction problem that jointly leverages measurements from all ISAC receivers. This is solved using a modified Orthogonal Matching Pursuit (OMP) algorithm, which iteratively alternates between jointly selecting candidate velocity vectors across all receivers (RXs) and independently estimating the complex channel gain amplitudes for each RX. Our results show that, provided sufficient spatial diversity, VISTA successfully reconstructs the targets' absolute uD spectrograms with a median error \leq 0.69 m/s for speeds \leq 15 m/s.

0.33

Physics-Guided Multi-Objective Deep Learning for Ultrasound RF Data Interpolation in Resource-Constrained Imaging

eess.IV cs.LG ·2026-09-23 ·Luoyuan Zhang, Yiyang You, Ananya Tandri, Yinan Feng, Hyunwoo Song +2 more
similarity 0.60
Abstract · Submitted to the Journal of Computational Design and Engineering

Ultrasound imaging increasingly targets portable, point-of-care, and wearable settings where constraints on power, bandwidth, and hardware complexity often necessitate sparse data acquisition in spatiotemporal scanning. However, image reconstruction using the sparse data can introduce insufficient phase information in coherent beamforming process, resulting in grating-lobe artifacts that degrade imaging contrast resolution. We present a physics-guided, data-driven framework for sparse-to-dense radio-frequency (RF) reconstruction that aligns training with downstream image formation. Our approach trains an end-to-end interpolation network using a hybrid supervision scheme that combines an RF-domain and a beamforming-domain loss with exponential moving average (EMA) to stabilize the multi-objective training. To improve generalization under variable acquisition layouts, we also introduce a random-skip masking strategy that varies sparsity patterns during training so a single model can handle diverse decimation factors and irregular channel configurations. We evaluate the framework on a held-out test set using the mean structural similarity index measure (SSIM) between reconstructed and ground-truth beamformed images. Across decimation factors $\times 2$ to $\times 13$, the best-performing configuration maintains mean SSIM around 0.95. Overall, the results show consistent gains in RF reconstruction and post-beamforming image quality across diverse acquisition conditions. This approach enables robust, high-quality ultrasound imaging at resource-constrained settings by allowing more sparse scanning in spatiotemporal domain.

0.18

Scaling of Wireless Foundation Models via Representation Diversity and Multi-Branch Architectures

eess.SP cs.AI ·2026-10-03 ·Ahmed Mohamed, Ahmed Aboulfotouh, Hatem Abou-Zeid
similarity 0.59
Abstract

Wireless foundation models learn representations from unlabeled radio signals for reuse across downstream tasks. Scaling model capacity is a common strategy for learning richer representations and improving downstream performance. However, its gains are less consistent in wireless self-supervised learning when pretraining data are limited. We investigate objective diversity as an alternative scaling axis: different self-supervised objectives emphasize different signal properties, and combining their representations can preserve more information to enable diverse tasks. We develop a fusion framework that combines frozen representations from independent encoders trained through reconstructive, predictive, and contrastive learning. This provides a reference for the benefits of diversity, but requires the maintenance of multiple encoders. To retain these benefits within the parameter budget of a standard single-objective encoder, we introduce a jointly trained multi-branch architecture with a shared trunk and objective-specific branches. We pretrain on a heterogeneous corpus of spectrogram and channel state information data and evaluate six downstream tasks that span communication, sensing, and positioning. At an equal output embedding dimension, fusion outperforms the evaluated single-objective encoders on all six tasks while using roughly one-third of their parameters. The multi-branch architecture retains much of the fusion benefit within a single-encoder parameter budget. Representation analyses indicate complementary contributions across objectives, with much of the added benefit retained in components orthogonal to the reconstruction representation subspace. These findings support objective diversity as an effective strategy for scaling wireless foundation models.

0.11

Predictive Beamforming via Echo-Beam Modality Separation with Cross-Attention for ISAC Systems

eess.SP ·2026-10-07 ·Seokjun Park, Jiwhan Jang, Jinseok Choi
published similarity 0.59
Abstract

In this paper, we propose a deep learning-based predictive beamforming algorithm for integrated sensing and communications in vehicle-to-infrastructure networks, where echo signals are leveraged to enhance communication performance. We formulate a sum spectral efficiency maximization problem under a sensing signal-to-interference-plus-noise ratio (SINR) constraint and construct a penalty-based unconstrained surrogate objective for network training. Based on this formulation, the beamforming matrix is predicted using sensing echoes and historical beamformers from previous time slots. To this end, we develop a predictive beamforming network that processes sensing echoes and beamforming inputs through separate encoders to preserve their modality-specific characteristics. This separate design enables each modality to learn rich and complementary representations which are more effectively exploited through a cross-attention mechanism compared to naive input fusion. The resulting features are fused via cross-attention, while gated recurrent units capture temporal dependencies across time slots. Simulation results validate the performance of the proposed method.

0.45

On-Device Super-Resolution Imaging for a Low-Cost SPAD Array on a 640-KB-SRAM Microcontroller

eess.IV ·2026-10-06 ·Zhenya Zang, Istvan Gyongy, Mike Davies
similarity 0.58
Abstract

This work presents a super-resolution (SR) deep-learning (DL) architecture to simultaneously upscale low-resolution (LR) depth and intensity images acquired by a consumer-grade VL53L9CX time-of-flight (ToF) sensor, a compact single-photon avalanche diode (SPAD) ranging device, while targeting deployment on a resource-constrained microcontroller (MCU) with only 640 KB SRAM and 2 MB Flash. The VL53L9CX provides 54 $\times$ 42 depth and intensity measurements. The proposed framework performs $\times$4 spatial SR to reconstruct 216 $\times$ 168 depth and intensity images simultaneously. The network employs separate reconstruction branches for intensity and high-resolution (HR) depth. We train the network with efficient loss functions and investigate the performance by testing multiple compact, hardware-oriented backbones, including Spatially Adaptive Feature Modulation (SAFM), Swift Parameter-Free Attention Network (SPAN), and Residual Local Feature Network (RLFN). We evaluate the network on both synthetic and real measurements, with particular attention to constrained activation and memory storage. We export the selected network to ONNX, quantize it to INT8, generate C inference code for an STM32H563ZI Arm Cortex-M33 MCU, and integrate it with our simplified sensor's firmware. This work is the first to demonstrate a compact DL SR model on a low-cost bare-metal MCU for a consumer-grade LR SPAD sensor, integrated with customized MCU firmware to enable on-device SR inference.

0.22

Super-Resolving Unseen Hyperspectral Sensors at Any Scale via Spatial Operators

cs.CV ·2026-09-30 ·Ji-Xuan He, Guohang Zhuang, Bo Junge, Tingyi Li, Lingchen +4 more
similarity 0.58
Abstract

Achieving cross-sensor generalization and arbitrary-scale reconstruction with a single model remains challenging in hyperspectral super-resolution (HSR). Although recent methods support arbitrary-scale reconstruction, applying them to new sensors or scales beyond the training range often requires additional data and computation to maintain reconstruction quality. To address these challenges, we propose OmniHSR, which predicts band-shared spatial operators rather than spectral values. Cross-Spectral Mapping (CSM) resamples inputs with any number of bands to fixed reference positions and predicts local operators with Gaussian supports. Continuous Operator-Field Reconstruction (COFR) composes these operators into a continuous field and applies them to all original bands for arbitrary-scale reconstruction. Experiments demonstrate that operator prediction outperforms direct spectral-value prediction on all seven datasets. Trained solely on ARAD with only 0.538M parameters, OmniHSR outperforms all directly transferred baselines on six unseen datasets without target-domain training data or adaptation. Across twelve upsampling factors from $\times2$ to $\times48$, it improves average PSNR on Pavia U and Chikusei by 0.55 dB over the strongest baseline. It also surpasses baselines trained from scratch or adapted on the target sensor and achieves up to $36\times$ faster inference. Our code will be publicly released soon.

0.20

You Only Encode Once: Sub-Pixel-Shifted Frames for Arbitrary-Scale Remote Sensing Super-Resolution with 2D Gaussian Splatting

cs.CV ·2026-11-17 ·Canjie Liu, Liuqian Wang, Qi Ming, Wei Lu, Xiaohan Hao +3 more
similarity 0.58
Abstract

Arbitrary-scale super-resolution (ASSR) of remote sensing images remains challenging due to the limited high-frequency information in a single low-resolution (LR) observation. While multi-frame SR can exploit complementary sub-pixel information, existing approaches often require real multi-frame training sequences and repeated feature encoding, increasing data acquisition demands and computational costs. To address these limitations, we propose SPS-GS (Sub-Pixel-Shifted Gaussian Splatting), an encode-once framework that learns from multi-phase LR observations synthesized from individual high-resolution images, without requiring real multi-frame training sequences. Specifically, Phase-Coherent Aliasing Demodulation (PCAD) exploits shift-dependent aliasing through Fourier-based alignment, phase-aware residual fusion, and least-squares aliasing estimation, enabling multi-frame information to be integrated before a single deep encoding pass. Gaussian Projection Alignment (GPA) then iteratively refines Gaussian centers and colors using multi-observation back-projection to promote observation consistency. The refined Gaussians support continuous rendering at arbitrary scales. Extensive experiments on five remote sensing datasets demonstrate strong reconstruction performance against single-image and multi-frame SR baselines, while maintaining competitive inference efficiency at common scales. The code will be publicly released.

0.36