CS Paper Radar

Efficient Multimodal Beam Prediction for Real-World 6G UAV Communications via Image-Position-Synthetic Fusion

Rifat Bin Reza, Md Ahadul Haque Suvo, Jia Uddin, Saifur Rahman Sabuj

eess.SP eess.IV ·submitted 2026-10-10 05:28 UTC · v1

Abstract

Camera-based beam selection can reduce the need for exhaustive channel measurement in highly mobile millimeter-wave UAV links, but the complementary value of visual and position-derived representations and their behavior under modality degradation require further investigation. This paper develops a staged multimodal framework for current-beam prediction using real-world UAV measurements. The proposed 32-beam classifier combines an ImageNet-pretrained EfficientNet-B3 visual encoder with two deterministic position-derived representations: an explicit Geometric-Kinematic vector and a nonlinear synthetic feature basis constructed from position. The image and synthetic representations are fused first, followed by late fusion with the geometry representation. Experiments on DeepSense 6G Scenario 23 show that Image-Synthetic fusion achieves 86.92% Top-1, 99.03% Top-3, and 99.56% Top-5 accuracy, improving Top-1 performance by 2.64 percentage points over image-only inference and outperforming the 60.71% Top-1 accuracy of synthetic-only inference. Adding the Geometric-Kinematic branch yields the highest measured performance, reaching 87.14% Top-1, 99.12% Top-3, and 99.60% Top-5 accuracy. Controlled input interventions further demonstrate complementary behavior under visual degradation and strong sensitivity to severe synthetic-input corruption. Position-derived nonlinear features provide complementary information to visual context and improve current-beam prediction in the evaluated real-world UAV setting.

Comment
19 pages, 7 figures, 5 tables, 2 algorithms

Closest papers

Nearest neighbours by meaning across the whole index.

CM-MAE: A Physics-Guided Cross-Modal Self-Supervised Learning Framework for Vision-Wireless Applications

cs.CV cs.AI ·2026-08-16 ·Yubo Zhang, Yiyao Liu
similarity 0.63
Abstract

Synchronized camera and wireless measurements observe the same scene through different physical channels. The central difficulty is that a representation learned in one deployment can fail when viewpoint, traffic, illumination, and propagation geometry change. This paper presents CM-MAE, a self-supervised vision--wireless pretraining framework for cross-scenario representation transfer. The evaluated real-data model uses only RGB frames and the measured 64-beam received-power vector available in DeepSense 6G; it does not use ray-traced paths, calibrated depth, or beam-index labels during pretraining. Its central pretraining term is a \emph{soft contrastive alignment loss}. Instead of making the synchronized image--wireless pair the only positive pair, this loss builds a target distribution from similarities between measured beam-power profiles, so nonidentical samples with similar directional responses are not forced apart as false negatives. A masked joint decoder provides the complementary local objective by reconstructing hidden visual patches and wireless angular clusters under modality dropout. After pretraining, a differential-rate fine-tuning rule lets a new fusion head adapt quickly while the encoders move slowly. Under a sequence-disjoint DeepSense 6G protocol, adding the soft alignment loss improves a matched linear-probe transfer average from 24.88\% to 29.49\%. Mild fusion fine-tuning reaches 77.38\% Top-1 accuracy on unseen Scenarios 6--8, and optional transductive normalization adaptation reaches 78.69\%. Since the fusion setting uses the contemporaneous 64-beam power vector at inference, these results should be read as representation-transfer diagnostics, not as proactive beam-prediction or reduced-sweeping claims.

0.04

Synthetic-to-Real ViT-Based Pose Estimation of a Noncooperative UAV

cs.CV ·2026-10-03 ·Krishnanujam Srinivas, Hanish Acharla, Brij Agrawal, Leonardo Herrera
similarity 0.61
Abstract · 12 pages, 8 figures, 5 tables

Remote pose estimation of noncooperative Unmanned Aerial Vehicles (UAVs) from imagery is critical, as they cannot be influenced or instrumented in advance. Deep-learning-based approaches offer a promising solution; however, their development is constrained by the cost and difficulty of acquiring large-scale real-world datasets with accurate pose labels. Synthetic imagery provides an alternative, but models trained on synthetic data must overcome the synthetic-to-real domain gap to generalize to real-world imagery. This work investigates the inherent synthetic-to-real generalization capability of a Vision Transformer (ViT)-based model for monocular UAV pose estimation. The proposed approach employs a self-supervised DINOv2 backbone and is trained exclusively on labeled synthetic imagery while being evaluated on labeled real-world imagery. Pose ambiguity-aware strategies are incorporated during training and inference to address ambiguities arising from the projection of a three-dimensional target onto a two-dimensional image plane and from target symmetries. An $α$-$β$ filter is further integrated during inference to improve pose estimations. To assess the model under operational requirements, it is evaluated in terms of Mean Angular Error (MAE) and inference time, both before and after filtering, using a real-world dataset containing 77,077 labeled UAV images. Before filtering, the model achieves an MAE of $19.18^{\circ}$ and an inference time of $13.25$ ms, whereas after filtering, these values are $8.74^{\circ}$ and $13.42$ ms, respectively.

0.21

YOLO12-MambaScan: An Efficient Object Detector with High-Frequency Enhancement and State-Space Modeling

cs.CV cs.IR ·2026-09-12 ·Hao Wang
benchmark similarity 0.60
Abstract

The rapid development of unmanned aerial vehicle (UAV) technology has made aerial-image object detection increasingly important for natural-resource monitoring, traffic management, and disaster response. Detecting small objects in aerial images remains difficult because objects occupy very few pixels, high-frequency cues are easily lost, and global context is hard to model in cluttered scenes. Existing detectors often retain insufficient edge, corner, and texture information. We propose \ours, an aerial-image detector built on the YOLO12 architecture. The model combines a triple-path high-frequency enhancement convolution module (TriPathHFConv), receptive-field coordinate-attention convolution (RFCAConv), and a Mamba-based global-context module. On VisDrone, at an input resolution of 960*960, ours achieves 60.0% mAP@50 and 38.6%mAP@50:95, demonstrating a favorable accuracy--efficiency trade-off for small-object detection. The benchmark and dataset protocol follow the VisDrone challenge setup.

0.17

GAAT: Geometry-Aware Alignment Transformer for Multimodal UAV Perception

cs.CV ·2026-08-28 ·Jingpu Yang, Debin Tang, Yilin Sun, Fengxian Ji, Jiahua Zhu +2 more
similarity 0.60
Abstract

Unmanned aerial vehicle (UAV) multimodal perception integrates visible (RGB), infrared (IR), synthetic aperture radar (SAR), and depth sensors for scene understanding under diverse conditions. However, differences in optics, resolution, and mounting often limit practical systems to global or image-center alignment. After tokenization, parallax, platform motion, and lens distortion can shift corresponding patch centers across modalities, weakening the spatial correspondence assumed by dense contrastive learning and cross-modal fusion. We propose GAAT (Geometry-Aware Alignment Transformer), an alignment-first pretrained model that estimates local correspondence reliability before cross-modal interaction. GAAT introduces syncPATC, which learns patch-center consistency under synchronized view transformations without correspondence annotations. It emits geometric priors, including token and query confidence, query centers, and sub-token offsets, that identify reliable local anchors across residual misalignment. Guided by these priors, MG-Sparse-MMA performs query-mediated sparse fusion over top-K_s reliable regions, replacing dense all-patch interaction with geometry-calibrated local updates. RA-QCGCL aligns pretraining supervision with this sparse query bottleneck through reliable patch-to-patch, patch-to-query, and query-to-query contrastive branches. We introduce UAVMeta and StateBench, which provide four acquisition-state scores derived from platform telemetry and image statistics: camera reliability, observation scale, viewpoint stability, and flight maneuver complexity. Extensive experiments across six downstream tasks demonstrate consistently superior transfer performance, establishing GAAT as a state-of-the-art multimodal foundation model for UAV perception. StateBench further enables a systematic diagnosis of real-world acquisition conditions.

0.25

Cross-View Vision-Aided Proactive BS Selection and Beam Prediction for mmWave V2I Communications

eess.SP ·2026-09-01 ·Zijiao Hu, Haiyao Yu, Gaoyang Pang, Guangchen Wang, Litianyi Zhang +4 more
code/data similarity 0.59
Abstract

This paper investigates environmental-sensing-aided proactive base station (BS) selection and beam prediction for millimeter-wave (mmWave) vehicle-to-infrastructure (V2I) wireless systems. We exploit onboard panoramic street-view images and a preloaded satellite map to predict communication-relevant environmental information around the vehicle, including nearby building footprints and heights. The predicted height map provides a compact environmental prior and is combined with historical mobility information to jointly predict the next-slot line-of-sight (LoS) state, transmission rate, and transmit and receive beam selections. On our dataset covering different real-world regions across New South Wales, Australia, the proposed framework achieves 91.4% LoS classification accuracy, 0.638 bps/Hz mean absolute error of data rate prediction, and more than 40% higher transmission rate than the conventional reactive baseline in geographically unseen regions, outperforming all evaluated deployable learning-based baselines. The dataset and code will be released at https://github.com/Huzijiao/Cross-view_V2I

0.25

EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection

cs.CV ·2026-08-12 ·Cunzheng Fan, Dawei Yan, Guanlin Wang, Xingshuo Yang, Yupeng Jia +2 more
similarity 0.59
Abstract · 14 pages, 7 figures, 6 tables

Joint use of RGB and infrared (IR) imagery can improve UAV-view object detection, but most existing methods fuse multimodal features with static or fixed weights and therefore overlook spatially varying modality reliability. We propose EGM-Det, an entropy-guided multimodal adaptive fusion framework for RGB-IR object detection. EGM-Det employs a dual-stream architecture to preserve modality-specific representations and introduces an Entropy Offset Gate Fusion module for adaptive multi-scale fusion. The module derives shallow entropy priors from input intensity, local entropy, and cross-modal discrepancy, and uses them to guide local offset alignment and spatial-channel gated fusion. It therefore selectively aggregates reliable RGB and infrared cues instead of uniformly combining heterogeneous features. We further introduce cross-modal distillation to regularize the learned fusion gates and reduce fusion degradation. Each student branch extracts complementary knowledge from the cross-modality teacher branch matched to the main branch, while entropy-adaptive supervision emphasizes uncertain modality decisions. Experiments on DroneVehicle, LLVIP, and VEDAI demonstrate state-of-the-art performance across all three benchmarks; in particular, EGM-Det outperforms prior approaches by more than 10 percentage points on VEDAI.

0.40

Intelligent Beamforming and Handover via Physics-Informed Beam-Aware CKM Diffusion

eess.SP ·2026-09-28 ·Le Zhao, Zesong Fei, Xinyi Wang, Nan Ha, Yining Wang +2 more
published similarity 0.59
Abstract

Downlink communications from base stations (BSs) to unmanned aerial vehicles (UAVs) in sixth-generation (6G) networks require precise beam alignment to overcome severe mobile communication path loss. However, traditional exhaustive beam sweeping relies on discrete codebooks and consumes valuable air-interface resources for online measurements, rendering it inefficient for highly dynamic aerial environments. In this paper, we propose BeamCKMDiff, a physics-informed generative diffusion framework designed to construct high-fidelity continuous beam-aware channel knowledge maps (BeamCKMs). Unlike existing empirical approaches, BeamCKMDiff employs a diffusion transformer (DiT) architecture featuring a dual-path conditioning mechanism. It integrates an analytical line-of-sight (LoS) beam prior with environmental topologies, while simultaneously embedding continuous beamforming vectors through an adaptive layer normalization (adaLN) mechanism. Building upon this differentiable generative mapping, we propose a CKM-based end-to-end continuous beamforming and proactive dual-BS handover algorithm. By analytically propagating gradients through the reverse diffusion process, the proposed framework enables successive convex approximation (SCA)-based beamforming optimization in the computational domain, effectively bypassing physical pilot scanning. Simulation results demonstrate that BeamCKMDiff achieves a normalized mean square error (NMSE) of --23.96 dB, establishing a highly reliable spatial prior. Compared to discrete beam sweeping and LoS-assumed baselines, the proposed CKM-based beamforming adapts to non-LoS (NLoS) conditions, providing accurate beam alignment and higher spectral efficiency. Furthermore, the integrated dual-BS handover mechanism eliminates blockage-induced link outages, ensuring robust connectivity for high-mobility aerial platforms.

0.31

DroneWAM: Efficient World Action Model for Drone Visual Navigation

cs.CV ·2026-09-27 ·Liang Yao, Fan Liu, Hongbo Lu, Wei Xu, Jianyu Jiang +3 more
code/data similarity 0.58
Abstract

World-action models give visual navigation agents a way to anticipate how candidate actions will change future observations and to act from the predicted consequences. For drones, this capability must operate under tight accuracy and efficiency constraints. We present DroneWAM, an efficient world-action model for drone visual navigation. DroneWAM adopts a JEPA-based architecture to model future states directly in representation space, avoiding the cost of explicit future image generation. A pretrained Resampler further compresses dense encoder features into fewer latent tokens, reducing the computation repeated at each imagined step. We also introduce adaptive rollout, where a preference-trained Gate adaptively allocates prediction depth according to the current scene. To support learning under richer aerial motion, we construct DroneNav-6D, a simulated visual navigation dataset with synchronized RGB observations, 6-DoF flight trajectories, control commands, and randomized wind disturbances. On DroneNav-6D, DroneWAM achieves the best trajectory accuracy among the compared methods. Adaptive rollout further reduces the average prediction depth from 8 to 4.58 while improving trajectory accuracy, demonstrating that predictive computation can be allocated more effectively across scenes. \href{https://github.com/1e12Leon/DroneWAM}{Codes and data} will be released.

0.51