CS Paper Radar

Camera Radar LiDAR Fusion via Point-Cloud Projection for Multi-Task Learning

Kavin Chandrasekaran, Sorin Grigorescu

cs.CV cs.AIcs.LG ·submitted 2026-11-02 20:42 UTC · v1
hot direction

Abstract

Autonomous vehicles require robust perception systems to operate safely under diverse environmental conditions. While cameras provide high-resolution visual data, their performance significantly degrades in low-light and adverse weather. High Definition (HD) radar offers superior reliability and angular resolution under such conditions but lacks rich visual feature recognition. LiDAR, though expensive, delivers highly precise distance measurements and dense point clouds. Each sensor exhibits unique trade-offs, necessitating the development of advanced sensor fusion algorithms to harness complementary strengths while minimizing computational complexity and maintaining accuracy. In this study, the proposed fusion architecture performs early fusion by combining camera images with perspective-view point cloud data from HD radar and LiDAR sensors. The proposed model is trained to detect vehicles and segment free drivable space simultaneously. Experiments are performed on the RADIal dataset where we demonstrate that fusing point cloud data with camera images in front-view surpasses the performances of the most recent standalone and fusion models.

Comment
IEEE International Workshop on Signal Processing Systems 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Adversarial Robustness of Camera-Radar Sensor Fusion for Autonomous Driving

cs.CR cs.CV ·2026-10-12 ·Ce Zhou, Yubo Wu, Khang Nguyen, Liyang Xiao, Thomas Hou +1 more
similarity 0.75
Abstract · 25 pages

Autonomous driving (AD) relies on multi-sensor perception to interpret the surrounding environment. However, the robustness of camera--radar fusion under adversarial conditions remains largely unexplored. This paper presents a systematic measurement study of the robustness of camera--radar sensor fusion in AD. We construct a unified evaluation pipeline on the nuScenes dataset that simulates attacks on both sensing modalities, including eight camera-image perturbations and three radar point-cloud perturbations. We evaluate five camera--radar fusion algorithms across two perception tasks, 3D object detection and depth estimation, and quantify their performance under clean and attacked inputs using standard metrics. Our evaluation reveals a clear asymmetry in the robustness of fusion systems. Camera perturbations substantially degrade both object detection and depth estimation, and moderate attack strengths are already sufficient to cause severe performance degradation. In contrast, the evaluated radar perturbations produce smaller and task-dependent effects, with object-detection models showing particularly limited degradation. Our analysis attributes this difference to the strong dependence of existing fusion architectures on dense camera features, together with the filtering, association, and confidence-weighting mechanisms commonly applied to sparse radar measurements. These results demonstrate that sensor fusion does not inherently guarantee robustness and that the contribution and reliability of each sensing modality must be explicitly considered under adversarial conditions. Based on four empirical findings, we derive measurement-guided design implications for future robust camera--radar fusion, including modality-specific reliability estimation, cross-modal consistency checking, adaptive fusion, and graceful failover under severe sensor degradation.

0.15

RLG-TPV: Radar- and LiDAR-Guided Tri-Perspective View Fusion for Camera-Radar 3D Object Detection

cs.CV ·2026-08-29 ·Ahmet Mete Dokgoz, A. Enes Doruk, Hasan F. Ates
similarity 0.74
Abstract

Tri-Perspective View (TPV) representations describe 3D scene structure through top, side, and front feature planes, but existing TPV lifting is primarily camera-based, leaving the depth of sampled image evidence ambiguous along projected camera rays. We propose RLG-TPV, a multimodal TPV framework for camera-radar 3D object detection in which radar and training-time LiDAR provide complementary geometric guidance during representation construction. A ray-guided deformable-attention lift weights sampled image features using LiDAR-supervised camera depth probabilities and radar frustum occupancy, while radar additionally refines the depth distribution before lifting. Because conventional radar provides limited elevation information, LiDAR-derived class-occupancy targets supervise the side and front planes during training; the corresponding heads are removed at inference, so deployment requires only cameras and radar. For temporal aggregation, Doppler-guided temporal fusion aligns past features using a motion field anchored by measured radar radial velocity, with gating that limits warping in regions without supported motion. An RCS-aware radar scatter further allows radar evidence to spread over spatial neighborhoods conditioned on radar cross section. On the nuScenes validation set, RLG-TPV achieves 0.4981 mAP and 0.5959 NDS, reducing orientation and velocity error by 31.9\% and 30.7\% relative to the published CRN baseline. Ablation studies show that ray-level geometric guidance is a major contributor to the final performance.

0.09

CLOCs-R: Radar-Assisted Camera-LiDAR Fusion for Long-Range Object Detection

cs.CV ·2026-11-03 ·Richard Beuck, John Nguyen, Sudip Dhakal
hot direction similarity 0.73
Abstract

Long-range 3D object detection remains difficult for autonomous vehicles because LiDAR returns become sparse and camera evidence weakens with distance, while automotive radar keeps returning measurements far beyond both. We present CLOCs-R, an extension of the CLOCs candidate-level fusion framework that adds radar evidence to camera-LiDAR rescoring. Radar returns are accumulated over several sweeps, compensated for ego-motion, and associated with each pre-NMS 3D candidate through a range-dependent gate. Compact features summarizing support, Doppler speed and spread, range residual, and offset are then appended to every camera-LiDAR pair before a lightweight fusion network rescores the candidates. We first verify that our fusion implementation reproduces CLOCs on KITTI. We then evaluate on the daytime clear-weather subset of the aiMotive dataset, reporting distance-binned metrics out to 200 m with frozen SECOND and Cascade R-CNN detectors. Camera-LiDAR CLOCs improves on SECOND up to 50 m but degrades beyond it. A residual scoring variant, introduced post hoc, corrects SECOND's score instead of replacing it and recovers most of this loss. Radar consistently improves BEV AP at 30-50 m (0.857 to 0.870), but beyond 50 m its gain is small (+0.003 center-distance AP), within seed variation, and CLOCs-R with residual scoring is statistically tied with LiDAR-only SECOND. Our analysis attributes the limited long-range benefit to weak separation of true and false detections by radar features, scarce far-range training positives, poor camera evidence at distance, and missing LiDAR proposals that rescoring cannot recover. We report all as-designed and post hoc results side by side.

0.22

Refining Roadside 4D Radar Vehicle Classification Using Point-Cloud Dispersion

cs.CV ·2026-11-01 ·Mohammed Alsakabi, Andreia Figueiredo, António Almeida, John Dolan, Miguel Luís +3 more
similarity 0.70
Abstract

Roadside sensing infrastructure is essential for Intelligent Transportation Systems (ITS), but cameras and legacy sensors are limited in range, weather robustness, and depth accuracy. 4D radar complements these modalities by resolving range, azimuth, elevation, and radial velocity under adverse conditions, though its sparse, noisy point clouds make reliable classification difficult. We present a roadside multi-sensor platform combining a 4D radar and stereo depth camera under synchronized acquisition, used to collect a real-world dataset. Our analysis reveals a substantial class-distribution discrepancy between modalities, most notably a strong radar confusion between passenger cars and motorcycles (49.01% and\ 48.60% of the dataset). To address this, we propose a Neyman-Pearson-based binary classifier on the temporally-smoothed 2D positional standard deviation of radar point cloud clusters, reducing classification error from 52.1% to 42.7%. This lightweight and edge-deployable approach improves radar-only classification reliability and lays the foundation for future camera-radar fusion.

0.20

RAHF-Net: Range-Aware Heterogeneous Fusion for LiDAR-Assisted RGB Semantic Segmentation

cs.CV ·2026-10-18 ·R. D. Lin, Pengcheng Weng, Ge Wang, Han Ding, Fei Wang
similarity 0.70
Abstract

This paper studies LiDAR-assisted RGB semantic segmentation, where LiDAR point-cloud features improve dense pixel-wise predictions in the RGB image plane rather than 3D point labels or detections. Existing 2D fusion methods often project point clouds into shallow depth/range maps and apply spatially uniform fusion, discarding rich 3D structure and ignoring the range-dependent reliability of outdoor LiDAR. We propose RAHF-Net, a Range-Aware Heterogeneous Fusion Network. Its Cylindrical-to-Perspective (C2P) module projects pretrained cylindrical voxel features into the image plane, preserving structured 3D semantic priors. A Range-Aware Heterogeneous Fusion (RAHF) module then uses a soft depth-derived range mask to perform LiDAR-guided geometric enhancement in reliable near-field regions and visual-context enhancement in sparse far-field regions. On KITTI-360, RAHF-Net achieves 65.76 mIoU, outperforming representative RGB-only and RGB-LiDAR baselines, including CMX by 1.45 points. Experiments on DELIVER further demonstrate improved robustness under adverse weather, illumination degradation, and sensor corruption.

0.19

Leveraging Vision-Based Point Cloud Map Priors for Camera-Based 3D Object Detection and Online Vectorized HD Mapping

cs.CV cs.RO ·2026-09-22 ·Markus Käppeler, Rohit Mohan, Abhinav Valada
published similarity 0.69
Abstract · IROS 2026 Workshop on Long-Term Perception for Human-Centric Autonomy

Camera-based 3D object detection and online vectorized HD mapping provide compact scene representations for autonomous driving, but both depend on accurate metric geometry and remain limited by depth ambiguity. Over long-term deployment, observations from repeated traversals can be accumulated into persistent point cloud priors that provide geometric context beyond the current observations. Existing explicit point cloud prior approaches, however, rely on LiDAR-based map construction and therefore require expensive 3D ranging sensors. We propose a framework that constructs a static point cloud prior map from previous camera traversals using Pi3X and augments each point with DINOv3 features. At runtime, a local prior patch is retrieved using global localization, encoded with a sparse voxel backbone, and fused in bird's-eye view (BEV) with lifted multi-view camera features. Task-specific sparse transformer heads then predict 3D objects and vectorized map elements from the fused representation. On Argoverse 2, the vision-based prior improves a strong baseline from 0.287 to 0.299 CDS and from 0.669 to 0.750 vectorized mapping mAP. Ablations show that semantic DINOv3 features are particularly important for vectorized mapping. These results demonstrate that vision-built geometric-semantic priors provide an effective form of long-term scene memory for camera-based perception, improving both tasks without LiDAR for prior-map construction or online inference.

0.25

RaSFNet: Ray-aligned and Scan-aware Fusion Network for LiDAR Semantic Segmentation

cs.CV cs.LG ·2026-10-22 ·Chuang Chen
similarity 0.69
Abstract

LiDAR point cloud semantic segmentation is a fundamental component of 3D perception for autonomous driving and robotics. Projection-based methods map 3D point clouds into 2D representations for feature extraction using 2D networks. However, aggregating multiple points during projection can obscure geometric differences within each frustum, while local feature extraction limits the modeling of spatial relationships across scanning regions. Moreover, proximity in the projected representation does not necessarily imply proximity in 3D space, and interactions between point and projected features may mix features from different ranges, reducing semantic discriminability. To address these challenges, we propose the Ray-aligned and Scan-aware Fusion Network (RaSFNet), which incorporates point cloud geometry and LiDAR scanning characteristics into projection-based semantic segmentation. Specifically, the Ray-Aligned Geometric Distribution Encoding (RAGDE) module characterizes point distributions within each frustum using a ray-aligned coordinate system. To model spatial context, a Scan-Topology-Aware Mamba (STAM) block organizes frustum features according to the LiDAR azimuth sampling pattern and captures long-range dependencies across scanning regions. In addition, a Range-Constrained Point-Frustum Fusion (RCPFF) mechanism uses shared range information to guide feature exchange between point and frustum representations while maintaining spatial correspondence. RaSFNet achieves mIoU scores of 70.2% and 77.7% on the SemanticKITTI and nuScenes validation sets, respectively, indicating competitive segmentation performance compared with the evaluated methods.

0.12

LiNO: Lightweight Network for Occupancy Estimation

cs.CV ·2026-11-10 ·Ruan Bispo, Tim Brophy, Reenu Mohandas, Anthony Scanlan, Ciaran Eising
similarity 0.68
Abstract · 13 pages / 6 figures / 10 tables

Occupancy estimation for automated vehicles is a crucial task that builds a 3D map using vehicle sensors. This map allows the vehicle to identify drivable areas and various obstacle classes. The task involves complex subtasks such as depth estimation, object classification, and segmentation to align semantic information with the 3D grid. Moreover, producing these estimations from multiple sensors demands significant computational power and memory, as high-dimensional data fusion requires complex feature extraction and processing. Consequently, most existing methods rely on large and computationally intensive architectures that are not suitable for real-time or real-world applications. We present a real-time, lightweight architecture that leverages camera and radar sensors through a residual multi-stage head. Unlike prior methods that rely on heavy camera-only blocks, our architecture employs camera-radar fusion and lightweight components. Additionally, we introduce a task-specific data augmentation strategy that enhances data diversity and improves generalisation. Our method achieves state-of-the-art (SOTA) performance, attaining 45.7 mean Intersection over Union (mIoU) on Occ3D and 41.9 Ray-level Intersection over Union (RayIoU) (41.9) on OpenOcc datasets, running at 17.2 frames per second (FPS) and using only 5.3 GB of memory, compared with 4.1 FPS and 7.7 GB for STCOcc.

0.24