CS Paper Radar

A Multi-Modal Perception Pipeline for Object Detection and Tracking in Autonomous Racing

Davide Malvezzi, Michele Pestarino, Vittoria Cavicchioli, Valentina La Gamba, Silvia Severi, Fabio Bagni, Luca Bartoli, Massimiliano Bosi, Francesco Gatti, Micaela Verucchi, Ayoub Raji, Marko Bertogna

cs.RO cs.AI ·submitted 2026-09-08 07:11 UTC · v2
hot direction

Abstract

Object detection and tracking are fundamental components of perception systems for autonomous driving. Achieving robust performance under adverse conditions such as limited visibility, sensor noise, and failures remains an open challenge, particularly in autonomous racing, where vehicles operate at very high speeds, experience strong vibrations, and interact under small safety margins. This paper presents a multi-modal late-fusion perception pipeline for object detection and tracking in the autonomous racing domain. The proposed system extends previous work by exploiting all onboard sensors through a late-fusion approach and a dedicated multi-object tracking framework. Independent detections from cameras, LiDARs, and RADARs are combined to provide timely and robust state estimates of surrounding vehicles. The tracking method explicitly compensates for detection delays and embeds in its model prior knowledge of vehicle dynamics and track layout. Experimental evaluation on real-world data across diverse critical scenarios, representative of challenging edge cases also in urban driving, confirms the effectiveness of the proposed pipeline and its suitability to support safe and adaptive planning decisions.

Comment
8 pages, 6 figures, ITSC 2026, Invited Session

Closest papers

Nearest neighbours by meaning across the whole index.

Multi-Modal Traffic Sign Detection with Semantic Attributes for Autonomous Driving

cs.CV cs.RO ·2026-08-21 ·Meda Lazar, Sourab Sridhar, Shashwata Gupta, Alexandra Tripcea, Varun Ravi +1 more
similarity 0.66
Abstract

Reliable traffic sign detection is a prerequisite for the global deployment of autonomous driving systems, where regulatory compliance and road safety depend on perceiving signs correctly across regions, ranges, and weather conditions. Despite recent progress, vision-based methods continue to face three fundamental limitations: poor cross-regional generalization due to high diversity across countries, degraded performance on small-object detection at long ranges (traffic signs occupy as little as $10{\times}10$ pixels at 200m), and fragile temporal tracking under the strongly non-linear perspective distortion that occurs as a vehicle approaches a sign. In this paper, we address the problem of robust, long-range, region-agnostic traffic sign perception by combining camera and Light Detection and Ranging (LiDAR) sensing. We present a multi-modal detection framework whose Intensity-Aware Deformable Fusion module aligns retro-reflective LiDAR cues with camera features, anchoring detection on geometric invariants rather than region-specific visual appearance. We further introduce a dual motion-model tracker that explicitly accounts for non-linear perspective transformations during vehicle approach, substantially improving temporal consistency over linear motion assumptions. Additionally, we develop a semantic attribute classification pipeline that estimates occlusion level, readability, sign embeddedness, and road relevance, providing actionable context to downstream planning. Extensive evaluation on our dataset, spanning 60+ countries and 2,500+ hours of driving data, shows that the proposed pipeline achieves an Object Miss Ratio (OMR) of 0.49% across 221,068 evaluation sequences, demonstrating globally generalizable traffic sign perception in commercial-grade autonomous driving systems.

0.23

Multi-Observer Vehicle Localization Case Study with Roadside Radar and Connected Vehicle Sensing

cs.CV cs.RO ·2026-08-17 ·Aleksi Pippuri, Nilusha Jayawickrama, Risto Ojala
datasetcode/data similarity 0.66
Abstract · 12 pages, 7 figures and 8 tables

In modern intelligent transportation systems, it is essential to accurately estimate vehicle positions, especially in mixed traffic conditions where both connected and conventional vehicles coexist. Roadside infrastructure and connected vehicles can provide complementary observations of the same traffic scene, but real-world evidence on decision-level fusion between these sources remains limited. This paper proposes a multi-observer vehicle localization framework that fuses compact object-level detections from a static roadside radar and a dynamic LiDAR-equipped connected vehicle. We evaluate the framework with real-world data collected at an urban intersection in Helsinki, Finland, with a separately instrumented target vehicle used as the reference trajectory. Two extended Kalman filter based strategies for the localization task were benchmarked. The performance of the radar and LiDAR sensors were evaluated separately, and the two fusion strategies were explored under nominal sensing conditions, reduced LiDAR update rates, simulated LiDAR occlusions, and different target-vehicle motion states. The results show that, under full LiDAR availability, fusion performance is dominated by the LiDAR observations, while the less accurate and less consistent radar observations provide only limited additional improvement. Nevertheless, AEKF achieves small gains over the LiDAR-only baseline, and object-level connected vehicle observations remain useful when shared at reduced update rates. These findings indicate that decision-level fusion provides scenario-dependent benefits rather than automatic improvement over a strong single-sensor baseline. We release the dataset and implementation on Github to support further research: https://github.com/AppuriAalto/multi-observer-vehicle-tracking

0.45

CRUISE: Vision-Language Model-Guided Uncertainty-Aware Cross-Modal Sensor Fusion for Robust Autonomous Driving

cs.AI ·2026-08-10 ·Junyao Wang, Yulin Xu, Yu Li, Pramod Khargonekar, Mohammad Abdullah Al Faruque
similarity 0.65
Abstract

Modern autonomous vehicles are equipped with multiple sensors, such as cameras, LiDAR, and radar, for comprehensive environmental perception. However, robust cross-modal feature fusion remains a critical challenge, as the reliability of each sensor varies significantly across diverse real-world driving conditions, including poor visibility and adverse weather. While uncertainty quantification (UQ) mitigates this issue by allowing models to prioritize reliable signals, existing uncertainty-aware fusion methods typically rely on simple feature-level uncertainty estimates and thus often fail to generalize effectively in complex, out-of-distribution scenarios. To address this limitation, we propose CRUISE, a novel uncertainty-aware cross-modal sensor fusion framework. CRUISE integrates a vision-language model (VLM)-guided UQ module that generates fine-grained, pixel-level uncertainty estimates. By leveraging the VLM's rich prior knowledge and superior contextual reasoning, our approach provides a highly informative guide for the fusion process. Furthermore, we introduce a dynamic adaptive mechanism that explicitly models and captures cross-modal dependencies, ensuring the framework fully exploits the inherent complementary nature of multi-sensor inputs.

0.34

Spatio-Temporal Calibration of a Vehicle-Mounted Microphone Array for Autonomous Driving

cs.SD eess.AS ·2026-11-10 ·Marvin Seegert, Richard Taupitz, Korbinian Moller, Johannes Betz
similarity 0.64
Abstract · Submitted to IEEE ICRA 2027

Microphones let an autonomous vehicle sense into areas its cameras and LiDAR cannot see, by picking up the sound that other traffic participants emit. Locating those sources and fusing them with the visual perception requires the microphone array to be calibrated and registered relative to the vehicle's other sensors. Existing calibration methods assume small, unobstructed indoor rigs, whereas a vehicle body casts an acoustic shadow that straight-line propagation cannot represent, and to our knowledge no calibration of an array distributed around a vehicle has been reported. We propose a pipeline that uses a balloon burst as a single cross-modal target, which the LiDAR locates in metric coordinates, every microphone hears as a sharp onset and a camera times. Each observation is weighted by its own timing variance, which discounts shadowed arrivals instead of discarding them and needs no model of the vehicle. On a full-size vehicle with eight microphones over a 4.9 m aperture, the held-out range residual of 6.37 cm approaches the 6.07 cm floor set by the onset timing noise, and the calibrated array tracks a moving source to within a few degrees of the vehicle's own LiDAR. A dozen bursts spread around the vehicle suffice, and spreading them matters more than adding more.

0.29

FastBEVFusion: Simple Multi-Modal Fusion for 3D Object Detection at 50 FPS

cs.CV cs.RO ·2026-10-27 ·Tom Sanitz, Emre Gülsoylu, Simone Frintrop, Christian Wilms
published similarity 0.64
Abstract · Accepted at Iberian Robotics Conference (ROBOT2026)

Reliable real-time 3D object detection based on LiDAR and camera data is critical for safety in autonomous driving, where systems must accurately perceive their surroundings with minimal latency. Many state-of-the-art systems achieve high-quality detection results but fail to meet real-time constraints, limiting their practical use. To address this challenge, we propose FastBEVFusion, a simple, fast, and robust fusion system for 3D object detection. Based on modality-specific features, FastBEVFusion projects these features into semi-aligned modality-specific BEV spaces, without relying on a computationally intensive and potentially error-prone depth estimation. Given these BEV spaces, we propose a light-weight, soft, and dynamic fusion module based on cross-attention, which adapts dynamically to the scene context and enables feature fusion across the entire BEV spaces. Experiments on the complex nuScenes dataset show that FastBEVFusion outperforms state-of-the-art real-time systems, and achieves over 50 FPS without deployment libraries. Additionally, our system shows strong robustness to calibration errors, making FastBEVFusion a powerful 3D object detection system for real-world applications.

0.28

One-Stage Object Detectors in Autonomous Driving

cs.CV cs.AI ·2026-08-19 ·Jonel Roman, Ryan Sirjue, Peter Nguyen, Daniel Krutky, Juan Jesus +1 more
surveybenchmark similarity 0.63
Abstract

Autonomous vehicles depend on fast and reliable perception systems to detect surrounding vehicles, pedestrians, cyclists, traffic signs, and other road objects in real time. This paper presents a comprehensive survey and analysis of one-stage object detectors for autonomous driving rather than an implementation of a new detection system. The survey reviews the evolution of major one-stage detectors, including YOLOv1, SSD, RetinaNet, EfficientDet, anchor-free detectors such as FCOS and CenterNet, and recent real-time models such as YOLOv10. It compares these architectures through their design choices, feature-fusion strategies, loss functions, deployment trade-offs, and reported benchmark performance. The paper also summarizes commonly used autonomous-driving datasets, evaluation metrics, open challenges, and future research directions. Overall, this survey highlights how one-stage detectors balance speed, accuracy, efficiency, and robustness, while also emphasizing the remaining gap between benchmark results and dependable real-world autonomous-driving performance.

0.26

LiNO: Lightweight Network for Occupancy Estimation

cs.CV ·2026-11-10 ·Ruan Bispo, Tim Brophy, Reenu Mohandas, Anthony Scanlan, Ciaran Eising
similarity 0.63
Abstract · 13 pages / 6 figures / 10 tables

Occupancy estimation for automated vehicles is a crucial task that builds a 3D map using vehicle sensors. This map allows the vehicle to identify drivable areas and various obstacle classes. The task involves complex subtasks such as depth estimation, object classification, and segmentation to align semantic information with the 3D grid. Moreover, producing these estimations from multiple sensors demands significant computational power and memory, as high-dimensional data fusion requires complex feature extraction and processing. Consequently, most existing methods rely on large and computationally intensive architectures that are not suitable for real-time or real-world applications. We present a real-time, lightweight architecture that leverages camera and radar sensors through a residual multi-stage head. Unlike prior methods that rely on heavy camera-only blocks, our architecture employs camera-radar fusion and lightweight components. Additionally, we introduce a task-specific data augmentation strategy that enhances data diversity and improves generalisation. Our method achieves state-of-the-art (SOTA) performance, attaining 45.7 mean Intersection over Union (mIoU) on Occ3D and 41.9 Ray-level Intersection over Union (RayIoU) (41.9) on OpenOcc datasets, running at 17.2 frames per second (FPS) and using only 5.3 GB of memory, compared with 4.1 FPS and 7.7 GB for STCOcc.

0.24

TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception

cs.CV cs.AI ·2026-09-25 ·Quinlan Sykora, Sourav Biswas, Christopher Diehl, Andrew Cunningham, Thomas Gilles +1 more
top venue similarity 0.63
Abstract · Published at ECCV 2026, 49 pages, 20 figures

We present TriO, a multi-modal unsupervised world model that predicts 4D occupancy, obstacle segmentation, flow and LiDAR. In contrast to prior work, TriO utilizes three distinct sensor modalities (camera, LiDAR, and RADAR) as both inputs and sources of self-supervision, eliminating the need for additional human annotations. Thanks to its novel supervision, the model is able to segment any occupancy from the drivable surface, overcoming the limitations of existing open-set methods in handling long-tail objects. TriO achieves state-of-the-art results in multiple 3D and 4D tasks, including occupancy, flow, and LiDAR prediction, as well as zero-shot road obstacle segmentation across multiple datasets such as Argoverse 2, and Spotting the Unexpected.

0.40