CS Paper Radar

You Should Be Properly Scoring Your Odometry

Ola Rønning, Usama Saqib, Andrzej Wąsowski

cs.RO ·submitted 2026-09-22 09:03 UTC · v1
code/data

Abstract

When we evaluate the performance of our odometry, it is common practice to score the estimated track against a ground truth. Unfortunately, scoring uses point metrics, such as the root mean square error, that ignore the covariance matrix which estimators like filters and smoothers already report. Using the covariance matters for two reasons. First, the covariance encodes the estimator's uncertainty, so it tells us whether the estimator trusts its own output. An overconfident estimator will not report itself lost. Second, the covariance weights the error in each direction of the estimate. Without the covariance, an estimator is unduly penalized for a high error in an uncertain direction. Instead of point metrics, we should use strictly proper scoring rules. These rules score the estimate together with its reported uncertainty. Strictly proper scoring rules recover the point metrics when no covariance is reported, and they diagnose covariance inconsistency when covariance is reported. Using a one-sided pairwise test, we show that two estimators can expose overconfidence in at least one of them without a ground truth. Strictly proper scoring rules and our pairwise test are available in our open-source framework smfeval. As a case study, we use smfeval to assess the uncertainty quality of the translational component of ground-based LiDAR-inertial odometry. Across four filters we find overconfidence - the worst case reports centimeter certainty with kilometer error. Knowing the filters are overconfident, we investigate the mechanism. The investigation traces overconfidence to filters crediting LiDAR measurements with more new information than they carry.

Closest papers

Nearest neighbours by meaning across the whole index.

Benchmarking Visual-Inertial Odometry in Subterranean Environments Under Sensor Degradation, Miscalibration, and Dynamic Occlusion

cs.RO ·2026-09-16 ·Yueying Zhu, Xiang Li, Thien-Minh Nguyen, Xuehe Wang, Shenghai Yuan
benchmark similarity 0.59
Abstract · 8 pages, 5 figures

Visual-inertial odometry (VIO) is a core capability for autonomous operation in GPS-denied subterranean environments, yet its reliability can degrade sharply under sensor drift, calibration errors, and dynamic occlusion. Existing evaluations mainly emphasize nominal-condition accuracy, offering limited insight into when practical deployment failures occur. In this work, we present a failure-centric stress-test benchmark for VIO in underground environments using the CERBERUS dataset. We systematically evaluate four representative VIO systems spanning filtering-, optimization-, and learning-based paradigms under nine practical perturbation settings, including IMU bias and noise variation, camera intrinsic and extrinsic drift, and dynamic scene occlusion. Beyond conventional trajectory error, we analyze robustness limits through coverage ratio and failure thresholds, revealing breakdown behaviors that are not captured by nominal-condition performance alone. Our study shows distinct vulnerability patterns across VIO paradigms: some methods are more sensitive to inertial degradation, while others are more affected by geometric miscalibration or dynamic interference. These results provide deployment-oriented guidance for VIO selection, calibration prioritization, and reliable operation in challenging underground scenarios. To support reproducible evaluation and future extensions, we will release the full benchmark scripts and evaluation pipeline.

0.15

When Your State Estimator Has Lost The Plot: Detecting Estimator Failures Via Spectral Analysis

cs.RO ·2026-08-11 ·Christian Lanegger, Helen Oleynikova, Roland Siegwart, Michael Pantic
similarity 0.57
Abstract

Reliable onboard state estimation is essential for safe robotic operation, yet unmodeled disturbances, such as sensor aliasing or out-of-distribution noise, still cause estimators to degrade or fail completely. While many methods aim to improve estimator robustness, only a few provide introspective mechanisms to assess estimate quality. Existing uncertainty measures, such as covariances, rely on idealized assumptions and tend to be overconfident, and more recent data-driven approaches are typically tied to their training data distributions. We propose a sensor-agnostic introspective method that assesses estimator health by analyzing the frequency-domain power distribution of recent velocity estimates. The method is evaluated using outdoor flight data from an aerial robot running visual-inertial, LiDAR-inertial, and radar-inertial odometry. The dataset includes multiple estimator failures, enabling analysis of several frequency-domain indicators, such as signal power, spectral bandwidth, and entropy. We observe consistent spectral power differences between healthy and degraded estimates, allowing detection of 51%-58% of labeled failures with 60%-84% precision across three fundamentally different state estimation frameworks. Our results show that even a simple frequency-domain analysis of a state estimator's output can serve as a lightweight introspective tool to complement existing robustness techniques in real-world robotic deployments, and opens promising avenues for future investigation.

0.04

Anchor-Free Hidden-Target Seeking via Certified Self-Calibration under Correlated Odometry

cs.RO eess.SY ·2026-08-26 ·Yash Bagla
similarity 0.57
Abstract · 16 pages, 8 figures

We study hidden-target seeking in an anchor-free regime where neither the vehicle, the relay, nor the target has an accessible global pose. The vehicle never senses the target directly; it receives only range-bearing observations through a single relay of unknown position and orientation, with motion known only via integrated body-frame odometry. Absolute localization is fundamentally impossible: the joint configuration retains an exact three-dimensional SE(2) gauge no estimator can resolve. Yet the quantities needed for control remain fully recoverable: motion collapses calibration to a task-relevant quotient, the relay-to-odometry yaw and target displacement, identifiable in closed form from two distinct vehicle views. We introduce an O(K) multi-view self-calibration estimator with an exact first-order yaw-uncertainty certificate that propagates the cross-view correlations integrated odometry induces: the full certificate attains 95.0% pooled coverage at the nominal 95% level, versus 82.1% when correlated poses are treated as independent. The certificate drives a hybrid policy that excites until calibration is trustworthy, refuses uncertified estimates, seeks using continuously re-measured geometry, and detects relay-frame changes via persistent certified inconsistency, avoiding unbounded dead-reckoning drift. Across 200 randomized closed-loop trials, the method attains 0.064 m median station error versus 0.065 m for an oracle given the true relay yaw, despite 27 m median dead-reckoning drift over long horizons; 90 physics-based ROS 2/Gazebo trials retain 30/30 success under nominal operation, communication degradation, and relay-frame disturbances. These results show global localization is unnecessary for reliable hidden-target seeking under this single-relay model, even when both the sensing infrastructure and the vehicle's own reference frame are uncalibrated.

0.43

Does Online Gravity Estimation Matter? Revisiting a Silent Design Split in LiDAR-Inertial Odometry

cs.RO ·2026-09-12 ·Jie Xu, Ziyi Jin, Kangjin Yu, Can Jiang, Hongjun Huang +3 more
code/data similarity 0.55
Abstract · 8 pages, 5 figures. Code, evidence, and video: https://github.com/jiejie567/rethink-lio-gravity

LiDAR-inertial odometry (LIO) systems differ in whether they continue estimating gravity after initialization. We compare four gravity-bias state configurations in each of FAST-LIO2 and LIO-SAM, then separately test a gravity-direction factor. Across 12 dataset sequences evaluated with FAST-LIO2, fixing gravity under continuous LiDAR correction produces mean paired changes in vertical and 3D position errors with 90% confidence intervals within $\pm 2\%$. Tests on 4 sequences with LIO-SAM likewise show no consistent benefit from online gravity. Multi-second LiDAR outages, unlike reduced range or field of view, reveal trajectory-dependent costs of fixing gravity. A history-matched 23D-to-21D switch places the repeatable 3D error increase after LiDAR updates resume. Under 5-s outages, a direction factor from the same IMU used for preintegration improves accuracy on Hall05 but worsens both errors with online gravity on TUHH. Dynamic-start tests also show fixed-bias failures at particular starting phases. We recommend keeping gravity and accelerometer bias online for robustness; use a direction factor only after verifying vertical and 3D accuracy gains under the intended operating conditions.

0.47

Probe Sets for Nonlinear Kalman Filtering: Multi-Center Gains and Covariance Recalibration

eess.SY eess.SP ·2026-09-13 ·Shida Jiang, Shengyu Tao, Scott Moura
code/data similarity 0.54
Abstract

Kalman filters (KFs) choose the gain as the product of the state-measurement cross-covariance and the inverse of the innovation covariance. In nonlinear systems, the predictive distribution generally changes shape, and these covariances require approximation. Conventional nonlinear KFs approximate them around a single center, the predicted state. They can produce inaccurate gains and overconfident covariance estimates when one local approximation does not represent the measurement geometry across the uncertainty region. To address this issue, we introduce probe sets, small covariance-scaled collections of states at which the filter repeats its local approximation. We then select the gain that minimizes the average covariance reported by these local approximations. The same construction is used in our previously proposed covariance recalibration step to improve covariance consistency. The construction applies to different KF variants without requiring a specific approximation rule. For quadratic measurements, we derive conditions under which probing yields lower normalized true error variance than the corresponding single-center gain when prediction uncertainty is sufficiently understated. We apply the framework to four KF variants and compare the resulting filters with multiple baselines across 300 randomized setups for each of two systems. The results show that probing preserves typical accuracy while substantially reducing covariance inconsistency and the rare large-error trajectories that dominate the root mean square of per-run errors. The code is available at https://github.com/Shida-Jiang/Probe_KF.

0.35

Dual Covariance Gaussian Splatting SLAM: Decoupling Rendering and Registration for Robust Real-Time Tracking

cs.RO cs.CV ·2026-09-22 ·Edward Beng Wai Tan, Siew-Kei Lam
similarity 0.54
Abstract

ICP-based 3D Gaussian Splatting (3DGS) SLAM tracks in real time by registering incoming frames against map Gaussians, using each primitive's covariance for both rendering and registration. These two uses place conflicting demands on one covariance. The mapper shapes it to minimize photometric error, often flattening it against surfaces, while robust registration typically benefits from measurement uncertainty. We propose a dual-covariance parameterization. Each Gaussian keeps a single mean but holds two covariances: a rendering covariance optimized by the mapper, and a tracking covariance derived from an RGB-D sensor noise model. We further use the tracking covariances as Gaussian anchors for image corners, providing constraints in directions where depth geometry is weak. We evaluate on TUM RGB-D, ScanNet, Replica, and two outdoor sequences recorded with a RealSense D435i on wheeled and handheld platforms. We achieve robust tracking performance across multiple scenes and reduced odometry drift, while tracking at $\sim$ 60 FPS.

0.32

Resilient Decentralized Radar-LiDAR-Inertial Odometry in Challenging Subterranean Environments

cs.RO ·2026-10-23 ·Moumita Mukherjee, Avijit Banerjee, Anton Koval, George Nikolakopoulos
similarity 0.53
Abstract

Accurate localization in GPS-denied and visually degraded environments affected by smoke, dust, or fog remains a critical challenge for autonomous systems. While LiDAR provides high-resolution spatial perception, its performance deteriorates under particulate occlusion. In contrast, frequency-modulated continuous-wave (FMCW) radar offers robustness under such conditions but suffers from sparse measurements and outliers. This article presents a resilient decentralized LiDAR Radar Inertial Odometry (LRIO) framework that fuses the complementary strengths of LiDAR, radar, and inertial sensing to achieve robust pose estimation in challenging subterranean environments. At the core of the framework is an Adaptive Residual Thresholding (ART) mechanism that autonomously detects temporal sensor degradation and selectively adjusts the fusion process to minimize unreliable measurements. Building on this, a Robust Error-Tolerant Optimal Information Filter (RETOIF) is employed within a decentralized fusion architecture to integrate LiDAR-inertial and multiple radar-inertial odometry estimates in a fault-aware manner, enhancing resilience to environmental disturbances and intermittent sensor failures. The proposed framework is evaluated using real-world sensor data collected in GPS-denied subterranean tunnels under smoke-free and smoke-degraded conditions. In the smoke-free experiment, the proposed RETOIF framework achieves a root mean square localization error (RMSE) of 2.1733, compared with 2.9929 for the conventional centralized LRIO approach, corresponding to a 27.4\% reduction in RMSE. Experimental results further demonstrate the framework's ability to maintain pose estimation under smoke-induced sensor degradation, highlighting its potential for resilient autonomous navigation and mapping in challenging subterranean environments.

0.05

Do Better Imagined Rollouts Mean Better Robot Control? A Controlled Study of World-Model Evaluation Under Feedback

cs.RO ·2026-09-02 ·Dharini Raghavan, Amritpal Singh
code/datahot direction similarity 0.53
Abstract · 20 pages, 10 figures

Predictive models are increasingly used in robotics for state estimation, planning, control, and policy evaluation, yet they are often judged by open-loop prediction accuracy over a fixed horizon. In closed-loop operation, a robot repeatedly acts, receives new measurements, updates its state estimate, and recomputes control. We study this difference in a differential-drive path-tracking task with biased odometry and intermittent landmark sensing. Six state estimators are evaluated across 24 sensing conditions using trajectory replay, a 20-step measurement-free rollout, and closed-loop tracking. Replay position RMSE correlates more strongly with closed-loop cross-track RMSE than rollout error (Spearman rho = 0.923 vs. 0.774) and selects a different estimator from the closed-loop optimum in 5/24 conditions, compared with 18/24 for the rollout metric. We then vary rollout horizon and measurement-update interval. With H=20, rank agreement decreases from rho = 0.916 with measurements at every step to rho = 0.774 with no measurements. A horizon-update grid shows that long prediction horizons remain informative when regular corrections are retained, whereas long rollouts without correction can produce rankings that differ substantially from closed-loop behavior. We also test recurrent estimators trained on longer sensing outages. This improves the EKF-anchored models under combined sensing degradation, reducing GRU-EKF cross-track RMSE from 1.72 m to 1.06 m, but the gain is not consistent across isolated outages or estimator architectures. These results show that predictive-model evaluation in robotics should specify both prediction horizon and measurement-update schedule. For models used in feedback, offline rollouts are most informative when their sensing and correction pattern reflects closed-loop operation. Code is available at https://github.com/rdharini2001/Robot_World_Model

0.48