CS Paper Radar

UrbanTwin Sim2Real LiDAR Challenge 2026: Site-Calibrated Synthetic Data and Multi-Detector Fusion for Roadside 3D Detection

Arnav Kumar, Brijesh Soni

cs.CV eess.SP ·submitted 2026-11-16 02:50 UTC · v1

Abstract

This report summarizes the work submitted by team arnav007 (submission ID 1054222) for the UrbanTwin Sim2Real LiDAR Challenge (V2X-Real track) at the IEEE BigData Cup 2026. The primary objective is to train 3D detectors for vehicles, pedestrians, and trucks using only synthetic roadside LiDAR data, and to generate synthetic scans that are realistic enough to be indistinguishable from real ones. A CARLA based pipeline is developed and calibrated to the fixed sensor site, accurately modeling the range image, including no-return beams, based on unlabelled real development scans. A combined inference detector, using multiple 3D detectors (VoxelNeXt, PV-RCNN, and SECOND) trained exclusively on synthetic data, is also proposed. Each scan is aligned to the synthetic ground plane and processed through the detectors at multiple ground heights. The resulting bounding boxes are fused using bird's-eye view (BEV) clustering, followed by class-specific post-processing to refine heading, box size, and filter low-confidence detections. The final submission achieved a combined score of 0.8947. The ablation study shows that per-scan ground leveling more than doubled the real-scan mAP, while fusing boxes across different heights and models further improved performance.

Closest papers

Nearest neighbours by meaning across the whole index.

Solution for UCF UrbanTwin LUMPI Track: Sim-to-Real Urban LiDAR 3D Object Detection

cs.CV cs.AI ·2026-09-07 ·Pu Luo, Cong Xu, Yumei Li, Kexin Zhang, Licheng Jiao +2 more
similarity 0.80
Abstract · 5 pages,1 figures

We present our solution to the LUMPI track of the UCF UrbanTwin Sim2Real LiDAR Challenge at the 6th DriveX Workshop, ECCV 2026. The detector must be trained only on synthetic data and is evaluated on 50 held-out real LiDAR frames; a separate 50-frame synthetic submission is evaluated for point-cloud realism. Our method addresses the Sim2Real gap at three levels. First, we align synthetic scans to the 50k-point test density and build a 30k-record training pool using UT-LUMPI geometry, RangeLDM-based sampling diversification, rare-class copy-paste, and pedestrian-oriented augmentation. Second, complementary DSVT detectors and Car/Bus PointPillars specialists are trained under the same synthetic-only constraint. Third, predictions are integrated by class-aware routing, asymmetric agreement fusion, constrained residual-recall supplementation, class-coverage auditing, and selective box-size calibration. The realism branch is optimized independently with radial-density matching, weak affine calibration, and calibrated set mixing. The final submission obtains a Combined Score of 0.4692, a Detection Score of 0.1797, a Realism Score of 0.9035, and 3D mAP@0.5 of 0.1258.

0.32

Solution for UCF UrbanTwin V2X-Real Track: Sim-to-Real Urban LiDAR 3D Object Detection

cs.CV ·2026-09-07 ·Pu Luo, Cong Xu, Yumei Li, Kexin Zhang, Licheng Jiao +2 more
similarity 0.80
Abstract · 7 pages,2 figures

Bridging the simulation-to-reality gap in roadside LiDAR requires addressing several coupled discrepancies, including scene geometry, sampling density, return patterns, and pedestrian scale. This report presents a multi-source collaborative training and class-aware fusion framework for Sim2Real 3D detection. The method organizes digital-twin scans, diffusion-redrawn scans, density-stabilized scans, and pedestrian morphology-aligned samples into a unified training pool with complementary roles. Within a common DSVT detection formulation, source-specialized expert branches preserve those roles while optimizing for the same detection objective. At inference, a predefined class-aware fusion pathway integrates geometry-stable and calibration-aware branches for vehicles, sampling-complementary branches for trucks, and morphology-consistent evidence for pedestrians. A label-free point-cloud center blend then refines geometric localization. On the UrbanTwin V2X-Real hidden test set, the unified system achieves a combined score of 0.7421, with 3D mAP@0.5 of 0.4518 and a realism score of 0.8871. The results indicate that a stable, interpretable collaboration among data sources is more valuable than unconstrained aggregation of model outputs.

0.29

Third-Place Solution for the UCF UrbanTwin V2X-Real Track at ECCV 2026: Synthetic Point Cloud Generation and Sim-to-Real 3D Object Detection

cs.CV ·2026-11-12 ·Antong Deng, Yuxin Zhang, Zhongxiang Xing, Zhenzhen Lin, Jiong Mu
published similarity 0.79
Abstract · 8 pages, 1 figure, 4 tables. 3rd place, Track 7 (V2X-Real), UCF UrbanTwin Sim2Real LiDAR Challenge, 6th DriveX Workshop at ECCV 2026

This report presents a complete technical solution for the UrbanTwin V2X-Real Sim2Real LiDAR Challenge, covering both synthetic point cloud generation and 3D object detection. For data generation, we adopt per-axis quantile calibration. Public V2X-Real training data are used to construct the real reference distribution, and independent one-dimensional quantile mappings are estimated for the x, y, and z axes of the base synthetic point clouds. The calibrated point clouds are then resampled using a fixed random seed and clipped to the valid spatial range, producing 50 frames with 2,000 float32 XYZ points per frame. For detection, we use a two-stage Voxel R-CNN (VoxelRCNN) detector trained with RangeLDM-enhanced synthetic point-cloud data, point clouds downsampled to 50k points per frame, and reconstructed pedestrian data. Training is performed on four NVIDIA GeForce RTX 4090 GPUs with a global batch size of 16 for 40 epochs. Final testing uses direct single-model inference with the original VoxelRCNN model. On the official evaluation server, the normalized Realism score is 0.7905 and the normalized Detection score is 0.5583, with mAP@0.5 of 0.3908, BEV mAP@0.5 of 0.4219, and mAP@0.7 of 0.1047. According to the official weighting rule, the final combined score is 0.6 x 0.5583 + 0.4 x 0.7905 = 0.6512. This report documents the complete implementation details of data construction, model training, inference, submission formatting, and official evaluation.

0.22

Geometric Refinement and Distributional Alignment for Sim2Real Roadside LiDAR Detection

cs.CV cs.RO ·2026-11-02 ·Aman Upganlawar
publishedcode/datahot direction similarity 0.76
Abstract · 8 pages. 1st place, Track 6 (UrbanTwin Sim2Real LiDAR, LUMPI), DriveX Grand Challenge, ECCV 2026 Workshop. Code: https://github.com/amanupg/…

We describe our approach to the LUMPI track of the UrbanTwin Sim2Real LiDAR Challenge, in which a 3D object detector must be trained exclusively on synthetic data and evaluated on unseen real roadside LiDAR, while the synthetic point clouds are simultaneously scored for distributional realism. Our final entry scores 0.4793 combined, from a detection mAP of 0.1333 and a realism score of 0.9126. Our contribution is not a new architecture. Working from a PointPillars baseline, we obtained most of our gains from three sources. First, a diagnosis that small object failure in roadside LiDAR is a localisation problem rather than a detection problem, which led to a parameter free geometric refinement worth 86 percent relative on pedestrian AP. Second, an audit of our own post processing which revealed that a shipped rule was destroying valid predictions, the repair of which produced our single largest detection gain at 335 percent relative on the Van class. Third, treating the realism metrics analytically rather than as a black box, which showed that one of the four components is a closed form function of nine summary statistics. We also report an experimental methodology that we believe is the more transferable result. Because mean AP decomposes into independent per class terms, a single submission can carry several mutually controlled experiments, and a submission assembled entirely from previously measured components has a score that is known before it is sent. We predicted our final score to four decimal places prior to submission. We report negative results at equal length, and give a full account of data provenance and compliance in Section 8, including a full account of every mechanism contributing to the submitted score.

0.43

Coordinate Frame Recovery, Per Class NMS and Null Return Modelling for Sim2Real V2X LiDAR Detection

cs.CV cs.RO ·2026-11-02 ·Aman Upganlawar
publishedcode/datahot direction similarity 0.75
Abstract · 8 pages. 1st place, Track 7 (UrbanTwin Sim2Real LiDAR, V2X-Real), DriveX Grand Challenge, ECCV 2026 Workshop. Code: https://github.com/amanu…

We describe our approach to the V2X-Real track of the UrbanTwin Sim2Real LiDAR Challenge, in which a 3D object detector must be trained exclusively on synthetic data and evaluated on unseen real roadside LiDAR from an Ouster OS1-64 unit, while the synthetic point clouds are simultaneously scored for distributional realism. Our final entry scores 0.7457 combined, from a detection mAP of 0.4382 and a realism score of 0.9251. Three findings account for most of our result, and all three were diagnostic rather than architectural. First, a coordinate frame mismatch between our predictions and the evaluation ground truth was suppressing detection AP entirely; recovering it took our score from 0.3520 to 0.5147 in a single submission. Second, a single global non maximum suppression threshold was destroying genuine pedestrian detections, because pedestrians occur in groups and a 0.3 IoU threshold applied to a 0.7 m box suppresses distinct neighbouring people; correcting this per class raised overall mAP from 0.0751 to 0.4295. Third, approximately 31 percent of the real reference point cloud consists of null returns encoded as the literal point (0, 0, 0), a format feature entirely absent from our synthetic clouds, which pinned Earth Mover's Distance near its floor until we modelled it. Negative results are reported at equal length, and Section 9 gives a full account of data provenance and compliance. That section also accounts for every mechanism contributing to the submitted score.

0.43

Fixing the Twin, Not the Detector: Sim-to-Real Roadside LiDAR Detection from Digital-Twin Data

cs.CV cs.LG ·2026-11-19 ·KuanLin Chen, Feng-jui Kuo
publishedcode/data similarity 0.75
Abstract · 8 pages, 1 figure, 5 tables. Accepted to the IEEE BigData 2026 Cup (UCF UrbanTwin Sim2Real LiDAR Challenge); first place, LUMPI track. Code:…

Roadside LiDAR detectors are expensive to train on real labels, and digital twins promise to replace these labels, but a detector trained on a twin meets a different sensor, different objects and different annotation conventions on the real site. We describe the pipeline that placed first in the LUMPI track of the UrbanTwin Sim2Real LiDAR Challenge (IEEE BigData 2026 Cup, 0.5541 combined score) and fifth on its V2X-Real track (0.8102). Every detector is a CenterPoint model trained only on synthetic data, on a single 8 GB laptop GPU. Most of the gain came from correcting differences between the twin and the real site rather than from the detector. We align the twin's coordinates, point density and box convention with the test sensor, generate the pedestrians and riders that the twins lack, plus street clutter as unlabeled negatives, and rebuild twin objects at real size and real point density. A non-learned resampling to the per-cell point density of unlabeled real frames lifts LUMPI realism from 0.589 to 0.947, against 0.949 between real frames. Label-free post-processing then exploits what a fixed roadside sensor guarantees: static structures recur in every frame, and real objects have characteristic point statistics. Reporting the validation and test gain of each component side by side, we find that corrections of systematic sim-to-real differences transfer to the test set, typically keeping a quarter to three quarters of their validation gain, while choices that fit the few objects of a validation set do not.

0.43

WorkIng: A Real-World and Synthetic Highway Work-Zone Benchmark Dataset for Autonomous Navigation

cs.CV ·2026-10-30 ·Xujun Xie, Dominik Rößle, Adithya Mohan, Daniel Cremers, Torsten Schön
publishedbenchmarkcode/data similarity 0.69
Abstract · Accepted to WACV 2027. 19 pages including supplementary material. (c) 2027 IEEE. Personal use of this material is permitted

Highway work zones are critical edge cases for autonomous vehicles (AVs), combining atypical geometries and jurisdiction-specific regulations that existing benchmarks fail to capture: real-world datasets lack 360° multi-modal, full-scene 3D work zone annotation, and no single recorded site can span the layouts and regulatory variants needed for generalizable models. We present WorkIng, a dual-domain benchmark and training resource. WorkIng-Highway provides a 360° multi-modal work zone benchmark with synchronized 128-channel LiDAR, six RGB cameras, and RTK-GPS across 32 km, yielding 12,000+ annotated frames and 227,399 tracked 3D bounding boxes. WorkIng-CARLA complements this with procedural simulation spanning 24 regulation-aware scenarios across six canonical topologies and six international traffic standards, populated by 100+ custom 3D assets. Benchmarking representative 3D detection and end-to-end planning architectures shows that real-world data remains the primary performance driver, and that scenario-stratified synthetic augmentation improves architectures across 2D recognition, 3D detection, and trajectory planning. Resources: https://github.com/cvims/WorkIng

0.42

GeoPillars: A Large-Kernel Pillar Detector for Single-Sweep LiDAR Under Point Sparsity, and a Decomposition of Its Gain

cs.CV ·2026-11-19 ·Chaitanya Dumbre, Shubham Sarkar, Akash Mohapatra, Ghanasham Patil, Sheetal Girase
code/datahot direction similarity 0.67
Abstract · 16 pages, 13 figures, 8 tables

GeoPillars is a pillar-based 3D detector for single-sweep LiDAR under a hard budget of eight points per 0.4 m pillar, a budget that discards 46.6% of in-range points. It adds geometric pillar features, grouped vector attention within each pillar, a learned absolute positional prior on the bird's-eye-view (BEV) grid, a large-kernel backbone, auxiliary surface-normal supervision and a full-resolution detection head. Trained on 123k synthetic CARLA scans and evaluated on cars, it reaches 81.33 BEV AP@0.7, 13.76 points above CenterPoint at the same input budget and 1.91 points above CenterPoint at its native, denser budget. These are in-distribution figures, because the validation split is per-frame rather than scene-disjoint. A rung-by-rung decomposition shows that the new components explain little of the gain. Raising the detection-head resolution alone recovers 10.19 of the 13.76 points, the joint change of encoder, backbone, neck and positional prior adds 3.37, and the normal-supervision branch adds 0.20, which is within run-to-run variation. With heading reversals folded out, orientation error is similar for all seven models, although GeoPillars reverses headings about half as often as a resolution-matched baseline. It is also the slowest of the seven models. Evaluated zero-shot on 6,743 real nuScenes keyframes, no model transfers, and each retains 3.7% to 12.2% of its CARLA accuracy. A collection defect left LiDAR intensity constant, so every model learned from geometry alone. Code and checkpoints are publicly available.

0.48