CS Paper Radar

The Operator Mismatch Problem: Deploying BEV Perception with Portable GPU Compute

Rohit Verma, Anand V Bodas

cs.AI ·submitted 2026-10-08 08:42 UTC · v1

Abstract

Modern autonomous driving systems rely on bird's-eye-view (BEV) perception models that fuse camera and LiDAR inputs to detect objects in 3D space. These models are accurate, but they cannot be deployed through standard inference runtimes. The reason is an operator mismatch between dense convolutions (which runtimes handle well), sparse 3D convolutions (which runtimes cannot represent), and geometric scatter operations (which runtimes have no vocabulary for). Today, every sparse convolution library is CUDA-only and PyTorch-coupled, locking BEV deployment to a single vendor's hardware and a single execution framework. We present BEVPIPE, a framework for deploying multimodal BEV perception pipelines using portable GPU compute APIs and integrating them with production inference runtimes. BEVPIPE partitions the model into runtime-managed dense subgraphs and three external operator extensions (voxelizer, sparse encoder, BEV projector), connected through a shared GPU memory space. BEVPIPE achieves a 19.5x end-to-end speedup over conventional deployments while retaining 98.5% of reference mAP. We also showcase that BEVPIPE is portable across different GPU backends.

Closest papers

Nearest neighbours by meaning across the whole index.

Achieving Near-Zero-Overhead Multi-Model Hierarchical Classification in Real-Time Detection Pipelines

cs.CV cs.DCcs.LG ·2026-08-12 ·Vaishnav Raju
similarity 0.72
Abstract · 14 pages, 17 figures, 5 tables. Submitted to Journal of Real-Time Image Processing

Edge-deployed vision systems in target recognition, surveillance, autonomous vehicles, and drone domains require hierarchical inference pipelines where a detection model identifies objects of interest and downstream classifiers provide fine-grained attribute analysis. Running all models on the GPU creates a serial bottleneck that limits real-time throughput as pipeline stages grow. Modern edge SoCs pair GPUs with dedicated neural accelerators (NPUs, DLAs) capable of concurrent execution, yet deploying custom models on these accelerators remains impractical due to strict operator constraints, quantization incompatibilities, and an undocumented end-to-end pipeline. We target NVIDIA Jetson DLA cores as the representative platform. We present a five-step methodology for zero GPU fallback DLA INT8 deployment of classification backbones, comprising architecture adaptation, manual dynamic range workaround to rescue TensorRT's implicit quantization (recovering 94.0% accuracy from implicit quantization's 75%) for rapid pipeline validation before explicit quantization, quantization-aware training, ONNX graph surgery for DLA compilation, and a concurrent GPU-detection/DLA-classification inference pipeline. We document nine engineering constraints with root-cause analysis and generalizable solutions. Validation on a dual-head person attribute classifier running on DLA alongside a GPU object detector on a Jetson Orin NX demonstrates near-zero pipeline overhead (12.5 vs. 13.3~FPS detector-only at 1080p), with dual-DLA scaling at no additional cost. The methodology is backbone-agnostic and generalizes to any detection-classification edge pipeline.

0.12

FieldRT: Reconfigurable Architecture from Geometric Sensitivity to Streaming Acceleration

cs.AR cs.PF ·2026-10-31 ·Haichuan Li
similarity 0.71
Abstract

Geometric perception on edge must simultaneously achieve two goals that are rarely addressed together: preserve accurate spatial understanding for object localization and operate within stringent latency, memory, energy, and precision constraints. This coupling is particularly evident in oriented bird's-eye-view (BEV) detection, where global context, multi-scale interactions, and geometric fusion affect both prediction quality and hardware dataflow. Conventional deployment flows first compress a model using aggregate quality or complexity proxies and then map the compressed operations to a fixed accelerator interface. This separation risks removing geometric structures that appear redundant under coarse metrics but are critical for precise localization, while leaving hardware-level efficiencies in the retained geometry unexploited. We present FieldRT, an algorithm-hardware co-design methodology that uses a common analysis of geometric sensitivity and data lifetime to derive both the deployment profile and its spatial realization. We deploy FieldRT with Axis-Fusion Network (AFN), a sub-million-parameter oriented-BEV detector, and construct controlled profiles that vary the capacity and organization of its geometric processing. The experiments show that redundancy is task-dependent rather than uniform across metrics. Dense prediction can improve while real instance detection deteriorates; coarse localization may remain stable while strict-IoU precision degrades. Moreover, transformations that are individually safe can become detrimental when combined. FieldRT translates the retained geometric structure into a parameterized streaming architecture that shares field access and storage across spatial operators. The resulting custom-PL primitives achieve production-scale numerical closure on an AMD Kria KV260 and make otherwise infeasible multi-dilation execution realizable.

0.21

IndoorBEV: A Lightweight Real-Time LiDAR BEV Perception System for Indoor Mobile Robots

cs.RO ·2026-09-30 ·Haichuan Li
code/data similarity 0.69
Abstract

Efficient indoor LiDAR perception is challenging because mobile robots must understand cluttered three-dimensional environments under strict latency and memory constraints. Existing point-based and voxel-based methods often incur substantial computational overhead, whereas conventional bird's-eye-view (BEV) representations improve efficiency at the cost of discarding vertical geometric information. We present IndoorBEV, a lightweight LiDAR perception framework that mitigates this tradeoff through a height-aware BEV representation and geometry-conditioned feature fusion. IndoorBEV summarizes the vertical point distribution in each BEV cell using statistical height features and multi-frequency height encoding, allowing informative three-dimensional cues to be processed efficiently by two-dimensional convolutions. A lightweight encoder then integrates complementary geometric features with multi-scale local representations and compact global scene context. Decoupled dense prediction heads jointly produce semantic BEV maps and oriented object bounding boxes. IndoorBEV contains only 0.6M parameters and requires 2.3 MB of model storage. On an NVIDIA AGX Orin, it uses 21.52 MB of GPU memory per inference and achieves a mean latency of 169.6 ms under a 200 ms perception deadline, with a deadline miss ratio of 1.8\%. Evaluations on simulated scenes, real-world robot scans, and an open-source indoor point-cloud dataset demonstrate a favorable tradeoff among perception accuracy, latency, and memory consumption. These results indicate that explicitly encoding vertical geometry within a compact BEV representation provides an effective approach to resource-efficient indoor LiDAR perception.

0.24

HiPFuse: Real-Time LiDAR Semantic Segmentation for Robotic Perception on Edge Devices

cs.CV cs.RO ·2026-11-10 ·Haiyang Wu, George Vosselman, Ville Lehtola
code/data similarity 0.67
Abstract · 13 pages, 6 figures

Real-time robotic perception requires accurate LiDAR semantics under limited resources. Bird's-eye-view (BEV) aggregation is efficient but can mix differently labeled surfaces in columns. We present HiPFuse, a point-BEV fusion framework that exchanges point and height-partitioned BEV features across horizontal scales, combining point-wise detail with 2D spatial context from one LiDAR scan. Fixed height assignments preserve separate vertical aggregation groups, with auxiliary supervision of initial BEV features using targets from the same 3D cells. A dual-scale BEV context module adds broader context from a coarser grid. On SemanticKITTI and MCD, HiPFuse achieves 63.7% and 62.5% mIoU at 14.2 and 22.4 FPS on Jetson AGX Orin, respectively, exceeding the 10 Hz scan rate. Separate training and evaluation show indoor applicability. Replacing only the baseline semantic front end with HiPFuse in fixed robotic systems reduces odometry absolute trajectory error RMSE by 10.6% on average across two outdoor sequences and increases accepted-loop recall by 8.7 percentage points on SemanticKITTI. Qualitative semantic maps show more continuous road labeling. Our code is available at https://github.com/haiyang2022/HiPFuse.

0.35

Pelorus: Resource-Aware Lidar-Inertial Odometry

cs.RO ·2026-11-17 ·Charlotte Sieh, Mario Porrmann
code/data similarity 0.67
Abstract

Lidar-inertial odometry (LIO) has an efficiency problem. High-resolution lidars produce millions of points per second that must be processed online on the resource-constrained hardware of robots, cars, or drones, yet most LIO methods neither bound their memory nor report their compute footprint. We introduce Pelorus, a resource-aware LIO with a fixed-capacity multiresolution map of voxelized normal distributions and an iterated equivariant state estimator. A relevance-based eviction keeps the map memory bounded, and a maturity-weighted map update suppresses inconsistent observations. The pipeline runs in parallel on CPUs and GPUs. On five sequences from three public datasets, Pelorus achieves the lowest relative pose error in seven of ten metrics with a small, bounded memory footprint and, with GPU offloading, the lowest CPU load of all compared methods. On an NVIDIA Jetson Orin NX, it processes a 128-beam lidar at 10 Hz. Pelorus is open source with CUDA, Vulkan, and Metal backends.

0.20

VkVIO: Cross-platform GPU Acceleration for Visual-Inertial Odometry with Vulkan

cs.RO cs.CV ·2026-09-24 ·Ole Hoffmann, Mateo de Mayo, Daniel Cremers
similarity 0.67
Abstract

Perception in robotics and XR fundamentally relies on good state estimation. Visual-inertial odometry (VIO) and Simultaneous Localization and Mapping (VI-SLAM) are proven ways of achieving this goal in a cost-effective and accurate manner. Efficiency in these systems allows for smaller, cooler, and lighter devices. GPU acceleration is a natural approach for reducing latency, thanks to their wide availability in platforms like embedded computers, mobile phones, and XR headsets. However, previous works in the literature have limited themselves to the use of CUDA for this task, significantly reducing deployment options to a single vendor. We instead leverage the vendor-agnostic Vulkan API, originally designed for the strict performance requirements of 3D graphics applications. In this work, we present VkVIO, the first, to the best of our knowledge, cross-platform GPU-accelerated VIO method. We provide state-of-the-art accuracy with causal estimates required for real-time operation. We deploy VkVIO on a diverse range of devices spanning a workstation, a laptop, and an extremely inexpensive single-board computer, while outperforming CUDA-based systems on the same hardware. VkVIO enables possibilities for low-latency, low-power, and low-cost VIO in robotics and XR.

0.25

Decoupling Logical Masks from GPU Execution for Dynamic Block-Sparse Attention

cs.AR ·2026-09-22 ·Shanghao Liu, Xiaoyun Yu, Wanting Li, Wenqi Jiang
similarity 0.66
Abstract · 15 pages, 10 figures, 9 tables. Preprint

Attention computation makes inference expensive in video diffusion transformers (vDiTs), which generate videos through iterative denoising. Block-sparse attention (BSA) reduces this cost by computing only blocks selected by a logical mask, which specifies attention interactions to compute. However, coupling logical block geometry to execution choices limits adaptation to varying masks and graphics processing units (GPUs), while runtime kernel specialization can incur preparation overhead that outweighs execution time savings. We present Tessera, a specialized runtime for dynamic BSA that decouples logical masks from GPU execution while preserving specified attention interactions. Its physical mapping layer retains, combines, or subdivides logical attention blocks into physical tiles suited to different attention mask shapes and GPU architectures. Its task organization layer groups and schedules tiles within GPU tasks to reuse data, expose parallelism, and overlap data movement with computation. Finally, profile-guided regime selection enables low- overhead execution plan selection through a lookup table constructed from offline profiling. We implement Tessera with specialized CUDA kernels supporting four NVIDIA GPU generations. Evaluated on 2,315 real attention masks and industrial video diffusion models, Tessera achieves up to 6.79x BSA request speedup over baseline systems in the evaluated video diffusion models.

0.49

MM-BEV: Enhancing Timeliness by Computing Where and When it Matters

cs.RO cs.CVcs.DCeess.SY ·2026-08-15 ·Liangkai Liu, Kang G. Shin
similarity 0.66
Abstract · 12 pages, 20 figures

Multimodal bird's-eye-view (BEV) perception combines LiDAR depth accuracy with dense camera semantics, but its high computational cost and imperfect sensing conditions make real-time deployment challenging. Existing methods largely compress individual detectors and overlook three opportunities: structured sparsity within camera and LiDAR inputs, timing misalignment between modalities, and the fact that many detected objects do not affect the planner's immediate action. We present MM-BEV, a real-time multimodal BEV system guided by a simple principle: compute where and when it matters. MM-BEV divides perception into mandatory work for safety-critical objects within braking distance of the ego vehicle and with short time-to-collision (TTC), and optional work for less urgent regions. It prioritizes mandatory work and reduces or sheds optional work under tight compute budgets. MM-BEV integrates four mechanisms: (1) a criticality-ranked temporal ROI selector based on motion-extrapolated detections from prior frames; (2) sparse, ROI-aware feature extraction using shared-shape camera crops at context-adaptive resolution and ROI-aware LiDAR voxelization; (3) a latency-aware coordinator that adapts LiDAR sweeps, image resolution, and keyframes according to scene dynamics and TTC; and (4) an asynchronous scheduler that decouples sensing from inference and skips stale frames. On nuScenes, MM-BEV reduces inference latency by 1.96x and end-to-end latency by 2.93x, with no loss in geometry-critical recall and only a 0.2 percentage-point drop in safety-critical recall. On a Clearpath Husky A300 equipped with an Ouster-128 LiDAR, BEV cameras, and a Jetson AGX Orin, MM-BEV further reduces mean latency by 2.11x, demonstrating its potential for real-world autonomous systems.

0.40