CS Paper Radar

Memory- and Bandwidth-Efficient SPAD-LiDAR Ranging via Coarse-to-Fine Spline Sketching

Zhenya Zang, Istvan Gyongy, Mike Davies

eess.IV eess.SP ·submitted 2026-09-28 13:21 UTC · v2

Abstract

This work presents an optimized compression framework for direct time-of-flight (dToF) light detection and ranging (LiDAR), using an accurate, compact timestamp-encoding strategy to address the high data-rate bottleneck from single-photon timing information in single-photon avalanche diode (SPAD) arrays. We propose a hardware-friendly timestamp-to-depth framework in which a sparse single-photon encoding strategy, namely coarse-to-fine spline sketches (CFSS), projects photon timestamps into fine-grained, low-dimensional representations, called sketch values. CFSS converts timestamps on-the-fly to sketch values and accumulates them with lower memory consumption, without constructing histograms, thereby saving on-chip/device memory. CFSS's closed-form solution for ToF retrieval ensures iteration-free reconstruction. Compared with the conventional sketched-LiDAR framework, the proposed method retains the same compression ratio in the single-peak case, ranging from hundreds$\times$ to thousands$\times$ depending on the accuracy-complexity trade-off, while improving depth-estimation accuracy. Both synthetic and real datasets are used to validate the performance. The CFSS framework is further extended to two-peak scenarios, enabling recovery of objects obscured by partially occluding camouflage and uniformly scattering semi-transparent media. Firmware implementation is also investigated by clearly separating online hardware processing from offline software workloads.

Closest papers

Nearest neighbours by meaning across the whole index.

High-Flux Count-Free Single-Photon 3D Cameras

cs.CV ·2026-08-18 ·Kaustubh Sadekar, Vivek K Goyal, David Maier, Atul Ingle
published similarity 0.68
Abstract · Presented at IEEE ICCP 2026 (Best Paper Award Winner). To appear in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

Single-photon cameras based on single-photon avalanche diode (SPAD) technology are gaining popularity for 3D sensing, thanks to their extreme sensitivity and time resolution. There are two key challenges with single-photon cameras that limit their widespread use: (i) they suffer from non-linear distortions called ''pile-up'' when operated in high-photon-flux conditions, and (ii) they generate a large volume of raw photon data, creating a severe data bottleneck at each sensor pixel. In this work, we show that while compressive capture techniques successfully mitigate data transfer challenges, they exacerbate the effects of dead-time distortion because they fail to retain sufficient information about the photon detection history to allow post-processing pile-up correction via existing methods. We propose a new computational-imaging method that combines free-running capture with an analysis-by-synthesis software pipeline to mitigate pile-up distortions. Our results with hardware emulations and full-scene and single-pixel simulations show that our method can reliably capture scene distance and reflectance over a wide range of illumination conditions. Our work will enable high-resolution SPAD cameras that are severely bandwidth-constrained to operate in real-world high-flux scenarios.

0.41

D2ToF: One-Step Diffusion for Depth Super-Resolution from Lightweight Single-Photon LiDAR Waveforms

eess.IV cs.CV ·2026-11-13 ·Pallabjyoti Deka, Lewis Wilson, Stephen McLaughlin, Jonathan Leach, Abderrahim Halimi
similarity 0.64
Abstract

Single-photon LiDAR (SPL) sensors offer exceptional temporal resolution and single-photon sensitivity for depth imaging. However, cost-effective SPL arrays typically suffer from extremely low spatial resolutions (e.g. $8\times 8$ pixels), limiting their real-world utility. In this paper, we propose a novel, one-step super-resolution framework, D2ToF, Distilled Diffusion for Time-of-Flight, that bridges the gap between high-fidelity generative modeling and real-time execution. Our approach uses an amortized variational distillation strategy that allows for a measurement-based data consistency term to be incorporated directly into the one-step training objective. This encourages that the synthesized high-resolution depth maps are consistent with the captured low-resolution Time-of-Flight (ToF) histograms. To overcome the severe ill-posedness of upsampling extremely degraded inputs, we introduce a multimodal guidance strategy. By leveraging a foundational monocular depth model to extract structural priors from a high-resolution reflectivity image, our network is guided by a rich triad of the ToF histogram, reflectivity map, and relative depth estimate. Experimental results outperform evaluated baselines, achieving $16\times$ spatial super-resolution (from $8\times 8 \times 100$ histograms to $128\times 128$ depth maps) at an accelerated inference speed of approximately $83$ milliseconds.

0.19

Memory-efficient GPU pipelines for real-time non-line-of-sight reconstruction

cs.DC cs.CV ·2026-08-28 ·Alfonso López-Ruiz, Diego Royo
similarity 0.63
Abstract

Non-line-of-sight (NLOS) imaging reconstructs scenes hidden around a corner from indirect light recorded by a single-photon avalanche diode (SPAD). A single reconstruction is a large inverse problem: billions of photon timestamps must be binned, moved through memory, transformed and inverted. As SPAD arrays raise acquisition throughput, reconstruction becomes the limiting stage. We rebuild the GPU execution of two established wave-based algorithms, f-k migration and phasor-fields, for both streaming and offline processing. On the phasor-fields side we assemble the ring-and-radius kernels of previous work once and offline, using the analytic Fourier transform of a ring, so the propagation kernel never exists in dense form at runtime, reducing the memory and bandwidth. We reorganize the pipeline of both algorithms with fused kernels, warp-level photon binning, batched transforms, CUDA graph replay, and FP16 storage applied only where it reduces the actual bottleneck. Our implementations are up to 42x faster than the reference streaming pipeline and up to 14x faster than the fastest published GPU baseline, all while using a fraction of the memory (down to 2.5%), enabling vastly larger and finer reconstructions on the same hardware, or comparable ones within a much lower memory budget. We report an ablation of each implementation choice and propose three denoising strategies enabled by the resulting frame budget for next-generation NLOS video processing.

0.20

On-Device Super-Resolution Imaging for a Low-Cost SPAD Array on a 640-KB-SRAM Microcontroller

eess.IV ·2026-10-06 ·Zhenya Zang, Istvan Gyongy, Mike Davies
similarity 0.61
Abstract

This work presents a super-resolution (SR) deep-learning (DL) architecture to simultaneously upscale low-resolution (LR) depth and intensity images acquired by a consumer-grade VL53L9CX time-of-flight (ToF) sensor, a compact single-photon avalanche diode (SPAD) ranging device, while targeting deployment on a resource-constrained microcontroller (MCU) with only 640 KB SRAM and 2 MB Flash. The VL53L9CX provides 54 $\times$ 42 depth and intensity measurements. The proposed framework performs $\times$4 spatial SR to reconstruct 216 $\times$ 168 depth and intensity images simultaneously. The network employs separate reconstruction branches for intensity and high-resolution (HR) depth. We train the network with efficient loss functions and investigate the performance by testing multiple compact, hardware-oriented backbones, including Spatially Adaptive Feature Modulation (SAFM), Swift Parameter-Free Attention Network (SPAN), and Residual Local Feature Network (RLFN). We evaluate the network on both synthetic and real measurements, with particular attention to constrained activation and memory storage. We export the selected network to ONNX, quantize it to INT8, generate C inference code for an STM32H563ZI Arm Cortex-M33 MCU, and integrate it with our simplified sensor's firmware. This work is the first to demonstrate a compact DL SR model on a low-cost bare-metal MCU for a consumer-grade LR SPAD sensor, integrated with customized MCU firmware to enable on-device SR inference.

0.22

FirstCross: Convex Single-Photon 3D Reconstruction from Sparse Views

cs.CV eess.IV ·2026-11-19 ·Chi-Wei Lee, Cheng-Han Chiang, Yi-Chun Chang, An-Kai Chen, Po-Chih Kuo
theorycode/data similarity 0.61
Abstract · Project page: https://arthur031221.github.io/firstcross/

Single-photon lidar records the arrival time of each returning photon, enabling 3D reconstruction from few views and scarce photons. However, leading neural methods such as Transientangelo require over an hour of non-convex per-scene optimization, and their accuracy degrades as photon flux decreases. We propose FirstCross, a network-free approach that instead solves a short sequence of convex problems on a voxel grid. For a solid object, the Poisson likelihood of a ray with given signal and background rates depends on the scene solely through its first crossing, the point where the ray first enters the object. Decomposing this likelihood yields two tractable terms. The first penalizes surfaces in front of detected photons and admits a tight convex closure, namely volume rendering with a running-maximum transmittance in place of NeRF's product transmittance. The second accounts for photons that no surface explains and admits a convex upper bound once the object is assumed to be thick behind each detected surface point. Compared with Transientangelo under its own real-data protocol, FirstCross achieves lower depth error across every test scene, view count, and flux level. With three views and 100 times fewer photons than at the protocol's lowest flux, our mean object depth error stays under half of Transientangelo's with all photons. Given our detected surface points, the best classical fusion has 1.3 to 1.8 times our three-view error across flux levels. On real room-scale captures with 1% of their photons, TransientNeRF has 2.8 to 7.7 times our error. FirstCross lets few-view single-photon scanners cut exposure to 1% and reconstruct an object in about a minute on one GPU.

0.23

ELiCv2: Learned LiDAR Geometry Compression under Real-Time Embedded Constraints

eess.IV ·2026-11-03 ·Soowoong Kim, Junsik Kim, Gun Bang, Jinsoo Choi
similarity 0.61
Abstract

Improving learned LiDAR compression increasingly relies on heavier inference, making it harder to meet the constraints of real-time embedded systems. We present ELiCv2, which removes two sources of redundant computation in progressive coding. Its discrete entropy interface selects a precomputed integer CDF instead of materializing one from a per-voxel distribution, while its occupancy-driven kernel-map builder derives finer-level sparse-convolution kernel-maps directly from the decoded hierarchy, eliminating coordinate-based neighbor search. ELiCv2 reinvests the recovered computation budget in lightweight ego-motion-aware temporal conditioning that gathers local occupancy context from the aligned previous reconstruction. On four LiDAR datasets and an NVIDIA Jetson Thor, ELiCv2 achieves a mean BD-rate reduction of 30.97% over G-PCC with channel width C=32 while sustaining the 20Hz nuScenes capture rate. Increasing the channel width to C=64 improves the mean reduction to 35.22%.

0.26

CRISP: Fixing Flying Pixels in Latent LiDAR Generation via Diffusion Decoding

cs.CV cs.AI ·2026-10-08 ·Andrea Ceron, Michael Schmidt, Alvaro Marcos-Ramiro, Sebastian Schmidt, Benjamin Busam
published similarity 0.59
Abstract · Accepted at NeurIPS 2026 (poster). 41 pages, 10 figures, 18 tables. Project page: https://andrea25512.github.io/CRISP/

Latent LiDAR pipelines suffer from flying pixels: convolutional VAEs blur sharp radial depth discontinuities, yielding edge depths that back-project to points floating between surfaces. We identify this as a major, directly correctable decoder bottleneck and introduce CRISP: a pixel-space diffusion decoder with a backbone-agnostic latent adapter, DiT-based denoiser, and support mask predictor. CRISP replaces video-VAE and LiDAR-native decoders alike while keeping the encoder and latent generator fixed. Across KITTI-360, SemanticKITTI, and nuScenes, replacing only the decoder reduces FSVD/FPVD by 50.5% on average across frozen backbones; for generic video VAEs, the reductions reach 71%/74%. On the LiDAR-native LiDM backbone, FRID drops by 71%, with the largest gains at depth discontinuities. In a pretrained LiDM world model, the same zero-shot replacement improves FSVD by 15.5%, narrowing the sim-to-real gap. Code and checkpoints will be released at https://andrea25512.github.io/CRISP/.

0.46

Harnessing SPAD Noise: Test-Time Gated Attention for Indoor Depth Estimation

cs.CV cs.LGcs.ROeess.SP ·2026-10-11 ·Yiquan Gao
code/data similarity 0.59
Abstract

Monocular depth estimation (MDE) has emerged as a cost-effective alternative to hardware-dependent 3D sensing modalities, yet predicting precise metric depth remains a persistent challenge due to intrinsic scale ambiguity, edge bleeding, and limited zero-shot generalization in unseen environments. Recent multi-modal approaches incorporate 1D/3D Single-Photon Avalanche Diode (SPAD) data to resolve these ambiguities, but they mainly treat SPAD noise as a harmful artifact required for denoising or leave the noise ignored. In this paper, we present an essentially different insight: native SPAD noise is an informative prior that implicitly encodes deterministic environmental characteristics. Driven by this paradigm shift, we introduce a novel, lightweight Test-Time Gated Attention framework by Harnessing SPAD Noise, therefore dubbed HSN. Tailored for indoor scenarios, HSN leverages the synergistic combination of 1D SPAD histograms and Gate Adapters (GAs). Through three complementary optimization phases, it generates channel-wise gated attention vectors to dynamically adjust encoder feature maps at inference time, effectively resolving ambiguous scale and boundary challenges without the heavy computational overhead or dense 3D modalities that other TTA methods suffer from. Extensive experiments on challenging real-world indoor benchmarks show that HSN achieves state-of-the-art metric depth accuracy in in-domain settings, while exhibiting even greater potential under zero-shot generalization to unfamiliar environments. Furthermore, interpretability analysis reveals that the proposed gated attention mechanism adaptively shifts attention toward deeper semantic layers under intensifying SPAD noise. Code will be made publicly available.

0.25