CS Paper Radar

High-Flux Count-Free Single-Photon 3D Cameras

Kaustubh Sadekar, Vivek K Goyal, David Maier, Atul Ingle

cs.CV ·submitted 2026-08-18 20:35 UTC · v1
published

Abstract

Single-photon cameras based on single-photon avalanche diode (SPAD) technology are gaining popularity for 3D sensing, thanks to their extreme sensitivity and time resolution. There are two key challenges with single-photon cameras that limit their widespread use: (i) they suffer from non-linear distortions called ''pile-up'' when operated in high-photon-flux conditions, and (ii) they generate a large volume of raw photon data, creating a severe data bottleneck at each sensor pixel. In this work, we show that while compressive capture techniques successfully mitigate data transfer challenges, they exacerbate the effects of dead-time distortion because they fail to retain sufficient information about the photon detection history to allow post-processing pile-up correction via existing methods. We propose a new computational-imaging method that combines free-running capture with an analysis-by-synthesis software pipeline to mitigate pile-up distortions. Our results with hardware emulations and full-scene and single-pixel simulations show that our method can reliably capture scene distance and reflectance over a wide range of illumination conditions. Our work will enable high-resolution SPAD cameras that are severely bandwidth-constrained to operate in real-world high-flux scenarios.

Comment
Presented at IEEE ICCP 2026 (Best Paper Award Winner). To appear in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

Closest papers

Nearest neighbours by meaning across the whole index.

One Photon, Many Worlds: Posteriors and Predictions with Single-Photon Cameras

eess.IV cs.CV ·2026-10-02 ·Haejoon Lee, Mohit Gupta, Vijayakumar Bhagavatula, Aswin C. Sankaranarayanan
similarity 0.71
Abstract · 18 pages, 18 figures

Single-photon avalanche diode (SPAD) cameras operate fundamentally differently from conventional cameras due to their photon-counting nature. Each frame produces a binary image: pixels report zero if no photons arrived during exposure, and one if one or more photons arrived. Reconstructing a scene or inferring its properties from a single binary frame is difficult because many different images could produce the same measurement; thus, the inverse problem is fundamentally one-to-many. As we gather more binary measurements, the inherent uncertainty associated with the inverse problem and any associated inference diminishes. With sufficient photon counts, photon noise becomes negligible relative to the signal mean, enabling near-deterministic scene recovery and inference. This work characterizes the transition from stochastic to near-deterministic scene understanding as photon budget increases, analyzing how the stochasticity in photon arrival affects downstream inference tasks. Technically, we develop a conditional generative framework based on a Hypergeometric frame-thinning process for accumulated binary SPAD measurements. Generative models capture the one-to-many nature of photon-starved inverse problems, enabling empirical characterization of how this ambiguity diminishes with increasing measurements and its impact on downstream tasks like character recognition, QR code decoding, and facial analysis.

0.20

UnSPAD: Uncertainty-aware Single Photon Burst Photography

cs.CV ·2026-11-13 ·Sachin Shah, Kalyan Bhetwal, Levi Burner, Christopher A. Metzler
published similarity 0.70
Abstract · Accepted at The Winter Conference on Applications of Computer Vision (WACV) 2027

We introduce UnSPAD, an uncertainty-aware burst photography framework capable of fusing and denoising arbitrarily long sequences of binary single photon avalanche diode (SPAD) measurements. Unlike prior methods that require fixed burst lengths or rely on powerful generative priors, UnSPAD combines learned motion estimation with uncertainty-informed measurement fusion and denoising to produce temporally consistent, hallucination-free reconstructions. UnSPAD first forms an initial scene estimate by averaging flow-aligned binary frames. UnSPAD then uses per-pixel uncertainty maps to perform uncertainty-aware denoising; the denoiser is informed about which regions accumulated the fewest observations (and thus contain the most relative shot noise) due to motion, occlusion, or alignment errors. This two-step reconstruction pipeline helps mitigate hallucinations---outside of the structure implicitly introduced by the flow estimates, the initial scenes estimates are entirely prior-free. Experiments on real and simulated SPAD and quantum image sensor (QIS) benchmarks demonstrate state-of-the-art reconstruction quality, improved temporal consistency, and superior color fidelity relative to the state of the art.

0.34

Spatiotemporal Flux Probing for Single-Photon Videography

cs.CV ·2026-09-18 ·Jerry Yan, Matteo Forlivesi, Bowen Tan, Andrew Xie, Siddharth Somasundaram +1 more
top venuecode/datahot direction similarity 0.69
Abstract · ECCV 2026, Project page: https://jerukan.github.io/spt-flux-probing

We address the problem of recovering high-speed videos from dynamic scenes under extreme photon sparsity. Existing methods rely on aggregating photon detections in local spatiotemporal windows to improve signal-to-noise ratio; however, this local grouping discards global structure and fails in low-light regimes where photon detections are sparse in space and time. In this work, we show that the information needed to recover both motion and illumination is encoded in correlations over the full space-time pattern of photon arrivals. Building on this insight, we develop a spatiotemporal flux probing theory and an algorithm that estimates the Fourier coefficients of the underlying intensity directly from the photon stream. We demonstrate that our approach (1) recovers fast motion and temporal illumination dynamics with substantially fewer photons than prior methods, (2) enables velocity-selective videography that automatically refocuses video onto specific detected motions, and (3) generalizes across sensing modalities including single-photon, event, and spike cameras.

0.62

Memory- and Bandwidth-Efficient SPAD-LiDAR Ranging via Coarse-to-Fine Spline Sketching

eess.IV eess.SP ·2026-09-28 ·Zhenya Zang, Istvan Gyongy, Mike Davies
similarity 0.68
Abstract

This work presents an optimized compression framework for direct time-of-flight (dToF) light detection and ranging (LiDAR), using an accurate, compact timestamp-encoding strategy to address the high data-rate bottleneck from single-photon timing information in single-photon avalanche diode (SPAD) arrays. We propose a hardware-friendly timestamp-to-depth framework in which a sparse single-photon encoding strategy, namely coarse-to-fine spline sketches (CFSS), projects photon timestamps into fine-grained, low-dimensional representations, called sketch values. CFSS converts timestamps on-the-fly to sketch values and accumulates them with lower memory consumption, without constructing histograms, thereby saving on-chip/device memory. CFSS's closed-form solution for ToF retrieval ensures iteration-free reconstruction. Compared with the conventional sketched-LiDAR framework, the proposed method retains the same compression ratio in the single-peak case, ranging from hundreds$\times$ to thousands$\times$ depending on the accuracy-complexity trade-off, while improving depth-estimation accuracy. Both synthetic and real datasets are used to validate the performance. The CFSS framework is further extended to two-peak scenarios, enabling recovery of objects obscured by partially occluding camouflage and uniformly scattering semi-transparent media. Firmware implementation is also investigated by clearly separating online hardware processing from offline software workloads.

0.22

Harnessing SPAD Noise: Test-Time Gated Attention for Indoor Depth Estimation

cs.CV cs.LGcs.ROeess.SP ·2026-10-11 ·Yiquan Gao
code/data similarity 0.66
Abstract

Monocular depth estimation (MDE) has emerged as a cost-effective alternative to hardware-dependent 3D sensing modalities, yet predicting precise metric depth remains a persistent challenge due to intrinsic scale ambiguity, edge bleeding, and limited zero-shot generalization in unseen environments. Recent multi-modal approaches incorporate 1D/3D Single-Photon Avalanche Diode (SPAD) data to resolve these ambiguities, but they mainly treat SPAD noise as a harmful artifact required for denoising or leave the noise ignored. In this paper, we present an essentially different insight: native SPAD noise is an informative prior that implicitly encodes deterministic environmental characteristics. Driven by this paradigm shift, we introduce a novel, lightweight Test-Time Gated Attention framework by Harnessing SPAD Noise, therefore dubbed HSN. Tailored for indoor scenarios, HSN leverages the synergistic combination of 1D SPAD histograms and Gate Adapters (GAs). Through three complementary optimization phases, it generates channel-wise gated attention vectors to dynamically adjust encoder feature maps at inference time, effectively resolving ambiguous scale and boundary challenges without the heavy computational overhead or dense 3D modalities that other TTA methods suffer from. Extensive experiments on challenging real-world indoor benchmarks show that HSN achieves state-of-the-art metric depth accuracy in in-domain settings, while exhibiting even greater potential under zero-shot generalization to unfamiliar environments. Furthermore, interpretability analysis reveals that the proposed gated attention mechanism adaptively shifts attention toward deeper semantic layers under intensifying SPAD noise. Code will be made publicly available.

0.25

FirstCross: Convex Single-Photon 3D Reconstruction from Sparse Views

cs.CV eess.IV ·2026-11-19 ·Chi-Wei Lee, Cheng-Han Chiang, Yi-Chun Chang, An-Kai Chen, Po-Chih Kuo
theorycode/data similarity 0.64
Abstract · Project page: https://arthur031221.github.io/firstcross/

Single-photon lidar records the arrival time of each returning photon, enabling 3D reconstruction from few views and scarce photons. However, leading neural methods such as Transientangelo require over an hour of non-convex per-scene optimization, and their accuracy degrades as photon flux decreases. We propose FirstCross, a network-free approach that instead solves a short sequence of convex problems on a voxel grid. For a solid object, the Poisson likelihood of a ray with given signal and background rates depends on the scene solely through its first crossing, the point where the ray first enters the object. Decomposing this likelihood yields two tractable terms. The first penalizes surfaces in front of detected photons and admits a tight convex closure, namely volume rendering with a running-maximum transmittance in place of NeRF's product transmittance. The second accounts for photons that no surface explains and admits a convex upper bound once the object is assumed to be thick behind each detected surface point. Compared with Transientangelo under its own real-data protocol, FirstCross achieves lower depth error across every test scene, view count, and flux level. With three views and 100 times fewer photons than at the protocol's lowest flux, our mean object depth error stays under half of Transientangelo's with all photons. Given our detected surface points, the best classical fusion has 1.3 to 1.8 times our three-view error across flux levels. On real room-scale captures with 1% of their photons, TransientNeRF has 2.8 to 7.7 times our error. FirstCross lets few-view single-photon scanners cut exposure to 1% and reconstruct an object in about a minute on one GPU.

0.23

Sparse Light Field Sampling Improves Casual 3D and 4D Reconstruction

eess.IV cs.CV ·2026-08-20 ·Shamus Li, Ruiming Cao, Laura Waller, Kristina Monakhova, Sara Fridovich-Keil
datasetcode/datahot direction similarity 0.63
Abstract · Project page: https://shamus.li/lightfield-gaussian-splatting

Many consumer smartphones, stereo cameras, and light field cameras record multiple synchronized viewpoints in a single exposure event. However, novel view synthesis pipelines commonly use only a monocular stream and rely on camera motion or learned priors to obtain angular coverage. In this paper, we ask: why do we use only one viewpoint? We analyze sensor-limited multi-view, where one sensor trades off spatial and angular resolution, and exposure-limited multi-view, where multiple sensors on one commodity device observe each event simultaneously. We introduce a new dataset incorporating three types of commodity multi-view cameras, and evaluate sparse-view 3DGS and 4DGS baselines measuring reconstruction quality as a function of number of exposures and angle between extreme views. Our results demonstrate that using multiple cameras, even with a low baseline, significantly improves reconstruction quality in single-shot, few-shot, and casual video settings. In addition, under a fixed sensor budget, angular sampling improves reconstruction when exposures are scarce despite lower spatial resolution. The gains are most pronounced for single-shot and dynamic scenes, where a stationary monocular camera lacks the angular diversity to recover scene geometry and motion.

0.36

D2ToF: One-Step Diffusion for Depth Super-Resolution from Lightweight Single-Photon LiDAR Waveforms

eess.IV cs.CV ·2026-11-13 ·Pallabjyoti Deka, Lewis Wilson, Stephen McLaughlin, Jonathan Leach, Abderrahim Halimi
similarity 0.61
Abstract

Single-photon LiDAR (SPL) sensors offer exceptional temporal resolution and single-photon sensitivity for depth imaging. However, cost-effective SPL arrays typically suffer from extremely low spatial resolutions (e.g. $8\times 8$ pixels), limiting their real-world utility. In this paper, we propose a novel, one-step super-resolution framework, D2ToF, Distilled Diffusion for Time-of-Flight, that bridges the gap between high-fidelity generative modeling and real-time execution. Our approach uses an amortized variational distillation strategy that allows for a measurement-based data consistency term to be incorporated directly into the one-step training objective. This encourages that the synthesized high-resolution depth maps are consistent with the captured low-resolution Time-of-Flight (ToF) histograms. To overcome the severe ill-posedness of upsampling extremely degraded inputs, we introduce a multimodal guidance strategy. By leveraging a foundational monocular depth model to extract structural priors from a high-resolution reflectivity image, our network is guided by a rich triad of the ToF histogram, reflectivity map, and relative depth estimate. Experimental results outperform evaluated baselines, achieving $16\times$ spatial super-resolution (from $8\times 8 \times 100$ histograms to $128\times 128$ depth maps) at an accelerated inference speed of approximately $83$ milliseconds.

0.19