CS Paper Radar

Modelling dynamic systems transfer functions from events in computational neuromorphic imaging

Nimrod Kruger, Gregory Cohen

cs.CV ·submitted 2026-09-24 14:22 UTC · v1
publishedunusual

Abstract

Event Vision Sensing (EVS) report threshold crossings of log-irradiance, so a static optical system imaging a static scene produces no output at all. The classical procedure for measuring a Point Spread Function (PSF), illuminating the system with a constant point source, therefore has no event-based equivalent: the probe must carry a temporal profile, and that profile becomes part of the measurement. A growing body of Computational Neuromorphic Imaging (CNI) work already exploits this, pairing engineered or modulated optics with event sensing, but each system adopts a particular excitation together with a particular reading of the event stream without the correspondence between the two being stated. We examine that correspondence directly within a analytical framework of an Linear Shift-Invariant (LSI) optical system with a specified Modulation Transfer Function (MTF), a first-order filter EVS pixel model, and three different temporal probes: a step function, a linear ramp and an exponential ramp. By analysing the inverse of the entire chain for different event-statistic, and comparing the results to the specified MTF, we identify the context where each probe is most relevant. We consider how photon-noise and cross-array threshold mismatch effects the analytical accuracy of the probe-inverse. Results show that the widely used step probe is highly susceptible to mismatch while resilient to photon shot-noise, while a linear rise probe and exponential rise probe retain their ability to infer signal levels even with high mismatch. We discuss the potential of dynamic-PSFs as components of a full forward operator from scene to events. In this, we use this analytical description to define dynamic-PSFs around EVS, and discuss the gaps toward a unified pixel model and a scene-composition framework required for CNI.

Comment
Conference paper - SPIE Sensors + Imaging 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Neuromorphic Camera for Multi-Target Pointing Acquisition and Tracking in Free Space Optical Communication

eess.IV physics.optics ·2026-11-03 ·Francesco Nardo, Nigitha Selvaraj, Klaus Schertler, Peter Krötz, Ricardo Barrios +2 more
published similarity 0.58
Abstract · 16 pages, 11 figures, 3 tables including 3 appendices. This paper has been accepted for invited presentation at SPIE Sensors + Imaging 2026,…

Pointing, Acquisition, and Tracking (PAT) in Free-Space Optical Communication (FSOC) applications must achieve both low jitter and a large Field of View (FoV) to support beacon acquisition and multi-target tracking. While conventional frame-based sensors provide the necessary angular coverage, their performance is limited by latency, available frame-rate, and motion blur. This work explores and demonstrates the application of an event-based vision sensor for PAT tasks in FSOC. The system employs a neuromorphic sensor that asynchronously records brightness changes (events) with microsecond-level temporal resolution. The main contribution of this paper is a comparative analysis of event-based tracking algorithms for the simultaneous acquisition and tracking of multiple optical beacons that operate directly on this asynchronous stream. The performance of these strategies is validated on an experimental dataset representing realistic operational conditions for dynamic High-Altitude Platform (HAP) and Unmanned Aerial Vehicle (UAV) networks. The analysis compares key metrics, such as tracking accuracy and processing latency, under various signal-to-background conditions.

0.38

Efficient Multi-Timescale Event Representations for Feed-Forward Object Detection

cs.CV ·2026-09-04 ·Fredrik Lundell, Per-Erik Forssen, Mårten Wadenbäck, Astrid Lundmark
published similarity 0.56
Abstract · Accepted to the Workshop on Neuromorphic Vision (NEVi) at ECCV 2026, 16 pages

Autonomous systems require robust low-latency perception under rapidly changing scene dynamics and challenging illumination. In event cameras object detection commonly relies on recurrent architectures to accumulate sparse temporal information over time. This work investigates how temporal information can be encoded directly within the event representation. We propose a confidence-normalized continuous multi-timescale representation based on logarithmic B-spline temporal encoding together with a geometry-aware local confidence mechanism that exploits the spatial structure of event generation. Using a fixed feed-forward EventCenterNet detector, we show that the proposed representations consistently outperform the compact CSTR representation on PEDRo and Gen1 datasets. We further introduce a recursive exponential-polynomial approximation that enables efficient event-by-event updates while largely preserving detection performance. These results demonstrate that carefully designed event representations can capture a substantial portion of the temporal information learned through recurrent temporal modeling, providing a promising foundation for efficient feed-forward, event-driven, and future neuromorphic object detection.

0.56

Real-Time Event-Based LED Tracking and Pose Estimation with PWM-Encoded Markers

cs.RO ·2026-11-05 ·Junkai Niu, Sheng Zhong, Delei Kong, Xin Wang, Kaizhen Sun +3 more
similarity 0.54
Abstract · This work has been submitted to the IEEE for possible publication

Event cameras are neuromorphic vision sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, supporting state estimation under rapid motion, motion blur, and high dynamic range (HDR) illumination. This also enables capture of high-frequency blinking patterns from light-emitting diodes (LEDs). Based on this, we propose a real-time event-based framework for LED tracking and six-degree-of-freedom (6-DoF) relative pose estimation using pulse-width modulation (PWM)-encoded LED markers. Multiple LEDs share a common blinking frequency and encode identities through distinct PWM duty cycles, reducing frequency-resource requirements and computational cost. Our dual-rate asynchronous tracking framework combines low-rate LED detection with high-rate raw-event updates, maintaining robust LED identity association and accurate pixel-level tracking under rapid motion. LED identities and positions establish reliable 2D-3D correspondences for real-time metric 6-DoF relative pose estimation. Extensive evaluations on a self-collected dataset demonstrate robustness under large motion flow and transient image-plane LED overlap. Furthermore, we develop a coordinated robotic platform comprising a quadrotor and a Mecanum-wheeled mobile robot, demonstrating the proposed framework's potential for closed-loop drone guidance through hovering and target-following experiments under time-varying HDR illumination, with onboard deployment on an NVIDIA Jetson Orin NX. The dataset and implementation will be released to facilitate further research.

0.36

Quanta Perception as Probabilistic Events

cs.CV cs.AI ·2026-08-27 ·Varun Sundar, Pavan Thodima, Sacha Jungerman, Mohit Gupta
similarity 0.53
Abstract · For project webpage, see https://wisionlab.com/project/probabilistic-events/

Autonomous systems rely on extracting information from light, yet remain brittle in extreme environments, from nighttime navigation to high-speed robotics. Conventional sensors aggregate photons over fixed exposures, imposing trade-offs between sensitivity, dynamic range, and temporal resolution that degrade perception when photons are scarce or dynamics are rapid. Quanta sensors detect individual photons, but their streams exceed real-time compute and latency budgets by orders of magnitude. Here we introduce $\textit{probabilistic events}$, a computational primitive for real-time quanta perception from individual photon detections. By computing the posterior over the time since the last intensity change, we represent photon streams as recursive belief states. Rather than fixed-threshold event-camera triggers, this recursive Bayesian formulation yields three low-latency signals: motion-adaptive scene flux, high-fidelity activity maps, and entropy-based perceptual uncertainty. This representation enables perception in extreme conditions, including pose estimation of a running person at $\sim$0.05 lux---without retraining vision models. Our approach processes input streams exceeding 50{,}000 quanta frames per second on commodity GPU hardware---yielding kilohertz-scale outputs up to four orders of magnitude faster than state-of-the-art quanta reconstruction baselines, even for megapixel arrays. By replacing frame reconstruction with direct probabilistic inference over photon streams, this work bridges photon-counting quanta sensing with robotic vision.

0.32

Measurement-Adaptive Light-Cone Transform for Robust Non-Line-of-Sight Imaging

eess.IV cs.CV ·2026-10-16 ·Lianfang Wang, Xueying Liu, Junnan Dong, Qinghua Zhang, Jun Liu +2 more
published similarity 0.53
Abstract

Practical non-line-of-sight (NLOS) measurements are affected by noise and deviations from idealized light transport, which can degrade reconstruction with conventional light-cone transform (LCT) methods. We propose the Measurement-Adaptive Light-Cone Transform (MALCT), a physics-guided framework that retains the efficient Fourier-domain inversion structure of LCT. {Within a unified variational formulation, MALCT combines the analytical response with a shared learned response through measurement-conditioned weighting and predicts a measurement-conditioned, frequency-dependent regularization map.} An attention-enhanced spatio-temporal feature extraction module captures long-range temporal dependencies and local spatial correlations to support reconstruction under low-signal conditions. Experiments on synthetic data and real measurements from different NLOS systems demonstrate improved reconstruction quality. Controlled evaluations further assess robustness to temporal-response broadening and reduced acquisition time. MALCT also preserves scene structures with spatial sampling grids down to $32\times32$, supporting its potential for NLOS imaging under limited acquisition budgets.

0.35

Ternary Visible Light Communication Using Event-Based Vision Sensors

eess.IV eess.SP ·2026-09-29 ·Sotaro Kuremoto, Junya Hara, Hiroshi Higashi, Yuichi Tanaka
similarity 0.53
Abstract · Submitted to ICASSP 2027. 5 pages, 5 figures, 1 table

This paper proposes a ternary visible light communication method using an event-based vision sensor (EVS) and a liquid crystal display (LCD). The throughput of optical camera communication (OCC) is limited by the frame rates of conventional frame-based cameras, and EVSs are expected to overcome this limitation because their pixels asynchronously trigger events with polarity in response to brightness changes, with a temporal resolution on the order of microseconds. However, existing event-based OCC relies on binary signaling that uses only the presence or absence of events and leaves the polarity unused. The proposed method maps the three brightness transitions (increase, decrease, and no change) to ternary symbols, thereby increasing the information carried per symbol. At the transmitter, the LCD displays a marker in which the transmitted data are encoded; at the receiver, the EVS captures the marker and records the brightness changes as events with positive and negative polarities. To suppress errors caused by the timing mismatch between display updates at the transmitter and demodulation at the receiver, we also propose a demodulation algorithm based on frame-transition detection. Experimental results demonstrate that the proposed method achieves a throughput of 336.5 kbps, approximately 150% higher than that of its binary counterpart.

0.28

Spatiotemporal Flux Probing for Single-Photon Videography

cs.CV ·2026-09-18 ·Jerry Yan, Matteo Forlivesi, Bowen Tan, Andrew Xie, Siddharth Somasundaram +1 more
top venuecode/datahot direction similarity 0.53
Abstract · ECCV 2026, Project page: https://jerukan.github.io/spt-flux-probing

We address the problem of recovering high-speed videos from dynamic scenes under extreme photon sparsity. Existing methods rely on aggregating photon detections in local spatiotemporal windows to improve signal-to-noise ratio; however, this local grouping discards global structure and fails in low-light regimes where photon detections are sparse in space and time. In this work, we show that the information needed to recover both motion and illumination is encoded in correlations over the full space-time pattern of photon arrivals. Building on this insight, we develop a spatiotemporal flux probing theory and an algorithm that estimates the Fourier coefficients of the underlying intensity directly from the photon stream. We demonstrate that our approach (1) recovers fast motion and temporal illumination dynamics with substantially fewer photons than prior methods, (2) enables velocity-selective videography that automatically refocuses video onto specific detected motions, and (3) generalizes across sensing modalities including single-photon, event, and spike cameras.

0.62

PIE-PS: Photometric Stereo from Physical Irradiance Event Streams

cs.CV ·2026-10-06 ·Xiangze Meng, Guangyu Li, Jing Li, Di Mei, Songchen Ma +2 more
top venue similarity 0.52
Abstract · 9 pages, 7 figures. Accepted to SIGGRAPH Asia 2026 Conference Papers

Event cameras record asynchronous log-image-irradiance changes with microsecond latency and high dynamic range. These properties are useful for photometric stereo under moving illumination, but raw events are sparse and depend on an unknown contrast threshold. We start from the event trigger model and derive a physical relation between adjacent events, light motion, and surface normals. This relation gives a direct physics-only solver, but the solver needs the threshold, enough events at each pixel, and independent per-pixel optimization. To address these limits, we introduce PIE-PS, a learning-based framework for dense surface normal reconstruction from raw event streams and known lighting. We form Physical Irradiance Events (PIEs) by pairing two adjacent events at the same pixel with their corresponding light directions. Each PIE provides a Physical Irradiance Event Feature (PIEF), defined as the signed event rate. PIEF does not require the unknown contrast threshold. To share spatial and temporal context across nearby PIEs, we introduce PIE-GNN, which treats each PIE as a graph node and encodes it with its light-pair geometry. Since the reliability of PIE observations can vary with local appearance, illumination geometry, and sensor noise, Reliability-Grading Attention (RGA) predicts reliability weights to down-weight unreliable PIEs. Pixel aggregation then produces dense normals. Experiments on synthetic and real data show that PIE-PS outperforms prior event-based photometric stereo methods and the direct solver baseline.

0.43