CS Paper Radar

Neuromorphic Camera for Multi-Target Pointing Acquisition and Tracking in Free Space Optical Communication

Francesco Nardo, Nigitha Selvaraj, Klaus Schertler, Peter Krötz, Ricardo Barrios, Sebastian Houben, Sebastian Randel

eess.IV physics.optics ·submitted 2026-11-03 19:01 UTC · v1
published

Abstract

Pointing, Acquisition, and Tracking (PAT) in Free-Space Optical Communication (FSOC) applications must achieve both low jitter and a large Field of View (FoV) to support beacon acquisition and multi-target tracking. While conventional frame-based sensors provide the necessary angular coverage, their performance is limited by latency, available frame-rate, and motion blur. This work explores and demonstrates the application of an event-based vision sensor for PAT tasks in FSOC. The system employs a neuromorphic sensor that asynchronously records brightness changes (events) with microsecond-level temporal resolution. The main contribution of this paper is a comparative analysis of event-based tracking algorithms for the simultaneous acquisition and tracking of multiple optical beacons that operate directly on this asynchronous stream. The performance of these strategies is validated on an experimental dataset representing realistic operational conditions for dynamic High-Altitude Platform (HAP) and Unmanned Aerial Vehicle (UAV) networks. The analysis compares key metrics, such as tracking accuracy and processing latency, under various signal-to-background conditions.

Comment
16 pages, 11 figures, 3 tables including 3 appendices. This paper has been accepted for invited presentation at SPIE Sensors + Imaging 2026, Edinburgh, UK. 2026 SPIE

Closest papers

Nearest neighbours by meaning across the whole index.

Asynchronous Tracking, Optical Communication and 3D Motion Capture using Event-based Sensors

cs.RO cs.CV ·2026-10-03 ·Ziwei Wang, Angus Apps, Holly Battisson, Iain Guilliard, Timothy Molloy +1 more
similarity 0.71
Abstract · 16 pages, 17 figures

Inter-robot communication is a key enabling technology in cooperative robotics applications. While wireless communication is ubiquitous, simple, and effective, it has inherent limitations: the receiving robot cannot easily localise the source of an incoming signal, clock synchronisation is challenging, and signals are broadcast to all nearby devices rather than targeted recipients. Optical communication provides a complementary channel that is inherently directional and spatially localised, directly addressing these limitations. Event cameras are bio-inspired dynamic vision sensors that respond to changes in image intensity with high temporal resolution, high dynamic range, and low latency, making them well-suited as receivers for high-rate optical communication in cooperative robotic systems. In this paper, we propose the Asynchronous Tracking and Optical Communication (ATOC) system, which integrates LED smart-beacon modulation, event-based detection, optical tracking, and event-data demodulation in a single pipeline. By simultaneously tracking and demodulating multiple LED smart beacons, ATOC transforms a conventional visual marker into a robust communication channel suitable for a wide range of high-impact robotics applications. We validate ATOC in a suite of laboratory studies and two 'applications': a smart-city demonstration and a 3D motion-capture system.

0.19

Real-Time Event-Based LED Tracking and Pose Estimation with PWM-Encoded Markers

cs.RO ·2026-11-05 ·Junkai Niu, Sheng Zhong, Delei Kong, Xin Wang, Kaizhen Sun +3 more
similarity 0.63
Abstract · This work has been submitted to the IEEE for possible publication

Event cameras are neuromorphic vision sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, supporting state estimation under rapid motion, motion blur, and high dynamic range (HDR) illumination. This also enables capture of high-frequency blinking patterns from light-emitting diodes (LEDs). Based on this, we propose a real-time event-based framework for LED tracking and six-degree-of-freedom (6-DoF) relative pose estimation using pulse-width modulation (PWM)-encoded LED markers. Multiple LEDs share a common blinking frequency and encode identities through distinct PWM duty cycles, reducing frequency-resource requirements and computational cost. Our dual-rate asynchronous tracking framework combines low-rate LED detection with high-rate raw-event updates, maintaining robust LED identity association and accurate pixel-level tracking under rapid motion. LED identities and positions establish reliable 2D-3D correspondences for real-time metric 6-DoF relative pose estimation. Extensive evaluations on a self-collected dataset demonstrate robustness under large motion flow and transient image-plane LED overlap. Furthermore, we develop a coordinated robotic platform comprising a quadrotor and a Mecanum-wheeled mobile robot, demonstrating the proposed framework's potential for closed-loop drone guidance through hovering and target-following experiments under time-varying HDR illumination, with onboard deployment on an NVIDIA Jetson Orin NX. The dataset and implementation will be released to facilitate further research.

0.36

ECO-COMM: An Ultra Low-Latency Event Camera based Optical Communication System

cs.NI ·2026-08-25 ·Chengling Xu, Keigo Hirakawa, Feng Ye
published similarity 0.63
Abstract · Conditionally accepted to appear ACM MobiHoc 2026

Ultralow-latency communication is critical for emerging next-generation applications such as XR, real-time control, and distributed sensing. We present ECO-COMM, an event-camera-based optical communication system for ultra-low-latency device association and lightweight information exchange. By exploiting the asynchronous sensing and microsecond-level temporal resolution of event cameras, ECO-COMM captures high-frequency optical signals without frame-based acquisition delays. We identify and analyze key hardware-induced challenges in event-camera communication, including timestamp inconsistency, readout contention, trailing effects, and the inevitable refractory period, and develop hardware-aware mitigation techniques to address them. Focusing on a single transmitter-receiver optical link, ECO-COMM establishes the feasibility of practical ultra-low-latency event-camera communication using commercially available hardware.A prototype implementation using an eight-LED transmitter and an off-the-shelf event camera achieves device association within 15 microseconds, symbol latency as low as 100 microseconds, and end-to-end latency below 8 milliseconds for 32-byte payloads at 0.1% bit error rate. ECO-COMM establishes a practical and complementary communication paradigm for ultra-low-latency systems where responsiveness and temporal precision are paramount.

0.44

A Height-Constrained 2-Point Minimal Solver for Pose Estimation from Active LED Markers with Event Cameras

cs.CV cs.RO ·2026-08-10 ·Runze Yuan, Alexander Kappler, Jun Zhang, Kuangyi Chen, Fabio Morbidi +3 more
published similarity 0.59
Abstract · 8 pages, 6 figures, accepted by IEEE/RSJ International Conference on INTELLIGENT ROBOTS & SYSTEMS (IROS) 2026

In many autonomous applications requiring real-time localization, active marker-based systems are preferred due to their low latency and ease of deployment compared to computationally demanding feature-based methods. Event~\mbox{cameras} offer high temporal resolution and minimal delay and are commonly used with active LED markers for robust real-time localization. Existing methods typically rely on Perspective-n-Point (PnP) solvers for pose estimation. However, structured marker layouts can be challenging to deploy in space-constrained scenarios, while partial self-motion information (e.g., gravity direction and altitude) is readily available from onboard sensors. We derive a robust and accurate minimal solver that estimates camera pose from only two LED markers by incorporating known tilt angle and camera height measured by an onboard sensor, such as an IMU or an altimeter. The proposed formulation uniquely determines the camera pose through both a closed-form and a linear least-squares solution. We further analyze degenerate configurations and characterize the conditions under which height information does not contribute to rotation estimation. For evaluation, we developed an event-based active marker system to collect real-world data with ground truth from a motion capture system. Experiments on both synthetic and real data demonstrate improved accuracy over the state-of-the-art P2P solver and competitive performance relative to P3P.

0.33

Event-Based Motion Estimation via Oriented Distance Fields

cs.CV cs.RO ·2026-08-25 ·Lei Sun, Yuqin Ma, Weilun Li, Haoran Liang, Runyi Yang +3 more
similarity 0.58
Abstract

Event-based motion estimation is central to tasks that demand high temporal resolution and robustness to fast motion. Existing methods typically rely on iterative optimization or repeated hypothesis comparison, offsetting the sensor's low-latency advantage. We propose Oriented Distance Field Motion Estimation (ODF Motion Estimation), which replaces this optimization with a single averaging step over a precomputed field of event distance vectors, combined with an adaptive event-count selection strategy and a parameter-free trail filter. On public and self-collected datasets, ODF motion estimation reaches sub-pixel accuracy at the lowest latency among compared methods. We validate its generality on two downstream applications rather than treating them as separate contributions. First, the estimated trajectory is converted into a blur kernel and paired with a compact iterative-unfolding network, trained on simulated motion-estimation noise, for real-time non-blind image deblurring, attaining competitive or superior PSNR/SSIM with under 1M parameters. Second, the same precomputed field is repurposed for directional event filtering in a low-power asynchronous pupil and glint tracker, sustaining stable tracking for tens of seconds while lowering a near-eye module's power draw.

0.18

Modelling dynamic systems transfer functions from events in computational neuromorphic imaging

cs.CV ·2026-09-24 ·Nimrod Kruger, Gregory Cohen
publishedunusual similarity 0.58
Abstract · Conference paper - SPIE Sensors + Imaging 2026

Event Vision Sensing (EVS) report threshold crossings of log-irradiance, so a static optical system imaging a static scene produces no output at all. The classical procedure for measuring a Point Spread Function (PSF), illuminating the system with a constant point source, therefore has no event-based equivalent: the probe must carry a temporal profile, and that profile becomes part of the measurement. A growing body of Computational Neuromorphic Imaging (CNI) work already exploits this, pairing engineered or modulated optics with event sensing, but each system adopts a particular excitation together with a particular reading of the event stream without the correspondence between the two being stated. We examine that correspondence directly within a analytical framework of an Linear Shift-Invariant (LSI) optical system with a specified Modulation Transfer Function (MTF), a first-order filter EVS pixel model, and three different temporal probes: a step function, a linear ramp and an exponential ramp. By analysing the inverse of the entire chain for different event-statistic, and comparing the results to the specified MTF, we identify the context where each probe is most relevant. We consider how photon-noise and cross-array threshold mismatch effects the analytical accuracy of the probe-inverse. Results show that the widely used step probe is highly susceptible to mismatch while resilient to photon shot-noise, while a linear rise probe and exponential rise probe retain their ability to infer signal levels even with high mismatch. We discuss the potential of dynamic-PSFs as components of a full forward operator from scene to events. In this, we use this analytical description to define dynamic-PSFs around EVS, and discuss the gaps toward a unified pixel model and a scene-composition framework required for CNI.

0.37

Ternary Visible Light Communication Using Event-Based Vision Sensors

eess.IV eess.SP ·2026-09-29 ·Sotaro Kuremoto, Junya Hara, Hiroshi Higashi, Yuichi Tanaka
similarity 0.58
Abstract · Submitted to ICASSP 2027. 5 pages, 5 figures, 1 table

This paper proposes a ternary visible light communication method using an event-based vision sensor (EVS) and a liquid crystal display (LCD). The throughput of optical camera communication (OCC) is limited by the frame rates of conventional frame-based cameras, and EVSs are expected to overcome this limitation because their pixels asynchronously trigger events with polarity in response to brightness changes, with a temporal resolution on the order of microseconds. However, existing event-based OCC relies on binary signaling that uses only the presence or absence of events and leaves the polarity unused. The proposed method maps the three brightness transitions (increase, decrease, and no change) to ternary symbols, thereby increasing the information carried per symbol. At the transmitter, the LCD displays a marker in which the transmitted data are encoded; at the receiver, the EVS captures the marker and records the brightness changes as events with positive and negative polarities. To suppress errors caused by the timing mismatch between display updates at the transmitter and demodulation at the receiver, we also propose a demodulation algorithm based on frame-transition detection. Experimental results demonstrate that the proposed method achieves a throughput of 336.5 kbps, approximately 150% higher than that of its binary counterpart.

0.28

Real-time Event-camera Stereo Visual Odometry via Keytime Gaussian Process Regression

cs.RO ·2026-10-01 ·Nikan Nobari, Jonathan D. Gammell
similarity 0.57
Abstract · Submitted to IEEE International Conference on Robotics and Automation (ICRA) 2027, 8 pages, 1 figures, 3 tables. The corresponding video can…

Event cameras have microsecond-level temporal resolution and high dynamic range which make them more resilient to motion blur and poor illumination than standard frame-based cameras. Event-camera visual odometry (VO) pipelines maximize these benefits when they process the asynchronous event stream at the native temporal resolution. Continuous-time Gaussian process (GP) regression and a white-noise-on-acceleration (WNOA) prior can handle asynchronous measurements but result in a prohibitively large estimation state when applied naively. This paper presents a continuous-time event-camera stereo VO pipeline that maintains the native measurement times of asynchronous events while also running in real time. It reduces the estimation states to keytimes while maintaining full temporal resolution by interpolating measurements to their exact timestamps with a physically founded WNOA prior. This decouples the state size from the dense number of measurements without discarding their asynchronous nature. The real-time continuous-time VO pipeline is evaluated on the MVSEC and DSEC datasets. It provides estimates in real time that are more accurate than ES-PTAM, a state-of-the-art discrete estimator, in all but one of the tested sequences. The pipeline respectively provides estimates at 22 Hz and 6 Hz on MVSEC and DSEC and RMS relative errors of 0.46 cm and 0.038 degrees across all valid sequences, which were 11 and 15 times better than ES-PTAM, respectively.

0.24