CS Paper Radar

3D Pose Estimation in Projection Space via Pixel-level Visible Light Communication

Michikuni Eguchi, Takefumi Hiraki

cs.RO ·submitted 2026-11-04 08:07 UTC · v1

Abstract

Projector-based augmented reality enables natural interaction between users and devices by overlaying digital information onto the physical world through projected imagery. A key technical requirement of such systems is accurate positional alignment between the projector and the devices. Pixel-level Visible Light Communication (PVLC) achieves this alignment without external sensing infrastructure by embedding invisible spatial codes into each projected pixel through high-speed brightness modulation that preserves image quality. However, the information that PVLC transmits is inherently planar. PVLC has therefore been limited to devices moving on a flat surface and cannot estimate the 3D pose of devices in three-dimensional space, such as aerial robots. To extend PVLC-based pose estimation to three dimensions, we formulate it as a Perspective-n-Point (PnP) problem. The 2D pixel positions that four coplanar photosensors on the device receive via PVLC serve as the image points, and their known 3D layout provides the object points. Experiments show that the proposed system estimates 3D pose with a latency of approximately 8~ms and a 3D position RMSE of 19.2~mm after Kalman filtering. These results confirm the first realization of real-time 3D pose estimation coordinated with high-quality projected image content without external sensing infrastructure.

Comment
submitted to IEEE Access

Closest papers

Nearest neighbours by meaning across the whole index.

A Height-Constrained 2-Point Minimal Solver for Pose Estimation from Active LED Markers with Event Cameras

cs.CV cs.RO ·2026-08-10 ·Runze Yuan, Alexander Kappler, Jun Zhang, Kuangyi Chen, Fabio Morbidi +3 more
published similarity 0.67
Abstract · 8 pages, 6 figures, accepted by IEEE/RSJ International Conference on INTELLIGENT ROBOTS & SYSTEMS (IROS) 2026

In many autonomous applications requiring real-time localization, active marker-based systems are preferred due to their low latency and ease of deployment compared to computationally demanding feature-based methods. Event~\mbox{cameras} offer high temporal resolution and minimal delay and are commonly used with active LED markers for robust real-time localization. Existing methods typically rely on Perspective-n-Point (PnP) solvers for pose estimation. However, structured marker layouts can be challenging to deploy in space-constrained scenarios, while partial self-motion information (e.g., gravity direction and altitude) is readily available from onboard sensors. We derive a robust and accurate minimal solver that estimates camera pose from only two LED markers by incorporating known tilt angle and camera height measured by an onboard sensor, such as an IMU or an altimeter. The proposed formulation uniquely determines the camera pose through both a closed-form and a linear least-squares solution. We further analyze degenerate configurations and characterize the conditions under which height information does not contribute to rotation estimation. For evaluation, we developed an event-based active marker system to collect real-world data with ground truth from a motion capture system. Experiments on both synthetic and real data demonstrate improved accuracy over the state-of-the-art P2P solver and competitive performance relative to P3P.

0.33

PnP*: A certifiable solver for Perspective-n-Point

cs.CV cs.ROmath.OC ·2026-10-22 ·Bang-Shien Chen, Caitrin E. Eaton, David M. Rosen
theory similarity 0.63
Abstract · 28 pages, 3 figures

Estimating a camera pose from 3D-2D correspondences - Perspective-n-Point - is one of the most fundamental problems in computer vision. Existing methods either solve a minimal problem or replace the objective with an easier one. Surprisingly, no scalable certifiable solver for the actual Maximum Likelihood Estimation (MLE) exists. We propose PnP*, the first practical solver that guarantees efficient computation and returns a certifiable globally optimal solution. We prove that the underlying convex relaxation is exact under moderate noise, which to our knowledge is the first stability guarantee for any relaxation for the MLE of PnP. Furthermore, we reduce the relaxation by exploiting chordal sparsity pattern and quotient ring reduction such that the relaxation is tractable and scalable. The stability and scalability results are verified numerically, as PnP* returns a provably correct camera pose at over 10 Hz.

0.19

PATH: Continuous Target Sensing among Autonomous Cooperative Drones

cs.RO ·2026-09-11 ·Heegyeong Kim, Alice James, Avishkar Seth, Endrowednes Kuantama, Jane Williamson +2 more
similarity 0.63
Abstract

Continuous target sensing by uncrewed aerial vehicles (UAVs) is constrained by limited flight endurance, motivating the transfer of tracking responsibility between cooperating UAVs. Such a handoff requires the receiver to identify the same physical target currently tracked by the sender despite differences in viewpoint, scale, and target appearance. Existing approaches based on global target localization or appearance-based cross-view association are limited by positioning uncertainty or ambiguous visual features. This paper presents Perspective Alignment \& Tracking Handoff (\textbf{PATH}), a platform-agnostic, geometry-assisted sensing and verification framework for target handoff between two moving UAVs. The sender reconstructs the tracked target as a metric 3D point using RGB-D sensing, while the receiver estimates its relative pose from a fiducial observation and projects the transmitted target point into its own image as a spatial prior for target acquisition. The receiver-generated candidate is then returned to the sender and verified through a cross-view Mutual Agreement Handshake before tracking responsibility is transferred. Real-world UAV experiments show mean relative-position and target-position errors of 0.047~m and 0.030~m, respectively. Under visually ambiguous conditions, PATH achieves 96.0\% frame-level receiver-side target acquisition accuracy, with 2.0\% false-positive and 2.0\% false-negative rates. A sensor-error sensitivity analysis shows that relative-pose uncertainty is the dominant contributor to receiver-view projection error. The implementation operates at video rate with compact inter-UAV communication below 16~kB/s at 60~Hz, demonstrating the feasibility of lightweight geometry-assisted target handoff on resource-constrained UAV platforms.

0.38

DSM-Free UAV Visual Localization via Latent-Depth Variable Projection and Structure-Adaptive Coupling

cs.CV ·2026-10-10 ·Yibin Ye, Xichao Teng, Shuo Chen, Xiaokai Song, Yuli Sun +2 more
code/data similarity 0.61
Abstract · 15 pages, 8 figures

Visual localization enables unmanned aerial vehicles (UAVs) to maintain absolute positioning in GNSS-denied environments. A common approach matches UAV images with geo-tagged orthographic maps and estimates camera pose using Perspective-n-Point (PnP) with elevations from Digital Surface Models (DSMs). However, accurate and up-to-date DSMs are often unavailable in practice. Orthographic-perspective geometry (OPG) enables DSM-free localization in scenes with varying elevations, yet existing methods typically estimate pose indirectly through intermediate matrix estimation and decomposition. To address this limitation, we propose Latent-Depth Variable Projection (LDVP), which treats the unknown scales along perspective viewing rays as latent depths. By analytically eliminating these depths and solving for the horizontal camera center in closed form given the rotation, LDVP reduces nonlinear estimation to a rotation-only problem, enabling direct recovery of metric horizontal position without assuming a planar scene. To improve localization robustness across different scene structures, we further introduce Structure-Adaptive Coupling (SAC), which uses relative scene structure inferred from monocular depth estimation to select LDVP when the matched points are not dominated by a single plane, and Planar-PnP otherwise. Our local observability analysis reveals how scene elevation variation affects the geometric constraints of OPG, providing a basis for this adaptive strategy. Extensive experiments show that LDVP consistently outperforms existing OPG-based DSM-free methods, while SAC further improves robustness across diverse scenes and substantially narrows the gap to DSM-based localization. Code will be released at https://github.com/UAV-AVL/LDVP-SAC.

0.29

Aligning Misaligned Benchmarks for Single-View Novel View Synthesis

cs.CV cs.AI ·2026-10-28 ·Yang-Che Tseng, Erqun Dong, Jeong-gi Kwak, Gopal Sharma, Shakiba Kheradmand +3 more
similarity 0.61
Abstract

While single-view novel view synthesis methods have been dramatically enhanced recently, how to quantify their performance has yet to be standardized. In this work, we demonstrate that an important factor lies in how each method's own reference frame is aligned with the so-called ground-truth pose. Existing methods often rely on test-time optimization of target poses for this alignment, which inherently is ill-posed due to the complexity of the optimization landscape of camera poses vs render accuracy, leading to methods being evaluated at an arbitrary local optimum. Some recognize this limitation and thus rely on 3D reconstructions of entire scenes using training and test views to align poses, which may be infeasible. In this work, instead, we propose to use traditional 2D-to-2D and 2D-to-3D image matching techniques to align 3D reconstructions. We also evaluate only within co-visible regions, between different views, which is the only place where a `correct' ground truth can be defined. Our evaluations show that with proper coordinate alignment, novel-view synthesis methods that use explicit 3D Gaussians achieve the state of the art, despite being previously reported in the literature to be outperformed by pure image-based 2D methods. We further provide a simple baseline that directly utilizes an off-the-shelf video generator and a feed-forward geometry estimator. Our simple baseline is already comparable to state-of-the-art methods specifically trained for novel-view synthesis, without any task-specific training.

0.18

VAComp: View-Aware Full Projector Compensation

cs.CV ·2026-11-07 ·Kai Zhou, Qingyue Deng, Bingyao Huang
code/data similarity 0.60
Abstract · 14 pages, 13 figures

Full projector compensation adjusts a projector's input image to compensate for both geometric and photometric distortions introduced by the projection surface. Existing methods generally treat the viewing condition as a static camera viewpoint. When applied to a moving viewer, existing methods face two key challenges: (1) view-dependent methods require model retraining or recalibration to compensate for novel viewpoints; and (2) NeRF/GS-based view-agnostic methods use structure from motion (SfM) to calibrate the projector-camera systems (ProCams) with an unknown global scale, while view-aware tracking requires calibrating a camera oriented opposite to the projector, which introduces an additional calibration challenge. To address these challenges, we introduce VAComp, the first view-aware framework for full projector compensation for a moving viewer in a naked-eye setting. Using a projector, VAComp enables the viewer to perceive a virtual object that visually replaces the real object or to observe view-consistent modifications of the real object's appearance and material properties. To resolve the SfM scale ambiguity, we use metric calibration as a reference to recover the global scale; to calibrate the oppositely facing camera, we use two checkerboards, one facing the RGB camera and the other facing the depth camera. To balance compensation quality and runtime efficiency, we develop two non-iterative hybrid methods, VAComp-SD and VAComp-SI. Both combine 2D Gaussian-based view-conditioned geometric correction with learned photometric compensation, predicting each compensation image through a single forward pass without per-frame iterative optimization. Real-world experiments show that both methods achieve competitive compensation quality, with VAComp-SD enabling real-time updates and VAComp-SI operating at interactive rates. Project page: https://realkaizhou.github.io/VAComp/.

0.44

OV-PEAR: Observation-Verified Pose-Scale Estimation and Reconstruction for Unknown Industrial Manipulation Objects

cs.RO cs.AI ·2026-10-19 ·Guoqin Tang, Qingxuan Jia, Yichen Tan, Zihang Lv, Ruiquan Wang +2 more
similarity 0.59
Abstract

Fine industrial robotic manipulation requires 6D pose, scale, and an operation-ready 3D model. Existing industrial pose estimation relies on CAD models, object-specific training data, or stable workcells, limiting use with unseen objects; zero-shot methods suffer from single-view occlusion, generative bias, and scale uncertainty. We present OV-PEAR, an observation-verified framework for 3D reconstruction and pose-scale estimation of unknown industrial objects. Its three-stage object-centric pipeline: (1) segments RGB-D instances and generates object-level 3D hypotheses; (2) anchors them to the robot frame via appearance-aware view retrieval, explicit scale search, and visibility-sensitive registration; (3) back-projects multi-view RGB-D evidence into the object-canonical frame to verify voxel occupancy, suppress hallucinations, and produce reusable trusted object models. On BOP-Industrial, OV-PEAR achieves 0.657 average AR and improves pose and voxel-level geometric consistency in real scenes; real-robot connector assembly reaches 99.67% grasp and 84.00% assembly success. The model library and pose-scale log enable short-term object-level state maintenance at 0.02 ms/frame.

0.12

3DROID: A Renderable 3D Gaussian Dataset with Measured Per-Scene Reliability

cs.CV cs.RO ·2026-10-01 ·Wonguen Cho, Junhoo Lee, Nojun Kwak
datasetcode/data similarity 0.59
Abstract · 12 pages, 3 figures

Robot manipulation models primarily reason from 2D observations while acting in the 3D physical world. To bridge this gap, recent work has augmented robot data with geometric priors such as depth, point clouds, and 3D trajectories, while renderable 3D Gaussian representations provide another promising form of 3D supervision. However, 3DGS representation is designed mainly for photometric fidelity and may not preserve real-world metric scale, particularly when the supplied camera extrinsics are unreliable. We study the effect of extrinsic reliability and pose conditioning on feed-forward 3DGS, and propose a calibration-aware pipeline that anchors reconstructed scenes to the robot's metric workspace. Our experiments show that pose conditioning improves novel-view fidelity, while its geometric benefit depends on the reliability of the injected extrinsics. Using this pipeline, we present a renderable, metric-pose-anchored dataset with scene-level reliability information for robot manipulation research. Our dataset is available at https://huggingface.co/datasets/wonguen/3DROID

0.30