CS Paper Radar

LAMP: 3D Source-Aware Lighting Estimation at Multiple Points

Yanqiao Wang, Jin Sun

cs.CV ·submitted 2026-10-13 22:36 UTC · v1

Abstract

Existing illumination estimation methods primarily recover lighting distributions for rendering, but typically leave the contributions of individual physical light sources at specific scene locations implicit. We address this gap by leveraging the scene understanding capabilities of vision-language models (VLMs) to connect local illumination with its physical sources. We introduce LAMP, a framework for point-conditioned, source-aware illumination reasoning from a single RGB image with marked query points. LAMP constructs a shared inventory of active light sources, ranks contributing sources for each query, and predicts their 3D query-to-source directions. We supervise these relationships using procedurally controlled indoor scenes, reusing independently rendered source responses to derive contribution targets across lighting configurations and obtaining exact directions from source and query positions. On the Infinigen test set, fine-tuning Qwen3-VL-8B increases dominant-source 3D PCK@0.5m from 0% to 70.6%, with 94.5% of directions within 12.5 degrees of ground truth among correctly localized sources. Combined with a lightweight network predicting dense RGB irradiance, LAMP achieves probe-rendering quality comparable to prior lighting-estimation methods while additionally localizing individual physical sources in 3D and explicitly attributing local illumination to them.

Closest papers

Nearest neighbours by meaning across the whole index.

Shedding Light: A Benchmark for Evaluating Lighting Understanding in Generative Image Models

cs.CV ·2026-09-09 ·Justine Giroux, Jack Oliver Hilliard, Yannick Hold-Geoffroy, Javier Vazquez-Corral, Jean-François Lalonde
top venuebenchmarkcode/data similarity 0.72
Abstract · Accepted to ACM Transactions on Graphics (SIGGRAPH Asia 2026), vol. 45, no. 6, article 227, December 2026. 25 pages. Project page: https://l…

Accurate modelling of illumination is central to realistic image synthesis and scene understanding. Yet, there is little exploration into whether image generative models are good at this task or whether physical plausibility remains a key challenge for them. Clearly, significant progress has been made in realistic image synthesis, but do models truly understand lighting in a physically accurate manner? To answer this question, this work proposes a benchmark to assess the lighting understanding and harmonisation capabilities of generative models. Our key insight is that evaluating lighting understanding for such models only requires testing how well they insert novel objects into real photographs whilst maintaining consistent illumination. To do so, we use a multi-illumination dataset with images containing simple objects serving as ``light probes'', and prompt models to inpaint the same object onto the original image, then compare the generated results against the ground-truth light probes. We then estimate the lighting direction, colour and radiance distribution from the inpainted probes, providing a quantitative measure of illumination accuracy and photometric realism. Our work establishes a scalable evaluation protocol to systematically assess how well generative models capture and reproduce real-world lighting, offering a foundation for benchmarking the photometric accuracy of any future models. All code and data are available at https://lvsn.github.io/SheddingLight/ .

0.70

A General Pipeline for Dense Illuminant Estimation via Physically Based Synthetic Data

cs.CV ·2026-10-05 ·Luca Cogo, Gianmarco Corti, Simone Bianco, Raimondo Schettini
published similarity 0.71
Abstract · Accepted at the 34th Color and Imaging Conference (CIC 2026), hosted by the Society for Imaging Science and Technology (IS&T)

Illuminant estimation is a fundamental problem in computational photography, as it enables the correction of color shifts induced by varying lighting conditions. While learning-based methods have demonstrated strong performance, their progress is hindered by the limited availability of large-scale datasets with accurate illuminant ground-truth. In this work, we propose a general and reusable pipeline to derive dense illuminant chromaticity maps from physically based 3D-rendered scenes. By repurposing an existing 3D scene collection, our approach enables the systematic generation of pixel-wise illuminant annotations under controlled lighting conditions, effectively lowering the barrier to data acquisition for learning-based illuminant estimation. Using this pipeline, we generate a large-scale synthetic set of 74,321 images, which we employ for pre-training both single- and multi-illuminant estimation models. Extensive experiments with state-of-the-art architectures show that synthetic pre-training consistently improves performance, with gains of up to 28% for single-illuminant estimation and up to 57% for multi-illuminant estimation, particularly in data-scarce regimes. These findings demonstrate that synthetic data generation pipelines offer an effective and scalable solution for the pre-training of illuminant estimation methods.

0.17

LUMEN: Geometry-Guided Diffusion Model for Spatially-Varying HDR Illumination Estimation

cs.CV ·2026-11-01 ·Hongze Chen, Fangyu Lin, Runze Zhang, Lingting Zhu, Weikai Chen +6 more
similarity 0.70
Abstract

Estimating scene illumination from images is challenging as lighting is inherently spatially varying, depending on geometry, occlusion, and light sources. Existing methods incorporate geometric cues by deriving local environment maps or probe-based representations, but largely rely on regression or parametric models, resulting in over-smoothed lighting and limited realism. We revisit spatially-varying illumination estimation from a generative perspective and introduce LUMEN, a geometry-conditioned diffusion framework for synthesizing high-dynamic-range (HDR) environment maps at arbitrary spatial locations from single or multi-view images. Specifically, we reconstruct a coarse 3D scene from input images and render partial environment maps at arbitrary locations to encode visibility and spatial structure. These signals are fused with multi-view image features and fed into a diffusion transformer to generate full HDR environment maps. To train the model, we build a synthetic dataset of 700 indoor scenes with dense spatial supervision, including multi-view RGB images, depth, and HDR illumination at multiple sampled locations. This dataset enables the model to learn spatially consistent lighting distributions across diverse scenes. Extensive experiments demonstrate that LUMEN significantly improves visual fidelity and spatial coherence compared to prior approaches.

0.14

When Depth Lies: Benchmarking Vision-Language Models on Mirror-Induced RGB-D Ambiguity

cs.CV ·2026-10-30 ·Hao Yin, Tianchen Guo, Heming Du, Yan Ke, Yanbin Liu +1 more
top venuebenchmarkdatasetcode/data similarity 0.68
Abstract · NeurIPS 2026 E&D Track accepted

Vision-language models (VLMs) have made significant progress in spatial reasoning, particularly by integrating RGB images with depth data, which is often assumed to faithfully reflect physical distances. However, mirrors violate this assumption: they distort depth readings while reproducing object appearances, making reflections difficult to distinguish from real objects. As existing benchmarks rarely consider such reflective scenarios, VLM performance in mirror scenes remains largely unexplored. In this work, we introduce a mirror-centric RGB-D dataset from reflective scenes to evaluate the spatial reasoning capabilities of state-of-the-art VLMs in these scenarios. Specifically, we collect 1,011 RGB-D images from 525 real-world scenes, spanning diverse indoor and outdoor environments, six reflective surface types, and both daytime and nighttime conditions. All images are annotated with reflective surface region masks and mask annotations for both directly observed and mirror-reflected objects. Then, we curate a benchmark named MIRA-Bench (Mirror-Induced Reasoning Ambiguity Benchmark), comprising 14,140 QA pairs across eight sub-tasks organized into three cognitive levels: Reflection-aware Perception, Spatial Reasoning, and Scene-level Decision-making. We evaluate state-of-the-art VLMs on MIRA-Bench and find consistent failures across tasks: models misclassify whether objects are directly observed or reflected, fail to match reflected instances to their real-world counterparts, and produce erroneous spatial inferences. Remarkably, providing depth data actually degrades reasoning performance, implying that VLMs cannot identify incorrect depth in mirror scenes. The code and data are available at https://github.com/EvaYin-cloud/MIRA-Bench

0.40

SpatialCast: Reading the 3D World via Cast Shadows

cs.CV ·2026-11-17 ·Enhan Li, Jinghao Zhou, Jiaze Zhang, Hongyang Du
benchmark similarity 0.67
Abstract · 10 pages (main paper), 7 pages (supplementary material), 5 figures and 6 tables in the main paper

As multimodal large language models (MLLMs) advance toward understanding and interacting with the physical world, spatial reasoning becomes essential for understanding scene structure and anticipating the effects of physical changes. Existing spatial benchmarks have made substantial progress in evaluating spatial relations, memory, and viewpoint reasoning, but less systematically assess how models connect physical scene conditions with visual observations. We introduce SpatialCast, a benchmark that uses cast shadows to evaluate this ability. Cast shadows provide observable evidence of a 3D scene. Their image appearance depends on object geometry and pose, illumination, and viewpoint. We examine this relationship through three complementary tasks that assess scene-projection consistency, predict intervention-induced changes, and infer the physical changes behind visual transitions. The benchmark contains 21,000 synthetic questions with separate scene and geometry test partitions, complemented by 200 SpatialCast-Real questions from 20 photographed scenes. Evaluations show that closed-source MLLMs outperform general-purpose open-weight MLLMs and MLLMs designed for spatial reasoning, which remain near chance on the synthetic benchmark, while even strong closed-source models struggle with shadow-length and light-height judgments. We also use SpatialCast training data for GRPO post-training, improving open-weight MLLM performance on the benchmark. Further evaluations show improvements on several external spatial benchmarks, suggesting transfer of the learned capabilities. These experiments support SpatialCast as a controlled resource for diagnosing spatial reasoning capabilities and studying their learning and generalization.

0.18

Where Should the Lamp Go? Ambient-Aware Lamp Placement for Portraits

cs.CV cs.GR ·2026-11-16 ·Yuyan Chen
similarity 0.67
Abstract · 15 pages, 12 figures

Portraits shape first impressions, and lighting is an important factor. Prior work, such as lighting estimation and portrait relighting, edits lighting after capture but does not guide where to place a light before the shot. This leaves an open question: where should a light go to best light a face? Therefore, we propose LightMatch, a preference-grounded inverse lighting framework that disentangles the controllable light from ambient illumination and optimizes its placement. We first learn a Bradley-Terry quality model from pairwise human judgments. A perception head then regresses spherical-harmonic lighting from frozen self-supervised features via cross-attention. Finally, a placement solver isolates the light via spherical-harmonic linearity and optimizes its reachable position. On held-out environments, LightMatch attains 87.8% of the oracle improvement, outperforming the conventional solver that ignores ambient light by a relative 28.4% and the same pipeline with the widely used DPR estimator by a relative 70.5%. It reaches 73.2% on real OLAT faces of unseen identities and, on our real captures, cuts lamp-direction error by 37.7%.

0.25

LightBridge: Feed-Forward Generative Relighting for 3D Gaussian Splatting

cs.GR eess.IV ·2026-09-02 ·Hezhi Cao, Panhao Cheng, huangsheng du, Qibiao Li, Youcheng Cai +1 more
code/data similarity 0.66
Abstract · 14pages, 8figures

3D Gaussian Splatting (3DGS) achieves high-quality, real-time novel view synthesis, but the resulting assets have baked-in illumination and cannot be easily relit. Inverse rendering methods optimize simplified reflectance and illumination models for each scene, limiting efficiency and relighting quality. Recent generative approaches leverage large diffusion models for realistic lighting edits, but applying them to 3DGS typically requires an additional per-scene optimization stage to bake the edited appearance into the representation. We present LightBridge, a feed-forward generative framework for controllable relighting of complete 3DGS assets in a single pass. To enable feed-forward training, we construct a large-scale Multi-Illumination Relighting Dataset with paired source and target observations of the same scenes. Latent Bridge Relighting Diffusion models relighting as source-to-target transport in latent space, enabling one-step extraction of 2D visual tokens without iterative diffusion sampling. A Gaussian Propagation Transformer uses a point transformer with sparse image-to-point self-attention followed by point-to-image cross-attention to efficiently propagate these cues across the complete 3DGS, while avoiding full attention over all image and Gaussian tokens. Experiments validate these designs, demonstrating competitive relighting quality and efficient single-pass prediction of complete relit 3DGS assets without scene-specific optimization. The code and dataset will be made publicly available upon acceptance.

0.25

RoomLight: A 2.5D Illumination Prior for Indoor Environments

cs.CV ·2026-09-23 ·Andreea Ardelean, Bernhard Egger
code/data similarity 0.66
Abstract

Ill-posed inverse problems require priors to constrain the solution space toward plausible outcomes. In inverse rendering, learned priors modeling the distribution of natural illumination improve the recovery of scene properties. However, existing models rely on the distant-illumination assumption, representing lighting as a far-field environment map. This limits their applicability to indoor scenes, where illumination is highly spatially varying due to finite-distance emitters, visibility changes, and parallax, all of which are poorly approximated by a single environment map. To address this, we introduce a spatially-aware illumination prior trained on real-world indoor panoramas and their estimated depth. Our variational autoencoder model learns a compact, optimizable latent space that decodes into HDR radiance and depth, parameterizing an area light emitter for direct integration into standard differentiable rendering pipelines. This design bridges the plausibility guarantees of a learned prior with the gradient flow required for downstream optimization. Crucially, by jointly modeling radiance and depth, our prior captures the spatial structure of indoor illumination, instead of treating the light sources as infinitely distant. We demonstrate that this formulation enables spatially-varying illumination modeling and achieves higher-fidelity recovery of indoor lighting compared to existing approaches. Project page: https://andreead-a.github.io/RoomLight

0.24