CS Paper Radar

A second-order theory of texture for depth from focus

Sreekar Ranganathan, Ioannis Gkioulekas

cs.CV ·submitted 2026-08-11 02:56 UTC · v2
published

Abstract

We present a theory of textured appearance of optically rough surfaces based on wave optics, emphasizing the role of texture for passive depth from focus. Our theory shows that even surfaces that traditional computer vision would consider textureless can produce textured appearance, due to subjective speckle from surface microgeometry. We analyze the properties of this second-order texture, and show that we can enhance its contrast under natural ambient lighting by simply using a narrowband spectral filter. Doing so results in dramatic improvements in passive depth reconstruction of seemingly textureless scenes, as we demonstrate through extensive theory, simulations, and real-world experiments.

Comment
ECCV 2026, project website: https://imaging.cs.cmu.edu/second-order-texture/

Closest papers

Nearest neighbours by meaning across the whole index.

Disparity Estimation of Planar Reflective Surfaces Using Specular Reflections From a Single Light Source

eess.IV ·2026-09-21 ·Katja Kossira, Frank Sippel, Jürgen Seiler, André Kaup
top venue similarity 0.54
Abstract

Multi-camera imaging and camera arrays have become ubiquitous in many applications, such as autonomous driving, robot control, or virtual reality, and accurate disparity maps of objects and their environment are essential for reliable operation. Despite recent advances in neural networks, correctly estimating the disparity of flat and textureless objects remains challenging. In particular, we consider a scenario defined by flat, textureless surfaces illuminated by a single fixed light source, resulting in one dominant specular reflection visible on the object surface. Under these conditions, reliable geometric and photometric cues are missing, and the specular reflection often causes mispredictions, especially when using conventional methods that rely on texture information to match corresponding pixels. To address this issue, the novel Specular Reflection Disparity Estimation SRDE algorithm is introduced, which is specifically designed for the constrained scenario of planar, textureless objects and single-source illumination. Unlike conventional stereo matching methods, SRDE ignores texture and instead leverages the geometric properties of specular reflections by incorporating the position information of the reflective region, the light source, and the camera setup. We show that SRDE outperforms existing methods by a notable margin, achieving more than a 52% improvement in End Point Error on synthetic images. Further tests demonstrate superior performance on real-world data. Furthermore, we integrate SRDE into existing neural disparity estimation pipelines by selectively replacing predictions in specular regions without modifying the backbone model. This hybrid strategy enables additional performance gains without requiring network retraining.

0.68

MicroSDF: Microfacet-Driven Hybrid Neural SDFs for Mixed-Reflectance Surface Reconstruction

cs.CV ·2026-10-14 ·Lejia Ye, Yuhua Xu, Yulan Guo, Lian Xu, Xiaohu Zhang
published similarity 0.53
Abstract · Published in IEEE Transactions on Image Processing, vol. 35, pp. 4106-4121, Apr. 2026. DOI: 10.1109/TIP.2026.3678096

Accurate 3D reconstruction in real-world environments remains a significant challenge due to the coexistence of reflective and non-reflective surfaces, which pose distinct modeling demands. Existing methods often treat these surface types separately, limiting their generalizability and physical plausibility. To bridge this gap, we propose MicroSDF, a novel neural implicit framework that facilitates geometry and reflectance modeling through microfacet theory. Our approach incorporates three core innovations: (i) a microfacet-guided geometry model that extracts multi-scale surface normals (macroscopic and microfacet) from a signed distance field (SDF), regularized by a proposed microfacet normal consistency loss to enforce physically plausible surface orientations; (ii) an enhanced dual-branch color model, where the specular branch leverages the microfacet normals to model high-frequency reflectance, and the vanilla branch, unlike prior works, uses reflection direction (instead of viewing direction) to better model diffuse and low-frequency specular components; and (iii) a detection-guided color blending strategy that adaptively fuses the color outputs based on reflection priors, providing more physically intuitive blending than implicitly learned blending weights. Combined with a tailored multi-stage optimization scheme, the proposed MicroSDF achieves robust and high-fidelity reconstruction across reflective and non-reflective surfaces. Extensive experiments on DTU, Shiny Blender, Ref-NeRF, and DeepVoxels datasets demonstrate state-of-the-art performance, establishing a new direction for physically grounded neural reconstruction.

0.46

A Lightweight Differentiable Method for Real-Time Skin Rendering and Digitization

cs.CV cs.GR ·2026-11-10 ·Mario Marquez-Balduque, Carlos Rodriguez-Pardo, Jorge Lopez-Moreno
publishedhot direction similarity 0.50
Abstract · 20 pages, 25 figures; includes 6 pages of supplementary material. Accepted to Computers and Graphics

Inverse rendering for skin typically relies on offline path tracing to estimate physical properties, which fail to map directly to the approximate shaders utilized in interactive applications, such as game engines. We bridge this gap by directly estimating subsurface scattering parameters tailored for real-time rendering pipelines. We introduce a differentiable Screen-Space Subsurface Scattering (SSSS) framework that enables the joint optimization of spatially varying base textures and scattering distances. To account for local variations in skin thickness, we estimate these parameters across distinct facial regions, generating a blended profile map ready for production runtime engines. Our pipeline utilizes a low-cost, single-light, single-view capture setup designed for accessible and practical appearance acquisition. Optimization is driven by a masked loss function targeting regions where subsurface scattering effects are dominant. We validate our approach on a real-world dataset of $45$ individuals across the Monk Skin Tone Scale, ensuring demographic generalization. Furthermore, we complement this evaluation with synthetic data, a multi-light setup validating our single-light assumption, and a tilted-view configuration confirming angular consistency. Results demonstrate that our method effectively recovers scattering parameters from lightweight inputs, yielding realistic skin appearance directly compatible with standard real-time shaders.

0.38

Sensitivity as an Arbitrary Output Variable for Differentiable Rendering

cs.GR ·2026-10-07 ·Linas Beresna, Eugene Fiume
top venue similarity 0.50
Abstract · 4 pages. SIGGRAPH Asia 2026 Technical Communications

Differentiable renderers expose the derivative of any scalar objective with respect to every scene parameter, yet unlike the primal image, which decades of arbitrary output variables (AOVs) have taught us to decompose, inspect, and composite, these derivatives have no established representation for human inspection. We introduce the sensitivity AOV, a render output carrying the sensitivity of an objective to the scene parameters that influence it. A single reverse-mode pass populates a sensitivity buffer over the scene's parameter hierarchy, from which many views are read rather than re-differentiated, in direct analogy to deferred shading: image-space sensitivity at object and parameter-type granularity, projections onto a freely navigable scene from any inspection viewpoint, and, for spatially varying parameters, per-texel fields carried to the surface through texture coordinates. We separate the fixed camera that defines the objective from the free camera used to inspect the result, and position reverse-mode attribution against its forward-mode dual. Our aim is not a single algorithm but a scaffolding that establishes derivative outputs as first-class render products alongside the primal image.

0.51

Deformable Texture Maps: Compiling Dynamic Microgeometry into Neural Textures

cs.GR ·2026-11-10 ·Antonio Arian Silaghi, Zhen Chen, Danny Kaufman, Élie Michel, Jean-Marc Thiery +2 more
similarity 0.50
Abstract · 12 pages, 12 figures

Deformable material surfaces, such as cloth, owe their appearance to fine-scale geometries that restructure under deformation. Even when coarse simulation models efficiently recover this microstructure, explicitly instantiating and rendering it remains expensive. We introduce Deformable Texture Maps (DTM), a representation that compiles deformation-driven microgeometry into a continuous and compact neural texture. Our model maps texture coordinates and local deformations to intrinsic maps describing opacity, surface normals, depth, and material identity. This captures the occlusion and visibility changes generated by deformation. At runtime, these maps provide a common texture-space representation of deformation-driven fine-scale structure that can be evaluated directly on coarse deforming surfaces. Since our representation encodes geometric attributes rather than radiance, lighting and material response remain decoupled from the learned model. This allows us to apply relighting, as well as shading and material editing without retraining. We further introduce a persistent texture-space rendering strategy that caches these intrinsic maps and selectively updates them as deformation changes. This effectively decouples neural evaluation from frame rendering and enables their reuse across frames and conventional rendering effects. We demonstrate our approach on yarn-based textiles, where non-linear material stretch and shear induce changes in yarn geometry, spacing and occlusion. We build our model by, first, sweeping tileable yarn patches through deformation space, then, rendering the detailed geometry into a canonical texture-space, and finally, compressing the resulting maps into a single neural representation. Our resulting model can be evaluated directly on arbitrary coarse garment meshes and integrated with conventional shading pipelines, keeping appearance authorable at render time.

0.20

Gaussian Splatting with Reflections Guided by What is Seen

cs.CV ·2026-10-09 ·Yiming Liang, Tianyu Xiao, Hiroshi Ishikawa
published similarity 0.50
Abstract · Accepted to IEEE ICIP 2026

While novel view synthesis methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have shown impressive performance on common scenes, they often struggle with capturing view-dependent specular effects, particularly near-field reflections. Existing methods utilize environment maps, which only represent lights from infinity, or environment Gaussians, which rely solely on indirect supervision from reflections, are insufficient for accurately modeling such effects. Here, we propose a framework based on 2D Gaussian Splatting (2DGS) that leverages directly observable environment information in input images. Our approach also integrates a hybrid reflection appearance representation and view-dependent specular modeling. Experiments on public benchmarks and a newly curated near-field reflection dataset demonstrate that our method achieves superior rendering quality and geometric accuracy.

0.17

Material and Defocus Cues for Occluded Object Localization in Single Hyperspectral Images: An Exploratory Study

cs.CV ·2026-10-15 ·Ali Zia, Jun Zhou
similarity 0.49
Abstract

Partial occlusion makes object detection difficult when visible regions have weak texture or similar appearance. Hyperspectral imaging provides material-dependent spectra and wavelength-dependent optical blur, which offer complementary evidence for separating objects. We investigate a pipeline that combines nonnegative-matrix-factorization-based material grouping, divergence-based optical cues, and graph Laplacian propagation. The propagated scores support region extraction and bounding-box localization without requiring a separate depth sensor at inference. A collection of 30 indoor and six outdoor hyperspectral scenes acquired using two camera systems provides qualitative examples spanning different materials, depth arrangements, and partial occlusions. The examples illustrate how material information can support separation when optical cues are weak, and how optical cues can distinguish some surfaces with similar material properties. Objects sharing both material and depth remain challenging. This exploratory study establishes the sensing rationale and mathematical formulation, while identifying the quantitative evaluation and implementation details needed to assess detection performance systematically.

0.23

Point-Based 3D Reconstruction from Sparse Views under Known Illumination

cs.CV ·2026-08-20 ·Magnus Kaufmann Gjerde, Joakim Bruslund Haurum, Jeppe Revall Frisvad, Markus Worchel, J. Andreas Bærentzen +1 more
similarity 0.49
Abstract

Sparse view 3D reconstruction is commonly addressed with neural implicit surfaces or dense point-based representations such as Gaussian splatting. Surface-aware splatting methods improve extracted geometry through oriented primitives and regularization, while RadiosityGS incorporates differentiable light transport through a radiosity inspired finite-element surfel formulation. We propose a differentiable point rendering method based on opacity-bearing beta surfels. An opacity explicit adjoint light transport formulation provides gradients for surfel geometry and appearance parameters, allowing physically based light transport to constrain reconstruction. Across five synthetic objects reconstructed from ten posed views, our method achieves the lowest mean symmetric Chamfer distance among the evaluated baselines and reduces mean Chamfer distance by 28.5% relative to the strongest point-based baseline while using only 267 surfels on average, approximately ~161 fewer primitives. Directional Chamfer results further show improved accuracy and competitive completion relative to related point-based methods. These results show that, in the controlled direct illumination setting, compact beta surfels combined with transport-based optimization can recover surfaces without relying on the tens to hundreds of thousands of primitives used by the evaluated baselines.

0.24