CS Paper Radar

Amortized Set Prediction for Inverse IFS Reconstruction from Density Maps

Yutaka Yamaguti

cs.CV ·submitted 2026-08-25 07:40 UTC · v2
benchmark

Abstract

Iterated Function Systems (IFS) generate self-similar fractals from a few contractive affine maps. The forward map from parameters to images is inexpensive and well understood, whereas the inverse problem is difficult and is typically solved by per-image optimization. We replace this loop with a single forward pass of a learned estimator that predicts the affine-map set directly from a visit-frequency density map, thereby amortizing the inverse problem. The design follows two constraints. First, density maps do not uniquely identify IFS parameters, so we evaluate by reconstruction rather than parameter recovery and handle unordered map sets by Hungarian matching. Training nevertheless matches the ground-truth parameters: reconstruction-only training gives a better one-shot output, but refinement from its predictions converges to poorer minima, whereas refinement from parameter-matched predictions keeps improving. Second, the fully known forward model lets us generate exact synthetic training pairs and supports image-only test-time refinement. On in-distribution tests, amortized initialization plus a few refinement steps gives a better quality-speed trade-off than equal-budget random-start optimization, and over 1000 steps it reaches high-quality reconstructions more often. It is robust to the sampling settings of the observation, and on an external benchmark outside its training prior it matches the mean accuracy of a recent hour-long optimizer within a minute. On real images (MNIST and Fashion-MNIST), it improves density metrics on average over a published per-image optimizer while being roughly 2 to 500 times faster per image.

Comment
35 pages, 16 figures. v2

Closest papers

Nearest neighbours by meaning across the whole index.

Livin' on a Prior: Likelihood Score Approximation for Inverse Problems

cs.LG ·2026-09-28 ·Rostislav Makarov, Tal Peer, Danilo de Oliveira, Timo Gerkmann
benchmarkhot direction similarity 0.63
Abstract

Generative models have found great success as data-driven methods of solving inverse problems. Two popular approaches work either by combining a pretrained generative prior with a known degradation model, or by training a conditional generative model directly from paired data. We target a setting that spans both regimes: unknown degradations can be learned from few paired examples, while known degradations can be learned from self-generated samples. We introduce Likelihood Score Approximation (LSA), a generative framework that keeps a pretrained unconditional model fixed and learns an observation-conditioned model that approximates the likelihood score from paired samples. Within a conditional stochastic-interpolant framework, LSA can be trained in either score or velocity coordinates, independently of the unconditional model's native parameterization, and supports both deterministic and stochastic sampling. We further show empirically that the prior model can be swapped post-training while keeping the same LSA model. Across speech and image inverse problems, LSA operates effectively even at roughly 0.01% of the full training dataset. On the ImageNet-256 benchmark it achieves competitive or better restoration quality than strong posterior-sampling baselines while requiring up to several orders of magnitude fewer network evaluations.

0.29

Retrieval-Augmented Initialization with Dual-Guided Inversion for Flow-Based Inverse Problem Solvers

cs.CV ·2026-10-30 ·Hansam Cho, Seoung Bum Kim
published similarity 0.62
Abstract · WACV 2027

Flow models have recently emerged as powerful generative priors for solving inverse problems. Despite their success, most existing flow-based inverse problem solvers initialize the reconstruction process with random Gaussian noise. However, recent studies in image generation have shown that the choice of initial noise can significantly influence generation outcomes. The role of initialization as a distinct design choice in flow-based inverse problem solving remains comparatively underexplored. In this work, we systematically evaluate noise initialization strategies while holding the downstream flow-based inverse solver fixed. We propose Retrieval-Augmented Initialization with Dual-Guided Inversion (RA-DGI), a method that constructs improved initial noise by leveraging retrieved proxy clean images and dual-guided inversion. Extensive experiments across multiple inverse problems demonstrate that the quality of the initial noise substantially affects reconstruction fidelity and perceptual performance. RA-DGI outperforms standard random initialization and achieves superior results compared to inference-time scaling strategies, while requiring less computational overhead. These findings highlight initialization as a critical yet previously overlooked factor in flow-based inverse problem solvers.

0.24

FIRM: Flow-based Imaging via Regularized Minimization

cs.CV ·2026-09-11 ·Shirin Shoushtari, Edward P. Chandler, Xiao Shi, Ulugbek S. Kamilov
similarity 0.61
Abstract

Flow matching methods for imaging inverse problems typically incorporate measurements through network conditioning or guidance during sampling. Neither approach explicitly applies the forward operator within the learned conditional velocity field. We develop a principled measurement-conditional velocity parameterization that does. For a linear interpolation path, we express the optimal velocity through the posterior mean $E[x_1|x_t, y]$ and show that this mean is the unique minimizer of a variational objective with an explicit data-consistency term. The velocity defined by this minimizer provably transports the source distribution to the measurement-conditioned posterior. This result leads to a forward operator-aware velocity field that is trained end-to-end and requires no separate guidance during sampling. Across five imaging tasks, our method achieves leading reconstruction quality with up to $50\times$ fewer network evaluations than competitive flow-based methods. Varying the number of sampling steps also controls the distortion-perception trade-off without retraining.

0.17

Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems

cs.LG math.NAstat.CO ·2026-10-01 ·Jan Blechschmidt, Oliver G. Ernst, Moritz Poguntke, Björn Sprungk
similarity 0.60
Abstract · 38 pages, 18 figures

Sampling from the posterior is the central task of computational Bayesian inverse problems. The standard workhorse in Bayesian inference---Markov chain Monte Carlo (MCMC)---is sequential, yields correlated samples, and must be rerun for each new observation. Conditional flow matching offers an amortized alternative: a transport map, trained once on joint samples of parameter and data, that yields independent approximate posterior samples for any observation at negligible online cost, without new likelihood evaluations. We give a careful, MCMC-literate assessment of flow matching for PDE-based inverse problems with function-valued parameters. Exploiting the flow's tractable density, we derive computable accuracy estimates for the learned posterior in total variation distance and Kullback--Leibler divergence and, moreover, propose a hybrid sampler that is made asymptotically exact by Metropolization. We validate the accuracy estimates and demonstrate amortization in several numerical examples, including electrical impedance tomography and a likelihood-free Lotka--Volterra model. Flow matching reproduces posterior means and credible bands closely at a fraction of the forward solves required by MCMC, but its accuracy degrades for concentrated posteriors and for observations that are atypical under the prior.

0.26

Training Calibrated Conditional Generative Models with a Sample-Only Coverage Loss

cs.LG stat.MEstat.ML ·2026-11-11 ·Benjamin Remy, Raphael Rossellini, Chihway Chang, Rebecca Willett
theory similarity 0.60
Abstract · 32 pages, 7 figures

Conditional generative models have become the tool of choice for sampling posteriors in scientific inverse problems, where we seek to sample images or signals conditioned on a set of indirect and noisy measurements. Yet they are often overconfident, which is a critical failure in scientific applications where accurate uncertainty quantification is essential. Existing regularizers fall short in complementary ways: differentiable coverage objectives require the model's density and do not identify the posterior, while strictly proper scoring rules such as the energy score need only samples but do not directly enforce coverage. We close this gap by introducing a differentiable loss function based on the expected coverage probability, estimated from samples alone. It applies to any conditional generative model, with no need for tractable densities or architectural changes. We prove that it is a strict divergence, vanishing if and only if the model equals the true posterior. On simulation-based inference benchmarks, it matches a density-based coverage loss without evaluating the density. On high-dimensional imaging inverse problems, it yields better-calibrated posteriors than scoring-rule regularizers, with little change in the reconstruction quality of point estimates.

0.22

FFAI: Flow Matching with Feedback Training and Adaptive Initialization for CT Reconstruction

cs.CV ·2026-10-29 ·Jigang Duan, Heran Wang, Genwei Ma, Xing Zhao
benchmark similarity 0.60
Abstract

Although low-dose, sparse-view, and limited-angle CT involve distinct degradation mechanisms, all three pose reconstruction challenges due to limited measurement information and can therefore be addressed within a unified framework. In iterative image restoration with flow matching, prediction errors may cause inference states to deviate from the prescribed training path, while a fixed initialization may be unsuitable for inputs with varying degradation severity. We propose a unified conditional flow matching method that conditions on the initial reconstruction and combines feedback-aware training with degradation-adaptive initialization. During training, a teacher network simulates inference updates to generate shifted states, from which a student network learns to recover high-quality reference images, improving its ability to handle prediction errors. During inference, the starting time is selected based on the noise level or extent of missing projection data, and the noise added to the initial state is adjusted accordingly to adapt the starting point to the input. Experiments on public benchmark datasets show that the proposed method achieves better quantitative results than the compared methods across all three reconstruction tasks, effectively suppressing noise and artifacts while preserving anatomical structures and details.

0.19

Making Every Step Count: Spatio-Temporal Information Allocation for Imaging Inverse Problems

cs.CV ·2026-08-12 ·Yi Cao, Xiangyong Cao, Pei Liu, Yong-Jin Liu, Deyu Meng
similarity 0.59
Abstract

Flow-based generative models have emerged as powerful image priors for training-free inverse problem solving, capturing coherent semantics and fine-grained structure. Despite these strengths, existing flow-based inverse solvers primarily focus on the design of individual updates, largely overlooking spatio-temporal information allocation under a fixed number of function evaluations (NFEs). Temporally, insufficient early exploration can trap the flow trajectory in an incorrect semantic basin, whereas excessive allocation of NFEs to early stages leaves little budget for late-stage refinement. Spatially, data consistency provides direct constraints only within observed regions, whereas the recovery of missing regions relies mainly on the generative prior. To address these two issues, we introduce two complementary and training-free components, i.e., Spectrum-Adaptive Scheduling (SAS) and Measurement-Prioritized Attention (MPA). For temporal allocation, SAS distributes the available NFEs over flow time according to the degradation spectrum and logSNR geometry, thus better balancing semantic exploration and detail refinement. For spatial propagation, MPA exploits data-prior conflicts to guide information toward weakly constrained regions, thereby enhancing semantic and structural fidelity. Extensive experiments on standard image inverse problems, e.g., super-resolution, motion deblurring, and inpainting, demonstrate that the proposed components can be integrated into existing flow-based inverse solvers in a plug-and-play manner without retraining or additional flow-model evaluations, and can also significantly improve the restoration quality of existing solvers.

0.04

Infer3D: Inverse Generative Modeling Enables Robust 3D Reconstruction

cs.CV cs.LGcs.RO ·2026-11-18 ·Amani Kiruga, Mark Hamilton, Joshua B. Tenenbaum, William T. Freeman, Vincent Sitzmann +2 more
code/data similarity 0.59
Abstract · 23 pages, 12 figures, 15 tables. Includes references and appendices. Project page: https://amanikiruga.github.io/infer3d-site/

Humans can understand the 3D structure of scenes even from highly unfamiliar or extreme viewpoints. Cognitive science suggests this generalization stems from a top-down generative inference strategy. In contrast, state-of-the-art 3D reconstruction neural networks typically rely on amortized, feedforward inference. While fast, this pattern-matching approach degrades when faced with out-of-distribution (OOD) scene conditions unseen during training. Inspired by the generative nature of human vision, we propose Infer3D, an inverse generative modeling framework that recovers the latent variables of a generative model to best match the monocular input. To prevent degenerate flat outputs, we couple the generative image prior with a frozen 3D lifting network that constrains the optimization space to a valid, multi-view-consistent 3D manifold. Across a variety of controlled synthetic and real-world tasks, Infer3D demonstrates superior OOD reconstruction accuracy compared to feedforward networks trained on equal or larger datasets. Furthermore, mirroring how human perception requires more time to process confusing stimuli, we introduce an adaptive inference strategy that expends more test-time compute on ambiguous OOD examples while efficiently routing in-distribution inputs. Project website: https://amanikiruga.github.io/infer3d-site/

0.34