CS Paper Radar

Multifidelity Operator Inference for Non-intrusive Model Reduction under Data Scarcity

Vivian Zhang, Tomoki Koike, Atticus Rex, Elizabeth Qian

cs.LG math.NAstat.CO ·submitted 2026-11-10 06:09 UTC · v1

Abstract

Many-query analyses, in which a computational model is repeatedly evaluated, play an important role in uncertainty quantification and optimization for engineering design. However, many-query analyses can be prohibitively expensive when considering high-fidelity numerical models, especially those governed by partial differential equations (PDEs). Surrogate modeling replaces costly simulations with cheap approximations, enabling practical uncertainty quantification and design workflows. Machine learning (ML) approaches to surrogate modeling have gained significant traction in the development of surrogate models due to their non-intrusive nature; they do not require access to the underlying simulation code. The Operator Inference (OpInf) approach, in particular, fits reduced operators to state and corresponding time derivative data. However, high-fidelity training data is scarce due to limited computational budgets, and OpInf models trained on such scarce data often exhibit high variance, a critical liability in high-risk engineering design. We propose Multifidelity Operator Inference (MF-OpInf), which reduces this variance by supplementing scarce high-fidelity snapshots with abundant low-fidelity data drawn from coarser discretizations of the same problem. MF-OpInf handles data across varying fidelity levels by utilizing a multifidelity linear regression framework to estimate the reduced operators. Numerical experiments on model PDE problems demonstrate that MF-OpInf predicts PDE solution fields with higher accuracy compared to the high-fidelity-only OpInf with the same computational budget.

Comment
21 pages, AIAA SciTech Forum 2027

Closest papers

Nearest neighbours by meaning across the whole index.

Multifidelity Operator Inference for Dynamical Systems

cs.CE stat.MLcs.LGmath.NA ·2026-11-12 ·Vignesh Sella, Anirban Chaudhuri, Karen Willcox
similarity 0.70
Abstract · 19 pages, AIAA SciTech Forum 2027 (Uncertainty Quantification Conference)

This paper presents a multifidelity (MF) formulation of Operator Inference (OpInf) for learning non-intrusive, physics-based reduced-order models (ROMs) of nonlinear dynamical systems. OpInf learns reduced operators from projections of snapshot data by solving a least squares regression problem. However, generating high-fidelity snapshots can be computationally expensive, limiting parameter coverage and the temporal extent of training trajectories for many aerospace applications, which can lead to inaccurate or unstable ROM predictions. This work proposes MF-OpInf, which combines limited high-fidelity snapshots with more plentiful low-fidelity snapshots through a control-variates-inspired MF least squares formulation to learn a single set of reduced operators across multiple trajectories. Low-fidelity trajectories provide additional parameter coverage and extend beyond the high-fidelity training time horizon. The low-fidelity snapshots are also used to construct an improved reduced basis through weighted proper orthogonal decomposition. We demonstrate MF-OpInf on a nonlinear reaction-diffusion problem with parametrized sinusoidal forcing and a two-dimensional incompressible flow past a cylinder with a parametrized inlet pulse. In these experiments, under an equal training simulation budget, MF-OpInf reduces the mean relative prediction error at unseen parameters by up to $87\%$ for the reaction-diffusion problem and $28\%$ for the cylinder flow problem over the test time regime, which extends beyond the high-fidelity training time horizon. These results demonstrate that MF-OpInf can improve temporal extrapolation and prediction at unseen parameters without increasing the training simulation budget.

0.15

On the Brittleness of Maximum Likelihood Estimation for Gaussian Process Hyperparameter Optimization

cs.LG stat.MLstat.ME ·2026-08-13 ·Tyler R. Johnson, Kian Ben-Jacob, Christopher P. Muller, Ramin Bostanabad
code/data similarity 0.63
Abstract · 26 pages, 10 figures, 2 tables

Machine learning (ML) has become an indispensable part of modern engineering design workflows. A crucial step in training an ML model is the selection of the loss function which can be systematically formulated via various techniques such as maximum likelihood estimation (MLE) and cross-validation . While MLE is one of the most popular, effective, and intuitive mechanisms for training ML models, it is brittle: if the assumptions underpinning it are not met, the trained ML model may generalize poorly. This brittleness affects even Gaussian processes (GPs) which are widely used in engineering design and are often (incorrectly) presumed to be very robust to overfitting. In this paper, we fundamentally evaluate the brittleness of MLE in the context of training GPs for probabilistic regression or classification tasks. We compare theoretically grounded metrics against MLE and propose practical solutions. Our extensive studies demonstrate the effectiveness of our solutions in downstream design tasks such as Bayesian optimization and provide a blueprint for practitioners to build accurate and robust GPs that can even outperform tabular foundation models in terms of prediction accuracy, uncertainty quantification, and inference cost. Our contributions are publicly available via GitHub at https://github.com/Bostanabad-Research-Group/GP-vs-TabPFN-vs-GPyTorch.

0.30

Compute-optimal data scaling for neural surrogates via multi-fidelity training

cs.LG cs.AI ·2026-10-12 ·Paul Setinek, Fabian Paischer, Nils Thuerey, Johannes Brandstetter, Felix Koehler
top venuecode/data similarity 0.63
Abstract · Accepted at NeurIPS 2026. Project page: https://psetinek.github.io/multi-fidelity-neural-surrogates/ Code: https://github.com/psetinek/multi…

Neural surrogates for Partial Differential Equations (PDEs) are usually trained on synthetic data from numerical simulators. As models grow and scaling laws demand larger datasets, the cost of data generation often outweighs the training cost and becomes the bottleneck in the scaling quest. Unlike data in other domains, however, each PDE sample has a tunable cost that depends on its numerical fidelity, with the resulting cost-accuracy trade-off governed by known convergence rates. We exploit this property by formalizing neural surrogate training as a function of both data quantity and fidelity under a fixed data generation compute budget, treating fidelity as a fine-grained discrete spectrum rather than a binary distinction. Building on this, we show that multi-fidelity training enables Pareto-optimal error scaling against data generation compute. By using simulations at lower fidelities as an additional training signal, training on the optimal fidelity mixture matches the accuracy of high-fidelity-only training, with up to $6.3\times$ less data generation compute across our settings. We validate these findings across six PDE problems spanning structured grids and unstructured meshes, two distinct error sources (discretization and iterative-solver error), and four surrogate architectures, totaling over 7,000 training runs.

0.59

Enabling risk and reliability analysis in next-generation computational tools via multifidelity methods

cs.CE math.OCstat.CO ·2026-11-03 ·Anirban Chaudhuri, Karen Willcox
similarity 0.61
Abstract · Review article

Risk and reliability analysis are fundamental to the design, assessment, and operation of safety-critical complex systems; however, their integration into next-generation computational tools is constrained by the cost of uncertainty quantification with expensive computational models. The computational challenge is particularly prohibitive for tail-sensitive quantities, such as probabilities of failure, conditional values-at-risk, and buffered probabilities of failure, where accurate estimation can require a large number of model evaluations. Multifidelity methods provide a framework for addressing this computational bottleneck by combining information from models of varying accuracy and cost while retaining the predictive accuracy of the high-fidelity model. This review summarizes the state of the art in methods that enable scalable risk and reliability analysis using multifidelity control variates, multifidelity importance sampling, and multifidelity active learning.

0.31

Efficient Robust Optimization via an Adaptive Mono-Level Gaussian Process Surrogate

cs.CE ·2026-11-17 ·Julius Schultz, Mustafa Essou, Ulrich Römer
benchmark similarity 0.61
Abstract

Optimization under uncertainty seeks designs that perform robustly despite varying operating conditions or uncertain model parameters. However, identifying such designs typically requires solving an optimization problem with a nested uncertainty analysis, resulting in a high computational burden. Surrogate models can alleviate this burden and enable optimization under uncertainty even with computationally expensive simulation models. This paper proposes a novel surrogate-based approach that reduces the number of costly model evaluations required for robust optimization. The approach adopts a mono-level strategy, in which a single Gaussian process surrogate is constructed and adaptively refined for both optimization and uncertainty analysis. Refinement in the design space is guided by a sampling-based uncertainty indicator that propagates the surrogate's epistemic uncertainty to the robustness measure. Efficient uncertainty estimation is crucial for the scalability of the algorithm and achieved through pathwise sampling combined with a weight-space representation of the Gaussian process. The proposed approach is evaluated on several analytical benchmark models and requires fewer model evaluations than a state-of-the-art bi-level surrogate approach while achieving comparable solution accuracy. Once trained, the mono-level surrogate can be reused to optimize different robustness measures at reduced cost. Moreover, the approach is successfully applied to a structural optimization problem with constraints and design variable uncertainty.

0.14

Risk-Averse Decision-Focused Learning: Minimizing CVaR of Regret

cs.LG cs.AI ·2026-11-14 ·Jayanta Mandi, Senne Berden, Victor Bucarey, Concepcion Dominguez, Tias Guns
similarity 0.61
Abstract

Many combinatorial optimization (CO) problems involve parameters that are uncertain when the problem must be solved. In such settings, the parameters have to be predicted from correlated contextual information before a solution can be computed. \emph{Decision-focused learning} (DFL) addresses this by training the predictive model to produce predictions that optimize downstream decision quality. Most existing DFL methods are risk-neutral and minimize the expected regret. In high-stakes applications this can lead to undesirable outcomes, because decision-makers must also account for the tail risk of decision quality. Prior risk-sensitive approaches typically model uncertainty explicitly and solve a stochastic or robust optimization problem at inference time, which increases computational complexity. We instead propose a DFL framework that achieves risk-sensitive behavior while retaining point predictions and deterministic optimization at inference time. Starting from the dual representation of the Conditional Value-at-Risk (CVaR) and adding entropy regularization, we derive a smooth surrogate for the CVaR of regret. Its gradient induces an instance-dependent reweighting within each minibatch that emphasizes high-risk outcomes, so the approach can turn existing surrogate-loss DFL methods, including the SPO+ loss, into risk-sensitive variants. When instantiated with SPO+, the resulting objective admits a generalization bound on the CVaR of the true regret. Experiments on knapsack, portfolio, shortest-path, and energy-aware scheduling problems show substantial reductions in tail regret when the uncertain parameters exhibit heterogeneous tail behavior.

0.11

In-Context Learning for Multi-Fidelity Multiscale Model Closure and Cross-Operator Out-Of-Distribution Generalization

cs.LG math.NA ·2026-10-28 ·Xinyuan Li, Matthew Zahr, Panos Stinis, Zecheng Zhang
similarity 0.61
Abstract

Multi-fidelity learning uses low-fidelity information (coarse-scale PDE models) together with limited high-fidelity data (fine-scale solutions) to obtain high-accuracy PDE solutions. It is a data-driven approach to downscale reduced-order models to solve PDEs efficiently and accurately. Standard multi-fidelity learning focuses on one operator or PDE; given a new PDE, retraining using additional coarse-fine solution pairs may be required, which is costly. In this work, we propose an in-context multi-fidelity learning framework for learning multiple multi-fidelity operators with one shared model. The model is trained using paired low- and high-fidelity examples as demonstrations from multiple operators, together with a new coarse-scale solution query, to predict the corresponding fine-scale solution. Once trained, given a new PDE operator, a few (fewer than five) coarse-fine demonstration pairs, and a low-cost coarse query, the model predicts the corresponding fine-scale solution without fine-tuning. When the target operator appears in the training dataset but has a different prediction goal, such as predicting beyond the training time interval, no additional demonstrations are needed. Instead, the model uses training coarse-fine pairs from that PDE and information from other training operators. Testing then only requires generating a low-cost coarse-scale solution query, without fine-tuning. Notably, our smallest model contains only 255,364 trainable parameters, corresponding to approximately 456 parameters per training operator for 560 operators.

0.12

Cost-Accuracy Trade-offs: Neural Operator vs Classical Numerical Solver

cs.LG math.NA ·2026-09-21 ·Daniel Zhengyu Huang, Andrew M. Stuart
benchmark similarity 0.61
Abstract · 41 pages; 10 figures

Neural operators are data-driven models that learn mappings from inputs that parameterize partial differential equations, such as spatially varying coefficients, initial conditions, forcing terms, boundary conditions, or geometries, to solution fields or quantities of interest. Once trained, they can serve as surrogates for classical numerical solvers in many-query settings that require repeated evaluations for varying inputs. We address the question of when, and then why, neural operator surrogates outperform classical numerical solvers, in terms of cost for a given accuracy. We focus on the post-training, many-query limit, in which data-acquisition and training costs are treated as fixed and fully amortized. Even in this deliberately favorable regime for neural operators, there are regimes in which classical solvers outperform the surrogate models. We compare the cost-accuracy performance of neural operator surrogates and classical numerical solvers through a reproducible benchmark study comparing neural operators with problem-matched classical solvers on representative problems in computational science and engineering, focusing on prediction error, per-query floating-point cost, and wall-clock runtime. Neural operators are most competitive at low-to-moderate accuracy requirements. Their floating-point cost advantage depends strongly on the problem structure, arising when they avoid temporal or nonlinear iterations or predict a reduced quantity of interest rather than a full solution field. Additional wall-clock speedups result from dense tensor operations that are well suited to modern hardware. As the target accuracy is tightened, achieving the required accuracy with neural operators becomes increasingly challenging, and classical solvers outperform surrogates in this regime; thus classical solvers will remain important for verification and high-accuracy computation.

0.45