CS Paper Radar

NANO: A Unified Optimization Framework for Nonlinear Gaussian Filtering

Zhenxuan Ma, Haoning Wang, Wenhan Cao, Shengbo Eben Li

eess.SY ·submitted 2026-10-12 10:56 UTC · v2

Abstract

Conventional Gaussian filtering typically approximates nonlinear models or their statistical moments and retains the Kalman update structure. The resulting approximation errors can affect both state estimates and their uncertainty. This paper presents the Natural Gradient Gaussian Approximation (NANO) framework for nonlinear filtering. Both prediction and measurement update are formulated as optimization problems over Gaussian distributions, with the entire Gaussian distribution serving as the optimization variable. The framework integrates observation losses, distributional discrepancies, natural-gradient optimization, and finite-step update rules, providing a unified formulation for constructing different filters. Natural-gradient iterations account for the geometry of the Gaussian family and jointly update the distribution parameters. The framework incorporates the existing NANO-P method using the Kullback-Leibler divergence and introduces NANO-W using the Wasserstein distance. NANO-P maintains covariance positive definiteness through a positive-semidefinite curvature approximation, while NANO-W uses a covariance transport update that accommodates indefinite curvature. Their update rules and sufficient conditions for covariance positive definiteness are established within the common framework. Through these two realizations, the proposed framework achieves the best estimation accuracy among the evaluated methods on each of six benchmarks. Runtime evaluations further assess its computational feasibility for practical applications.

Comment
This manuscript was posted prematurely, before all coauthors had approved its public release. We therefore request its withdrawal

Closest papers

Nearest neighbours by meaning across the whole index.

Stochastic Gradient Descent over P2

cs.LG stat.MLmath.PR ·2026-09-11 ·Maria Oprea, Qin Li, Yunan Yang
similarity 0.61
Abstract

Stochastic gradient descent (SGD) admits diffusion approximations that replace the complicated randomness of stochastic gradients by Gaussian noise, providing a powerful tool for understanding its dynamics and long-time behavior. We investigate whether an analogous approximation principle holds for optimization over probability measures, where the objective is a functional defined on the Wasserstein space P2. The nonlinear geometry and infinite-dimensional nature of P2 prevent a direct extension of the classical Euclidean theory. Using Lions differentiability, we lift the problem to a linear Hilbert space, where higher-order differential calculus becomes available. We then construct a Gaussian random-field approximation whose velocity field matches the mean and covariance of the original stochastic gradient. By exploiting this moment matching through higher-order Taylor expansions, we show that the Gaussian approximation captures the SGD dynamics with second-order weak accuracy. Our result provides a rigorous foundation for replacing sample-driven randomness by analytically tractable Gaussian fluctuations in stochastic optimization over probability measures.

0.24

A Survey of Nonlinear Filtering Methods for Data Assimilation and Recent Advances in Generative AI-Enabled Score Filters

cs.LG math.PR ·2026-11-04 ·Cath Pham, Kadesha Reynolds
survey similarity 0.55
Abstract

Many real-world applications, such as numerical weather prediction or wildfire tracking, involve modeling highly nonlinear dynamical systems that are chaotic and influenced by various sources of uncertainty. To improve model prediction accuracy and reduce such uncertainties, data assimilation (DA) combines the dynamical model with partial, noisy observations to estimate the evolution of the system's state. Doing so depends on solving the optimal filtering problem: computing the conditional distribution of a hidden state given a sequence of observations. Classical approaches fail along three distinct axes: Kalman-based methods assume Gaussian posteriors and produce biased estimates under nonlinear observation operators; particle-based methods represent distributions with weighted samples and collapse in high dimensions due to the curse of dimensionality; and kernel-based methods move beyond discrete particle representations but cannot maintain their density approximation affordably as dimension grows. The Ensemble Score Filter (EnSF) avoids all three constraints by encoding the filtering density in a score function and approximating that score with a training-free Monte Carlo estimator, enabling stable filtering in high-dimensional, chaotic systems. This survey traces EnSF's mathematical foundations, establishes it as the state of the art for non-Gaussian, high-dimensional filtering, and examines fourteen extensions that expand its operational reach. Twelve of the fourteen extensions preserve this training-free Monte Carlo estimator at the core of the update, with only two departing from it to achieve planetary-scale deployment.

0.21

Square Root Gauss-Newton iLQR

cs.RO math.OC ·2026-09-17 ·Maximilian Haas-Heger, Jur van den Berg
similarity 0.55
Abstract · 16 pages, 2 figures, ISRR 2026

The iterative Linear Quadratic Regulator (iLQR) is a widely used algorithm for nonlinear trajectory optimization. At each iteration, it solves a local linear-quadratic approximation of the problem via dynamic programming, propagating a quadratic cost-to-go function. If the Hessian of the cost-to-go approximation is positive-semidefinite, one can derive a square root formulation of iLQR that propagates its Cholesky factor instead. This offers significant numerical advantages - much as square root Kalman filters improve upon their conventional counterparts - particularly when iLQR is used within an augmented Lagrangian framework for handling constraints, where large penalties degrade conditioning. Previous square root formulations of iLQR and related algorithms exist, but they are either numerically suboptimal, algorithmically complex, or both. In this paper, we show that the key to an effective square root formulation lies in the Gauss-Newton (weighted least-squares) structure of the cost function: this yields a positive semidefiniteness property that extends beyond the Hessian to the full augmented cost-to-go matrix, and enables a backward pass of remarkable simplicity in which each step reduces to a single QR-decomposition, from which the feedback gain and propagated Cholesky factor are extracted directly.

0.23

Probe Sets for Nonlinear Kalman Filtering: Multi-Center Gains and Covariance Recalibration

eess.SY eess.SP ·2026-09-13 ·Shida Jiang, Shengyu Tao, Scott Moura
code/data similarity 0.55
Abstract

Kalman filters (KFs) choose the gain as the product of the state-measurement cross-covariance and the inverse of the innovation covariance. In nonlinear systems, the predictive distribution generally changes shape, and these covariances require approximation. Conventional nonlinear KFs approximate them around a single center, the predicted state. They can produce inaccurate gains and overconfident covariance estimates when one local approximation does not represent the measurement geometry across the uncertainty region. To address this issue, we introduce probe sets, small covariance-scaled collections of states at which the filter repeats its local approximation. We then select the gain that minimizes the average covariance reported by these local approximations. The same construction is used in our previously proposed covariance recalibration step to improve covariance consistency. The construction applies to different KF variants without requiring a specific approximation rule. For quadratic measurements, we derive conditions under which probing yields lower normalized true error variance than the corresponding single-center gain when prediction uncertainty is sufficiently understated. We apply the framework to four KF variants and compare the resulting filters with multiple baselines across 300 randomized setups for each of two systems. The results show that probing preserves typical accuracy while substantially reducing covariance inconsistency and the rare large-error trajectories that dominate the root mean square of per-run errors. The code is available at https://github.com/Shida-Jiang/Probe_KF.

0.35

Wiener Space Cubature for Nonlinear Gaussian Filtering

eess.SP ·2026-10-02 ·Luke Snow, Daniel Waxman, Matthew E. Levine
theory similarity 0.55
Abstract

Continuous-discrete filtering is a classical problem in signal processing and control theory, whereby the state driving a stochastic differential equation (SDE) must be inferred from noisy, partial discrete-time observations. Corresponding filtering algorithms are often formed as discrete-time methods and applied to a discretization of the SDE, an approach limited by the incurred discretization error, or model a type of Gaussian moment ordinary differential equation (ODE) flow, which is approximate for nonlinear dynamics. We introduce a new method, the Wiener-space cubature Kalman filter (WSC-KF), for Gaussian filtering in this continuous-discrete setting. The WSC-KF combines spatial sigma-point representations with Wiener space cubature integration, effectively approximating iterated integral terms of the driving Brownian motion to higher-order than standard discretization (e.g., Ito-Taylor expansions), or approximate moment-ODE flows (e.g., as in the unscented Kalman filter). Theoretically, we characterize the path-law approximation degree of WSC-KF through a tunable model order parameter. We prove that, depending on the model order chosen, WSC-KF provides weak order bounds comparable to or dominating that of a cubature Kalman filter using a first-order Ito predict step. Numerically, we confirm that rate on a globally Lipschitz Levy-area diffusion. Degree-5 Wiener cubature matches mixed iterated integrals that existing continuous-discrete Kalman predictors omit, while remaining efficiently computable; this is the mechanism enabling higher-order weak path-law approximation.

0.25

Zeroth-Order Langevin Dynamics for Stochastic Nonconvex Optimization: Curvature Regularization and Non-Asymptotic Guarantees

cs.LG math.OCmath.PRstat.ML ·2026-11-07 ·Stefano Bruno, Dong-Young Lim
similarity 0.54
Abstract

Zeroth-order methods provide a way to optimize stochastic nonconvex objectives using only noisy function evaluations. However, the smoothing inherent in their gradient estimators effectively modifies the objective being optimized. In particular, Gaussian smoothing with smoothing parameter $η>0$ induces, to leading order, an implicit Hessian-trace regularization, which motivates the study of the curvature-regularized objective of the form $f(x)+(η^2/2) tr(\nabla^2 f(x))$. We develop a non-asymptotic theory in the nonconvex setting for a class of algorithms called stochastic zeroth-order Langevin dynamics (ZOLD), which use only noisy function evaluations and do not have access to first- or higher-order derivative information, and derive excess-risk guarantees for the curvature-regularized objective. We first control the finite-time distribution of the ZOLD iterates in Wasserstein distance with respect to the Gibbs measure associated with the Gaussian-smoothed objective. An explicit coupling between the smoothing and inverse temperature parameters then quantifies the proximity of this measure to the Gibbs measure associated with the corresponding Hessian-trace-regularized objective. These distributional estimates in Wasserstein distance lead to excess-risk bounds for the curvature-regularized objective. Our analysis accommodates assumptions under which the stochastic function evaluations are allowed to have non-uniform dependence on the data stream. In addition, the constants in the non-asymptotic bounds do not rely on log-Sobolev or Poincaré inequalities as in prior zeroth-order Langevin analyses. To the best of our knowledge, this is the first non-asymptotic analysis of a zeroth-order Langevin method that simultaneously establishes excess-risk guarantees for a curvature-regularized objective and Wasserstein guarantees for the associated Gibbs measure in the nonconvex setting.

0.14

Generative models for simulation based filtering: Formulations and Empirical Comparisons

cs.LG ·2026-09-14 ·Mohammad Al-Jarrah, Wei Deng, Bamdad Hosseini, Amirhossein Taghvaei
similarity 0.54
Abstract · 6 pages, 2 figures, 1 table

This letter presents a unified formulation and a controlled numerical comparison of generative-model approaches to the nonlinear filtering problem. Under this formulation the analysis step is realized by a transport of the forecast distribution to the posterior, the approaches differing only in how that transport is selected and learned. We derive three new filters, based on stochastic interpolants, their deterministic flow-matching limit, and Schrödinger bridges realized through forward--backward SDEs. We develop a two-stage tuning procedure that separates the training of the generative model from its online refinement. The resulting methods are compared against the optimal transport filter (OTF), the Knothe--Rosenblatt filter (KRF), the sequential importance resampling (SIR) particle filter and the ensemble Kalman filter (EnKF), in terms of accuracy, computational time, and sensitivity to ensemble size and state dimension. The results indicate that every generative filter resolves multimodal posteriors that the EnKF and SIR do not, that no single generative framework dominates, the preferred method being set by the available online budget and ensemble size, and that the filters differ in the regularity of the particle trajectories they produce.

0.25

ZENO: Nonlinear filtering with Zakai Equation Neural Operators

cs.LG q-bio.NC ·2026-10-23 ·Aaditya Prasad, Steven W. Flavell, Brian DePasquale
similarity 0.54
Abstract

State inference and parameter learning for nonlinear filters is a core computation for scientific machine learning with varied applications in engineering and neuroscience. The unnormalized filtering distribution is governed by the Zakai equation, a stochastic partial differential equation (SPDE) that combines the forward Kolmogorov operator (known as the Fokker-Planck equation) with observed likelihood updates. Although solving these high-dimensional SPDEs is often impossible analytically and intractable numerically, advances in operator learning and physics-informed machine learning have made it feasible. Harnessing these advances, we introduce the Zakai Equation Neural Operator (ZENO), a model that optimizes a neural operator against a Zakai Equation based physics-informed loss to approximate the exact unnormalized filtering distribution for latent stochastic dynamical systems. We use this ZENO-inferred posterior distribution in an expectation-maximization (EM) framework to perform parameter learning and state inference. We demonstrate ZENO's ability to accurately infer the posterior distribution and learn its corresponding SDE parameters in three simulated stochastic dynamical systems (Double-Well bistable system, Van Der Pol (VDP) oscillator, and Lorenz system) and compare its performance against established methods for latent SDE inference and learning. We show that ZENO recovers previously observed limit cycles representing forward and backward locomotion from a whole-brain imaging dataset of immobilized \textit{C. elegans} nematodes. ZENO is a novel approach to the problem of nonlinear filtering and dynamical inference through the lens of partial differential equations and operator theory.

0.23