CS Paper Radar

Non-Asymptotic Confidence Regions for Separable Nonlinear Models

Szabolcs Szentpéteri, Péter Kovács, Balázs Csanád Csáji

eess.SY math.STstat.ML ·submitted 2026-10-29 07:49 UTC · v1
publishedtheory

Abstract

This paper studies robust uncertainty quantification for separable nonlinear least squares (SNLLS) problems. We propose two identification methods, (i) Residual-Perturbed Gradient (RPG) and (ii) Global-Fit-Gap (GFG), to construct joint distribution-free confidence regions for both the linear and nonlinear parameters. By exploiting assumed group invariance of the noise (such as symmetry or exchangeability), we apply random perturbations (such as sign-changes or permutations) to build the regions based on resampling and ranking. We prove that both methods guarantee non-asymptotically exact coverage (type I error). Finally, we illustrate the constructions through two numerical experiments: (i) identifying generalized finite impulse response (GFIR) systems using Laguerre polynomials with an adjustable decay rate and (ii) fitting Gaussian radial basis function (RBF) models with variable knots and widths.

Comment
Accepted for publication at the 65th IEEE Conference on Decision and Control (CDC 2026)

Closest papers

Nearest neighbours by meaning across the whole index.

MiNCE: Nonparametric, Strongly Consistent Confidence Envelopes for Band-Limited Functions and their Smoothed Spectra

cs.LG math.STeess.SPstat.ME+1 ·2026-09-08 ·Balázs Csanád Csáji, Bálint Horváth
similarity 0.54
Abstract

Minimum-norm confidence envelope strategies offer a nonparametric approach to constructing nonasymptotic, simultaneous confidence regions for band-limited functions, exploiting the theory of Reproducing Kernel Hilbert Spaces (RKHS). While the finite-sample coverage guarantees of these envelopes have been established, their consistency has not been analyzed so far. In this paper, we study this construction, here termed the Minimum-Norm Confidence Envelope (MiNCE) framework, and establish the strong uniform consistency of the resulting bands, both for noise-free and noisy observation models, under mild assumptions on the measurement noises. We further extend this formulation to the frequency domain, deriving nonasymptotic, simultaneous, strongly uniformly consistent confidence bands for the smoothed spectra. Numerical experiments in nonparametric regression and spectral estimation empirically confirm our theoretical results, illustrating the contraction of the confidence envelopes toward the target function as the sample size increases.

0.29

Direct Inference on Admissible Sets (Diona)

cs.LG math.OCeess.SY ·2026-10-22 ·Lorenzo Sabug, Eric Kerrigan
benchmark similarity 0.54
Abstract · 27 pages, 4 figures

We propose a new regression method named Direct Inference on Admissible Sets (Diona) that directly uses samples to estimate the state and its derivatives at a query point, along with their admissible bounds, without long global training. By casting the values and derivatives at the query and neighbouring samples as latent variables in a constrained optimisation problem, the burden of long training loops becomes one of on-demand local solves. We establish Diona's theoretical properties including finite-sample coverage and, in the noiseless case, convergence to the true state and derivatives under dense sampling. In a nonlinear regression benchmark on six functions of three to eight dimensions, Diona was more accurate than a Gaussian process (GP) on noiseless data especially with denser data sets. Furthermore, as with GP, it produced uncertainty bounds that contained the true value at more than 95% of query points in four functions. Diona's time to a first prediction after a data set update was three to four orders of magnitude shorter than that of the GP, and two to three orders shorter than that of a neural network, which was more accurate but offers no uncertainty quantification. This low latency is paid for by on-demand predictions, so this latency advantage applies when relatively few predictions are needed between data updates.

0.22

Subspace Uncertainty and Sharp Sampling Thresholds on the Boolean Cube

cs.IT math.PRcs.LGmath.CO ·2026-10-08 ·Thomas Weinberger
theory similarity 0.54
Abstract

For a known finite-dimensional model subspace, classical Gaussian regression theory guarantees that least squares eventually attains the parametric rate, with prediction error scaling as the subspace dimension divided by the sample size. Yet subspaces with the same number of unknown coefficients can require very different sample sizes to reach this regime. For subspaces of polynomials on the Boolean cube, a fundamental question is how restrictions on degree and ambient dimension jointly determine this threshold. Our main result sharply characterizes the worst-subspace minimax sample threshold over a substantial parameter range, revealing an exponential dependence on degree and precisely quantifying the joint effects of degree, ambient dimension, and subspace dimension. A small modification of our lower-bound construction reveals a strong separation: a single subspace can admit noiseless recovery with optimal sample complexity independent of degree, yet require essentially the worst-case sample size, exponential in degree, to attain the noisy parametric rate. To establish the sampling lower bound, we sharpen the Polyanskiy-Samorodnitsky uncertainty principle (J. Funct. Anal., '19) to uniformly optimal precision and extend its concentration guarantee to large subspaces in which every function concentrates most of its energy on one shared Hamming ball. The sampling upper bound follows from the sharp moment bound of Kirshner and Samorodnitsky (IEEE Trans. Inf. Theory, '21) combined with empirical-process bounds.

0.33

Tight Sequential Confidence Intervals for Online Kernel Regression

cs.LG stat.MLmath.PR ·2026-11-15 ·Haichen Hu, David Simchi-Levi
similarity 0.54
Abstract

We characterize, in the worst case over kernels, the sharp minimax width of online confidence intervals for elements of a reproducing kernel Hilbert space (RKHS). This resolves the open problem posed by Vakili et al. (2021) in its original discrete-time form. A learner observes $y_i=f(x_i)+ε_i$ at adaptively chosen points, with $\|f\|_{\mathcal H_k}\le B$ and i.i.d. $R$-sub-Gaussian noise, and reports the interval $|f(x)-μ_n(x)|\leρ_n(δ)σ_n(x)$, where $μ_n$ and $σ_n$ are the posterior mean and standard deviation of a surrogate Gaussian process model with regularization parameter $λ$. Writing $ρ_n^*(δ)$ for the minimax confidence width for a fixed kernel, we show that, for the sum of a linear kernel and a periodic Sobolev kernel, all sufficiently large $n$, and all $δ\in(0,1/4]$, $ρ_n^*(δ)\asymp B+(R/λ)\sqrt{γ_n+\log(1/δ)}$, where $γ_n$ is the maximal information gain and grows polynomially in $n$. Building on Lattimore (2023), we construct an adaptive design on a fixed two-dimensional domain for which, under Gaussian noise, the normalized error at a prespecified point is $Ω(B+(R/λ)\sqrt{γ_n})$ with constant probability. This rules out the conjectured polylogarithmic width in the worst case over kernels and online designs.

0.18

Rank-one variance inflation in polynomial estimation of signal parameters with a sign-preserving fractional-power basis

eess.SP math.STstat.ME ·2026-10-04 ·Serhii Zabolotnii
code/data similarity 0.53
Abstract · 29 pages, 3 figures, 8 tables. v2: corrected Sections 1.2 and 5.5 (relation to arXiv:2606.11421 is a shared mechanism, not a special case). …

The polynomial maximization method estimates signal parameters in non-Gaussian noise from a finite description of the noise by its moments and cumulants, without a density. Known accuracy results assume that these noise characteristics are known, or use a design in which estimating them jointly costs no accuracy. In practice they are estimated from the residuals of a preliminary fit, and the effect of this substitution on accuracy is unknown. We treat the feasible scheme as a two-step estimator and, for a general smooth signal model in a fixed design, find the increment of its asymptotic covariance in closed form: $Σ_{\rm fe}-Σ_{\rm or}=c_2(1-g_{\rm or}) Q^{-1}\bar{d}\bar{d}^{\top}Q^{-1}$, a positive semidefinite matrix of rank at most one. Here $c_2$ is the noise variance, $g_{\rm or}\le 1$ the oracle variance reduction factor, and $Q$ and $\bar{d}$ the second-moment matrix and mean of the signal gradient over the design. Estimating the fractional moments that form the weights costs no first-order accuracy, because that block is orthogonal; the whole increment comes from estimating the sign-preserving centering means. The increment lies along the single direction $Q^{-1}\bar{d}$, so linear combinations of the parameters orthogonal to it keep oracle accuracy. In the trace metric the variance reduction factor becomes $g_{\rm fe}=g_{\rm or}+(1-g_{\rm or})R_D$, where $R_D=\|Q^{-1}\bar{d}\|^2/{\rm tr} Q^{-1}$ depends on the design alone. We give four sufficient conditions for first-order oracle equivalence; noise symmetry is not among them. Monte Carlo experiments with a harmonic signal confirm the direction and size of the increment. A covariance estimate that ignores the increment undercovers where non-Gaussianity gives the largest gain, symmetric noise included.

0.65

Locally Adaptive Conformal Prediction for Polynomial Chaos Expansions

cs.CE stat.COstat.MEstat.ML ·2026-11-06 ·Dimitrios Loukrezis, Arthur Hatstatt, Bruno Sudret, Dimitris G. Giovanis
benchmark similarity 0.52
Abstract · 22 pages, 11 figures, 2 tables

This work develops a locally adaptive jackknife+ conformal prediction framework for polynomial chaos expansions. Therein, the non-conformity scores are normalized with an auxiliary polynomial chaos expansion trained on the log-transformed leave-one-out residuals of the base model. We show that the coverage guarantee of jackknife+ conformal prediction carries over to the adaptive intervals if the leave-one-out auxiliary models are trained on nested leave-one-out residuals. For fixed-basis polynomial chaos expansions fitted by ordinary least squares, all required quantities are derived in closed form and the guarantee-preserving adaptive intervals are obtained from a single model fit, without any model retraining. For sparse polynomial chaos expansions, where multiple model fits are necessary, we propose a heuristic simplification that limits the computational overhead of adaptivity at the cost of the formal coverage guarantee. Numerical experiments on an academic benchmark with known heteroscedastic noise and on an engineering simulation dataset show that the suggested method attains the target coverage while providing intervals that adapt to local prediction uncertainty, therefore being considerably more statistically efficient. These advantages are particularly prominent in high-uncertainty regions of the input space, where the performance of non-adaptive methods degrades substantially.

0.26

Optimal estimation for Functional Linear Regression with Noisy Discretized Data

cs.LG stat.ML ·2026-09-08 ·Sixtine Sphabmixay
similarity 0.51
Abstract · 63 pages, 25 figures

In this paper, we consider the scalar-on-function linear regression model under a realistic sampling scheme in which the functional covariates are observed on a regular grid and contaminated by additive noise. We propose a two-step estimation procedure: first, the underlying curves are reconstructed from the discrete noisy observations using a Fourier-based projection method; second, the slope function is estimated by a penalized least-squares criterion over finite-dimensional trigonometric spaces, with data-driven selection of the model dimension. We establish oracle-type inequalities for the prediction error, both with respect to the reconstructed curves and to the true latent curves. Under regularity assumptions on the slope function and polynomial decay of the eigenvalues of the covariate, we derive convergence rates for the prediction error and show that our estimator attains the minimax rate when the number of grid points is sufficiently large. Finally, the proposed method is illustrated on simulated data and on a real meteorological dataset.

0.24

Adaptive Robust Confidence Sets in High Dimensions

cs.DS math.STcs.LGstat.ME+1 ·2026-10-11 ·Ilias Diakonikolas, Chao Gao, Daniel M. Kane, Yuetian Luo
theory similarity 0.51
Abstract

We study adaptive confidence sets for mean vectors of multivariate Gaussians with identity covariance under Huber contamination when the contamination proportion is unknown. We show that an information-theoretically optimal adaptive confidence set can be constructed by simultaneously quantifying uncertainty across all one-dimensional projections. The resulting confidence set achieves the optimal diameter and reveals a substantial adaptation cost compared with the minimax estimation error. Although statistically optimal, this construction is computationally infeasible. We establish a Statistical Query (SQ) lower bound showing that achieving diameter $O(m^{-1/2})$ requires either exponentially many queries or query tolerance $d^{-Ω(m)}$. Thus, in contrast to robust mean estimation, adaptation to the unknown contamination proportion incurs both a statistical cost and a computational barrier within the SQ framework. Finally, we give a low-degree moment-matching algorithm that constructs an adaptive confidence set using $d^{O(m)}$ samples and time. Its diameter bound contains an additional $O(m^{-1/2})$ term beyond the information-theoretic rate, matching the dependence on $m$ in the SQ lower bound. Together, these results characterize the statistical limits of adaptive robust confidence sets under Huber contamination and quantify a computational barrier within the SQ framework.

0.36