CS Paper Radar

Robust Space-Filling Input Design via Stochastic Optimization

Máté Kiss, Roland Tóth, Maarten Schoukens

eess.SY ·submitted 2026-08-13 15:24 UTC · v1

Abstract

The space-filling input design approach generates a so-called space-filling dataset in the feature space of the system model. The design method is applicable on a broad class of model structures with wide selection of signals and also incorporates information measures through optimality criteria into the signal design. However, during the signal design, knowledge of a hypothesized model is required. The designed signal can perform far from the optimal if the true system is significantly different from the hypothesized system model. This paper proposes a robust space-filling input design algorithm that can generate a space-filling dataset for an entire class of models. The proposed algorithm takes the expectation of an optimality criterion over the population of the model class, and a stochastic approximation technique is employed to optimize this robust criteria. The efficiency of the proposed algorithm is demonstrated in a simulation example.

Closest papers

Nearest neighbours by meaning across the whole index.

A Reinforcement Learning Approach to Space-Filling Input Design for Nonlinear System Identification

eess.SY ·2026-11-13 ·Charly Witmeur, Gonçalo G. Cruz, Maarten Schoukens, Mark C. Runacres, Jan Decuyper
benchmark similarity 0.57
Abstract

In this paper, we propose a reinforcement learning (RL) approach to space-filling input design for nonlinear system identification, in which an agent learns a control policy that actively drives the system to explore its state space. Classical input design, developed mainly for linear models, minimises the covariance of the estimated parameters. For nonlinear black-box models, however, the dominant error is the bias induced by the model structure over the region of interest, which motivates space-filling strategies that seek to cover the system state space. Existing space-filling approaches rely on predefined input signal classes and require solving explicit optimisation problems. Unlike these methods, the proposed approach requires no input class to be selected and no frequency band to be tuned for the system at hand, satisfies an amplitude bound by construction, and adapts to varying initial conditions through its observation-based policy without re-optimisation. The learned excitation signals exhibit physically meaningful behaviours, such as resonance excitation and amplitude modulation. The approach is evaluated against tuned swept sine and crest-factor-optimised multisine excitations, and against a state-of-the-art space-filling multisine design, on two nonlinear benchmark systems: a mass-spring-damper with nonlinear stiffness and a Van der Pol oscillator. Under the same amplitude bound, the learned policy achieves the best overall space-filling performance on both systems and distributes its samples more uniformly than the state-of-the-art design.

0.23

Deterministic Bounds on Accuracy and Stability of Metric-Space Regression and Classification

cs.LG math.OC ·2026-10-15 ·Nina Maria Gottschling, Jakob Gawlikowski, Michele Caprio
similarity 0.47
Abstract

Data-driven approaches often face ambiguity in datasets due to sensor or target noise, insufficient information, and noise in the input data. While best-fit machine learning methods can accommodate such noise through uncertainty estimates, they are prone to overfitting and it is challenging to quantify ambiguity of the relation between inputs and targets. We introduce a geometric framework that quantifies this ambiguity through the average kernel size, computed with data-dependent, i.e. model-independent, feasible sets derived from worst-case input and target noise. The framework provides computable, deterministic, non-asymptotic bounds on the best achievable empirical error and accuracy for a finite, constructed dataset, independent of the predictive model class, and applies to regression and classification with pseudo-metric input spaces and metric target spaces. The ambiguity of the input-target relation implies three consequences: an empirical error floor, an accuracy-stability trade-off, and, under standard probabilistic assumptions, a strictly positive distribution-training error gap when training below the accuracy bounds. Under standard Borel assumptions for Euclidean norms, the distributional analogue of the average kernel size is related to Bayes risk as it reduces to the optimal distribution root-mean-squared error times the square root of two. Experiments spanning tabular regression, image regression and classification datasets yield nontrivial bounds on the best achievable empirical error and accuracy on a constructed dataset, alongside descriptive comparisons with model performance on the original datasets. The bounds provide a computable, model-independent reference scale on the constructed dataset and comparisons with model performance on the original dataset are descriptive.

0.23

Safe Performance-aware Online Input Design

eess.SY ·2026-11-09 ·Diyou Liu, Mohammad Khosravi
similarity 0.46
Abstract

In this paper, we propose an input design framework for the safe online excitation and identification of uncertain linear systems without requiring informative initial data. The objective is to actively excite the system and collect sufficiently informative input-output data while maintaining safe operation and satisfactory control performance. Achieving this objective is challenging due to the potential conflict between the excitation required for learning and the safety and performance requirements, particularly when limited prior knowledge of the system is available. Inspired by the dual control principle, the proposed scheme balances exploitation for control performance with exploration for model improvement. A Bayesian learning approach is employed to update the model uncertainty online, starting from a conservative initial uncertainty set. As informative data are collected, the uncertainty set contracts, enabling progressively less conservative operation. To reduce the online computational burden, the resulting program is reformulated into a computationally tractable form. Furthermore, an online ellipsoidal terminal set with non-increasing conservatism is designed to guarantee the recursive feasibility of the proposed scheme. Numerical experiments are conducted to demonstrate the effectiveness of the proposed framework, particularly in terms of uncertainty reduction and control performance.

0.21

Active Informativity: Online Input Design for Data-Driven Control

eess.SY ·2026-09-29 ·Yishu Wang, Paolo Rapisarda, Jianquan Lu, Yang Liu
similarity 0.45
Abstract

We develop a unified framework for online input design in data-driven control from noisy data. We introduce a performance measure and analyze how such quality measure changes as new data are added. Based on such analysis we propose an online method for selecting inputs that ensures that data quality does not decrease over time. We demonstrate the validity of our approach by formulating six common noisy data-driven control problems in our framework, and by solving numerical examples for three of them.

0.24

PAC-Bayesian Bounds for Learning Partially Observed Stochastic Linear Time-Invariant State-Space Systems with Inputs and Sub-Gaussian Noise

cs.LG math.OC ·2026-09-08 ·Mihaly Petreczky, Mohamad Al Ahdab, John Leth
similarity 0.45
Abstract

In this paper we derive a Probably Approximately Correct (PAC)-Bayesian error bound for partially observed linear time-invariant (LTI) stochastic dynamical systems in state-space form with inputs and sub-Gaussian noise. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points. The bound derived in this paper relates the expectation of prediction errors with the prediction error generated by the model on the data used for learning. In addition, we show that it can also be used to derive bounds for the parameter estimation error. In turn, this allows us to provide finite-sample error bounds for the prediction error and parameter estimation error for a wide class of system identification algorithms. Furthermore, as LTI systems are a sub-class of recurrent neural networks (RNNs), these error bounds could be a first step towards PAC-Bayesian bounds for RNNs.

0.49

Bayesian Optimization on Function Spaces via Sparse RKHS Manifolds

cs.LG stat.ML ·2026-10-05 ·Davide Sartor, Meghan E. Huber, Donghyun Kim, Nathan Wycoff
benchmark similarity 0.44
Abstract

Bayesian Optimization (BO) has become an established methodology for minimizing black-box functions of a vector input. Often, however, this parameter vector arises from the discretization of an inherently functional relationship. Several recent articles have considered the Functional Bayesian Optimization (FBO) setting, in which the variable to be optimized is not a member of a finite dimensional vector space, but rather an infinite dimensional function space. In this work, we propose $L^0$ Manifold Optimization (L0MO), a simple approach to FBO which searches the subset of a Reproducing Kernel Hilbert Space (RKHS) consisting of functions with a sparse representation in the kernel functions, optimizing both the kernel locations and their coefficients. We discuss in detail the relationship between our method and existing ones, providing a unifying lens through which to view prior works. To assess our method against the state of the art, we conduct an extensive computational study, and along the way develop a novel set of benchmark test functions which port standard finite-dimensional ones to the infinite dimensional domain. Our experiments demonstrate that, on balance, the proposed method achieves superior performance across a wide range of test benchmarks.

0.19

On the Complexity of Bayesian Signal Processing

cs.CC econ.TH ·2026-08-30 ·Yi Liu
similarity 0.42
Abstract

We develop a computational framework for Bayesian decision-making. We show that as long as no action is optimal in every state, Bayes-optimal choice is intractable. This hardness need not arise from large action, state, or signal spaces, nor from a complicated represented utility function: extracting enough information from a hard-to-interpret signal to act optimally can itself be computationally hard. We also characterize tractability across approximation notions and identify their sources of difficulty. Under the probably approximately correct criterion, sample-based Bayesian learning is tractable if and only if the signal support is bounded. Our results provide justifications for bounded rationality, costly Bayesian inference, and sample-based Bayesian learning.

0.22

Constraint-Aware Discrete Black-Box Optimization Using Tensor Decomposition

cs.LG ·2026-09-08 ·Keisuke Onoue, Ryosuke Kojima
published similarity 0.42
Abstract · 32 pages, including supplementary material. ECML PKDD 2026

Discrete black-box optimization is often addressed using approaches such as Sequential Model-Based Optimization (SMBO), which aims to improve sample efficiency by fitting surrogate models that approximate a costly objective function over a discrete search space. In many real-world problems, the set of feasible inputs is often given by logical constraints known in advance. However, existing surrogate modeling techniques generally fail to capture the symbolic rules governing feasibility in discrete input spaces. In this paper, we propose a surrogate modeling approach based on tensor decomposition that captures the structure of discrete search spaces while directly integrating feasibility information. To implement this approach, we formulate surrogate model training as a constrained polynomial optimization problem and solve a relaxed formulation using a differentiable penalty term derived from T-norms. Our experiments on both synthetic and real-world benchmarks, including a pressure vessel design task, demonstrate that the proposed method improves sample efficiency by effectively guiding the search away from infeasible regions.

0.47