CS Paper Radar

Dynamics Aware Past-Future Information Bottleneck

Wooyoung Chung, Stas Tiomkin

cs.IT eess.SY ·submitted 2026-11-05 18:21 UTC · v1
theoryunusual

Abstract

The simplification of complex dynamical systems is essential for efficient estimation and control. A principled approach to this task is through the lens of information-theoretic compression. The Past-Future Information Bottleneck (PF-IB) provides a theoretical framework for compressing linear dynamical systems. However, the standard formulation often lacks structural consistency, yielding representations that are not realizable as dynamical systems. By constraining the objective to a specific manifold of block Hankel matrices, which we prove is a Lie group, we bridge the gap between PF-IB and realization theory. Our approach ensures that the resulting compressed dynamics are realizable, thereby enabling its direct application to estimation and control. We demonstrate our approach in numerical simulations.

Closest papers

Nearest neighbours by meaning across the whole index.

Tensor-Train Weak SINDy: Identifying High-Dimensional Nonlinear Dynamics

cs.LG cs.CEstat.ML ·2026-09-08 ·Will Houser, Vanja Dukic, David M. Bortz
similarity 0.54
Abstract · 34 pages, 8 figures

Weak Sparse Identification of Nonlinear Dynamics (WSINDy) provides a noise-robust approach for learning dynamical systems from data without requiring numerical differentiation. However, for high-dimensional systems, tensor-product libraries of candidate functions grow exponentially with the state dimension, making standard WSINDy expensive in both computation and memory. The Multidimensional Approximation of Nonlinear Dynamics (MANDy) addresses this scaling through a tensor-train (TT) representation of the candidate library, but does not provide a mechanism for sparse model selection. Here, we combine these approaches to develop TT-WSINDy, which performs the weak-form transformation, regression, and sparsification in TT format. We show that the TT formulation recovers the corresponding WSINDy regression problem and derive polynomial time and memory complexity bounds for the tensor-train sparsification procedure. Numerical experiments demonstrate robustness to measurement noise and computational savings for high-dimensional systems.

0.19

Dynamics-Aware Sampling and Recovery for Tensor-Valued Dynamical Systems

cs.IT math.NA ·2026-10-25 ·Seok-Young Chung, Longxiu Huang, Yisen Wang
theory similarity 0.53
Abstract · 55 pages, 9 figures

We study sampling and recovery for tensor-valued dynamical systems evolving under the t-product, using partial spatial observations over time. Although the t-product Fourier transform yields slicewise matrix dynamics, recovery does not generally decouple across slices. Eigenvalues shared across Fourier-domain frontal slices couple the corresponding eigenspaces into aggregated spectral blocks, which determine the tensor-specific recovery geometry and space--time tradeoff. We establish deterministic identifiability conditions for the initial tensor state. In the diagonalizable case, these are frame conditions on the aggregated blocks. We extend them to non-diagonalizable systems through generalized eigenspaces and Jordan structure. A temporal--spectral factorization of the finite-horizon stacked recovery system underlying Tensor Dynamical Sampling (DS) yields a decoupled two-stage formulation once the temporal threshold is reached. Under independent spatial subsampling, we prove high-probability recovery guarantees using empirical Gram matrices on the aggregated blocks. The bounds depend on aggregated multiplicities, incoherence, and population conditioning. Stability estimates separate the spatial frame factor from the temporal Vandermonde factor and show that increasing temporal depth may worsen numerical conditioning even after identifiability is ensured. We implement Tensor DS and evaluate it on synthetic and real data. Phase transitions track the predicted blockwise sampling thresholds, and stability experiments illustrate Vandermonde conditioning. Experiments on sea surface temperature and turbulence velocity data show that a learned t-product system operator generally yields more accurate and structurally faithful reconstructions than vectorized low-rank, Tucker-type, graph-regularized, graph-spectral, and vectorized dynamical sampling baselines.

0.28

Data-driven representation of continuous-time linear time-invariant behaviors via Hankel integral operators

eess.SY math.OC ·2026-11-06 ·Fritz A. Engeln, Marta Zagórowska, Jan-Willem van Wingerden
similarity 0.52
Abstract · 9 pages, 1 figure

We present a data-driven representation of the finite-horizon behavior of deterministic controllable continuous-time LTI systems. We first introduce a notion of persistency of excitation for continuous-time signals that does not require the signal to be differentiable. We then show that, when the input is persistently exciting in this sense, the closure of the range of a Hankel integral operator built from a single input-output trajectory equals the behavior of the system over a finite time horizon. This result provides a nonparametric representation of system trajectories directly in terms of measured data, without requiring an explicit parametric model of the system.

0.31

Dynamics as Code: On Model Compression via Dynamic System

cs.LG ·2026-10-08 ·Fan Gao, Wei Su, Juntong Fan, Renfeng Peng, Hongyu Liu +2 more
theoryunusual similarity 0.52
Abstract · 17 pages including supplementary material

The escalating size of pretrained neural networks has rendered model compression a prerequisite for deployment under stringent memory and compute constraints. With the irrational winding as an example, earlier work introduced a dynamic system (DS) paradigm that reconceptualizes compression as compact weight representation: high-dimensional parameters are encoded by the index of a trajectory produced by a dynamic system, from which the vector is recovered during decompression. This mechanism is fundamentally distinct from pruning, quantization, knowledge distillation, and low-rank decomposition. Along this direction, we prove that under a Diophantine condition, a finite trajectory of \(M = O(ε^{-(d+ν)})\) states in the irrational winding constitutes an \(ε\)-net over the \(d\)-dimensional weight space, thereby linking state resolution, decompression error, and compression ratio in a predictable manner. Furthermore, we propose a generalized DS-based model compression framework by unifying four DS families---space-filling curves (Hilbert, Peano, Morton/Z-order, Snake), chaotic systems (Lorenz), congruential and pseudo-random generators (LCG, PCG), and low-discrepancy sequences (Halton). Also, we introduce the KD-tree and coordinate-template acceleration to scale to large models as well as outlier identification to control the error. Experiments on ResNet-18 and Qwen2.5-1.5B/Qwen1.5-7B validate that DS-based compression achieves competitive compression ratios without post-hoc retraining, with controllable decompression error and flexible state-space design, establishing it as a principled and practical compression approach.

0.29

On the Information Bottleneck for Ergodic Finite-State Markov Chains: Stationary Approximation and Asymptotic Optimality

cs.IT ·2026-10-19 ·Wael Tabbara, Jihad Fahs, Mohamad Assaad
similarity 0.52
Abstract

In this paper, we investigate the information bottleneck problem for sources that follow discrete-time ergodic finite-state Markov chains. We propose treating observations generated by the time-varying Markov chain as if they were drawn from its unique stationary distribution, and we use the solution to the information bottleneck problem for the stationary process to compress samples from the time-varying process. The resulting error is upper bounded by several information gap terms. We establish sufficient conditions under which these gaps converge to zero and analyze the convergence rates of the different information gap terms. The proofs are based on intricate analysis using convergence of subsequences along with the derivation of properties of the optimization sets associated with the information bottleneck problem. In addition to the inequalities and convergence results we establish, our analysis offers insights into how simple properties of ergodic Markov chains give rise to results concerning the information bottleneck problem for such chains.

0.31

What Should an Information Bottleneck Encoder Learn? Separating Representation Learning from Rate Control

cs.LG cs.IT ·2026-10-26 ·Hassan Tavakoli, Bella Bose
similarity 0.52
Abstract

The information bottleneck (IB) trades compression of an input $X$ against prediction of a target $Y$ through a multiplier $β=1/λ$, yet each operating point is usually served by training, or fully conditioning, a new encoder. For jointly Gaussian data, we show that the IB solution factorizes as $A(β)=D(β)W$: the information-bearing directions $W$ do not depend on $β$, and $β$ acts only through scalar gains. We ask whether this separation of \emph{representation} from \emph{rate control} survives beyond Gaussian models, treating the budget as a given operating parameter rather than a quantity to be optimized. For a budget-free representation \(F(X)\), the \emph{separation gap} is controlled by the \emph{sufficiency deficit} $ε_F=I(X;Y\mid F(X))$: We show there is at least $ε_F$ in the worst case for finite inputs, and it is bounded uniformly in $λ$ by a function of $ε_F$ for finite label alphabets. The same certificate extends to a new target distribution and to pooled multi-domain data, where a conditional-alignment term appears. On MNIST, SVHN, CIFAR-10 and STL-10 (10 seeds, eight prespecified budgets), a frozen CNN representation with one compressor--decoder pair per budget reduces estimated training multiply--accumulate operations (MACs) by $7.7\times$ and wall-clock time by about $2\times$ relative to end-to-end VIBs, and attains higher budget-averaged accuracy ($65.6\%$ versus $57.3\%$ per domain; $67.1\%$ versus $57.1\%$ pooled), with the advantage concentrated at strong compression, where the VIBs collapse. At weak compression ($λ\le0.036$), VIB has higher cross-entropy and ECE; its surrogate Lagrangian is higher per domain, indistinguishable from the pooled VIB in the macro average, and lower on the pooled test set.

0.10

Ladder of information limits on prediction for reduced-order models

cs.IT nlin.CDphysics.comp-ph ·2026-09-16 ·Adrian Lozano-Duran
code/data similarity 0.51
Abstract

What can and cannot be predicted by a model when only limited information is available? We answer this question by constructing a ladder of information limits that organizes prediction tasks according to their information requirements: from memoryless and memory-augmented trajectory forecasts to event prediction, stationary statistics, and generative laws. The analysis, which is independent of the model structure or architecture, identifies the minimum attainable error along with the information limitations that give rise to it. The approach distinguishes information hidden in unresolved variables from information recovered through time-delayed observations and connects these contributions to Mori--Zwanzig memory. The ladder also reveals why Lyapunov growth alone cannot characterize reduced-order prediction error, why stationary statistics may remain predictable even when individual trajectories become unpredictable, and why stochasticity can represent (but cannot recover) missing information. Numerical studies of the Kuramoto--Sivashinsky equation and the Lorenz system illustrate these results across the ladder. This unified perspective provides a principled language for understanding the fundamental limitations of ROMs for a given prediction task and for clarifying whether improved predictions require richer observations, greater precision in the input, or additional memory.

0.51

Direct and Indirect Data-Driven Control with Prior Information about the Equilibrium Manifold

eess.SY ·2026-09-24 ·Yi-Chun Liao, Valentina Breschi, Marco M. Nicotra
published similarity 0.51
Abstract · Accepted at the 65th IEEE Conference on Decision and Control (CDC 2026)

By hinging on the assumption that a system to be controlled is fully unknown, many data-driven control approaches do not leverage available or readily inferable priors. In contrast to this viewpoint, this paper analyzes the impact of using the system's equilibrium subspace to inform direct and indirect linear quadratic regulation. For the indirect case, we show how including a constraint on the equilibrium subspace in the identification problem changes the statistical properties of the learned model. In particular, we show that enforcing consistency with respect to the equilibrium subspace leads to a reduction in the estimator variance that, in turn, enhances model-based control performance. In the direct case, we show how this prior can be leveraged to gain insight into the controlled system without requiring an explicit identification step. These results are supported by both numerical and experimental evidence, showcasing the advantages of explicitly leveraging the equilibrium manifold as a prior in data-driven control.

0.25