CS Paper Radar

Fundamental Limits of Transferability and Equivariance in Algebraic Signal Models I: Finite Dimensions

Alejandro Parada-Mayorga

eess.SP cs.LG ·submitted 2026-09-28 18:44 UTC · v1

Abstract

We study the fundamental limits of transferability in algebraic signal processing through homomorphisms between algebraic signal models. Homomorphisms are linear maps between the signal spaces of two models that commute with filtering, so filtering a signal and transferring it across domains can be done in either order. The existence of such maps is governed entirely by coincidences among the filtered eigenvalues of the two models' shift operators, but existence alone is insufficient: the space of homomorphisms always contains trivial elements that destroy all information. We introduce the spectral transfer efficiency $η(θ)\in[0,1]$ to quantify information-preserving quality, prove that every homomorphism decomposes into unconstrained blocks over coincidence classes, derive the dimension of the homomorphism space, and characterize exactly when lossless transfer is achievable. Beyond normal shift operators, we quantify a departure-from-normality penalty and show how filter derivatives can repair spectral defectiveness. The theory yields concrete consequences in three settings: for sampling, eigenvalue interlacing converts transferability under subsampling into an explicit filter design constraint; for compressed sensing, $η(θ)$ controls the restricted isometry constant and the coherence of the resulting measurements, and recovery decouples across coincidence classes; and for machine learning, spectral aliasing emerges as the controlled symmetry breaking that makes transfer between mismatched domains possible at all.

Closest papers

Nearest neighbours by meaning across the whole index.

Sampling of Signals on Multigraphs: From Removable and Uniqueness sets to Spectral Proxies

eess.SP cs.LG ·2026-10-19 ·Alejandro Parada-Mayorga
theory similarity 0.58
Abstract

In this paper, we study the sampling of bandlimited signals on multigraphs, where several graphs with different shift operators are defined on the same set of nodes. Since the shift operators do not commute in general, they admit only a joint block diagonalization, and the frequencies of the multigraph signal model are matrices rather than scalars. Relying on a formal algebraic convolutional signal model, we introduce a notion of frequency and bandwidth for multigraph signals that does not depend on any individual layer, and we extend the notions of removable and uniqueness sets to multigraphs. We prove that the optimality of a sampling set is determined exclusively by the common Fourier basis and its invariant subspaces, and not by any individual layer, although the individual layers provide computable sufficient conditions that certify a sampling set as a uniqueness set. Since identifying which layers provide reliable certificates requires spectral information, we propose an aggregate operator that pools the information of all the layers, is computable directly from the shift operators, and certifies uniqueness without any assumption on the individual layers. This operator leads to a greedy sampling algorithm that does not require computing the joint block diagonalization. Numerical experiments on multigraphs whose layers order the frequencies inconsistently show that the proposed algorithm selects better-conditioned sampling sets and attains lower reconstruction errors than single-layer methods.

0.28

Score the Algebra, Not the Span: Dimension Reduction for Transfer Operator Models of Dynamical Systems

cs.LG ·2026-08-19 ·Mark Kozdoba, Shie Mannor
benchmark similarity 0.56
Abstract

Dimension reduction for dynamical systems is standard practice, and the standard route is spectral: model the transfer (Koopman) operator by its leading modes. We show that on systems assembled from several weakly interacting components --- a structure common in physical and biological settings --- this may either require an exponential number of modes, or drop an entire component: the component is absent from the model rather than modeled coarsely, and no function of it can be predicted at any accuracy. We call this linear masking. The cause is that a rank-based model pays one coordinate per mode. We propose to score instead the $σ$-algebra the coordinates generate, so that products and powers come free and a component's cost is governed only by its generators rather than by all its interactions. The criterion is a $χ^2$-divergence between the embedded present and future, and it carries a budget guarantee: twice the intrinsic dimension of the dynamics is enough coordinates for an embedding whose algebra carries the operator's entire spectrum, with its full infinite rank. In variational form the criterion admits off-the-shelf estimators, and restricting its critic to the bilinear class returns the VAMP score on the span, so rank-based methods are one end of the same family. We demonstrate the proposed objective on a composite of published benchmark systems. We exhibit examples where the rank-based methods completely miss the masked components at all ranks $k<100$, while ten algebra coordinates recover all of them. In addition, the resulting algebra representation supports predicting the masked components from few labels, while direct regression from the high-dimensional observation or from the VAMP features fail.

0.26

A Hardy-Space Proof of the Filter-Only Gaussian Feedback-Capacity Formula

cs.IT ·2026-09-14 ·Jun Su, Yang Xu, Guangyue Han
theory similarity 0.52
Abstract

The feedback capacity of power-constrained channels with additive stationary Gaussian noise was formulated by Kim in [Kim, 2010] as an infinite-dimensional optimization over the spectrum of an independent stationary Gaussian component and a strictly causal feedback filter. Kim further asserted that the independent stationary component can be omitted. However, Derpich and Østergaard [Derpich and Østergaard, 2022] identified a gap in the original proof, leaving the original derivation of the filter-only capacity formula incomplete. In this paper, we establish the filter-only formula at the level of capacity suprema for noise spectra satisfying the Paley-Wiener condition. In particular, we show that polynomial feedback filters suffice in the capacity supremum, without a positive lower bound on the noise spectrum or a rationality assumption. Under a positive essential lower bound, a complementary approximation preserves the output spectrum and rate exactly, while its input power converges to that of the original pair. An interleaved AR(1) example connects these results with the SK(2) coding scheme. The argument does not assert attainment by a single filter.

0.34

Tight Bounds for Data-driven Multiple Hyper-parameter Tuning with Structured Loss Function

cs.LG stat.ML ·2026-08-18 ·Anh Tuan Nguyen, Viet Anh Nguyen
theory similarity 0.51
Abstract · 19 pages, 2 figures

Data-driven algorithm design frames hyperparameter tuning as a statistical learning problem, but establishing generalization guarantees remains challenging due to the implicit, non-smooth dependence of model performance on hyperparameters. Existing multi-dimensional bounds under piecewise-polynomial assumptions remain theoretically loose and lack comprehensive lower bounds. We resolve this by establishing tight pseudo-dimension bounds for multi-dimensional data-driven tuning. First, we refine the learning-theoretic upper bound using real algebraic geometry; by analyzing invariant connected sign cells during block elimination rather than isolated sign vectors, we avoid topological over-counting to derive strictly sharper sample complexities. Second, we present a multi-regime lower-bound framework that disentangles combinatorial and algebraic capacities. By constructing shattered problem instances across distinct regimes, we prove our upper bounds are tightly saturated. Finally, we extend our topological framework to accommodate general bi-level validation-loss tuning and broader semi-algebraic applications.

0.30

Generic Local Identifiability Does Not Imply Generic Global Identifiability in Linear Networks

eess.SY math.OC ·2026-10-13 ·Anders Hansson
similarity 0.51
Abstract

We give a counterexample to the implication from generic local to generic global identifiability for linear networks with a fixed topology, independently unknown edge coefficients, and selected excited and measured nodes. A six-node network with nine unknown coefficients has exactly two compatible parameter vectors generically. Adding six excited and measured nodes preserves this ambiguity while making the added coefficients directly recoverable. The resulting twelve-node structure has forty-two independent unknown coefficients and zero diagonal. For every positive bound on the largest eigenvalue magnitude, a nonempty open set of parameters has two distinct compatible networks satisfying that bound. This holds for both real and complex coefficients. Other open regions have exactly one admissible solution. The proof uses a rational transformation exchanging the two solutions and a common limiting matrix whose fourth power vanishes. Exact rational-number certificates verify an explicit pair. The construction also extends algebraically when the unknown coordinates are whole rational transfer functions. Genericity in prescribed numerator and denominator coefficient families, and the requirement that every transfer function vanish at infinite frequency, remain separate questions.

0.23

A Walk From Free Probability to Matrix Discrepancy III: Higher Rank Kadison-Singer and Spectrally Thin Trees

cs.DS ·2026-09-18 ·Tarun Kathuria
theory similarity 0.50
Abstract · The author explicitly reserve all rights in this work. No permission is granted for the reproduction, storage, or use of this document for t…

Let $A_1,\ldots,A_N$ be positive semidefinite matrices of rank at most $r$, with $\sum_i A_i=I$ and $\norm{A_i}\le\varepsilon$. We prove that one sign can be assigned to each original matrix with discrepancy $O(\sqrt{\varepsilon\log(2r)})$, independently of their dimension and number, which is known to be optimal upto constants. We give a separate existence proof and a deterministic algorithm with polynomial work in a real-arithmetic model with semidefinite-value and exact spectral primitives. Separate Lean formalizations verify the existence proof and the algorithm in this arithmetic model. The proof extends the variational approach to discrepancy developed in the companion papers, motivated by operator-valued free interpolation, Lehner's formula, and spectral Tsallis regularization. A concave matrix power interpolates between a trace source and a sandwich source. Its concavity controls the response of the optimizing density. The walk maintains independent source reserves and a second matrix recording reserve expenditure. This matrix certifies contraction of the unfinished input mass between epochs; within an epoch, preparation and a negative-curvature step control the spectral potential while the coefficients advance toward signs. As applications, one spanning tree can be chosen simultaneously $O(\varepsilon\log(2s))$-spectrally thin for $s$ positive weightings of a common graph whose edge leverages are at most $\varepsilon$. The diagonal specialization gives discrepancy $O(\sqrt{L\log(2k)})$ for matrices with row sums at most $L$ and at most $k$ nonzero entries per column, through a walk of fractional colorings.

0.38

Discrete-time polynomial systems with inputs and outputs: structure, reachability, observability, and minimal realizations

eess.SY math.OC ·2026-09-29 ·Eduardo D. Sontag
similarity 0.50
Abstract

We study the realization problem for discrete-time input/output polynomial systems. These are formalized using tools from commutative algebra and algebraic geometry as systems whose state spaces are algebraic varieties, or more abstractly the set of k-points of an affine k-scheme, where k is an arbitrary infinite field. The input/output behaviors of such systems are described by "polynomial response maps" in which outputs are polynomial functions of past inputs. The main results show that every polynomial response map admits a canonical (quasi-reachable and algebraically observable) realization, which is unique up to isomorphism. The key to the approach is to linearize dynamics by considering a "dual" system in which states are functions defined on states. Finite dimensionality of the canonical realization, and its polynomiality, are characterized in terms of the space, algebra, and field of observables of the map, as well as in terms of algebraic input/output difference equations, and by a Jacobian rank criterion. A particular subclass consists of the maps that we call "bounded," defined by the property that their degree in the past inputs is uniformly bounded. Bounded maps are shown to be finitely realizable if and only if they are realizable by finite-dimensional state-affine systems, whose theory in turn reduces to that of rational formal power series. We also study the lattice of quasi-reachable realizations of a given map, including normal realizations. This work is an update of the PhD thesis written by the author in 1976; connections to recent work are briefly discussed in the last section.

0.38

What Should an Information Bottleneck Encoder Learn? Separating Representation Learning from Rate Control

cs.LG cs.IT ·2026-10-26 ·Hassan Tavakoli, Bella Bose
similarity 0.50
Abstract

The information bottleneck (IB) trades compression of an input $X$ against prediction of a target $Y$ through a multiplier $β=1/λ$, yet each operating point is usually served by training, or fully conditioning, a new encoder. For jointly Gaussian data, we show that the IB solution factorizes as $A(β)=D(β)W$: the information-bearing directions $W$ do not depend on $β$, and $β$ acts only through scalar gains. We ask whether this separation of \emph{representation} from \emph{rate control} survives beyond Gaussian models, treating the budget as a given operating parameter rather than a quantity to be optimized. For a budget-free representation \(F(X)\), the \emph{separation gap} is controlled by the \emph{sufficiency deficit} $ε_F=I(X;Y\mid F(X))$: We show there is at least $ε_F$ in the worst case for finite inputs, and it is bounded uniformly in $λ$ by a function of $ε_F$ for finite label alphabets. The same certificate extends to a new target distribution and to pooled multi-domain data, where a conditional-alignment term appears. On MNIST, SVHN, CIFAR-10 and STL-10 (10 seeds, eight prespecified budgets), a frozen CNN representation with one compressor--decoder pair per budget reduces estimated training multiply--accumulate operations (MACs) by $7.7\times$ and wall-clock time by about $2\times$ relative to end-to-end VIBs, and attains higher budget-averaged accuracy ($65.6\%$ versus $57.3\%$ per domain; $67.1\%$ versus $57.1\%$ pooled), with the advantage concentrated at strong compression, where the VIBs collapse. At weak compression ($λ\le0.036$), VIB has higher cross-entropy and ECE; its surrogate Lagrangian is higher per domain, indistinguishable from the pooled VIB in the macro average, and lower on the pooled test set.

0.10