CS Paper Radar

A One-Sided McDiarmid-Type Inequality for Heavy Tails via Reverse KL Tensorization

Hanuka Eshima, Amedeo Roberto Esposito, Andrea Paudice

cs.IT math.STmath.PRstat.ML ·submitted 2026-10-15 02:06 UTC · v1

Abstract

We revisit McDiarmid's inequality, a central tool in statistical learning theory and algorithm analysis. We show that large deviations above the mean are exponentially unlikely for functions of independent random variables that are almost surely bounded above and satisfy a conditional moment assumption of order $p\in(1,2]$. When $p<2$, the result applies even when the function has infinite variance. Our proof relies on a new tensorization inequality for the Kullback--Leibler divergence, termed reverse KL tensorization, which may be of independent interest. Applications include concentration of the sample mean and median-of-means estimator, and one-sided uniform risk bounds for unbounded losses.

Closest papers

Nearest neighbours by meaning across the whole index.

Distribution-Dependent Improvements of McDiarmid's Inequality via Rényi Divergence

cs.IT math.PR ·2026-10-13 ·Amedeo Roberto Esposito, Adrien Vandenbroucque, Michael Gastpar
similarity 0.68
Abstract

We investigate whether Rényi divergence can sharpen McDiarmid's inequality and its distribution-dependent refinement induced by KL divergence. Starting from Marton's transportation proof, we show that its transport inequality fails in general when KL divergence is replaced by Rényi divergence of order $0<α<1$. For sums of independent bounded random variables, however, the additivity of Rényi divergence allows us to adapt the argument and derive concentration bounds. For Bernoulli sums, these bounds strictly improve on the distribution-dependent bound induced by KL divergence over a range of deviation thresholds. We also establish a distribution-dependent refinement of Gilardoni's inequality relating total variation to Rényi divergence, providing explicit constants for our general concentration bounds.

0.30

A Complete Characterization of Tensorizable $f$-divergences

cs.IT math.PRmath.ST ·2026-08-28 ·Rodrigo Cruz, Flavio P. Calmon, Qian Yu
similarity 0.66
Abstract

Csiszar's formulation of the $f$-divergence introduced a vast family of functionals for quantifying dissimilarity between probability distributions. However, many applications in statistics and information theory rely only on a few $f$-divergences, such as the Kullback-Leibler divergence, the $χ^2$-divergence, and the squared Hellinger distance. These divergences are especially useful because they admit simple compositional formulas under product measures, a property sometimes referred to as tensorization. In this work, we refine a formalism of tensorization previously introduced in the literature. Then, we show that any possible tensorization formula has a multi-affine form characterized by a single parameter, and identify all tensorizable $f$-divergences under our adopted notion of tensorization.

0.45

Contraction of Rényi Divergences for Discrete Channels

cs.IT math.PR ·2026-09-28 ·Adrien Vandenbroucque, Amedeo Roberto Esposito, Michael Gastpar
highly cited similarity 0.63
Abstract

We investigate Strong Data-Processing Inequality (SDPI) constants for Rényi Divergences on finite spaces. We study their dependence on the Rényi order $α$, proving that they are non-decreasing and that their scaling by $(α-1)$ is convex for $α\geq1$. We also identify several support restrictions on the probability measures involved in determining these constants. In particular, in the distribution-independent setting, measures supported on a common set of at most two points suffice to evaluate the Rényi-SDPI constant. For $α\in[0,1]$, we further prove equality with the $χ^2$-SDPI constant, while at order infinity we obtain a closed-form expression. In order to link contraction over product spaces to contraction along individual coordinates, we provide tensorisation bounds for arbitrary product channels analogous to those known for $\varphi$-Divergences. At finite orders, the Rényi-SDPI constants are bounded above and below through comparisons with the $χ^2$-Divergence and Hellinger Divergences, with sharpness established in multiple cases. At order infinity, we instead relate these constants to the contraction of Total Variation Distance. Finally, our findings are applied to local differential privacy (LDP) and the analysis of Markov chains. This yields sharp contraction guarantees for pure-LDP mechanisms, and connects Rényi-LDP to Rényi-SDPI constants. For Markov chains, we derive finite-time convergence bounds and exhibit a family of chains for which Rényi-SDPIs improve on classical $χ^2$-based bounds by arbitrarily large factors.

0.34

Maximal correlation under cardinality constraints

cs.IT ·2026-08-17 ·Dror Drach, Tomer Berg, Or Ordentlich, Ofer Shayevitz
theory similarity 0.60
Abstract

In this paper, we define and analyze the quantized maximal correlation, an extension of the notion of maximal correlation restricted to functions taking values in sets of bounded cardinality. We derive an upper bound on the quantized maximal correlation by showing that the correlation between any quantized functions of $X$ and $Y$ is related to the MMSE distortion in quantization of a particular linear combination of random variables. Following this, we leverage rate-distortion techniques and anti-concentration inequalities to further bound this MMSE, which results in explicit bounds on the quantized maximal correlation. Unlike the quantized maximal correlation itself, which does not generally tensorize, our bounds on the mean squared error do tensorize, resulting in a dimension-free upper bound on the quantized maximal correlation for product distributions. Our results also lead to improved bounds on the isoperimetric constants of reversible Markov chains and product chains, strengthening classical results such as those by Alon and Milman.

0.31

Generalization Bounds via Vector-Valued $L_p$ Correlation $V_α$

cs.LG cs.IT ·2026-10-21 ·Mohammad Hossein Momeni Hamaneh, Mohammad Mahdi Mojahedian, Mohammad Hossein Yassaee
similarity 0.57
Abstract

Understanding generalization of randomized learning algorithms remains a central challenge. Mutual-information and total-variation bounds provide complementary ways to quantify data--hypothesis dependence, but they can behave quite differently in high-dimensional regimes. We study a one-parameter family of dependence measures, the vector-valued $L_p$-norm correlation $V_α$, and derive generalization bounds whose dependence terms interpolate between total variation (TV) at $α=1$ and variance and $χ^2$ related behavior at $α=2$. For $α>2$, the Hölder-conjugate loss moment is below two, yielding heavy-tailed specializations that can remain finite even when the loss variance is infinite. A particularly interpretable specialization occurs at $α=2$, where the bound can be expressed in terms of the variability of the algorithm's output density across datasets, $\int \sqrt{\mathrm{Var}_S[p(w\mid S)]}~ dw$. We show that the entire $V_α$ family is invariant under smooth bijective reparameterizations of the hypothesis space. We also introduce an adaptive density stability (ADS) condition that implies explicit control of $V_2$, and we verify it for Gibbs-type posteriors under bounded losses (yielding rates that depend on the inverse temperature and sample size). To address scaling, we extend the framework to a conditional formulation using supersamples, analogous in spirit to conditional mutual information (CMI), and discuss evaluations in the output space as a practical surrogate for estimating dependence. Simulations on Bayesian linear regression illustrate how tuning $α$ can change the tightness of the resulting bounds across regimes. Additional supersample experiments on noisy MNIST and on a pretrained ResNet-50 for CIFAR-10 show similar behavior of the projected loss-space bounds beyond the original clean-MNIST setting.

0.32

Statistical-to-computational gaps in central limit theorems for high-dimensional tensors

cs.CC math.STmath.PR ·2026-11-11 ·Debsurya De, Dmitriy Kunisky
theory similarity 0.57
Abstract · 44 pages, 1 figure

We study high-dimensional central limit theorems (CLTs) as hypothesis testing problems, comparing the number of summands needed for statistical and computational indistinguishability of the normalized sum from the limiting Gaussian distribution as the dimension and number of summands grow together. For the off-diagonal entries of sums of i.i.d. rank-one tensors formed from standard Gaussian vectors (equivalently, the problem of distinguishing Wigner from Wishart tensors), we prove total variation bounds sharpening previous estimates of Mikulincer (2020) and give matching polynomial-time tests. This shows that the statistical and computational thresholds coincide for this CLT, contrary to a proposal by Kunisky, Moore, and Wein (2024). On the other hand, we also show that, for tensors of order at least three, sufficient Gaussian dilution of the summands by adding independent Gaussian noise and rescaling gives a CLT model with a statistical-to-computational gap (in the above setting and ones with more general rank-one summand distributions). To the best of our knowledge, such a phenomenon has not been observed before in any high-dimensional CLT. In particular, in the relevant parameter regimes, we show that a brute-force search achieves strong detection, while polynomials of degree up to $n^{Ω(1)}$ cannot achieve strong separation. The latter lower bounds control low-degree $χ^2$-divergences by a novel method based on cumulant generating functions, and we further illustrate the use of these tools by applying them to masked tensor CLT models where only a prescribed subset of tensor entries is revealed.

0.26

Matrix $Φ$-Entropic Measures of Dependence

cs.IT ·2026-10-21 ·Chenyu Wang, Amin Gohari
similarity 0.56
Abstract

We introduce matrix $Φ$-ribbons, extending the scalar $Φ$-ribbons to the matrix case. For $Φ$ in the subadditivity class, we establish data processing and tensorization properties of matrix $Φ$-ribbons. We establish scalar and matrix $Φ$-entropy inequalities and use them to show that both $Φ$-ribbons characterize the corresponding $Φ$-strong data processing inequality (SDPI) constants. We obtain exact evaluations of the SDPI constants for several sources. We provide examples showing that the matrix $Φ$-SDPI constant can exceed the scalar counterpart. Finally, we derive inner bounds for $Φ$-ribbons from prescribed independence structures and prove that these bounds are optimal when $Φ(t)=t\log t$.

0.25

Finite-Sample Binary Hypothesis Testing via Rényi Divergences: Strong Converse and Local Privacy

cs.IT math.ST ·2026-09-23 ·Roberto Bruno, Adrien Vandenbroucque, Amedeo Roberto Esposito
published similarity 0.55
Abstract · Extended version of a paper presented at the 2026 IEEE International Symposium on Information Theory (ISIT). Submitted to the IEEE Transacti…

We study asymmetric simple binary hypothesis testing between $H_0:P_0^{n}$ and $H_1:P_1^{n}$, based on $n$ independent and identically distributed observations. Leveraging a variational representation of Rényi divergence of order $α$, we derive our main result: a finite-sample converse with $α>1$. The bound uses both directions of the divergence $D_α(P_1\|P_0)$ and $D_α(P_0\|P_1)$, tensorises under product measures, and contains familiar data-processing converses as boundary cases. For comparison, we apply the same variational approach to general $f$-divergences and specialise it to total variation, $E_γ$, Hellinger, and Kullback Leibler divergences, thereby recovering familiar converses within a unified framework. Together with an achievability bound involving Rényi divergence with $α\in (0,1)$, the main converse recovers the phase transition of the optimal Type II error under the exponentially decaying Type I error constraint $\varepsilon_n=e^{-nr}$. Under regularity conditions, the optimal Type II error vanishes exponentially when $r<D(P_1\|P_0)$ and converges exponentially fast to one when $r>D(P_1\|P_0)$. We also derive sample-complexity bounds and extend both the converse and achievability analyses to locally differentially private observations, quantifying the cost of privacy and recovering the non-private achievability bound as the privacy constraint vanishes.

0.34