CS Paper Radar

Finite-Sample Binary Hypothesis Testing via Rényi Divergences: Strong Converse and Local Privacy

Roberto Bruno, Adrien Vandenbroucque, Amedeo Roberto Esposito

cs.IT math.ST ·submitted 2026-09-23 09:41 UTC · v1
published

Abstract

We study asymmetric simple binary hypothesis testing between $H_0:P_0^{n}$ and $H_1:P_1^{n}$, based on $n$ independent and identically distributed observations. Leveraging a variational representation of Rényi divergence of order $α$, we derive our main result: a finite-sample converse with $α>1$. The bound uses both directions of the divergence $D_α(P_1\|P_0)$ and $D_α(P_0\|P_1)$, tensorises under product measures, and contains familiar data-processing converses as boundary cases. For comparison, we apply the same variational approach to general $f$-divergences and specialise it to total variation, $E_γ$, Hellinger, and Kullback Leibler divergences, thereby recovering familiar converses within a unified framework. Together with an achievability bound involving Rényi divergence with $α\in (0,1)$, the main converse recovers the phase transition of the optimal Type II error under the exponentially decaying Type I error constraint $\varepsilon_n=e^{-nr}$. Under regularity conditions, the optimal Type II error vanishes exponentially when $r<D(P_1\|P_0)$ and converges exponentially fast to one when $r>D(P_1\|P_0)$. We also derive sample-complexity bounds and extend both the converse and achievability analyses to locally differentially private observations, quantifying the cost of privacy and recovering the non-private achievability bound as the privacy constraint vanishes.

Comment
Extended version of a paper presented at the 2026 IEEE International Symposium on Information Theory (ISIT). Submitted to the IEEE Transactions on Information Theory

Closest papers

Nearest neighbours by meaning across the whole index.

A local-to-global principle for $f$-divergence homogenization

cs.IT math.PR ·2026-10-22 ·Aryeh Kontorovich
similarity 0.68
Abstract

We study the effect of replacing a family of independent statistical experiments by independent repetitions of their average. We give a simple, complete characterization of the $f$-divergences for which this operation increases divergence by at most a constant depending only on the generator, uniformly in the number of observations and the underlying measurable space. The characterization is governed by a local-to-global principle: a comparison restricted to two observations on a binary alphabet already determines the validity of the general inequality. It admits an explicit scalar formulation, which simplifies further for symmetric divergences. The result applies to arbitrary convex generators and requires no differentiability or symmetry assumptions. It recovers the homogenization inequality for total variation and identifies exactly the admissible members of the power-divergence family. In particular, whenever uniform homogenization fails, the obstruction is already present in a family of binary experiments with two observations. Moreover, the global comparison constant is controlled uniformly over generators with a common bound in the local inequality.

0.22

Finite Sample Bounds for Composite Hypothesis Testing

cs.IT ·2026-08-28 ·Elías Vera-Sigüenza, Amedeo Roberto Esposito
similarity 0.65
Abstract · 5 figures, 34 pages

We investigate composite binary hypothesis testing in the finite sample regime under asymmetric error constraints. Using Rényi divergences, we derive explicit achievability and converse bounds for the optimal Type II error. When the Type I error is constrained to decay exponentially with sample size, the bounds identify a phase transition and yield a strong converse above it. In the composite problem, the phase transition threshold is given by the joint KL projection over the alternative and null classes. Achievability is obtained through a joint Rényi projection whose log likelihood ratio defines a single test with uniform error control over both hypothesis classes, without requiring the projected pair to be least favourable. For compact convex classes with full support on a finite alphabet, we determine the exact error exponents on both sides of the transition and show that the achievable exponent is attained at a unique Rényi order. The same framework recovers the fixed Type I composite Chernoff--Stein exponent and yields a polynomial refinement of the finite sample achievability result. We further identify conditions under which the projected pair is least favourable at finite sample size.

0.13

On the Estimation of Chernoff Information

cs.IT ·2026-08-18 ·Kadircan Aksoy, Peter Jung
theory similarity 0.64
Abstract

Chernoff information is a fundamental divergence measure characterizing the optimal error exponent in Bayesian binary hypothesis testing, with applications in information fusion, time-series analysis, and statistical learning theory. However, closed-form expressions exist only for simple parametric families, and nonparametric estimation remains difficult because the quantity is defined as an optimization of the unnormalized Rényi divergence over its order. We reformulate this optimization via a derivative condition, whose zero locates the optimal mixture parameter, and estimate the derivative directly using a $k$-nearest-neighbor method. We prove the $L_2$-consistency of the derivative estimator under mild regularity conditions on the densities and their domain. Coupled with a bisection procedure that locates the optimal parameter up to arbitrary precision, this yields an estimator for Chernoff information.

0.24

Second-Order Expansion of Privacy Amplification Under f-Divergence Criteria

cs.IT ·2026-09-10 ·Mario Berta, Hao-Chung Cheng, Marco Tomamichel
similarity 0.64
Abstract

We derive the second-order asymptotics of randomness extraction from memoryless sources with side information under security criteria based on a broad class of Csiszàr f-divergences, treating both a fixed reference side-information marginal and optimization over that marginal. The conditional varentropy decomposes into fluctuations of the conditional entropy across different values of the side information and the average variance of the conditional surprisal for each value. Without marginal optimization, these contributions yield a Gaussian-mixture second-order profile. With marginal optimization, they combine into the total conditional varentropy, yielding a single Gaussian profile. As corollaries, we obtain second-order expansions for Rényi-entropy criteria of all orders $α\in (0,1)$ and recover the known expansion for total variation distance.

0.30

Contraction of Rényi Divergences for Discrete Channels

cs.IT math.PR ·2026-09-28 ·Adrien Vandenbroucque, Amedeo Roberto Esposito, Michael Gastpar
highly cited similarity 0.64
Abstract

We investigate Strong Data-Processing Inequality (SDPI) constants for Rényi Divergences on finite spaces. We study their dependence on the Rényi order $α$, proving that they are non-decreasing and that their scaling by $(α-1)$ is convex for $α\geq1$. We also identify several support restrictions on the probability measures involved in determining these constants. In particular, in the distribution-independent setting, measures supported on a common set of at most two points suffice to evaluate the Rényi-SDPI constant. For $α\in[0,1]$, we further prove equality with the $χ^2$-SDPI constant, while at order infinity we obtain a closed-form expression. In order to link contraction over product spaces to contraction along individual coordinates, we provide tensorisation bounds for arbitrary product channels analogous to those known for $\varphi$-Divergences. At finite orders, the Rényi-SDPI constants are bounded above and below through comparisons with the $χ^2$-Divergence and Hellinger Divergences, with sharpness established in multiple cases. At order infinity, we instead relate these constants to the contraction of Total Variation Distance. Finally, our findings are applied to local differential privacy (LDP) and the analysis of Markov chains. This yields sharp contraction guarantees for pure-LDP mechanisms, and connects Rényi-LDP to Rényi-SDPI constants. For Markov chains, we derive finite-time convergence bounds and exhibit a family of chains for which Rényi-SDPIs improve on classical $χ^2$-based bounds by arbitrarily large factors.

0.34

Contraction and Statistical Inference under Privacy for Uniformly Bounded Distributions

cs.IT cs.CRmath.ST ·2026-09-23 ·Leonhard Grosse, Sara Saeidian, Tobias J. Oechtering, Mikael Skoglund
similarity 0.63
Abstract

We investigate $c$-interior pointwise maximal leakage (PML) as a tool for contraction analyses and disclosure control. Based on the strong adversarial threat models from maximal leakage, $c$-interior PML generalizes local differential privacy (LDP) to data-generating distributions with densities uniformly bounded away from zero by $c>0$. Viewing $c$-interior PML as an algebraic constraint on a kernel yields more flexible (and often tighter) contraction analyses than standard LDP. We provide tight bounds on the Dobrushin coefficient, and bound the contraction coefficient of the Hockeystick-divergence. We further derive strong data processing inequalities on $f$-divergences under $c$-interior PML constraints when the input distributions to the divergence are restricted to be in the $c$-interior. These results extend beyond the regime of pure LDP to cover a larger class of kernels, including, e.g., arbitrary stochastic matrices. We apply the results to minimax theory and provide asymptotically optimal strategies under $c$-interior PML constraints for binary hypothesis testing and mean estimation. The results show that disclosure control with PML allows analysts to reason about systems in a more differentiated manner: For example, it allows us to quantify the privacy leakage of deterministic systems, and can give precise adversarial guarantees with respect to arbitrary distributional assumptions. Interestingly, a recurring theme in the disclosure analyses is that if the privacy problem is relatively regular (if the density bound $c$ is large), private inference can be possible without incurring any additional cost in terms of sample complexity.

0.17

Classical and Quantum Information Measures Characterizing Convertibility: An Axiomatic Approach

cs.IT quant-phmath-phmath.PR ·2026-10-12 ·Frits Verhagen
similarity 0.62
Abstract · PhD thesis at the Centre for Quantum Technologies, National University of Singapore; contains corrections of some minor errors appearing in …

In this PhD thesis, we take an axiomatic approach to studying multivariate divergences, which are measures of the information content of a tuple of classical or quantum states. These divergences are studied in the context of large-sample and catalytic convertibility of tuples of states. This type of convertibility is characterized by the ordering of an axiomatically defined family of divergences. In this thesis, we identify this family of divergences explicitly for two scenarios, one classical and one quantum. In the classical setting, we show that the relevant divergences are certain generalizations of the bivariate Rényi divergences. Unlike previous studies, we do not require the distributions to have the same support, yielding interesting new divergences. In the quantum setting, we consider pairs of classical-quantum states with pure component states, and find that majorization is characterized by all $α$-$z$ relative entropies for $α<1$. This demonstrates the first operational interpretation of these relative entropies.

0.15

On Inherent Privacy of Posterior Sampling: A Unified Rényi-Divergence Framework

cs.LG stat.MLmath.ST ·2026-10-14 ·Ruyu Zhou, Fang Liu
top venue similarity 0.61
Abstract · Accepted to NeurIPS 2026

Differential Privacy (DP) provides a rigorous framework for quantifying privacy guarantees. While most methods achieve DP by injecting calibrated random noise, the intrinsic randomness of certain procedures such as sampling can yield DP guarantees ``for free''. In this work, we develop a unified Rényi divergence framework to characterize the inherent DP guarantees of sampling from a Bayesian posterior distribution. We show that the privacy loss incurred by One-Posterior-Sampling (OPS) is governed by posterior exponential moments of the single-record log-likelihood ratio, and establish explicit DP guarantees under uniformly bounded, sub-Gaussian, or sub-exponential tail regimes. We apply the framework to representative Bayesian models, including categorical likelihoods with arbitrary priors, Gaussian-Gaussian conjugate pair, generalized linear models from the exponential family and linear regression with Gaussian likelihood, both with Gaussian priors on the regression coefficients. Our results elucidate sample size, model structure, neighboring relations in DP (bounded or unbounded), and how prior if applicable, jointly determine the inherent DP guarantees of OPS. Furthermore, our results recover the existing DP guarantees for OPS as special cases -- often with tighter privacy loss bounds -- and substantially broaden the class of Bayesian models for which the inherent DP guarantees of OPS can be rigorously characterized.

0.27