CS Paper Radar

Multiplicative comparisons of Rényi entropies for weighted Bernoulli sums

Jiange Li

cs.IT math.PRmath.CO ·submitted 2026-09-01 16:57 UTC · v2
theory

Abstract

We establish multiplicative comparisons between Rényi entropies of different orders for weighted sums of independent Bernoulli random variables. In particular, we prove a logarithmic comparison between the zeroth- and infinity-order Rényi entropies, which yields a polynomial improvement over the square-root bound of Jain, Sah, and Sawhney. As an application, this leads to an improved parameterized running time for the randomized bin-packing algorithm of Nederlof, Pawlewicz, Swennenhuis, and Wȩgrzycki. We also obtain explicit dimension-free, constant-factor comparisons between Rényi entropies of positive orders.

Closest papers

Nearest neighbours by meaning across the whole index.

Settling the Sample Complexity of Rényi Entropy Estimation

cs.IT cs.DSmath.ST ·2026-10-07 ·Qisheng Wang
theory similarity 0.61
Abstract · 23 pages, 1 table

Rényi entropy estimation has been comprehensively investigated by Acharya, Orlitsky, Suresh and Tyagi (SODA 2015; IEEE Trans. Inf. Theory 2017) and consequent works, whereas only the sample complexity of Rényi entropy estimation of integer order has been settled. In this paper, we settle the sample complexity of Rényi entropy estimation of noninteger order, thereby completing the complexity picture of Rényi entropy estimation. Specifically, we show that for any noninteger $α> 0$, it is sufficient and necessary to use \[ Θ\!\left(\frac{d^{\max\{1/α,1\}}}{\varepsilon^{1/α}\log(d)} + \frac{d^{|1-1/α|}}{\varepsilon^2}\right) \] samples to estimate the Rényi entropy of order $α$ of an unknown discrete distribution over an alphabet of size $d$ to within additive error $\varepsilon$. For the upper bound, we reduce the bias using a refined polynomial approximation estimator for large probabilities. For the lower bound, we employ a different hard instance equipped with a new moment matching construction. The constructive moment matching has constant bounded high-order moments, while attaining a fixed ratio between the $α$-th moments, which is of independent interest.

0.20

Distribution-Dependent Improvements of McDiarmid's Inequality via Rényi Divergence

cs.IT math.PR ·2026-10-13 ·Amedeo Roberto Esposito, Adrien Vandenbroucque, Michael Gastpar
similarity 0.59
Abstract

We investigate whether Rényi divergence can sharpen McDiarmid's inequality and its distribution-dependent refinement induced by KL divergence. Starting from Marton's transportation proof, we show that its transport inequality fails in general when KL divergence is replaced by Rényi divergence of order $0<α<1$. For sums of independent bounded random variables, however, the additivity of Rényi divergence allows us to adapt the argument and derive concentration bounds. For Bernoulli sums, these bounds strictly improve on the distribution-dependent bound induced by KL divergence over a range of deviation thresholds. We also establish a distribution-dependent refinement of Gilardoni's inequality relating total variation to Rényi divergence, providing explicit constants for our general concentration bounds.

0.30

The binomial distribution maximises entropy among sums of negatively related indicators with fixed mean and support

cs.IT math.PR ·2026-10-19 ·Fraser Daly
similarity 0.58
Abstract · 10 pages

We show that the binomial distribution supported on $\{0,1,\ldots,n\}$ and with mean $np$ for some $0<p<1$ maximises entropy within the class of sums of negatively related indicator random variables with this mean and support. Our results generalise the main result of Yu (2008), who showed that this binomial distribution maximises entropy within the smaller class of ultra-log-concave random variables of order $n$. Our proof makes use of a Markov chain construction of Yu (2008), together with a stochastic ordering argument.

0.25

Sharp Rogers--Shephard Inequalities for Rényi Entropy

cs.IT math.PRmath.MG ·2026-10-11 ·Bang-Xian Han, Guoqing Zhan, Chiyu Zhou, Zhuo-Nan Zhu
theory similarity 0.56
Abstract · 29 pages; comments are welcome!

We establish sharp weighted Rogers--Shephard inequalities for Rényi differential entropy. For independent copies $X,X'$ of a log-concave random vector in $\R^n$, we prove $h_p(aX-bX')\le h_p(X)+n\log(a+b)$ for all $a,b>0$ and $1\le p\le\infty$, where orders $1$ and $\infty$ correspond to Shannon entropy and min-entropy, respectively. The Shannon case settles the entropic Rogers--Shephard conjecture of Madiman and Kontoyiannis and its higher-dimensional formulation. For $1\le p<\infty$, equality holds precisely for invertible affine images of random vectors with independent exponential coordinates. At $p=\infty$, the larger equality class is characterized by possibly extended-valued convex potentials that are positively homogeneous of degree one. For every $0<p<1$, uniform laws on sufficiently high-dimensional simplices violate the unweighted bound. Thus $p=1$ is the critical order for this bound to hold in every dimension. The proof combines weighted convex intersections with an exponential lift to a log-affine probability density. At the Shannon endpoint, this lift yields an exact deficit decomposition with a conditional mutual information term for an additive channel with asymmetric Laplace noise.

0.43

Nearly Sample-Optimal Estimators for Quantum Rényi and Tsallis Entropies

cs.IT quant-ph ·2026-08-18 ·Kean Chen, Qisheng Wang
similarity 0.56
Abstract · 32 pages, 1 table, 4 algorithms

In this paper, we provide estimators for quantum Rényi and Tsallis entropies with nearly optimal sample complexity. Specifically, for order $α$, dimension $d$, and additive error $\varepsilon$, 1. For $0 < α< 1$, the sample complexity is $O(d^{1+1/α}/\varepsilon^{1/α} + d^{1/α-1}/\varepsilon^{2})$ for Rényi entropy and $O(d^{1+1/α}/\varepsilon^{1/α} + d^{2-2α}/\varepsilon^2)$ for Tsallis entropy. In particular, for $0 < α\leq 1/2$, the sample complexity for both entropies is $O(d^{1+1/α}/\varepsilon^{1/α})$. 2. For non-integer $α> 1$, the sample complexity is $O(d^2/\varepsilon^{1/α} + d^{1-1/α}/\varepsilon^2)$ for Rényi entropy. Our upper bounds improve the quantum Rényi entropy estimators due to Acharya, Issa, Shende, and Wagner (2017) and the quantum Tsallis entropy estimators due to Chen, Liu, and Wang (2026), and match the lower bounds recently established by Wang (2026).

0.51

Optimal Rényi and Logarithmic Additive Functional Entropy Estimation

cs.IT cs.CCeess.SPmath.ST ·2026-10-20 ·Arman Adibi, Piotr Krysta
theory similarity 0.55
Abstract

We characterize the minimax sample complexity of estimating Rényi entropy from independent samples on an alphabet of size at most $k$, when the known order $α>1$ may vary with $k$. For every fixed additive accuracy $\varepsilon\in(0,1/2)$ and success probability $2/3$, we prove matching bounds with constants independent of $k$ and $α$. For $1<α<\log k$, the rate is $Θ_\varepsilon(αk^{1-1/α}X_α(k,\varepsilon))$, where a monotone scalar inequality defines $X_α$ at noninteger orders. The bounds remain uniform as the order approaches one or an integer, or grows with the alphabet. Integer orders below $\log k$ have rate $Θ_\varepsilon(αk^{1-1/α})$; orders at least $\log k$ and min-entropy have rate $Θ_\varepsilon(k\log k)$. We extend the analysis to logarithms of positive additive functionals. For every fixed continuous $f$ satisfying $f(0)=0$, $f(x)>0$ for $x>0$, and $f(x)\sim ax^r$ at zero, with $a>0$ and $r>1$, estimating $-β^{-1}\log\sum_i f(p_i)$ at fixed accuracy, for fixed $β>0$, has sample complexity $Θ(k^{1-1/r})$ for integer $r$ and $Θ(k/\log k)$ otherwise. A replication reduction preserves the sampling experiment and transfers the rates for powers. Smooth flat functions show that regularity alone does not determine the complexity. The upper bound combines polynomial approximation, Gamma count statistics, and an independent pilot, with a global affine correction near order one. The lower bound uses interpolation to construct positive mean-one priors and controls their distinguishability and entropy separation after normalization. Exact count-variance identities and weighted approximation duality connect the two arguments.

0.17

Improved Bounds on the Ingleton Score of Four Random Variables

cs.IT ·2026-10-04 ·Jiang Chang
theory similarity 0.55
Abstract · 5 pages. Ancillary files contain the parsed inequality catalogues, exact rational certificates for the bounds 5/32 and 19/124, exact extensi…

For four finite discrete random variables, we prove that the negative of the Ingleton expression, divided by the joint entropy, is at most $19/124 \approx 0.15323$, improving the previously reported upper bound $3/19$ by $11/2356$. The proof uses a copy program with four copy steps and six auxiliary variables, certified by exact rational linear-programming certificates. We also show that two published information inequalities, due to Dougherty-Freiling-Zeger and Csirmaz, imply the bound $5/32$ by an explicit nonnegative combination with Shannon inequalities, and that $5/32$ is the exact optimum both of the union of the three published inequality catalogues considered here and of the full copy-lemma extensions of all 213 distinct copy programs occurring in them. The new program is not among these programs, and the bound $19/124$ remains optimal when all of them and all catalogue inequalities are added. Ancillary files provide the parsed catalogues, exact primal and dual certificates, exact extension certificates, and verifiers that use only the Python standard library.

0.27

Relative Entropy under Ordered Chi-Squared Constraints: Exact Bounds and Extremizers

cs.IT ·2026-11-18 ·Igal Sason
theory similarity 0.54
Abstract

We derive a sharp upper bound on the relative entropy $D(P\|Q)$ in terms of the two ordered chi-squared divergences $χ^2(P\|Q)$ and $χ^2(Q\|P)$. For any $A,B>0$, the constraints $χ^2(P\|Q)\leq A$ and $χ^2(Q\|P)\leq B$ yield upper bounds on $D(P\|Q)$ and $D(Q\|P)$. Two pairs of Bernoulli distributions each attain both bounds simultaneously. As a consequence, for sequences of probability mass functions $\{P_n\}$ and $\{Q_n\}$ on a common finite or countably infinite alphabet, both $D(P_n\|Q_n)$ and $D(Q_n\|P_n)$ tend to zero whenever one of the two ordered chi-squared divergences tends to zero and the other remains uniformly bounded. Without this boundedness assumption, $D(P_n\|Q_n)$ can fail to converge to zero even when $χ^2(Q_n\|P_n)\to 0$. Finally, in contrast to the attainable upper bounds, on any finite or countably infinite alphabet containing at least three elements, the infimum of $D(P\|Q)$ under the exact constraints $χ^2(P\|Q)=A$ and $χ^2(Q\|P)=B$ is zero and is not attained.

0.35