CS Paper Radar

Thermodynamic Formalism for Non-Invariant Probabilities: Entropy, Divergence, Channels

Artur O. Lopes, Elismar R. Oliveira

cs.IT math.DScond-mat.stat-mechmath.PR ·submitted 2026-11-10 11:00 UTC · v1

Abstract

Let $\mathcal{M}$ and $\mathcal{M}_σ$ denote, respectively, the sets of Borel and $σ$-invariant probabilities on the uncountable set $Ω=\{1,2,...,d\}^\mathbb{N}$. Given a Hölder Jacobian $J$, the dual Ruelle operator $μ\in \mathcal{M} \mapsto \mathcal{L}_{J}^* (μ)$ will play the role of a dynamical channel; generalizes the action of a stochastic matrix on probability vectors. Since $\mathcal{L}_{J}^*$ generally breaks $σ$-invariance, we introduce the \emph{ni-entropy} $h_{ni}(μ)$ for non-invariant measures, employing the concept of the expected value function on cylinders of size one. $h_{ni}(μ)$ differs from Shannon entropy but extends the Kolmogorov-Sinai entropy from $\mathcal{M}_σ$ to $\mathcal{M}$; $h_{ni}$ is concave and u.s.c. This leads us to the concept of ni-pressure $\mathcal{P}(A)$ of a potential $A:Ω\to \mathbb{R}$. $μ\in \mathcal{M}$ is not a density with respect to a dominating measure. We define an {\it ni-Kullback-Leibler divergence} $D_{ni}(ρ|| ν)$ not assuming absolute continuity. We investigate concepts of Information Theory such as Minimal Output Entropy, Donsker-Varadhan variational principle, a variational formula for $h_{ni}$, bounds for divergence, information projections, and their connections to the ni-pressure $\mathcal{P}$.

Closest papers

Nearest neighbours by meaning across the whole index.

A Structural Characterization of Entropy Functionals

cs.IT math.ST ·2026-08-14 ·Daniel Lazarev
similarity 0.57
Abstract · 31 pages, 1 figure

Entropy functionals and their associated divergences underlie many statistical methods, including maximum entropy inference, minimum divergence estimation, and goodness-of-fit testing, yet choosing among Shannon, Rényi, Tsallis, and more general entropies is often a matter of convention rather than structural principle. We introduce a measure theoretic framework in which admissibility requires the entropy of an input measure to be bounded above by that of its reference measure whenever the former is absolutely continuous with respect to the latter. Under generalized mean-value composition, we characterize all such entropy functionals and obtain a four-level hierarchy determined successively by the mean generator, entropy scale, and additivity assumptions. A continuous strictly monotone generator $g$ is admissible exactly when $t\mapsto g(1/t)$ is strictly convex for increasing $g$, or strictly concave for decreasing $g$. This resolves a question posed by Rényi (Proc. 4th Berkeley Sympos. Math. Statist. Prob., 1961) concerning which generalized means may replace the arithmetic mean in his entropy axiomatization. The same criterion is equivalent to strict convexity of an associated Csiszár $f$-divergence generator and therefore yields data processing under Markov kernels with an exact equality condition. Within this hierarchy, product additivity singles out the Rényi family, while internal additivity, or product additivity together with arithmetic mean-value composition, singles out Shannon entropy. The characterization is constructive and yields new admissible entropy and divergence families, including integral-transform examples.

0.10

Sharp Rogers--Shephard Inequalities for Rényi Entropy

cs.IT math.PRmath.MG ·2026-10-11 ·Bang-Xian Han, Guoqing Zhan, Chiyu Zhou, Zhuo-Nan Zhu
theory similarity 0.56
Abstract · 29 pages; comments are welcome!

We establish sharp weighted Rogers--Shephard inequalities for Rényi differential entropy. For independent copies $X,X'$ of a log-concave random vector in $\R^n$, we prove $h_p(aX-bX')\le h_p(X)+n\log(a+b)$ for all $a,b>0$ and $1\le p\le\infty$, where orders $1$ and $\infty$ correspond to Shannon entropy and min-entropy, respectively. The Shannon case settles the entropic Rogers--Shephard conjecture of Madiman and Kontoyiannis and its higher-dimensional formulation. For $1\le p<\infty$, equality holds precisely for invertible affine images of random vectors with independent exponential coordinates. At $p=\infty$, the larger equality class is characterized by possibly extended-valued convex potentials that are positively homogeneous of degree one. For every $0<p<1$, uniform laws on sufficiently high-dimensional simplices violate the unweighted bound. Thus $p=1$ is the critical order for this bound to hold in every dimension. The proof combines weighted convex intersections with an exponential lift to a log-affine probability density. At the Shannon endpoint, this lift yields an exact deficit decomposition with a conditional mutual information term for an additive channel with asymmetric Laplace noise.

0.43

Regularized barycentric Rényi divergences

cs.IT quant-phmath-phmath.FA ·2026-09-15 ·Milán Mosonyi
similarity 0.56
Abstract · 27 pages. v5: Improved presentation, slightly stronger results

Barycentric Rényi divergences were introduced in [Mosonyi, Bunth, Vrana, Linear Algebra and its Applications, 2024] as an alternative to standard Kubo-Ando constructions to define multivariate quantum Rényi divergences. They are defined via a variational expression and depend on a finite collection of quantum relative entropies $D^{q_x}$. When all the relative entropies are monotone under CPTP maps then so are the corresponding barycentric Rényi divergences, and when all the relative entropies are additive then the corresponding barycentric Rényi divergences are subadditive under tensor product. Additivity has only been established before for the case where all $D^{q_x}$ are chosen to be the Umegaki relative entropy, which is also the only case where the barycentric Rényi divergence (called the minimal one) was known to admit an explicit expression. Here we settle the problem of additivity by showing that for any choice of monotone quantum relative entropies, the regularized barycentric Rényi divergence coincides with the minimal barycentric Rényi divergence on invertible inputs. This in turn implies that if at least two of the $D^{q_x}$ are strictly larger than the Umegaki relative entropy (in a precise sense), then the resulting barycentric Rényi divergence is not even weakly additive.

0.20

High-Dimensional Ultra-Log-Concave Distributions

cs.DS math.PR ·2026-09-21 ·Zongchen Chen, Sihan Wang
similarity 0.56
Abstract

Ultra-log-concave distributions are ubiquitous in probability, combinatorics, and statistical mechanics and have been studied extensively. In this paper, we introduce a quantitative high-dimensional extension of this notion, called $δ$-ultra-log-concavity, for probability measures on $\mathbb{N}^d$ with downward closed support. When $δ= 1$, this notion coincides with the class studied by Gurvits (2009) via strongly log-concave generating functions, and with the class defined by Anari, Oveis Gharan, and Vinzant (2021) via completely log-concave generating functions; in one dimension, it reduces to classical ultra-log-concavity. We establish several functional inequalities, including a weighted Poincaré inequality, a discrete Brascamp--Lieb inequality, and a weighted Wu-type modified log-Sobolev inequality. Our approach combines integrated Bakry--Émery calculus for a canonical birth-death chain with Poisson stochastic localization, which arises as the time reversal of coordinatewise binomial thinning. We further establish concentration of measure, maximum-entropy principles, and several closure properties for ultra-log-concave measures, and develop applications to queueing models, polymatroids, antiferromagnetic Potts models, and hardcore models. Finally, a lattice scaling limit of the discrete theory yields Poincaré and Brascamp--Lieb inequalities for Laguerre diffusions.

0.47

Continuity of Regularized Channel Rényi Divergences

cs.IT quant-phmath-ph ·2026-09-23 ·Jinzhao Wang, Yuxiang Yang
theorycode/data similarity 0.55
Abstract · 12 pages; comments are welcome; v2: updated references for concurrent works, add a Lean certificate for our proof: https://github.com/JWang2…

We prove that the regularized, stabilized sandwiched Rényi divergence of finite-dimensional quantum channels converges to their regularized relative entropy as the Rényi order tends to one. The key tool is the channel hockey-stick divergence: Gour's Stinespring approximation bound and a Schatten norm estimate amplify an asymptotic bound below one into exponential decay at higher threshold rates. For channel pairs with finite max-relative entropy, known operational connections then give exponential strong converses for parallel and adaptive discrimination, a sharp zero--one testing law, and the subchannel asymptotic equipartition property.

0.48

Matrix $Φ$-Entropic Measures of Dependence

cs.IT ·2026-10-21 ·Chenyu Wang, Amin Gohari
similarity 0.54
Abstract

We introduce matrix $Φ$-ribbons, extending the scalar $Φ$-ribbons to the matrix case. For $Φ$ in the subadditivity class, we establish data processing and tensorization properties of matrix $Φ$-ribbons. We establish scalar and matrix $Φ$-entropy inequalities and use them to show that both $Φ$-ribbons characterize the corresponding $Φ$-strong data processing inequality (SDPI) constants. We obtain exact evaluations of the SDPI constants for several sources. We provide examples showing that the matrix $Φ$-SDPI constant can exceed the scalar counterpart. Finally, we derive inner bounds for $Φ$-ribbons from prescribed independence structures and prove that these bounds are optimal when $Φ(t)=t\log t$.

0.25

Copula Operad and Copula Entropy

cs.IT math.PRmath.CTmath.ST ·2026-09-17 ·Xuexing Lu
theoryunusual similarity 0.54
Abstract · Find mistakes. Copulas are not closed under composition

We construct a symmetric operad $\mathfrak{C}$ on the class of all multivariate copulas, where operadic composition is given by Sklar substitution. We prove that the absolutely continuous subclass $\mathfrak{C}^{ac}$---which coincides with the $L^1$ class of copula densities---forms a suboperad; under composition, the density of the composite copula is given by the explicit Sklar substitution density formula $g(v)=φ\big(Ψ_1(v^{(1)}),\dots,Ψ_n(v^{(n)})\big)\prod_{k=1}^{n}ψ_{k}(v^{(k)})$. Furthermore, we show that copulas with finite copula entropy---identified with the $L\log L$ class of copula densities---are closed under substitution and hence constitute a suboperad $\mathfrak{C}^{L\log L}$. On this suboperad, copula entropy is strictly additive: $H(γ(Φ;Ψ_{1},\ldots,Ψ_{n}))=H(Φ)+\sum_{k=1}^{n}H(Ψ_{k})$.

0.25

On Measures of Dissimilarity on the Majorization Simplex

cs.IT ·2026-10-12 ·Tarun Hariharan, Prasad Krishnan, Abhishek Deshpande
theory similarity 0.53
Abstract

The majorization simplex $\mathcal{P}_n$ is the set of probability vectors of length $n$ whose entries are arranged in non-increasing order. Under the majorization order, $\mathcal{P}_n$ is a lattice, and it is the natural space for comparing distributions when the labels of the outcomes are not relevant. In this work, we study measures of dissimilarity (and, similarity) on $\mathcal{P}_n$. Our first set of results concerns the Rényi-entropy based distance $d_α(p,q)=H_α(p)+H_α(q)-2H_α(p\vee q)$ proposed by Yadav and Shkel (2026), which generalizes a metric of Cicalese, Gargano and Vaccaro (2013). We show that $d_α$ is a metric on $\mathcal{P}_n$ for $α\in[1,\infty)$, while for $α\in[0,1)\cup\{\infty\}$, $d_α$ is not a metric in general. Our main result in this part identifies the necessary and sufficient conditions for equality in the triangle inequality of $d_α$ when $α>1$, for which we identify the equality conditions for the supermodularity of the Rényi entropy $H_α$. Our second set of results concerns the total variation distance and the Hellinger affinity (also called the Bhattacharyya coefficient) on $\mathcal{P}_n$. We obtain a number of bounds on these measures, along with some of their equality conditions. The resulting Hellinger affinity based upper bound on the total variation distance can be smaller than the bounds obtained from Pinsker's and the Bretagnolle--Huber inequalities.

0.24