Tight Lower Bounds for α-Divergences Under Moment Constraints and Relations Between Different α
share
The α-divergences include the well-known Kullback-Leibler divergence, Hellinger distance and χ^2-divergence. In this paper, we derive differential and integral relations between the α-divergences that are generalizations of the relation between the Kullback-Leibler divergence and the χ^2-divergence. We also show tight lower bounds for the α-divergences under given means and variances. In particular, we show a necessary and sufficient condition such that the binary divergences, which are divergences between probability measures on the same 2-point set, always attain lower bounds. Kullback-Leibler divergence, Hellinger distance, and χ^2-divergence satisfy this condition.
READ FULL TEXT