Potential Conditional Mutual Information: Estimators, Properties and Applications

10/13/2017
by   Arman Rahimzamani, et al.
0

The conditional mutual information I(X;Y|Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model inference, causal strength estimation and time-series problems. In several applications, it is desirable to have a functional purely of the conditional distribution p_Y|X,Z rather than of the joint distribution p_X,Y,Z. We define the potential conditional mutual information as the conditional mutual information calculated with a modified joint distribution p_Y|X,Z q_X,Z, where q_X,Z is a potential distribution, fixed airport. We develop K nearest neighbor based estimators for this functional, employing importance sampling, and a coupling trick, and prove the finite k consistency of such an estimator. We demonstrate that the estimator has excellent practical performance and show an application in dynamical system inference.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset