A General Derivative Identity for the Conditional Mean Estimator in Gaussian Noise and Some Applications
share
Consider a channel Y= X+ N where X is an n-dimensional random vector, and N is a Gaussian vector with a covariance matrix šŖ_ N. The object under consideration in this paper is the conditional mean of X given Y= y, that is yā E[ X| Y= y]. Several identities in the literature connect E[ X| Y= y] to other quantities such as the conditional variance, score functions, and higher-order conditional moments. The objective of this paper is to provide a unifying view of these identities. In the first part of the paper, a general derivative identity for the conditional mean is derived. Specifically, for the Markov chain Uā Xā Y, it is shown that the Jacobian of E[ U| Y= y] is given by šŖ_ N^-1 Cov ( X, U | Y= y). In the second part of the paper, via various choices of U, the new identity is used to generalize many of the known identities and derive some new ones. First, a simple proof of the Hatsel and Nolte identity for the conditional variance is shown. Second, a simple proof of the recursive identity due to Jaffer is provided. Third, a new connection between the conditional cumulants and the conditional expectation is shown. In particular, it is shown that the k-th derivative of E[X|Y=y] is the (k+1)-th conditional cumulant. The third part of the paper considers some applications. In a first application, the power series and the compositional inverse of E[X|Y=y] are derived. In a second application, the distribution of the estimator error (X-E[X|Y]) is derived. In a third application, we construct consistent estimators (empirical Bayes estimators) of the conditional cumulants from an i.i.d. sequence Y_1,...,Y_n.
READ FULL TEXT