Communication Complexity of Distributed High Dimensional Correlation Testing
share
Two parties observe independent copies of a d-dimensional vector and a scalar. They seek to test if their data is correlated or not, namely they seek to test if the norm ρ_2 of the correlation vector ρ between their observations exceeds τ or is it 0. To that end, they communicate interactively and declare the output of the test. We show that roughly order d/τ^2 bits of communication are sufficient and necessary for resolving the distributed correlation testing problem above. Furthermore, we establish a lower bound of roughly d^2/τ^2 bits for communication needed for distributed correlation estimation, rendering the estimate-and-test approach suboptimal in communication required for distributed correlation testing. For the one-dimensional case with one-way communication, our bounds are tight even in the constant and provide a precise dependence of communication complexity on the probabilities of error of two types.
READ FULL TEXT