Optimal linear estimation under unknown nonlinear transform

05/13/2015
by   Xinyang Yi, et al.
0

Linear regression studies the problem of estimating a model parameter β^* ∈R^p, from n observations {(y_i,x_i)}_i=1^n from linear model y_i = 〈x_i,β^* 〉 + ϵ_i. We consider a significant generalization in which the relationship between 〈x_i,β^* 〉 and y_i is noisy, quantized to a single bit, potentially nonlinear, noninvertible, as well as unknown. This model is known as the single-index model in statistics, and, among other things, it represents a significant generalization of one-bit compressed sensing. We propose a novel spectral-based estimation procedure and show that we can recover β^* in settings (i.e., classes of link function f) where previous algorithms fail. In general, our algorithm requires only very mild restrictions on the (unknown) functional relationship between y_i and 〈x_i,β^* 〉. We also consider the high dimensional setting where β^* is sparse ,and introduce a two-stage nonconvex framework that addresses estimation challenges in high dimensional regimes where p ≫ n. For a broad class of link functions between 〈x_i,β^* 〉 and y_i, we establish minimax lower bounds that demonstrate the optimality of our estimators in both the classical and high dimensional regimes.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset