LASSO extension: using the number of non-zero coefficients to test the global model hypothesis
share
In this paper, we propose a test procedure based on the LASSO methodology to test the global null hypothesis of no dependence between a response variable and p predictors, where n observations with n < p are available. The proposed procedure is similar to the F-test for a linear model, which evaluates significance based on the ratio of explained to unexplained variance. However, the F-test is not suitable for models where p ≥ n. This limitation is due to the fact that when p ≥ n, the unexplained variance is zero and thus the F-statistic can no longer be calculated. In contrast, the proposed extension of the LASSO methodology overcomes this limitation by using the number of non-zero coefficients in the LASSO model as a test statistic after suitably specifying the regularization parameter. The method allows reliable analysis of high-dimensional datasets with as few as n = 40 observations. The performance of the method is tested by means of a power study.
READ FULL TEXT