Randomization Tests that Condition on Non-Categorical Covariate Balance
share
A benefit of randomized experiments is that covariate distributions of treatment and control groups are balanced on avearge, resulting in simple unbiased estimators for treatment effects. However, it is possible that a particular randomization yields substantial covariate imbalance, in which case researchers may want to employ covariate adjustment strategies such as linear regression. As an alternative, we present a randomization test that conditions on general forms of covariate balance without specifying a model by only considering treatment assignments that are similar to the observed one in terms of covariate balance. Thus, a unique aspect of our randomization test is that it utilizes an assignment mechanism that differs from the assignment mechanism that was actually used to conduct the experiment. Previous conditional randomization tests have only allowed for categorical covariates, while our randomization test allows for any type of covariate. Through extensive simulation studies, we find that our conditional randomization test is more powerful than unconditional randomization tests that are standard in the literature. Furthermore, we find that our conditional randomization test is similar to a randomization test that uses a model-adjusted test statistic, thus suggesting a parallel between conditional randomization-based inference and inference from statistical models such as linear regression.
READ FULL TEXT