Is My Matched Dataset As-If Randomized, More, Or Less? Unifying the Design and Analysis of Observational Studies
share
Matching alleviates the problem of covariate imbalance in observational studies by finding a subset of treatment groups that exhibits covariate balance. However, standard analyses for matched datasets do not condition on the strong covariate balance that modern matching algorithms instigate by design. We find that, as a result, these analyses can be unnecessarily conservative. We develop an alternative approach that involves three contributions. First, we formulate an "as-if randomized" assumption that---unlike previous works---can incorporate any assignment mechanism of interest, including mechanisms that ensure covariate balance constraints, as in matching algorithms. Second, we develop a valid randomization test for this as-if randomized assumption, which allows researchers to determine the assignment mechanism that their matched dataset best approximates. Thus, this test encapsulates the design stage of an observational study. Third, we provide a treatment effect estimation strategy that uses the same assignment mechanism that was determined during the design stage, thereby unifying the design and analysis stages of the observational study, similar to how they are unified in the design and analysis of randomized experiments. Through simulation, we find that our approach yields more precise inferences than standard approaches by conditioning on the covariate balance in a given matched dataset.
READ FULL TEXT