The Most Difference in Means: A Statistic for Null and Near-Zero Results
share
Two-sample p-values test for statistical significance. Yet p-values cannot determine if a result has a negligible (near-zero) effect size, nor compare evidence for negligibility among independent studies. We propose the most difference in means (δM) statistic to assess the practical insignificance of results by measuring the evidence for a negligible effect size. Both δM and the relative form of δM allow hypothesis testing for negligibility and outperform other candidate statistics in identifying results with stronger evidence of negligible effect. We compile results from broadly related experiments and use the relative δM to compare practical insignificance across different measurement methods and experiment models. Reporting the relative δM builds consensus for negligible effect size by making near-zero results more quantitative and publishable.
READ FULL TEXT