A New Argument for p<0.005
share
Analysis of Credibility is a reverse-Bayes technique that has been proposed to overcome some of the shortcomings of significant tests. The notion of intrinsic credibility has been recently introduced to check the credibility of "out of the blue" findings without any prior support. A significant result is deemed intrinsically credible if it is in conflict with a sceptical prior derived from the very same data that would make the effect non-significant. In this note I propose to use Bayesian predictive tail probabilities to check for intrinsic credibility. For the standard 5 new threshold for credibility that is remarkably close to the recently proposed 0.005 threshold. I also introduce the notion of the credibility ratio, the ratio of the upper to the lower limit of a confidence interval, as an easy tool to check for intrinsic credibility. I show that the credibility ratio has to be smaller than 5.85 such that a significant finding is also intrinsically credible.
READ FULL TEXT