What do we know about the disruption indicator in scientometrics? An overview of the literature
share
The purpose of this paper is to provide a review of the literature on the original disruption indicator (DI1) and its variants in scientometrics. The DI1 has received much media attention and prompted a public debate about science policy implications, since a study published in Nature found that papers in all disciplines and patents are becoming less disruptive over time. This review explains in the first part the DI1 and its variants in detail by examining their technical and theoretical properties. The remaining parts of the review are devoted to studies that examine the validity and the limitations of the indicators. Particular focus is placed on (1) the time-sensitivity of disruption scores, (2) the convergent validity of disruption scores with expert judgments, and (3) the comparative performance of the DI1 and its variants. The review shows that, while the literature on convergent validity is not entirely conclusive, it is clear that some modified indicator variants, in particular DI5, show higher degrees of convergent validity than DI1. Limitations of the DI1 and its variants are summarized, and best practice guidelines are provided. The review encourages users of the indicator to inform about the variety of DI1 variants and to apply the most appropriate variant. More research on the convergent validity of the DI1 and its variants as well as on the time-sensitivity of disruption scores is needed before the indicators can be used in the research evaluation practice.
READ FULL TEXT