“The Natural Selection of Bad Science”

Paul E. Smaldino

University of California, Merced

Science is an enterprise that uses rationality and investigation to build increasingly accurate models of the natural world. Practitioners of science have a vast toolkit of investigatory and analytic methods at their disposal—a toolkit that changes over time. Are their discernible patterns in the adoption of methods over time? Empirical science can in many cases be thought of as a problem of signal detection for facts, in which the challenge is to accurately detect a true signal among potential noise. If scientists are incentivized only or primarily by the quest for truth, then any methodological improvement to signal strength should be adopted, provided any associated costs are manageable. Instead, scientists are often rewarded for the acquisition of tokens, such as publication count or journal impact factor. If methodological strategies that maximize token counts are misaligned with those that increase signal strength, the probability that scientific results reflect reality will fall. I will present empirical evidence and the results of formal modeling to support this conclusion and suggest that many current incentive structures in professional science organization are detrimental to the ultimate aims of scientific research. I will conclude with some thoughts on how organizations can act to improve the quality of science research.

All are welcome!