Department of Biostatistics Seminar/Workshop Series

Is most published research false?

Jeffrey Leek, PhD

Associate Professor of Biostatistics and Oncology, John Hopkins Bloomberg School of Public Health

The accuracy of published research is critical for scientists, physicians and patients who rely on these results. But the fundamental belief in the scientific literature was called into serious question by a paper suggesting most published medical research is false. This claim has launched an entire discipline focused on the crisis of reproducibility and replicability of science. In this talk I will discuss a new science-wise false discovery rate estimation method and its application to data in the form of P-values from the abstracts of all 77,430 papers published in The Lancet, The Journal of the American Medical Association, The New England Journal of Medicine, The British Medical Journal, and The American Journal of Epidemiology between 2000 and 2010. Among these papers, we found 5,322 reported P-values. Using these data, I will show how we estimate the overall rate of false discoveries among reported results as 14% (s.d. 1%), contrary to previous claims. I will also discuss the controversy surrounding both the original publication and the pre-publication controversy surrounding our paper. Time permitting I will discuss some extensions of our estimation approach to a new universal false discovery rate estimate.