Department of Biostatistics Seminar/Workshop Series

Measuring Reproducibility of High-Throughput Biological Experiments

Qunhua Li, PhD

Department of Statistics, University of Washington

Wednesday, December 15, 1:30-2:30pm, MRBIII Conference Room 1220

Reproducibility is essential to reliable scientific discovery in large-scale high-throughput biological studies. In this talk, I will present a unified approach to measure reproducibility of findings identified from replicate experiments and select discoveries using reproducibility between replicates.

Unlike the usual scalar measures of reproducibility, our approach views reproducibility as when the findings are no longer consistent across replicates. To measure the pairwise consistency between replicates, we develop a graphical statistic based on empirical copulas and a copula mixture model to quantitatively describe the change of consistency in the decreasing significance of findings. Based on the copula mixture procedure, we define a quantity, called “irreproducible discovery rate”, in a fashion analogous to the false discovery rate. This quantity, which describes the lack of reproducibility for the identifications selected at each threshold, provides a reproducibility criterion for selecting reliable signals and assessing the overall reproducibility of findings. Our approach can be applied to both probabilistic- and heuristic-based significance scores, and permits principled setting of selection thresholds.

This method has been adopted by ENCODE consortium for selecting ChIP-seq signal identification algorithms and monitoring the performance of their experimental facility. I will illustrate the effectiveness of our method using some ENCODE examples.
Topic revision: r2 - 26 Apr 2013, JohnBock

This site is powered by FoswikiCopyright © 2013-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Vanderbilt Biostatistics Wiki? Send feedback