Department of Biostatistics Seminar/Workshop Series

Assessing Reproducibility and Value in Genomic Signatures

Prasad Patil

Department of Biostatistics

Johns Hopkins Bloomberg School of Public Health

A handful of genomic discoveries have made their way into the clinic as tests based on genomic signatures. Prominent examples in breast cancer include MammaPrint, Oncotype DX, and Prosigna, all of which provide the clinician with a prediction of risk of cancer recurrence or severity. None of these tests are currently part of the standard course of care for a breast cancer patient. We examine two aspects of applying a genomic signature in a clinical setting that could hinder widespread use of these types of tests. First, we describe how common data normalization and rescaling steps may affect the reproducibility of a prediction for a single patient. We suggest a rank-based modeling procedure using Top-Scoring Pairs (TSPs) as one approach to avoiding what we call “test set bias”. Second, the results from genomic tests are often used only to confirm what an oncologist already expects about a patient’s prognosis. To determine whether a risk prediction from a genomic test can provide additional value beyond what is already known from standard clinical quantities, we conduct simulations in a clinical trial setting where adjusting for predictive covariates can improve the precision of a treatment effect estimator. Through the application of a propensity score/outcome regression framework, we compare precision gains due to adjusting for clinical quantities with the additional inclusion of the MammaPrint prediction to determine if information from the genomic test can provide supplementary value.