On behalf of MAQC, Leming Shi, Wendell D Jones, Roderick V Jensen, Russell D Wolfinger, Ernest S Kawasaki, Damir Herman, Lei Guo, Federico M Goodsaid and Weida Tong reply:
We are pleased that Klebanov et al. find the MAQC study1,2 “important” for the establishment of the reproducibility of DNA microarray data. Their letter provides us with an opportunity to briefly clarify some important issues regarding our views on microarray data analysis. First, we would like to emphasize that we are not abandoning sensitivity and specificity as criteria when developing methods and rules for determining a list of differentially expressed genes in a microarray study. Rather, we would like to add an essential criterion related to the ability of the rule or method to reproduce this gene list. Just as there are trade-offs between sensitivity and specificity (more sensitivity usually implies less specificity and vice versa), there may be trade-offs related to reproducibility versus sensitivity and specificity. We feel from a fundamental, scientific viewpoint that a study that cannot be reliably reproduced has little value. We also concede that most current methodologies are deficient to varying degrees with respect to measuring and characterizing the reproducibility of a microarray experiment, not necessarily because of the microarray technology itself but rather due to the complexity of the highly parallel measurement, with its many sources of variation. There is not yet general agreement on how reproducibility manifests itself in this context. Even so, because researchers typically identify differentially expressed genes and carry them forward for biological pathway analysis or use them as the basis for mechanistic inferences, we propose that a reasonable first step is to present methods that will enhance the reproducibility or stability of a list of differentially expressed genes. In summary, reproducibility can and should be used as a quality measurement, but it is not the only quality measurement (that is, sensitivity and specificity are also important).
This is a preview of subscription content, access via your institution