Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Analysis
  • Published:

Evaluation of DNA microarray results with quantitative gene expression platforms

Abstract

We have evaluated the performance characteristics of three quantitative gene expression technologies and correlated their expression measurements to those of five commercial microarray platforms, based on the MicroArray Quality Control (MAQC) data set. The limit of detection, assay range, precision, accuracy and fold-change correlations were assessed for 997 TaqMan Gene Expression Assays, 205 Standardized RT (Sta)RT-PCR assays and 244 QuantiGene assays. TaqMan is a registered trademark of Roche Molecular Systems, Inc. We observed high correlation between quantitative gene expression values and microarray platform results and found few discordant measurements among all platforms. The main cause of variability was differences in probe sequence and thus target location. A second source of variability was the limited and variable sensitivity of the different microarray platforms for detecting weakly expressed genes, which affected interplatform and intersite reproducibility of differentially expressed genes. From this analysis, we conclude that the MAQC microarray data set has been validated by alternative quantitative gene expression platforms thus supporting the use of microarray platforms for the quantitative characterization of gene expression.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Effect of the number of transcript molecules on assay precision.
Figure 2: Analysis of assay accuracy.
Figure 3: Correlation of fold change between alternative quantitative platforms.
Figure 4: Performance of microarray platforms relative to alternative quantitative platforms.
Figure 5: Assessment of true positive rates and false discovery rates using TaqMan assays.
Figure 6: Resolution of fold-change discrepancy results.

Similar content being viewed by others

References

  1. Vondracek, M. et al. Transcript profiling of enzymes involved in detoxification of xenobiotics and reactive oxygen in human normal and simian virus 40 T antigen-immortalized oral keratinocytes. Int. J. Cancer 99, 776–782 (2002).

    Article  CAS  Google Scholar 

  2. Urdea, M. et al. Branched DNA amplification multimers for the sensitive, direct detection of human hepatitis virus. Nucleic Acids Symp. Ser. 24, 197–200 (1991).

    CAS  Google Scholar 

  3. Gleaves, C.A. et al. Multicenter evaluation of the Bayer VERSANT HIV-1 RNA 3.0 assay: analytical and clinical performance. J. Clin. Virol. 25, 205–216 (2002).

    Article  CAS  Google Scholar 

  4. Bustin, S.A. (ed.). A-Z of Quantitative PCR. (International University Line Biotechnology Series, La Jolla, California, USA, 2004).

    Google Scholar 

  5. Wong, M.L. & Medrano, J.F. Real-time PCR for mRNA quantitation. Biotechniques 39, 75–85 (2005).

    Article  CAS  Google Scholar 

  6. Lee, L.G., Connell, C.R. & Bloch, W. Allelic discrimination by nick-translation PCR with fluorogenic probes. Nucleic Acids Res. 21, 3761–3766 (1993).

    Article  CAS  Google Scholar 

  7. Heid, C.A., Stevens, J., Livak, K.J. & Williams, P.M. Real time quantitative PCR. Genome Res. 6, 986–994 (1996).

    Article  CAS  Google Scholar 

  8. Gibson, U.E., Heid, C.A. & Williams, P.M. A novel method for real time quantitative RT-PCR. Genome Res. 6, 995–1001 (1996).

    Article  CAS  Google Scholar 

  9. Qin, L.X. et al. Evaluation of methods for oligonucleotide array data via quantitative real-time PCR. BMC Bioinformatics 7, 23 (2006).

    Article  Google Scholar 

  10. Kuo, W.P. et al. A sequence-oriented comparison of gene expression measurements across different hybridization-based technologies. Nat. Biotechnol. 24, 832–840 (2006).

    Article  CAS  Google Scholar 

  11. Wang, Y. et al. Large scale real-time PCR validation on gene expression measurements from two commercial long-oligonucleotide microarrays. BMC Genomics 7, 59 (2006).

    Article  Google Scholar 

  12. Willey, J.C. et al. Standardized RT-PCR and the standardized expression measurement center. Methods Mol. Biol. 258, 13–41 (2004).

    CAS  PubMed  Google Scholar 

  13. Rots, M.G. et al. mRNA expression levels of methotrexate resistance-related proteins in childhood leukemia as determined by a standardized competitive template-based RT-PCR method. Leukemia 14, 2166–2175 (2000).

    Article  CAS  Google Scholar 

  14. Mullins, D.N. et al. CEBPG transcription factor correlates with antioxidant and DNA repair genes in normal bronchial epithelial cells but not in individuals with bronchogenic carcinoma. BMC Cancer 5, 141 (2005).

    Article  Google Scholar 

  15. Flagella, M. et al. A multiplex branched DNA assay for parallel quantitative gene expression profiling. Anal. Biochem. 352, 50–60 (2006).

    Article  CAS  Google Scholar 

  16. Yao, J.D. et al. Multicenter Evaluation of the VERSANT Hepatitis B Virus DNA 3.0 Assay. J. Clin. Microbiol. 42, 800–806 (2004).

    Article  CAS  Google Scholar 

  17. Elbeik, T. et al. Multicenter Evaluation of the Performance Characteristics of the Bayer VERSANT HCV RNA 3.0 Assay (bDNA). J. Clin. Microbiol. 42, 563–569 (2004).

    Article  CAS  Google Scholar 

  18. Stenman, J. & Orpana, A. Accuracy in amplification. Nat. Biotechnol. 19, 1011–1012 (2001).

    Article  CAS  Google Scholar 

  19. Cleveland, W. Robust locally weighted regression and smoothing scatter plots. J. Am. Stat. Assoc. 74, 829–836 (1979).

    Article  Google Scholar 

  20. MAQC Consortium. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat. Biotechnol. 24, 1151–1161 (2006).

  21. Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate - a practical and powerful approach to multiple testing. J. R. Stat. Soc. B. Met. 57, 289–300 (1995).

    Google Scholar 

  22. Shippy, R. et al. Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in making cross-platform correlations. BMC Genomics 5, 61 (2004).

    Article  Google Scholar 

  23. Livak, K.J. & Schmittgen, T.D. Analysis of relative gene expression data using real-time quantitative PCR and the 2-ΔΔCT Method. Methods 25, 402–408 (2001).

    Article  CAS  Google Scholar 

  24. Kern, D. et al. An enhanced-sensitivity branched-DNA assay for quantification of human immunodeficiency virus type 1 RNA in plasma. J. Clin. Microbiol. 34, 3196–3202 (1996).

    Article  CAS  Google Scholar 

  25. Wang, J. et al. Regulation of insulin preRNA splicing by glucose. Proc. Natl Acad. Sci. USA 94, 4360–4365 (1997).

    Article  CAS  Google Scholar 

  26. Shippy, R. et al. Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat. Biotechnol. 24, 1123–1131 (2006).

    Article  CAS  Google Scholar 

  27. Tong, W. et al. Evaluation of external RNA controls for the assessment of microarray performance. Nat. Biotechnol. 24, 1132–1139 (2006).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the contribution to this manuscript from the following members of the MAQC team: Shawn B. Baker, Anne Bergstrom Lucas, Jim Collins, Eugene Chudin, Stephanie Fulmer-Smentek, Damir Herman, Richard Shippy, Chunlin Xiao and Necip Mehmet.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Federico M Goodsaid.

Ethics declarations

Competing interests

J.C.W. is a consultant for and has significant financial interest in Gene Express, Inc.

Supplementary information

Supplementary Fig. 1

TaqMan® assays SD vs. CT plot. (DOC 31 kb)

Supplementary Fig. 2

TaqMan® assays R2 distribution plot. (DOC 166 kb)

Supplementary Fig. 3

Boxplot of fold change as a function of signal. (DOC 43 kb)

Supplementary Fig. 4

Expression characteristics of EPHA7. (DOC 91 kb)

Supplementary Fig. 5

Endogenous control expression. (DOC 204 kb)

Supplementary Table 1

Gene lists used for the analysis of performance metrics of alternative quantitative platforms and for comparison with microarrays. (DOC 51 kb)

Supplementary Table 2

Inter-site discordance in detection among genes expressed at low level. (DOC 156 kb)

Supplementary Table 3

TPR, FDR tables. (DOC 100 kb)

Supplementary Table 4

Discordant probes among the 997 genes. (DOC 36 kb)

Supplementary Table 5

Discordant gene expression values in alternative quantitative platforms FC expression values for all discordant genes. (DOC 44 kb)

Supplementary Methods (DOC 48 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Canales, R., Luo, Y., Willey, J. et al. Evaluation of DNA microarray results with quantitative gene expression platforms. Nat Biotechnol 24, 1115–1122 (2006). https://doi.org/10.1038/nbt1236

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nbt1236

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing