Figures
Abstract
Background
Influenza is major public health threat worldwide, yet the diagnostic accuracy of rapid tests in developing country settings is not well described.
Methodology/Principal Findings
To investigate the diagnostic accuracy of the QuickVue Influenza A+B test in a primary care setting in a developing country, we performed a prospective study of diagnostic accuracy of the QuickVue Influenza A+B test in comparison to reverse transcriptase-polymerase chain reaction (RT-PCR) in a primary healthcare setting in children aged 2 to 12 years in Managua, Nicaragua. The sensitivity and specificity of the QuickVue test compared to RT-PCR were 68.5% (95% CI 63.4, 73.3) and 98.1% (95% CI 96.9, 98.9), respectively, for children with a fever or history of a fever and cough and/or sore throat. Test performance was found to be lower on the first day that symptoms developed in comparison to test performance on days two or three of illness.
Citation: Gordon A, Videa E, Saborio S, López R, Kuan G, Reingold A, et al. (2009) Performance of an Influenza Rapid Test in Children in a Primary Healthcare Setting in Nicaragua. PLoS ONE 4(11): e7907. https://doi.org/10.1371/journal.pone.0007907
Editor: Robyn Klein, Washington University School of Medicine, United States of America
Received: August 31, 2009; Accepted: October 21, 2009; Published: November 19, 2009
Copyright: © 2009 Gordon et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was funded by the Pediatric Dengue Cohort Initiative, J. Craig Venter Institute, and the United States Centers for Disease Control and Prevention. Rapid tests were kindly provided by Quidel Corporation. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: None of the authors has any relationships or support that might be perceived as constituting a competing interest, except that Quidel Corporation covered the costs of airfare and hotel for AG at one conference in February 2009.
Introduction
Influenza is a major health threat throughout the world [1], yet many countries lack even rudimentary influenza surveillance systems [2], [3]. Furthermore, influenza surveillance in developing countries often includes only major cities due to limited laboratory resources in outlying areas and poor infrastructure, such as lack of electrical power. Consequently, data are lacking about the types of influenza viruses that are circulating and the burden of influenza-related disease in the developing world.
Logistical constraints in developing countries largely dictate the types of laboratory tests available for use in influenza surveillance. While reverse transcriptase-polymerase chain reaction (RT-PCR) and viral isolation are the preferred diagnostic methods, rapid influenza tests are valuable for their ease of use and laboratory-independence. In the developed world, rapid influenza tests have been evaluated in both laboratory and clinical settings, yielding sensitivities of 45–90% and specificities of 86–100% [4], [5], [6], [7], [8], [9], [10] in comparison to viral isolation and RT-PCR. In developing countries, the performance characteristics of rapid influenza tests are not well documented in primary health care settings, where infrastructure is unreliable, conditions are often hot and humid, and staff may have little or no experience with the collection of respiratory samples, yet where rapid tests may aid in influenza surveillance.
Historically, viral isolation has been regarded as the “gold standard” for diagnosis of influenza infection. However, recently a number of studies have demonstrated the superior sensitivity of RT-PCR for the detection of influenza viruses [11], [12], [13], [14], and influenza experts have recommended that RT-PCR be regarded as the new diagnostic standard [14], [15]. Surveillance systems in developed countries and some developing countries have switched to using RT-PCR as the primary laboratory diagnostic test, with viral isolation performed primarily on RT-PCR-positive specimens. For this reason, the QuickVue rapid test was compared to RT-PCR in this study.
To examine the diagnostic accuracy of the QuickVue Influenza A+B test in comparison to RT-PCR in children treated in a primary healthcare facility in a developing country, we performed a prospective study in Managua, Nicaragua and report the results here.
Methods
Ethics Statement
This study was approved by the institutional review boards at the University of California (UC) Berkeley and the Nicaraguan Ministry of Health. Written informed consent was obtained from the parents or guardians of all participants.
Study Population and Site
This study was performed as part of the Nicaraguan Influenza Cohort Study (NICS), a prospective community-based cohort study of approximately 3,800 children 2–14 years old in Managua, Nicaragua [16]. Healthy children aged 2–12 years residing in the study area were enrolled beginning in June 2007. Children were eligible to remain in the study through the age of 14 or until they moved from the study area. To maintain the age structure of the cohort, new two-year-olds were recruited in July-September of each year. The primary study site was the Health Center Socrates Flores Vivas (HCSFV), a public primary health care facility in the capital of Nicaragua. Respiratory specimens were collected and the rapid test was performed in the HCSFV's clinical laboratory. A number of measures were taken to encourage participants to attend the HCSFV at the first sign of illness. These measures included: 1) at enrollment, parents agreed to bring their child to the health center at the first sign of illness; 2) participants were provided with free healthcare 24 hours a day, 365 days a year; 3) participants were provided with rapid diagnostic testing free of charge; and 4) a study ambulance was available at all times at the heath center. Children who required hospitalization or tertiary-level care were transferred to a hospital by study staff. In yearly participation surveys, 1.7% to 2.5% of participants reported attending a healthcare provider outside of the study.
Procedures
All specimens collected between January 1, 2008 and December 31, 2008 from children in the NICS who presented with a fever, or history of fever or feverishness, and cough and/or sore throat within five days of symptom onset were included in the study. Information from all medical appointments was systematically collected using standardized data collection forms. Data included age, sex, two measurements of temperature, date of fever onset, and other symptoms. Upon presentation with testing criteria, children were randomly selected for respiratory testing using a random number generator and then referred to the HCSFV clinical laboratory. There, a nasal swab was first collected for the rapid test, QuickVue Influenza A+B, using the swab provided in the kit. Then nasal and throat Dacron swabs were collected and placed into one tube containing three mL of viral transport medium for RT-PCR and viral culture. The QuickVue test was performed onsite immediately. Samples for RT-PCR and viral isolation collected on workdays (Monday–Saturday) were kept at 4°C and processed within 16 hours. Samples collected on Sundays and holidays were stored for up to 48 hours prior to processing.
QuickVue Influenza A+B is a lateral-flow immunoassay that detects and differentiates between influenza A and B. Tests were performed according to the manufacturer's instructions. Tests were read after exactly ten minutes and were considered negative if, at ten minutes, a pink line had not appeared to indicate a positive influenza A or B result. Tests were considered invalid if the blue control line did not appear or if the background color made it impossible to read the test result. Invalid tests were repeated once.
Laboratory Methods
Samples were transported to the National Virology Laboratory at the Nicaraguan Ministry of Health, where RNA was extracted from nasal swabs using the QIAamp® Viral RNA Mini Kit. (Qiagen Sciences, Germantown, MD) Influenza viruses A and B were amplified using Promega Access RT-PCR (Promega Corporation, Madison, WI) and typed in-house according to a protocol provided by the Centers for Disease Control and Prevention (CDC). Laboratory technicians were trained in how to perform RT-PCR, including attendance at influenza testing workshops held by the CDC and World Health Organization and hands-on training by study personnel from UC Berkeley. Laboratory technicians performing RT-PCR testing were blinded to the results of the rapid test and to the patients' clinical presentation.
Rapid Test Training
To replicate the typical training resources that would be available if rapid tests were widely distributed in a developing country setting, a 1.5-hour training session was held. This training consisted of a short presentation explaining the symptoms and burden of influenza, influenza surveillance, collection of nasal swabs, principles of the QuickVue rapid test, and instructions on how to perform the test. Laboratory technicians then practiced taking one nasal swab specimen and performing a rapid test. Prior to the training session, technicians had never performed an influenza rapid test and many had little or no experience collecting nasal swab specimens.
Statistical Analyses
The main outcomes of this study were the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the QuickVue Influenza A+B test compared with RT-PCR. Agreement between tests was calculated using the kappa statistic. Sub-analyses were performed to examine test performance stratified by testing criteria, day of presentation, age, and sex. All statistical analyses were performed using STATA 10.1 (StataCorp, College Station, Texas).
Results
Between January 1, 2008, and December 31, 2008, 3,916 children participated in the cohort study. During this period, study participants attended 13,666 medical visits, of which 3,935 (29%) met the testing criteria. Of these, 1,195 (30%) were randomly selected for influenza testing, and 1,157 duplicate specimens (97%) were collected from 989 children and tested by both rapid test and RT-PCR, while 38 children received a rapid test only (Figure 1). Demographic data for the children that contributed samples are presented in Table 1. In total, 359 samples (31%) tested positive for influenza by RT-PCR, with 224 (19%) testing positive for influenza A and 135 (12%) for influenza B. Of the 74 influenza A-positive samples that underwent subtyping by RT-PCR, 73 were positive for seasonal influenza A H1N1 and one for influenza A H3N2. The QuickVue test detected virus in 246 influenza RT-PCR-positive samples, including 146 influenza A-positive samples and 100 influenza B-positive samples. One invalid QuickVue test result was repeated and found to be negative for influenza.
Flowchart diagramming the selection of samples used to analyze the performance of the QuickVue Influenza A+B rapid test. Due to complications related to transport and sample separation, samples (n = 37) collected during holiday periods lasting longer than 3 days did not undergo RT-PCR and therefore were not included in this study.
Data on the agreement of RT-PCR and rapid test are presented in Table 2. Overall, the sensitivity and specificity of the QuickVue test compared to RT-PCR for influenza A or B were 68.5% (95% CI 63.4, 73.3) and 98.1% (95% CI 96.9, 98.9), respectively. The corresponding PPV was 93.9% (95% CI 90.3, 96.5), and the NPV was 88.1% (95% CI 85.9, 90.1) (Table 3). The sensitivity and specificity of the test in children under six and in those aged six and over were similar (data not shown).
Limiting the analysis to the 578 samples from patients who met the CDC definition of influenza-like illness, defined as a fever of ≥37.8°C plus cough or sore throat, resulted in a slightly higher sensitivity for the rapid test but did not significantly change the overall test performance (Tables 2 and 3). In this sub-analysis, the sensitivity of the rapid test was 72.1% (95% CI 66.3, 77.5) and the specificity was 98.1% (95% CI 95.9, 99.3). Performance of the rapid test by day of presentation was examined both for the entire study population and for the subgroup meeting the definition of ILI. In both analyses, the rapid test had a lower sensitivity on the day of symptom onset and on the fourth day of illness when compared to the sensitivity on day two or three of illness (Table 4).
Discussion
Our study found that the QuickVue Influenza A+B test performed with a moderate sensitivity (68.5%) and high specificity (98.1%) in comparison to RT-PCR in a developing country primary care setting. Laboratory technicians found the test easy to use and needed minimal training to adequately perform the test.
In the US and other developed countries, the QuickVue Influenza A+B rapid test has been found to have a sensitivity of 67%–85% and specificity of 97%–100% [8], [17]. However, one recent study conducted in three sites in the US found the test performed with a significantly lower sensitivity (19%–32%)[18]. One study performed in a developing country reported the QuickVue test to have a sensitivity and specificity of 71% and 98%, respectively, in patients aged one month to 85 years in hospital outpatient clinics in Thailand [15]. As most of these prior studies included only patients meeting the CDC/WHO definition of ILI, it is more appropriate to compare the results of our sub-analysis limited to children with ILI to these studies. Thus, the sensitivity of 72.1% (95% CI 66.3, 77.5) and specificity of 98.1% (95% CI 95.9, 99.3) we observed are well within the ranges of the sensitivities and specificities previously reported.
Importantly, a substantial number of participants (23%) presented on the first day of illness, and the sensitivity of the rapid test was found to be lower among children who presented and were tested on the first day. Few studies of rapid tests have captured data on these early cases. In a study in Hong Kong, researchers examined the performance of the QuickVue rapid test and did not find a significant difference in the first 24 hours following symptom onset [19]. One possible explanation for our differing results is that the Hong Kong study population was very different from ours. Subjects were predominantly adults and were recruited because they were seeking medical care at an outpatient clinic, and therefore might have been sicker and hence have a higher viral load than our study subjects at the time of testing. In a recent meta-analysis of influenza volunteer challenge studies, symptoms were reported in the first 24 hours following virus inoculation, a day before peak viral load and two days before the peak of symptoms [20]. It is possible that early in illness, children are symptomatic at relatively low viral loads, leading to the lower sensitivity we report on day 1. More studies are needed to elucidate the natural history of influenza infection in children.
The large number of children presenting so early resulted from the cohort study protocol, in which families of participants were asked to bring in their children at the first sign of illness. However, persons with influenza typically do not present until several days into the illness. Therefore, it is likely that the sensitivity and resulting positive predictive values we observed are underestimates of the test's performance in the clinical setting, since it is less common for patients to present on the day of symptom onset. However, during an influenza pandemic, patients may present very early following symptom onset; thus, it is imperative to evaluate the performance of any rapid test in detecting influenza very early after onset of illness.
One of the limitations of this study is the age range of the patients. Because the study was restricted to children aged 2–14 years, the sensitivities and specificities we found apply only to this group. However, because the rapid test performed at approximately the same level in this study as in studies performed in children in developed country settings, the rapid test may well perform at the same level across all age ranges in developing country settings as observed in studies conducted in developed countries. Another limitation of this study is the use of RT-PCR as the gold standard. However, considering the very high (∼98%) specificity of the rapid test in this study and the fact that RT-PCR is substantially more sensitive than viral isolation, it is unlikely that use of RT-PCR alone, instead of a combination of RT-PCR and viral isolation, significantly affected the results.
In conclusion, the rapid influenza test we used performed well in a primary healthcare setting in Managua, Nicaragua. Clinical laboratory staff needed minimal training to correctly perform the test and found the test easy to perform. Further evaluation of rapid tests in developing countries is needed to determine the feasibility of using these tests to augment existing surveillance systems and for quick deployment in the event of an influenza pandemic, as well as for use in clinical decision-making.
Acknowledgments
The authors would like to thank the Nicaraguan Ministry of Health and all of the study staff for their excellent work. In particular, we thank Maria Celina Perez Zúnigas for assistance in performing RT-PCR; Carolina Flores Hernandez, Heyri Roa Solís, Moises Navarro Martinez, and Patricia Castillo Palma for performing the influenza rapid tests, and Drs. Sergio Benito Ojeda Munguía, Luis Nery Sanchez Palma, Grethel María Navas Pasquier, and Jackeline Herrera Morales for providing medical attention to the study participants.
Author Contributions
Conceived and designed the experiments: AG GK ALR AB EH. Performed the experiments: AG EV SS RL GK AB. Analyzed the data: AG. Contributed reagents/materials/analysis tools: AG EH. Wrote the paper: AG EH.
References
- 1. Simonsen L (1999) The global impact of influenza on morbidity and mortality. Vaccine 17: Suppl 1S3–10.
- 2.
WHO Global Agenda on Influenza Surveillence and Control.
- 3. Schoub BD, McAnerney JM, Besselaar TG (2002) Regional perspectives on influenza surveillance in Africa. Vaccine 20: Suppl 2S45–46.
- 4. Uyeki TM (2003) Influenza diagnosis and treatment in children: a review of studies on clinically useful tests and antiviral treatment for influenza. Pediatr Infect Dis J 22: 164–177.
- 5. Agoritsas K, Mack K, Bonsu BK, Goodman D, Salamon D, et al. (2006) Evaluation of the Quidel QuickVue test for detection of influenza A and B viruses in the pediatric emergency medicine setting by use of three specimen collection methods. Journal of Clinical Microbiology 44: 2638–2641.
- 6. Grijalva CG, Poehling KA, Edwards KM, Weinberg GA, Staat MA, et al. (2007) Accuracy and interpretation of rapid influenza tests in children. Pediatrics 119: e6–11.
- 7. Ruest A, Michaud S, Deslandes S, Frost EH (2003) Comparison of the Directigen flu A+B test, the QuickVue influenza test, and clinical case definition to viral culture and reverse transcription-PCR for rapid diagnosis of influenza virus infection. Journal of Clinical Microbiology 41: 3487–3493.
- 8. Hurt AC, Alexander R, Hibbert J, Deed N, Barr IG (2007) Performance of six influenza rapid tests in detecting human influenza in clinical specimens. Journal of Clinical Virology 39: 132–135.
- 9. Poehling KA, Zhu Y, Tang YW, Edwards K (2006) Accuracy and impact of a point-of-care rapid influenza test in young children with respiratory illnesses. Archives of Pediatrics and Adolescent Medicine 160: 713–718.
- 10. Rodriguez WJ, Schwartz RH, Thorne MM (2002) Evaluation of diagnostic tests for influenza in a pediatric practice. Pediatric Infectious Disease Journal 21: 193–196.
- 11. Stockton J, Ellis JS, Saville M, Clewley JP, Zambon MC (1998) Multiplex PCR for typing and subtyping influenza and respiratory syncytial viruses. Journal of Clinical Microbiology 36: 2990–2995.
- 12. Pregliasco F, Mensi C, Camorali L, Anselmi G (1998) Comparison of RT-PCR with other diagnostic assays for rapid detection of influenza viruses. Journal of Medical Virology 56: 168–173.
- 13. Steininger C, Kundi M, Aberle SW, Aberle JH, Popow-Kraupp T (2002) Effectiveness of reverse transcription-PCR, virus isolation, and enzyme-linked immunosorbent assay for diagnosis of influenza A virus infection in different age groups. Journal of Clinical Microbiology 40: 2051–2056.
- 14. Weinberg GA, Erdman DD, Edwards KM, Hall CB, Walker FJ, et al. (2004) Superiority of reverse-transcription polymerase chain reaction to conventional viral culture in the diagnosis of acute respiratory tract infections in children. Journal of Infectious Diseases 189: 706–710.
- 15. Simmerman JM, Chittaganpitch M, Erdman D, Sawatwong P, Uyeki TM, et al. (2007) Field performance and new uses of rapid influenza testing in Thailand. International Journal of Infectious Diseases 11: 166–171.
- 16. Kuan G, Gordon A, Aviles W, Ortega O, Hammond SN, et al. (2009) The Nicaraguan pediatric dengue cohort study: study design, methods, use of information technology, and extension to other infectious diseases. American Journal of Epidemiology 170: 120–129.
- 17. Mehlmann M, Bonner AB, Williams JV, Dankbar DM, Moore CL, et al. (2007) Comparison of the MChip to viral culture, reverse transcription-PCR, and the QuickVue influenza A+B test for rapid diagnosis of influenza. Journal of Clinical Microbiology 45: 1234–1237.
- 18. Uyeki TM, Prasad R, Vukotich C, Stebbins S, Rinaldo CR, et al. (2009) Low sensitivity of rapid diagnostic test for influenza. Clinical Infectious Diseases 48: e89–92.
- 19. Cheng CK, Cowling BJ, Chan KH, Fang VJ, Seto WH, et al. (2009) Factors affecting QuickVue Influenza A+B rapid test performance in the community setting. Diagnostic Microbiology and Infectious Disease 65: 35–41.
- 20. Carrat F, Vergu E, Ferguson NM, Lemaitre M, Cauchemez S, et al. (2008) Time lines of infection and disease in human influenza: a review of volunteer challenge studies. American Journal of Epidemiology 167: 775–785.