Diagnostic test systematic reviews: Bibliographic search filters (“Clinical Queries”) for diagnostic accuracy studies perform well

doi:10.1016/j.jclinepi.2008.11.006

Journal of Clinical Epidemiology

Volume 62, Issue 9, September 2009, Pages 974-981

https://doi.org/10.1016/j.jclinepi.2008.11.006 Get rights and content

Abstract

Background

Systematic reviews of health care topics are valuable summaries of all pertinent studies on focused questions. However, finding all relevant primary studies for systematic reviews remains challenging.

Objectives

To determine the performance of the Clinical Queries sensitive search filter for diagnostic accuracy studies for retrieving studies for systematic reviews.

Methods

We compared the yield of the sensitive Clinical Queries diagnosis search filter for MEDLINE and EMBASE to retrieve studies in diagnostic accuracy systematic reviews reported in ACP Journal Club in 2006.

Results

Twelve of 22 diagnostic accuracy reviews (452 included studies) met the inclusion criteria. After excluding 11 studies not in MEDLINE or EMBASE, 95% of articles (417 of 441) were captured by the sensitive Clinical Queries diagnosis search filter (MEDLINE and EMBASE combined). Of 24 studies not retrieved by the filter, 22 were not diagnostic accuracy studies. Reanalysis of the Clinical Queries filter without these 22 nondiagnosis articles increased its performance to 99% (417 of 419). We found no substantive impact of the two articles missed by the Clinical Queries filter on the conclusions of the systematic reviews in which they were cited.

Conclusion

The sensitive Clinical Queries diagnostic search filter captured 99% of articles and 100% of substantive articles indexed in MEDLINE and EMBASE in diagnostic accuracy systematic reviews.

Section snippets

Background

Systematic reviews are valuable resources for clinicians and researchers, because they summarize all pertinent studies on a specific clinical question, can improve the understanding of inconsistencies among diverse evidence, help users to keep up with the medical literature, define future research agendas, and inform the management of health problems [1], [2]. However, finding all primary studies for systematic reviews is challenging, because an overwhelming amount of information is available

Methods

The methods used to derive the Clinical Queries search filters in both MEDLINE and EMBASE have been described elsewhere [5], [6], [7], [8], [9]. In this study, we compared the yield of the sensitive Clinical Queries diagnostic filters (Table 1) with the studies included in a sample of systematic reviews of diagnostic accuracy from the ACP Journal Club collection (http://www.acpjc.org) for the year 2006 (search carried out in May, 2007).

Diagnostic accuracy reviews were searched from the ACP

Results

Of 94 diagnosis accuracy reviews found in ACP Journal Club, 22 were published in 2006. When reviewed in full text, 13 systematic reviews met our original inclusion criteria (both MEDLINE and EMBASE searches as data sources, and available in electronic format) [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23]. The addition of the third eligibility criterion during full-text review resulted in the exclusion of one diagnostic review [11], and nine systematic reviews were

Discussion

We showed that the sensitive Clinical Queries diagnostic search filter performed extremely well by capturing 99% of included articles indexed in MEDLINE and EMBASE from our sample of 12 diagnostic systematic reviews. The original capture estimate of 95% was improved once we determined that 92% of the 24 missed articles (22 of 24) were not about diagnosis. A large proportion of missed articles that were not about diagnosis occurred in reviews that addressed both diagnosis and treatment

References (34)

J.M. Wardlaw et al.
Non-invasive imaging compared with intra-arterial angiography in the diagnosis of symptomatic carotid stenosis: a meta-analysis
Lancet
(2006)
N. Vakil et al.
Limited value of alarm features in the diagnosis of upper gastrointestinal malignancy: systematic review and meta-analysis
Gastroenterology
(2006)
J.J.Y. Sung et al.
Incidence of gastroesophageal malignancy in patients with dyspepsia in Hong Kong: implications for screening strategies
Gastrointest Endosc
(2001)
M.M.G. Leeflang et al.
Use of methodological search filters to identify diagnostic accuracy studies can lead to the omission of relevant studies
J Clin Epidemiol
(2006)
D.J. Cook et al.
Systematic reviews: synthesis of best evidence for clinical decisions
Ann Intern Med
(1997)
P.J. Devereaux et al.
Reviewing the reviewers: the quality of reporting in three secondary journals
CMAJ
(2001)
J.W. Ely et al.
Obstacles to answering doctors’ questions about patient care with evidence: qualitative study
BMJ
(2002)
InterTASC Information Specialists' Sub-Group Search Filter Resource. Available at...
R.B. Haynes et al.
Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytic survey
BMJ
(2005)
R.B. Haynes et al.
Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytic survey
BMJ
(2004)

N.L. Wilczynski et al.

Developing optimal search strategies for detecting clinically sound causation studies in MEDLINE

AMIA Annu Symp Proc

(2003)

N.L. Wilczynski et al.

Developing optimal search strategies for detecting clinically sound prognostic studies in MEDLINE

BMC Med

(2004)

S.S. Wong et al.

Developing optimal search strategies for detecting sound clinical prediction studies in MEDLINE

AMIA Annu Symp Proc

(2003)

Clinical queries in Ovid. Available at...

J.P. Gisbert et al.

Accuracy of monoclonal stool antigen test for the diagnosis of H. pylori infection: a systematic review and meta-analysis

Am J Gastroenterol

(2006)

C. Davenport et al.

Assessing the diagnostic test accuracy of natriuretic peptides and ECG in the diagnosis of felt ventricular systolic dysfunction: a systematic review and meta-analysis

Br J Gen Pract

(2006)

J.L. Martin et al.

Systematic review and evaluation of methods of assessing urinary incontinence

Health Technol Assess

(2006)

Cited by (28)

Electromyography in diagnosing temporomandibular disorders
2012, Journal of the American Dental Association
Citation Excerpt :
― We also searched key words in a selection of truncated and Medical Subject Headings terms, with the help of a librarian who specializes in health sciences databases. This search strategy has been successful in identifying diagnostic studies in other areas.33 We complemented the literature search with a manual search of references in the identified reports.
Although electromyography (EMG) has been used extensively in dentistry to assess masticatory muscle impairments in several conditions, especially temporomandibular disorders (TMDs), many investigators have questioned its psychometric properties and accuracy in diagnosing TMD.
The authors conducted a systematic review to analyze the literature critically and determine the accuracy of EMG in diagnosing TMDs. They conducted an electronic search of MEDLINE, Embase, all Evidence-Based Medicine Reviews, Allied and Complementary Medicine, Ovid HealthSTAR and SciVerse Scopus. The authors selected abstracts that fulfilled the inclusion criteria, retrieved the original articles, verified the inclusion criteria and hand searched the articles' references. They used a methodological tool (Quality Assessment of Diagnostic Accuracy Studies [QUADAS]) to evaluate the quality of the selected articles.
The electronic database search resulted in a total of 130 articles. The authors selected eight articles as potentially meeting eligibility for the review. Of these eight articles, only two fulfilled the study inclusion criteria, and the authors analyzed them. Investigators in both studies reported low sensitivity (values ranged from 0.15 to 0.40 in one study and a mean of 0.69 in the second study). In addition, investigators in the two studies reported contradictory levels of specificity (values ranged from 0.95 to 0.98 in one study, and the mean value in the second study was 0.67). The likelihood ratios and predictive values were not helpful in diagnosing TMD by means of EMG. The quality of the two studies was poor on the basis of the QUADAS checklist.
The authors of this systematic review found no evidence to support the use of EMG for the diagnosis of TMD.
Inclusion of methodological filters in searches for diagnostic test accuracy studies misses relevant studies
2012, Journal of Clinical Epidemiology
Inclusion of methodological filters in searches for diagnostic test accuracy studies misses relevant studies - Reply
2012, Journal of Clinical Epidemiology
Inclusion of methodological filters in searches for diagnostic test accuracy studies misses relevant studies
2011, Journal of Clinical Epidemiology
Citation Excerpt :
An additional important consideration is the impact of excluding “missed” studies on the review results and conclusions. Kastner et al. [33] assessed the impact of excluding studies that they judged to have been missed by their filter, but the limitations and scale of this study mean that the results of this analysis are unlikely to be reliable. This is an area that has not currently been adequately investigated and where further research is needed.
To compare the performance of MEDLINE searches using index test(s) and target condition (subject searches) with the same searches combined with methodological filters for test accuracy studies.
We derived a reference set of 506 test accuracy studies indexed on MEDLINE from seven systematic reviews that conducted extensive searches. We compared the performance of “subject” with “filtered” searches (same searches combined with each of 22 filters). Outcome measures were number of reference set records missed, sensitivity, number needed to read (NNR), and precision (Number of reference set studies identified for every 100 records screened).
Subject searches missed 47 of the 506 reference studies; filtered searches missed an additional 21 to 241 studies. Sensitivity was 91% for subject searches and ranged from 43% to 87% for filtered searches. The NNR was 56 (precision 2%) for subject searches and ranged from 7 to 51 (precision 2–15%) for filtered searches.
Filtered searches miss additional studies compared with searches based on index test and target condition. None of the existing filters provided reductions in the NNR for acceptable sensitivity; currently available methodological filters should not be used to identify studies for inclusion in test accuracy reviews.
Users' guide to the surgical literature: How to perform a high-quality literature search
2015, Canadian Journal of Surgery
The article “Users’ guide to the surgical literature: how to perform a literature search” was published in 2003, but the continuing technological developments in databases and search filters have rendered that guide out of date. The present guide fills an existing gap in this area; it provides the reader with strategies for developing a searchable clinical question, creating an efficient search strategy, accessing appropriate databases, and skillfully retrieving the best evidence to address the research question.
Development and evaluation of a search filter to identify prognostic factor studies in Ovid MEDLINE
2022, BMC Medical Research Methodology

View all citing articles on Scopus

View full text

Original ArticleDiagnostic test systematic reviews: Bibliographic search filters (“Clinical Queries”) for diagnostic accuracy studies perform well

Abstract

Background

Objectives

Methods

Results

Conclusion

Section snippets

Background

Methods

Results

Discussion

Lancet

Gastroenterology

Gastrointest Endosc

J Clin Epidemiol

Systematic reviews: synthesis of best evidence for clinical decisions

Ann Intern Med

Reviewing the reviewers: the quality of reporting in three secondary journals

CMAJ

Obstacles to answering doctors’ questions about patient care with evidence: qualitative study

BMJ

Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytic survey

BMJ

Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytic survey

BMJ

Developing optimal search strategies for detecting clinically sound causation studies in MEDLINE

AMIA Annu Symp Proc

Developing optimal search strategies for detecting clinically sound prognostic studies in MEDLINE

BMC Med

Developing optimal search strategies for detecting sound clinical prediction studies in MEDLINE

AMIA Annu Symp Proc

Accuracy of monoclonal stool antigen test for the diagnosis of H. pylori infection: a systematic review and meta-analysis

Am J Gastroenterol

Assessing the diagnostic test accuracy of natriuretic peptides and ECG in the diagnosis of felt ventricular systolic dysfunction: a systematic review and meta-analysis

Br J Gen Pract

Systematic review and evaluation of methods of assessing urinary incontinence

Health Technol Assess

Original Article
Diagnostic test systematic reviews: Bibliographic search filters (“Clinical Queries”) for diagnostic accuracy studies perform well