Mass colorectal cancer screening: Methodological quality of practice guidelines is not related to their content validity
Introduction
The practice of evidence-based medicine (EBM) means integrating the best research evidence with clinical expertise and patient values [1]. It is clearly established that the current methodological quality of clinical practice guidelines (GLs), including evidence-based GLs, needs to improve considerably [2], [3]. As a consequence, a lot of energy is currently being spent to try to improve the situation [4]. However, it is not really established whether or not better methodological quality leads to more valid recommendations, i.e. those that are supported with consistent research evidence or, when evidence is conflicting or lacking, with sufficient consensus among the GL development team, and that, when implemented, are more likely to improve the balance between benefits and harms/costs.
In previous work [5], we showed that validity, and methodological quality of recommendations in GLs were not related to each other in 11 GLs providing advice for the use of laboratory tests in non-small cell lung cancer patients. However, this work had two main limitations: (1) some of the systematic reviews (SRs) that we used to establish the validity of the laboratory-related recommendations were our own SRs; (2) the evidence in the field we investigated was not of high quality; in particular, no randomized controlled trials (RCTs) had been published. We therefore wished to undertake similar work in an area where the evidence is of higher quality (i.e. RCTs are available), and where more SRs have been published, by workers other than ourselves.
In colorectal cancer (CRC) patients, cure or long-term survival is much more likely to be achieved when the disease is diagnosed at an early stage. SRs of RCTs have shown that fecal occult blood test (FOBT) is a screening test that can decrease CRC-mortality at a population level. Many governmental or professional organizations worldwide advocate FOBT mass-screening for CRC.
Section snippets
Practice guidelines
We systematically searched for GLs providing advice for the use of FOBT as a screening test in colorectal cancer, using the strategy described previously which includes manual searches [6]. Key search terms were “colorectal cancer” and “practice guidelines”. We excluded GLs published before 2000 for the same reasons as those which led us to exclude SRs published before 2000 (see next section). We also excluded GLs whose recommendations were unclear or not relevant to our work, e.g. GLs
Practice guidelines
Our literature search retrieved 36 GLs, and our selection criteria enabled us to include 12 of these [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23]. The FOBT-related recommendations made in these 12 GLs are summarized in Table 1. All GLs agree, formally or implicitly, that FOBT can only be used as a screening test for colorectal cancer in asymptomatic, average-risk patients, thus excluding patients with a familial or personal history of colorectal cancer, as well as
Discussion
Many of the 12 GLs fell short of basic quality criteria, a result that confirms previous observations in various areas of medicine [3]. Among the six domains comprising the AGREE instrument which are generally accepted to cover the key elements of the GL development process [4], the 1st domain, i.e. scope and purpose, is the one which is the most satisfactorily addressed, with scores above 80% in 11 of 12 GLs. Likewise, the domain which is the least satisfactorily addressed is the 6th domain,
Conclusion
It is generally believed that evidence-based GLs can provide health-care professionals with valid recommendations, i.e., that are supported with consistent research evidence or sufficient consensus among the GL development team when evidence is conflicting or lacking. These, when implemented, should be more likely to improve the balance between benefits and harms, than the opposite [32]. Our preliminary work [5] as well as the present study would not really confirm this belief, at least in the
Acknowledgments
The authors acknowledge the helpful comments from Dr Catherine Dubé, Gastroenterologist, Calgary, Alberta, Canada. This work was presented during a symposium entitled “evidence in action” (S.Sandberg, W.Oosterhuis, Chairmen) at EUROMEDLAB on the 3 rd of June 2007 (RAI Congress Centre, Amsterdam, The Netherlands).
References (37)
- et al.
Systematic reviews in laboratory medicine: principles, processes and practical considerations
Clin. Chim. Acta
(2004) - et al.
Colorectal cancer screening and surveillance: clinical guidelines and rationale. Update based on new evidence
Gastroenterology
(2003) - et al.
Choice of fecal occult blood tests for colorectal cancer screening: recommendations based on performance characteristics in population studies: a WHO (World Health Organization) and OMED (World Organization for Digestive Endoscopy) report
Am. J. Gastroenterol.
(2002) - et al.
Beyond the evidence in clinical guidelines
Lancet
(2004) - et al.
Screening for gestational diabetes: variation in guidelines
Eur. J. Obstet Gynecol. Reprod. Biol.
(2000) - et al.
- et al.
Practice guidelines developed by specialty societies: the need for a critical appraisal
Lancet
(2000) - et al.
Critical appraisal of guidelines (chapter 14)
- AGREE (Appraisal of Guidelines for Research and Evaluation) Collaboration. Development and validation of an...
- et al.
Conflict between guideline methodological quality and recommendation validity: a potential problem for practitioners
Clin. Chem.
(2006)
Quality of guidelines for the laboratory management of diabetes mellitus
Scand J. Clin. Lab. Invest Suppl.
For the Hedges Team. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey
BMJ
Screening for colorectal cancer using the faecal occult blood test
Hemoccult. Cochrane Database Syst Rev.
Users' guides to the medical literature. VI. How to use an overview. Evidence-based medicine working group
JAMA
Evidence-based medicine: what it is and what it isn't
BMJ
American Cancer Society guidelines for the early detection of cancer
CA Cancer J. Clin.
Colorectal cancer screening in the UK: joint position statement by the British Society of Gastroenterology, The Royal College of Physicians, and The Association of Coloproctology of Great Britain and Ireland
Gut
Canadian Association of Gastroenterology and the Canadian Digestive Health Foundation: guidelines on colon cancer screening
Can. J. Gastroenterol.
Cited by (16)
Critical care guidelines on pain, agitation and delirium management: Which one to use? A systematic literature search and quality appraisal with AGREE II
2020, Journal of Critical CareCitation Excerpt :AGREE II evaluates the methodology of guideline development, technical aspects and the dissemination, but not the quality of a guideline's content or validity of its statements [24]. This is an important distinction, as high-quality developmental process does not necessarily translate into high-quality content [40]. The number of recommendations varies significantly, which suggests a difference in either depth and/or covered scope of the CPGs.
Systematic review of the methodological quality of clinical guideline development for the management of chronic disease in Europe
2012, Health PolicyCitation Excerpt :The reasons for study exclusion during the full text screening are reported in Table 2. Nine studies reported in 10 papers were included, analysing a total of 28 European guidelines from eight European countries and two pan-European organisations [33–42] (Table 3). The clinical remit and scope of the guidelines varied, with four of the nine papers focusing on type 2 diabetes (management of diabetes in pregnancy [37], laboratory management of type 2 diabetes [36], clinical management of diabetes type 2, e.g. management of blood glucose, lipids and renal disease [34,40,41]); two studies focused on cancer (colorectal cancer screening [33] and breast cancer [42]), one on cardiovascular disease [38], one on depression [35] and one on the management of knee osteoarthritis [39].
AGREE II - Improving the quality of clinical care
2010, The LancetIs it time to develop AGREE III?
2019, CMAJQuality evidence important for quality guidelines
2010, CMAJ. Canadian Medical Association Journal