Research reportBrief self-rated screening for depression on the Internet
Introduction
Estimations of 12-month prevalence rates of depressive disorders vary between 6.6% (Andrews et al., 2001) to 11% (Kessler et al., 1994) in the general population and 12% in a primary care population (Sartorius et al., 1996). Depressive disorders are among the top four leading causes of disease burden worldwide (Lopez and Murray, 1998). They cause serious disability (van Schaik et al., 2007), reduced quality of life (Cuijpers et al., 2004), and incur huge economic costs (Cuijpers et al., 2007b). Their identification and treatment are therefore important. Screening for such disorders, using reliable and valid self-report questionnaires, needs to be improved and made more user-friendly, both to researchers and health care providers.
Detection would improve with simpler, shorter, more reliable and valid depression self-rating (U.S. Preventive Services Task Force, 2002). This kind of improved self-rating would be conducted more readily by subjects (Cuijpers et al., 2009) and save time for health care providers who have many competing demands on their time.
Screening conducted via the Internet offers easy and quick access to large numbers of users at low cost (Austin et al., 2006, Buchanan, 2003). Collecting data on the Internet saves researchers time and organization, minimizes data-entry errors by allowing automatic transcription into a computerized database (Coles et al., 2007), and can reduce missing values by making responses to all items obligatory before submission (Austin et al., 2006). Moreover, people sometimes disclose more sensitive information in computer-based compared to face-to-face interviews (Buchanan, 2002, Davis, 1999, Joinson, 1999) as ‘the computer has no eyebrows’ (Marks et al., 2007). However, there are several disadvantages of web-based screening as well. For example, the anonymous nature of the Internet allows people to participate frivolously or with malicious intent, which can affect the data quality. Furthermore, regarding ethical principles, it is more difficult to assess the subjects' identities or their reactions to the research experience online in case surveys might upset the test-taker (Kraut et al., 2004).
Though several studies have found equivalent psychometric properties in web-based versus paper–pencil questionnaires (Andersson et al., 2003, Carlbring et al., 2007, Houston et al., 2001, Spek et al., 2008), other studies did not (reviewed by Buchanan (2002), which means that the psychometric equivalence cannot be assumed (Buchanan, 2002, Buchanan, 2003). One factor which might affect the reliability and validity of self-ratings on the Internet is variability in presentation of the test across different computers due to technical discrepancies between different hardware and software configurations (Austin et al., 2006, Buchanan and Smith, 1999). Furthermore, the heterogeneity (e.g. age, education, socio-economic status) of Internet users is increasing which may introduce unknown confounding variables, possibly adding to ‘noise’ in the data and reducing the proportion of variance in responses accounted for by differences in whatever (e.g. depression) one is trying to measure (Buchanan and Smith, 1999). Variations in the amount of control over the testing environment (e.g. at home versus the lab, and in rater mood or fatigue) might influence the validity of web questionnaires (Buchanan and Smith, 1999, Davis, 1999), as they do too for paper–pencil administration. And, as mentioned earlier, social-desirability effects might be less pronounced for web-based than paper–pencil administration (Joinson, 1999). It is therefore necessary to check the validity of rating each measure on the Internet before adopting it (Buchanan, 2002, Buchanan and Smith, 1999).
This study aims to validate Internet-based screening of depression by three self-rated measures – the Center for Epidemiological Studies Depression Scale (CES-D) and the Kessler psychological distress scale (K10; see below for details), and the Single-Item Depression (SID) scale. Selection was based on the psychometric properties of the paper–pencil versions, their understandability, and their availability without charge. Diagnosis in a standard diagnostic interview was used as the ‘gold standard’.
Section snippets
Participants and procedure
Data for this study were collected as part of a larger investigation of a brief, web-based screener (WSQ) for common mental disorders (detailed in Donker et al., 2009). In short, participants were recruited from the general population by using Internet banners (Google, Dutch Internet-sites on mental health issues). We targeted adults aged 18 or older who were anxious, depressed or thought of themselves as drinking too much alcohol – the kind of people for whom the WSQ is intended. We expected
Demographics
The total sample (N = 502) had a mean age of 43 (SD 13, range 18–80); and 285 (57%) of the subjects were female; the majority was Dutch (n = 474, 94%) and 217 (43%) subjects received medium education (Intermediate Vocational Training [community college], school of higher general secondary education or pre-university education). Of the 157 subjects who had a CIDI interview, the mean age was 43 (SD 15, range 18–80); 89 (57%) were female; the majority was Dutch (n = 146, 94%) and 73 (47%) subjects
Discussion
Findings from our study suggest that both the web-based CES-D and web-based K10 yield reliable (Cronbach's α 0.90–0.92) and valid (AUC 0.81–0.84) self-ratings for depressive disorders which are similar to the paper–pencil ratings (Beekman et al., 1997, Donker et al., 2010). The SID was moderately accurate (AUC 0.71) and a cut-off score of 5 gave high sensitivity (0.87) but lower specificity (0.51) compared to findings from previous research (McKenzie and Marks, 1999). Reducing the number of
Role of funding source
None.
Conflict of interest
None.
Acknowledgements
This study is funded by the Faculty of Psychology and Education of the VU University, Amsterdam.
References (53)
- et al.
Internet administration of the Hospital Anxiety and Depression Scale in a sample of tinnitus patients
J. Psychosom. Res.
(2003) - et al.
Interpreting scores on the Kessler Psychological Distress Scale (K10)
Aust. N.Z.J. Public Health
(2001) - et al.
Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research
Comput. Hum. Behav.
(2007) - et al.
Assessing obsessive compulsive symptoms and cognitions on the Internet: evidence for the comparability of paper and Internet administration
Behav. Res. Ther.
(2007) - et al.
Minor depression: risk profiles, functional disability, health care use and risk of developing major depression
J. Affect. Disord.
(2004) - et al.
Screening for mood and anxiety disorders with the five-item, the three-item, and the two-item Mental Health Inventory
Psychiatry Res.
(2009) - et al.
The validity of the Dutch K10 and EK10 screening scales for depressive and anxiety disorders
Psychiatry Research
(2010) - et al.
Internet administration of the Edinburgh Depression Scale
J. Affect. Disord.
(2008) Reliability and validity studies of the WHO-Composite International Diagnostic Interview (CIDI): a critical review
J. Psychiatr. Res.
(1994)Diagnosis and statistical manual of mental disorders
(1994)
The psychometric properties of the Composite International Diagnostic Interview
Soc. Psychiatry Psychiatr. Epidemiol.
Prevalence, comorbidity, disability and service utilization. Overview of the Australian National Mental Health Survey
Br. J. Psychiatry
Internet administration of three commonly used questionnaires in panic research: equivalence to paper administration in Australian and Swedish samples of people with panic disorder
Int. J. Test.
Cognitive Therapy of Depression
Criterion validity of the Center for Epidemiologic Studies Depression scale (CES-D): results from a community-based sample of older subjects in The Netherlands
Psychol. Med.
Online assessment: desirable or dangerous?
Prof. Psychol.
Internet-based questionnaire assessment: appropriate use in clinical contexts
Cogn. Behav. Ther.
Using the Internet for psychological research: personality testing on the world-wide web
Br. J. Psychol.
Evaluation of 2 measures of psychological distress as screeners for depression in the general population
Can. J. Psychiatry
Screening of depression in adolescents through the Internet: sensitivity and specificity of two screening questionnaires
Eur. Child Adolesc. Psychiatry
Economic costs of minor depression: a population-based study
Acta Psychiatr. Scand.
Web-based administration of a personality questionnaire: comparison with traditional methods
Behav. Res. Meth. Instrum. Comput.
Assessing mental health in primary care research using standardized scales: can it be carried out over the telephone?
Psychol. Med.
A readers' guide to the interpretation of diagnostic test properties: clinical example of sepsis
Intensive Care Med.
The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-Being
Psychol. Med.
Cited by (73)
How useful is the center for epidemiologic studies depression scale in screening for depression in adults? An updated systematic review and meta-analysis<sup>✰</sup>
2021, Psychiatry ResearchCitation Excerpt :In the sensitivity analysis, the pooled sensitivity of the studies applying a cutoff of 16 was slightly lower at 0.84, but the pooled specificity was similar. The sensitivity was high in two studies (Donker et al., 2010; Wada et al., 2007) where the cutoff scores were higher than 16, but the specificity was 0.62 and 0.93, respectively. In the 14 studies for patients with chronic diseases, most (eight studies, 57.1%) applied cutoff scores of 20–23.
Associations of combined exposures to surrounding green, air pollution and traffic noise on mental health
2019, Environment InternationalA computerized version of the Patient Health Questionnaire-4 as an ultra-brief screening tool to detect emotional disorders in primary care
2018, Journal of Affective DisordersCitation Excerpt :Several authors have proposed internet-based testing to avoid some of the limitations of paper-and-pencil tests or face-to-face interviews (e.g. sameness through stigma or social desirability) and to improve the cost-effectiveness of screening (Aboraya et al., 2005; Barak and English, 2002; Luxton et al., 2014). Web-based screening questionnaires have also been proposed to screen for mental disorders, with several studies showing that such tests are both reliable and feasible, with the benefit of facilitating online data collection (Donker et al., 2010, 2011; Lin et al., 2007; Nguyen et al., 2015; van Ballegooijen et al., 2012). Online screening questionnaires have also shown good psychometric properties in PC settings to identify ED (Farvolden et al., 2003; Muñoz-Navarro et al., 2017a, 2017b, 2016).