Research report
Brief self-rated screening for depression on the Internet

https://doi.org/10.1016/j.jad.2009.07.013Get rights and content

Abstract

Background

The Internet offers promising possibilities for the quick screening of depression for treatment and research purposes. This paper aims to validate three self-rated measures to screen for depression on the Internet: SID (single-item depression scale), CES-D (Center for Epidemiological Studies Depression scale) and K10 (Kessler psychological distress scale).

Methods

Of the 502 subjects aged 18–80 who rated the SID, CES-D and K10 measures on the Internet, 157 (31%) subjects were also interviewed by telephone using the WHO Composite International Diagnostic Interview (C)IDI) for DSM-IV-disorders.

Results

Cronbach's α for both web self-rated measures CES-D and K10 was 0.90. The SID correlated 0.68 (P < 0.001) with the CES-D and with the K10. The CES-D correlated 0.84 with the K10 (P < 0.001). Subjects with a DSM-IV diagnosis for any depressive disorder had significantly higher means (P < 0.001) on the three self-rated measures for depressive symptoms than subjects without a diagnosis of any depressive disorder. Using any depressive disorder as the gold standard, the area under the curve (AUC) of the SID was 0.71 (95% CI: 0.63–0.79), which was significantly lower than the AUC of the CES-D (AUC: 0.84; 95% CI: 0.77–0.90, P = 0.003) and of the K10 (AUC: 0.81; 95% CI: 0.73–0.88, P = 0.0024). The AUCs for the K10 and CES-D did not differ significantly from each other.

Limitations

The CIDI interviews were not recorded, so inter-rater reliability could not be calculated.

Conclusions

The CES-D and K10 are reliable, valid tools for care providers to quickly screen depressive patients on the Internet and for researchers to collect data.

Introduction

Estimations of 12-month prevalence rates of depressive disorders vary between 6.6% (Andrews et al., 2001) to 11% (Kessler et al., 1994) in the general population and 12% in a primary care population (Sartorius et al., 1996). Depressive disorders are among the top four leading causes of disease burden worldwide (Lopez and Murray, 1998). They cause serious disability (van Schaik et al., 2007), reduced quality of life (Cuijpers et al., 2004), and incur huge economic costs (Cuijpers et al., 2007b). Their identification and treatment are therefore important. Screening for such disorders, using reliable and valid self-report questionnaires, needs to be improved and made more user-friendly, both to researchers and health care providers.

Detection would improve with simpler, shorter, more reliable and valid depression self-rating (U.S. Preventive Services Task Force, 2002). This kind of improved self-rating would be conducted more readily by subjects (Cuijpers et al., 2009) and save time for health care providers who have many competing demands on their time.

Screening conducted via the Internet offers easy and quick access to large numbers of users at low cost (Austin et al., 2006, Buchanan, 2003). Collecting data on the Internet saves researchers time and organization, minimizes data-entry errors by allowing automatic transcription into a computerized database (Coles et al., 2007), and can reduce missing values by making responses to all items obligatory before submission (Austin et al., 2006). Moreover, people sometimes disclose more sensitive information in computer-based compared to face-to-face interviews (Buchanan, 2002, Davis, 1999, Joinson, 1999) as ‘the computer has no eyebrows’ (Marks et al., 2007). However, there are several disadvantages of web-based screening as well. For example, the anonymous nature of the Internet allows people to participate frivolously or with malicious intent, which can affect the data quality. Furthermore, regarding ethical principles, it is more difficult to assess the subjects' identities or their reactions to the research experience online in case surveys might upset the test-taker (Kraut et al., 2004).

Though several studies have found equivalent psychometric properties in web-based versus paper–pencil questionnaires (Andersson et al., 2003, Carlbring et al., 2007, Houston et al., 2001, Spek et al., 2008), other studies did not (reviewed by Buchanan (2002), which means that the psychometric equivalence cannot be assumed (Buchanan, 2002, Buchanan, 2003). One factor which might affect the reliability and validity of self-ratings on the Internet is variability in presentation of the test across different computers due to technical discrepancies between different hardware and software configurations (Austin et al., 2006, Buchanan and Smith, 1999). Furthermore, the heterogeneity (e.g. age, education, socio-economic status) of Internet users is increasing which may introduce unknown confounding variables, possibly adding to ‘noise’ in the data and reducing the proportion of variance in responses accounted for by differences in whatever (e.g. depression) one is trying to measure (Buchanan and Smith, 1999). Variations in the amount of control over the testing environment (e.g. at home versus the lab, and in rater mood or fatigue) might influence the validity of web questionnaires (Buchanan and Smith, 1999, Davis, 1999), as they do too for paper–pencil administration. And, as mentioned earlier, social-desirability effects might be less pronounced for web-based than paper–pencil administration (Joinson, 1999). It is therefore necessary to check the validity of rating each measure on the Internet before adopting it (Buchanan, 2002, Buchanan and Smith, 1999).

This study aims to validate Internet-based screening of depression by three self-rated measures – the Center for Epidemiological Studies Depression Scale (CES-D) and the Kessler psychological distress scale (K10; see below for details), and the Single-Item Depression (SID) scale. Selection was based on the psychometric properties of the paper–pencil versions, their understandability, and their availability without charge. Diagnosis in a standard diagnostic interview was used as the ‘gold standard’.

Section snippets

Participants and procedure

Data for this study were collected as part of a larger investigation of a brief, web-based screener (WSQ) for common mental disorders (detailed in Donker et al., 2009). In short, participants were recruited from the general population by using Internet banners (Google, Dutch Internet-sites on mental health issues). We targeted adults aged 18 or older who were anxious, depressed or thought of themselves as drinking too much alcohol – the kind of people for whom the WSQ is intended. We expected

Demographics

The total sample (N = 502) had a mean age of 43 (SD 13, range 18–80); and 285 (57%) of the subjects were female; the majority was Dutch (n = 474, 94%) and 217 (43%) subjects received medium education (Intermediate Vocational Training [community college], school of higher general secondary education or pre-university education). Of the 157 subjects who had a CIDI interview, the mean age was 43 (SD 15, range 18–80); 89 (57%) were female; the majority was Dutch (n = 146, 94%) and 73 (47%) subjects

Discussion

Findings from our study suggest that both the web-based CES-D and web-based K10 yield reliable (Cronbach's α 0.90–0.92) and valid (AUC 0.81–0.84) self-ratings for depressive disorders which are similar to the paper–pencil ratings (Beekman et al., 1997, Donker et al., 2010). The SID was moderately accurate (AUC 0.71) and a cut-off score of 5 gave high sensitivity (0.87) but lower specificity (0.51) compared to findings from previous research (McKenzie and Marks, 1999). Reducing the number of

Role of funding source

None.

Conflict of interest

None.

Acknowledgements

This study is funded by the Faculty of Psychology and Education of the VU University, Amsterdam.

References (53)

  • G. Andrews et al.

    The psychometric properties of the Composite International Diagnostic Interview

    Soc. Psychiatry Psychiatr. Epidemiol.

    (1998)
  • G. Andrews et al.

    Prevalence, comorbidity, disability and service utilization. Overview of the Australian National Mental Health Survey

    Br. J. Psychiatry

    (2001)
  • D.W. Austin et al.

    Internet administration of three commonly used questionnaires in panic research: equivalence to paper administration in Australian and Swedish samples of people with panic disorder

    Int. J. Test.

    (2006)
  • A.T. Beck et al.

    Cognitive Therapy of Depression

    (1979)
  • A.T. Beekman et al.

    Criterion validity of the Center for Epidemiologic Studies Depression scale (CES-D): results from a community-based sample of older subjects in The Netherlands

    Psychol. Med.

    (1997)
  • T. Buchanan

    Online assessment: desirable or dangerous?

    Prof. Psychol.

    (2002)
  • T. Buchanan

    Internet-based questionnaire assessment: appropriate use in clinical contexts

    Cogn. Behav. Ther.

    (2003)
  • T. Buchanan et al.

    Using the Internet for psychological research: personality testing on the world-wide web

    Br. J. Psychol.

    (1999)
  • J. Cairney et al.

    Evaluation of 2 measures of psychological distress as screeners for depression in the general population

    Can. J. Psychiatry

    (2007)
  • P. Cuijpers et al.

    Screening of depression in adolescents through the Internet: sensitivity and specificity of two screening questionnaires

    Eur. Child Adolesc. Psychiatry

    (2007)
  • P. Cuijpers et al.

    Economic costs of minor depression: a population-based study

    Acta Psychiatr. Scand.

    (2007)
  • R.N. Davis

    Web-based administration of a personality questionnaire: comparison with traditional methods

    Behav. Res. Meth. Instrum. Comput.

    (1999)
  • Donker, T., van Straten, A., Marks, I.M., Cuijpers, P., 2009. A brief web-based screening questionnaire for common...
  • M. Evans et al.

    Assessing mental health in primary care research using standardized scales: can it be carried out over the telephone?

    Psychol. Med.

    (2004)
  • J.E. Fischer et al.

    A readers' guide to the interpretation of diagnostic test properties: clinical example of sepsis

    Intensive Care Med.

    (2003)
  • T.A. Furukawa et al.

    The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-Being

    Psychol. Med.

    (2003)
  • Cited by (73)

    • How useful is the center for epidemiologic studies depression scale in screening for depression in adults? An updated systematic review and meta-analysis<sup>✰</sup>

      2021, Psychiatry Research
      Citation Excerpt :

      In the sensitivity analysis, the pooled sensitivity of the studies applying a cutoff of 16 was slightly lower at 0.84, but the pooled specificity was similar. The sensitivity was high in two studies (Donker et al., 2010; Wada et al., 2007) where the cutoff scores were higher than 16, but the specificity was 0.62 and 0.93, respectively. In the 14 studies for patients with chronic diseases, most (eight studies, 57.1%) applied cutoff scores of 20–23.

    • A computerized version of the Patient Health Questionnaire-4 as an ultra-brief screening tool to detect emotional disorders in primary care

      2018, Journal of Affective Disorders
      Citation Excerpt :

      Several authors have proposed internet-based testing to avoid some of the limitations of paper-and-pencil tests or face-to-face interviews (e.g. sameness through stigma or social desirability) and to improve the cost-effectiveness of screening (Aboraya et al., 2005; Barak and English, 2002; Luxton et al., 2014). Web-based screening questionnaires have also been proposed to screen for mental disorders, with several studies showing that such tests are both reliable and feasible, with the benefit of facilitating online data collection (Donker et al., 2010, 2011; Lin et al., 2007; Nguyen et al., 2015; van Ballegooijen et al., 2012). Online screening questionnaires have also shown good psychometric properties in PC settings to identify ED (Farvolden et al., 2003; Muñoz-Navarro et al., 2017a, 2017b, 2016).

    View all citing articles on Scopus
    View full text