Research report
Validation of the Center for Epidemiologic Studies Depression Scale in screening for major depressive disorder among retired firefighters exposed to the World Trade Center disaster

https://doi.org/10.1016/j.jad.2009.05.028Get rights and content

Abstract

Background

We evaluated the performance of a modified Center of Epidemiologic Studies Depression Scale (CES-D-m), which captured symptoms in the past month, in comparison to the Diagnostic Interview Schedule (DIS) in identification of major depressive disorder (MDD) in World Trade Center (WTC)-exposed retired Fire Department, City of New York (FDNY) firefighters.

Methods

From 12/2005 to 7/2007, FDNY enrolled retired firefighters in its Medical Monitoring and Treatment Program. All participants completed the CES-D-m and the DIS on the same day. Sensitivity, specificity, receiver operating characteristic (ROC) curves, and Youden's index were used to assess properties of the CES-D-m. Multivariate logistic regression analyses were also used.

Results

7% of 1915 retired male firefighters were diagnosed with MDD using the DIS. Using the most common CES-D cutoff score of 16, the prevalence of elevated risk was 36%, which declined to 23% using a cutoff score of 22, as determined by Youden's index. At 22, CES-D-m sensitivity was 0.84, specificity was 0.82, and the area under the ROC curve was 0.89 relative to DIS MDD diagnosis.

Limitations

Participants were more likely than non-participants to live in the New York City area.

Conclusions

This is the first study of WTC rescue/recovery workers to assess the performance of a one-month version of the CES-D. The CES-D-m performed well in identifying those at elevated risk. Since diagnostic follow-up is time consuming and costly, it is important to correctly distinguish those at elevated risk using a screening tool that has been validated in the population under study.

Introduction

On September 11, 2001 (9/11), approximately 11,500 Fire Department, City of New York (FDNY) firefighters responded to the terrorist attacks on the World Trade Center (WTC). The rescue and recovery effort exposed many of the responders to physical and emotional trauma, putting them at risk for various mental health disorders including major depressive disorder (MDD). Although the etiology of MDD is not clearly understood (Person et al., 2006), several studies performed on specific post-disaster populations (North et al., 1999, Tak et al., 2007), including 9/11 (Galea et al., 2002, Gross et al., 2006, Long et al., 2007, Stellman et al., 2008, Tapp et al., 2005), highlight the concern of experts in the mental health community over depressive symptoms following traumatic events (Katz et al., 2002). A study conducted in October and November 2001 of Manhattan residents living less than 1 mile from the WTC reported 16.8% with depression-like symptoms (Galea et al., 2002). In addition, 12% of a sample of NYC transit workers stationed near the WTC towers screened in 2002 (Tapp et al., 2005) and 16.1% of a sample of WTC clean up and recover workers screened in 2003 reported symptoms consistent with major depression (Gross et al., 2006).

Given the prevalence of and health risks associated with depression, self-administered screening tools provide practical means for assessing depressive symptoms. Screening tools identify persons who reach a predetermined threshold or cutoff score for follow-up with more intensive and time-consuming diagnostic tests. The Center for Epidemiologic Studies Depression Scale (CES-D) was designed in 1977 to screen for depressive symptoms in the general population. It consists of 20 self-reported items, rated on a 4-point scale of symptom frequency, and is scored from 0 to 60 (Radloff, 1977), with higher scores more likely to be associated with a MDD diagnosis. While the original work reported an optimum cutoff score of 16 in the general population, others have reported cutoff scores from 12 (Watson et al., 2004) to 25 (Haringsma et al., 2004) in specific populations, with 16 most frequently used (Mair et al., 2009, Schein and Koenig, 1997, Shinar et al., 1986, Snoek et al., 2008).

Following administration of a screening instrument, those who exceed a specified threshold are advised to complete a follow-up diagnostic interview for a more comprehensive review of MDD symptoms. The Diagnostic Interview Schedule (DIS) is a structured diagnostic interview designed to assist in making a MDD diagnosis according to Diagnostic and Statistical Manual for Mental Disorders, version IV (DSM-IV-TR®) criteria (Robins et al., 1995). Full assessment of accepted diagnostic criteria (such as with the DIS or clinical interviews) is required for diagnosis of MDD.

The DSM-IV-TR® identifies nine symptoms that are associated with a diagnosis of MDD and requires that at least five of these nine symptoms be present over a two-week period. In contrast, the CES-D assesses symptoms only during the past week (Radloff, 1977) and therefore encompasses a time frame that is not consistent with the DSM-IV-TR® definition of MDD.

To our knowledge, no study has determined the optimal cutoff score for depression using the CES-D or any other tool in a population of rescue/recovery workers after a major disaster. The purpose of the current study was to evaluate the performance characteristics of the CES-D modified to encompass a one-month assessment of depressive symptoms (CES-D-m) in relation to a full diagnostic assessment using a structured diagnostic interview (the DIS) in a large retired population of WTC-exposed firefighters. Further, since the FDNY is currently using a cutoff score of 16, which means that only persons scoring at or above that level receive a full diagnostic assessment, our intent was to evaluate the screening efficiency of this cutoff.

Section snippets

Methods

Since 1997, the FDNY Bureau of Health Services (BHS) performs periodic health evaluations on active FDNY members approximately every 18 months, which include both physician examinations and, since 2001, self-administered health questionnaires. In 2005, a more extensive mental health questionnaire incorporated the CES-D. This occurred co-incident with outreach to retired WTC-exposed firefighters, who were invited to resume participation in health screenings, which were previously limited to the

Characteristics of the study population

As shown in Table 1, the 1915 retired firefighters averaged 47.0 years of age on 9/11 (SD = 6.9; range 28.0 to 73.4 years). The participants were predominantly married (83%) and about half had some college education or an associate degree (48%). At the time of the exam, 4 to 6 years after 9/11, 10% reported a change in marital status.

Most participants arrived at the WTC site on 9/11 (72%), with the majority arriving after the collapse of the WTC towers, during the afternoon of 9/11 (56%). By the

Discussion

This study tests the performance of the CES-D-m in comparison to DIS identification of MDD in WTC-exposed retired FDNY firefighters. Previous studies in other populations have used the CES-D in its original form, assessing symptom occurrence in the past week (Beekman et al., 1995, Cloninger et al., 2006, Schein and Koenig, 1997, Zauszniewski and Graham, 2009). The DSM-IV-TR® criteria require symptoms to extend beyond the immediate 7 days prior to screening. We chose to modify the CES-D to

Conclusions

We believe that the CES-D-m is an effective way to screen for MDD in retired male firefighters. On the basis of ROC analyses and Youden's index, where neither sensitivity nor specificity is favored, the optimal cutoff score in our population was 22. This cutoff score is higher than what was estimated by the original study carried out in the general population rather than in firefighters or other disaster workers (Radloff, 1977). Studies in other populations, disaster and non-disaster exposed,

Role of funding source

Funding for this work was provided by the National Institute for Occupational Safety and Health (NIOSH) RO1-OH07350. NIOSH had no further role in study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.

Conflict of interest

The authors declare that they have no competing interests.

Acknowledgements

This work was supported by the National Institute for Occupational Safety and Health.

References (40)

  • ChengS.T. et al.

    The Center for Epidemiologic Studies Depression Scale in older Chinese: thresholds for long and short forms

    Int. J. Geriatr. Psychiatry

    (2005)
  • Corrigan, M., McWilliams, R., Kelly, K.J., Niles, J., Cammarata, C., Jones, K., Glass, L., Wartenberg, D., Halliman,...
  • CuijpersP. et al.

    Screening of depression in adolescents through the Internet: sensitivity and specificity of two screening questionnaires

    Eur. Child Adolesc. Psychiatry

    (2008)
  • DboukN. et al.

    Assessment of the PHQ-9 as a screening tool for depression in patients with chronic hepatitis C

    Dig. Dis. Sci.

    (2008)
  • FountoulakisK. et al.

    Reliability, validity and psychometric properties of the Greek translation of the Center for Epidemiological Studies—Depression (CES-D) Scale

    BMC Psychiatry

    (2001)
  • GaleaS. et al.

    Psychological sequelae of the September 11 terrorist attacks in New York City

    N. Engl. J. Med.

    (2002)
  • GhubashR. et al.

    The performance of the Center for Epidemiologic Study Depression Scale (CES-D) in an Arab female community

    Int. J. Soc. Psychiatry

    (2000)
  • GoebertD. et al.

    Depressive symptoms in medical students and residents: a multischool study

    Acad. Med.

    (2009)
  • GrossR. et al.

    Posttraumatic stress disorder and other psychological sequelae among World Trade Center clean up and recovery workers

    Ann. N. Y. Acad. Sci.

    (2006)
  • HaringsmaR. et al.

    The criterion validity of the Center for Epidemiological Studies Depression Scale (CES-D) in a sample of self-referred elders with depressive symptomatology

    Int. J. Geriatr. Psychiatry

    (2004)
  • Cited by (33)

    • Pandemic-related mental health risk among front line personnel

      2021, Journal of Psychiatric Research
      Citation Excerpt :

      As such, the mental health of emergency and medical personnel is critical to public health and national adaptation and recovery efforts. During typical non-crisis times, emergency personnel are at increased risk for psychiatric disorders, including depression (Berg et al., 2006; Chiu et al., 2010; Fullerton et al., 2004; Hartley et al., 2011; Stanley et al., 2018) and posttraumatic stress disorder (PTSD; Boffa et al., 2017; Boffa et al., 2018; Kim et al., 2018). These risks increase when emergency personnel respond to events that are extraordinarily stressful and potentially traumatic (Fullerton et al., 2004), such as natural disasters (Osofsky et al., 2011) and terrorist attacks (Soo et al., 2011; Luft et al., 2012).

    • Occupational stress and suicidality among firefighters: Examining the buffering role of distress tolerance

      2018, Psychiatry Research
      Citation Excerpt :

      Participants self-report the frequency of depression symptoms in the past week on a 4-point scale (0 = never or rarely/less than 1 day per week to 3 = almost all the time/5–7 days per week); higher total scores indicate greater severity of depression symptoms. This instrument has been validated in a sample of firefighters (Chiu et al., 2010). In the current study, the CES-D was analyzed as a continuous variable and internal consistency was good (α = 0.82).

    • Psychological impacts of disaster on rescue workers: A review of the literature

      2018, International Journal of Disaster Risk Reduction
      Citation Excerpt :

      Ten of the 20 studies focused solely on the depression of rescue workers. The instruments using to measure depression were the studies employed the Centers for Epidemiological Studies Depression Scale (CESD) [40-44], the Zung Self-Rating Depression Scale (ZSRD) [27,28,45,46], and the Traumatic Stress Symptom Checklist (TSSC) [47]. The prevalence of anxiety was high at 27.2%, 31.7%, and 25.5% among firefighters, police officers, and rescue teams, respectively.

    • Relationships between depression, pain and sleep quality with doctor visits among community-based adults in north-west China

      2017, Public Health
      Citation Excerpt :

      The internal consistency coefficient (Cronbach's alpha) of the whole scale in the present study was 0.90. Radloff suggested that a score of 16 should be the cut-off point for depression on this scale,14 but some studies have adopted 21 or 22 as the cut-off score.15 Studies in China have suggested that the original cut-off score of 16 has a low positive predictive value16 and may be too low for use in Chinese populations.13

    View all citing articles on Scopus
    View full text