Depressive Response Sets due to gender and culture-based Differential Item Functioning

Abstract

Two studies tested a “strong” version of Nolen-Hoeksema's [Nolen-Hoeksema, S. (1987). Sex difference in unipolar depression: evidence and theory. Psychological Bulletin, 101, 259–282.] hypothesis of depressive response sets using samples of Australian respondents (Study I, n=1111) as well as US respondents (Study II, n=300), using a Rasch version of Thalbourne's Manic-Depressiveness Scale (MDS) whose contents are consistent with atypical depression (i.e. depressive episodes with hypomanic symptoms). As predicted, tests for differential item functioning in both studies revealed that women are more likely to worry about “being poor” than equally depressive men (P<0.05), thus ruling out the alternative hypothesis that depressive response sets are simply a byproduct of more frequent or stronger depression in women. Australian women and men, and US women used the MDS items in a similar fashion, whereas equally depressed US men seriously underreported their symptoms. Yet, using top-down purification to derive an unbiased baseline set of items, a “split” 12-item Rasch measure (R-MDS) could be developed that is not affected by differential test functioning due to gender or cultural differences. Comparison of the R-MDS measure and the original MDS scores revealed that while women are more depressive than men by 0.4 SD, the absence of gender bias in the R-MDS decreased this effect by about 20% (0.08 SD). Moreover, gender bias interacted with culture, suggesting that comparisons of depression levels of diverse groups within the same culture may be biased as well. Raw score to R-MDS conversion tables are included.

Introduction

The available literature agrees that women are almost twice more likely than men to report protracted sadness, apathy, low self-esteem and other symptoms that are indicative of depression. Yet, there continues to be less than perfect agreement on how men and women differ with respect to their expression of depressive symptoms (Nolen-Hoeksema, 1987, Nolen-Hoeksema, 1994, Santor and Ramsay, 1998). For instance, based on factor analytic techniques Williamson (1998) identified social withdrawal, indecisiveness, and irritability as characteristic of depressed men. Wilhelm and Hadzi-Pavlovic (1997) suggest that women are more likely to report anxiety, but this hypothesis disagrees with Breslau, Schultz, and Peterson's (1996) finding that anxiety disorders predict the occurrence of depressive disorders for men and women alike. Given this conflicting picture, it is understandable that “negative symptoms” (i.e. symptoms of depression) are often difficult to identify in psychiatric practice (Greden and Tandon, 1991, Muller and Davids, 1999). This problem is exacerbated by the fact that depressive symptoms may vary with the nature of patients' psychiatric illnesses (Gibbons, Clark, Cananaugh, & Davis, 1985).

A coherent framework to address the expression of depressive symptoms is provided by Nolen-Hoeksema's (1987) “response set” explanation according to which women tend to show passive ruminating responses, whereas men actively use distraction to cut off depression before it ramifies. The response set explanation has received consistent empirical support in descriptive research (e.g. Butler & Nolan-Hoeksema, 1994) as well as in laboratory research (e.g. Lyobomirsky & Nolen-Hoeksema, 1993). The detrimental effects of negative rumination are further demonstrated in an experiment showing that the level of depression in mildly-to-moderately depressed subjects increased when they were induced to ruminate, while depression decreased when distractive behaviors were induced (Nolen-Hoeksema & Morrow, 1994). Despite such findings, the possibility remains that in daily life similar levels of depression naturally produce similar negative ruminative behaviors in men and women. Thus, the greater rumination exhibited by women might be a direct consequence of their more frequent depression, thereby implying that gender differences merely serve to moderate the expression of depressive response sets.

This paper tests the “strong” response set hypothesis according to which women express depression differently than equally depressed men. Although Nolen-Hoeksema's theorizing clearly implies the existence of qualitative gender differences, the response set hypothesis has not been tested in this form. We attribute this to the fact that the stronger version poses technical problems that are difficult to solve within the framework of classical test theory (Thissen, Steinberg, & Gerrard, 1986). In particular, a rigorous test of the strong response set hypothesis requires that depression first should be assessed in a gender-neutral fashion. No bias tests were performed in previous research on depressive symptoms (Butler and Nolen-Hoeksema, 1994, Christensen et al., 1999, Lyobomirsky and Nolen-Hoeksema, 1993, Nolen-Hoeksema, 1994, Nolen-Hoeksema and Morrow, 1994), and it is not clear therefore whether the requirement of gender-neutral measurement was met. The following describes how the strong version of Nolen-Hoeksema's basic hypothesis can be tested by removing gender bias within a Rasch (1960) scaling approach.

Most applications of Rasch scaling, as well as related Item Response Theory methods, in clinical and personality testing (e.g. Birenbaum, 1986, Carter and Wilkinson, 1984, Cooke and Michie, 1997, Gibbons et al., 1985, Lange and Houran, 1999 are primarily intended to obtain unidimensional scales with interval level measurement properties. Unfortunately, as pointed out by Thissen et al. (1986) and Santor and Ramsay (1998), one of the most useful properties of such methods, namely, their ability to detect and quantify a particular kind of item bias called Differential Item Functioning (DIF), is often ignored. DIF violates the assumption of local independence which requires that items' measurement properties should not be affected by external variables such as gender or culture. As such, DIF is evidenced by the fact that equally depressed men and women respond systematically differently to the same item. Although formal methods to determine the presence of differential item functioning are discussed in a later section, we point out that DIF is also often accompanied by psychometric anomalies (Tanzer, 1996) and that it may produce “phantom” (i.e. actually non-existing) factors in factor analysis (Lange, Irwin, & Houran, 2000).

Because Nolen-Hoeksema's (1987) response set hypothesis entails that women are more ruminative than equally depressed men, it follows that measures of depression and rumination should show DIF (cf. Santor & Ramsay, 1998). There is indirect evidence however that gender is not the only source of DIF in depression related measures and that cultural factors may play a role as well. For instance, Thompson, Kaslow, Weiss, and Nolen-Hoeksema (1998) administered a revised version of the Children's Attributional Style Questionnaire (CASQ) to large samples of African American and Caucasian American girls and boys. Suggestive of cultural DIF, the psychometric properties of the CASQ varied by race since the Caucasian data showed higher internal consistency. To address issues related to cultural DIF we compare samples of North American and Australian respondents. Differential item and test functioning in depression due to age have recently been reported as well (Christensen et al., 1999). We judged, however, that our samples contained too few older respondents to address this factor in a satisfactory manner.

The strong response set hypothesis will be tested in two studies. To address the possible effects of cultural DIF, Study I uses Australian respondents, while Study II is based on North American respondents. Both studies combine the data sets of projects in which Thalbourne's Manic-Depressiveness Scale (MDS) was administered for purposes not related to the present research. The MDS addresses history of manic symptoms as well as history of depression (Thalbourne, Delin, & Bassett, 1994). While relatively new, the MDS has profitably been used in clinical reserach (for a review see Thalbourne, Keough, & Crawley, 1999) and two preliminary validation studies have been reported (Lester, 1999, Thalbourne and Bassett, 1998). The MDS consists of 18 True/False type questions which were largely derived from the clinical criteria listed in the DSM-III and DSM-III-R. Table 1 lists twelve of the MDS questions in abbreviated form. The remaining questions are listed in the Method section to Study I.

Note that Item 1 (“worried about being poor”) corresponds most closely to Nolen-Hoeksema's notion of negative rumination, and we predict therefore that women are more likely to endorse this item than equally manic-depressive men. A test of this hypothesis requires that the remaining MDS items contain no gender DIF as such DIF may combine to produce bias at the test level, thereby making it impossible to identify equally manic-depressive respondents. Bias at the test level is customarily referred to as Differential Test Functioning (DTF). When tests consist of many nearly equivalent items (e.g. as in educational testing), DTF can be removed simply by discarding the items showing DIF. However, this approach is not feasible for shorter instruments like the MDS as too few items would remain. For this reason we use Rasch (1960) scaling as this allows DTF to be neutralized by equating differentially functioning items across genders or cultures based on an unbiased, i.e. DIF-free, baseline set of items. Where the context allows, we use the terms DIF and DTF as synonymous with bias.