Introduction

Assertive community treatment (ACT) is an intensive treatment model in which multidisciplinary teams provide community care for non-motivated patients with a severe mental illness (Bond et al. 2001). If correctly implemented, ACT is regarded as an evidence-based intervention (McHugo et al. 1999). Its primary objectives are to reduce hospital admissions, keep patients in contact with services, and improve psychosocial outcome (Marshall and Lockwood 1998).

Several studies, most of them American, have provided evidence for the effectiveness of ACT (Burns and Santos 1995), whose main effects were to reduce admissions and to keep patients in contact with the mental health services. ACT’s effects on symptoms, housing stability, and subjective quality of life were less clear (Marshall and Lockwood 1998). However, in European studies that compared it with standard community care, ACT had no effects on psychiatric hospital use, symptoms, or quality of life (Kent and Burns 2005; Sytema et al. 2007).

Despite the lack of European evidence for its beneficial clinical effects, ACT has been widely implemented in Great-Britain and other European countries (Killaspy et al. 2006), including the Netherlands. For this reason, with regard to ACT’s effect on symptom-reduction, functioning and quality of life, it should be established which patients may benefit from ACT and which do not.

Several studies which defined treatment outcome as level of symptoms, level of functioning, employment and quality of life have identified the predictive factors associated with poor response to various types of treatment (including ACT). Briefly, these studies identified nine such factors: (1) male gender (Grossman et al. 2006); (2) age (Roberts et al. 2000); (3) low educational level (Lauronen et al. 2007); (4) concomitant substance abuse (Dixon 1999; Drake et al. 1993; Batel 2000; Greenfield et al. 2006); (5) early manifestations of symptoms (Remschmidt et al. 1994); (6) negative symptoms (Wieselgren et al. 1996); (7) lack of awareness of symptoms (Rossi et al. 2000); (8) poor treatment compliance (Gerlach 2002); and (9) duration of untreated psychosis (Singh 2007).

However, to our knowledge, no studies have specifically investigated the influence of these predictive factors on psychosocial outcome in the context of ACT. Using an observational study design in patients receiving ACT over a two to 3-year period, we therefore examined the influence of three of these predictors on treatment outcome, which was defined as level of symptoms and social functioning over time (Health of the Nation Outcome Scales (HoNOS) total score). We used the following predictors: substance abuse, motivation for treatment, and demographic factors (age, ethnicity, level of education, and gender).

Methods

Setting

The study involved patients from six ACT teams in the city of Rotterdam, the Netherlands. There were three criteria for treatment by an ACT team: (1) age 18 or older, (2) diagnosis with a severe mental illness (usually a psychotic or bipolar disorder, with or without a comorbid addiction disorder); and (3) lack of motivation for treatment at the start of ACT, which made assertive outreach necessary.

Data were collected as part of a routine outcome-monitoring (ROM) procedure, and were used in clinical practice to discuss treatment outcome with the patient and the clinician. The collection of routine outcome monitoring data, which was done by independent raters with a Master’s degree in psychology, was approved by the Dutch Committee for the Protection of Personal Data. All data were analyzed anonymously.

Model Fidelity

To assess the fidelity of the six treatment programs to ACT, we used the Dartmouth Assertive Community Treatment Scale (DACTS), which assesses fidelity on the basis of 28 items using anchored five-point scales (Teague et al. 1998; Salyers et al. 2003; Bond and Salyers 2004). Psychometric properties such as internal consistency, inter-rater reliability and sensitivity to change over time have been found to be acceptable (Winter and Calsyn 2000; Bond and Salyers 2004). A mean score of all items between 0 and 2.9 means that a treatment team has failed to implement ACT; a score between 3.0 and 4.1 means that ACT has been implemented to a moderate degree, and a score between 4.2 and 5 means that it has been fully implemented (Teague et al. 1998; Salyers et al. 2003).

Outcome Measures

Data were collected over the period from January 2003 to August 2008. At the start of the treatment and then at 6-month intervals, patients were assessed using the Health of Nation Outcome Scales to determine psychosocial outcome (HoNOS; Wing et al. 1998; Mulder et al. 2004). To this we added one additional observer-rated item to assess motivation for treatment.

Health of the Nation Outcome Scales was originally developed as a standardized assessment tool for routine use by the mental-health services. It consists of 12 observer-rated scales, each using five points from 0 (no problem) to 4 (severe/very severe), and thus yielding a total score from 0 to 48. The psychometric properties of the English and Dutch HoNOS total scores have been found to be acceptable (Wing et al. 1998; Mulder et al. 2004). HoNOS covers the following domains: (1) overactive, aggressive, disruptive or agitated behaviour, (2) non-accidental self-harm, (3) problem drinking and drug-taking, (4) cognitive problems, (5) physical illness and disability, (6) hallucinations and delusions, (7) depressed mood, (8) other psychological symptoms, (9) relationship problems, (10) problems with activities of daily living, (11) problems with living conditions, and (12) problems with occupation and activities.

Predictors Variables

The scale for assessing motivation for treatment was adapted from the Severity of Psychiatric Illness scale (Lyons 1998; Mulder et al. 2005), and was scored in five categories in the same way as the HoNOS scale: (0) strong motivation: significant degree of motivation for treatment; (1) clear motivation: there may be some hesitation, but this does not lead to problems with motivation; (2) some motivation: there is motivation for treatment but also ambivalence or mild passive resistance; (3) poor motivation: the individual appears not to be motivated and there is passive resistance; and (4) no motivation/resistance: the individual actively resists treatment. On the basis of an interview with the patient and the clinician, the motivation for treatment scale was scored by independent raters who were not involved in the patients’ treatment.

To assess substance abuse, patients were routinely assessed on the basis of two items—alcohol use and drug use—taken from the Camberwell Assessment of Need (CAN). The ratings were based on the interviewee’s perspective (as opposed to the patient’s). The CAN severity ratings are 0 (no need), 1 (met need) and 2 (unmet need) (Wennström 2008).

We collected socio-demographic and diagnostic data on gender, age, ethnicity (according to the definition of the Statistics Netherlands: i.e., parents’ countries of birth), level of education, and DSM-IV-TR diagnoses as made by the psychiatrists of the ACT team.

Data Analysis

SPSS version 15.0 was used for all analyses. Treatment outcome was defined as the HoNOS total score. Linear Mixed Models with repeated measures were used to assess the association of the predictors and psychosocial functioning over time.

Factors of primary interest included time (treatment duration) and psychosocial functioning (HoNOS total score). To capture a curvilinear decline which would assume a more rapid change in the early months, the model also included a square-root transformation of time.

Predictors All covariates were selected on the basis of a theoretically or empirically documented association with treatment outcome. Demographic information included education, age and ethnicity; other covariates were problems with motivation and substance abuse at baseline.

Model For the initial specification of the model, we included linear time, square-root time, HoNOS total score, demographics (age, level of education and ethnicity), motivation at baseline, and substance abuse at baseline. Fixed factors To obtain the most parsimonious model, fixed effects were dropped in subsequent iterations and eliminated, since each effect was either not significantly related to the HoNOS total score, or did not appreciably alter outcome (likelihood ratio test; Fitzmaurice et al. 2004). Patient’s identification number was used as a random factor (random intercept deviation). Random effects were modeled if they significantly contributed to the model (likelihood ratio test).

Final model The fixed effects in the final model were intercept, time and a square-root transformation of time (which fitted the data better than a linear time slope alone); motivation at baseline; substance abuse at baseline; age; and level of education. Repeated measures were modeled on the assumption of a first-order autoregressive covariance structure (based on REML) (Fitzmaurice et al. 2004).

Lastly, in an effort to replicate and supplement earlier findings (Grossman et al. 2006), we performed a variation of the primary analyses for men and women separately. Because the sample included only a small number of women, we included all covariates from the model (as defined above), not just covariates that were statistically significant. Non-significant results for the smaller group were examined to ascertain whether their size (β) and direction were similar to those of the larger group. Although non-significant results might indicate that the sample size was not great enough to allow comparison between the groups, any differences in their magnitude or direction indicates that the results are not explained solely by sample size.

To test whether the differences in associations were significantly different for men and women, gender was added to the full model as an interaction term with all covariates (substance abuse, age, level of education and motivation, linear time, and square-root transformation of time). In this model, a significant interaction would indicate dissimilarities in the associations for men and women between the covariates and the HoNOS total score over time.

Results

Patients

The data included 637 assessments from a total of 139 patients. On average, assessments were 6.9 months apart (SD = 1.4). The mean treatment duration of follow-up was 27.4 months (SD = 5.4). The mean age was 38.3 years (SD = 9.5). Diagnosis was schizophrenia or other psychosis for 72.3% of the patients. The patients’ characteristics are described in Table 1.

Table 1 Patient characteristics
Full size table

Model Fidelity

The mean of the total DACTS scores of the six ACT teams was 3.5 (range: 3.4–3.6), meaning that ACT had been implemented with moderate success. The lowest scores were awarded to various categories pertaining to substance abuse: substance-abuse specialist on staff (m = 2.8), individualized substance-abuse treatment (m = 2.8), dual-disorder treatment groups (m = 1.2) and dual disorders (dd) model (m = 3).

Determinants of Treatment Outcome for the Whole Group

In the final model, the following predictors were significantly associated with the HoNOS total score: substance abuse at baseline (CAN score 2: serious problem on items 12 (alcohol) or 13 (drugs) versus 0: no problem or 1: intervention); motivation for treatment at baseline; education level (no education or elementary school versus lower high school and over); and age (<30 years versus ≥30 years).

Analysis of changes during follow-up in the HoNOS total score revealed a significant improvement over time (Table 2: linear time: F = 7.841, P = .005, square-root time: F = 14.534, P < .000). Of all predictors, substance abuse at baseline was most strongly associated with the HoNOS total score: the main effect was (β = 3.47, F = 24.414, P < .001). Because the HoNOS incorporates problematic alcohol use and drug taking it is evident that the HoNOS total score will positively correlate with substance abuse. Therefore we did 2 analyses, (1) on the relation between the HoNOS total score (including problematic alcohol use and drug taking (above analysis) and substance abuse and (2) on the relation between the HoNOS total score (excluding problematic alcohol use and drug taking) and substance abuse. The second analysis shows that, substance abuse still remains a predictive factor (β = 1.60, F = 5.874, P = .017).

Table 2 Prediction of HoNOS total scores among men and women
Full size table

The results also showed that age was independently associated with the HoNOS total score (β = 2.26, F = 7.341, P = .007), meaning that older patients had higher overall HoNOS total scores. Problems with motivation at baseline were also associated with higher overall HoNOS total scores (β = .733, F = 6.460, P = .012). Lastly, analyses revealed that the level of education was significantly associated with HoNOS total score (β = −1,916, F = 5.028, P = .027), as patients without education or elementary school had higher overall HoNOS total scores than patients whose education level was lower high school or above.

Gender and Outcome

Table 2 also shows the men and women’s respective levels of psychosocial functioning. The results showed differences in the significance and direction of the associations. In men, poorer treatment outcome was predicted by substance abuse, age (30 or older), and level of education (no education, or elementary school only). In women, the pattern of associations was different, in that only substance abuse and problems with motivation for treatment were strongly and significantly associated. In addition, there was also a non-significant association with age, in the opposite direction than that of male patients. Because the association with education was also non-significant in women, its significance was different than it was with men.

The two covariates—time and square-root transformations of time—were also different for men and women, men showing a significant decline over time, and women showing a non-significant decline.

To test whether the associations of the covariates with treatment outcome differed between men and women, gender was added as an interaction term. Our results showed significant differences between men and women for substance abuse (F = 23.145, df = 110.874, P < .001), age (F = 10.299, df = 225,753, P = .002), motivation for treatment (F = 10.682, df = 121,094, P = .001) and time (sqrt) (F = 5.997, df = 367.394, P = .015). Level of education (F = 1.864, df = 143.689, P = .174) and time (linear) (F = 1.508, df = 387.370, P = .220) revealed no significant interaction with gender.

Discussion

The fidelity score of the DACTS model showed that the six teams had implemented ACT moderately successfully, but that treatment for dual disorder had been implemented relatively unsuccessfully.

Although the patients’ psychosocial functioning improved significantly over time, the gains seem to have been concentrated mainly in the first months of treatment; later on, the level of functioning appeared to stabilize. Despite this early improvement, two factors indicate a need for long-term ACT: patients’ level of functioning over time, and the risk that their lack of motivation for treatment will cause their situation to worsen. It should also be stated that the significant improvement in psychosocial functioning was restricted to men, although the non-significant results for women may have been a product of the sample size.

Our analysis also showed that the level of psychosocial functioning was significantly hampered by substance abuse, age over 30, low level of education (either no education, or elementary school only), and problems with motivation for treatment. In that these patient characteristics were associated with significantly more problematic functioning over time, our results confirmed earlier findings on treatment outcome in other patients with a severe mental illness (Dixon 1999; Drake et al. 1993; Batel 2000; Greenfield et al. 2006; Lauronen et al. 2007; Gerlach 2002; Roberts et al. 2000). Our finding that older patients had higher HoNOS total scores may have been due to the fact that the duration of mental illness (Jenner 2003) or of untreated psychosis was longer in these patients, each a factor that has been associated with worse prognosis (Singh 2007).

Our study further demonstrated that the pattern with which these variables were associated with psychosocial outcome was different between men and women. The differences between the sexes’ levels of psychosocial functioning—women tending to have fewer psychosocial problems over time, but also improving less—may have been due to a floor effect.

However, the fact that substance abuse had more adverse consequences for women than for men may have been because women seemed more prone to perilous activities, such as turning to prostitution as a means to earn the money they needed to support their substance use. This led to problems regarding physical health and daily living conditions, and is in line with previous research by RachBeisel et al. (1999), who suggested that substance abuse among women is associated with increased risks for physical health problems and sexually transmitted diseases.

The third difference between men and women, problems with motivation for treatment, also resulted in a higher risk (i.e., stronger association) for psychosocial problems in women than in men, which may be related to more disruptive behavior, and which therefore leads women to have more problems with motivation for treatment.

For men, risk factors were being aged 30 or older, low level of education, and substance abuse. This is in agreement with findings that older patients had poorer global functioning (Roberts et al. 2000), and may indicate that these patients are more at risk of neglecting their personal care than women are. These findings also supplement those of Gur et al. (1996) by showing more specifically how the clinical features of patients with a severe mental illness are moderated by aging and gender.

The association with low level of education may indicate that such patients have more difficulty managing or coping with problems in their lives. Neisser et al. (1996) showed that because educational level was moderately highly correlated with intelligence, it may also reflect a patient’s ability to make use of any services on offer, and to foresee the consequences of their behavior. Because a low level of education may also lead to greater isolation from the labour market (Wolbers 2000), it may also complicate rehabilitation.

Bhugra et al. (1997) showed that non-western ethnicity was associated with poorer treatment outcome, a finding we were unable to replicate, due possibly to differences in outcome assessment: whereas Bhugra et al. used employment status, we defined outcome more broadly in terms of psychosocial functioning as measured by the HoNOS. Our study therefore suggests that, in terms of psychosocial functioning over time, non-western immigrants do not differ from other patients.

Limitations of the Study

We should acknowledge two limitations of the present study. The first concerns the design. Because this was a naturalistic follow-up study that used routine outcome-monitoring data, we had no information on other factors that may have co-determined the outcomes, such as negative symptoms, lack of awareness of symptoms, and duration of untreated psychosis. Neither does the design make it possible to draw any causal inferences, although Shrier et al. (2007) suggests that, like randomized controlled trials, an observational study design can also contribute to evidence-based research.

The second limitation concerns the small number of women in the analyses, which was a product of the substantial overrepresentation of male patients in the ACT teams. We therefore checked non-significant results for the female patients to see if they were similar in magnitude (β) and direction to those in the larger group. Although non-significant results may indicate that the sample size was not enough for purposes of comparing the groups, the differences in magnitude or direction we found here indicate that the results were not explained solely by sample size.

Conclusion

Since model fidelity has been shown to be associated with better outcome (McHugo et al. 1999; McGrew et al. 1994; Latimer 1999; Bond et al. 2000), our results suggest that our ACT teams should improve their fidelity with the ACT model. We also conclude that special attention should be paid to patients who seem to benefit less from ACT.

Our results emphasize the importance of implementing the ACT-model fully, including substance abuse treatment programs. This can be done by implementing IDDT (Drake et al. 2001), or other substance abuse programs. McHugo et al. (1999) showed that faithfully implemented dual-disorder programs achieved better treatment outcomes. These recommendations agreed with our DACTS findings, which also support a better implementation of ACT, especially with regard to the dual-disorder elements that achieved low DACTS scores in this study.

Because our results also indicate that treatment outcome was significantly hampered by low education (including mental retardation), we propose the implementation of programs based on behavioral therapy for mentally retarded patients. One example of such a program is token economy (Comaty et al. 2001), which has also shown to increase adaptive behavior in schizophrenic patients (Dickerson et al. 2005).

To address problems of motivation for treatment, we recommend the structural implementation of Motivational Interviewing (Martino et al. 2000; Gerlach 2002). The central purpose of motivational interviewing is to examine and resolve ambivalence in treatment goals. Research by Bien et al. (1993) and Brown and Miller (1993) has shown that patients who were given motivational interviewing had participated more fully in treatment, and appeared to be more motivated than those who had not received this intervention. If motivational interviewing is implemented, motivationally challenged patients may benefit more from assertive community treatment.

To meet the special needs of patients in different age categories, we also argue for the development of innovative programs such as the differentiation of ACT teams according to patients’ age (i.e., young, adult and elderly). Because clinicians working in ACT teams serving a subpopulation such as the elderly, may have special skills for dealing with specific needs, such as somatic and cognitive problems. Therefore these teams may be better equipped to deal with specific problems related to age. Finally, because several important factors such as problems with recovery and substance abuse have shown to be different for men and women (Mangrum et al. 2006; Grossman et al. 2006), it may be necessary to adopt a gender-specific approach. This will mean that separate treatment programs are adapted to the specific needs of men and women.