Multiple Informants: A New Method to Assess Breast Cancer Patients’ Comorbidity

Abstract

Past assessments of comorbidity indices have sought to recommend a single index that performs better than others. The authors used a multiple informants approach as an alternative method to simultaneously assess five indices of comorbidity. This approach provides a single estimate of the overall effect of comorbidity and evaluates the relation any individual index has to the outcomes of interest. Association of comorbidity with definitive primary therapy, discussion of tamoxifen, and receipt of tamoxifen was evaluated in a cohort of 830 older breast cancer patients enrolled at four geographically distinct centers in the United States from 1996 to 1999. The estimated adjusted effect of a unit increase in comorbidity on the odds of discussing tamoxifen therapy was 0.70 (95% confidence interval: 0.56, 0.88). An increase in comorbidity was not associated with receipt of definitive primary therapy (odds ratio = 0.94, 95% confidence interval: 0.79, 1.13) or receipt of tamoxifen (odds ratio = 0.96, 95% confidence interval: 0.72, 1.27). The multiple informants regression proved superior to separate regression models that included only one index. In analyses that require comorbidity adjustment and for which no single index is expected to be ideal, the multiple informants approach is an attractive alternative to selecting a single index and to other methods of using multiple indices.

Received for publication April 1, 2002; accepted for publication August 28, 2002.

Studies that compare the effectiveness, cost, or utilization of therapies must account for confounding by differences in underlying health. Methods to control for this confounding (1) have been the topic of a broad literature. Such methods have been developed in studies of the effectiveness of breast cancer therapy (2–9) or of disparities in the distribution of breast cancer therapies in underserved populations (5, 7–14). Many of the indices of comorbid disease were developed (3, 5, 7, 15) or validated (16) in breast cancer cohorts.

Several studies have examined multiple measures of comorbid disease, with an emphasis on choosing the best method for control of confounding. Comparisons have been made between the same measures derived from different information sources (10, 15, 17–19) and between different measures of comorbidity (15, 19, 20). In general, measures are marginally correlated with one another (10, 15, 18, 19) and add little control when a second index supplements the first (15, 18, 20).

Each comorbidity index measures a common concept, the health of the participant, but each also contributes different information, reflecting the purpose for which it was developed. The common objective explains why indices are correlated, but the different purposes and data sources explain why correlations are seldom very strong. Currently, no single index or information source can be uniformly recommended for studies of breast cancer patients (10). New methods to reliably account for comorbidity in older cancer patients have been urgently solicited (1).

In this context, we introduce a new analytic approach. This multiple informants approach uses information from parallel comorbidity indices by simultaneously fitting separate logistic regression models. It then merges the results into a unified regression equation to yield a single measure of the effect of comorbid disease on outcome. Additionally, this method examines individual indices and their relative associations with the outcome. Use of multiple informants data as a predictor has been described previously (21, 22).

The multiple informants approach is ideally suited for analyses in which multiple measures of the same concept are available but none of the measures definitively assesses the underlying concept. An additional advantage over traditional approaches is that all available data are used and all subjects contribute to the analysis. In our analysis, the comorbidity indices measure health, but their differences create the possibility that they will yield different assessments of comorbid disease status for the same person. Furthermore, not all indices were assessed for all participants. In this paper, we present results from multiple informants analysis of the association between comorbidity, as measured by five indices, and three outcomes.

MATERIALS AND METHODS

Study sample

We conducted a prospective cohort study of elderly women diagnosed with early-stage breast cancer. Our enrollment and data collection procedures have been described elsewhere (23). Briefly, we identified women with early-stage breast cancer (stage I and a tumor diameter of 1 cm or greater, stage II, or stage IIIa) at hospitals in Rhode Island and southeastern Massachusetts, North Carolina, Minnesota, and Los Angeles, California, between December 1, 1996, and September 30, 1999. With their physician’s permission, we invited patients aged 65 years or older, with no earlier breast cancer and no concurrent second primary tumor, to complete three interviews and to allow review of their medical record. We excluded women who were 1) non-English-speaking, 2) not competent to undergo interview, 3) without satisfactory hearing, or 4) not enrolled within 5 months of the date of their breast cancer surgery.

Women who agreed to participate returned a signed consent form approved by local institutional review boards. Participants completed telephone interviews 3, 6, and 15 months after their surgery. We collected demographic data and information on primary and systemic adjuvant therapy, treatment decision making, and comorbid disease during the patient interviews. At least 3 months after participants’ surgery, medical record reviewers collected data from medical records on tumor characteristics, comorbidity, and treatments received. We asked patients’ surgeons and medical oncologists to complete patient-specific forms that asked for an assessment of patient health at the time of presentation and to rate the importance of various factors that influenced their decision making regarding the prescription of tamoxifen.

Dependent variables

We examined the association between patient’s comorbidity and three outcomes: primary tumor therapy, discussion of adjuvant tamoxifen, and prescription of adjuvant tamoxifen.

Primary tumor therapy

We classified patients according to receipt of definitive primary therapy or less-than-definitive therapy. We considered definitive therapy to be axillary node dissection and breast-conserving surgery plus radiation therapy or mastectomy (24, 25). Less-than-definitive therapy applied to all other combinations of surgery and radiation therapy.

Physician discussion of adjuvant tamoxifen

Patients were asked whether they had discussed tamoxifen therapy with their physicians. If they had discussions by the 6-month interview, we classified discussion of tamoxifen therapy as yes; otherwise, we classified it as no.

Prescription of adjuvant tamoxifen

If patients were prescribed tamoxifen by the 6-month interview, we classified tamoxifen prescription as yes; otherwise, we classified it as no. In a subset of 45 patients who received a prescription benefit through their health maintenance organization, self-report of tamoxifen prescription had a sensitivity of 94 percent and a specificity of 91 percent when compared with the pharmacy database.

Independent variables

We collected information on patients’ comorbid diseases by using five indices from three data sources. Multiple reports of comorbidity were solicited because comorbidity is inherently difficult to assess. Covariates included a patient’s demographic characteristics, disease characteristics, receipt of adjuvant chemotherapy, and factors influencing physicians’ decisions regarding tamoxifen treatment.

Comorbidity indices

Table 1 summarizes construction of the five comorbidity indices. The final measure of comorbidity was an assessment of a patient’s health, aside from her breast cancer diagnosis, by the patient’s surgeon and/or medical oncologist indicated on the patient-specific treatment recommendation forms. Ratings from patients’ surgeons were analyzed as the fourth comorbidity index, and ratings from patients’ medical oncologists were analyzed as the fifth comorbidity index. We calculated the Charlson index of comorbidity (16) on the basis of information collected from the 3-month interview. The original Charlson measure predicted 1-year mortality in 559 medical patients (16) and 10-year mortality rates for deaths attributable to comorbid diseases among breast cancer patients (16). We followed the adaptation to an interview format rather than a medical record review format (26). Both forms have test-retest reliability of approximately 0.9, correlate well with one another, and correlate with indicators of resource utilization (26).

We based the Index of Coexistent Diseases (ICED) (3) on data from the medical record. The ICED was developed to consider conditions that may impact breast cancer management and outcome during a 2-year period following hospitalization (3). This index incorporates an individual disease value reflecting the stage of recorded health conditions, which is combined with complications of the diseases to form an overall disease value. A second component measures functional status for 11 system categories, such as circulation, respiration, and vision. The disease value and function value are combined to yield a scale ranging from 0 to 3. We obtained the details of the components and scoring guidelines from the original authors (3). Interrater reliability examined in a subpopulation of 30 breast cancer patients found that all four raters derived the same ICED scores for 20 patients, and three of four raters derived the same ICED scores for an additional six patients (3). The ICED score has been directly associated with postoperative complications and function 1 year after hip replacement surgery (27) and with mortality rates among prostate cancer patients (28) and kidney dialysis patients (29).

The remaining indices did not assess individual comorbid disease but rather represented global ratings of patients’ health. The first is the American Society of Anesthesiologists (ASA) physical status score (30, 31), which was assigned by the patient’s anesthesiologist at the time of surgery and was abstracted from the medical record. The ASA physical status score is directly associated with the risk of complications in the postoperative period (32) and with the risk of mortality within 7 days of surgery (33). The final measure of comorbidity was an assessment of a patient’s health, aside from her breast cancer diagnosis, by the patient’s surgeon and/or medical oncologist and was indicated on the patient-specific treatment recommendation forms. The index was adapted from a similar item described by Charlson et al. (16, 34). In-hospital mortality rates approximately tripled with a unit increase along the index (34). For those who survived, the physician’s assessment was an important predictor of 1-year mortality (16).

Patient characteristics

We categorized a patient’s age as 65–69 years, 70–79 years, or ≥80 years. We used the physical function scale (PFI10) from the Medical Outcomes Study 36-item short form (MOS–SF36) (35) to measure function 3 months after initial surgery and the general health status scale (35) to measure self-perceived health before breast cancer diagnosis. As a crude measure of health-services utilization, we asked participants at the 6-month interview to indicate the number of medications currently prescribed by a physician.

Disease characteristics

Categorical stage groups were I, IIa, IIb, or IIIa (36), which were based on measures of tumor size and axillary node status. Estrogen-receptor status was categorized as positive, negative, or indeterminate.

Systemic adjuvant chemotherapy

If, during the 3- or 6-month interview, patients indicated receiving chemotherapy, they were categorized as yes. Otherwise, they were classified as no.

Factors influencing physicians’ decisions regarding tamoxifen treatment

The patient-specific treatment recommendation forms asked physicians to rate the importance of 11 factors regarding their recommendation for or against adjuvant tamoxifen therapy to a specific patient (for example, “Treatment with tamoxifen will reduce her risk of local recurrence of her breast cancer.”). We described the development and psychometric properties of this measure previously (23).

Analytic strategy

Descriptive statistics

In this paper, we present descriptive characteristics of the study population as frequencies or means within categories of the variables. To characterize the five comorbidity indices, we calculated 1) the proportion of the study population at each level of each comorbidity index, 2) the missingness pattern for the combinations of comorbidity indices per patient, and 3) the Spearman correlations with one another and with other measures of health status.

Modeling the influence of individual comorbid disease measures on outcomes

We used separate logistic regression models to compute the crude and adjusted estimates of association of each comorbidity index on each outcome, controlling for the clustering of patients treated by individual physicians. Each comorbidity scale, coded ordinally, served as the independent variable of primary interest. On the basis of a model-building strategy used in an earlier report (23), we adjusted for age, enrollment site, cancer stage, physical function, estrogen receptor protein status, chemotherapy, physician’s decisional balance score, and receipt of breast-conserving surgery with or without radiation. We excluded chemotherapy, primary therapy, and the decisional balance variables when primary therapy was the dependent variable; the first two define the outcome, and the last applies to only tamoxifen prescription. We adjusted for enrollment site as a categorical variable.

Modeling using the multiple informants approach

The primary objective was to substitute multiple informants modeling for the traditional modeling described above, adjusting for the same covariates. We used new methodology that allowed for 1) inclusion of all indices in a single multivariate regression to obtain a single estimate of the association of comorbidity with each dependent variable; 2) testing for index-specific associations; 3) testing of whether the associations of other independent variables differ by index, and estimation of those differences; and 4) inclusion of partial data from subjects for whom observations were missing for a subset of the indices.

We used generalized estimating equations (GEE) to incorporate available comorbidity information from all five indices simultaneously while controlling for other covariates. Use of this procedure for a predictor variable has been described previously (21, 22). We extended the method to analysis of comorbidity, as presented in the following five logistic models:

Logit E(Y|X₁, Z) = b₀ + b₁X₁ + b₂Z (1)

Logit E(Y|X₂, Z) = (b₀ + a₀₂) + (b₁ + a₁₂)X₂ + b₂ Z

Logit E(Y|X₃, Z) = (b₀ + a₀₃) + (b₁ + a₁₃)X₃ + b₂Z

Logit E(Y|X₄, Z) = (b₀ + a₀₄) + (b₁ + a₁₄)X₄ + b₂Z

Logit E(Y|X₅, Z) = (b₀ + a₀₅) + (b₁ + a₁₅)X₅ + b₂Z

where Y = the dependent variable, X₁ = the ICED score, X₂ = the Charlson score, X₃ = ASA performance status, X₄ = the surgeon’s rating, X₅ = the oncologist’s rating, and Z = vector of covariates for patient characteristics, disease characteristics, and decisional balance.

In these models, b₀ represents the baseline log-odds. The a_0(i) measures the difference in baseline log-odds due to the ith comorbidity index compared with the reference (ICED), thereby assessing the relative effect of nonmissingness of the individual indices on the dependent variable. For example, a₀₁ measures the difference in log-odds associated with nonmissingness of the Charlson index compared with nonmissingness of the ICED index.

The b₁ represents the log-odds associated with comorbidity (that is, the overall effect of comorbidity on the dependent variable). This log-odds is allowed to vary for each index by the a_1(i) parameters, representing the difference in association for the different comorbidity indices. For example, the a₁₍₁₎ parameter measures the differential effect of comorbidity on the outcome for the Charlson index relative to the ICED. This interaction between individual indices and the summary estimate of association was tested (H₀: a_1(i) equal 0 vs. H₁: a_1(i) not equal to 0 for all i) to evaluate whether the association of comorbidity with the outcome was modified by type of comorbidity index. There was little evidence for such differences (p = 0.57 for primary therapy, p = 0.22 for tamoxifen discussion, and p = 0.62 for tamoxifen prescription), so the models were refitted without the a_1(i) parameters. Furthermore, the b₂ parameters for the covariates were assumed to be the same for all indices, although separate effects could be modeled.

The parameters were estimated by using the SAS PROC GENMOD procedure (37) for each dependent variable, assuming an exchangeable working correlation and empirical variance estimates for the regression parameters to account for the correlation of the logistic regressions given by equation 1. The generalized estimating equation approach treats the correlation between indices, and of patients treated by individual physicians, as a nuisance, and it estimates these correlations to account for the multiple reports and clustering.

RESULTS

We included 830 women who consented to full study participation and for whom data were available on at least one of the five comorbidity indices. Eighty-seven percent (n = 725) of the women completed the 3-month interview, and 77 percent (n = 643) completed the 6-month interview. We collected a medical record review for 96 percent (n = 793) of the women, a surgeon’s treatment recommendation form for 70 percent (n = 581), and an oncologist’s treatment recommendation form for 35 percent (n = 290). Table 2 displays the descriptive characteristics of the study population. Although the majority of the women were aged 70–79 years, younger and older women were well represented. About half of the women had stage I disease, and nearly one third had stage IIA disease. About 70 percent of the women were node negative, and almost three quarters of the tumors were estrogen-receptor positive. By three to one, women who received definitive primary therapy outnumbered those who received less-than-definitive therapy.

Almost 80 percent of the women reported that they discussed tamoxifen therapy with a physician, and three quarters received a prescription. Of the women who did not receive a tamoxifen prescription, 43 percent discussed tamoxifen with a physician. Of the women who received a tamoxifen prescription, 14 percent did not discuss the risks and benefits of tamoxifen prescription. The positive average physician decisional balance score (24.9; standard deviation, 23.9; range, –43 to 100) suggests that, on average, physicians viewed a tamoxifen prescription favorably for these patients.

All five of the comorbidity indices were statistically significantly correlated (all p < 0.005) with physical function, self-perceived health status before breast cancer diagnosis, and total number of prescriptions for medications (table 3), suggesting that the indices measure a similar underlying concept. However, the five indices did show substantial differences in their distributions, particularly for extreme values (table 4). The proportion of the population with a zero score, suggesting no impairment, ranged from 4 percent for the ASA to 74 percent for the surgeon’s index. The proportion of the population with the highest score ranged from 0 percent for the ASA to 18 percent for the ICED.

Each comorbidity index was available within a different subfraction of the cohort. However, this observation is unlikely to explain all of the differences in the distributions. The complete pattern of missingness for the five indices is available from the authors. Of the 830 participants, 692 (83 percent) had a value for the Charlson index and at least one of the medical-record-based indices (ICED or ASA). Missingness was more important for the physician indices, for which 185 (22 percent of the total) participants had neither a surgeon’s nor a medical oncologist’s assessment. The physician indices were both assessed on the patient-specific treatment recommendation forms, for which response rates were the lowest of the data collection instruments. Nonetheless, these indices made important contributions to the assessment of comorbidity, because 97 (12 percent) of the participants had at least one of the physician indices but were missing either the Charlson or both of the medical-record-based indices.

The crude associations shown in table 5 suggest that the odds of receiving definitive primary therapy decreased as comorbidity increased. However, after adjustment for the covariates, most of the associations migrated toward the null, and the width of the 95 percent confidence intervals increased substantially. From the multiple informants analysis for which information on all five indices was used, the overall effect of a unit increase in comorbidity score, adjusted for other covariates, was 0.94 (95 percent confidence interval: 0.79, 1.13).

All of the crude and adjusted associations suggested that the odds of discussing tamoxifen therapy decreased as comorbidity increased (table 5). The adjusted overall effect of a unit increase in comorbidity on the odds of discussing tamoxifen therapy equaled 0.70 (95 percent confidence interval: 0.56, 0.88). This multiple informants finding is consistent with the difference in mean number of comorbid diseases among women who discussed tamoxifen and those who did not reported previously (23).

None of the crude and adjusted associations strongly suggested that the odds of receiving a tamoxifen prescription changed as comorbidity increased (table 5). The multiple informants analysis (odds ratio = 0.96, 95 percent confidence interval: 0.72, 1.27) provided an efficient test of the association between comorbidity and the odds of receiving a tamoxifen prescription. The estimate from the pooled informant model was nearer to the null than all but one of the estimates of association from the individual indices (odds ratio = 1.09 for ICED; table 5), and its interval was narrower than any of the intervals about the estimates derived for the individual indices (table 5). The multiple informants finding is consistent with the null difference in mean number of comorbid diseases among women who received a tamoxifen prescription and those who did not reported previously (23).

DISCUSSION

The effect of comorbidity on the type of therapy received by breast cancer patients and on the outcomes of breast cancer has been extensively investigated (10). The five comorbidity indices available in our analyses correlated moderately well with one another and with three independent measures of health. These observations show that all five indices measured the same underlying concept but did so with error, perhaps because of differences in their original purpose or their data source. It would be difficult to choose one scale as the best for the analyses at hand, and we would run the potential risk of reporting results specific to that source had we selected a single index to the exclusion of others. For the three dependent variables we considered, the multiple informants approach proved superior to multivariable modeling that included only one index.

The multiple informants approach simultaneously provided a single estimate of the association of comorbidity with the outcomes and an assessment of whether the associations depended on the individual comorbidity indices used in the analysis. We observed that comorbid diseases had little association with receipt of definitive primary therapy or prescription for tamoxifen. The proportion of women who discussed tamoxifen therapy decreased as comorbidity increased, however, suggesting that physicians were less likely to discuss tamoxifen therapy with older women who had other diseases. In addition, we found no statistically significant differences between the comorbidity indices for the three outcomes we studied, supporting the finding that they were all similarly associated with the outcomes. When such interactions exist, the multiple informants approach suggests that one index might be measuring a different concept, and the investigator should generate hypotheses to explain the difference. In contrast, had only a single index of comorbidity been examined in such a context, one may be led to spurious conclusions about the association, particularly if the index was selected on the basis of the strength of its association.

The multiple informants regression allowed participants to be included in the analysis as long as at least one index of comorbidity was available, thereby making best use of the data in which, for many subjects, information on at least one of the indices was missing. These participants would ordinarily have been excluded from analyses involving complete data (38), for example, in the case of fitting a single logistic regression model with all five comorbidity covariates. All of these models assume that missingness was not associated with the outcome or other predictors. Extensions to allow other types of ignorable missingness are straightforward (22).

A second advantage of multiple informants analysis over fitting all five covariates in a single model is that the single model yields parameter estimates for a particular comorbidity measure, conditional on holding the other comorbidity measures constant. Interpretation of these parameters may not be straightforward, because one expects comorbidity measures to be correlated. In contrast, the parameter estimates from the multiple informants model are interpreted as the marginal associations for the respective indices. Finally, multicollinearity is not a concern because the indices are fitted in separate equations and the parameter estimates unified into one equation.

Although multiple informants analysis has several advantages, the limitations must also be considered. First, collecting comorbidity information from more than one source requires resources that might be spent elsewhere. In our study, three of the indices derived from single items. Two were survey items asked of physicians (34), and the third was the ASA physical status score (30, 31) abstracted from the medical records. It is not unusual for studies to collect more than one comorbidity index, and it is not burdensome to collect the single-question indices, so the multiple informants approach can be viewed as the best use of the information often available. Second, multiple informants analysis requires correlated data analyses. Although the analyses are more complicated, interpretation of the summary effect of comorbidity, and its interaction with the individual indices, is accessible to investigators, and the methods are implemented in standard statistical software (39).

Last, and perhaps most important, multiple informants analysis should not be a substitute for development of comorbidity indices directly relevant to the research questions at hand. When a single comorbidity index applies directly to a research question, or such an index can be developed, then that single index should be given preference. However, as long as multiple indices are available and new indices continue to be developed, we expect that some confusion will remain about the best choice for control of confounding by comorbidity or for direct assessment of its effects. In this situation, the multiple informants approach is an attractive alternative to selecting a single index.

ACKNOWLEDGMENTS

Supported by grants R01 CA/AG70818 from the National Cancer Institute and National Institute on Aging, National Institutes of Health; K07 CA87724 from the National Cancer Institute; and R01 MH54693 from the National Institute of Mental Health.

References