Original Article
Absolute risk reductions and numbers needed to treat can be obtained from adjusted survival models for time-to-event outcomes

https://doi.org/10.1016/j.jclinepi.2009.03.012Get rights and content

Abstract

Objective

Cox proportional hazards regression models are frequently used to determine the association between exposure and time-to-event outcomes in both randomized controlled trials and in observational cohort studies. The resultant hazard ratio is a relative measure of effect that provides limited clinical information.

Study Design and Setting

A method is described for deriving absolute reductions in the risk of an event occurring within a given duration of follow-up time from a Cox regression model. The associated number needed to treat can be derived from this quantity. The method involves determining the probability of the outcome occurring within the specified duration of follow-up if each subject in the cohort was treated and if each subject was untreated, based on the covariates in the regression model. These probabilities are then averaged across the study population to determine the average probability of the occurrence of an event within a specific duration of follow-up in the population if all subjects were treated and if all subjects were untreated.

Results

Risk differences and numbers needed to treat.

Conclusions

Absolute measures of treatment effect can be derived in prospective studies when Cox regression is used to adjust for possible imbalance in prognostically important baseline covariates.

Introduction

In randomized controlled trials, the effect of treatment on dichotomous outcomes can be reported using a variety of measures of treatment effect: absolute risk reduction, relative risk, relative risk reduction, the number needed to treat (NNT), and the odds ratio. Schechtman argues that both relative and absolute measures should be reported [1]. Cook and Sackett argue that for clinical decision making the NNT is more meaningful than the relative risk, the relative risk reduction, or the odds ratio [2]. Jaeschke et al. suggest that the odds ratio and the relative risk provide limited information [3]. Finally, Sinclair and Bracken argue that clinically important questions are best addressed using relative risks, relative risk reductions, risk differences, and the NNT [4]. Common to all of these perspectives is the notion that relative measures of treatment effect provided limited clinical information, and that absolute measures of treatment effect (or measures derived from them such as the NNT) are more relevant for clinical decision making. In the face of these proposals, some medical journals require that absolute risk reductions and the associated NNT be reported for any randomized controlled trial with a dichotomous outcome [5].

In many randomized controlled trials, the outcome is time-to-event in nature. In such settings, Kaplan–Meier survival curves are often used to compare survival times among the different arms of the trial. When Kaplan–Meier survival curves are used to estimate the effect of treatment on survival, the difference in survival probabilities between treated and untreated subjects can be estimated for different durations of follow-up. This difference in survival probabilities provides an estimate at the population level of the effect of treatment on the probability of survival over a specified duration of time. Some statisticians have advocated that adjusted estimates of treatment effect be reported in randomized controlled trials [6], [7], [8], [9]. This allows one to adjust for potential residual imbalance in prognostically important baseline covariates between treatment arms. For this reason, Cox proportional hazards regression models have been used to estimate adjusted hazard ratios in randomized controlled trials. In this context, adjusted hazard ratios are reported to quantify the magnitude of the treatment effect. The hazard ratio provides a relative measure of treatment effect. Unfortunately, neither the absolute reduction in the probability of an event occurring during a specific length of follow-up nor the NNT to avoid one such event are conveyed by the hazard ratio.

Although time-to-event outcomes are common in randomized controlled studies, they also occur frequently in observational studies. Researchers are increasingly using observational studies to estimate the effect of treatment on outcomes. In nonrandomized studies, unlike in randomized trials, treated subjects often differ systematically from untreated subjects. Therefore, outcomes cannot be compared directly between treated and untreated subjects. Statistical methods must be used to adjust for systematic differences between treated and untreated subjects when estimating the effect of treatment on outcomes. A commonly used method in the medical literature for this purpose is the Cox proportional hazards regression model. The use of this model allows investigators to estimate the effect of treatment on survival after adjusting for baseline covariates. As in the case of randomized controlled trials, the use of Cox regression models allows one to estimate the relative impact of treatment on survival. However, the hazard ratio does not allow one to infer the absolute reduction in the probability of the outcome occurring during a specified duration of follow-up, nor the number of subjects that must be treated to avoid one event over a specified duration of follow-up.

Several authors have described methods to derive relative risks, relative risk reductions, absolute risk reductions, and NNTs from a logistic regression model in either a randomized controlled trial or in an observational study [10], [11], [12], [13]. Many of these methods are based on estimating the marginal (or population-average) probability of the outcome if all subjects in the population were treated, and the marginal probability of the outcome if all subjects were untreated. The use of these methods allows for reporting of more clinically meaningful measures of treatment effect when logistic regression is used to estimate adjusted odds ratios.

The objective of this paper was to describe how risk differences and NTTs can be derived from a Cox proportional hazards model in either a randomized controlled trial or in an observational study with time-to-event outcomes. I also demonstrate that this method can be used with parametric accelerated failure time (AFT) survival models. The paper is organized as follows: In Section 2, I describe how absolute measures of treatment effect can be derived from a time-to-event regression model. In Section 3, a case study is presented in which the utility of the proposed methods is demonstrated. Finally, in Section 4, I summarize my findings.

Section snippets

Estimating clinically meaningful measures of treatment effect from survival models

In this section, we describe how to estimate absolute reductions in the risk of an event occurring within a specific duration of follow-up using both Cox proportional hazards regression models and parametric AFT survival models. The NNT to avoid one event within the specified duration of follow-up can be calculated as the reciprocal of the absolute risk reduction.

Let us assume that in a randomized controlled trial or an observational study, a time-to-event outcome variable Ti is observed for

Data sources

Detailed clinical data were obtained by retrospective chart review on a sample of 7,613 patients discharged alive with a diagnosis of heart failure between April 1, 1999 and March 31, 2001 from 103 acute care hospitals in Ontario, Canada. Further detail of the data obtained is provided elsewhere [22]. These data were collected as part of the Enhanced Feedback for Effective Cardiac Treatment (EFFECT) Study, an ongoing initiative intended to improve the quality of care for patients with

Discussion

We have described a method to derive the absolute reduction in the probability of an event occurring within a specific duration of follow-up from an adjusted survival model. The NNT is calculated as the reciprocal of the absolute risk reduction. The method can be implemented using either a Cox proportional hazards regression model or with a parametric AFT model. This method is applicable in both randomized controlled trials and in observational cohort studies that report adjusted estimates of

Acknowledgments

The Institute for Clinical Evaluative Sciences (ICES) is supported in part by a grant from the Ontario Ministry of Health and Long-Term Care. The opinions, results, and conclusions are those of the authors and no endorsement by the Ministry of Health and Long-Term Care or by the ICES is intended or should be inferred. Dr. Austin is supported in part by a Career Investigator award from the Heart and Stroke Foundation of Ontario. This research was supported by an operating grant from the Canadian

References (29)

  • D.G. Altman et al.

    Baseline comparisons in randomized clinical trials

    Stat Med

    (1991)
  • P.W. Lavori et al.

    Designs for experiments—parallel comparisons of treatment

    N Engl J Med

    (1983)
  • Austin PC. Absolute risk reductions, relative risks, relative risk reductions, and numbers needed to treat can be...
  • R. Bender et al.

    Estimating adjusted NNT measures in logistic regression analysis

    Stat Med

    (2007)
  • Cited by (99)

    • Effectiveness of COVID-19 Booster on the Risk of Hospitalization Among Medicare Beneficiaries

      2022, Mayo Clinic Proceedings
      Citation Excerpt :

      We computed vaccine effectiveness as (1 − hazard ratio) × 100. We multiplied risk difference per 1 million person-days with 130 to calculate risk difference over 130 days per 1 million persons and again multiplied this with 15 to estimate the number of COVID-19 hospitalizations prevented by boosters in approximately 15 million currently unboosted persons aged 65 years or older (also see Supplemental Methods section).22,23 For the analysis of the optimal interval between original vaccination and booster, we used the Cox regression model to estimate the risk of hospitalization for boosted individuals compared with unboosted controls for different intervals between vaccination and receipt of boosters, controlling for all patient characteristics, original vaccination type, and the type of booster.

    View all citing articles on Scopus
    View full text