Abstract
Individuals not fully complying with their assigned treatments is a common problem encountered in randomized evaluations of behavioral interventions. Treatment group members rarely attend all sessions or do all “required” activities; control group members sometimes find ways to participate in aspects of the intervention. As a result, there is often interest in estimating both the effect of being assigned to participate in the intervention, as well as the impact of actually participating and doing all of the required activities. Methods known broadly as “complier average causal effects” (CACE) or “instrumental variables” (IV) methods have been developed to estimate this latter effect, but they are more commonly applied in medical and treatment research. Since the use of these statistical techniques in prevention trials has been less widespread, many prevention scientists may not be familiar with the underlying assumptions and limitations of CACE and IV approaches. This paper provides an introduction to these methods, described in the context of randomized controlled trials of two preventive interventions: one for perinatal depression among at-risk women and the other for aggressive disruptive behavior in children. Through these case studies, the underlying assumptions and limitations of these methods are highlighted.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The exact line of code for R is “tsls(outcome~D + x1 + x2, ~T + x1 + x2, data = dataset).”
The exact line of code for Stata is “ivregress 2sls outcome x1 x2 (D = T).”
References
Angrist, J., & Imbens, G. W. (1995). Two-stage least squares estimation of average causal effects in models with variable treatment intensity. Journal of the American Statistical Association, 90, 431–442. doi:10.2307/2291054.
Angrist, J., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91, 444–472. doi:10.2307/2291629.
Baker, S. G., & Kramer, B. S. (2005). Simple maximum likelihood estimates of efficacy in randomized trials and before-and-after studies, with implications for meta-analysis. Statistical Methods in Medical Research, 14, 349–367. doi:10.1191/0962280205sm404oa.
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck Depression Inventory (2nd ed.). San Antonio, TX: The Psychological Corporation.
Black, M. M., Bentley, M. E., Papas, M. A., Oberlander, S., Teti, L. O., McNary, S., et al. (2006). Delaying second births among adolescent mothers: A randomized, controlled trial of a home-based mentoring program. Pediatrics, 118, e1087–e1099. doi:10.1542/peds.2005-2318.
Bloom, H. S. (1984). Accounting for no-shows in experimental evaluation designs. Evaluation Review, 8, 225–246. doi:10.1177/0193841X8400800205.
Connell, A. M., Dishion, T. J., Yasui, M., & Kavanagh, K. (2007). An adaptive approach to family intervention: Linking engagement in family-centered intervention to reductions in adolescent problem behavior. Journal of Consulting and Clinical Psychology, 75, 568–579. doi:10.1037/0022-006X.75.4.568.
Dunn, G., Maracy, M., Dowrick, C., Ayuso-Mateos, J. L., Dalgard, O. S., Page, H., et al. (2003). Estimating psychological treatment effects from a randomised controlled trial with both non-compliance and loss to follow-up. The British Journal of Psychiatry, 183, 323–331. doi:10.1192/bjp.183.4.323.
Dunn, G., Maracy, M., & Tomenson, B. (2005). Estimating treatment effects from randomized controlled trials with noncompliance and loss to follow-up: The role of instrumental variables methods. Statistical Methods in Medical Research, 14, 369–395. doi:10.1191/0962280205sm403oa.
Foster, E. M. (2000). Is more better than less? An analysis of children’s mental health services. Health Services Research, 35, 1135–1158.
Foster, E. M. (2003). Propensity score matching: An illustrative analysis of dose response. Medical Care, 41, 1183–1192. doi:10.1097/01.MLR.0000089629.62884.22.
Frangakis, C. E., & Rubin, D. B. (1999). Addressing complications of intention-to-treat analysis in the combined presence of all-or-none treatment noncompliance and subsequent missing outcomes. Biometrika, 86, 365–379. doi:10.1093/biomet/86.2.365.
Frangakis, C. E., & Rubin, D. B. (2002). Principal stratification in causal inference. Biometrics, 58, 21–29. doi:10.1111/j.0006-341X.2002.00021.x.
Hill, J. L., Brooks-Gunn, J., & Waldfogel, J. B. B. (2003). Sustained effects of high participation in an early intervention for low-birth-weight premature infants. Developmental Psychology, 39, 730–744. doi:10.1037/0012-1649.39.4.730.
Hirano, K., Imbens, G. W., Rubin, D. B., & Zhou, X. (2000). Assessing the effect of influenza vaccine in an encouragement design with covariates. Biostatistics (Oxford, England), 1, 69–88. doi:10.1093/biostatistics/1.1.69.
Ialongo, N., Werthamer, L., Brown, C. H., Kellam, S., & Wai, S. B. (1999). The proximal impact of two first grade preventive interventions on the early risk behaviors for later substance abuse, depression and antisocial behavior. American Journal of Community Psychology, 27, 599–642. doi:10.1023/A:1022137920532.
Imbens, G. W., & Rubin, D. B. (1997). Bayesian inference for causal effects in randomized experiments with noncompliance. The Annals of Statistics, 25, 305–327. doi:10.1214/aos/1034276631.
Jin, H., & Rubin, D. B. (2008). Principal stratification for causal inference with extended partial compliance. Journal of the American Statistical Association, 103, 101–111. doi:10.1198/016214507000000347.
Jo, B. (2002a). Estimation of intervention effects with noncompliance: Alternative model specifications. Journal of Educational and Behavioral Statistics, 27, 385–409. doi:10.3102/10769986027004385.
Jo, B. (2002b). Statistical power in randomized intervention studies with noncompliance. Psychological Methods, 7, 178–193. doi:10.1037/1082-989X.7.2.178.
Jo, B., Asparouhov, T., Muthen, B. O., Ialongo, N. S., & Brown, C. H. (2008). Cluster randomized trials with treatment noncompliance. Psychological Methods, 13, 1–18. doi:10.1037/1082-989X.13.1.1.
Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55–63. doi:10.1023/A:1021786811186.
Lin, J. Y., Ten Have, T. R., & Elliott, M. R. (2008). Longitudinal nested compliance class model in the presence of time-varying noncompliance. Journal of the American Statistical Association, 103, 462–473.
Little, R. J., & Rubin, D. B. (2000). Causal effects in clinical and epidemiological studies via potential outcomes: Concepts and analytical approaches. Annual Review of Public Health, 21, 121–145. doi:10.1146/annurev.publhealth.21.1.121.
Little, R. J. A., & Yau, L. (1998). Statistical techniques for analyzing data from prevention trials: Treatment of no-shows using Rubin’s causal model. Psychological Methods, 3, 147–159. doi:10.1037/1082-989X.3.2.147.
Muñoz, R. F., Le, H. N., Ghosh Ippen, C., Diaz, M. A., Urizar Jr, G. G., Soto, J., et al. (2007). Prevention of postpartum depression in low-income women: Development of the Mamás y Bebés/Mothers and Babies Course. Cognitive and Behavioral Practice, 14, 70–83. doi:10.1016/j.cbpra.2006.04.021.
Muthén, L. K., & Muthén, B. O. (1998–2007). Mplus user’s guide (5th ed.). Los Angeles, CA: Author.
O’Malley, A. J., & Normand, S. T. (2005). Likelihood methods for treatment noncompliance and subsequent nonresponse in randomized trials. Biometrics, 61, 325–334. doi:10.1111/j.1541-0420.2005.040313.x.
Peng, Y., Little, R. J., & Raghunathan, T. E. (2004). An extended general location model for causal inferences from data subject to noncompliance and missing values. Biometrics, 60, 598–607. doi:10.1111/j.0006-341X.2004.00208.x.
R Core Development Team. (2007). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0. www.r-project.org.
Radloff, L. S. (1977). The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401. doi:10.1177/014662167700100306.
Robins, J. M., & Greenland, S. (1992). Identifiability and exchangeability of direct and indirect effects. Epidemiology (Cambridge, Massachusetts), 3, 143–155. doi:10.1097/00001648-199203000-00013.
Rosenbaum, P. R. (2002). Observational studies (2nd ed.). New York: Springer.
Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. The Annals of Statistics, 6, 34–58. doi:10.1214/aos/1176344064.
SPSS Inc. (2007). SPSS Base 16.0 for Windows. Chicago, IL: Author.
Stata Corp. (2007). Stata statistical software: Release 10. College Station, TX: Author.
Werthamer-Larsson, L., Kellam, S. G., & Wheeler, L. (1991). Effect of first-grade classroom environment on child shy behavior, aggressive behavior, and concentration problems. American Journal of Community Psychology, 19, 585–602. doi:10.1007/BF00937993.
Acknowledgment
This research supported in part by grant R40 MC 02497 from the Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services (PI: Le) as well as by the Center for Prevention and Early Intervention, jointly funded by the National Institute of Mental Health and the National Institute on Drug Abuse (MH066247; PI: Ialongo).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Stuart, E.A., Perry, D.F., Le, HN. et al. Estimating Intervention Effects of Prevention Programs: Accounting for Noncompliance. Prev Sci 9, 288–298 (2008). https://doi.org/10.1007/s11121-008-0104-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-008-0104-y