Abstract
The primary goals of the present study were 1) to explore the types and frequencies of adaptation reported by facilitators; 2) to document their reasons for making those adaptations; and 3) to propose an efficient approach to the study of prevention program fidelity based on interview data. We interviewed 42 program facilitators involved in a large-scale dissemination about their implementation of a community-based prevention program. Interview questions addressed facilitators’ attitudes about program fidelity and the various types of changes, deletions, and additions they made. Although nearly all facilitators reported that fidelity to program curriculum was important, most also reported adapting the program. The most frequent reason facilitators gave for adaptation was deleting or changing material because they ran out of time. We employed a method used in business and health care settings (the Pareto principle, or the “law of the vital few”) to identify the minority (30%) of types of adaptation that accounted for a majority (70%) of all adaptations reported. Similarly, we identified the minority of reasons (25%) for adaptation that accounted for a majority (75%) of reasons reported. This approach helps to focus the assessment of fidelity on those adaptations that occur with high frequency, since low-frequency events are unlikely to have a significant impact on large-scale outcomes. High-frequency adaptations can then be targeted to determine their effect on outcomes; to address in training; and to assess on an ongoing basis for continuous quality improvement.
Similar content being viewed by others
References
Berwick, D. M., & Leape, L. L. (1999). Reducing errors in medicine: It's time to take this more seriously. British Medical Journal, 319, 136–137.
Botvin, G. J. (2004). Advancing prevention science and practice: Challenges, critical issues, and future directions. Prevention Science, 5(1), 69–72.
Botvin, G. J., & Griffin, K. W. (2005). Prevention science, drug abuse prevention, and life skills training: Comments on the state of the science. Journal of Experimental Criminology, 1(1), 63–78.
Castro, F. G., Barrera, M. J., & Martinez, C. R. J. (2004). The cultural adaptation of prevention interventions: Resolving tensions between fidelity and fit. Prevention Science, 5(1), 41–45.
Chen, H. (1990). Theory-driven evaluations. Newbury Park, CA: Sage Publications.
Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18(2), 237–256.
Dusenbury, L., Brannigan, R., Falco, M., & Lake, A. (2004). An exploration of fidelity of implementation in drug abuse prevention among five professional groups. Journal of Alcohol and Drug Education, 47(3), 4–19.
Elliott, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5(1), 47–52.
Foxcroft, D. R., Ireland, D., Lister-Sharp, D. J., Loew, G., & Breen R. (2003). Primary prevention for alcohol misuse in young people (Cochrane Review). In The Cochrane Library, Issue 1. Oxford: Update Software.
Greenberg, M. T., Domitrovich, C., & Bumbarger, B. (2001). The prevention of mental disorders in school-aged children: Current state of the field. Prevention and Treatment, 4(1), 1.
Gryna, F. M., Chua, R. C. H., & DeFeo, J. A. (Eds.). (2005). Juran's quality planning and analysis for enterprise quality. New York: McGraw-Hill.
Heflinger, C. A. (1996). Implementing a system of care: Findings from the Fort Bragg evaluation project. The Journal of Mental Health Administration, 23, 16–29.
Hogue, A., Liddle, H. A., Singer, A., & Leckrone, J. (2005). Intervention fidelity in family-based prevention counseling for adolescent problem behaviors. Journal of Community Psychology, 33(2), 191–211.
Juran, J. M. (1954). Universals in management planning and controlling. The Management Review, 43, 748–761.
Kaftarian, S., Robinson, E., Compton, W., Davis, B. W., & Volkow, N. (2004). Blending prevention research and practice in schools: Critical issues and suggestions. Prevention Science, 5(1), 1–3.
Koch, R. (1999). The 80-20 principle. Redfern, Australia: Currency Press.
Kumpfer, K. L., Molgaard, V., & Spoth, R. (1996). The strengthening families program for the prevention of delinquency and drug use. In R. D. Peters & R. J. McMahon (Eds.), Preventing childhood disorders, substance abuse, and delinquency (pp. 241–267). Thousand Oaks CA: Sage Publications.
Kumpfer, K. L., Alvarado, R., Smith, P., & Bellamy, N. (2002). Cultural sensitivity and adaptation in family-based prevention interventions. Prevention Science, 3, 239–244.
Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education and Behavior, 31, 242–257.
Limber, S. P. (2006). The Olweus bullying prevention program: An overview of its implementation and research basis. Mahwah, NJ: Lawrence Erlbaum Associates Publishers.
Marsden, J. R., & Mathayalakan, S. (1999). A multisession comparative study of group size and group performance in an electronic meeting system environment. IEEE Transactions on Systems, Man, and Cybernetics—Part C: Applications and Reviews, 29, 169–185.
Molgaard, V. K., Kumpfer, K., & Fleming, E. (nd). Strengthening Families Program for Parents and Youth 10-1-4: A Video Based Curriculum (Leader Guide). Ames, Iowa: Iowa State University Extension.
Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315–340.
Mrazek, P. J., & Haggerty, R. J. (1994). Reducing risks for mental disorders: Frontiers for prevention research (summary). (pp. 1–42). Washington, DC: National Academy Press.
Ogden, T., Forgatch, M. S., Askeland, E., Patterson, G. R., & Bullock, B. M. (2005). Implementation of parent management training at the national level: The case of Norway. Journal of Social Work Practice, 19(3), 317–329.
Ringwalt, C. L., Ennett, S., Johnson, R., Rohrbach, L. A., Simons-Rudolph, A., Vincus, A., & Thorne, J. (2003). Factors associated with fidelity to substance use prevention curriculum guides in the nation's middle schools. Health Education and Behavior, 30, 375–391.
Ringwalt, C., Ennett, S. T., Vincus, A., & Simons-Rudolph, A. (2004). Students’ special needs and problems as reasons for the adaptation of substance abuse prevention curricula in the nation's middle schools. Prevention Science, 5, 197–206.
Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237–260.
Shoham, A. (1999). Bounded rationality, planning, standardization of international strategy, and export performance: a structural model examination. Journal of International Marketing, 7, 24–50.
Spoth, R., Guyll, M., Trudeau, L., & Goldberg-Lillehoj, C. (2002). Two outcome studies of proximal outcomes and implementation quality of universal preventive interventions in a community-university collaboration context. Journal of Community Psychology, 30, 499–518.
Spoth, R., Redmond, C., & Shin, C. (2001). Randomized trial of brief family interventions for general populations: Adolescent substance use outcomes four years following baseline. Journal of Consulting and Clinical Psychology, 69, 627–642.
St Pierre, T. L., & Kaltreider, D. L. (2004). Tales of refusal, adoption, and maintenance: Evidence-based substance abuse prevention via school-extension collaborations. American Journal of Evaluation, 25(4), 479–491.
Walker, H. M., Golly, A., McLane, J. Z., & Kimmich, M. (2005). The Oregon first step to success replication initiative: Statewide results of an evaluation of the program's impact. Journal of Emotional and Behavioral Disorders, 13(3), 163–172.
Acknowledgments
This work was supported by National Institute of Drug Abuse (NIDA) grant R21 DA 19758-01. This work was also supported in part by the Margaret M. Hard award through the graduate school of Washington State University. Our thanks to Angel Cantu for assistance with research and to the facilitators who completed interviews, to Linda Becker of the Division of Alcohol and Substance Abuse of Washington State, and to the program facilitators who were willing to be interviewed.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hill, L.G., Maucione, K. & K. Hood, B. A Focused Approach to Assessing Program Fidelity. Prev Sci 8, 25–34 (2007). https://doi.org/10.1007/s11121-006-0051-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-006-0051-4