Abstract
This report presents a comprehensive,multi-dimensional mental health program performancemonitoring system that has recently been implemented inthe Department of Veterans Affairs. Principlesunderlying the development of the system are reviewed and68 specific monitors are described addressing four majorperformance domains: access, inpatient care, outpatientcare, and economic performance. Simple methods are presented for identifying outliers, forgenerating summary performance scores across series' ofrelated monitors, and for adjusting results fordifferences in patient characteristics across locales. Although still technically imperfect, andtherefore requiring continuous improvement, monitoringsystems such as the one presented can be useful toolsguiding and improving service delivery and mental health system performance, and providing a medium ofaccountability to consumers and otherstakeholders.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
Allen HE, Darling H, McNeill DN and Bastien F (1994) The Employee Health Care Value Survey: Round one. Health Affairs 13(1) 25–41.
American Psychiatric Association (1987) Diagnostic and Statistical Manual, Third Edition, Revised, Washington, DC, American Psychiatric Press.
Center for Mental Health Services (1994) Mental Health Components of a Health Plan Report Card, Washington, Center for Mental Health Services.
Cicchetti DV (1994) Guidelines, criteria and rules of thumb for evaluating normed and standardized instruments in psychology. Psychological Assessment 6: 284–290.
Corrigan JM and Neilson DM (1993) Toward the Development of uniform reporting standards for managed care Organizations: The Health Plan Employer Data and Information Set (Version 2.0) Journal on Quality Improvement 19(12) 566–575.
Dickey B (in press) The development of report cards for mental health care in L Sederer and B Dickey (Eds.) Outcomes assessment in clinical practice. Baltimore, Williams & Wilkins
Ellwood PM (1988) Shattuck lecture— Outcomes Management: A Technology of Patient Experience. New England Journal of Medicine 325: 1549–1556.
Endicott J, Spitzer RL, Fleiss JL, and Cohen J (1976): The Global Assessment Scale: A procedure for measuring overall severity of psychiatric disturbance. Archives of General Psychiatry 1976; 33: 766–771.
Epstein A (1995) Performance reports on quality— prototypes, problems and prospects. New England Journal of Medicine 333: 57–61.
Frisman L, Rosenheck RA (1995) Health Care for Homeless Veterans Programs: Eighth Progress Report. West Haven, Ct., Northeast Program Evaluation Center
Fontana AF, Rosenheck RA, Spencer HS and Grey S (1995). The long journey home IV: The Fourth Progress Report on the Department of Veterans Affairs Specialized PTSD Programs. West Haven, Ct., Northeast Program Evaluation Center.
Gatsonis CA, Epstein AM, Newhouse JP, Normand S-L, McNeil B (1995) Variations in the Utilization of Coronary Angiography for Elderly Patients with an Acute Myocardial Infarction: An Analysis Using Hierarchical Logistical Regression. Medical Care 33: 625–642.
Green J and Wintfield N (1995) Sounding board— report cards on cardiac surgeons: Assessing New York State’s Approach. New England Journal of Medicine 332: 1229–1232.
Hammermeister KE, Shroyer L, Sethi GK, Grover FL (1995) Why is it important to demonstrate linkages between outcomes of care and processes and structures of care. Medical Care 33: OS5–OS16.
Health Care Financing Administration (1987) Medicare hospital mortality information, 1986. Washington, DC; HCFA.
Hedeker D, Gibbons RD and Flay BR (1994) Random-effects regression models for clustered data with an example from smoking prevention research. Journal of Consulting and Clinical Psychology 62: 757–765.
Iezzoni LI (1995) Risk Adjustment for measuring health care outcomes, Ann Arbor Michigan, Health Administration Press.
Journal of Educational and Behavioral Statistics (1995) Hierarchical Linear Models: Problems and Prospects, Volume 20(2), Summer/1995.
Kamis-Gould E (1987) The New Jersey performance management system: A state system and uses of simple measures. Evaluation and Program Planning 10: 249–255.
Kizer KW (1995) Vision for change: A plan to restructure the veterans health administration. Washington, DC; Department of Veterans Affairs.
Leda CL, Rosenheck RA, Corwel L, and Olson RA (1993): The Fourth Progress Report on the Domiciliary Care for Homeless Veterans Program. West Haven, Ct., Northeast Program Evaluation Center
Leda CL, Rosenheck RA and Medak S (1994) First Progress Report on Veterans Industries/Therapeutic Residences Program. West Haven, Ct., Northeast Program Evaluation Center.
McNeil BJ, Pedersen SH and Gatsonis C (1992) Current Issues in Profiling Quality of Care. Inquiry 29: 298–307.
National Advisory Mental Health Council (1993) Health care reform for Americans with severe mental illnesses: Report of the national advisory mental health council. American Journal of Psychiatry 150: 1447–1465.
National Committee on Quality Assurance (1995) Report card pilot project. Washington, DC, The National Committee on Quality Assurance.
Rogers RH, Wells KB, Sturm MR, Burnam MA (1993) Outcomes for adult depressed outpatients under prepaid or fee-for-service financing. Archives of General Psychiatry 50: 517–525.
Roper WL, Winkwerder W, Hackbarth GM and Lakauer H (1988) Effectiveness in health care: An initiative to evaluate and improve medical practice. New England Journal of Medicine 319:1197
Rosenheck RA and Cicchetti D (1995) A Mental Health Program PerformanceMonitoring System for the Department of Veterans Affairs, West haven, CT, Northeast Program Evaluation Center.
Rost K, Smith GR, Burnam MA and Burns BJ (1992) measuring the outcomes of care for mental health problems: The case of depressive disorders. Medical Care Supplement 30(5): MS266–MS273.
Rupp A (1991) Underinsurance for severe mental illness. Schizophrenia Bulletin 17: 401–405.
Sorenson JE, Zelman W, Hanbery GW, Kucic AR (1987). Managing mental health organizations with twenty-five key performance indicators. Evaluation and Program Planning 10: 239–247.
Steinwachs DM, Wu AW and Skinner EA (1994) How will outcomes management work? Health Affairs 13(1) 153–162.
Stephenson GM and Findlay S (1995) HEDIS: Almost ready for prime time. Business and Health (March 1995): 39–50.
Tomich N (1995) VHA Reorganization: Interim steps sought. US Medicine 31: 1–13.
Veterans Health Administration (1995) An Integrated Performance Process Framework, Veterans Health Administration, Washington, D.C.
Williamson OE (1975). Markets and Hierarchies, New York, The Free Press.
Rights and permissions
About this article
Cite this article
Rosenheck, R., Cicchetti, D. A Mental Health Program Report Card: A Multidimensional Approach to Performance Monitoring in Public Sector Programs. Community Ment Health J 34, 85–106 (1998). https://doi.org/10.1023/A:1018720414126
Issue Date:
DOI: https://doi.org/10.1023/A:1018720414126