Elsevier

Psychiatry Research

Volume 111, Issues 2–3, 30 August 2002, Pages 191-197
Psychiatry Research

Inpatient diagnostic assessments: 3. Causes and effects of diagnostic imprecision

https://doi.org/10.1016/S0165-1781(02)00147-6Get rights and content

Abstract

Preceding studies found that clinicians using the Traditional Diagnostic Assessment (TDA, the standard of clinical practice) often made imprecise diagnoses, compared with gold standards. Those same studies found excellent diagnostic agreement (kappa>0.75) between Computer Assisted Diagnostic Interview (CADI) and gold standards, thus warranting CADI's use as the standard for data collection and diagnosis in this study. When TDA and CADI users independently examined 106 inpatient-subjects, TDA users agreed only 45.3% (48/106) with CADI's primary diagnosis and found only 50.5% as many total diagnoses. This study searched for the causes and effects of those differences. To test the hypothesis that insufficient data collection was the cause, the 106 TDA write-ups were analyzed word-by-word. Only 46.2% (49/106) of the TDA write-ups listed enough symptom criteria (e.g. hallucinations, depression) to meet DSM-IV requirements for diagnosis, a likely cause of TDA's inaccuracy. TDA write-ups evaluated only 52.9% of the 18 Key Criteria necessary to screen for 10 diagnostic groups, a likely cause of TDA's incompleteness. TDA's diagnostic imprecision had effects on (1) length of stay (LOS) for hospitalized patients and (2) associated costs. Patients evaluated with TDA had a mean LOS of 12.5 days versus 7.7 days for CADI patients, a reduction of 4.8 days (12.5–7.7). If CADI replaced TDA, then annual savings of $3 000 000 system-wide could be projected for inpatient care. Remedies for TDA's diagnostic imprecision are proposed.

Introduction

The Traditional Diagnostic Assessment (TDA) is the standard of practice for making initial psychiatric diagnoses. Textbooks (Othmer and Othmer, 1994, Hales et al., 1995, Sadock and Sadock, 2000) agree generally about the TDA's processes and formats. Although textbooks accept its diagnostic precision as sufficient for clinical practice, research finds that TDA users often make inaccurate diagnoses (Hill et al., 1996, Lipton and Simon, 1985, McGorry et al., 1995, Miller, 2001, Miller et al., 2001, Mojtabai and Nicholson, 1995, Skodol et al., 1984, van Praag, 1997, Williams et al., 1996). Clinicians continue their universal use of the TDA in spite of contrary evidence, but investigators avoid using it as a solo diagnostic instrument in research (Basco et al., 2000, Shear et al., 2000, Widiger et al., 1994, Widiger et al., 1996).

The Computer Assisted Diagnostic Interview (CADI) is a structured computer-based interview. Previous studies (Miller et al., 2001, Miller, 2001) found that CADI's inpatient diagnoses agree with SCID-CV (kappa>0.75) and Consensus Diagnosis (kappa>0.81) and that CADI has excellent interrater reliability (kappa>0.91), thus warranting CADI's use as the standard for data collection and diagnosis in this study.

When TDA users and CADI users independently examined 106 inpatient-subjects, TDA users agreed only 45.3% (48/106) with the CADI diagnoses and named on average only 50.5% (1.53 versus 3.03) as many diagnoses per patient as did CADI. The purpose of this study is to research the causes and effects of those findings.

Section snippets

Subjects and evaluators

Subjects came from two groups. Group A (Miller et al., 2001, Miller, 2001) included 56 inpatient-subjects in an acute psychiatric unit of a publicly funded hospital affiliated with a medical school. Six experienced clinicians (five psychiatrists, one Ph.D. psychologist) made the TDA evaluations. Five research investigators (including the author) made the CADI evaluations blind (data came only from subjects).

Group B included 50 inpatient-subjects admitted consecutively to a publicly funded

TDA users’ diagnostic accuracy

Table 2, bottom line, shows that only 46.2% (49/106) of TDA write-ups assessed enough symptom criteria to meet DSM-IV diagnostic requirements. The consequent diagnoses agreed only 45.3% (48/106) with CADI diagnoses. CADI requires users to collect all data; the program then computes the diagnosis by matching the collected data exactly with DSM-IV algorithms.

TDA users’ diagnostic completeness

DSM-IV requires that Key Criteria must be evaluated to make the linked diagnoses. Table 1 (bottom two lines) shows that TDA users assessed

Limitations

These include regionalism (multi-ethnicity and urbanism of Los Angeles County) and non-representativeness of subjects (inpatients with severe chronic disorders). Clinicians represented public, private and medical school sectors about equally. Results are provisional, awaiting other studies for wider validation.

Clinicians make the best diagnoses possible, given constraints of money and time

Health care financing drives HMOs, insurance companies, and government agencies to restrict time for initial evaluations to a fraction of Klein's (1995) recommendation for 90–120 min.

References (30)

  • M.R. Basco et al.

    Methods to improve diagnostic accuracy in a community mental health setting

    American Journal of Psychiatry

    (2000)
  • A.S. Elstein et al.

    Medical Problem Solving. An Analysis of Clinical Reasoning

    (1978)
  • W.S. Fenton et al.

    Diagnostic efficiency of DSM-III schizophrenia

    Journal of Nervous and Mental Disease

    (1989)
  • H.N. Garb

    Studying the Clinician: Judgment Research and Psychological Assessment

    (1998)
  • W.M. Glazer

    Defining best practices: a prescription for greater autonomy

    Psychiatric Services

    (1998)
  • J.H. Greist

    Clinical computing. The computer as clinician assistant: assessment made simple

    Psychiatric Services

    (1998)
  • Hales, R.E., Yudofsky, S.C., Talbott, J.A. (Eds.), 1995. Textbook of Psychiatry. Second ed. American Psychiatric Press,...
  • C. Hill et al.

    Problem of diagnosis in post-mortem brain studies of schizophrenia

    American Journal of Psychiatry

    (1996)
  • M. Hunt

    The Universe Within. A New Science Explores the Human Mind

    (1977)
  • Jason, H., 1978. Foreword. In: Elstein, A.S., Shulman, L.S., Sprafka, S.A. (Eds.), Medical Problem Solving. An Analysis...
  • D.F. Klein

    Psychopharmacological practice and health reform

    Psychiatric Annals

    (1995)
  • A.A. Lipton et al.

    Psychiatric diagnosis in a state hospital: Manhattan State revisited

    Hospital and Community Psychiatry

    (1985)
  • C.J. McDonald

    Protocol-based computer reminders, the quality of care and the non-perfectibility of man

    New England Journal of Medicine

    (1976)
  • P.D. McGorry et al.

    Spurious precision: procedural validity of diagnostic assessment in psychotic disorders

    American Journal of Psychiatry

    (1995)
  • J. May et al.

    Note-taking and information recall

    Journal of American Medical Education

    (1977)
  • Cited by (28)

    • Agreement between clinical and MINI diagnoses in outpatients with mood and anxiety disorders

      2017, Journal of Affective Disorders
      Citation Excerpt :

      So why do the MINI and the clinical evaluation of the same patient not agree to the degree we expected? Previous studies explained the moderate agreement rates of clinician's diagnoses (Miller et al., 2001) (compared to structured clinical assessment) through insufficient data collection; i.e. clinicians focus on the most likely (primary) diagnosis and are therefore more likely to disregard other symptoms and comorbid diagnoses (Miller, 2002; Rettew et al., 2009). Clinicians may also be more focused on identification of the primary disorder to ensure a diagnosis that responds best to treatment (de Beurs et al., 2011; Rettew et al., 2009; Weinstein et al., 1989; Welner et al., 1987).

    • Implementing the K-SADS-PL as a standard diagnostic tool: Effects on clinical diagnoses

      2016, Psychiatry Research
      Citation Excerpt :

      There is strong evidence for a low to moderate agreement between research diagnostic assessments and clinical evaluations, as well as for the poor inter-rater reliability of clinician-generated diagnoses (Miller, 2001, 2002; Jewell et al., 2004; Rettew et al., 2009).

    • Clinical Utility of a Dimensional Model of Personality Disorder

      2010, Professional Psychology: Research and Practice
    • Clinical Utility and DSM-V

      2009, Psychological Assessment
    View all citing articles on Scopus

    Similar findings with an overlapping sample were presented at the Annual Meetings, American Psychiatric Association, 6 May, 2000, in Chicago, IL, and 18 May, 2002, in Philadelphia, PA.

    View full text