Validation of a tool to assess health practitioners’ decision support and communication skills

https://doi.org/10.1016/S0738-3991(03)00043-0Get rights and content

Abstract

As patients become more involved in decisions affecting their health, it is important to monitor and improve the support clinicians provide to facilitate shared decision making. The Decision Support Analysis Tool (DSAT) was developed as a research tool to evaluate practitioners’ use of decision support and related communication skills during a clinical encounter. The DSAT, consisting of six categories of decision support skills and four categories of communication skills, was tested with 34 actual transcripts of patient–physician dialogue. The patients were prepared for the clinical encounter with either a detailed decision aid plus worksheet (n=16) or a pamphlet (n=18). Pairs of raters, blinded to the intervention allocation, coded each transcript independently. The overall inter-rater agreement and kappa coefficients were, respectively 75% and 0.59 for the decision support skills and 76% and 0.68 for the communication skills categories. The frequency of DSAT skills coded: (a) were significantly correlated with three out of six patient and physician outcome measures (r>0.30, P<0.05); and (b) showed significant discrimination (P=0.05) or trends (P<0.15) in discrimination between the decision aid and pamphlet groups. The DSAT shows promise as a reliable and valid evaluation tool but requires further testing with larger samples.

Introduction

Health professionals’ ability to communicate effectively is increasingly recognized as a core clinical skill [1], [2]. The quality of practitioner–patient interactions directly impact on health outcomes, adherence to treatment, as well as patients’ and practitioners’ satisfaction [3], [4]. Moreover, patient-centered approaches have emerged from a new paradigm in health care that stresses patients’ perspectives of their well-being and involvement in care [5], [6]. The paternalistic model, in which practitioners decide for the patient, is gradually being replaced by a model of shared decision making involving both the practitioner and the patient [5]. The role of the health professional is now not only to inform the patient about his disease or treatment but also to create an effective relationship by assessing patients’ needs and concerns, showing understanding, empathy and providing support [7]. Hence, in a shared decision making situation, patients’ and practitioners’ active cognitive and affective participation is imperative for the success of the interaction. Practitioners actively elicit patients’ points of view, help them to express themselves openly, and ask questions about issues that affect decision making [8]. It is this type of dialogue that provides the fundamental vehicle through which the shared decision support process is defined and operationalized [9]. However, the quality of the interaction not only depends on practitioners’ abilities to involve patients in discussing health-related decisions but also on their knowledge and skills in supporting patients through all phases of the decision making process (Appendix A, Part 1).

Despite the wide acceptance of a shared decision making model in supporting patients with difficult health-related decisions, very few practicing clinicians have been formally prepared to effectively respond to patients’ needs in this domain [10]. Practice guidelines for these difficult decisions recommend that clinicians help patients to: (a) understand the options and their potential benefits and harms; (b) consider the personal value they attach to the benefits and the harms; and (c) participate in decision making [11], [12]. Decision aids have been developed as adjuncts to counseling to prepare patients for the clinical encounter with the practitioner [13], [14]. Studies evaluating decision aids have found that they are acceptable to patients, help the uncertain to make decisions, increase participation in decision making, and increase the likelihood that choices are based on better knowledge, more realistic expectations of outcomes and personal values [15], [16]. However, less attention has been paid to the follow-up counseling session, which is a core activity in the practitioner’s decision supporting role [11]. We know little about the effects of decision aids on physician–patient interactions [15], [16].

A key issue in analyzing the practitioner–patient interactions during shared decision making is measurement. While several tools focus on communication or general problem-solving abilities [17], they are limited in evaluating the decision support skills needed when counseling patients about specific, value-sensitive, decisions. Moreover, we do not know which decision support and communication skills are associated with greater patient and physician satisfaction with the decision making process and the decision itself.

We have developed the Decision Support Analysis Tool (DSAT), which was initially used in educational settings to evaluate students’ decision support and related communication skills (Appendix A). The theoretical underpinnings of the DSAT are the Ottawa Decision Support Framework (ODSF) and Ivey’s problem-solving model [18]. A detailed description of the Ottawa Decision Support Framework is presented elsewhere [13]. The key elements of the framework are: assessing patients’ decision making needs; providing decision support; and evaluating the success of support in improving decision making and the outcomes of the decisions [13]. The scope of needs identified and addressed by the DSAT relate to the current decision making status of the patient and focus on the modifiable factors contributing to decisional conflict, including information deficits, values clarity problems, and support and resource deficits.

In a single clinical encounter, progress through the entire phase of decision making may not be feasible; therefore, we limited our evaluation to the assessment of clinicians’ success in facilitating progress in decision making. According to Ivey’s [18] problem-solving model, fostering commitment to act is an important element in an interview.

To emphasize the importance of decision making skill development in the patient–practitioner encounter, we added an additional element to the DSAT. We asserted that skills learned in the decision making process should be transferable to future decision making. Therefore, we wanted to assess whether practitioners foster awareness relating to key principles used in the decision making process by encouraging patients to reflect on their experiences and the learning gained from making a difficult decision [19], [20].

Finally, decision support also includes managing relationships and communicating effectively. Ivey’s [18] problem-solving model served as the basis for some of the elements in the DSAT, notably establishing structure and rapport during an interview. Other interpersonal communication skills assessed in the DSAT were derived from those described by Arnold and Boggs [21]. These skills included listening, questioning and sending messages.

Section snippets

Methods

Since the DSAT had not been evaluated as a research tool, our study objective was to evaluate its reliability and validity. We tested the DSAT with transcripts of actual physician–patient dialogue gathered in a randomized trial of a decision support intervention.

Reliability

Overall inter-rater agreement for decision support (75%, kappa=0.58) and communication skills codes (76%, kappa=0.68) were similar. The inter-rater percentage agreement scores for the categories of the decision support process were highest for discussion of knowledge/information (90%), followed by discussion of values (56%), discussion of commitment to progress to the next stage of decision making (52%), discussion of decision making status (40%), discussion of support (33%), and finally

Discussion and conclusion

This first evaluation of the DSAT as a research tool indicates that adequate levels of inter-rater reliability and construct validity with some outcome measures can be reached. The tool also shows promise in discriminating between patients’ exposure to different types of decision support interventions.

These results need to be interpreted in light of the study limitations. First, the sample size was very small; it was underpowered to detect anything other than large correlations and differences

References (35)

  • C.H. Braddock et al.

    Informed decision making in outpatient practice time to get back to basics

    J. Am. Med. Assoc.

    (1999)
  • American College of Physicians. Guidelines for counseling postmenopausal women about preventive hormone therapy. Ann...
  • J.P. Kassirer

    Incorporating patients preferences into medical decisions

    N. Engl. J. Med.

    (1994)
  • E.A. Mort

    Clinical decision making in the face of uncertainty: hormone replacement therapy as an example

    J. Fam. Pract.

    (1996)
  • Hersey J, Matheson J, Lohr K. Consumer health informatics and patient decision making. Research Triangle Institute:...
  • A.M. O’Connor et al.

    Decision aids for patients considering health care options: evidence of efficacy and policy implications

    J. Natl. Cancer Inst.

    (1999)
  • G. Elwyn et al.

    Measuring the involvement of patients in shared decision making: a systematic review of instruments

    Patient Educ. Couns.

    (2000)
  • Cited by (76)

    • Using formal methods for content validation of medical procedure documents

      2017, International Journal of Medical Informatics
    • Formal Verification of Health Assessment Tools: a Case Study

      2016, Electronic Notes in Theoretical Computer Science
    • The psychometric properties of Observer OPTION<sup>5</sup>, an observer measure of shared decision making

      2015, Patient Education and Counseling
      Citation Excerpt :

      Observer (direct) measures, where trained assessors directly observe behavior via, for example, recorded (audio, video, transcript) encounters, are theoretically less prone to these effects, and can be triangulated with patient reported measures to produce more comprehensive assessment [2]. There are several observer measures of SDM, ranging 7–70 items [3–9], with variable levels of reliability and largely unexamined validity [10]. Observer OPTION12 (OPTION12) [6] is the most commonly-used observer measure.

    View all citing articles on Scopus
    View full text