What is new?
There is variation in the use of item scaling within questionnaires designed to measure patient experiences and satisfaction with care. Evidence supporting alternative scaling formats is scant.
This study found that a five-point scale with all-point-defined scale produced data of better quality with lower means, floor, and ceiling effects than a 10-point end-point-defined scale.
Questionnaire items designed to assess aspects of patient experiences and satisfaction with care should use five-point all-point-defined scales in preference to 10-point end-point-defined scales.
Self-completed questionnaires are widely used within health-related research for collecting information from patients and health professionals. The measurement of patient-reported outcomes, in particular, has seen a huge growth in the number of questionnaires as documented by several systematic reviews [1], [2], [3], [4]. The number of published articles reporting the application, development, or evaluation of questionnaires that are designed to assess patient experiences or satisfaction with health care is considerable [3], [5], [6], [7], [8].
There are a number of options for developing questionnaires in relation to questionnaire appearance, wording, and sequencing [9]. The evidence for questionnaire appearance has been described as scant, but evidence in relation to the structure of questions, including response categories, suggests that they can strongly influence data quality [9]. The ordering of categories, the number of categories, whether there is an odd or even number of categories, and how categories are labeled have all been found to influence results [9], [10].
Most questionnaires within the field of patient experiences and satisfaction have used items with all-point-defined scales [9] where each scale point has a descriptor [11], [12], [13], [14], [15], [16], [17], a well-known example being the five-point Likert scale. After a systematic review, it was found that patient satisfaction questionnaires include response scales with between 3 and 11 points, the mode being five points [5]. There is little evidence to support the use of these alternate response scales. Research, including randomized studies, has been recommended to determine which form of response scale produces data with the best quality. This includes missing data and floor and ceiling effects, the latter being an important consideration with the field of patient experiences where results are often highly skewed toward more positive levels of satisfaction [8], [12], [16], [18], [19].
Studies that have assessed different response scales for patient experience and satisfaction questionnaires have largely compared all-point-defined scales that differ in their use of descriptors or number of scale points [20]. Five-point scales were found to produce more variance than six-point scales in a study of patients receiving outpatient care in the United States [21]. Three all-point-defined scales, two with 5 points and one with 10 points, were compared in a study of inpatients in The Netherlands [22]. At the item level, a five-point scale that ranged from “dissatisfied” to “very satisfied” was considered optimal. More recently, the same group compared 5- and 10-point scales in a larger sample of hospital inpatients [20]. The former used “dissatisfied” to “very satisfied,” and the latter used “very poor” to “excellent” descriptors. The five-point scale produced less missing data, more variance, less skewed, and less kurtotic distributions. Finally, a study of patient satisfaction with general practice out-of-hours services compared three different five-point all-point-defined scales—a Likert scale of “very dissatisfied” to “very satisfied”, the same scale with smiley faces alongside, a scale of “very poor” to “excellent”—and a 10-point end-point-defined scale of “very dissatisfied” to “very satisfied [23].” The authors concluded that there were no significant differences in response rates or data quality for the different scales.
National surveys of patient experiences with health care have been conducted over the past 12 years within the Norwegian health care system [16], [17], [24], [25], [26]. Earlier surveys used 10-point end-point-defined scales or bipolar adjective scales, where only the scale end points have descriptors [16], [24], [25], [26]. However, this scale produced poor levels of data quality in pilot studies with psychiatric patients. Subsequent national surveys have used five-point all-point-defined scales that have produced data that was more normally distributed with lower floor and ceiling effects both for psychiatry and other groups of patients [17], [27], [28]. The finding that such scales produce better data quality with less of a skew toward positive experiences led to their adoption in a national survey of parent experiences of child inpatient care [17].
The Patient Experiences Questionnaire (PEQ) uses 10-point end-point-defined scales and has evidence for reliability and validity [24], [26]. The encouraging results after more recent Norwegian national surveys that used the five-point scale [17], [27], [28] have meant that this form of scaling has been considered for use with the PEQ. The work that follows presents a randomized comparison of the original version of the PEQ with 10-point scales with a version that uses a five-point all-point-defined scale.