Original Article
Five-point scales outperform 10-point scales in a randomized comparison of item scaling for the Patient Experiences Questionnaire

https://doi.org/10.1016/j.jclinepi.2010.02.016Get rights and content

Abstract

Objective

To assess the data quality of two approaches to scaling items within the Patient Experiences Questionnaire (PEQ); a five-point scale with descriptors for all scale points and a 10-point scale with descriptors only at the end points.

Study Design and Setting

The two versions were pretested through cognitive interviews with 14 patients. The PEQ was then mailed to 1,000 patients after inpatient treatment at a large university hospital in Norway, randomized to receive the 5- or 10-point scale versions. Response rates, missing data, item means, floor, and ceiling effects were assessed. Regression analysis was used to examine the impact of response scale on missing data, floor, and ceiling effects after controlling for age, education level, and health status.

Results

The five-point scale produced data with unimodal and fairly symmetric distributions in contrast to the highly skewed J- and U-shaped distributions for the 10-point scale. The five-point scale data had significantly lower item means, floor, and ceiling effects. Regression analysis showed that the type of scale explained a significant component of the variation in both floor and ceiling effects.

Conclusion

The five-point scale performed better than the 10-point scale and is more suitable for assessing patient experiences. The revised PEQ will be used in Norwegian national surveys.

Introduction

What is new?

  • There is variation in the use of item scaling within questionnaires designed to measure patient experiences and satisfaction with care. Evidence supporting alternative scaling formats is scant.

  • This study found that a five-point scale with all-point-defined scale produced data of better quality with lower means, floor, and ceiling effects than a 10-point end-point-defined scale.

  • Questionnaire items designed to assess aspects of patient experiences and satisfaction with care should use five-point all-point-defined scales in preference to 10-point end-point-defined scales.

Self-completed questionnaires are widely used within health-related research for collecting information from patients and health professionals. The measurement of patient-reported outcomes, in particular, has seen a huge growth in the number of questionnaires as documented by several systematic reviews [1], [2], [3], [4]. The number of published articles reporting the application, development, or evaluation of questionnaires that are designed to assess patient experiences or satisfaction with health care is considerable [3], [5], [6], [7], [8].

There are a number of options for developing questionnaires in relation to questionnaire appearance, wording, and sequencing [9]. The evidence for questionnaire appearance has been described as scant, but evidence in relation to the structure of questions, including response categories, suggests that they can strongly influence data quality [9]. The ordering of categories, the number of categories, whether there is an odd or even number of categories, and how categories are labeled have all been found to influence results [9], [10].

Most questionnaires within the field of patient experiences and satisfaction have used items with all-point-defined scales [9] where each scale point has a descriptor [11], [12], [13], [14], [15], [16], [17], a well-known example being the five-point Likert scale. After a systematic review, it was found that patient satisfaction questionnaires include response scales with between 3 and 11 points, the mode being five points [5]. There is little evidence to support the use of these alternate response scales. Research, including randomized studies, has been recommended to determine which form of response scale produces data with the best quality. This includes missing data and floor and ceiling effects, the latter being an important consideration with the field of patient experiences where results are often highly skewed toward more positive levels of satisfaction [8], [12], [16], [18], [19].

Studies that have assessed different response scales for patient experience and satisfaction questionnaires have largely compared all-point-defined scales that differ in their use of descriptors or number of scale points [20]. Five-point scales were found to produce more variance than six-point scales in a study of patients receiving outpatient care in the United States [21]. Three all-point-defined scales, two with 5 points and one with 10 points, were compared in a study of inpatients in The Netherlands [22]. At the item level, a five-point scale that ranged from “dissatisfied” to “very satisfied” was considered optimal. More recently, the same group compared 5- and 10-point scales in a larger sample of hospital inpatients [20]. The former used “dissatisfied” to “very satisfied,” and the latter used “very poor” to “excellent” descriptors. The five-point scale produced less missing data, more variance, less skewed, and less kurtotic distributions. Finally, a study of patient satisfaction with general practice out-of-hours services compared three different five-point all-point-defined scales—a Likert scale of “very dissatisfied” to “very satisfied”, the same scale with smiley faces alongside, a scale of “very poor” to “excellent”—and a 10-point end-point-defined scale of “very dissatisfied” to “very satisfied [23].” The authors concluded that there were no significant differences in response rates or data quality for the different scales.

National surveys of patient experiences with health care have been conducted over the past 12 years within the Norwegian health care system [16], [17], [24], [25], [26]. Earlier surveys used 10-point end-point-defined scales or bipolar adjective scales, where only the scale end points have descriptors [16], [24], [25], [26]. However, this scale produced poor levels of data quality in pilot studies with psychiatric patients. Subsequent national surveys have used five-point all-point-defined scales that have produced data that was more normally distributed with lower floor and ceiling effects both for psychiatry and other groups of patients [17], [27], [28]. The finding that such scales produce better data quality with less of a skew toward positive experiences led to their adoption in a national survey of parent experiences of child inpatient care [17].

The Patient Experiences Questionnaire (PEQ) uses 10-point end-point-defined scales and has evidence for reliability and validity [24], [26]. The encouraging results after more recent Norwegian national surveys that used the five-point scale [17], [27], [28] have meant that this form of scaling has been considered for use with the PEQ. The work that follows presents a randomized comparison of the original version of the PEQ with 10-point scales with a version that uses a five-point all-point-defined scale.

Section snippets

Patient Experiences Questionnaire

The PEQ is a 24-item questionnaire that is designed for use with inpatients discharged from medical and surgical departments. The questionnaire has been used in several national surveys of Norwegian patients, and it has evidence for reliability and validity [24]. The content of the PEQ, including the scales, has been published [24]. It uses 10-point end-point-defined scales that follow the content of each item. For example, the item relating to whether the patient received enough information

Data collection

The five-point version of the PEQ was deemed acceptable to the 14 patients who were interviewed, and the consensus among the three interviewers was that patients preferred this scale to the 10-point scale when responding to the questions relating to patient experiences. Of the 1,000 patients mailed a questionnaire, 466 (46.6%) responded. The response rates for those who received the 5- and 10-point scaled questionnaires were 45.4% (n = 227) and 47.8% (n = 239), respectively, the difference not

Discussion

The present study sought to assess whether the data quality of an existing measure of patient experiences, the PEQ [24], could be improved through the adoption of a five-point all-point-defined response scale. Evidence from patient interviews and pilot surveys support the use of this scale, and there is good evidence for the data quality, reliability, and validity of other Norwegian PEQs that use this form of scaling [17], [27], [28]. The present study provides further evidence supporting the

References (40)

  • E. McColl et al.

    Design and use of questionnaires: a review of best practice applicable to surveys of health service staff and patients

    Health Technol Assess

    (2001)
  • Streiner DL, Norman GR. Health Measurement Scales; A Practical Guide to Their Development and Use. Oxford, UK: Oxford...
  • S. Grogan et al.

    Validation of a questionnaire measuring patient satisfaction with general practitioner services

    Qual Saf Health Care

    (2000)
  • C. Jenkinson et al.

    Patient's experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care

    Qual Saf Health Care

    (2002)
  • A.A. Hendriks et al.

    Reliability and validity of the Satisfaction with Hospital Care Questionnaire

    Int J Qual Health Care

    (2002)
  • J. Sweeney et al.

    Development of the Irish National Patient Perception of Quality of Care Survey

    Int J Qual Health Care

    (2003)
  • S.A. Margolis et al.

    Patient satisfaction with primary health care services in the United Arab Emirates

    Int J Qual Health Care

    (2003)
  • A.M. Garratt et al.

    The OutPatient Experiences Questionnaire (OPEQ): data quality, reliability, and validity in patients attending 52 Norwegian hospitals

    Qual Saf Health Care

    (2005)
  • A.M. Garratt et al.

    Parent experiences of paediatric care (PEPC) questionnaire: reliability and validity following a national survey

    Acta Paediatr

    (2007)
  • S. Bruster et al.

    National survey of hospital patients

    BMJ

    (1994)
  • Cited by (44)

    • Measuring patient satisfaction in acute care hospitals: nationwide monitoring in Switzerland

      2021, Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen
      Citation Excerpt :

      To deal with the observed ceiling effect of the previous version of the questionnaire, which had a 11-point scale, the pilot survey tested two additional scale responses: a 5-point verbal scale and a 7-point scale, both with only the extreme rates labeled. The inclusion of the additional scales was based on recommendations of related literature, which suggested that lower point scales tend to have a better performance to deal with ceiling effects [11,12]. Dimensional analysis tested the structure of the items for the three scales, as well as the adequacy of the questions.

    • First-contact care with a medical vs chiropractic provider after consultation with a swiss telemedicine provider: Comparison of outcomes, patient satisfaction, and health care costs in spinal, hip, and shoulder pain patients

      2015, Journal of Manipulative and Physiological Therapeutics
      Citation Excerpt :

      Work loss (absenteeism) was determined by asking patients to report on the number of days off work over the study period due to their complaint. Satisfaction with the care received and the result of care were measured using a 5-point Likert scale ranging from “very satisfied” to “very unsatisfied.”6 Variables referring to the use of health care services included providers consulted, treatments undergone, and diagnostic procedures received over the study period.

    • Polynucleotides HPT for Asian Skin Regeneration and Rejuvenation

      2024, Clinical, Cosmetic and Investigational Dermatology
    View all citing articles on Scopus
    View full text