A simple imputation algorithm reduced missing data in SF-12 health surveys

Abstract

Objective

The SF-12 Health Survey is a 12-item questionnaire that yields two summary scores (physical and mental health). Neither score can be computed when an item is missing. We explored imputation methods for missing scores for this instrument.

Study design and setting

Using data from a population-based survey, we tested several ways of imputing simulated missing data.

Results

Among 1250 participants, 118 (9.6%) had at last one missing SF-12 item. Missing data were more common among women, older respondents, non-Swiss nationals, and health service users. Among the 1132 respondents with complete data, replacement of any item with the mean population item weight yielded good results: the mean correlation between imputed and true score was 0.979 for both the physical and mental score. Results remained satisfactory when up to three of the six key items for each score (items that contribute predominantly to a given score), and any number of non-key items, were replaced by the mean. Application of this imputation algorithm to the original survey reduced the proportion of missing scores to <1%. Respondents with incomplete surveys, hence imputed scores, had lower scores than respondents with complete data (physical score: 44.9 vs. 49.8, p < 0.001, mental score: 44.4 vs. 46.3, p = 0.064).

Conclusions

A simple imputation algorithm can substantially reduce the proportion of missing scores for the SF-12 health survey, and consequently reduce non-response bias.

Introduction

Valid research requires complete data. Missing values reduce statistical power, and more importantly, may cause selection bias. Many studies have documented differences between respondents and non-respondents [1], [2], [3], [4], [5], or between early and late respondents [5], [6], [7], in health research. However, even among study participants, substantial proportions of some variables may have missing values. Possible bias due to partial survey response has attracted only limited attention [7], [8], [9].

Incomplete data cause particular concern when multiple data elements are combined to form a single variable, such as a composite score, a clinical prediction rule, or a multi-item psychometric scale. Without imputation, a single bit of missing information may cause the composite score to be missing. Obviously, the more items are combined, the greater the problem: if the probability of a missing value was 2% per item in a composite score, and if missing item probabilities were mutually independent, the probability of a missing score value would be 10% for a 5-item score, 18% for a 10-item score, and 33% for a 20-item score.

Imputation rules alleviate this problem by substituting an acceptable value for the missing element, hence salvaging useful information available in non-missing items. When the composite score is essentially a sum of similar parts, a common imputation rule is “if up to half of the items are missing, replace missing values with mean of available items; if more than half are missing, declare score as missing.” In theory, replacement with the respondent's mean is appropriate only for strictly parallel tests, that is, when all items have the same mean, variance, and correlation with the underlying latent variable [10]. In practice, this rule is often used even when these conditions do not apply, and with good results as long as missing values are fairly rare. For instance, this imputation method is recommended for the SF-36 health survey, which consists of scales based on items that have the same number of response options, but not necessarily the same distribution [11].

When the composite score is derived through a more complex formula, or when items are not identically distributed, replacement with the mean of available items will not work. Often, no imputation rule is available. This is the case for the mental and physical component summary scores (MCS and PCS) based on the SF-36 [12] and the SF-12 [13], [14] health surveys. Each summary score is a sum of 36 or 12 item weights. As no imputation rules exist, a single missing item value will cause missing values for both summary scores. As a result, the SF-12 summary scores have rates of missing data among respondents that are frequently around 10%, and sometimes exceed 25% [15], [16], [17], [18], [19], [20], [21]. Corresponding figures are necessarily higher for the SF-36 summary scores. This issue is of concern to anyone who considers using these popular instruments.

In this article, we examine selection bias due to incomplete responses to an SF-12 survey, explore several ways of imputing summary scores when one or more items have a missing value, and propose a simple and effective imputation algorithm.