Multiple correlations and bonferroni’s correction

Abstract

Correlation coefficients between biological measurements and clinical scales are often calculated in psychiatric research. Calculating numerous correlations increases the risk of a type I error, i.e., to erroneously conclude the presence of a significant correlation. To avoid this, the level of statistical significance of correlation coefficients should be adjusted. Threshold levels of significance for correlation coefficients were adjusted for multiple comparisons in a set of k correlation coefficients (k = 1, 5, 10, 20, 50, 100) by Bonferroni’s correction. Significant correlation coefficients were then calculated according to sample size. The change in the threshold values of significance is larger when the number of correlations goes from 1 to 5 than when it goes from 50 to 100. A correlation coefficient, statistically significant at 5% when calculated alone, can be under the threshold level of significance when calculated even among a few other coefficients. Focusing on the most relevant variables or the use of multivariate statistics is advocated.

Introduction

In psychiatry, it is frequent to search for relationships between biological measurements, for example hormones, neurotransmitters levels, or enzymatic activity, and quantitative measurements of clinical symptoms or personality traits, often assessed by diagnostic instruments with several subscales. Correlation coefficients are often used for these calculations; however, performing multiple statistical tests is not without pitfalls; when many correlation coefficients are calculated from a given set of data, the risk of obtaining significant results by chance only is increased.

Correlation coefficients are generally tested against the null hypothesis that the correlation is equal to 0. The α-level of the test or the probability of a type I error (the probability of obtaining by chance a correlation differing from 0 when there is no true relationship) is generally set at 5%. In a set of k tests, the overall risk of a type I error, or the probability of at least one false rejection when the null hypothesis of all correlations being equal to 0 is true, becomes higher than 5%. This overall α-level for a set of k tests is defined by an upper bound determined by Bonferroni’s inequality Cupples et al 1984, Morrison 1976:

$$\text{overall-}\text{α≤k·α′}$$

where α′ is the level of type I error of each single test, most often 5%.

Thus practically, when looking for significant correlations, to keep the overall-α at 5%, the α′-level of each correlation must be divided by k(Meinert 1986). This rule was applied in a recent paper on the relationships between benzodiazepine receptor binding and severity of schizophrenia (Busatto et al 1997). Correlations were sought between benzodiazepine receptor binding in different cerebral regions and the types of schizophrenic symptoms (positive or negative). Two of six relationships were significant, but the authors underlined that these correlations could not be considered significant when Bonferroni’s correction for multiple comparisons was applied. This comment is particularly relevant to exploratory research protocols. There, a great number of correlation coefficients is usually calculated, and the risk of a type I error is large. At the opposite, protocols testing a definite hypothesis where the size of the relationship between variables is the main interest are more subject to a type II error, i.e., the inability to conclude a relationship when it truly exists. Here, we provide a graphical tool to estimate the variation of the threshold for significant values of correlation coefficients when numerous correlation coefficients are calculated.