Dietary assessment in the California Teachers Study: reproducibility and validity

Abstract

Objective

To evaluate the reproducibility and validity of the food-frequency questionnaire (FFQ) used in the California Teachers Study (CTS) cohort and to use this data to quantify the effects of correcting nutrient-breast cancer relative risks for measurement error.

Methods

One hundred and ninety five CTS cohort members participated in a 10-month dietary validation study that included four 24-h dietary recalls and pre- and post-study FFQs. Shrout–Fleiss intraclass correlations for reproducibility were computed. Under several standard assumptions concerning the correlations of errors in the FFQs and 24-h recalls, we calculated energy-adjusted deattenuated Pearson correlations for validity and tested for differences in validity according to a number of demographic and other risk factors. For each nutrient, we compared the performance of the FFQ versus the 24-h recalls, estimating the number of days of recalls that give equivalent information about true intake as does a single FFQ. Finally, the effects of adjustment for measurement error on risk estimates were evaluated in 44,423 postmenopausal cohort members, 1,544 of whom developed breast cancer during seven years of follow-up. Relative risks (RR) and confidence intervals (CI) were calculated using Cox proportional hazards with and without correction for measurement error.

Results

Reproducibility correlations for the nutrients ranged from 0.60 to 0.87. With a few exceptions, validity correlations were reasonably high (range: 0.55–0.85), including r = 0.74 for alcohol. Performance of the FFQ differed by age for percent of calories from fat and by body mass index and hormone therapy use for alcohol consumption. For most nutrients examined, our FFQ is comparable to two to six recalls for each subject in capturing true intake. In the measurement error-adjusted risk analyses, corrected RRs were within 13% of uncorrected values for all nutrients examined except for linoleic acid. For alcohol consumption the corrected RR (per 20 g/1,000 kcal/d) was 1.36 (95% CI: 1.03–1.51) compared to the uncorrected estimate of 1.25 (95% CI: 1.10–1.42).

Conclusion

The FFQ dietary assessment used in the CTS is reproducible and valid for all nutrients except the unsaturated fatty acids. Correcting relative risk estimates for measurement error resulted in relatively small changes in the associations between the majority of nutrients and the risk of postmenopausal breast cancer.

Appendix

Estimation of the number of days, n, of 24-h recalls a FFQ report is worth

We define n by equating the correlation, r, between the true nutrient intake and the FFQ with the correlation between true nutrient intake and n 24-h recalls (which increases as n increases). Note that this is equivalent to equating Var(True Intake|FFQ value) to Var(True Intake|n 24-h recalls) since 1 − r² is equal to Var(True|Estimated)/Var(True), i.e., it is the “unexplained” portion of the variance of True not captured by the estimate. If we assume that all variables are distributed normally then we can compute both Var(True Intake|FFQ value) as well as Var(True Intake|n 24-h recalls) by a standard variance components analysis.

Let X_i be true intake of the nutrient, then by fitting a random intercept model (as in SAS proc mixed) to the four recalls (R_ij, j = 1,4, i = 1, N) per subject, then R_ij is estimated by:

$$ {\text{R}}_{{{\text{ij}}}} = {\text{X}}_{{\text{i}}} + {\varepsilon}_{{{\text{ij}}}} $$

where ε_ij is the random error in our measurement. We can estimate Var(X) as the variance of the random intercept and Var(ε) as the residual variance (i.e., the variance of one 24-h recall given true intake). By an elementary application of conditional probability for normal random variables, and defining $\ifmmode\expandafter\bar\else\expandafter\=\fi{R}_{i}$ = R_ij/n, we have

$$ \begin{aligned}{} & {\text{Var}}({\text{X $|$ }}\ifmmode\expandafter\bar\else\expandafter\=\fi{R}_{i} {\text{) = Var}}({\text{X}}) - {\text{Cov}}({\text{X}},{\text{ }}\ifmmode\expandafter\bar\else\expandafter\=\fi{R}_{i} {\text{)}}^{{\text{2}}} {\text{/Var(}}\ifmmode\expandafter\bar\else\expandafter\=\fi{R}_{i} {\text{)}} \\ & \quad \quad \quad \quad {\text{ = Var(X)}} - {\text{Var(X)}}^{{\text{2}}} {\text{/(Var(X) + Var(}}\varepsilon {\text{)/n)}} \\ \end{aligned} $$

Note also that by fitting a second-random intercept model

$$ {\text{R}}_{{{\text{ij}}}} = {\text{a}} + {\text{bFFQ}}_{{\text{i}}} + {\text{W}}_{{\text{i}}} + \varepsilon _{{{\text{ij}}}} $$

and equating X_i = a + bFFQ_i + W_i (so that the random intercept W is the portion of true X unexplained by the regression on the FFQ value), we can estimate Var(X|FFQ) ≡ Var(W) as the variance of the random intercept term. Thus to estimate the number of days of 24-h recalls that one FFQ is “worth” we equate Var(W) = Var(X|$\ifmmode\expandafter\bar\else\expandafter\=\fi{R}_{i}$) = Var(X) − Var(X)²/[Var(X) + Var(ε)/n] and solve for n as:

$$ {\text{n}} = {\text{Var}}(\varepsilon ){\text{ }}({\text{Var}}({\text{X}}) - {\text{Var}}({\text{W}}))/({\text{Var}}({\text{X}})*{\text{Var}}({\text{W}})) $$

Since the square of the corrected correlation between FFQ and the 24-h recalls can be written as:

$$ {\text{r}}^{{\text{2}}} = ({\text{Var}}({\text{X}}) - {\text{Var}}({\text{W}}))/{\text{Var}}({\text{X}}),\,\,{\text{so that Var}}({\text{W}}) = (1 - {\text{r}}^{{\text{2}}} ){\text{Var}}({\text{X}}) $$

(this follows since X − W is the portion of X that is “explained” by the FFQ and r² is always the ratio of the variance of the “explained” portion of X to the total variance of X)

we have

$$ {\text{n}} = {\text{r}}^{{\text{2}}} {\text{Var}}(\varepsilon )/(1 - {\text{r}}^{{\text{2}}} ){\text{Var}}({\text{X}}) $$

which is identical to Eq. 1.