Skip to main content
Log in

Evaluating Pharmacokinetic/Pharmacodynamic Models Using the Posterior Predictive Check

  • Published:
Journal of Pharmacokinetics and Pharmacodynamics Aims and scope Submit manuscript

Abstract

The posterior predictive check (PPC) is a model evaluation tool. It assigns a value (pPPC) to the probability that the value of a given statistic computed from data arising under an analysis model is as or more extreme than the value computed from the real data themselves. If this probability is too small, the analysis model is regarded as invalid for the given statistic. Properties of the PPC for pharmacokinetic (PK) and pharmacodynamic (PD) model evaluation are examined herein for a particularly simple simulation setting: extensive sampling of a single individual's data arising from simple PK/PD and error models. To test the performance characteristics of the PPC, repeatedly, “real” data are simulated and for a variety of statistics, the PPC is applied to an analysis model, which may (null hypothesis) or may not (alternative hypothesis) be identical to the simulation model. Five models are used here: (PK1) mono-exponential with proportional error, (PK2) biexponential with proportional error, (PK2ε) biexponential with additive error, (PD1) E max model with additive error under the logit transform, and (PD2) sigmoid E max model with additive error under the logit transform. Six simulation/analysis settings are studied. The first three, (PK1/PK1), (PK2/PK2), and (PD1/PD1) evaluate whether the PPC has appropriate type-I error level, whereas the second three (PK2/PK1), (PK2ε/PK2), and (PD2/PD1) evaluate whether the PPC has adequate power. For a set of 100 data sets simulated/analyzed under each model pair according to a stipulated extensive sampling design, the pPPC is computed for a number of statistics in three different ways (each way uses a different approximation to the posterior distribution on the model parameters). We find that in general; (i) The PPC is conservative under the null in the sense that for many statistics, prob(pPPC≤α)<α for small α. With respect to such statistics, this means that useful models will rarely be regarded incorrectly as invalid. A high correlation of a statistic with the parameter estimates obtained from the same data used to compute the statistic (a measure of statistical “sufficiency”) tends to identify the most conservative statistics. (ii) Power is not very great, at least for the alternative models we tested, and it is especially poor with “statistics” that are in part a function of parameters as well as data. Although there is a tendency for nonsufficient statistics (as we have measured this) to have greater power, this is by no means an infallible diagnostic. (iii) No clear advantage for one or another method of approximating the posterior distribution on model parameters is found.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

REFERENCES

  1. L. B. Sheiner and J.-L. Steimer. Pharmacokinetic pharmacodynamic modeling in drug development, Ann. Reυ. Pharmacol. Toxicol. 40:67–96 (2000).

    Google Scholar 

  2. Guidance for IndustryPopulation Pharmacokinetics (available at http:__www.fda.gov_ cder_guidance_index.htm), CDER, FDA, 1999.

  3. F. Mentré and M.-E. Ebelin. Validation of population pharmacokinetic/pharmacodyn-amic analyses: Review of proposed approaches. In The Population Approach: Measuring and Managing Variability in Response Concentration and Dose (COST B1), Office for official publications of the European Communities, Brussels, 1997, pp. 149–160.

    Google Scholar 

  4. R. Bruno, N. Vivier, J. C. Vergniol, S. L. De Phillips, G. Montay, and L. B. Sheiner. A population pharmacokinetic model for docetaxel (Taxotere): model building and validation. J. Pharmacokin. Biopharm. 24:153–172 (1996).

    Google Scholar 

  5. E. B. Roecker. Prediction error and its estimation for subset-selected models. Techno-metrics 33:459–468 (1991).

    Google Scholar 

  6. D. B. Rubin. Bayesian justifiable and relevant frequency calculations for the applied statistician. Ann. Statist. 12:1151–1172 (1984).

    Google Scholar 

  7. A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin. Bayesian Data Analysis, Chapman and Hall, London, 1995.

    Google Scholar 

  8. A. Gelman, F. Bios, and J. Jiang. Physiological pharmacokinetic analysis using population modeling and informative prior distribution, J. Am. Stat. Assoc. 91:1400–1412 (1996).

    Google Scholar 

  9. T. R. Belin and D. B. Rubin. The analysis of repeated-measures data on schizophrenic reaction times using mixture models. Statist. Med. 14:747–768 (1995).

    Google Scholar 

  10. A. Gelman, X.-L. Meng, and H. Stern. Posterior predictive assessment of model fitness via realized discrepancies, Statist. Sinica 6:733–807 (1996).

    Google Scholar 

  11. P. Girard, T. F. Blaschke, H. Kastrissios, and L. B. Sheiner. A Markov mixed effect regression model for drug compliance. Statist. Med. 17:2313–2333 (1998).

    Google Scholar 

  12. E. H. Cox, C. V.-Follet, A. E. Fuseau, S. Kenkare, S. L. Beal, and L. B. Sheiner. A population pharmacokinetic–pharmacodynamic analysis of repeated measures time-to-event pharmacodynamic responses: The Antiemetic Effect of Ondansetron. J. Pharmaco-kin. Biopharmaceut. 27:625–644 (1999).

    Google Scholar 

  13. E. O. George and G. S. Mudholkar. P values for two-sided tests. Biometrical J. 32:747–751 (1990).

    Google Scholar 

  14. S-plus Users Manual, Statistical Sciences, Inc., Seattle, WA (1993).

  15. G. E. P. Box and M. E. Muller. A note on the generation of random normal deviates. Ann. Math. Statist. 29:160–161 (1958).

    Google Scholar 

  16. P. Lewis, A. Goodman, and J. Miller. A pseudonormal number generator for the system 360. IBM Systems J. 8:135–146 (1969).

    Google Scholar 

  17. N. M. Laird and T. A. Louis. Empirical Bayes confidence intervals for a series of related experiments. Biometrics 45:481–495 (1989).

    Google Scholar 

  18. S. L. Beal and L. B. Sheiner. NONMEM Users' Guides, NONMEM Project Group, University of California, San Francisco, CA, 1996.

    Google Scholar 

  19. L. Breiman and J. H. Friedman. Estimating optimal transformation for multiple regression and correlation. J. Am. Statist. Assoc. 80:580–619 (1985).

    Google Scholar 

  20. W.-Y. Loh. Calibrating confidence coefficients. J. Am. Statist. Assoc. 82:155–162 (1987).

    Google Scholar 

  21. M. A. Newton and C. J. Geyer. Bootstrap recycling: A Monte Carlo alternative to the nested bootstrap. J. Am. Statist. Assoc. 89:905–912 (1994).

    Google Scholar 

  22. R. Beran. Prepivoting test statistics: A bootstrap view of asymptotic refinements. J. Am. Statist. Assoc. 83:687–697 (1988).

    Google Scholar 

  23. P. Hall and M. A. Martin. On bootstrap resampling and iteration. Biometrika 75:661–771 (1988).

    Google Scholar 

  24. D. V. Hinkley and S. Shi. Importance sampling and the nested bootstrap. Biometrika 76:435–446 (1989).

    Google Scholar 

Download references

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yano, Y., Beal, S.L. & Sheiner, L.B. Evaluating Pharmacokinetic/Pharmacodynamic Models Using the Posterior Predictive Check. J Pharmacokinet Pharmacodyn 28, 171–192 (2001). https://doi.org/10.1023/A:1011555016423

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1011555016423

Navigation