Abstract
Evidence produced by multinational trial-based cost-effectiveness studies is often used to inform decisions concerning the adoption of new healthcare technologies. A key issue relating to the use of this type of evidence is the extent to which trial-wide economic results are applicable to every single country participating in the study. We consider what role cost-effectiveness analysis alongside multinational trials should have in assisting reimbursement decisions at jurisdiction and national levels. Using the proposed framework as a benchmark to evaluate their relative pros and cons, we then describe and review the statistical approaches used in the multinational trial-based costeffectiveness literature. The results of the review are used to define the desirable characteristics a statistical method for the analysis of data collected from different jurisdictions should have in order to be consistent with the proposed framework.
It is argued that Bayesian hierarchical models that use both patient- and country-level information are the most appropriate tool to analyse multinational trial-based cost-effectiveness data and facilitate the between-country generalizability assessment of the study findings. The merits of each approach are discussed, highlighting problems and limitations, in order to identify areas of future research.

Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.Notes
Here, we use the word ‘jurisdictions’ to mean circumstances where more than one decision maker exists within a country. A typical example is that of Canada where different provinces may impose different requirements for reimbursement submissions.
This is typically the case in multinational RCTs investigating the clinical effectiveness of drug therapies, where the treatment protocol is being rigidly followed in each treatment centre. There are a number of reasons why the absolute benefit of treatment may vary across countries participating in the trial. In particular, variation in baseline (i.e. control group) events may exist because of differences in demographics, case mix (severity), clinical practice and resource constraints,[8] leading to between-location variability in the observed clinical effectiveness and resource use. In principle, such differences may also generate important variability in the relative treatment effect of the new intervention relative to control. Although patients treated within the same hospital may share treatment protocols, these could differ between treatment centres and countries, leading to potential between-country variation in clinical outcome[9,10] and use of healthcare resources. Similarly, the availability of specific resources (e.g. beds in high-dependency units, outpatient facilities, etc.) will influence the length of stay in hospital, and possibly the clinical outcome.[11] Finally, the relative costs (or relative pricesfftariffs) of healthcare resources at the country level could influence the way these factors are combined in the process of care.[12]
Methods to transfer the study results to countries beyond the trial participants are outside the scope of this article, and although some of the statistical techniques discussed herein might prove to be useful to assess such a type of transferability, this is often more appropriately done using decision models.[2]
Researchers need to be aware of the role and limitations of the alternative analytical methods to conduct multinational assessment of cost-effectiveness data in order to use the most appropriate vehicle to fulfill the information requirements relevant for jurisdiction-specific decision-making activity. At the same time, decision makers need to be aware of the limitations of the simpler approaches to trial-based CEA, to be able to interpret the information provided to them by the research community.
In some RCTs, patients receive treatment from several healthcare professionals (at once or at different points in time); similarly, patients may attend different treatment centres during the course of their therapy. This feature of the data creates particular non-strictly hierarchical structures, known as multiple membership or cross-classified. Variability by location in the results of the analysis can be ascribed to one or more of these centres or healthcare professionals, and multilevel models can be used to analyse this type of data structure to quantify the extent of impact of contextual factors, such as treatment centre or healthcare professional.[16]
References
Drummond MF, Pang F. Transferability of economic evaluation results. In: Drummond MF, McGuire AL, editors. Economic evaluation of health care: merging theory with practice. Oxford: Oxford University Press, 2001
Sculpher MJ, Pang FS, Manca A, et al. Generalisability in economic evaluation studies in health care: a review and case studies. Health Technol Assess 2004; 8 (49): 1–206
Goeree R, Burke B, Manca A, et al. Generalizability of economic evaluations: using results from other geographic areas or from multinational trials to help inform health care decision making in Canada. Report to the CCOHTAHTA Capacity Building Grants Program (Grant N. 67). Toronto: Canadian Coordinating Office for Health Technology Assessment, 2005
Gallo PP. Center-weighting issues in multicenter clinical trials. J Biopharm Stat 2000; 10 (2): 145–63
Huster WJ, Louv WC. Demonstration of the reproducibility of treatment efficacy from a single multicenter trial. J Biopharm Stat 1992; 2 (2): 219–38
Patel HI. Robust analysis of a mixed-effect model for a multicenter clinical trial. J Biopharm Stat 2002; 12 (1): 21–37
Chen L, Tamura RN. Analysis of homogeneity of treatment effect in adaptive multicenter clinical trials. J Biopharm Stat 1998; 8 (1): 55–67
O’Brien BJ. A tale of two (or more) cities: geographic transferability of pharmacoeconomic data. Am J Manag Care 1997; 3: S33–9
Localio AR, Berlin JA, Ten Have TR. Confounding due to cluster in multicenter studies: causes and cures. Health Serv Outcomes Res Methodol 2002; 3 (3-4): 195–210
Localio AR, Berlin JA, Ten Have TR, et al. Adjustment for center in multicenter studies: an overview. Ann Int Med 2001; 135: 112–23
Bourne RB, DeBoer D, Hawker G, et al. Total hip and knee replacement. In: Tu JV, Pinfold SP, McColgan P, et al.,editors. Access to health services in Ontario. Toronto(ON): Institute for Clinical Evaluative Sciences, 2005: 91–118
Raikou M, Briggs A, Gray A, et al. Centre-specific or average unit costs in multi-centre studies: some theory and simulation. Health Econ 2000; 9: 191–8
Drummond MF, Manca A, Sculpher MJ. Increasing the generalisability of economic evaluations: recommendations for the design, analysis and reporting of studies. Int J Technol Assess Health Care 2005; 21 (2): 165–71
Claxton K, Sculpher MJ, Drummond MF. A rational framework for decision making by the National Institute for Clinical Excellence. Lancet 2002; 360: 711–5
Sculpher MJ, Claxton K, Akerhurst R. It’s just evaluation for decision making: recent developments in, and challenges for, cost-effectiveness research. In: Smith PC, Ginnelly L, Sculpher MJ, editors. Health policy and economics: opportunities and challenges. Maidenhead: Oxford University Press, 2005: 8–41
Browne WJ, Goldstein H, Rasbash J. Multiple membership multiple classification models. Stat Modelling 2001; 1: 103–24
Manca A, Rice N, Sculpher MJ, et al. Assessing generalisability by location in trial-based cost-effectiveness analysis: the use of multilevel models. Health Econ 2005; 14 (5): 471–85
Roberts C. The implication of variation in outcome between health care professionals for the design and analysis of randomised controlled trials. Stat Med 1999; 18: 2605–15
Roberts C, Roberts SA. Design and analysis of clinical trials with clustering effects due to treatments. Clin Trials 2005; 2: 152–62
Badia X, Roset M, Herdman M, et al. A comparison of United Kingdom and Spanish general population time trade-off values for EQ-5D health states. Med Decis Making 2001; 21 (7): 7–16
Greiner W, Schoffski O, Graf VD, et al. The transferability of international economic health results to national study questions. Health Econ Prevent Care 2000; 1: 94–102
Johnson JA, Luo N, Shaw JW, et al. Valuations of EQ-5D health states: are the United States and United Kingdom different? Med Care 2005; 43 (3): 221–8
Johnson JA, Ohinmaa A, Murti B, et al. Comparison of Finnish and US-based visual analog scale valuations of the EQ-5D measure. Med Decis Making 2000; 20 (3): 281–9
Briggs AH. Handling uncertainty in economic evaluation and presenting the results. In: Drummond MF, McGuire A, editors. Economic evaluation in health care: merging theory with practice. Oxford: Oxford University Press, 2001
Hoch JS, Briggs AH, Willan AR. Something old, something new, something borrowed, something BLUE: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ 2002; 11 (5): 415–30
Willan AR, Briggs AH, Hoch JS. Regression methods for covariate adjustment and subgroup analysis for non-censored cost-effectiveness data. Health Econ 2004; 13: 461–75
Greene WH. Econometric analysis. 3rd ed. Upper Saddle River (NJ): Prentice-Hall, Inc, 1997
Barber JA, Thompson SG. Analysis and interpretation of cost data in randomised controlled trials: review of published studies. BMJ 1998; 317: 1195–200
Manning WG, Basu A, Mullahy J. Generalized modeling approaches to risk adjustment of skewed outcomes data. J Health Econ 2005; 24 (3): 465–88
Manning WG, Mullahy J. Estimating log models: to transform or not to transform? J Health Econ 2001; 20 (4): 461–94
Snijders TAB, Bosker RJ. Multilevel analysis: an introduction to basic and advanced multilevel modeling. London: SAGE Publications Ltd, 1999
Rice N, Leyland A. Multilevel models: applications to health data. J Health Serv Res Policy 1996; 1: 154–64
Gilks WR, Richardson S, Spiegelhalter DJ. Markov chain Monte Carlo in practice. London: Chapman and Hall, 1996
Willan AR, Pinto EM, O’Brien BJ, et al. Country specific cost comparisons from multinational clinical trials using empirical Bayesian shrinkage estimation: the Canadian ASSENT-3 economic analysis. Health Econ 2005; 14: 327–38
Berlin JA, Santanna J, Schmid CH, et al., Anti-Lymphocyte Antibody Induction Therapy Study Group. Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head. Stat Med 2002; 21 (3): 371–87
Willke RJ, Glick HA, Polsky D, et al. Estimating countryspecific cost-effectiveness from multinational clinical trials. Health Econ 1998; 7: 481–93
Rutten-van MÖlken MP, van Doorslaer EKA, Till MD. Cost-effectiveness analysis of formoterol versus salmeterol in patients with asthma. Pharmacoeconomics 1998; 14 (6): 671–84
Koopmanschap MA, Touw KCR, Rutten FFH. Analysis of costs and cost-effectiveness in multinational trials. Health Policy 2001; 58 (2): 175–86
Carey K. A multilevel modelling approach to analysis of patient costs under managed care. Health Econ 2000; 9 (5): 435–46
Carey K. Hospital length of stay and costs: a multilevel modelling analysis. Health Serv Outcomes Res Methodol 2002; 3: 41–56
Coyle D, Drummond MF. Analyzing differences in the costs of treatment across centers within economic evaluations. Int J Technol Assess Health Care 2001; 17 (2): 155–63
Cook JR, Drummond MF, Glick H, et al. Assessing the appropriateness of combining economic data from multinational clinical trials. Stat Med 2003; 22 (12): 1955–76
Grieve R, Nixon R, Thompson SG, et al. Using multilevel models for assessing the variability of multinational resource use and cost data. Health Econ 2005; 14 (2): 185–96
Nixon RM, Thompson SG. Methods for incorporating covariate adjustment, subgroup analysis and between centre differences into cost-effectiveness analysis. Health Econ 2005; 14 (12): 1217–29
Manca A, Lambert PC, Sculpher MJ, et al. Cost effectiveness analysis using data from multinational trials: the use of bivariate hierarchical modelling. MedDec Making 2007; 27 (4): 471–90
Thompson SG, Nixon R, Grieve R. Addressing the issues that arise in multicentre cost data, with application to a multinational study. J Health Econ 2006; 25 (6): 1015–28
Assmann SF, Pocock SJ, Enos LE, et al. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000 Mar 25; 355 (9209): 1064–9
Caro JJ, Huybrechts KF, De Backer G, et al. Are the WOSCOPS clinical and economic findings generalizable to other populations? A case study for Belgium. Acta Cardiol 2000; 55 (4): 239–46
Drummond MF. Comparing cost-effectiveness across countries: the model of acid-related disease. Pharmacoeconomics 1994; 5 (S3): 60–7
Sculpher MJ, Poole L, Cleland J, et al. Low doses vs high doses of the angiotensin converting-enzyme inhibitor lisinopril in chronic heart failure: a cost-effectiveness analysis based on the Assessment of Treatment with Lisinopril and Survival(ATLAS) study. Eur J Heart Fail 2000; 2: 447–54
Schwartz JS, Wang YR, Cleland J, et al. High- versus low dose angiotensin converting enzyme inhibitor therapy in the treatment of heart failure: an economic analysis of the Assessment of Treatment With Lisinopril and Survival (ATLAS) Trial. Am J Manag Care 2003; 9: 417–24
Ess SM, Luscher TF, Szucs TD. High dose lisinopril in heart failure: economic considerations. Cardiovasc Drugs Ther 2002; 16 (4): 365–71
Scalone L, Mantovani LG. Economic evaluation of high vs low dosage lisinopril in patients with chronic heart failure. Pharmacoeconomics Ital Res Articles 2002; 4 (2): 45–55
Brown R, Henderson RA, Koster D, et al. Cost effectiveness of eptifibatide in acute coronary syndromes: an economic analysis of Western European patients enrolled in the PURSUIT trial. Eur Heart J 2002; 23 (1): 50–8
Brown R, Armstrong PW. Cost effectiveness in Canada of eptifibatide treatment for acute coronary syndrome patients using PURSUIT subgroup analysis. Can J Cardiol 2003; 19 (2): 161–6
Henderson RA, Brown R. The costs of routine eptifibatide use in acute coronary syndromes in Western Europe: an economic substudy of the PURSUIT trial. Eur Heart J Suppl 1999; 1 (N): N35–41
Scott WG, Scott HM. Heart failure: a decision analytic analysis of New Zealand data using the published results of the SOLVD treatment trial. Studies of Left Ventricular Dysfunction. Pharmacoeconomics 1996; 9 (2): 156–67
Butler JR, Fletcher PJ. A cost-effectiveness analysis of enalapril maleate in the management of congestive heart failure in Australia. Aust N Z J Med 1996; 26 (1): 89–95
Szucs TD, Goedde M, Berger K, et al. Economic benefits of ACE inhibitors in the treatment of chronic heart failure in Switzerland. Schweiz Med Wochenschr 1997; 127 (29-30): 1234–41
Mason J, Young P, Freemantle N, et al. Safety and costs of initiating angiotensin converting enzyme inhibitors for heart failure in primary care: analysis of individual patient data from studies of left ventricular dysfunction. BMJ 2000; 321 (7269): 1113–6
Glick H, Cook J, Kinosian B, et al. Costs and effects of enalapril therapy in patients with symptomatic heart failure: an economic analysis of the Studies of Left Ventricular Dysfunction (SOLVD) treatment trial. J Card Fail 1995; 1: 371–80
Reed SD, Anstrom KJ, Bakhai A, et al. Conducting economic evaluations alongside multinational clinical trials: toward a research consensus. AmHeart J 2005; 149 (3): 434–43
Bjorholt I, Andersson FL, Kahan T, et al. The cost-effectiveness of ramipril in the treatment of patients at high risk of cardiovascular events: a Swedish sub-study to the HOPE study. J Int Med 2002 June; 251 (6): 508–17
Liljas B, Stahl E, Pauwels RA. Cost effectiveness analysis of a dry powder inhaler (Turbuhaler) versus a pressurised metered dose inhaler in patients with asthma. Pharmacoeconomics 1997; 12 (2): 267–77
Mark DB, Harrington RA, Lincoff AM, et al. Cost-effectiveness of platelet glycoprotein IIb/IIIa inhibition with eptifibatide in patients with non-ST-elevation acute coronary syndromes. Circulation 2000; 101 (4): 366–71
Baltagi BH. Econometric analysis of panel data. Chichester: John Wiley & Sons, Ltd, 2001
Manca A, Willan AR. ‘Lost in translation’: accounting for between-country differences in the analysis of multinational cost effectiveness data. Pharmacoeconomics 2006; 24 (11): 1101–19
Fox J. Applied regression analysis, linear models, and related methods. Thousand Oaks (CA): SAGE Publications, Inc., 1997
Armitage P, Berry G, Matthews JNS. Statistical methods in medical research. 4th ed. Oxford: Blackwell Science, 2002
Saunders M, Dische S, Barrett A, et al. Continuous, hyperfractionated, accelerated radiotherapy (CHART) versus conventional radiotherapy in non-small cell lung cancer: mature data from the randomised multicentre trial. Radiother Oncol 1999 Aug; 52 (2): 137–48
Sutton AJ, Abrams KR, Jones DR, et al. Methods for metaanalysis in medical research. Chichester: John Wiley & Sons, 2000
Manca A, Lambert PC, Sculpher MJ, et al. Cost effectiveness analysis using data from multinational trials: the use of bivariate hierarchical modelling. Med Decis Making 2007; 27 (4): 471–90
Willan AR, Kowgier ME. Cost-effectiveness analysis of a multinational RCT with a binary measure of effectiveness and an interacting covariate. Health Econ 2008; 17 (7): 777–91
Gelman A, Carlin JB, Stern HS, et al. Bayesian data analysis. New York: Chapman & Hall/CRC, 2004
Draper D, Hodges JS, Mallows CL, et al. Exchangeability and data analysis. J R Stat Soc Ser A 1993; 156 (1): 9–37
Glick H, Doshi JA, Sonnad SS, et al. Economic evaluation in clinical trials. Oxford: Oxford University Press, 2007
Grieve R, Nixon R, Thompson SG, et al. Multilevel models for estimating incremental net benefits in multinational studies. Health Econ 2007; 16 (8): 815–26
Schulman K, Burke J, Drummond MF, et al. Resource costing for multinational neurologic clinical trials: methods and results. Health Econ 1998; 7: 629–39
Wordsworth S, Ludbrook A. Comparing costing results in across countries economic evaluations: the use of technology specific purchasing parities. Health Econ Lett 2005; 14 (1): 93–9
Wordsworth S, Ludbrook A, Caskey F, et al. Collecting unit cost data in multicentre studies: creating comparable methods. Eur J Health Econ 2005; 6 (1): 38–44
Gauthier A, Manca A, Anton S. Bayesian modelling of healthcare resource use in multinational randomized clinical trials. Pharmacoeconomics 2009; 27 (12): 1017–29
Adam T, Koopmanschap MA. Cost-effectiveness analysis: can we reduce variability in costing methods? Int J Technol Assess Health Care 2003; 19 (2): 407–20
Reed SD, Friedman JY, Gnanasakthy A, et al. Comparison of hospital costing methods in an economic evaluation of a multinational clinical trial. Int J Technol Assess Health Care 2003; 19 (2): 396–406
Oostenbrink JB, Koopmanschap MA, Rutten FFH. Standardisation of costs: the Dutch manual for costing in economic evaluations. Pharmacoeconomics 2002; 20 (7): 443–54
Graves N, Walker D, Raine R, et al. Cost data for individual patients included in clinical studies: no amount of statistical analysis can compensate for inadequate costing methods. Health Econ 2002; 11: 735–9
Acknowledgements
This article was conceived and written while A. Manca was the recipient of a Wellcome Trust-funded post-doctoral Training Fellowship in Health Services Research (grant number GR071304MA). M.J. Sculpher was Public Health Career Scientist funded by the UK NHS R&D Programme. Work by AM and MS on generalizability in economic evaluation was supported by the NHS Health Technology Programme in the UK. R. Goeree was funded by a Health Technology Assessment and Economic Evaluation research grant from the Ontario Ministry of Health and Long Term Care (♯06129). The views and opinions expressed herein are those of the authors and do not necessarily reflect those of the funding institutions.
The authors have no conflicts of interest that are directly relevant to the content of this article.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Manca, A., Sculpher, M.J. & Goeree, R. The Analysis of Multinational Cost-Effectiveness Data for Reimbursement Decisions. Pharmacoeconomics 28, 1079–1096 (2010). https://doi.org/10.2165/11537760-000000000-00000
Published:
Issue Date:
DOI: https://doi.org/10.2165/11537760-000000000-00000