A rational framework for decision making by the National Institute For Clinical Excellence (NICE)

doi:10.1016/S0140-6736(02)09832-X

The Lancet

Volume 360, Issue 9334, 31 August 2002, Pages 711-715

https://doi.org/10.1016/S0140-6736(02)09832-X Get rights and content

Summary

Regulatory and reimbursement authorities face uncertain choices when considering the adoption of health-care technologies. In this Viewpoint, we present an analytic framework that separates the issue of whether a technology should be adopted on the basis of existing evidence from whether more research should be demanded to support future decisions. We show the application of this framework to the assessment of heath-care technologies using a published analysis of a new drug treatment for Alzheimer's disease. The results of the analysis show that the amount and type of evidence required to support the adoption of a health technology will differ substantially between technologies with different characteristics. Additionally, the analysis can be used to aid the efficient design of research. We discuss the implications of adoption of this new framework for regulatory and reimbursement decisions.

Section snippets

Randomised controlled trials as a source of evidence

The importance of the randomised controlled trial as a source of data for decision making is widely accepted. However, for many technologies, such data are few or non-existent. Indeed, no formal licensing process requiring trial data exists for new non-pharmaceutical interventions. As a result, many interventions provided by health services have never been assessed in trials. For example, in the recent NICE appraisal of prophylactic removal of wisdom teeth, only one completed randomised

Decisions

How should organisations such as NICE make decisions despite weak or non-existent evidence from randomised controlled trials? Reimbursement authorities face four possible choices with respect to technologies: adopt the technology on the basis of existing information, adopt now but demand further information to inform this choice in the future, reject on the basis of existing information, or reject and demand further research to inform this choice in the future. An explicit framework is needed

Implications of adopting the new framework

The framework raises a number of issues that will need to be resolved: whether bodies such as NICE have sufficient powers to sustain this form of decision making, and how to prevent disincentives when companies use the evidence generated by those whose product was the first to be commercially released.²² Also, in some circumstances, adoption of a cost-effective technology might have to be delayed until further research has been done. For example, reversal of an initial adoption decision may be

References (22)

Guidelines for the pharmaceutical industry on preparation of submissions to the Pharmaceutical Benefits Advisory Committee.
(1992)
Ontario Guidelines for Economic Analysis of Pharmaceutical Products.
(1994)
PC Langley
Formulary submission guidelines for Blue Cross and Blue Shield of Colorado and Nevada: structure, application and manufacturer responsibilities.
Pharmacoeconomics
(1999)
F Song et al.
The effectiveness and cost-effectiveness of prophylactic removal of wisdom teeth.
Health Technol Assess
(2000)
MF Drummond et al.
Economic analysis alongside clinical trials.
Int J Technol Assess Health Care
(1991)
MS McDonagh et al.
A rapid and systematic review of the clinical effectiveness of glycoprotein IIb/IIa antagonists in the medical management of unstable angina.
Health Technol Assess
(2000)
MJ Sculpher et al.
The iterative use of economic evaluation as part of the process of health technology assessment.
J Health Serv Res Policy
(1997)
K Claxton
The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies.
J Health Econ
(1999)
A Briggs et al.
An introduction to Markov modelling for economic evaluation.
Pharmacoeconomics
(1998)
PJ Neumann et al.
Cost-effectiveness of donepezil in the treatment of mild or moderate Alzheimer's disease.
Neurology
(1999)

A Clegg et al.

Clinical and cost-effectiveness of donepezil, rivastigmine and galantamine for Alzheimer's disease: a rapid and systematic review.

Health Technol Assess

(2000)

Cited by (381)

Cost-Effectiveness and Budget Impact of Future Developments With Whole-Genome Sequencing for Patients With Lung Cancer
2023, Value in Health
This study aimed to investigate the cost-effectiveness, budget impact (BI), and impact of uncertainty of future developments concerning whole-genome sequencing (WGS) as a clinical diagnostic test compared with standard of care (SoC) in patients with locally advanced and metastatic non–small cell lung cancer.
A total of 3 likely scenarios to take place within 5 years (according to experts) were simulated using a previously developed, peer reviewed, and published decision model. The scenarios concerned “WGS results used for treatment selection” (scenario 1), “WGS-based biomarker for immunotherapy” (scenario 2), and “off-label drug approval for WGS results” (scenario 3). Two diagnostic strategies of the original model, “SoC” and “WGS as a diagnostic test” (base model), were used to compare our scenarios with. Outcomes were reported for the base model, all scenarios separately, combined (combined unweighted), and weighted by likelihood (combined weighted). Cost-effectiveness, BI, and value of information analyses were performed for WGS compared with SoC.
Total costs and quality-adjusted life-years for SoC in metastatic non–small cell lung cancer were €149 698 and 1.235. Incremental outcomes of WGS were €1529/0.002(base model), −€222/0.020(scenario 1), −€2576/0.023(scenario 2), €388/0.024(scenario 3), −€5041/0.060(combined unweighted), and −€1715/0.029(combined weighted). The annual BI for adopting WGS for this population in The Netherlands ranged between €682 million (combined unweighted) and €714 million (base model). The consequences of uncertainty amounted to €3.4 million for all scenarios (combined weighted) and to €699 000 for the diagnostic yield of WGS alone (combined weighted).
Our findings suggest that it is likely for WGS to become cost-effective within the near future if it identifies more patients with actionable targets and show the impact of uncertainty regarding its diagnostic yield. Modeling future scenarios can be useful to consider early adoption of WGS while timely anticipating on unforeseen developments before final conclusions are reached.
Can polygenic risk scores contribute to cost-effective cancer screening? A systematic review
2022, Genetics in Medicine
Polygenic risk influences susceptibility to cancer. We assessed whether polygenic risk scores could be used in conjunction with other predictors of future disease status in cost-effective risk-stratified screening for cancer.
We undertook a systematic review of papers that evaluated the cost-effectiveness of screening interventions informed by polygenic risk scores compared with more conventional screening modalities. We included papers reporting cost-effectiveness outcomes with no restriction on type of cancer or form of polygenic risk modeled. We evaluated studies using the Quality of Health Economic Studies checklist.
A total of 10 studies were included in the review, which investigated 3 cancers: prostate (n = 5), colorectal (n = 3), and breast (n = 2). Of the 10 papers, 9 scored highly (score >75 on a 0-100 scale) when assessed using the quality checklist. Of the 10 studies, 8 concluded that polygenic risk-informed cancer screening was likely to be more cost-effective than alternatives.
Despite the positive conclusions of the included studies, it is unclear if polygenic risk stratification will contribute to cost-effective cancer screening given the absence of robust evidence on the costs of polygenic risk stratification, the effects of differential ancestry, potential downstream economic sequalae, and how large volumes of polygenic risk data would be collected and used.
How can early stage economic evaluation help guide research for future vaccines?
2022, Vaccine
Implementation Barriers to Value of Information Analysis in Health Technology Decision Making: Results From a Process Evaluation
2021, Value in Health
Citation Excerpt :
This work is placed in the context of the increasing theoretical literature on VOI analysis and the realization that VOI analysis is still underused in policy making.15,16 As Claxton et al1 pointed out, adoption of a decision-making framework that includes VOI analysis to assess further research has radical implications, including a shift toward increasing demands for further evidence to make a recommendation on technologies where there is substantial uncertainty and cost-effectiveness is borderline or for which there is a large eligible population of patients. Our work has shown that the adoption of this framework and the shift have not yet occurred in practice as was highlighted by other articles describing barriers to VOI use.17,29,47
Value of information (VOI) analysis can support health technology assessment decision making, but it is a long way from being standard use. The objective of this study was to understand barriers to the implementation of VOI analysis and propose actions to overcome these.
We performed a process evaluation of VOI analysis use within decision making on tomosynthesis versus digital mammography for use in the Dutch breast cancer population screening. Based on steering committee meeting attendance and regular meetings with analysts, we developed a list of barriers to VOI use, which were analyzed using an established diffusion model. We proposed actions to address these barriers. Barriers and actions were discussed and validated in a workshop with stakeholders representing patients, clinicians, regulators, policy advisors, researchers, and the industry.
Consensus was reached on groups of barriers, which included characteristics of VOI analysis itself, stakeholder’s attitudes, analysts’ and policy makers’ skills and knowledge, system readiness, and implementation in the organization. Observed barriers did not only pertain to VOI analysis itself but also to formulating the objective of the assessment, economic modeling, and broader aspects of uncertainty assessment. Actions to overcome these barriers related to organizational changes, knowledge transfer, cultural change, and tools.
This in-depth analysis of barriers to implementation of VOI analysis and resulting actions and tools may be useful to health technology assessment organizations that wish to implement VOI analysis in technology assessment and research prioritization. Further research should focus on application and evaluation of the proposed actions in real-world assessment processes.
Uncertainty tolerance among experts involved in drug reimbursement recommendations: Qualitative evidence from HTA committees in Canada and Poland
2021, Health Policy
Citation Excerpt :
The low quality of evidence creates uncertainty around the effectiveness of new drugs, but we have insufficient understanding of how uncertainty, in particular non-quantifiable uncertainty, is perceived by stakeholders and how it is or should be managed. In the context of HTA, we have long known that uncertainty presents a challenge to the immediate funding decisions (even when in some cases future evidence reduces uncertainty) [10,11]. Three revealed preferences studies measure the impact of a clinical uncertainty variable on committee recommendations [12–14] and two show it to be a significant predictor [12,13].
Drug reimbursement decisions often rely on health technology assessment (HTA). Increasingly, new drugs have limited clinical evidence and uncertain clinical benefit. Our goal was to describe how members of drug advisory committees and other stakeholders conceptualize and tolerate uncertainty and how they rationalize uncertainty tolerance.
Our triangulated parallel design applied two qualitative methods. We interviewed 31 members of drug advisory committees in Canada and Poland about their information needs and included hypothetical scenarios with uncertain clinical benefits. Respondents speculated about their likely reimbursement recommendation. We analyzed written recommendations of the pan Canadian Oncology Drug Review for drugs with uncertain benefit and compared initial recommendations to the responses from patient and clinician groups.
Uncertainty tolerance varied among committee members and across jurisdictions. In the scenario analysis, 7 Canadian and 11 Polish respondents leaned against recommending a hypothetical drug with uncertain clinical benefit, whereas 5 Canadian and 5 Polish respondents leaned in favour. Those against rationalized that uncertainty increases potential harm; those in favour rationalized that patients often have no alternatives. The document analysis revealed that patients had higher uncertainty tolerance in general.
Uncertainty tolerance varies among committee members and other stakeholders depending on their backgrounds and on the decision contexts. We argue that policy guidance around uncertainty management could improve the transparency and consistency of recommendations.
Cost-Effectiveness Analysis of Frailty Assessment in Older Patients Undergoing Coronary Artery Bypass Grafting Surgery
2020, Canadian Journal of Cardiology
In perioperative settings, frailty assessment has been shown to reduce mortality. This study examined the cost effectiveness of frailty assessment among patients aged 65 with coronary artery disease under consideration for coronary artery bypass grafting surgery.
A combined decision tree and Markov model was developed to estimate costs and quality-adjusted life years (QALYs) over a 21-year time horizon. Clinical parameters were obtained from published literature. Utilities were derived from the literature and the Canadian Community Health Survey. Costs were obtained from the Ontario fee schedule and published literature. Sensitivity and scenario analyses were conducted to assess the robustness of the results. Expected value of perfect information (EVPI) analysis was conducted to estimate the value of further research.
The frailty assessment initiative had a lower average cost than no frailty assessment ($19,567 compared with $20,062). QALYs with frailty assessment were 0.47 years more than with no frailty assessment. Thus, frailty assessment was dominant compared with no frailty assessment. Results were robust to changes in the input parameters. At a willingness to pay (WTP) threshold of $50,000/QALY, there was 100% probability of frailty assessment being cost-effective, and the EVPI per patient was $0. Scenario and sensitivity analysis showed frailty screening remained cost effective when changing the cohort average age, removing health benefits for nonfrail patients, and using subjective judgement to modify effectiveness parameters.
Frailty assessment may be good value for money. However, limited availability of geriatric consultation services, may hinder implementation. Thus, the estimated benefits of frailty screening may not be achievable in practice.
Il a été déterminé que l’évaluation de la fragilité permettait de réduire la mortalité en contexte périopératoire. Les auteurs du présent article ont examiné le rapport coût-efficacité de l’évaluation de la fragilité des patients âgés de 65 ans atteints d’une coronaropathie chez qui un pontage aortocoronarien était envisagé.
Les coûts et les années de vie ajustées en fonction de la qualité (AVAQ) ont été estimés sur un horizon de 21 ans au moyen d’un arbre décisionnel et d’un modèle de Markov. Les paramètres cliniques sont extraits d’études publiées, et les valeurs d’utilité sont dérivées d’études publiées et de l’Enquête sur la santé dans les collectivités canadiennes. Les coûts sont tirés du barème des frais et honoraires en vigueur en Ontario et d’études publiées. Des analyses de sensibilité et de scénarios ont été réalisées pour évaluer la robustesse des résultats. Enfin, une analyse de la valeur espérée de l’information parfaite (VEIP) a été effectuée pour estimer la valeur de recherches futures.
Le coût moyen de l’exécution d’une évaluation de la fragilité était inférieur à celui de la non-exécution d’une telle évaluation (19 567 $ comparativement à 20 062 $). Les AVAQ associées à l’évaluation de la fragilité s’établissaient à 0,47 année de plus que les AVAQ en l’absence d’évaluation. L’évaluation de la fragilité était donc dominante par rapport à l’absence d’évaluation. Les résultats se sont révélés robustes aux changements des paramètres d’entrée. À un seuil de volonté de payer de 50 000 $ par AVAQ, la probabilité que l’évaluation de la fragilité soit rentable était de 100 % et la VEIP par patient s’établissait à 0 $. Les analyses de scénarios et de sensibilité ont montré que l’évaluation de la fragilité demeure rentable même lorsqu’on change l’âge moyen de la cohorte, qu’on ne tient pas compte des bienfaits pour la santé des patients qui ne sont pas considérés comme étant fragiles et qu’on modifie de manière subjective les paramètres relatifs à l’efficacité.
L’évaluation de la fragilité pourrait permettre d’optimiser les ressources. Toutefois, l’offre restreinte de services de consultation en gériatrie pourrait faire obstacle à la mise en œuvre d’une telle initiative. Les avantages estimés d’une évaluation de la fragilité ne sont donc peut-être pas réalisables dans la pratique.

View all citing articles on Scopus

View full text

ViewpointA rational framework for decision making by the National Institute For Clinical Excellence (NICE)

Summary

Section snippets

Randomised controlled trials as a source of evidence

Decisions

Implications of adopting the new framework

Guidelines for the pharmaceutical industry on preparation of submissions to the Pharmaceutical Benefits Advisory Committee.

Ontario Guidelines for Economic Analysis of Pharmaceutical Products.

Formulary submission guidelines for Blue Cross and Blue Shield of Colorado and Nevada: structure, application and manufacturer responsibilities.

Pharmacoeconomics

The effectiveness and cost-effectiveness of prophylactic removal of wisdom teeth.

Health Technol Assess

Economic analysis alongside clinical trials.

Int J Technol Assess Health Care

A rapid and systematic review of the clinical effectiveness of glycoprotein IIb/IIa antagonists in the medical management of unstable angina.

Health Technol Assess

The iterative use of economic evaluation as part of the process of health technology assessment.

J Health Serv Res Policy

The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies.

J Health Econ

An introduction to Markov modelling for economic evaluation.

Pharmacoeconomics

Cost-effectiveness of donepezil in the treatment of mild or moderate Alzheimer's disease.

Neurology

Clinical and cost-effectiveness of donepezil, rivastigmine and galantamine for Alzheimer's disease: a rapid and systematic review.

Health Technol Assess

Viewpoint
A rational framework for decision making by the National Institute For Clinical Excellence (NICE)