A framework for the evaluation of new interventional procedures

Abstract

Objectives

The introduction of new interventional procedures is less regulated than for other health technologies such as pharmaceuticals. Decisions are often taken on evidence of efficacy and short-term safety from small-scale usually observational studies. This reflects the particular challenges of evaluating interventional procedures – the extra facets of skill and training and the difficulty defining a ‘new’ technology. Currently, there is no framework to evaluate new interventional procedures before they become available in clinical practice as opposed to new pharmaceuticals. This paper proposes a framework to guide the evaluation of a new interventional procedure.

Proposed framework

A framework was developed consisting of a four-stage progressive evaluation for a new interventional procedure: Stage 1: Development; Stage 2: Efficacy and short-term safety; Stage 3: Effectiveness and cost-effectiveness; and Stage 4: Implementation. The framework also suggests the types of studies or data collection methods that can be used to satisfy each stage.

Conclusions

This paper makes a first step on a framework for generating evidence on new interventional procedures. The difficulties and limitations of applying such a framework are discussed.

Introduction

The introduction of new technologies into health care should reflect a judgement that benefits outweigh any harm. Those making such decisions should be guided by evidence that is both reliable and timely and thereby avoid erroneous decisions that put patients at unnecessary risk or delay the introduction of helpful treatments. Interventional procedures are health technologies that can be used ‘for diagnosis or treatment involving an incision, puncture, entry into a body cavity or the use of electromagnetic radiation’ [1]. Many interventional procedures also involve the use of medical devices. A medical device is defined as ‘any instrument, apparatus, appliance, software, material or other article, whether used alone or in combination, together with any accessories, including the software intended by its manufacturer to be used specifically for diagnostic and/or therapeutic purposes and necessary for its proper application, intended by the manufacturer to be used for human beings for the purpose of diagnosis, prevention, monitoring, treatment or alleviation of disease.’ [2]. Compared with other types of health technology, new interventional procedures including medical devices are subject to relatively little regulation, often entering clinical practice without a thorough assessment of safety and efficacy [3], [4], [5]. This contrasts with pharmaceuticals, which are subjected to rigorous assessment before they are routinely used in patients [6], [7], [8].

In the United Kingdom, the evaluation of pharmaceuticals follows a framework made up of four distinct progressive phases (Box 1) [9]. Currently, there is no formal framework for the evaluation of interventional procedures and the framework used for the evaluation of medical devices is suboptimal [5]. The lack of a defined set of steps for the approval of interventional procedures may lead to patients being offered treatments without access to adequate information on the nature and likelihood of benefits and harms and clinicians being uncertain which new treatments they should adopt. Furthermore, health systems have difficulty in managing the risks associated with new procedures and hospitals do not want to be taken by surprise by unexpected consequences arising from treatments [3], [10]. Typically, the evaluation of interventional procedures stops at studies assessing safety and efficacy in small groups of patients [11]. For example, in the late 1980s optic nerve decompression surgery for nonarteritic anterior ischaemic optic neuropathy, a condition that is associated with a sudden and painless loss of vision, was performed very frequently with supporting data coming from small short-term case-series [11]. Years later, findings from a pragmatic randomised controlled trial (RCT) [12] indicated that this surgical treatment is both ineffective and potentially harmful as patients receiving surgery had a significantly greater risk of losing visual acuity [11]. Based on these findings the investigators recommended that its use should be abandoned [11]. Another example where early data was misleading is electronic fetal heart rate monitoring during labour. This procedure was used for two decades based on evidence from case-series and comparative studies using historical or non-randomised concurrent controls. When results from RCTs became available, these showed no evidence of benefit to the fetus and that electronic fetal heart rate monitoring was potentially harmful to the mother by increasing the caesarean section rate two to three fold [13]. Box 2 shows examples of other interventional health technologies that were widely used in routine clinical practice before they were found to be ineffective or harmful [14].

In this paper we therefore argue that the evaluation of interventional procedures including or not a medical device (from this point onwards we refer to this as ‘interventional procedures’) should also be regarded as a progressive process rather than a discrete event focusing on short-term data on efficacy and safety by proposing a framework for their evaluation, and this echoes arguments of Ergina and colleagues on the need for a framework for generating evidence on interventional procedures [15].