Current Controlled Trials: an opportunity to help improve the quality of clinical research

Abstract

Some problems with the quality of controlled clinical trials can be addressed by following these procedures: registering all trials at inception; using systematic reviews to inform the design of new studies; posting and obtaining feedback on preprints; reporting all well conducted trials, regardless of their results; reducing biased and inefficient assessment of reports submitted for publication; publishing sufficiently detailed reports; linking trial reports to relevant external information; providing readier access to reports; and reviewing and amending reports after initial publication. The launch of a new range of electronic journals by Current Controlled Trials offers an opportunity to contribute to progress in these ways.

Full text

What's wrong with clinical research?

Six years ago Altman drew attention to 'the scandal of poor medical research' [1]. He called for less research, better research, and research done for the right reasons. In the continuing deluge of reports of biomedical research, studies that really do seem to have been worth doing are refreshing. The example I would single out from the field of research in which I worked previously is the Collaborative Eclampsia Trial [2]. The study showed that a dirt-cheap drug (magnesium sulphate) was more effective than more expensive alternatives in controlling convulsions in eclampsia - a condition that leads to the deaths of about 50 000 women every year, almost all of them in developing countries. Not only did the trial demonstrate the great importance of an inexpensive medicine that was not even on WHO's list of essential drugs, it also challenged assumptions about the pathophysiology of convulsions [3], prompting some neuroscientists to take a new interest in the role of magnesium in seizure activity. Why are there so few studies of this quality and relevance?

Failure to recognise controlled clinical trials as 'indispensable ordeals'

One reason is that the need for reliable evaluation of clinical interventions seems often to be overlooked in the excitement surrounding more basic research. Fifty years after the structure of DNA was discovered, for example, the cacophony of claims about the potential benefits for health care of this advance in basic knowledge is becoming almost deafening. As one geneticist has observed, however, 'for twenty years geneticists have issued a stream of promises about what they will achieve. Few have been fulfilled, and some never will be' [4].

There is no way of responsibly bypassing the need to use well-designed controlled clinical experiments to test the validity of therapeutic theories, whether these have been derived from basic research, or from clinical impressions. In a recently reported controlled trial of E5 murine mono-clonal anti-endotoxin involving over 1100 patients, for example, no benefit of this product could be detected, despite the fact that basic understanding of Gram-negative sepsis has grown importantly over the past 20 years [5]. A decade after the genetic defect leading to cystic fibrosis was identified, people with the condition are asking when they will see dividends to their health resulting from the discovery.

Very occasionally, the effects of new treatments are so dramatic that carefully controlled research is unnecessary. Usually, however, research designed to distinguish both the effects of biases and the play of chance from treatment effects is needed to protect the interests of patients. These controlled clinical trials are 'indispensable ordeals' [6] for testing the validity of therapeutic hypotheses derived from more basic research or clinical anecdote. They are the principal means of ensuring that the health care interventions on offer to the public are useful, and that they are acceptably safe, at least in the short term.

Failure to study outcomes that matter to patients

Not only must theories be validated in controlled trials, they must be validated in ways that are meaningful to patients. Psaty and his colleagues have drawn attention to the need to study effects on outcome measures that matter to patients and practitioners [7]. Reliance on surrogate outcomes sometimes has catastrophic consequences. For example, as people who develop rhythm abnormalities during heart attacks are more likely to die prematurely, the demonstration that drugs could reduce these arrhythmias was taken to be an important therapeutic advance. Although Furberg, in an early systematic review of the relevant controlled trials warned that a beneficial effect of these drugs on mortality could not be assumed [8], they continued to be used for nearly a decade. At the peak of their use in the late 1980s, it has been estimated that anti-arrhythmic drugs were causing between 20 000 and 70 000 premature deaths every year in the United States alone [9]. This yearly total of deaths is of the same order of magnitude as the total number of Americans who died in the Vietnam War.

Failure to cumulate the results of all well-designed studies scientifically

The scale of this disaster might have been contained if clinical researchers behaved in ways that acknowledge that science is cumulative. Over 50 controlled trials of anti-arrhythmic drugs were performed [10] before their lethal capacity was finally acknowledged. That would not have happened if clinical researchers, before embarking on further studies, had routinely prepared or consulted updates [11,12] of Furberg's worrying systematic review [8]. In addition, the lethal trend would have emerged sooner if the results of new trials had been presented in the context of an updated systematic review of all the other relevant evidence from controlled trials. This process is still extremely rare, even among reports of trials in the most prominent general journals [13].

The scale of the anti-arrhythmic drug disaster might also have been reduced if all the relevant research had been published. In 1993, Cowley and his colleagues [14], commendably, pointed out how an unpublished study performed in 1980 might have 'provided an early warning of trouble ahead'. Nine patients had died among those assigned to the anti-arrhythmic drug (lorcainide) compared with only one patient among those assigned placebo. 'When we carried out our study in 1980', they reported, 'we thought that the increased death rate was an effect of chance…..The development of lorcainide was abandoned for commercial reasons, and this study was therefore never published; it is now a good example of 'publication bias', the bias through which 'negative' results of research are less likely to be reported [15].

Acquiescence in distorted clinical research agendas

Failure to publish the results of controlled trials for commercial reasons is one of the reasons for growing doubts about the integrity of clinical research. Recent commentaries in both The Lancet [16] and the New England Journal of Medicine [17] have drawn attention to the worrying incentives that drive those involved in clinical research, and the increasingly dubious relationships that are developing between academia and industry. One of the editorialists asks bluntly 'Is academic medicine for sale?'

Not only does the influence of industry within academia raise worrying questions about the research that does get done and reported; it also raises questions about the research that does not get done. Commercial priorities distort the ways in which inevitably limited clinical research capacity is used. The 'opportunity cost' of this tendency is that many questions about the effects of interventions intended to improve health, particularly aspects of health (such as eclampsia) in the poorer parts of the world [18], may not be addressed because they do not interest the commercial sector. Examples of issues of importance to the public's health include aspirin for myocardial infarction [19], indomethacin for early dementia [20], corticosteroids for head injury [21,22]; carotid endarterectomy for cerebral ischaemia [23], investigations for benign chronic headache [24], and counselling for psychological distress [25].

Commercial interests are not the only influences leading to patterns of clinical research that do not serve the interests of patients. Perverse reward systems within academia must share some of the responsibility. Relman, a former editor of the New England Journal of Medicine, had this scientifically bizarre advice for his readers: 'Large-scale multi-institutional clinical trials provide less opportunity for authorship than individual or small-group research . . . . Increased opportunities for authorship can be provided if the National Institutes of Health encourage small-scale clinical trials carried out by individual investigators.' [26]. Given the continuing problem of trials that are too small to rule out effects of likely importance to patients, such advice suggests a concern to ensure that, first and foremost, research should serve the interests of academic investigators, not patients.

Other evidence that researchers do not have patients sufficiently in mind when they design controlled trials comes from the results of surveys to assess whether doctors themselves would agree to participate in the trials to which they are expected to recruit patients. For example, asked whether they would consent to participate in each of six lung cancer trials for which, as patients, they might be eligible, between 36 and 89 per cent of a sample of Canadian physicians who treat the disease said that they would not participate [27].

How can electronic media be exploited to improve the quality of clinical research?

Electronic publishing offers scope for improving the quality, relevance and reporting of clinical research [28]. For the reasons already noted, in no field of biomedical research is improvement more important than in the design, conduct, analysis and reporting of controlled trials. Guided by some principles relevant to promoting the quality and relevance of controlled trials, the new range of electronic journals being launched by Current Controlled Trials thus offers an important opportunity to contribute to progress.

Registering all controlled trials, at inception

The scientific and ethical reasons for prospective registration of controlled trials are now widely accepted [29,30,31]. Among the major international pharmaceutical companies, Glaxo Wellcome has led the way in developing a disclosure policy. The company has introduced a policy of registering information on its clinical trials programmes, and is committed to publishing all clinical trials, and stating the protocol number in every report of each trial, to avoid any confusion and double counting [32].

There are three main reasons for requiring prospective registration of controlled trials. First, agencies that fund research need to take their decisions in the light of information about relevant ongoing research - to avoid duplication of effort, to promote appropriate replication, and to promote collaboration, for example, in multicentre trials and/or prospective meta-analyses. Second, patients, clinicians and other decision makers need to be informed about trials in which they can participate, or to which they can contribute in other ways. Third, people using evidence from controlled trials to guide policies and practice, and decisions about further research, need to be confident that they are aware of all the trial evidence relevant to a particular question.

Working in collaboration with research funding organisations and others around the world, Current Controlled Trials has established a meta-Register of Controlled Trials [http://controlled-trials.com], and this now contains basic details about thousands of ongoing controlled trials. In addition, Current Controlled Trials has collaborated with the UK's Medical Research Council in piloting and establishing a system for assigning a unique identifier to each of the trials registered in the meta-Register. Current Controlled Trials will help investigators to ensure that trials are registered and allocated an International Standard Randomised Controlled Trial Number (ISRCTN), and this will become a required component of all reports of trials published in their journals. In addition, contact details for the centres contributing to multicentre trials can also be provided on and after registration. Many of the trials in the United States registered by CenterWatch [http://www. centerwatch.com] and the National Institutes of Health Clinical Trials Registry [http://clinicaltrials.gov] have supplied this information, and Current Controlled Trials' international meta-Register of Controlled Trials will be developed along these lines.

Given that some questionable methods are now used to recruit patients to trials [33], it seems likely that groups representing patients will increasingly insist on details of ongoing trials being made publicly available, and that criteria for consumer endorsement of and encouragement of participation in particular trials will be developed and applied. Indeed, I hope that a 'Good Trials Guide' will be developed by groups representing the interests of consumers. This might be a powerful lever in improving the relevance of clinical trials to patients.

Using systematic reviews of existing trials to inform the design of further trials

Consumer groups and others will wish know the extent to which the design of a trial has taken account of the results of systematic reviews of all relevant previous research. This background information is required both for scientific and for ethical reasons. Funding agencies and research ethics committees have begun to require those applying for support for new controlled trials to refer to systematic reviews of relevant existing trials [34,35]. Current Controlled Trials will establish links between electronically held information about ongoing controlled trials and relevant electronically published systematic reviews, for example, those published in The Cochrane Library [http://www.update-software.com/cochrane/cochrane-frame.html].

Obtaining feedback on preprints of controlled trials

Publication of preprints of research reports to obtain feedback has been commonplace for many years in some spheres of scientific activity [36]. Although it is a relatively new notion in clinical research, both the Lancet [37] and the British Medical Journal [38] now offer a 'preprint' service to the research community. Current Controlled Trials will also provide a non-refereed preprint depository to which any article can be submitted and which all individuals can access free of charge. Any article can be submitted by its authors, but they will remain responsible for the article's content. The only screening process will be to ensure relevance of the article to the scope of Current Controlled Trials and to avoid abusive, libellous or indecent articles.

Reporting all well-conducted controlled trials, regardless of their results

Studies which have yielded 'disappointing' or 'negative' results are less likely to be presented at scientific meetings, reported in print, published promptly, in full reports, in journals that are widely read, in English, and in more than one report; and they are less likely to be cited in reports of later studies [15]. On average, these reporting biases will tend to lead to inferences that interventions are more effective than they are in fact. Journals can help to reduce these biases by requiring authors of reports of controlled trials to state the trial registration details. In addition, although only very basic information about a trial need be submitted to achieve registration, this process offers an opportunity to provide more details about studies. The Lancet, for example, has pioneered journal peer review and publication of trial protocols, undertaking to accelerate peer review of reports of such trials on completion [39], a process that will reduce result-dependent biases among reviewers. Current Controlled Trials will offer a similar service.

One of the most important undertakings of Current Controlled Trials is to publish all trials judged by peer review to have been carried out correctly, irrespective of their results. This undertaking is particularly important in respect of reports of controlled trials that are 'unexciting', but nevertheless can contribute, however modestly, to the sum of knowledge relevant to a particular question. Authors of reports of such trials often find themselves submitting to a succession of print journals, each of which requires submissions in a slightly different format, only to find that none of the journals is prepared to allocate their limited page space to the studies concerned. Not surprisingly, authors may stop trying to publish these studies. The undertaking made by Current Controlled Trials will help to reduce these unproductive efforts. This will be of particular help to pharmaceutical companies, which often undertake routine or repetitive trials to fulfil regulatory requirements. Several companies have recently committed to publish results from all their trials by endorsing a set of guidelines on Good Publication Practice [40], so developments that will facilitate this are particularly welcome.

Reducing biased and inefficient assessment of trial reports submitted for publication

Contrary to widely held assumptions, the effectiveness of the much vaunted 'black box' of peer review is not based on empirical evidence supporting most of the various procedures and rituals which it comprises. Indeed, there is evidence that the process can be biased [41]. Largely as a result of initiatives taken under the aegis of the Journal of the American Medical Association [42], there has been an encouraging growth in the empirical research required to sort out which of the elements of peer review are worth retaining, and which should be jettisoned. Systematic reviews of this evidence are being prepared (T Jefferson, personal communication) and these will be used to inform the procedures adopted by Current Controlled Trials.

Publishing sufficiently detailed reports of controlled trials

A recurrent criticism of reports of controlled trials is that they are not detailed enough to allow readers to judge the merit of the research, or to apply it sensibly. The reporting guidelines published by the CONSORT Group, and any revisions of these [43], will be used as a basis for judging the acceptability of submissions to Current Controlled Trials. Current Controlled Trials are working towards providing authors with a standard electronic template for reporting their studies to help them comply with the guidelines. All articles accepted for publication by Current Controlled Trials will be published electronically without restrictions on length.

These electronically published reports will exploit all the possibilities of this medium, including the opportunity to publish large datasets and to display data in a form that can be read directly by other software packages, to allow readers to manipulate the data for themselves. Again, other areas of scientific enquiry have been ahead of medicine in depositing electronic datasets generated from research, both for probity and to facilitate further analysis. Such developments will clearly facilitate the more flexible and robust meta-analyses that become possible using individual patient data [44].

It will be possible to use large numbers of still illustrations, for example, to describe the characteristics of patients who participated in a particular trial, and also to publish video illustrations, for example to show just what the interventions studied involved, and interviews with patients about their experiences of these.

Linking trial reports to relevant external information

Electronically published reports of controlled trials can be linked to other relevant information held electronically. Most obviously, links can be established with the bibliographic or full text records of other studies cited in the report, and with other reports of the same trial. In addition, because the CONSORT guidelines state that data from a new trial should be interpreted 'in the light of the totality of the available evidence' [43] and current reports of controlled trials rarely do this [13], links can be made to relevant systematic reviews. Furthermore, because many reports of trials end with an indication of the important questions that remain unanswered, links can be made to records of ongoing trials that are addressing those questions.

In addition to these links of very direct relevance to the report of a particular trial, considerable additional scope exists for links to other material of possible relevance, such as reports of similar trials, or background documents about the health problems, interventions or outcomes studied.

Providing readier access to reports of controlled trials

Although the World Wide Web has transformed access to the results of research, in principle, 20^th century publishing norms mean that price barriers still stand between the results of the research and those whose taxes, charitable donations and expenditure on drugs have supported the research. Current Controlled Trials believe that original papers publishing information about trials should be free to everyone. Research articles will therefore be available free both on the Current Controlled Trials website and in BioMed Central. This will help to make postpublication peer review more effective, using facilities similar to the BMJ's Rapid Response System.

Reviewing and amending reports of controlled trials after initial publication

Postpublication review of published reports of controlled trials will sometimes uncover correctable errors, or the need for more appropriate analyses. Improvement of trial reports in response to these suggestions raises concerns about what should be regarded as the 'archival' version of a report, but this should not be a reason for failing to improve a report when it is clear that this is possible. All the pieces of information about and reports of a particular trial will need to be electronically threaded together, so that they can be assessed separately and together. As Tony Johnson, formerly editor of Statistics in Medicine, said at a meeting of the European Association of Science Editors:

'Current systems of peer review have failed to detect, let alone correct, even elementary mistakes in design, analysis, presentation and conclusions. Electronic publication will enable us to move away from the single version, printed paper, refereed by a small number of recognised experts, to a dynamic multiversion paper which can be updated as necessary to address criticisms of referees throughout the world' [45].

Conclusions

Important progress was made during the 20^th century in developing robust research methods to assess the effects of medical and other healthcare interventions. Very substantial room for improvement remains, however [46]. A variety of strategies are required to improve matters so that clinical research meets more effectively the needs of people using the health services. Among these, electronic publication of information about and derived from controlled trials seems set to make an important impact.