Critical reflections on the rise of qualitative research
BMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b3425 (Published 15 September 2009) Cite this as: BMJ 2009;339:b3425
- 1School of Health Sciences, University of Southampton, Southampton SO17 1BJ
- 2Health Services Research Unit, Department of Public Health and Policy, London School of Hygiene and Tropical Medicine, London WC1E 7HT
- Correspondence to: C Pope cjp{at}soton.ac.uk
- Accepted 9 March 2009
In 1993 the BMJ published an unusual article about qualitative research in which we reported a fictitious encounter in the corridors of health services research.1 The article was a socratic dialogue between a quantitatively trained director of a health services research unit and a more junior qualitative sociologist. The dialogue was designed to stimulate debate about the dominance of quantitative research and, in particular, the randomised controlled trial in health related research. It went on to suggest that qualitative methods should be taken more seriously. Since then qualitative research methods have become far more widely accepted in health services research and many areas of medical and nursing research.2 3 4 5 6 7 8
Although it seems that qualitative research is established in healthcare settings, we are not convinced that it is always conducted appropriately. This article critically evaluates how far qualitative research has come and asks some searching questions about whether researchers are using its full potential to inform and improve health service organisation and the delivery of care.
In keeping with the spirit of our original paper, we have used the conversational format to revisit the two fictional characters and the debate about the place of qualitative research. The scene is the retirement party in honour of the director of the research unit in the 1993 dialogue. We join him just as he concludes his speech of thanks.
DIRECTOR: And so, the unit has grown, both in terms of the number of people and the range of disciplines they draw on. We use diverse research methods, notably much more qualitative research. This is something I am proud of, and something I look forward to my successor taking forward. I wish her and all my former colleagues the very best of fortune. But I’ve talked long enough. There is food and drink waiting for us, so let’s enjoy ourselves.
[The director sees someone he recognises approaching him]
DIR: So it is you. I didn’t think you’d respond to my invitation considering our differences of opinion. How long has it been?
SOCIOLOGIST: 15 years.
DIR: You must be amused to hear me talking about qualitative research with such enthusiasm. But I have come round to your view. There is such a range of qualitative research going on and it’s making a huge contribution. We even have qualitative work embedded in randomised controlled trials, helping to improve trial design and shape the outcome measures we use.9 Most of the researchers here are using interviews and focus groups in their research. You’ll have noticed that there was barely a regression equation in sight all afternoon during the scientific meeting in my honour.
SOC: You sound like me 15 years ago. What’s happened?
DIR: Well, I have to admit, you really made me think. And after you left the unit I hired more social scientists, and we’ve come a long way since then.
SOC: I’m delighted that qualitative research is so much more mainstream. For one thing, I don’t have to have to revisit those tired old arguments about small samples and “soft” science. But don’t you think it’s all got a bit out of hand?
DIR: I thought this was what you wanted.
Quality of qualitative research
SOC: Yes, but I do worry about the quality of some of the qualitative research out there.10 Everyone thinks they can do qualitative research now, and I’m not always convinced we do justice to what we were trying to achieve when I started doing health research.
DIR: What do you mean?
SOC: Oh, you know, those papers that report thematic analyses. The kind that report six themes and don’t explain what the relationship is between the themes and so don’t really go anywhere in terms of trying to explain the data. Sometimes the problem is the word limits set by journals, but sometimes it’s just superficial work.
DIR: But wasn’t I reviewing one of your reports the other day? It had exactly that kind of analysis if I remember correctly.
SOC: Well, yes, you’ve caught me out there. But we had a tight deadline and were already late starting our next project. I just didn’t have time to get beyond a basic analysis.
DIR: But haven’t you got software like ATLAS.ti and NVIVO now? It must speed things up from the old days when you had to use sheets of paper and marker pens.
SOC: I still analyse my data like that, I’m afraid. The trouble with software packages is that you can actually get quite a long way without having to make sense of what you’ve found.
DIR: Now you sound as disapproving as me. That is how those of us with a proper statistical training feel about today’s user friendly SPSS software.
SOC: Yes, I agree, though I suppose qualitative analysis software has helped make qualitative research acceptable. It certainly makes it look technical, which is quite appealing in health research. Software is useful, but it doesn’t produce the interpretation for you; all too often that creative turn is missing.
DIR: Some of our researchers use something called framework for their research.11 What do you think of that approach?
SOC: It’s seductive because it looks systematic. But you have to use it with care and remember it was designed for tightly framed, commissioned policy research. It can sometimes be very deductive, which is odd for qualitative analysis. A real strength of qualitative research is induction—interpreting the data. That’s where you find the unexpected. I sometimes worry that we don’t push qualitative research far enough. I’d like it to be less descriptive and for us to try harder to explain things.
Ubiquity of interviews
DIR: Finally, you’re talking like a real scientist. But, to be fair, qualitative researchers have changed the way the rest of us do research. We wouldn’t design a study these days without incorporating people’s views—talking to patients in depth—it’s so important to get the users’ as well as the professionals’ perspectives.
SOC: But there are so many studies that are just based on “one shot” interviews. You end up with 20 minutes of talk, no private accounts, just surface description. Researchers short change themselves by taking talk at face value. I’m particularly concerned about interviews with health professionals, service managers, and policy makers. They’re well educated and experienced at presenting themselves in public. It makes them hard to interview, but few people seem to try to get behind their façades. When you do, of course, things can seem very different. Checkland and colleagues highlighted this in their study of the implementation of clinical best practice guidelines in general practices.12 The GPs they interviewed listed commonsense implementation barriers such as lack of time. However their accounts could not be understood literally; instead, they related to the GPs’ underlying beliefs about how work should be allocated in a general practice and what it meant to be a GP. The things that stopped GPs implementing guidelines actually had little to do with information management and shortage of time, the main reasons they gave to the interviewer.
DIR: I can see that one to one interviews taken at face value might be problematic. But maybe focus groups are less superficial, because people in a group talk to each other in a less self conscious way, don’t they?
SOC: But focus groups are often only used to get hold of a bigger sample, which misses the point. People think that three focus groups of six people equates to 18 face to face interviews, but ideally you would use the group dynamics of a focus group to get more interesting data.13 14 I know I shouldn’t say this, but I’m fed up with interviews. I’d really like to get back to doing more observation, or ethnography, so we can see what people really do, not just what they say they do.
DIR: But ethnography takes far too long and is difficult to get funded. It’s just not efficient. It’s also sometimes difficult to get permission to do observations from research ethics committees or healthcare organisations. Even if you do obtain funding, ethics approval and access to sites, you spend days hanging around and can’t guarantee you’ll see anything important. You also have to rely on an individual researcher to collect good data. With interview studies, I can send a team of less experienced researchers out with a topic guide and a tape recorder and they usually come back with useable data if they’ve been briefed properly. I’m not sure that ethnography is predictable enough in that way.
SOC: I think your enthusiasm for interviews is getting the better of you. I can see the practical constraints of ethnography, but, in fact, these are as much the result of assumptions from quantitative research about the appropriate length of time for research projects as inherent weaknesses in ethnography. As it happens, ethnographers are adapting to new time requirements. Health researchers are using ethnographic methods successfully and in the timescales of conventional research projects.15
DIR: But what about the more fundamental ethical issues with observation? How can you get consent from people when you don’t know in advance who you’re going to observe and what you might see or overhear? That book you gave me once about psychiatric hospitals and their effect on their patients was great,16 but I can’t see it getting through research ethics these days, what with the author pretending to be a physical education instructor on the hospital staff and collecting data covertly.
SOC: Don’t get me started on the problems of ethics in qualitative research. I admit it’s difficult, especially explaining to research ethics committees how it’s possible for an observer to behave ethically without obtaining written consent from all the people she is going to observe.17
Mixed up methods?
DIR: Despite all these difficulties, more and more qualitative research is being done, and many studies use mixed qualitative and quantitative methods these days.18
SOC: I call it “mixed up” methods.
DIR: But you were always pushing us to consider qualitative methods alongside quantitative research.1
SOC: Mixed method approaches have made researchers take qualitative methods more seriously, but we are not really making as much of mixed methods as we could. Many studies use parallel methods, keeping qualitative and quantitative methods separate in the same study. And qualitative research still tends to be in a supporting role.19 Researchers seldom think about how to integrate methods and findings20 or take notice of the fact that different methods give you a different picture of what’s going on (box 1). You can’t just add everything together. What happened to the idea that we could learn more from the fact that qualitative research looks at things in a different way?
Box 1 Qualitative and quantitative methods can be integrated at different stages in a research project
Design—eg, using qualitative interviews to develop a quantitative measure
Sampling—eg, using an initial survey to determine or provide a sampling frame for qualitative interviews
Analysis—eg, using qualitative research to inform priors for bayesian statistical analysis
Interpretation—eg, integrating the findings in chapter, papers, reports
O’Cathain suggests that health services research tends to be weakest at integrating at the analysis stage.20
DIR: So we don’t always make the best of all the data in a study, but how do you square different accounts of the same thing, especially when you’re up against a deadline?
SOC: I agree it makes things more complicated, but the creative tension between methods can be positive—it can lead to better, more complex explanations. The challenge is to provide an account that is dialectical—that uses the different insights from different methods to tell us something new (box 2).
Box 2 Three examples of integrating mixed methods
Turnbull et al examined geographical variation in use of general practice out of hours services using statistical analysis of routine data.21 This generated hypotheses for qualitative research and prompted a choice of two very different locations to explore the role of place in determining service use.22 The quantitative results also showed that the greatest variation in service use between urban and rural areas was in children aged 0-4 years (compared with other age groups), so the qualitative study focused on the experiences of parents of young children. Using mixed methods this research showed the importance of place in addition to distance or rurality in access to care
Mannion et al used semistructured interviews and documentary analysis in six English NHS acute hospital trusts to look at the relation between culture and performance (in terms of the government’s star rating system).23 This was followed by a large quantitative study to see if the findings from the case studies were replicated nationally.24 The research established that organisational culture varied across hospitals and that some of this variation was associated, in predictable ways, with organisational characteristics and performance
Murtagh et al did a qualitative process evaluation alongside a randomised controlled trial.25 As a result of observation and semistructured interviews the researchers decided to discontinue one arm of the trial because “the intervention in that arm, a standard gamble values elicitation exercise, was causing confusion and was unlikely to produce valid data on participant values”
DIR: That sounds ambitious in a typical two year, commissioned study. You sound so negative about most qualitative health research—isn’t there anything positive?
Possibilities of synthesis
SOC: Well, I have been excited by the development of methods for qualitative synthesis.26 For a long time, qualitative research has been criticised because it produces lots of seemingly unique and small scale case studies. Qualitative synthesis provides ways of integrating qualitative studies to build a cumulative knowledge base, and potentially for combining qualitative reviews with quantitative systematic reviews and meta-analyses. Of course, it won’t be much good if the original studies were under-analysed. Then you just end up with an assembly of reports of people’s views on a subject.
DIR: Well I am glad to hear you positive about at least one aspect of qualitative health research. It has been fascinating talking to you again.
Notes
Cite this as: BMJ 2009;339:b3425
Footnotes
We thank all our colleagues who have contributed to our thinking and the debates about qualitative methods over the past 20 years and the BMJ’s two reviewers. An earlier version of this article was presented at the Society for Social Medicine annual scientific meeting, University of Southampton, 17-19 September 2008.
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.