By: Kelly Devers, Ph.D., Chair, AcademyHealth Methods Council and James F. Burgess, Ph.D., Vice Chair, AcademyHealth Methods Council
The majority of AcademyHealth’s more than 4,400 individual and organizational members are American, but an increasing portion are based internationally, and the Board of Directors recently approved the establishment of a Global Health and Health Care Interest Group. As the professional home for professionals that conduct research using a wide variety of relevant and rigorous methods to address complex questions about the delivery of care and the performance of the health system, we are concerned about the British Medical Journal’s apparent reversal of position on the importance of qualitative and mixed methods research and by extension the kinds of questions these methods are best able to address. To better understand the issue and options, we turned to the co-leaders of our Methods Council – an esteemed group of experts who advise AcademyHealth in developing strategies for professional development in health services research; and provide guidance on emerging and relevant methods themes, relevant content for HSRMethods.org and on ongoing AcademyHealth projects and activities on an ad hoc basis. Below is their summary and response.
In September 2015, McGill Qualitative Health Research Group tweeted an excerpt of a rejection letter received from the editors of the British Medical Journal that indicated qualitative studies were “an extremely low priority” for the journal. In February 2016, 76 senior academics from 11 countries responded with an open letter urging the journal to develop a formal policy to address the inclusion of qualitative and mixed methods research.
As Chair and Vice-Chair of the AcademyHealth Methods Council, part of our charge is understanding, assessing and helping to explain research methods that underpin the work of our field. In this role, we see immense value in the contributions of qualitative and mixed methods research, and we fundamentally disagree with the assumption that such studies are “low priority,” “unlikely to be highly cited,” “lacking practical value,” or “not of interest to [BMJ] readers.” Qualitative and mixed research methods are particularly well suited to addressing some of the most pressing medical, health services, and policy research questions as noted here and here. Additionally, as Greenhalgh et al point out in their letter, qualitative papers have been among the top papers in BMJ over the past 20 years. Weiner et al report similar results from a review of nine major US health services and management research journals over a ten year periods (between 1998 and 2008) here and here, with citation analysis showing that although qualitative research articles comprised only nine percent of all published research articles, some were among the top papers in the field and overall qualitative research articles contributed comparably to the field's knowledge base.
We therefore support the call by Greenhalgh and colleagues for the BMJ to develop and publish a formal policy on qualitative and mixed methods research, including explicit criteria for judging the relevance of such manuscript submissions. Such a policy would prevent qualitative and mixed methods research from being automatically and arbitrarily rejected and would help researchers, reviewers, and editorial staff distinguish exemplar from flawed qualitative and mixed methods research and identify which manuscripts should be prioritized for publication. There is a rich base from which the BMJ can start, including not only articles previously appearing in the BMJ but prominent examples from US funders and journals.
For example, in 1999, with support from the U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality (AHRQ) and the Robert Wood Johnson Foundation, Health Services Research, one of AcademyHealth’s two official journals, published a special issue on Qualitative Methods here and here, with one article focusing on criteria for assessing the quality of qualitative research*, here. Following the HSR special issue, the U.S. National Institute of Health and National Science Foundation published guidelines for qualitative and mixed methods proposals and other journals (including the Journal of the American Medical Association and Circulation: Cardiovascular Quality and Outcomes) published guidelines about writing and reviewing qualitative research references here, here , here and here. More recently, Tong et al (2007) synthesized 22 guidelines for publishing qualitative research, resulting in the Consolidated Criteria for Reporting Qualitative Studies.
Finally, Greenhalgh and colleagues’ challenge to the BMJ to publish an occasional series of “qualitative notes” and “landmark” qualitative papers with an accompanying methodological commentary from an international expert provides both an excellent opportunity to put these ideas to the test and possible opportunities for international collaboration. We think that the AcademyHealth membership and US journals, including Health Services Research, would be natural partners for further dialogue, collaboration, and commentary on qualitative theory, methods, and “landmark” papers. We would also point to Health Services Research for their best practice of Methods Articles, some on various aspects of qualitative research.
Over the years Health Services Research developed a policy to explicitly welcome qualitative and mixed methods research studies and increased the number of qualitative and mixed methods editorial board members and reviewers. Finally, current collaborations between UK and US based scholars, such as the one between Dixon-Woods and Bosk and colleagues (see for example Dixon-Woods, Bosk, Aveling et al, 2011), illustrate the tremendous power and promise of international collaboration at all stages of qualitative and mixed methods research and the publishing process.
In summary, we strongly encourage the BMJ Editorial Board to accept the Greenhalgh et al proposal for formal policies and concrete steps that continue to move us beyond tired and distracting debates of qualitative versus quantitative research to a richer understanding of the unique and valuable contributions of qualitative research and consensus on about how qualitative and quantitative methods can best be used in complementary ways. We would also support AcademyHealth’s continued engagement as enthusiastic partners in further dialogue about criteria for assessing qualitative and mixed methods research, including how and when to best use these methods to engage, inform, and stimulate doctors, other health professionals, researchers, and policymakers to make better decisions and improve outcomes for patients and the public.
To follow the conversation online read letters to the editors via rapid responses, here and follow the conversation on Twitter here.
*Both authors (KD and JB) are, or have been, members of the editorial board for Health Services Research.