Working against bias: double blind peer review at Journal of Osteopathic Medicine

The International Committee of Medical Journal Editors (ICMJE) comprises a small assembly of editors from leading general medical journals, including British Medical Journal, Journal of the American Medical Association, and New England Journal of Medicine [1]. Their Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work in Medical Journals (ICMJE Recommendations), issued regularly since 1978 [2], are made publicly available by the ICMJE to support evidence based editorial standards. While the ICMJE Recommendations are “intended primarily for use by authors who might submit their work for publication to ICMJE member journals,” [3] Journal of Osteopathic Medicine (JOM) is among the many academic medical publications that views these recommendations as the seminal guiding document for editorial operations.

In the latest set of ICMJE Recommendations updated in December 2019 [4], the committee of editors said this about peer review:

Peer review is the critical assessment of manuscripts submitted to journals by experts who are usually not part of the editorial staff. Because unbiased, independent, critical assessment is an intrinsic part of all scholarly work, including scientific research, peer review is an important extension of the scientific process … Journals may differ in the number and kinds of manuscripts they send for review, the number and types of reviewers they seek for each manuscript, whether the review process is open or blinded, and other aspects of the review process. For this reason and as a service to authors, journals should publish a description of their peer-review process. [4]

With that guidance and a commitment to transparency in mind, we’d like to share with you an overview of our manuscript evaluation process at JOM, including details about our recent transition from single blind to double blind peer review.

When an author submits a manuscript to JOM, it undergoes preliminary review by the Senior Publishing Editor, a staff member in the editorial office who ensures that all manuscripts contain the appropriate information, are formatted properly according to our Instructions for Authors [5], and conform to data reporting standards, including Institutional Review Board vetting and clinical trial registry where applicable [6]. If any information is missing or if the article is improperly formatted, it is returned to the author with a list of necessary updates, usually within 2 business days after submission.

If the article passes through preliminary review by the Editorial Office, it is assigned directly to a Section Editor specific to the topic area chosen by the author upon submission. You might recall from reading our January Editorial [7] that JOM is now divided into subsections by domain (or topic) area, an update made in 2020 for two purposes: to ensure that all articles submitted to the journal are vetted by experts in the specialty area being described in the manuscript, and to ensure that readers can easily find published articles related most closely to their area(s) of clinical practice. If an author submits a paper related to Cardiopulmonary Medicine, for example, it is assigned directly to the Section Editor who oversees that subspecialty. The Section Editor then conducts a “desk review” or “triage” of the manuscript, vetting it for methodological rigor and suitability for JOM’s readership. If the Section Editor determines that the manuscript is not likely to be of interest to our readers or that the research is otherwise lacking in a critical area, she or he may choose to reject it outright. This practice is common amongst academic medical journals [8] and its purpose is twofold: to provide authors with timely feedback rather than a lengthy peer review process that ultimately results in delayed rejection, and to limit our requests of peer reviewers (who volunteer their time) to only manuscripts with the potential to achieve publication. Of note, Case Reports, Clinical Images, Commentaries, and Letters to the Editor are assigned directly to the Editor in Chief for desk review rather than a Section Editor, as we accept a very small percentage of these article types.

If a manuscript is deemed by the Section Editor as appropriate for JOM and warrants peer review, the Section Editor invites a minimum of two peer reviewers with expertise matched to the article’s topic. These reviewers are found in a variety of ways through ScholarOne, our submission system. Keywords entered by the authors are first matched to the areas of interest specified by reviewers who are already registered in our system; the authors’ keywords can also be matched to reviewers not yet registered with JOM via the Web of Science Reviewer Locator [9], a new feature offered through our ScholarOne submission portal. We utilize this feature frequently to ensure authors get the “best” reviewer match for their paper, rather than being automatically confined to our known pool of referees. An article’s references section is another source we use to identify reviewers for the manuscript’s topic, since the authors of referenced works are often experts in their field and, as published authors themselves, are familiar with peer review and the scholarly publication process. Section Editors may also utilize their vast knowledge and connections in the field to subjectively identify expert clinicians and educators to review a manuscript.

When at least two reviews are completed, ideally within 2 weeks of submission, the manuscript returns to the Section Editor, who provides their own substantive review and synthesizes the comments of the reviewers into consistent, meaningful feedback for the authors. The Section Editor’s recommended decision is then reviewed by the Editor in Chief, who issues a final ruling on the manuscript. The manuscript then proceeds to the Editorial Office, where the decision letter is given a final edit and transmitted to the authors.

Our goal is to have the entire review process completed in a mean of 30 days or fewer. Revisions are handled in much the same fashion, although the process is sometimes shorter. Our Section Editors consult a biostatistician as needed on selected manuscripts, generally when they are close to being accepted, and some authors may receive additional statistics related revision requests at that time.

All accepted papers undergo substantial copyediting for clarity, consistency, data transparency, and completeness of reporting. We make every effort to return copyedited manuscripts to authors for approval within 1 month after acceptance; when a final version is reached, the article is transferred to our publishing partner (De Gruyter) for typesetting, proofing, and ultimately, publication in our Advance Articles section. We are proud to report that the Advance Articles option, which we began to utilize in earnest in the summer of 2020, obviates the artificial waiting time previously required when articles were published only when a print issue was available. Now, publication happens within a mean of approximately 3 months or fewer after acceptance – and in most cases so far in 2021, much sooner.

With that framework in mind, we’d like to highlight one important update to the peer review process at JOM: as of February 2021, we have transitioned from a single blind review process wherein the authors’ identity is known to the reviewers, to a double blind process wherein neither the reviewers nor the authors are aware of the other’s identity. As you know if you’ve read our Editorials in this year’s prior issues [6, 7, 10], part of updating JOM – your journal of record for the osteopathic profession – has involved close examination of our editorial processes at a granular level. As we engaged in that close examination, a few priorities remained at the fore. Chief among them was our commitment to equity of all kinds – gender, ethnicity, ability status, authors’ country of origin, type of practice, and professional life cycle stage. As we considered how and whether our processes support that commitment, we studied the effects of single blind and double blind review. The research is far from definitive, with some previous studies showing that double blind review increased the number of accepted papers with women as the first author [11] and decreased bias associated with institutional prestige [12], while others have found no difference in acceptance outcomes [13], [14], [15]. Previous studies [16, 17] have also attempted to quantify the extent to which challenges inherent with double blind review – namely, that reviewers may still guess the identity of the authors, which may still result in bias – render it void.

After weighing the research and our subjective experience, we ultimately decided to implement double blind peer review at JOM for an important reason: as with conflict of interest disclosures, perception is important. Our mission is to advance the art and science of osteopathic medicine by disseminating the best possible research to support it, and to do that, we must have the investment of both authors and reviewers. That investment is made on trust that we are engaged in the most thorough, fair, equitable, impartial, and timely iteration of peer review possible. We believe double blind review reduces, at minimum, the perception of bias and signals to our research community that we are invested in their future. We are aware of several journals with different models, including some that allow authors to select whether they prefer single blind or double blind review [18], and we remain committed – as your osteopathic colleagues engaged in research are – to following the evidence as we continue to evaluate and update our review and publication processes in the coming years.

As always, we welcome feedback from our authors, reviewers, and readers on how we can continue to ensure a lively, trustworthy, and equitable experience for those engaged with JOM at every level.