Accelerating Communication between Scientists

When scientists submit a paper to a journal for publication, their next step is to wait for the reviewers’ comments and the journal’s final verdict. The clock ticks; and in the meantime, the results remain locked from the public eye for months. Many researchers complain that the process delays the opportunity to obtain feedback from others and foster collaboration.

The website medRxiv (pronounced med-archive) was created to address this delay. Launched in July 2019, the website is a preprint server where medical researchers can upload their studies before peer review, which can take years, and involves back-and-forth between researcher and reviewer. According to two of its founders, Harlan Krumholz, MD, SM, the Harold H. Hines, Jr. Professor of Medicine (Cardiology) and professor in the Institute for Social and Policy Studies; and Joseph Ross, MD, MHS ’06, professor of medicine (general medicine) and of public health, medRxiv accelerates this process.

“There’s a lot of research where it’s imperfect, but we can learn from it,” Ross said. “And the long road from finalizing the paper to getting it published prevents us from learning from one another. And so that’s part of what we’re trying to address here.”

Krumholz’s deep interest in open science began with the recognition that scientific progress had been impeded by a culture of academic competition that sequestered data and blocked full transparency of methods. Together, he and Ross involved themselves in several open-science efforts, most notably founding the Yale Open Data Access (YODA) project, which facilitates access to clinical trial data. And according to Ross, medRxiv was the next step in their quest to foster transparency in medical research.

Preprint servers exist for many disciplines, ranging from economics to mathematics to data science, and provide platforms for scientists to share work that is not yet peer reviewed. They offer an opportunity to share findings before they appear in journals; medRxiv is a clinical research preprint service.

medRxiv’s partnerships grew out of the 2017 Peer Review Congress in Chicago, where Krumholz gave a talk announcing a preprint service for the medical sciences. Krumholz and Ross had planned the announcement and were working out the details of how it would take place. After the presentation, Krumholz and Ross met with attendees from both Cold Spring Harbor Laboratory (CSHL) and The BMJ, a medical journal published by the British Medical Association that had tried to create a similar preprint service a decade ago. The groups soon decided to join forces after talking, in what Ross called “a natural collaboration.”

“When you’re in medical research, your work is focused on helping people,” said Krumholz. “And if it can help people, then time matters, because you’re trying to create progress. If your work is languishing within the peer-review process, which can take years, then it’s by definition slowing down scientific effort.”

Krumholz’s speech at the congress sparked several questions among medical editors. Some people expressed concerns that posting research before peer review could potentially harm patients and spread misinformation. And while Krumholz and Ross said they could answer most of the questions, they recognized that medRxiv required more guardrails than other preprint services because those services didn’t necessarily have implications for the reception of medical treatment.

These safeguards are important during the screening process. The research cannot identify participants, use unregistered clinical trials, or be submitted by an anonymous author. The final check is a question: could any harm result from the research? Usually, the answer is no; but if the study draws particularly strong conclusions—Krumholz gave the hypothetical example of someone claiming that cigarettes increase life expectancy—the team at medRxiv would reach out to the author and suggest the study go through standard peer review instead. Ross added that during weekly meetings, questionable submissions are one item on the agenda along with general medRxiv policies that might require alteration.

One safeguard of medRxiv stands out on the homepage of its website: a disclaimer in red type that cautions readers against using the studies posted for “clinical practice or health-related behavior.”

Krumholz noted other benefits of preprints. As significant amounts of research ultimately end up unpublished, preprints allow for registered trials and their results to appear online. Preprints also protect the provenance of an idea: two people could be working on the same project at the same time, but first publication might be based simply on luck. “In terms of actually making progress, you want to see today what people did yesterday, so tomorrow you can learn from it and maybe build on it,” Krumholz said.

The medRxiv site, which is owned by CSHL, works on a volunteer basis, with members of the scientific community signing up to read papers on their own schedules. These readers, called affiliates, examine the ethics of the papers and make note of any concerns. Krumholz and Ross emphasize that medRxiv is really a resource to the scientific community and that it benefits greatly from the contributions of many, including a dedicated staff at CSHL.

One of those affiliates is Rachel Dreyer, PhD, assistant professor of emergency medicine, who has worked with medRxiv since mid-2019 and regularly reserves time to review papers. She first reads the abstract to check for offensive material or disturbing claims or conclusions; then ensures that the ethics of the research, particularly regarding human subjects, have been correctly addressed. Dreyer estimates that she passes about 95% of the papers she reviews.

Dreyer added that she thinks the service has been particularly important in the pandemic. Since COVID-19 began, medRxiv has seen an influx of COVID-19 related papers—papers that would typically take months to appear in a peer-reviewed journal.

“I’m interested in and passionate about open science, and just the idea of preprints,” Dreyer said. “Submitting an article for publication to most journals is such a lengthy process, and I think there’s a lot of value in submitting unpublished research that you’re working on that can get out there in the scientific literature to share with others in the field.”

Another medRxiv affiliate, Joshua Wallach, PhD, MS, assistant professor of epidemiology, also emphasized the time it takes for medical research to make it through the peer-review process. When he reviews research for peer-reviewed journals, he typically dedicates one to four hours to reading, making comments, and fleshing out those comments into a structured document. Scanning pieces for medRxiv does not require as much time—though Wallach also emphasized that despite medRxiv’s expedited posting process, researchers are still expected to submit their work to journals.

After receiving a submission, editors at peer-reviewed journals screen papers and identify peer reviewers with proper expertise. But there’s never any guarantee that the journal will ultimately accept the research—and if a study is rejected, the process begins anew. As a researcher who has waited for review, Wallach added, the long time frame can be frustrating.

“If you’re working in a rapidly changing field, six to 12 months is a very long time to wait to get that information out there, especially if other people are working in that field and are hoping to be able to read your paper, build upon what’s been done, maybe learn from the techniques and methodology,” Wallach said.

But medRxiv has alleviated some of that frustration. As the COVID-19 pandemic has progressed, Wallach added, medRxiv and other preprint servers have allowed for mass dissemination of research among scientists. Krumholz also noted that in June, the website posted a study by British researchers on the effects of dexamethasone in COVID-19 patients almost a full month before the New England Journal of Medicine published it. As of spring 2021, dexamethasone is widely used to treat severe cases of COVID-19, according to the World Health Organization (WHO). Nevertheless, Krumholz emphasized that the preprint server provides a service that is complementary to peer review.

Since its inception, medRxiv has boomed. According to Ross, while it’s impossible to quantify the exact impact of COVID-19 on the site’s growth, the number of submissions exploded from 7.2 papers a day in January 2020 to 77.1 in May. Now, Ross says, the number has settled around 50 papers per day in November and December 2020.

Wallach added that while he expects the bulk of research to continue to revolve around COVID-19, he’s also looking forward to seeing what’s submitted once the pandemic wanes. But no matter what the topic, Wallach emphasized that medRxiv is a worthy enterprise. “It is something that is just done out of interest and belief in medRxiv as a preprint service and as a service to the scientific community,” Wallach said.