Research Literacy
Preprints, Peer Review, and What the Difference Is Worth
The short answer
A preprint is a study posted publicly before independent experts have scrutinized it — and a meaningful share never complete that step at all. Of 5,061 COVID-19 preprints posted in early 2020, only 5.7% had appeared in a peer-reviewed journal by the time researchers checked. Of 1,471 orthopaedic preprints posted from 2014-2021, 52% were later matched to a peer-reviewed publication. And peer review isn't a rubber stamp for the ones that make it: in a study matching 21 preprints of COVID-19 critical-care therapy trials to their published versions, 33% changed a primary outcome's effect measure, 24% changed the statistical significance of a secondary outcome, and 19% changed the study's stated conclusion between the two versions.
Quick answers
- What is it?
- A preprint is a complete study manuscript posted publicly before independent experts have reviewed it, while peer review is the outside-expert vetting process a journal applies before publishing a study.
- Why are people interested in it?
- People often see health claims sourced to a preprint online and need to know how much less vetted that is than a peer-reviewed study.
- What has actually been studied?
- Tracking studies of preprint-to-publication rates: 5,061 COVID-19 preprints from 2020, 1,471 orthopaedic preprints from 2014-2021, and a study matching 21 COVID-19 critical-care preprints to their eventual peer-reviewed versions to see what changed.
- How strong is the evidence?
- Reasonably solid tracking data: only 5.7% of the 2020 COVID-19 preprint batch was published within the study's window versus 52% of the slower-moving orthopaedic batch, and among matched pairs, 19% saw their stated conclusion change during peer review.
- What don't we know yet?
- What the eventual publication rate looks like with more time, whether the 19% conclusion-change rate generalizes beyond COVID-19 critical-care research, why specific preprints never get published, and whether peer review's changes make studies more accurate or just more cautious.
- Where can I read the original research?
- Sources at the bottom of this page: Añazco D, Nicolalde B, Espinosa I, et al., PeerJ, 2021; and Rizk AA, Arza RA, Jella TK, et al., Clinical Orthopaedics and Related Research, 2023.
The short answer: A preprint is a complete manuscript posted publicly — usually to a server like medRxiv, bioRxiv, or Research Square — before independent experts have reviewed it, and before an editor has decided whether it meets a journal's bar. That gap isn't hypothetical. Of 5,061 COVID-19 preprints posted in the first five months of 2020, researchers later found only 5.7% had appeared in a peer-reviewed journal. In a slower-moving field with a longer observation window — orthopaedic surgery, tracked from 2014 to 2021 — 52% of 1,471 preprints were eventually matched to a peer-reviewed publication. And peer review isn't just a formality for the ones that get there: a study that matched 21 COVID-19 critical-care therapy preprints to their later published versions found that 33% changed their primary outcome's effect measure, 24% changed the statistical significance of a secondary outcome, and 19% changed the study's stated conclusion somewhere along the way.
What the research shows
What "preprint" actually means
A preprint is the complete manuscript — full methods, full results, full discussion — posted to a public server, typically at the same time it's submitted to a journal or sometimes before. Nothing about that process involves an outside expert checking the statistics, questioning the study design, or catching an error in how the authors interpreted their own data. That checking is what peer review is supposed to add: a journal sends the manuscript to researchers with relevant expertise, who evaluate it, usually anonymously, and send back questions, required corrections, or a rejection. The manuscript that eventually appears in the journal — if it appears at all — has typically been revised in response to that process. The preprint sitting on the server is whatever the authors submitted, unfiltered.
A large share of preprints never complete the trip
The most direct way to see what this means in practice is to track a large batch of preprints and see how many eventually turn into peer-reviewed publications. A 2021 study in PeerJ did exactly that for COVID-19 research: researchers identified 5,061 preprints posted to bioRxiv, medRxiv, and Research Square between January 1 and May 31, 2020, then checked publication records using the Crossref API. Only 288 of them — 5.7% — had been published in a peer-reviewed journal by the time the study measured this. The researchers also found that preprints which did eventually get published had noticeably higher citation counts, both as preprints and afterward, than preprints that never made the transition — suggesting that publication status correlates with something about a study's underlying quality or importance, not just luck.
That said, the observation window matters enormously, and a lower publication rate in a fast-moving, high-volume field during a pandemic surge shouldn't be read as universal. A separate 2023 study in Clinical Orthopaedics and Related Research tracked 1,471 orthopaedic preprints posted across a much longer span, 2014 through September 2021, across the same three servers. Over that longer window, in a smaller and slower-moving research area, 52% — 762 of 1,471 — were matched to a corresponding peer-reviewed publication. The same study found that preprints, even when eventually published, had a smaller academic and public footprint on their own than their published counterparts: fewer citations, fewer views, less engagement, measured independently of the later journal version.
What peer review actually changes, measured directly
Publication rate answers "did it get reviewed?" A separate, more pointed question is "did the review change anything?" A 2023 study in the Journal of Intensive Care Medicine answered this directly for a narrow, high-stakes slice of COVID-19 research: preprints evaluating specific treatments — anticoagulation, dexamethasone, hydroxychloroquine, remdesivir, and tocilizumab — for critically ill patients. Starting from 40 matched preprint-to-publication pairs identified through Altmetric, the researchers randomly selected 21 for detailed comparison, checking whether the primary outcome, the effect size of that outcome, the statistical significance of secondary outcomes, or the study's stated conclusion changed between the preprint and the peer-reviewed version.
The results were specific rather than sweeping. Only 1 of the 21 studies (4.8%) saw an actual change to its primary outcome — the headline claim mostly held. But 7 of the 21 (33.3%) saw a change in that primary outcome's effect measure — the size of the reported effect shifted, even when the basic claim didn't. Five studies (23.8%) saw a change in the statistical significance of at least one secondary outcome. And 4 studies (19.0%) — nearly 1 in 5 — saw their overall stated conclusion change between the preprint version and the peer-reviewed version. The authors' own summary judgment was measured: primary outcomes were "generally reliable" in preprint form, but secondary outcomes and framing "should be made with caution" until peer review had run its course.
What this adds up to for a reader
Taken together, these three studies describe a specific, boundaried role for preprints rather than a case for or against them. They get findings into public view faster than the journal pipeline allows — which mattered enormously during a pandemic — but a substantial share never complete the trip to peer review within any given observation window, and even among those that do, roughly 1 in 5 sees a real change to what the study claims to have found. Both bioRxiv and medRxiv display a "published in" link once a preprint's peer-reviewed version appears, which is the most direct way to check whether a specific preprint you're reading has been through that process yet — and, if it has, to read the version that actually survived it.
What we don't know yet
What the eventual publication rate looks like with more time. The PeerJ study's 5.7% figure reflects preprints checked soon after posting during an unprecedented publishing surge; some of the rest may have been published later, outside the study's window. The orthopaedic study's longer window and higher rate (52%) hints at this, but doesn't resolve it for the COVID-19 sample specifically.
Whether the 19% conclusion-change rate generalizes beyond COVID-19 critical-care therapies. That study examined a narrow, high-urgency slice of pandemic-era research. Whether peer review changes conclusions at a similar rate for slower-moving research areas — including most peptide and metabolic research — wasn't measured here.
Why specific preprints never get published. None of the three studies distinguish between preprints rejected after review, preprints never submitted anywhere, and preprints still working through a slow process at the time of measurement — very different outcomes a raw publication rate can't separate.
Whether peer review's changes make studies more accurate, or just more cautious. The Intensive Care Medicine study documents that conclusions changed; it doesn't independently verify which version — preprint or published — was closer to the truth.
Sources
- Añazco D, Nicolalde B, Espinosa I, et al. Publication Rate and Citation Counts for Preprints Released During the COVID-19 Pandemic: The Good, the Bad and the Ugly. PeerJ, 2021;9:e10927. DOI
- Rizk AA, Arza RA, Jella TK, et al. Characterization and Reach of Orthopaedic Research Posted to Preprint Servers: Are We "Undercooking" Our Science? Clinical Orthopaedics and Related Research, 2023;481(8):1491-1500. DOI
- Morin C, Padki A, Wong A, et al. Comparison of COVID-19 Preprint and Peer-Reviewed Versions of Studies on Therapies for Critically Ill Patients. Journal of Intensive Care Medicine, 2023;38(11):1060-1067. DOI
Article records maintained via PubMed. Peptide Proof reports what studies found. It does not provide medical advice, dosing guidance, or treatment recommendations.
Sources
- Publication Rate and Citation Counts for Preprints Released During the COVID-19 Pandemic: The Good, the Bad and the Ugly
- PeerJ, 2021 · n=5061 · DOI
- Characterization and Reach of Orthopaedic Research Posted to Preprint Servers: Are We \"Undercooking\" Our Science?
- Clinical Orthopaedics and Related Research, 2023 · n=1471 · DOI
- Comparison of COVID-19 Preprint and Peer-Reviewed Versions of Studies on Therapies for Critically Ill Patients
- Journal of Intensive Care Medicine, 2023 · n=21 · DOI
Get the evidence, not the hype.
New research briefs, plainly written and fully sourced. No dosing advice, ever.