Research Literacy
What Is a Systematic Review, and Why Does It Outrank a Single Study?
The short answer
A single 1999 trial of 11,324 heart attack survivors found that fish-oil supplements cut the combined risk of death, another heart attack, or stroke by up to 15%, with cardiovascular death itself down 30% in the trial's four-way analysis. That result helped shape both clinical guidelines and a consumer supplement industry. When a 2020 Cochrane systematic review pooled 86 randomized trials and 162,796 participants asking the same question, cardiovascular mortality fell by only 8% (a much smaller, though still real, effect), and all-cause mortality showed no significant change at all. Neither result was fraudulent or wrong on its own terms — the single trial reported an honest finding from one population, in one era, on one draw of chance. Pooling every trial that asked the same question is what let researchers see whether that finding held up.
Quick answers
- What is it?
- A systematic review pools every comparable trial on a question using a pre-registered protocol and statistically combines their results, rather than relying on any one study's findings alone.
- Why are people interested in it?
- A single dramatic-sounding trial often makes headlines, and readers need to know why researchers trust a pooled review of many trials more than that one striking result.
- What has actually been studied?
- The 1999 GISSI-Prevenzione trial (11,324 heart attack survivors) testing fish oil, compared against a 2020 Cochrane systematic review pooling 86 randomized trials and 162,796 participants asking the same question.
- How strong is the evidence?
- A clear, well-documented contrast: the single trial found large benefits (up to 30% lower cardiovascular death), while the pooled 86-trial review found a much smaller, still-real benefit for cardiovascular death (8%) and no significant effect on all-cause mortality.
- What don't we know yet?
- Whether the original large effect reflected a real signal that shrank with more data or a chance finding from one well-timed trial, how much of the gap owes to changes in background cardiac care over the decades between trials, and whether an even larger future pool of trials would find the effect stabilize, shrink further, or disappear.
- Where can I read the original research?
- Sources at the bottom of this page: GISSI-Prevenzione Investigators, The Lancet, 1999; and Abdelhamid AS, Brown TJ, Brainard JS, et al., Cochrane Database of Systematic Reviews, 2020.
The short answer: A single randomized trial found that fish-oil supplements cut the risk of death, heart attack, or stroke after a heart attack — a result that helped shape both medical guidelines and a supplement industry. When researchers later pooled every comparable trial into a systematic review, the same category of benefit was still present, but much smaller, and less certain. Neither result was wrong. The single trial reported what happened in one population, in one era of medicine, on one roll of the statistical dice. A systematic review exists specifically to check whether that roll comes up the same way again.
What the research shows
One trial, a striking result
In 1999, the Lancet published the GISSI-Prevenzione trial: 11,324 people who had survived a heart attack within the previous three months, randomized in a two-by-two design to fish oil (omega-3 polyunsaturated fatty acids, 2,836 patients), vitamin E (2,830), both (2,830), or neither (2,828), and followed for 3.5 years. The results were notable. Fish oil lowered the risk of the trial's primary combined endpoint — death, non-fatal heart attack, or stroke — by 10% in the standard two-way analysis (95% CI 1–18%) and by 15% in a four-way analysis that isolated the fish-oil-only group (95% CI 2–26%). The effect on cardiovascular death specifically was larger still: down 17% in the two-way analysis and 30% in the four-way analysis (95% CI 13–44%). Vitamin E showed no benefit at all. This was one trial, published once, and it landed hard — feeding directly into cardiology guidelines recommending omega-3 supplementation after a heart attack, and into a consumer fish-oil market that suddenly had a randomized trial behind it.
What a systematic review does differently
A systematic review is not just a summary of what's been published. It follows a pre-registered protocol, searches multiple databases for every trial that addressed the same question — not just the well-known ones — applies the same inclusion and exclusion criteria to all of them, formally assesses each trial's risk of bias, and, when the trials are similar enough, statistically pools their results into a single combined estimate (a meta-analysis), weighting larger and more precise trials more heavily. The question a systematic review asks is different from the question a single trial answers. A trial asks: what happened in this specific group of people, treated this way, during this stretch of time? A systematic review asks: does this result hold up across every trial anyone has run on this question?
The pooled evidence gives a smaller, hedgier answer
Twenty-one years after GISSI, a 2020 Cochrane systematic review updated the evidence base: 86 randomized controlled trials, 162,796 total participants, trial durations from 12 to 88 months, 28 of them rated at low risk of bias. Pooled across that full body of evidence, all-cause mortality showed a relative risk of 0.97 (95% CI 0.93–1.01; 143,693 participants; 45 trials; rated high-certainty evidence) — a precise estimate close enough to "no effect" that the researchers could say so with confidence, not a vague or underpowered null. Cardiovascular mortality specifically came in at relative risk 0.92 (95% CI 0.86–0.99; 117,837 participants; moderate-certainty evidence) — an 8% reduction, real, but far short of GISSI's 30%. Overall cardiovascular events were not significantly reduced (relative risk 0.96, 95% CI 0.92–1.01). The one outcome that stayed statistically favorable was coronary heart disease death specifically: relative risk 0.90 (95% CI 0.81–1.00), a number the review's authors translated into a number needed to treat of 334 — meaning 334 people would need to take omega-3 supplements for the review's studied duration to prevent a single coronary death. That surviving benefit was rated low-certainty, not high.
Why pooling narrows the answer
A systematic review reduces several specific risks a lone trial cannot address on its own. Chance: any single trial is one draw from a range of possible results, and pooling many independent trials of the same question narrows the confidence interval, making an isolated statistical fluke less likely to survive. Population specificity: GISSI enrolled Italian adults recovering from a recent heart attack in the mid-1990s; 86 trials run across different countries, decades, doses, and patient groups test whether an effect generalizes, rather than confirming it only worked in the specific group that happened to be enrolled. Publication bias: a well-run systematic review explicitly searches for null and unpublished trials rather than relying on whichever results made it into print. And design idiosyncrasy: any one trial can carry a specific quirk — how a comparison group was assembled, how an outcome was defined — that only becomes visible when many different designs are compared side by side.
None of this makes single trials worthless. Nearly every new medical finding starts as a single trial, and a well-run randomized trial remains far stronger evidence than an observational study. But a systematic review sits above any individual trial it includes for a specific, checkable reason: it exists to test whether that trial's result reproduces, not simply to report that it happened once.
What we don't know yet
Whether the original GISSI effect reflected a real biological signal that shrank with more data, or a chance finding from one well-timed trial. The pooled review cannot fully distinguish between these possibilities after the fact — it can only report that the larger effect size was not reproduced at that magnitude.
How much of the gap owes to changes in background cardiac care between the mid-1990s and the trials run since. Medical practice around heart attack recovery has evolved substantially over the decades separating these trials; none of the reviewed evidence isolates how much of the shrinking effect reflects that shift, as opposed to chance or differences in the populations studied.
Why even the pooled evidence disagrees with itself at the edges. The coronary-heart-disease-specific mortality benefit stayed statistically significant while broader all-cause and cardiovascular-event categories did not, and the review's own certainty rating for that surviving benefit is low, not high.
Whether a still larger, more diverse pool of trials would find the effect stabilize, shrink further, or disappear. This review is itself a revision of earlier Cochrane reviews on the same question; it is unlikely to be the final word.
Sources
- GISSI-Prevenzione Investigators (Gruppo Italiano per lo Studio della Sopravvivenza nell'Infarto miocardico). Dietary supplementation with n-3 polyunsaturated fatty acids and vitamin E after myocardial infarction: results of the GISSI-Prevenzione trial. Lancet, 1999;354(9177):447-455. PubMed
- Abdelhamid AS, Brown TJ, Brainard JS, et al. Omega-3 fatty acids for the primary and secondary prevention of cardiovascular disease. Cochrane Database of Systematic Reviews, 2020;3(3):CD003177. DOI
Article records maintained via PubMed. Peptide Proof reports what studies found. It does not provide medical advice, dosing guidance, or treatment recommendations.
Sources
- Dietary supplementation with n-3 polyunsaturated fatty acids and vitamin E after myocardial infarction: results of the GISSI-Prevenzione trial
- Lancet, 1999 · n=11324
- Omega-3 fatty acids for the primary and secondary prevention of cardiovascular disease
- Cochrane Database of Systematic Reviews, 2020 · n=162796 · DOI
Get the evidence, not the hype.
New research briefs, plainly written and fully sourced. No dosing advice, ever.