Research Literacy
What 'Statistically Significant' Actually Means
The short answer
A p-value answers one narrow question: if the treatment truly did nothing, how likely is a gap this large between groups to appear by chance alone? In the SELECT trial, 17,604 people with cardiovascular disease and overweight or obesity were randomized to semaglutide or placebo; a major cardiovascular event occurred in 6.5% of the semaglutide group versus 8.0% of placebo (P<0.001) — a result very unlikely to be chance. But a later subgroup analysis of the same trial found a 34% relative risk reduction in a smaller subgroup that wasn't statistically significant at all (P=0.11), while a 26% reduction in a larger subgroup was (P=0.0004). The bigger-looking number was the less certain one — significance tracks precision, not just the size of an effect.
Quick answers
- What is it?
- "Statistically significant" means a p-value crossed a conventional threshold, answering a narrow question: if the treatment truly did nothing, how likely is a gap this large to appear by chance alone?
- Why are people interested in it?
- The phrase gets used constantly in health headlines as if it means "proven" or "big," and readers need to know it actually measures something else.
- What has actually been studied?
- The SELECT trial (17,604 people) and its p=0.001 primary result, plus a subgroup analysis of the same trial where a smaller, more dramatic-looking 34% risk reduction failed to reach significance while a smaller, less dramatic 26% reduction did.
- How strong is the evidence?
- Grounded in a single large, well-run randomized trial and its prespecified subgroup analyses — a clear illustration rather than a broad evidence review.
- What don't we know yet?
- Whether the trial's headline result is clinically important enough to matter for a given person, whether a non-significant subgroup finding reflects a smaller true effect or just a small sample, and how much running multiple subgroup analyses itself raises the chance of a false-positive finding.
- Where can I read the original research?
- Sources at the bottom of this page: Lincoff AM, Brown-Frandsen K, Colhoun HM, et al., New England Journal of Medicine, 2023; and Meyhöfer SM, Cariou B, Cercato C, et al., Nature Medicine, 2026.
The short answer: A p-value answers one narrow question: if the treatment truly did nothing, how likely is a gap this large between groups to appear by chance alone? In the SELECT trial, 17,604 people with cardiovascular disease and overweight or obesity were randomized to semaglutide or placebo; a major cardiovascular event occurred in 6.5% of the semaglutide group versus 8.0% of placebo (P<0.001) — a result very unlikely to be chance. But a later subgroup analysis of the same trial found a 34% relative risk reduction in a smaller subgroup that wasn't statistically significant at all (P=0.11), while a 26% reduction in a larger subgroup was (P=0.0004). The bigger-looking number was the less certain one — significance tracks precision, not just the size of an effect.
What the research shows
A significant result means the gap probably isn't noise
According to PubMed, the SELECT trial (Semaglutide Effects on Heart Disease and Stroke in Patients With Overweight or Obesity), published in the New England Journal of Medicine in 2023, randomized 17,604 adults with existing cardiovascular disease and a BMI of 27 or higher, but no diabetes, to weekly semaglutide or placebo. Over a mean follow-up of 39.8 months, a primary cardiovascular event — cardiovascular death, nonfatal heart attack, or nonfatal stroke — occurred in 569 of 8,803 patients (6.5%) on semaglutide and 701 of 8,801 patients (8.0%) on placebo (hazard ratio 0.80, 95% CI 0.72-0.90, P<0.001). (DOI) That p-value doesn't mean semaglutide is "99.9% likely to work," and it isn't a measure of how big the benefit is. It means that if semaglutide truly had no effect on cardiovascular risk, seeing a gap this size in a trial this large would be extremely unlikely to happen by chance alone.
The size of an effect and its statistical certainty are two different things
A 2026 secondary analysis of the SELECT trial, published in Nature Medicine, tested whether the cardiovascular benefit held up in patients at higher risk for liver fibrosis, using three ways of defining that subgroup by Fibrosis-4 score. In the broadest subgroup (score ≥1.3), the reduction was 26% (hazard ratio 0.74, 95% CI 0.63-0.88, P=0.0004) — clearly significant. In an age-adjusted subgroup, it was 21% (hazard ratio 0.79, 95% CI 0.63-0.98, P=0.035) — still significant, though closer to the line. In the narrowest subgroup — any age, Fibrosis-4 score above 2.67 — the reduction looked biggest of all, 34% (hazard ratio 0.66), yet the confidence interval ran from 0.39 to 1.10, and the result was not statistically significant (P=0.11). (DOI) The subgroup with the most dramatic-looking number was also the smallest, and the smallest subgroup produced the least precise estimate.
Why a bigger number can be a less certain one
This isn't a contradiction — it's how the statistics are supposed to work. Confidence intervals widen as sample size shrinks, because there's less data to pin down where the true effect actually sits. A hazard ratio of 0.66 sounds more impressive than 0.74, but its 95% confidence interval (0.39 to 1.10) means the study couldn't rule out results ranging from a 61% risk reduction to a 10% risk increase. The 0.74 estimate, from a larger group, had a much narrower range (0.63 to 0.88), and every value in that range still points toward benefit. Significance isn't a verdict on how large or how real an effect is — it's a statement about how much the data can pin that effect down.
The same pattern shows up on a continuous measure too
A separate prespecified SELECT substudy tracked a blood marker of inflammation, high-sensitivity CRP (hsCRP), across the full 17,604-patient trial. Baseline hsCRP was similar between groups (geometric mean 1.96 mg/L on semaglutide vs 1.91 mg/L on placebo), and semaglutide reduced it by 37.8% at 104 weeks — a reduction present across every baseline-risk subgroup and detectable as early as four to eight weeks, before most of the weight loss had occurred. (DOI) The point here isn't the biology; it's the statistics. Run across the trial's full sample size, even a continuous, noisy lab value like hsCRP produced a precise, tightly bounded estimate. Precision follows sample size and consistency of effect — it doesn't care whether the outcome is a hard endpoint like a heart attack or a soft one like an inflammatory marker.
Significance is a threshold, not a wall
It's also worth noticing how close some of these p-values sit to the conventional 0.05 cutoff used to call a result "significant." The age-adjusted subgroup's P=0.035 cleared that line; the narrowest subgroup's P=0.11 didn't. Nothing biologically different separates a result at P=0.04 from one at P=0.06 — the line itself is a convention, not a law of nature. Treating "significant" and "not significant" as a hard yes-or-no on whether an effect is real, rather than as a continuous measure of how well the data pin it down, is one of the most common ways trial results get overinterpreted.
What we don't know yet
Whether the trial's headline result is clinically important enough to matter for a given person. A 20% relative reduction in major cardiovascular events being statistically real doesn't by itself say how much weight it should carry against other considerations — that's a judgment a p-value can't supply.
Whether the non-significant subgroup finding reflects a smaller true effect, a similar effect obscured by a small sample, or both. Distinguishing those possibilities would need a larger, dedicated study in that specific population, which wasn't located in this search.
How much the multiple-subgroup design itself affects how these results should be read. Testing a trial's data across several different subgroup definitions raises the chance that at least one comparison turns up "significant" by chance alone — a well-documented statistical hazard that applies to any subgroup finding, favorable or not, more than it applies to a trial's single pre-specified primary result.
Sources
- Lincoff AM, Brown-Frandsen K, Colhoun HM, et al. Semaglutide and Cardiovascular Outcomes in Obesity without Diabetes. New England Journal of Medicine, 2023;389(24):2221-2232. DOI
- Meyhöfer SM, Cariou B, Cercato C, et al. Semaglutide on liver fibrosis and heart outcomes in patients at high risk of liver fibrosis: a prespecified analysis of the SELECT randomized trial. Nature Medicine, 2026;32(5):1686-1693. DOI
Article records maintained via PubMed. Peptide Proof reports what studies found. It does not provide medical advice, dosing guidance, or treatment recommendations.
| Evidence Index: semaglutide
Sources
- Semaglutide and Cardiovascular Outcomes in Obesity without Diabetes
- New England Journal of Medicine, 2023 · n=17604 · DOI
- Semaglutide on liver fibrosis and heart outcomes in patients at high risk of liver fibrosis: a prespecified analysis of the SELECT randomized trial
- Nature Medicine, 2026 · n=17604 · DOI
- Effect of Semaglutide on the Inflammatory Biomarker High-Sensitivity CRP in Patients With Established Cardiovascular Disease and Overweight or Obesity in SELECT: A Prespecified Secondary Analysis
- Circulation, 2026 · n=17604 · DOI
Get the evidence, not the hype.
New research briefs, plainly written and fully sourced. No dosing advice, ever.