When dozens of studies address the same question, a meta-analysis pools their results into a single estimate — often displayed as a forest plot, a diagram of horizontal lines where each line is one study and a diamond marks the combined answer. Done well, pooling raises statistical power and averages out the noise of small samples. Done carelessly, it averages biased studies into a confident-sounding number that is more misleading than any one of them. The method's credibility therefore rests less on the final diamond than on the pipeline that produced it, and methodologists have spent forty years codifying that pipeline.
This is an explainer about research methodology, not medical guidance; treatment decisions belong with a clinician.
What separates a systematic review from an ordinary review?
A narrative review is an expert's essay; the selection of studies is invisible to the reader. A systematic review commits in advance to a written protocol: which databases will be searched, with which keywords, over which date range, and — critically — which studies will be excluded and why. Teams then screen thousands of titles, usually with two independent reviewers, and record reasons for exclusion study by study. The point is auditability. A reader who distrusts the conclusion can retrace every step. Registration in a public registry such as PROSPERO, and reporting standards like PRISMA, were created because reviews written without these commitments drifted toward the studies their authors already believed.
How does pooling actually work?
The core move is a weighted average. Each study contributes an effect estimate — say, the difference in blood-pressure reduction between a treatment and control group — and studies with more precise estimates, typically because they enrolled more participants, get more weight. The output is a pooled effect with a confidence interval. Heterogeneity statistics describe how much the studies disagree; when disagreement is high, many analysts now report the pooled number with visible reluctance or not at all, because averaging over genuinely different populations or interventions can produce a number that describes none of them.
Where does the strength of evidence come from?
Three checks carry most of the weight, and each is a body of replicated methodological work rather than a single study. The first is publication-bias assessment: because journals prefer positive results, the published literature may be a skewed sample of all research conducted. Funnel plots and related tests look for the telltale asymmetry this produces; the cautionary example is that early meta-analyses of some drugs looked strong until unpublished trials surfaced and pulled estimates toward zero. The second is risk-of-bias appraisal — grading each included study on blinding, allocation and attrition, and sometimes re-running the analysis with weaker studies removed to see if the answer survives. The third is sensitivity analysis: does the pooled estimate hold when slightly different inclusion rules are applied? Evidence-grading frameworks such as GRADE, now used by major guideline bodies, combine these checks into an explicit rating from high down to very low.
Related stories: Preprints Are Not Unpublished Results. But They Are Not Proven Either. · Correlation Is Not Causation: How Study Design Decides the Argument.
Why did the method earn its reputation?
The case for pooling was made most famously in cardiology. In 1985, a meta-analysis of randomized trials of streptokinase after heart attack, led by Salim Yusuf and colleagues, found a survival benefit from the combined data even though several individual trials were individually inconclusive; larger trials later confirmed the effect, and the therapy became standard. The episode became a textbook argument for meta-analysis: when single trials are too small to detect a real but moderate effect, the structured synthesis of what already exists can get there years earlier. The reverse lesson came later, when meta-analyses built on small trials in other fields overstated effects that large follow-up studies then shrank. Both directions teach the same thing — the method extracts whatever signal the underlying studies contain, and no more.
Can garbage really go in and come out polished?
Yes, and the phrase 'garbage in, garbage out' is used by meta-analysts themselves. Pooling cannot fix a literature of weak observational studies; it can only make their combined uncertainty explicit. If every included trial suffered from unblinded outcome assessment, the diamond in the forest plot inherits that flaw at higher resolution. This is why the same field can host two meta-analyses reaching opposite conclusions — different inclusion criteria can load the pool with different studies — and why serious papers report how their conclusions change under alternative rules.
How should a reader judge one?
Check four things in order. Was the protocol registered before the search began? Is the study selection flow, usually a PRISMA diagram, complete enough to retrace? Does the paper grade the risk of bias in included studies rather than merely listing them? And do the authors address heterogeneity and publication bias, or silently assume they are absent? A meta-analysis that clears all four belongs near the top of the evidence hierarchy — above any single trial, because it is the structured synthesis of many. One that skips them is an essay with statistics attached.
What would confirm or overturn a pooled result?
The strongest meta-analyses are updated. When new trials accumulate, a living systematic review recomputes the diamond; pooled estimates have shifted meaningfully — sometimes enough to reverse recommendations — after large new trials arrived. For readers, the honest summary is this: a meta-analysis is not the end of a question, it is the current best-weighted position of all the evidence gathered so far, and its authority is exactly as good as the transparency of its assembly.
