When observational studies and randomized trials disagree, the observational studies are not necessarily wrong — but only one of the designs can, by construction, isolate cause from circumstance. In an observational study, researchers record what people do and watch what happens; in a randomized experiment, they assign the exposure by chance. Randomization is the difference between a story and a controlled comparison, because it breaks the link between the treatment and everything else that distinguishes the people who receive it. That single mechanical difference is why a century of methodological work keeps returning to one conclusion: correlation is a clue, but assignment is the argument.
This is an explainer on research design; it does not offer medical or lifestyle advice.
What exactly does randomization buy?
People who take up a behavior differ from people who do not — in age, income, health consciousness, underlying risk. These are confounders: characteristics that cause both the exposure and the outcome, manufacturing apparent links. Observational analysts can adjust for the confounders they measured, but residual confounding persists for everything they missed or mismeasured, and confounders in health research are notoriously hard to measure. Randomization solves the problem structurally rather than statistically: with adequate sample size, chance alone distributes known and unknown confounders evenly across arms, so any outcome difference between groups traces — with known probabilities — to the assignment itself. No amount of clever modeling on observational data can reproduce that guarantee, because the guarantee is not statistical; it is about how the groups were formed.
What is the classic case of the designs diverging?
Hormone therapy after menopause. Through the 1990s, large observational cohorts consistently found that women taking estrogen had less heart disease, and prescribing followed. In 2002 the Women's Health Initiative, a randomized trial run by the National Heart, Lung, and Blood Institute, reported the opposite direction: the treatment increased risks of several cardiovascular outcomes, and the observational signal turned out to reflect confounding — women who chose hormone therapy were, on average, healthier, wealthier and better monitored than those who did not. The episode became the textbook demonstration that measured adjustment cannot always rescue self-selection, and it changed how guideline bodies weigh evidence tiers.
Does that make observational research useless?
No — and treating it as second-rate across the board is its own error. Randomization is frequently impossible or unethical: you cannot randomize smoking, poverty or toxic exposure. For many questions, observational data are the only evidence that will ever exist, and methods have grown far more disciplined: natural experiments that exploit quasi-random variation, instrumental variables, sibling and twin comparisons, negative controls, and preregistered analysis plans. Evidence frameworks such as GRADE grade observational evidence on its own scale rather than dismissing it. The honest hierarchy is question-dependent: a rigorous natural experiment can beat a sloppy trial, but when a clean randomized result and a naive observational association conflict, the tie usually goes to randomization.
Related stories: Randomization and Blinding: The Boring Machinery Behind Trustworthy Trials · It Worked in Mice: Why Animal Findings Rarely Reach the Clinic.
What tools do observational researchers use to fight back?
The modern toolkit is genuinely sophisticated. Mendelian randomization uses genetic variants — randomly assigned at conception by Mendelian segregation — as natural instruments to probe whether an exposure plausibly causes an outcome. Difference-in-differences designs compare trends across groups before and after a policy change, subtracting out shared background shifts. Regression discontinuity exploits arbitrary cutoffs: people just above and just below a threshold are nearly identical, so differences across the line can be attributed to the treatment. None of these is bulletproof — instruments can be weak, cutoffs can be gamed — but each converts some of the raw selection problem into something closer to the assignment logic of a trial. When several such designs, plus cohorts, converge on the same answer, the causal case becomes strong without a single randomized participant.
How can a reader tell how much a claim deserves?
Ask what was assigned by the researchers versus chosen by the subjects. If nothing was assigned, the finding is an association, and the paper's own limitations section should be discussing confounding. Check whether the analysis adjusted for obvious confounders and whether the authors ran sensitivity analyses for unmeasured ones. Look for dose-response patterns, timing, and consistency across independent designs — a correlation that appears across cohorts, countries and methods is more credible than a single cross-section, though still not proof. And be suspicious of the strongest language: well-trained epidemiologists rarely write that observational results 'prove' anything, and a paper that claims proof from association has outrun its own design. The habit costs nothing and filters out a large share of overconfident coverage before it reaches you.
How strong is all this evidence about evidence?
unusually well-documented. Beyond hormone therapy, researchers have repeatedly run paired comparisons — same question, observational and randomized designs — and found that observational estimates often match trials, but with enough divergent cases to justify caution, and the divergent cases cluster where self-selection is strongest. This is replicated, converging methodological knowledge, though the frequency of disagreement remains debated; meta-research does not get meta-replicated often, and honest practitioners say so.
What would settle future disagreements faster?
Trials with broader populations, larger natural experiments from policy changes, and registries that make confounder measurement better rather than merely more voluminous. For the reader, the durable habit is simple: when a headline says a habit is linked to an outcome, ask whether anyone was randomized — because until someone is, the honest summary is 'associated, possibly causal, confounding not excluded.'
