The journal impact factor — the average number of citations a journal's recent articles receive — was invented in the 1960s as a librarian's tool for choosing which journals to buy. It became, instead, the number researchers were hired, funded, and promoted by. The backlash is now institutional. The San Francisco Declaration on Research Assessment (DORA), launched in 2012, had gathered more than 25,000 individual and organizational signatories by 2025, and in 2023 the European University Association and the European Commission launched COARA, the Coalition for Advancing Research Assessment, whose members commit to reforming how they evaluate researchers. As of 2025, COARA counted more than 600 member organizations, most of them European universities and funders.
This is an explainer on the debate over research metrics; institutions set their own hiring criteria.
What is wrong with the number itself?
Three technical criticisms dominate. First, the distribution problem: citations follow a power law, so a journal's average describes almost no individual paper. Nature's own editors noted in 2005 that some 89% of the impact factor's value came from a minority of articles — meaning most published work in a high-impact journal is cited below its journal's headline number. Second, opacity: Thomson Reuters (now Clarivate) calculates the metric from proprietary data, and studies comparing Web of Science counts with Scopus found impact factors differing by meaningful amounts depending on source. Third, gaming: citations to front-matter, review-article inflation, and coerced citations — editors pressuring authors to cite the journal — have each been documented in publishing-ethics literature.
What is the human cost of using it?
Reviews, grants, and jobs keyed to journal prestige push researchers toward fashionable, low-risk, positive-result topics in high-impact venues. The 2012 San Francisco meeting that produced DORA began from cell biology: researchers argued that where a paper appeared said little about its quality and skewed career decisions against careful, slower work. Surveys of researchers across Europe conducted during the COARA rollout found broad agreement that publication counts and journal metrics dominate evaluations — and broad dissatisfaction with that state.
Related stories: Preprints Changed Who Controls Scientific Priority · Transformative Agreements: One Contract for Reading and Publishing.
How is the impact factor calculated, and who owns it?
The mechanics are simple and the machinery is not. Each June, Clarivate computes the metric as citations in one year to items a journal published in the prior two, divided by the number of citable items in those two years. Everything depends on what counts as a citable item: editorials, letters, and news pieces are typically excluded from the denominator while their citations still count in the numerator, a structure that mechanically rewards journals publishing front matter that attracts citations. The data come from Clarivate's proprietary Web of Science index, and the calculation has never been independently audited, which is the governance criticism beneath the statistical ones. Clarivate disputes the significance of these choices, and the metric remains the most-used journal-level number in the world regardless, which is precisely why reformers target evaluation practice rather than the formula itself.
What did funders actually change?
Funder policy is where the reform has teeth. Requirements that applicants list publications as contributions without journal branding, review panels instructed to assess the paper rather than the venue, and CV templates that remove journal-impact fields are all live changes in European systems. In the United States, momentum has been slower, though a 2022 White House office of science policy memorandum inviting agencies to reconsider evaluation practices gave reformers a federal hook. The gap between signature and behavior is the measurable thing: audits of review-panel documentation in several countries still find journal names weighted heavily in written assessments, which is why COARA members report on implementation rather than only signing pledges, and why skeptics within the reform movement describe the current phase as the test of whether declarations alter deliberation.
What are the alternatives actually in use?
Reformers advocate narrative and structural evidence: annotated CVs describing an applicant's real contributions, statements of open science practice, peer review of full portfolios rather than counts. Some funders have acted — the Dutch ZonMw and UK's Wellcome published policies telling reviewers to ignore journal brands, and in 2021 more than 40 heads of German institutions signed the "Berlin Statement" committing to stop using the impact factor in hiring. Quantitative alternatives exist — h-index, field-normalized citation percentiles from Leiden's CWTS, article-level metrics — but COARA's position is deliberately careful: replacing one flawed number with another is not reform, so the coalition emphasizes qualitative assessment supported, not replaced, by data.
Has anything actually changed?
Slowly, at the edges. Some European universities now run annotated-CV pilots; DORA's signatories include major funders such as the Howard Hughes Medical Institute. But Clarivate still publishes the impact factor annually, and a 2024 analysis of hiring criteria across countries found journal metrics still embedded in most templates. The durable measure of progress: whether a strong paper in a modest journal can win a professorship. In most systems, it cannot yet.
