// NATURE NEWS — SPAZIO & SCIENZA
Reply to: Dataset artefacts can partially drive the measured decline in disruption
Nature
volume 656, pages E14–E21 (2026) Cite this article
The Original Article was published on 12 August 2026
replying to: V. Holst et al. Nature https://doi.org/10.1038/s41586-026-10787-y (2026).
In the accompanying Comment1, Holst et al. claim that the decline in disruptiveness that we documented in Park et al.2 is an artefact of including works that do not cite any references (that is, have zero backward citations). Using the dataset, metric and method advocated by Holst et al.1, we find declines equivalent to benchmark transformations in science. Their own regression model—designed to address their concerns about works with zero references—yields large, significant declines for papers and patents (P < 0.01), a result that is presented in their supplementary tables yet left unaddressed, despite directly contradicting their central claim. Their critique is further undermined by severe quality issues in their data, which contain three times as many works with zero references as our data. We trace this excess to their inclusion of at least 2.8 million editorials, obituaries and comments, 1.5 million books and proceedings and 254,000 product and artistic reviews. Twenty per cent of their sample is non-research content that almost by definition lacks references. Simple keyword searches highlight the problem’s severity, identifying among others 456 For Dummies guides, 50 Dr. Seuss and Curious George books, and the Captain Underpants series, all without references, in their sample. Applying granular document-type classification reveals that non-research content decreased from 40% to 8% of their sample between 1945 and 2010—a shift that is sufficient to generate the decline in works with zero references that they attribute to metadata errors in our study. Standard practice excludes such content to safeguard against the metadata quality concerns at the centre of their critique—concerns that their dataset exemplifies rather than addresses. Their supplementary Web of Science analysis, framed as a direct replication of our study, rests on shifting and impossible accounts of its methodology. Declining disruptiveness has been documented in nearly 100 studies across databases, metrics and non-citation-based measures3. The evidence does not support an artefact-based explanation.
This is a preview of subscription content, access via your institution
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
Prices may be subject to local taxes which are calculated during checkout
This study uses data from Funk et al. (https://doi.org/10.5281/zenodo.7258379), SciSciNet (https://doi.org/10.1038/s41597-023-02198-9), Dimensions.ai (https://docs.dimensions.ai/bigquery/) and Li et al. (https://doi.org/10.7910/DVN/6NJ5RN).
Our code builds on Funk et al. (https://doi.org/10.5281/zenodo.7258379); additional scripts will be made available upon request.