Reproducibility vs. Replicability: A Brief History of a Confused Terminology

Citation: Hans E. Plesser (2017) Reproducibility vs. Replicability: A Brief History of a Confused Terminology. Frontiers in Neuroinformatics (RSS)
DOI (original publisher): 10.3389/fninf.2017.00076
Semantic Scholar (metadata): 10.3389/fninf.2017.00076
Sci-Hub (fulltext): 10.3389/fninf.2017.00076
Internet Archive Scholar (search for fulltext): Reproducibility vs. Replicability: A Brief History of a Confused Terminology
Download: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5778115/
Tagged: reproducibility (RSS)

Summary

• Jon Claerbout defines reproducibility as running same software on same inputs, getting same outputs in Electronic documents give reproducible research a new meaning.
• Pre-existing scientific tradition defines
  • Repeatability: the exact same procedure and exact same results.
  • Reproducibility: is testing the same hypothesis with variations in the unspecified part of the method.
• ACM adopted:
  • Repeatability: Same team, same experimental setup
  • Replicability: Different team, same experimental setup
  • Reproducibility: Different team, different experimental setup
• Three years after this was published, ACM changed their tune to be more consistent with metrology [1]:
  • Reproducibility: Different team, same experimental setup
  • Replicability: Different team, different experimental setup
• Goodman lexicon resolves the ambiguity.

Goodman	Claerbout	ACM (unknown date <2017)	ACM 2020
		Repeatability	Repeatability
Methods reproducibility	Reproducibility	Replicability	Reproducibility
Results reproducibility	Replicability	Reproducibility	Replicability
Inferential reproducibility