Science Is Shaped by Wikipedia: Evidence From a Randomized Control Trial

Citation: Neil Thompson, Douglas Hanley (2018) Science Is Shaped by Wikipedia: Evidence From a Randomized Control Trial.
Internet Archive Scholar (search for fulltext): Science Is Shaped by Wikipedia: Evidence From a Randomized Control Trial
Download: https://ssrn.com/abstract=3039505
Tagged:

Summary

Test "several of the implications about public goods in science, in particular (i) is Wikipedia used to inform academic research, (ii) does it generate demand for the underlying papers, and (iii) are those with less access to journals more likely to benefit."

Data:

• complete edit history of Wikipedia
• ull-text version of all articles since 1995 across more than 2,061 Elsevier journals (representing the scientific literature)
• citations to academic journals from Web of Science
• a set of Wikipedia articles created as part of the randomized control experiment

(i)

Look at word similarity in scientific articles in windows 3 months before article creation, and 3-6 months after article creation (an article typically expands in its first 3 months).

Hypothesis: "if Wikipedia has an impact on the progression of the literature, science published after the creation of the Wikipedia article will bear a closer similarity to the article than the science published before it did."

Result suggests "the effect on the scientific literature from a Wikipedia article is similar, but weaker, to the effect from a review article."

Also do an experiment, creating new Wikipedia articles about chemistry and econometrics, publishing half of them, and holding back the other half as a control. Similarity of literature published after article creation becomes more similar to published article, dissimilar to control (latter reflecting expected word drift).

(ii)

Look at average monthly citations in the 2 year windows before and after experimental intervention. Finds positive effect for all tested specification, but only some are statistically significant, providing suggestive but not conclusive evidence that referencing a scientific article in Wikipedia generates more citations.

(iii)

Finds those with more access to closed literature benefit more from Wikipedia references, presumably because many of the references are to closed literature.

Also suggests policy interventions (eg grant conditions) could encourage development of public resources for science such as scientific articles in Wikipedia, or separate repositories such as the Stanford Encyclopedia of Philosophy, and suggests a way of estimating the costs vs benefits of such interventions.