Soylent: a word processor with a crowd inside

Citation: Michael S. Bernstein, Greg Little, Robert C. Miller, Björn Hartmann, Mark S. Ackerman, David R. Karger, David Crowell, Katrina Panovich (2010) Soylent: a word processor with a crowd inside. Proceedings of the 23nd annual ACM symposium on User interface software and technology - UIST '10 (RSS)
DOI (original publisher): 10.1145/1866029.1866078
Semantic Scholar (metadata): 10.1145/1866029.1866078
Sci-Hub (fulltext): 10.1145/1866029.1866078
Internet Archive Scholar (search for fulltext): Soylent: a word processor with a crowd inside
Download: http://portal.acm.org/citation.cfm?doid=1866029.1866078
Tagged: Computer Science (RSS)

Summary

This article proposes and implements a crowdwork design pattern (Find-Fix-Verify)m targeting editing tasks: text shortening, proofreading, and repetitive edits. Bernstein et al integrates this tool with Microsoft Word, and share their experience measuring the quality of the results they receive as well as the associated time and task cost. They observe that the results were valuable and largely successful, but also uneven and needed author review (especially when coordination among ideas from multiple Turkers was needed due to conflicts in how to resolve a given issue, which was not possible due to the pattern).

Theoretical and Practical Relevance

This article offers multiple practical strategies for engaging with crowdworkers and explains a valuable design pattern that might be used in a wide variety of contexts. This article is also an example of a particularly high-quality empirical systems paper: it invokes some amount of theory, and demonstrates a series of interconnected activities; it makes a theoretical contribution, describes the work clearly and concisely, and offers empirical findings; it achieved a best paper award at UIST.