Task-based evaluation of summary quality: Describing relationships between scientific papers
This paper concerns the task of determining how a scientific paper relates to previous work it describes and cites. It uses this task to evaluate summaries generated from based on the rhetorical status.
6 groups (with a total of 24 subjects) and 6 experiments were used, where each task was limited to 10 minutes to ensure completion in a 1 hour time frame.
This is part of the 'argumentation zoning' project of Simone Teufel's thesis. See also An annotation scheme for discourse-level argumentation in research articles (which introduces the zones), Discourse-level argumentation in scientific articles: Human and automatic annotation, and What's yours and what's mine: Determining intellectual attribution in scientific text.