Rubric Reliability and Annotation of Content and Argument in Source-Based Argument Essays
Yanjun Gao, Alex Driban, Brennan Xavier McManus, Elena Musi, Patricia Davies, Smaranda Muresan, Rebecca J. Passonneau
Abstract
We present a unique dataset of student source-based argument essays to facilitate research on the relations between content, argumentation skills, and assessment. Two classroom writing assignments were given to college students in a STEM major, accompanied by a carefully designed rubric. The paper presents a reliability study of the rubric, showing it to be highly reliable, and initial annotation on content and argumentation annotation of the essays.- Anthology ID:
- W19-4452
- Volume:
- Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications
- Month:
- August
- Year:
- 2019
- Address:
- Florence, Italy
- Venues:
- ACL | BEA | WS
- SIG:
- SIGEDU
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 507–518
- Language:
- URL:
- https://www.aclweb.org/anthology/W19-4452
- DOI:
- 10.18653/v1/W19-4452
- PDF:
- http://aclanthology.lst.uni-saarland.de/W19-4452.pdf