Aggregation Driven Progression System for GWAPs

Osman Doruk Kicikoglu, Richard Bartle, Jon Chamberlain, Silviu Paun, Massimo Poesio


Abstract
As the uses of Games-With-A-Purpose (GWAPs) broadens, the systems that incorporate its usages have expanded in complexity. The types of annotations required within the NLP paradigm set such an example, where tasks can involve varying complexity of annotations. Assigning more complex tasks to more skilled players through a progression mechanism can achieve higher accuracy in the collected data while acting as a motivating factor that rewards the more skilled players. In this paper, we present the progression technique implemented in Wormingo , an NLP GWAP that currently includes two layers of task complexity. For the experiment, we have implemented four different progression scenarios on 192 players and compared the accuracy and engagement achieved with each scenario.
Anthology ID:
2020.gamnlp-1.11
Volume:
Workshop on Games and Natural Language Processing
Month:
May
Year:
2020
Address:
Marseille, France
Venues:
GAMESandNLP | LREC | WS
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
79–84
Language:
English
URL:
https://www.aclweb.org/anthology/2020.gamnlp-1.11
DOI:
Bib Export formats:
BibTeX MODS XML EndNote
PDF:
http://aclanthology.lst.uni-saarland.de/2020.gamnlp-1.11.pdf