On the Importance of Diversity in Question Generation for QA

Md Arafat Sultan, Shubham Chandel, Ramón Fernandez Astudillo, Vittorio Castelli


Abstract
Automatic question generation (QG) has shown promise as a source of synthetic training data for question answering (QA). In this paper we ask: Is textual diversity in QG beneficial for downstream QA? Using top-p nucleus sampling to derive samples from a transformer-based question generator, we show that diversity-promoting QG indeed provides better QA training than likelihood maximization approaches such as beam search. We also show that standard QG evaluation metrics such as BLEU, ROUGE and METEOR are inversely correlated with diversity, and propose a diversity-aware intrinsic measure of overall QG quality that correlates well with extrinsic evaluation on QA.
Anthology ID:
2020.acl-main.500
Volume:
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Month:
July
Year:
2020
Address:
Online
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5651–5656
Language:
URL:
https://www.aclweb.org/anthology/2020.acl-main.500
DOI:
10.18653/v1/2020.acl-main.500
Bib Export formats:
BibTeX MODS XML EndNote
PDF:
http://aclanthology.lst.uni-saarland.de/2020.acl-main.500.pdf
Video:
 http://slideslive.com/38929093