Pretraining Sentiment Classifiers with Unlabeled Dialog Data

Toru Shimizu, Nobuyuki Shimizu, Hayato Kobayashi


Abstract
The huge cost of creating labeled training data is a common problem for supervised learning tasks such as sentiment classification. Recent studies showed that pretraining with unlabeled data via a language model can improve the performance of classification models. In this paper, we take the concept a step further by using a conditional language model, instead of a language model. Specifically, we address a sentiment classification task for a tweet analysis service as a case study and propose a pretraining strategy with unlabeled dialog data (tweet-reply pairs) via an encoder-decoder model. Experimental results show that our strategy can improve the performance of sentiment classifiers and outperform several state-of-the-art strategies including language model pretraining.
Anthology ID:
P18-2121
Volume:
Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
July
Year:
2018
Address:
Melbourne, Australia
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
764–770
Language:
URL:
https://www.aclweb.org/anthology/P18-2121
DOI:
10.18653/v1/P18-2121
Bib Export formats:
BibTeX MODS XML EndNote
PDF:
http://aclanthology.lst.uni-saarland.de/P18-2121.pdf
Note:
 P18-2121.Notes.pdf
Video:
 https://vimeo.com/285806152
Presentation:
 P18-2121.Presentation.pdf