Dynamic Language Models for Streaming Text

Dani Yogatama, Chong Wang, Bryan R. Routledge, Noah A. Smith, Eric P. Xing


Abstract
We present a probabilistic language model that captures temporal dynamics and conditions on arbitrary non-linguistic context features. These context features serve as important indicators of language changes that are otherwise difficult to capture using text data by itself. We learn our model in an efficient online fashion that is scalable for large, streaming data. With five streaming datasets from two different genres—economics news articles and social media—we evaluate our model on the task of sequential language modeling. Our model consistently outperforms competing models.
Anthology ID:
Q14-1015
Volume:
Transactions of the Association for Computational Linguistics, Volume 2
Month:
Year:
2014
Address:
Venue:
TACL
SIG:
Publisher:
Note:
Pages:
181–192
Language:
URL:
https://www.aclweb.org/anthology/Q14-1015
DOI:
10.1162/tacl_a_00175
Bib Export formats:
BibTeX MODS XML EndNote
PDF:
http://aclanthology.lst.uni-saarland.de/Q14-1015.pdf