Learning Structured Text Representations

Yang Liu, Mirella Lapata


Abstract
In this paper, we focus on learning structure-aware document representations from data without recourse to a discourse parser or additional annotations. Drawing inspiration from recent efforts to empower neural networks with a structural bias (Cheng et al., 2016; Kim et al., 2017), we propose a model that can encode a document while automatically inducing rich structural dependencies. Specifically, we embed a differentiable non-projective parsing algorithm into a neural model and use attention mechanisms to incorporate the structural biases. Experimental evaluations across different tasks and datasets show that the proposed model achieves state-of-the-art results on document modeling tasks while inducing intermediate structures which are both interpretable and meaningful.
Anthology ID:
Q18-1005
Volume:
Transactions of the Association for Computational Linguistics, Volume 6
Month:
Year:
2018
Address:
Venue:
TACL
SIG:
Publisher:
Note:
Pages:
63–75
Language:
URL:
https://www.aclweb.org/anthology/Q18-1005
DOI:
10.1162/tacl_a_00005
Bib Export formats:
BibTeX MODS XML EndNote
PDF:
http://aclanthology.lst.uni-saarland.de/Q18-1005.pdf
Video:
 https://vimeo.com/276396538