Incorporating Source-Side Phrase Structures into Neural Machine Translation

Akiko Eriguchi, Kazuma Hashimoto, Yoshimasa Tsuruoka


Abstract
Neural machine translation (NMT) has shown great success as a new alternative to the traditional Statistical Machine Translation model in multiple languages. Early NMT models are based on sequence-to-sequence learning that encodes a sequence of source words into a vector space and generates another sequence of target words from the vector. In those NMT models, sentences are simply treated as sequences of words without any internal structure. In this article, we focus on the role of the syntactic structure of source sentences and propose a novel end-to-end syntactic NMT model, which we call a tree-to-sequence NMT model, extending a sequence-to-sequence model with the source-side phrase structure. Our proposed model has an attention mechanism that enables the decoder to generate a translated word while softly aligning it with phrases as well as words of the source sentence. We have empirically compared the proposed model with sequence-to-sequence models in various settings on Chinese-to-Japanese and English-to-Japanese translation tasks. Our experimental results suggest that the use of syntactic structure can be beneficial when the training data set is small, but is not as effective as using a bi-directional encoder. As the size of training data set increases, the benefits of using a syntactic tree tends to diminish.
Anthology ID:
J19-2003
Volume:
Computational Linguistics, Volume 45, Issue 2 - June 2019
Month:
June
Year:
2019
Address:
Venue:
CL
SIG:
Publisher:
Note:
Pages:
267–292
Language:
URL:
https://www.aclweb.org/anthology/J19-2003
DOI:
10.1162/coli_a_00348
Bib Export formats:
BibTeX MODS XML EndNote
PDF:
http://aclanthology.lst.uni-saarland.de/J19-2003.pdf