Hierarchically-Attentive RNN for Album Summarization and Storytelling

Licheng Yu, Mohit Bansal, Tamara Berg


Abstract
We address the problem of end-to-end visual storytelling. Given a photo album, our model first selects the most representative (summary) photos, and then composes a natural language story for the album. For this task, we make use of the Visual Storytelling dataset and a model composed of three hierarchically-attentive Recurrent Neural Nets (RNNs) to: encode the album photos, select representative (summary) photos, and compose the story. Automatic and human evaluations show our model achieves better performance on selection, generation, and retrieval than baselines.
Anthology ID:
D17-1101
Volume:
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Month:
September
Year:
2017
Address:
Copenhagen, Denmark
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
966–971
Language:
URL:
https://www.aclweb.org/anthology/D17-1101
DOI:
10.18653/v1/D17-1101
Bib Export formats:
BibTeX MODS XML EndNote
PDF:
http://aclanthology.lst.uni-saarland.de/D17-1101.pdf
Attachment:
 D17-1101.Attachment.pdf