Luciana Benotti


2020

pdf bib
They Are Not All Alike: Answering Different Spatial Questions Requires Different Grounding Strategies
Alberto Testoni | Claudio Greco | Tobias Bianchi | Mauricio Mazuecos | Agata Marcante | Luciana Benotti | Raffaella Bernardi
Proceedings of the Third International Workshop on Spatial Language Understanding

In this paper, we study the grounding skills required to answer spatial questions asked by humans while playing the GuessWhat?! game. We propose a classification for spatial questions dividing them into absolute, relational, and group questions. We build a new answerer model based on the LXMERT multimodal transformer and we compare a baseline with and without visual features of the scene. We are interested in studying how the attention mechanisms of LXMERT are used to answer spatial questions since they require putting attention on more than one region simultaneously and spotting the relation holding among them. We show that our proposed model outperforms the baseline by a large extent (9.70% on spatial questions and 6.27% overall). By analyzing LXMERT errors and its attention mechanisms, we find that our classification helps to gain a better understanding of the skills required to answer different spatial questions.

pdf bib
On the role of effective and referring questions in GuessWhat?!
Mauricio Mazuecos | Alberto Testoni | Raffaella Bernardi | Luciana Benotti
Proceedings of the First Workshop on Advances in Language and Vision Research

Task success is the standard metric used to evaluate referential visual dialogue systems. In this paper we propose two new metrics that evaluate how each question contributes to the goal. First, we measure how effective each question is by evaluating whether the question discards objects that are not the referent. Second, we define referring questions as those that univocally identify one object in the image. We report the new metrics for human dialogues and for state of the art publicly available models on GuessWhat?!. Regarding our first metric, we find that successful dialogues do not have a higher percentage of effective questions for most models. With respect to the second metric, humans make questions at the end of the dialogue that are referring, confirming their guess before guessing. Human dialogues that use this strategy have a higher task success but models do not seem to learn it.

bib
Effective questions in referential visual dialogue
Mauricio Mazuecos | Alberto Testoni | Raffaella Bernardi | Luciana Benotti
Proceedings of the The Fourth Widening Natural Language Processing Workshop

An interesting challenge for situated dialogue systems is referential visual dialog: by asking questions, the system has to identify the referent to which the user refers to. Task success is the standard metric used to evaluate these systems. However, it does not consider how effective each question is, that is how much each question contributes to the goal. We propose a new metric, that measures question effectiveness. As a preliminary study, we report the new metric for state of the art publicly available models on GuessWhat?!. Surprisingly, successful dialogues do not have a higher percentage of effective questions than failed dialogues. This suggests that a system with high task success is not necessarily one that generates good questions.

2018

pdf bib
Modeling Student Response Times: Towards Efficient One-on-one Tutoring Dialogues
Luciana Benotti | Jayadev Bhaskaran | Sigtryggur Kjartansson | David Lang
Proceedings of the 2018 EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text

In this paper we investigate the task of modeling how long it would take a student to respond to a tutor question during a tutoring dialogue. Solving such a task has applications in educational settings such as intelligent tutoring systems, as well as in platforms that help busy human tutors to keep students engaged. Knowing how long it would normally take a student to respond to different types of questions could help tutors optimize their own time while answering multiple dialogues concurrently, as well as deciding when to prompt a student again. We study this problem using data from a service that offers tutor support for math, chemistry and physics through an instant messaging platform. We create a dataset of 240K questions. We explore several strong baselines for this task and compare them with human performance.

2015

pdf bib
Zoom: a corpus of natural language descriptions of map locations
Romina Altamirano | Thiago Ferreira | Ivandré Paraboni | Luciana Benotti
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)

2014

pdf bib
A Natural Language Instructor for pedestrian navigation based in generation by selection
Santiago Avalos | Luciana Benotti
Proceedings of the EACL 2014 Workshop on Dialogue in Motion

pdf bib
Mining human interactions to construct a virtual guide for a virtual fair
Andrés Luna | Luciana Benotti
Proceedings of the EACL 2014 Workshop on Dialogue in Motion

2012

pdf bib
Corpus-based Interpretation of Instructions in Virtual Environments
Luciana Benotti | Martín Villalba | Tessa Lau | Julián Cerruti
Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

pdf bib
Probabilistic Refinement Algorithms for the Generation of Referring Expressions
Romina Altamirano | Carlos Areces | Luciana Benotti
Proceedings of COLING 2012: Posters

2011

pdf bib
Giving instructions in virtual environments by corpus based selection
Luciana Benotti | Alexandre Denis
Proceedings of the SIGDIAL 2011 Conference

pdf bib
The GIVE-2.5 C Generation System
David Nicolás Racca | Luciana Benotti | Pablo Duboue
Proceedings of the 13th European Workshop on Natural Language Generation

pdf bib
CL system: Giving instructions by corpus based selection
Luciana Benotti | Alexandre Denis
Proceedings of the 13th European Workshop on Natural Language Generation

pdf bib
Prototyping virtual instructors from human-human corpora
Luciana Benotti | Alexandre Denis
Proceedings of the ACL-HLT 2011 System Demonstrations

2010

pdf bib
Dialogue Systems for Virtual Environments
Luciana Benotti | Paula Estrella | Carlos Areces
Proceedings of the NAACL HLT 2010 Young Investigators Workshop on Computational Approaches to Languages of the Americas

pdf bib
Negotiating causal implicatures
Luciana Benotti | Patrick Blackburn
Proceedings of the SIGDIAL 2010 Conference

2009

pdf bib
A computational account of comparative implicatures for a spoken dialogue agent
Luciana Benotti | David Traum
Proceedings of the Eight International Conference on Computational Semantics

pdf bib
Clarification Potential of Instructions
Luciana Benotti
Proceedings of the SIGDIAL 2009 Conference

pdf bib
Frolog: an Accommodating Text-Adventure Game
Luciana Benotti
Proceedings of the Demonstrations Session at EACL 2009