Enrique Amigó

Also published as: Enrique Amigo


2020

pdf bib
An Effectiveness Metric for Ordinal Classification: Formal Properties and Experimental Results
Enrique Amigo | Julio Gonzalo | Stefano Mizzaro | Jorge Carrillo-de-Albornoz
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

In Ordinal Classification tasks, items have to be assigned to classes that have a relative ordering, such as “positive”, “neutral”, “negative” in sentiment analysis. Remarkably, the most popular evaluation metrics for ordinal classification tasks either ignore relevant information (for instance, precision/recall on each of the classes ignores their relative ordering) or assume additional information (for instance, Mean Average Error assumes absolute distances between classes). In this paper we propose a new metric for Ordinal Classification, Closeness Evaluation Measure, that is rooted on Measurement Theory and Information Theory. Our theoretical analysis and experimental results over both synthetic data and data from NLP shared tasks indicate that the proposed metric captures quality aspects from different traditional tasks simultaneously. In addition, it generalizes some popular classification (nominal scale) and error minimization (interval scale) metrics, depending on the measurement scale in which it is instantiated.

2012

pdf bib
UNED: Improving Text Similarity Measures without Human Assessments
Enrique Amigó | Jesús Giménez | Julio Gonzalo | Felisa Verdejo
*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)

pdf bib
The Heterogeneity Principle in Evaluation Measures for Automatic Summarization
Enrique Amigó | Julio Gonzalo | Felisa Verdejo
Proceedings of Workshop on Evaluation Metrics and System Comparison for Automatic Summarization

2011

pdf bib
Corroborating Text Evaluation Results with Heterogeneous Measures
Enrique Amigó | Julio Gonzalo | Jesús Giménez | Felisa Verdejo
Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing

2009

pdf bib
The role of named entities in Web People Search
Javier Artiles | Enrique Amigó | Julio Gonzalo
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

pdf bib
The Contribution of Linguistic Features to Automatic Machine Translation Evaluation
Enrique Amigó | Jesús Giménez | Julio Gonzalo | Felisa Verdejo
Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP

pdf bib
The Impact of Query Refinement in the Web People Search Task
Javier Artiles | Julio Gonzalo | Enrique Amigó
Proceedings of the ACL-IJCNLP 2009 Conference Short Papers

2006

pdf bib
MT Evaluation: Human-Like vs. Human Acceptable
Enrique Amigó | Jesús Giménez | Julio Gonzalo | Lluís Màrquez
Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions

pdf bib
Iqmt: A Framework for Automatic Machine Translation Evaluation
Jesús Giménez | Enrique Amigó
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)

We present the IQMT Framework for Machine Translation Evaluation Inside QARLA. IQMT offers a common workbench in which existing evaluation metrics can be utilized and combined. It provides i) a measure to evaluate the quality of any set of similarity metrics (KING), ii) a measure to evaluate the quality of a translation using a set of similarity metrics (QUEEN), and iii) a measure to evaluate the reliability of a test set (JACK). The first release of the IQMT package is freely available for public use. Current version includes a set of 26 metrics from 7 different well-known metric families, and allows the user to supply its own metrics. For future releases, we are working on the design of new metrics that are able to capture linguistic aspects of translation beyond lexical ones.

2005

pdf bib
QARLA: A Framework for the Evaluation of Text Summarization Systems
Enrique Amigó | Julio Gonzalo | Anselmo Peñas | Felisa Verdejo
Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05)

pdf bib
Evaluating DUC 2004 Tasks with the QARLA Framework
Enrique Amigó | Julio Gonzalo | Anselmo Peñas | Felisa Verdejo
Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization

2004

pdf bib
An Empirical Study of Information Synthesis Task
Enrique Amigo | Julio Gonzalo | Victor Peinado | Anselmo Peñas | Felisa Verdejo
Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)

pdf bib
Using syntactic information to extract relevant terms for multi-document summarization
Enrique Amigó | Julio Gonzalo | Víctor Peinado | Anselmo Peñas | Felisa Verdejo
COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics