Frank Lin


2006

pdf bib
Modular Approach to Error Analysis and Evaluation for Multilingual Question Answering
Hideki Shima | Mengqiu Wang | Frank Lin | Teruko Mitamura
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)

Multilingual Question Answering systems are generally very complex, integrating several sub-modules to achieve their result. Global metrics (such as average precision and recall) are insufficient when evaluating the performance of individual sub-modules and their influence on each other. In this paper, we present a modular approach to error analysis and evaluation; we use manually-constructed, gold-standard input for each module to obtain an upper-bound for the (local) performance of that module. This approach enables us to identify existing problem areas quickly, and to target improvements accordingly.

pdf bib
Keyword Translation Accuracy and Cross-Lingual Question Answering inChinese and Japanese
Teruko Mitamura | Mengqiu Wang | Hideki Shima | Frank Lin
Proceedings of the Workshop on Multilingual Question Answering - MLQA ‘06