Katrin Kirchhoff


2020

pdf bib
Masked Language Model Scoring
Julian Salazar | Davis Liang | Toan Q. Nguyen | Katrin Kirchhoff
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

Pretrained masked language models (MLMs) require finetuning for most NLP tasks. Instead, we evaluate MLMs out of the box via their pseudo-log-likelihood scores (PLLs), which are computed by masking tokens one by one. We show that PLLs outperform scores from autoregressive language models like GPT-2 in a variety of tasks. By rescoring ASR and NMT hypotheses, RoBERTa reduces an end-to-end LibriSpeech model’s WER by 30% relative and adds up to +1.7 BLEU on state-of-the-art baselines for low-resource translation pairs, with further gains from domain adaptation. We attribute this success to PLL’s unsupervised expression of linguistic acceptability without a left-to-right bias, greatly improving on scores from GPT-2 (+10 points on island effects, NPI licensing in BLiMP). One can finetune MLMs to give scores without masking, enabling computation in a single inference pass. In all, PLLs and their associated pseudo-perplexities (PPPLs) enable plug-and-play use of the growing number of pretrained MLMs; e.g., we use a single cross-lingual model to rescore translations in multiple languages. We release our library for language model scoring at https://github.com/awslabs/mlm-scoring.

pdf bib
Robust Prediction of Punctuation and Truecasing for Medical ASR
Monica Sunkara | Srikanth Ronanki | Kalpit Dixit | Sravan Bodapati | Katrin Kirchhoff
Proceedings of the First Workshop on Natural Language Processing for Medical Conversations

Automatic speech recognition (ASR) systems in the medical domain that focus on transcribing clinical dictations and doctor-patient conversations often pose many challenges due to the complexity of the domain. ASR output typically undergoes automatic punctuation to enable users to speak naturally, without having to vocalize awkward and explicit punctuation commands, such as “period”, “add comma” or “exclamation point”, while truecasing enhances user readability and improves the performance of downstream NLP tasks. This paper proposes a conditional joint modeling framework for prediction of punctuation and truecasing using pretrained masked language models such as BERT, BioBERT and RoBERTa. We also present techniques for domain and task specific adaptation by fine-tuning masked language models with medical domain data. Finally, we improve the robustness of the model against common errors made in ASR by performing data augmentation. Experiments performed on dictation and conversational style corpora show that our proposed model achieves 5% absolute improvement on ground truth text and 10% improvement on ASR outputs over baseline models under F1 metric.

2019

pdf bib
Simple, Fast, Accurate Intent Classification and Slot Labeling for Goal-Oriented Dialogue Systems
Arshit Gupta | John Hewitt | Katrin Kirchhoff
Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue

With the advent of conversational assistants, like Amazon Alexa, Google Now, etc., dialogue systems are gaining a lot of traction, especially in industrial setting. These systems typically consist of Spoken Language understanding component which, in turn, consists of two tasks - Intent Classification (IC) and Slot Labeling (SL). Generally, these two tasks are modeled together jointly to achieve best performance. However, this joint modeling adds to model obfuscation. In this work, we first design framework for a modularization of joint IC-SL task to enhance architecture transparency. Then, we explore a number of self-attention, convolutional, and recurrent models, contributing a large-scale analysis of modeling paradigms for IC+SL across two datasets. Finally, using this framework, we propose a class of ‘label-recurrent’ models that otherwise non-recurrent, with a 10-dimensional representation of the label history, and show that our proposed systems are easy to interpret, highly accurate (achieving over 30% error reduction in SL over the state-of-the-art on the Snips dataset), as well as fast, at 2x the inference and 2/3 to 1/2 the training time of comparable recurrent models, thus giving an edge in critical real-world systems.

2018

pdf bib
Context Models for OOV Word Translation in Low-Resource Languages
Angli Liu | Katrin Kirchhoff
Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)

2016

pdf bib
Unsupervised Resolution of Acronyms and Abbreviations in Nursing Notes Using Document-Level Context Models
Katrin Kirchhoff | Anne M. Turner
Proceedings of the Seventh International Workshop on Health Text Mining and Information Analysis

2015

pdf bib
Morphological Modeling for Machine Translation of English-Iraqi Arabic Spoken Dialogs
Katrin Kirchhoff | Yik-Cheung Tam | Colleen Richey | Wen Wang
Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2014

pdf bib
Submodularity for Data Selection in Machine Translation
Katrin Kirchhoff | Jeff Bilmes
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)

2013

pdf bib
Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Lucy Vanderwende | Hal Daumé III | Katrin Kirchhoff
Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

pdf bib
Using Document Summarization Techniques for Speech Data Subset Selection
Kai Wei | Yuzong Liu | Katrin Kirchhoff | Jeff Bilmes
Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2012

pdf bib
Evaluating User Preferences in Machine Translation Using Conjoint Analysis
Katrin Kirchhoff | Daniel Capurro | Anne Turner
Proceedings of the 16th Annual conference of the European Association for Machine Translation

2010

pdf bib
Hand Gestures in Disambiguating Types of You Expressions in Multiparty Meetings
Tyler Baldwin | Joyce Chai | Katrin Kirchhoff
Proceedings of the SIGDIAL 2010 Conference

pdf bib
Contextual Modeling for Meeting Translation Using Unsupervised Word Sense Disambiguation
Mei Yang | Katrin Kirchhoff
Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)

2009

pdf bib
Graph-based Learning for Statistical Machine Translation
Andrei Alexandrescu | Katrin Kirchhoff
Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics

2008

pdf bib
Beyond Log-Linear Models: Boosted Minimum Error Rate Training for N-best Re-ranking
Kevin Duh | Katrin Kirchhoff
Proceedings of ACL-08: HLT, Short Papers

pdf bib
The University of Washington Machine Translation System for ACL WMT 2008
Amittai Axelrod | Mei Yang | Kevin Duh | Katrin Kirchhoff
Proceedings of the Third Workshop on Statistical Machine Translation

2007

pdf bib
Data-Driven Graph Construction for Semi-Supervised Graph-Based Learning in NLP
Andrei Alexandrescu | Katrin Kirchhoff
Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference

2006

pdf bib
Lexicon Acquisition for Dialectal Arabic Using Transductive Learning
Kevin Duh | Katrin Kirchhoff
Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing

pdf bib
Phrase-Based Backoff Models for Machine Translation of Highly Inflected Languages
Mei Yang | Katrin Kirchhoff
11th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
Factored Neural Language Models
Andrei Alexandrescu | Katrin Kirchhoff
Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers

2005

pdf bib
Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing
Raymond Mooney | Chris Brew | Lee-Feng Chien | Katrin Kirchhoff
Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing

pdf bib
The Vocal Joystick: A Voice-Based Human-Computer Interface for Individuals with Motor Impairments
Jeff A. Bilmes | Xiao Li | Jonathan Malkin | Kelley Kilanski | Richard Wright | Katrin Kirchhoff | Amar Subramanya | Susumu Harada | James Landay | Patricia Dowden | Howard Chizeck
Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing

pdf bib
POS Tagging of Dialectal Arabic: A Minimally Supervised Approach
Kevin Duh | Katrin Kirchhoff
Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages

pdf bib
Improved Language Modeling for Statistical Machine Translation
Katrin Kirchhoff | Mei Yang
Proceedings of the ACL Workshop on Building and Using Parallel Texts

2004

pdf bib
Automatic Learning of Language Model Structure
Kevin Duh | Katrin Kirchhoff
COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics

pdf bib
Automatic Diacritization of Arabic for Acoustic Modeling in Speech Recognition
Dimitra Vergyri | Katrin Kirchhoff
Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages

2003

pdf bib
Directions For Multi-Party Human-Computer Interaction Research
Katrin Kirchhoff | Mari Ostendorf
Proceedings of the HLT-NAACL 2003 Workshop on Research Directions in Dialogue Processing

pdf bib
Factored Language Models and Generalized Parallel Backoff
Jeff A. Bilmes | Katrin Kirchhoff
Companion Volume of the Proceedings of HLT-NAACL 2003 - Short Papers