Berlin Chen


2019

pdf bib
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling
Alex Wang | Jan Hula | Patrick Xia | Raghavendra Pappagari | R. Thomas McCoy | Roma Patel | Najoung Kim | Ian Tenney | Yinghui Huang | Katherin Yu | Shuning Jin | Berlin Chen | Benjamin Van Durme | Edouard Grave | Ellie Pavlick | Samuel R. Bowman
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

Natural language understanding has recently seen a surge of progress with the use of sentence encoders like ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2019) which are pretrained on variants of language modeling. We conduct the first large-scale systematic study of candidate pretraining tasks, comparing 19 different tasks both as alternatives and complements to language modeling. Our primary results support the use language modeling, especially when combined with pretraining on additional labeled-data tasks. However, our results are mixed across pretraining tasks and show some concerning trends: In ELMo’s pretrain-then-freeze paradigm, random baselines are worryingly strong and results vary strikingly across target tasks. In addition, fine-tuning BERT on an intermediate task often negatively impacts downstream transfer. In a more positive trend, we see modest gains from multitask training, suggesting the development of more sophisticated multitask and transfer learning techniques as an avenue for further research.

2018

pdf bib
會議語音辨識使用語者資訊之語言模型調適技術 (On the Use of Speaker-Aware Language Model Adaptation Techniques for Meeting Speech Recognition ) [In Chinese]
Ying-wen Chen | Tien-hong Lo | Hsiu-jui Chang | Wei-Cheng Chao | Berlin Chen
Proceedings of the 30th Conference on Computational Linguistics and Speech Processing (ROCLING 2018)

pdf bib
探討聲學模型的合併技術與半監督鑑別式訓練於會議語音辨識之研究 (Investigating acoustic model combination and semi-supervised discriminative training for meeting speech recognition) [In Chinese]
Tien-Hong Lo | Berlin Chen
Proceedings of the 30th Conference on Computational Linguistics and Speech Processing (ROCLING 2018)

pdf bib
探討鑑別式訓練聲學模型之類神經網路架構及優化方法的改進 (Discriminative Training of Acoustic Models Leveraging Improved Neural Network Architecture and Optimization Method) [In Chinese]
Wei-Cheng Chao | Hsiu-Jui Chang | Tien-Hong Lo | Berlin Chen
Proceedings of the 30th Conference on Computational Linguistics and Speech Processing (ROCLING 2018)

pdf bib
探索結合快速文本及卷積神經網路於可讀性模型之建立 (Exploring Combination of FastText and Convolutional Neural Networks for Building Readability Models) [In Chinese]
Hou-Chiang Tseng | Berlin Chen | Yao-Ting Sung
Proceedings of the 30th Conference on Computational Linguistics and Speech Processing (ROCLING 2018)

2017

pdf bib
探究不同領域文件之可讀性分析 (Exploring Readability Analysis on Multi-Domain Texts) [In Chinese]
Hou-Chiang Tseng | Yao-Ting Sung | Berlin Chen
Proceedings of the 29th Conference on Computational Linguistics and Speech Processing (ROCLING 2017)

pdf bib
使用查詢意向探索與類神經網路於語音文件檢索之研究 (Exploring Query Intent and Neural Network modeling Techniques for Spoken Document Retrieval) [In Chinese]
Tien-Hong Lo | Ying-Wen Chen | Berlin Chen | Kuan-Yu Chen | Hsin-Min Wang
Proceedings of the 29th Conference on Computational Linguistics and Speech Processing (ROCLING 2017)

pdf bib
序列標記與配對方法用於語音辨識錯誤偵測及修正 (On the Use of Sequence Labeling and Matching Methods for ASR Error Detection and Correction) [In Chinese]
Chia-Hua Wu | Chun-I Tsai | Hsiao-Tsung Hung | Yu-Chen Kao | Berlin Chen
Proceedings of the 29th Conference on Computational Linguistics and Speech Processing (ROCLING 2017)

pdf bib
當代非監督式方法之比較於節錄式語音摘要 (An Empirical Comparison of Contemporary Unsupervised Approaches for Extractive Speech Summarization) [In Chinese]
Shih-Hung Liu | Kuan-Yu Chen | Kai-Wun Shih | Berlin Chen | Hsin-Min Wang | Wen-Lian Hsu
International Journal of Computational Linguistics & Chinese Language Processing, Volume 22, Number 1, June 2017

pdf bib
語音文件檢索使用類神經網路技術 (On the Use of Neural Network Modeling Techniques for Spoken Document Retrieval) [In Chinese]
Tien-Hong Lo | Ying-Wen Chen | Kuan-Yu Chen | Hsin-Min Wang | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 22, Number 2, December 2017-Special Issue on Selected Papers from ROCLING XXIX

pdf bib
探究使用基於類神經網路之特徵於文本可讀性分類 (Exploring the Use of Neural Network based Features for Text Readability Classification) [In Chinese]
Hou-Chiang Tseng | Berlin Chen | Yao-Ting Sung
International Journal of Computational Linguistics & Chinese Language Processing, Volume 22, Number 2, December 2017-Special Issue on Selected Papers from ROCLING XXIX

2016

pdf bib
Learning to Distill: The Essence Vector Modeling Framework
Kuan-Yu Chen | Shih-Hung Liu | Berlin Chen | Hsin-Min Wang
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers

In the context of natural language processing, representation learning has emerged as a newly active research subject because of its excellent performance in many applications. Learning representations of words is a pioneering study in this school of research. However, paragraph (or sentence and document) embedding learning is more suitable/reasonable for some tasks, such as sentiment classification and document summarization. Nevertheless, as far as we are aware, there is only a dearth of research focusing on launching unsupervised paragraph embedding methods. Classic paragraph embedding methods infer the representation of a given paragraph by considering all of the words occurring in the paragraph. Consequently, those stop or function words that occur frequently may mislead the embedding learning process to produce a misty paragraph representation. Motivated by these observations, our major contributions are twofold. First, we propose a novel unsupervised paragraph embedding method, named the essence vector (EV) model, which aims at not only distilling the most representative information from a paragraph but also excluding the general background information to produce a more informative low-dimensional vector representation for the paragraph. We evaluate the proposed EV model on benchmark sentiment classification and multi-document summarization tasks. The experimental results demonstrate the effectiveness and applicability of the proposed embedding method. Second, in view of the increasing importance of spoken content processing, an extension of the EV model, named the denoising essence vector (D-EV) model, is proposed. The D-EV model not only inherits the advantages of the EV model but also can infer a more robust representation for a given spoken paragraph against imperfect speech recognition. The utility of the D-EV model is evaluated on a spoken document summarization task, confirming the effectiveness of the proposed embedding method in relation to several well-practiced and state-of-the-art summarization methods.

pdf bib
評估尺度相關最佳化方法於華語錯誤發音檢測之研究(Evaluation Metric-related Optimization Methods for Mandarin Mispronunciation Detection) [In Chinese]
Yao-Chi Hsu | Ming-Han Yang | Hsiao-Tsung Hung | Yi-Ju Lin | Berlin Chen
Proceedings of the 28th Conference on Computational Linguistics and Speech Processing (ROCLING 2016)

pdf bib
融合多任務學習類神經網路聲學模型訓練於會議語音辨識之研究(Leveraging Multi-task Learning with Neural Network Based Acoustic Modeling for Improved Meeting Speech Recognition) [In Chinese]
Ming-Han Yang | Yao-Chi Hsu | Hsiao-Tsung Hung | Ying-Wen Chen | Berlin Chen | Kuan-Yu Chen
Proceedings of the 28th Conference on Computational Linguistics and Speech Processing (ROCLING 2016)

pdf bib
使用字典學習法於強健性語音辨識(The Use of Dictionary Learning Approach for Robustness Speech Recognition) [In Chinese]
Bi-Cheng Yan | Chin-Hong Shih | Shih-Hung Liu | Berlin Chen
Proceedings of the 28th Conference on Computational Linguistics and Speech Processing (ROCLING 2016)

pdf bib
運用序列到序列生成架構於重寫式自動摘要(Exploiting Sequence-to-Sequence Generation Framework for Automatic Abstractive Summarization)[In Chinese]
Yu-Lun Hsieh | Shih-Hung Liu | Kuan-Yu Chen | Hsin-Min Wang | Wen-Lian Hsu | Berlin Chen
Proceedings of the 28th Conference on Computational Linguistics and Speech Processing (ROCLING 2016)

pdf bib
基於深層類神經網路及表示學習技術之文件可讀性分類(Classification of Text Readability Based on Deep Neural Network and Representation Learning Techniques)[In Chinese]
Hou-Chiang Tseng | Hsiao-Tsung Hung | Yao-Ting Sung | Berlin Chen
Proceedings of the 28th Conference on Computational Linguistics and Speech Processing (ROCLING 2016)

pdf bib
使用字典學習法於強健性語音辨識 (The Use of Dictionary Learning Approach for Robustness Speech Recognition) [In Chinese]
Bi-Cheng Yan | Chin-Hong Shih | Shih-Hung Liu | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 21, Number 2, December 2016

pdf bib
評估尺度相關最佳化方法於華語錯誤發音檢測之研究 (Evaluation Metric-related Optimization Methods for Mandarin Mispronunciation Detection) [In Chinese]
Yao-Chi Hsu | Ming-Han Yang | Hsiao-Tsung Hung | Yi-Ju Lin | Kuan-Yu Chen | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 21, Number 2, December 2016

pdf bib
融合多任務學習類神經網路聲學模型訓練於會議語音辨識之研究 (Leveraging Multi-Task Learning with Neural Network Based Acoustic Modeling for Improved Meeting Speech Recognition) [In Chinese]
Ming-Han Yang | Yao-Chi Hsu | Hsiao-Tsung Hung | Ying-Wen Chen | Kuan-Yu Chen | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 21, Number 2, December 2016

2015

pdf bib
表示法學習技術於節錄式語音文件摘要之研究(A Study on Representation Learning Techniques for Extractive Spoken Document Summarization) [In Chinese]
Kai-Wun Shih | Berlin Chen | Kuan-Yu Chen | Shih-Hung Liu | Hsin-Min Wang
Proceedings of the 27th Conference on Computational Linguistics and Speech Processing (ROCLING 2015)

pdf bib
使用詞向量表示與概念資訊於中文大詞彙連續語音辨識之語言模型調適(Exploring Word Embedding and Concept Information for Language Model Adaptation in Mandarin Large Vocabulary Continuous Speech Recognition) [In Chinese]
Ssu-Cheng Chen | Kuan-Yu Chen | Hsiao-Tsung Hung | Berlin Chen
Proceedings of the 27th Conference on Computational Linguistics and Speech Processing (ROCLING 2015)

pdf bib
可讀性預測於中小學國語文教科書及優良課外讀物之研究(A Study of Readability Prediction on Elementary and Secondary Chinese Textbooks and Excellent Extracurricular Reading Materials) [In Chinese]
Yi-Nian Liu | Kuan-Yu Chen | Hou-Chiang Tseng | Berlin Chen
Proceedings of the 27th Conference on Computational Linguistics and Speech Processing (ROCLING 2015)

pdf bib
調變頻譜分解之改良於強健性語音辨識(Several Refinements of Modulation Spectrum Factorization for Robust Speech Recognition) [In Chinese]
Ting-Hao Chang | Hsiao-Tsung Hung | Kuan-Yu Chen | Hsin-Min Wang | Berlin Chen
Proceedings of the 27th Conference on Computational Linguistics and Speech Processing (ROCLING 2015)

pdf bib
融合多種深層類神經網路聲學模型與分類技術於華語錯誤發音檢測之研究(Exploring Combinations of Various Deep Neural Network based Acoustic Models and Classification Techniques for Mandarin Mispro-nunciation Detection)[In Chinese]
Yao-Chi Hsu | Ming-Han Yang | Hsiao-Tsung Hung | Yuwen Hsiung | Yao-Ting Hung | Berlin Chen
Proceedings of the 27th Conference on Computational Linguistics and Speech Processing (ROCLING 2015)

pdf bib
節錄式語音文件摘要使用表示法學習技術 (Extractive Spoken Document Summarization with Representation Learning Techniques) [In Chinese]
Kai-Wun Shih | Kuan-Yu Chen | Shih-Hung Liu | Hsin-Min Wang | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 20, Number 2, December 2015 - Special Issue on Selected Papers from ROCLING XXVII

pdf bib
調變頻譜分解技術於強健語音辨識之研究 (Investigating Modulation Spectrum Factorization Techniques for Robust Speech Recognition) [In Chinese]
Ting-Hao Chang | Hsiao-Tsung Hung | Kuan-Yu Chen | Hsin-Min Wang | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 20, Number 2, December 2015 - Special Issue on Selected Papers from ROCLING XXVII

2014

pdf bib
運用概念模型化技術於中文大詞彙連續語音辨識之語言模型調適 (Leveraging Concept Modeling Techniques for Language Model Adaptation in Mandarin Large Vocabulary Continuous Speech Recognition) [In Chinese]
Po-Han Hao | Su-Cheng Chen | Berlin Chen
Proceedings of the 26th Conference on Computational Linguistics and Speech Processing (ROCLING 2014)

pdf bib
探究新穎語句模型化技術於節錄式語音摘要 (Investigating Novel Sentence Modeling Techniques for Extractive Speech Summarization) [In Chinese]
Shih-Hung Liu | Kuan-Yu Chen | Yu-Lun Hsieh | Berlin Chen | Hsin-Min Wang | Wen-Lian Hsu
Proceedings of the 26th Conference on Computational Linguistics and Speech Processing (ROCLING 2014)

pdf bib
使用概念資訊於中文大詞彙連續語音辨識之研究 (Exploring Concept Information for Mandarin Large Vocabulary Continuous Speech Recognition) [In Chinese]
Po-Han Hao | Ssu-Cheng Chen | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 19, Number 4, December 2014 - Special Issue on Selected Papers from ROCLING XXVI

pdf bib
Leveraging Effective Query Modeling Techniques for Speech Recognition and Summarization
Kuan-Yu Chen | Shih-Hung Liu | Berlin Chen | Ea-Ee Jan | Hsin-Min Wang | Wen-Lian Hsu | Hsin-Hsi Chen
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)

2013

pdf bib
改良語句模型技術於節錄式語音摘要之研究 (Improved Sentence Modeling Techniques for Extractive Speech Summarization) [In Chinese]
Shih-Hung Liu | Kuan-Yu Chen | Hsin-Min Wang | Wen-Lian Hsu | Berlin Chen
Proceedings of the 25th Conference on Computational Linguistics and Speech Processing (ROCLING 2013)

pdf bib
改良調變頻譜統計圖等化法於強健性語音辨識之研究 (Improved Modulation Spectrum Histogram Equalization for Robust Speech Recognition) [In Chinese]
Yu-Chen Kao | Berlin Chen
Proceedings of the 25th Conference on Computational Linguistics and Speech Processing (ROCLING 2013)

2012

pdf bib
改良式統計圖等化法強鍵性語音辨識之研究 (Improved Histogram Equalization Methods for Robust Speech Recognition) [In Chinese]
Hsin-Ju Hsieh | Jeih-weih Hung | Berlin Chen
Proceedings of the 24th Conference on Computational Linguistics and Speech Processing (ROCLING 2012)

pdf bib
遞迴式類神經網路語言模型應用額外資訊於語音辨識之研究 (Recurrent Neural Network-based Language Modeling with Extra Information Cues for Speech Recognition) [In Chinese]
Bang-Xuan Huang | Hank Hao | Menphis Chen | Berlin Chen
Proceedings of the 24th Conference on Computational Linguistics and Speech Processing (ROCLING 2012)

pdf bib
A Comparative Study of Methods for Topic Modeling in Spoken Document Retrieval
Shih-Hsiang Lin | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 17, Number 1, March 2012

pdf bib
語音辨識使用統計圖等化方法 (Speech Recognition Leveraging Histogram Equalization Methods) [In Chinese]
Hsin-Ju Hsieh | Jeih-weih Hung | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 17, Number 4, December 2012-Special Issue on Selected Papers from ROCLING XXIV

2011

pdf bib
An Effective and Robust Framework for Transliteration Exploration
Ea-Ee Jan | Niyu Ge | Shih-Hsiang Lin | Berlin Chen
Proceedings of 5th International Joint Conference on Natural Language Processing

pdf bib
實證探究多種鑑別式語言模型於語音辨識之研究 (Empirical Comparisons of Various Discriminative Language Models for Speech Recognition) [In Chinese]
Min-Hsuan Lai | Bang-Xuan Huang | Kuan-Yu Chen | Berlin Chen
Proceedings of the 23rd Conference on Computational Linguistics and Speech Processing (ROCLING 2011)

pdf bib
機率式調變頻譜分解於強健性語音辨識 (Probabilistic Modulation Spectrum Factorization for Robust Speech Recognition) [In Chinese]
Wen-Yi Chu | Yu-Chen Kao | Berlin Chen | Jeih-Weih Hung
ROCLING 2011 Poster Papers

2010

pdf bib
A Risk Minimization Framework for Extractive Speech Summarization
Shih-Hsiang Lin | Berlin Chen
Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics

pdf bib
鑑別式語言模型於語音辨識結果重新排序之研究 (Exploiting Discriminative Language Models for Reranking Speech Recognition Hypotheses) [In Chinese]
Chia-Wen Liu | Shih-Hsiang Lin | Berlin Chen
Proceedings of the 22nd Conference on Computational Linguistics and Speech Processing (ROCLING 2010)

pdf bib
整合邊際資訊於鑑別式聲學模型訓練方法之比較研究 (A Comparative Study on Margin-Based Discriminative Training of Acoustic Models) [In Chinese]
Yueng-Tien Lo | Berlin Chen
Proceedings of the 22nd Conference on Computational Linguistics and Speech Processing (ROCLING 2010)

2009

pdf bib
相似度比率式鑑別分析應用於大詞彙連續語音辨識 (Likelihood Ratio Based Discriminant Analysis for Large Vocabulary Continuous Speech Recognition) [In Chinese]
Hung-Shin Lee | Berlin Chen
Proceedings of the 21st Conference on Computational Linguistics and Speech Processing

pdf bib
主題語言模型於大詞彙連續語音辨識之研究 (On the Use of Topic Models for Large-Vocabulary Continuous Speech Recognition) [In Chinese]
Kuan-Yu Chen | Berlin Chen
Proceedings of the 21st Conference on Computational Linguistics and Speech Processing

2008

bib
Proceedings of the 20th Conference on Computational Linguistics and Speech Processing
Chao-Lin Liu | Berlin Chen
Proceedings of the 20th Conference on Computational Linguistics and Speech Processing

pdf bib
Improved Minimum Phone Error based Discriminative Training of Acoustic Models for Mandarin Large Vocabulary Continuous Speech Recognition
Shih-Hung Liu | Fang-Hui Chu | Yueng-Tien Lo | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 13, Number 3, September 2008: Special Issue on Selected Papers from ROCLING XIX

2007

bib
Proceedings of the 19th Conference on Computational Linguistics and Speech Processing
Kuang-Hua Chen | Berlin Chen
Proceedings of the 19th Conference on Computational Linguistics and Speech Processing

pdf bib
改善以最小化音素錯誤為基礎的鑑別式聲學模型訓練於中文連續語音辨識之研究 (Improved Minimum Phone Error based Discriminative Training of Acoustic Models for Chinese Continuous Speech Reconigtion) [In Chinese]
Shih-Hung Liu | Fang-Hui Chu | Berlin Chen
Proceedings of the 19th Conference on Computational Linguistics and Speech Processing

bib
ROCLING 2007 Poster Papers
Kuang-Hua Chen | Berlin Chen
ROCLING 2007 Poster Papers

pdf bib
A Comparative Study of Histogram Equalization (HEQ) for Robust Speech Recognition
Shih-Hsiang Lin | Yao-Ming Yeh | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 12, Number 2, June 2007

2006

pdf bib
統計圖等化法於雜訊語音辨識之進一步研究 (An Improved Histogram Equalization Approach for Robust Speech Recognition) [In Chinese]
Shih-Hsiang Lin | Yao-Ming Yeh | Berlin Chen
Proceedings of the 18th Conference on Computational Linguistics and Speech Processing

pdf bib
An Empirical Study of Word Error Minimization Approaches for Mandarin Large Vocabulary Continuous Speech Recognition
Jen-Wei Kuo | Shih-Hung Liu | Hsin-Min Wang | Berlin Chen
International Journal of Computational Linguistics & Chinese Language Processing, Volume 11, Number 3, September 2006: Special Issue on Selected Papers from ROCLING XVII

2005

pdf bib
風險最小化準則在中文大詞彙連續語音辨識之研究 (Risk Minimization Criterion for Mandarin Large Vocabulary Continuous Speech Recognition) [In Chinese]
Jen-Wei Kuo | Shih-Hung Liu | Berlin Chen
Proceedings of the 17th Conference on Computational Linguistics and Speech Processing

pdf bib
Lightly Supervised and Data-Driven Approaches to Mandarin Broadcast News Transcription
Berlin Chen | Jen-Wei Kuo | Wen-Hung Tsai
International Journal of Computational Linguistics & Chinese Language Processing, Volume 10, Number 1, March 2005

pdf bib
MATBN: A Mandarin Chinese Broadcast News Corpus
Hsin-Min Wang | Berlin Chen | Jen-Wei Kuo | Shih-Sian Cheng
International Journal of Computational Linguistics & Chinese Language Processing, Volume 10, Number 2, June 2005: Special Issue on Annotated Speech Corpora

2004

pdf bib
非監督式學習於中文電視新聞自動轉寫之初步應用 (Unsupervised Learning for Chinese Broadcast News Transcription) [In Chinese]
Jen-Wei Kuo | Wen-Hung Tsai | Berlin Chen
Proceedings of the 16th Conference on Computational Linguistics and Speech Processing

2001

pdf bib
Mandarin-English Information: Investigating Translingual Speech Retrieval
Helen Meng | Berlin Chen | Sanjeev Khudanpur | Gina-Anne Levow | Wai-Kit Lo | Douglas Oard | Patrick Shone | Karen Tang | Hsin-Min Wang | Jianqiang Wang
Proceedings of the First International Conference on Human Language Technology Research