Summary of the paper
| Title | Practical Evaluation of Speech Recognizers for Virtual Human Dialogue Systems |
|---|---|
| Authors | Xuchen Yao, Pravin Bhutada, Kallirroi Georgila, Kenji Sagae, Ron Artstein and David Traum |
| Abstract | We perform a large-scale evaluation of multiple off-the-shelf speechrecognizers across diverse domains for virtual human dialogue systems. Ourevaluation is aimed at speech recognition consumers and potential consumerswith limited experience with readily available recognizers. We focus onpractical factors to determine what levels of performance can be expected fromdifferent available recognizers in various projects featuring different typesof conversational utterances. Our results show that there is no singlerecognizer that outperforms all other recognizers in all domains. Theperformance of each recognizer mayvary significantly depending on the domain, the size and perplexity of thecorpus, the out-of-vocabulary rate, and whether acoustic and language modeladaptation has been used or not. We expect that our evaluation will proveuseful to other speech recognition consumers, especially in the dialoguecommunity, and will shed some light on the key problem in spoken dialoguesystems of selecting the most suitable available speech recognition system fora particular application, and what impact training will have. |
| Language | Evaluation methodologies |
| Topics | Dialogue, Speech Recognition/Understanding, Evaluation methodologies |
Full paper  |
Practical Evaluation of Speech Recognizers for Virtual Human Dialogue Systems |
| Bibtex | @InProceedings{YAO10.675, author = {Xuchen Yao, Pravin Bhutada, Kallirroi Georgila, Kenji Sagae, Ron Artstein and David Traum}, title = {Practical Evaluation of Speech Recognizers for Virtual Human Dialogue Systems}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |