Summary of the paper
| Title | Estimation Method of User Satisfaction Using N-gram-based Dialog History Model for Spoken Dialog System |
|---|---|
| Authors | Sunao Hara, Norihide Kitaoka and Kazuya Takeda |
| Abstract | In this paper, we propose an estimation method of user satisfaction for aspoken dialog system using an N-gram-based dialog history model. We havecollected a large amount of spoken dialog data accompanied by usabilityevaluation scores by users in real environments. The database is made by afield-test in which naive users used a client-server music retrieval systemwith a spoken dialog interface on their own PCs. An N-gram model is trainedfrom the sequences that consist of users' dialog acts and/or the system'sdialog acts for each one of six user satisfaction levels: from 1 to 5 and φ(task not completed). Then, the satisfaction level is estimated based on theN-gram likelihood. Experiments were conducted on the large real data and theresults show that our proposed method achieved good classification performance;the classification accuracy was 94.7% in the experiment on a classificationinto dialogs with task completion and those without task completion. Even ifthe classifier detected all of the task incomplete dialog correctly, ourproposed method achieved the false detection rate of only 6%. |
| Language | Usability, user satisfaction |
| Topics | Dialogue, Evaluation methodologies, Usability, user satisfaction |
Full paper  |
Estimation Method of User Satisfaction Using N-gram-based Dialog History Model for Spoken Dialog System |
| Bibtex | @InProceedings{HARA10.579, author = {Sunao Hara, Norihide Kitaoka and Kazuya Takeda}, title = {Estimation Method of User Satisfaction Using N-gram-based Dialog History Model for Spoken Dialog System}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |