Summary of the paper
| Title | DICIT: Evaluation of a Distant-talking Speech Interface for Television |
|---|---|
| Authors | Timo Sowa, Fiorenza Arisio and Luca Cristoforetti |
| Abstract | The EC-funded project DICIT developed distant-talking interfaces forinteractive TV. The final DICIT prototype system processes multimodal userinput by speech and remote control. It was designed to understand both naturallanguage and command-and-control-style speech input. We conducted an evaluationcampaign to examine the usability and performance of the prototype. Thetask-oriented evaluation involved naive test persons and consisted of asubjective part with a usability questionnaire and an objective part. We usedthree groups of objective metrics to assess the system: one group related tospeech component performance, one related to interface design and userawareness, and a final group related to task-based effectiveness and usability.These metrics were acquired with a dedicated transcription and annotation tool.The evaluation revealed a quite positive subjective assessments of the systemand reasonable objective results. We report how the objective metrics helped usto determine problems in specific areas and to distinguish design-relatedissues from technical problems. The metrics computed over modality-specificgroups also show that speech input gives a usability advantage over remotecontrol for certain types of tasks. |
| Language | Speech Recognition/Understanding |
| Topics | Corpus (creation, annotation, etc.), Evaluation methodologies, Speech Recognition/Understanding |
Full paper  |
DICIT: Evaluation of a Distant-talking Speech Interface for Television |
| Bibtex | @InProceedings{SOWA10.894, author = {Timo Sowa, Fiorenza Arisio and Luca Cristoforetti}, title = {DICIT: Evaluation of a Distant-talking Speech Interface for Television}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |