Summary of the paper
| Title | Contrastive Lexical Evaluation of Machine Translation |
|---|---|
| Authors | Aurélien Max, Josep Maria Crego and François Yvon |
| Abstract | This paper advocates a complementary measure of translation performance thatfocuses on the constrastive ability of two or more systems or system versionsto adequately translate source words. This is motivated by three main reasons :1) existing automatic metrics sometimes do not show significant differencesthat can be revealed by fine-grained focussed human evaluation, 2) thesemetrics are based on direct comparisons between system hypotheses with thecorresponding reference translations, thus ignoring the input words that wereactually translated, and 3) as these metrics do not take input hypotheses fromseveral systems at once, fine-grained contrastive evaluation can only be doneindirectly. This proposal is illustrated on a multi-source Machine Translationscenario where multiple translations of a source text are available.Significant gains (up to +1.3 BLEU point) are achieved on these experiments,and contrastive lexical evaluation is shown to provide new information that canhelp to better analyse a system's performance. |
| Language | |
| Topics | Machine Translation, SpeechToSpeech Translation, Evaluation methodologies |
Full paper  |
Contrastive Lexical Evaluation of Machine Translation |
| Bibtex | @InProceedings{MAX10.823, author = {Aurélien Max, Josep Maria Crego and François Yvon}, title = {Contrastive Lexical Evaluation of Machine Translation}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |