Summary of the paper
| Title | Semantic Evaluation of Machine Translation |
|---|---|
| Authors | Billy Tak-Ming Wong |
| Abstract | It is recognized that many evaluation metrics of machine translation in usethat focus on surface word level suffer from their lack of tolerance oflinguistic variance, and the incorporation of linguistic features can improvetheir performance. To this end, WordNet is therefore widely utilized by recentevaluation metrics as a thesaurus for identifying synonym pairs. On this basis,word pairs in similar meaning, however, are still neglected. We investigate thesignificance of this particular word group to the performance of evaluationmetrics. In our experiments we integrate eight different measures of lexicalsemantic similarity into an evaluation metric based on standard measures ofunigram precision, recall and F-measure. It is found that a knowledge-basedmeasure proposed by Wu and Palmer and a corpus-based measure, namely LatentSemantic Analysis, lead to an observable gain in correlation with humanjudgments of translation quality, in an extent to which better than the use ofWordNet for synonyms. |
| Language | Semantics |
| Topics | Machine Translation, SpeechToSpeech Translation, Evaluation methodologies, Semantics |
Full paper  |
Semantic Evaluation of Machine Translation |
| Bibtex | @InProceedings{WONG10.837, author = {Billy Tak-Ming Wong}, title = {Semantic Evaluation of Machine Translation}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |