Summary of the paper
| Title | Using a Grammar Checker for Evaluation and Postprocessing of Statistical Machine Translation |
|---|---|
| Authors | Sara Stymne and Lars Ahrenberg |
| Abstract | One problem in statistical machine translation (SMT) is that the output oftenis ungrammatical. To address this issue, we have investigated the use of agrammar checker for two purposes in connection with SMT: as an evaluation tooland as a postprocessing tool. To assess the feasibility of the grammar checkeron SMT output, we performed an error analysis, which showed that the precisionof error identification in general was higher on SMT output than in previousstudies on human texts. Using the grammar checker as an evaluation tool givesa complementary picture to standard metrics such as Bleu, which do not accountwell for grammaticality. We use the grammar checker as a postprocessing tool byautomatically applying the error correction suggestions it gives. There areonly small overall improvements of the postprocessing on automatic metrics, butthe sentences that are affected by the changes are improved, as shown both byautomatic metrics and by a human error analysis. These results indicate thatgrammar checker techniques are a useful complement to SMT. |
| Language | Grammar and Syntax |
| Topics | Machine Translation, SpeechToSpeech Translation, Evaluation methodologies, Grammar and Syntax |
Full paper  |
Using a Grammar Checker for Evaluation and Postprocessing of Statistical Machine Translation |
| Bibtex | @InProceedings{STYMNE10.426, author = {Sara Stymne and Lars Ahrenberg}, title = {Using a Grammar Checker for Evaluation and Postprocessing of Statistical Machine Translation}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |