Summary of the paper
| Title | Syntactic Testsuites and Textual Entailment Recognition |
|---|---|
| Authors | Paul Bedaride and Claire Gardent |
| Abstract | We focus on textual entailments mediated by syntax and propose a newmethodology to evaluate textual entailment recognition systems on suchdata. The main idea is to generate a syntactically annotated corpus ofpairs of (non-)entailments and to use error mining methodology fromthe parsing field to identify the most likely sources of errors. Togenerate the evaluation corpus we use a template based generationapproach where sentences, semantic representations and syntacticannotations are all created at the same time. Furthermore, we adaptthe error mining methodology initially proposed for parsing to thefield of textual entailment. To illustrate the approach, we apply theproposed methodology to the Afazio RTE system (an hybrid systemfocusing on syntactic entailment) and show how it permits identifyingthe most likely sources of errors made by this system on a testsuiteof 10 000 (non-)entailment pairs which is balanced in term of(non-)entailment and in term of syntactic annotations. |
| Language | |
| Topics | Textual Entailment and Paraphrasing, Semantics |
Full paper  |
Syntactic Testsuites and Textual Entailment Recognition |
| Bibtex | @InProceedings{BEDARIDE10.379, author = {Paul Bedaride and Claire Gardent}, title = {Syntactic Testsuites and Textual Entailment Recognition}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |