Summary of the paper
| Title | Test Suite Design for Biomedical Ontology Concept Recognition Systems |
|---|---|
| Authors | K. Bretonnel Cohen, Christophe Roeder, William A. Baumgartner Jr., Lawrence E. Hunter and Karin Verspoor |
| Abstract | Systems that locate mentions of concepts from ontologies in free text are knownas ontology concept recognition systems. This paper describes an approach tothe evaluation of the workings of ontology concept recognition systems throughuse of a structured test suite and presents a publicly available test suite forthis purpose. It is built using the principles of descriptive linguisticfieldwork and of software testing. More broadly, we also seek to investigatewhat general principles might inform the construction of such test suites. Thetest suite was found to be effective in identifying performance errors in anontology concept recognition system. The system could not recognize 2.1% ofall canonical forms and no non-canonical forms at all. Regarding the questionof general principles of test suite construction, we compared this test suiteto a named entity recognition test suite constructor. We found that they hadtwenty features in total and that seven were shared between the two models,suggesting that there is a core of feature types that may be applicable to testsuite construction for any similar type of application. |
| Language | Ontologies |
| Topics | Evaluation methodologies, Corpus (creation, annotation, etc.), Ontologies |
Full paper  |
Test Suite Design for Biomedical Ontology Concept Recognition Systems |
| Bibtex | @InProceedings{COHEN10.31, author = {K. Bretonnel Cohen, Christophe Roeder, William A. Baumgartner Jr., Lawrence E. Hunter and Karin Verspoor}, title = {Test Suite Design for Biomedical Ontology Concept Recognition Systems}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |