Summary of the paper
| Title | Design and Application of a Gold Standard for Morphological Analysis: SMOR as an Example of Morphological Evaluation |
|---|---|
| Authors | Gertrud Faaß, Ulrich Heid and Helmut Schmid |
| Abstract | This paper describes general requirements for evaluating and documenting NLPtools with a focus on morphological analysers and the design of a GoldStandard. It is argued that any evaluation must be measurable and documentationthereof must be made accessible for any user of the tool. The documentationmust be of a kind that it enables the user to compare different tools offeringthe same service, hence the descriptions must contain measurable values. A GoldStandard presents a vital part of any measurable evaluation process, therefore,the corpus-based design of a Gold Standard, its creation and problems thatoccur are reported upon here. Our project concentrates on SMOR, a morphologicalanalyser for German that is to be offered as a web-service. We not only utilizethis analyser for designing the Gold Standard, but also evaluate the toolitself at the same time. Note that the project is ongoing, therefore, we cannotpresent final results. |
| Language | Evaluation methodologies |
| Topics | Morphology, Usability, user satisfaction, Evaluation methodologies |
Full paper  |
Design and Application of a Gold Standard for Morphological Analysis: SMOR as an Example of Morphological Evaluation |
| Bibtex | @InProceedings{FAA10.409, author = {Gertrud Faaß, Ulrich Heid and Helmut Schmid}, title = {Design and Application of a Gold Standard for Morphological Analysis: SMOR as an Example of Morphological Evaluation}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |