Summary of the paper
| Title | Number or Nuance: Which Factors Restrict Reliable Word Sense Annotation? |
|---|---|
| Authors | Susan Windisch Brown, Travis Rood and Martha Palmer |
| Abstract | This study attempts to pinpoint the factors that restrict reliable word senseannotation, focusing on the influence of the number of senses annotators useand the semantic granularity of those senses. Both of these factors may bepossible causes of low interannotator agreement (ITA) when tagging withfine-grained word senses, and, consequently, low WSD system performance (Ng etal., 1999; Snyder & Palmer, 2004; Chklovski & Mihalcea, 2002). If number ofsenses is the culprit, modifying the task to show fewer senses at a time couldimprove annotator reliability. However, if overly nuanced distinctions are theproblem, then more general, coarse-grained distinctions may be necessary forannotator success and may be all that is needed to supply systems with thetypes of distinctions that people make. We describe three experiments thatexplore the role of these factors in annotation performance. Our resultsindicate that of these two factors, only the granularity of the sensesrestricts interannotator agreement, with broader senses resulting in higherannotation reliability. |
| Language | Semantics |
| Topics | Word Sense Disambiguation, Corpus (creation, annotation, etc.), Semantics |
Full paper  |
Number or Nuance: Which Factors Restrict Reliable Word Sense Annotation? |
| Bibtex | @InProceedings{BROWN10.927, author = {Susan Windisch Brown, Travis Rood and Martha Palmer}, title = {Number or Nuance: Which Factors Restrict Reliable Word Sense Annotation?}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |