Summary of the paper
| Title | Word Sense Annotation of Polysemous Words by Multiple Annotators |
|---|---|
| Authors | Rebecca J. Passonneau, Ansaf Salleb-Aoussi, Vikas Bhardwaj and Nancy Ide |
| Abstract | We describe results of a word sense annotation task using WordNet, involvinghalf a dozen well-trained annotators on ten polysemous words for three parts ofspeech. One hundred sentences for each word were annotated. Annotators had thesame level of training and experience, but interannotator agreement (IA) variedacross words. There was some effect of part of speech, with higher agreement onnouns and adjectives, but within the words for each part of speech there waswide variation. This variation in IA does not correlate with number of sensesin the inventory, or the number of senses actually selected by annotators. Infact, IA was sometimes quite high for words with many senses. We claim that theIA variation is due to the word meanings, contexts of use, and individualdifferences among annotators. We find some correlation of IA with senseconfusability as measured by a sense confusion threshhold (CT). Data mining forassociation rules on a flattened data representation indicating eachannotator's sense choices identifies outliers for some words, and systematicdifferences among pairs of annotators on others. |
| Language | Lexicon, lexical database |
| Topics | Word Sense Disambiguation, Text mining, Lexicon, lexical database |
Full paper  |
Word Sense Annotation of Polysemous Words by Multiple Annotators |
| Bibtex | @InProceedings{PASSONNEAU10.922, author = {Rebecca J. Passonneau, Ansaf Salleb-Aoussi, Vikas Bhardwaj and Nancy Ide}, title = {Word Sense Annotation of Polysemous Words by Multiple Annotators}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |