Summary of the paper
| Title | Inter-Annotator Agreement on a Linguistic Ontology for Spatial Language - A Case Study for GUM-Space |
|---|---|
| Authors | Joana Hois |
| Abstract | In this paper, we present a case study for measuring inter-annotator agreementon a linguistic ontology for spatial language, namely the spatial extension ofthe Generalized Upper Model. This linguistic ontology specifies semanticcategories, and it is used in dialogue systems for natural language of space inthe context of human-computer interaction and spatial assistance systems. Itscore representation for spatial language distinguishes how sentences can bestructured and categorized into units that contribute certain meanings to theexpression. This representation is here evaluated in terms of inter-annotatoragreement: four uninformed annotators were instructed by a manual how toannotate sentences with the linguistic ontology. They have been assigned toannotate 200 sentences with varying length and complexity. Their resultingagreements are calculated together with our own 'expert annotation' of the samesentences. We show that linguistic ontologies can be evaluated with respect tointer-annotator agreement, and we present encouraging results of calculatingagreements for the spatial extension of the Generalized Upper Model. |
| Language | Language modelling |
| Topics | Evaluation methodologies, Ontologies, Language modelling |
Full paper  |
Inter-Annotator Agreement on a Linguistic Ontology for Spatial Language - A Case Study for GUM-Space |
| Bibtex | @InProceedings{HOIS10.231, author = {Joana Hois}, title = {Inter-Annotator Agreement on a Linguistic Ontology for Spatial Language - A Case Study for GUM-Space}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |