Summary of the paper
| Title | Evaluation of Textual Knowledge Acquisition Tools: a Challenging Task |
|---|---|
| Authors | Haïfa Zargayouna and Adeline Nazarenko |
| Abstract | A large effort has been devoted to the development of textual knowledgeacquisition (KA) tools, but it is still difficult to assess theprogress that has been made. The results produced by these tools are difficultto compare, due to the heterogeneity of the proposed methods and of theirgoals. Various experiments have been made to evaluate terminologicaland ontological tools. They show that in terminology as well as inontology acquisition, it remains difficult to compare existingtools and to analyse their advantages and drawbacks.From our own experiments in evaluating terminology and ontology acquisitiontools, it appeared that the difficultiesand solutions are similar for both tasks. We propose a unifiedapproach for the evaluation of textual KA tools thatcan be instantiated in different ways for various tasks. Themain originality of this approach lies in the way it takes intoaccount the subjectivity of evaluation and the relativity ofgold standards. In this paper, we highlight the major difficulties of KA evaluation,we then present a unified proposal for the evaluationof terminologies and ontologies acquisition tools and theassociated experiments. The proposed protocols take intoconsideration the specificity of this type of evaluation. |
| Language | Evaluation methodologies |
| Topics | Ontologies, Acquisition, Evaluation methodologies |
Full paper  |
Evaluation of Textual Knowledge Acquisition Tools: a Challenging Task |
| Bibtex | @InProceedings{ZARGAYOUNA10.311, author = {Haïfa Zargayouna and Adeline Nazarenko}, title = {Evaluation of Textual Knowledge Acquisition Tools: a Challenging Task}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |