Summary of the paper
| Title | An Evaluation of Technologies for Knowledge Base Population |
|---|---|
| Authors | Paul McNamee, Hoa Trang Dang, Heather Simpson, Patrick Schone and Stephanie M. Strassel |
| Abstract | Previous content extraction evaluations have neglected to address problemswhich complicate the incorporation of extracted information into an existingknowledge base. Previous question answering evaluations have likewise avoidedtasks such as explicit disambiguation of target entities and handling a fixedset of questions about entities without previous determination of possibleanswers. In 2009 NIST conducted a Knowledge Base Population track at its TextAnalysis Conference to unite the content extraction and question answeringcommunities and jointly explore some of these issues. This exciting newevaluation attracted 13 teams from 6 countries that submitted results in twotasks, Entity Linking and Slot Filling. This paper explains the motivation anddesign of the tasks, describes the language resources that were developed forthis evaluation, offers comparisons to previous community evaluations, andbriefly summarizes the performance obtained by systems. We also identifyrelevant issues pertaining to target selection, challenging queries, andperformance measures. |
| Language | Evaluation methodologies |
| Topics | LR national/international projects, organizational/policy issues, Knowledge Discovery/Representation, Evaluation methodologies |
Full paper  |
An Evaluation of Technologies for Knowledge Base Population |
| Bibtex | @InProceedings{MCNAMEE10.634, author = {Paul McNamee, Hoa Trang Dang, Heather Simpson, Patrick Schone and Stephanie M. Strassel}, title = {An Evaluation of Technologies for Knowledge Base Population}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |