Summary of the paper
| Title | Evaluation of Document Citations in Phase 2 Gale Distillation |
|---|---|
| Authors | Olga Babko-Malaya, Dan Hunter, Connie Fournelle and Jim White |
| Abstract | The focus of information retrieval evaluations, such as NIST’s TRECevaluations (e.g. Voorhees 2003), is on evaluation of the information contentof system responses. On the other hand, retrieval tasks usually involve twodifferent dimensions: reporting relevant information and providing sources ofinformation, including corroborating evidence and alternative documents. Underthe DARPA Global Autonomous Language Exploitation (GALE) program, Distillationprovides succinct, direct responses to the formatted queries using the outputsof automated transcription and translation technologies. These responses areevaluated in two dimensions: information content, which measures the amount ofrelevant and non-redundant information, and document support, which measuresthe number of alternative sources provided in support of reported information.The final metric in the overall GALE distillation evaluation combines theresults of scoring of both query responses and document citations. In thispaper, we describe our evaluation framework with emphasis on the scoring ofdocument citations and an analysis of how systems perform at providing sourcesof information. |
| Language | Other |
| Topics | Evaluation methodologies, Information Extraction, Information Retrieval, Other |
Full paper  |
Evaluation of Document Citations in Phase 2 Gale Distillation |
| Bibtex | @InProceedings{BABKOMALAYA10.108, author = {Olga Babko-Malaya, Dan Hunter, Connie Fournelle and Jim White}, title = {Evaluation of Document Citations in Phase 2 Gale Distillation}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |