Summary of the paper
| Title | Building a Web Corpus of Czech |
|---|---|
| Authors | Drahomíra "johanka" Spoustová, Miroslav Spousta and Pavel Pecina |
| Abstract | Large corpora are essential to modern methods of computational linguisticsand natural language processing. In this paper, we describe an ongoingproject whose aim is to build a largest corpus of Czech texts. We arebuilding the corpus from Czech Internet web pages, using (and, if needed,developing) advanced downloading, cleaning and automatic linguisticprocessing tools. Our concern is to keep the whole process languageindependent and thus applicable also for building web corpora of otherlanguages. In the paper, we briefly describe the crawling, cleaning, andpart-of-speech tagging procedures. Using a prototype corpus, we provide acomparison with a current corpora (in particular, SYN2005, part of theCzech National Corpora). We analyse part-of-speech tag distribution, OOVword ratio, average sentence length and Spearman rank correlationcoefficient of the distance of ranks of 500 most frequent words. Ourresults show that our prototype corpus is now quite homogenous. The challenging task is to find a way to decrease the homogeneity of the text while keeping the high quality of the data. |
| Language | LR Infrastructures and Architectures |
| Topics | Corpus (creation, annotation, etc.), Tools, systems, applications, LR Infrastructures and Architectures |
Full paper  |
Building a Web Corpus of Czech |
| Bibtex | @InProceedings{SPOUSTOV10.810, author = {Drahomíra "johanka" Spoustová, Miroslav Spousta and Pavel Pecina}, title = {Building a Web Corpus of Czech}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |