Summary of the paper
| Title | The POETICON Corpus: Capturing Language Use and Sensorimotor Experience in Everyday Interaction |
|---|---|
| Authors | Katerina Pastra, Christian Wallraven, Michael Schultze, Argyro Vataki and Kathrin Kaulard |
| Abstract | Natural language use, acquisition, and understanding takes place usually inmultisensory and multimedia communication environments. Therefore, for one tomodel language in its interaction and integration with sensorimotorexperiences, one needs a representative corpus of such interplay. In thispaper, we will present the first corpus of language use and sensorimotorexperience recordings in everyday human:human interaction, in which spontaneouslanguage communication has been recorded along with corresponding multiviewvideo recordings, recordings of 3D full body kinematics, and 3D tracking ofobjects in focus. It is a twelve-hour corpus which comprises of six everydayhuman:human interaction scenes, each one performed 3 times by 4 differentEnglish-speaking couples (interaction between a male and a female actor), eachcouple acting each scene in two settings: a fully naturalistic setting in which5-camera multi-view video recordings take place, and a high-tech setting, withfull body motion capture for both individuals, a 2-camera multiview videorecording, and 3D tracking of focus objects. The corpus has been developedwithin an EU-funded cognitive systems research project, POETICON(http://www.poeticon.eu), and represents a new type of language resources forcognitive systems. Namely, a corpus that reveals the dynamic role of languagein its interplay with sensorimotor experiences and which allows one tocomputationally model this interplay. |
| Language | Knowledge Discovery/Representation |
| Topics | Corpus (creation, annotation, etc.), Multimedia Document Processing, Knowledge Discovery/Representation |
Full paper  |
The POETICON Corpus: Capturing Language Use and Sensorimotor Experience in Everyday Interaction |
| Bibtex | @InProceedings{PASTRA10.778, author = {Katerina Pastra, Christian Wallraven, Michael Schultze, Argyro Vataki and Kathrin Kaulard}, title = {The POETICON Corpus: Capturing Language Use and Sensorimotor Experience in Everyday Interaction}, booktitle = {Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC'10)}, year = {2010}, month = {may}, date = {19-21}, address = {Valletta, Malta}, editor = {Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odjik, Stelios Piperidis, Mike Rosner, Daniel Tapias}, publisher = {European Language Resources Association (ELRA)}, isbn = {2-9517408-6-7}, language = {english} } |