Upload
ronald-chapman
View
213
Download
0
Embed Size (px)
Citation preview
Towards a definition of GestBase
- an open database of gestures
Milan Rusko
Institute of Informatics of the Slovak Academy of Sciences, Bratislava
COST 2102 DUBLIN 2009
MotivationPublishing studies on gestures face_and_body gestures voice gestures
Video – often not included in the electronic publication, no widely accepted form of annotation suitable for basic comparisons etc.
Photos – occupy too much space in the printed publication , no standard or widely accepted annotation suitable for basic comparisons Audio – not available in the printed publications =>replaced by ambiguous orthographic code (mhm, oooh, etc. )
Text (periphrastic) description – ambiguous, insufficient, no standard form
All suffer from lack of comprehensive method of referencing.It is hard for the reader to imagine the gesture.
Vision of solution - GestBase
Possible solution - an open database of Gestures
Such a database should have standardized format of entries allow for storage of gestures in multiple modalities
and their annotations allow for storage of gestures in context allow for search, comparison, and other database
operations be accessible for all potential users be open for new submissions
Database structureVisual gestures
database
pictures (photos)
textual description of the gesture
coded description of the picture
...
Semantic & Pragmatic content (meaning)
Comments
Nonverbal speech gestures database
sounds (recordings)
orthoepic (IPA) transcription
orthographic (text) trsc.
Automatically derived acoustical chars (F0 contour, ...)
Coded acoustical char.
Para and Extra-linguistic
Semantic & Pragmatic content
Function in the dialogue
Comments
RELATIONS
Pilot study: first submisson – GestBase_Sk ver0.1Standard procedure:
Natural communicationrecordings
Classification usingexisting class. schemes
Annotation usingexisting ann. conventions
GestBase file
Recording of speakers(stimuli induced gestures)
Annotation
Collecting prototype gestures,developing annot. and class.
schemes
Prototype GestBase files
Procedure in ver0.1:
Collection of stimuli induced gestures
The speaker gets a stimulus – a gesture represented in one modality, and he has to add other modalities and additional information.
1. stimulus=photo, speaker ads sound (or several sounds), context (creates a sentence) his own description of the meaning and typical usage.(initial set of photos taken from Ruzickova)
2. stimulus=text (orthographic form of the gesture etc.) speaker adds sound (in future we plan also video recording)
(list of interjections and particles taken from Slovak National Corpus)
3. stimulus=sound (nonverbal speech gesture, possibly in context)
Ideas on usability of GestBase
Possible usage of GestBase could be a photo-sound-text electronic upgraded version of Ružičková: American, Slovak, Japanese and Chinese Picture dictionary of Gestures. (possibly extended with other languages)
This is a publication which is aimed at giving information to the foreign language learners on the gestures in the given language and their comparison with the ones in his mother tongue.
Inter-lingual and inter-cultural comparison studies:Slovak – Hungarian (one culture, very different languages)Slovak – Japanese (very different cultures) etc.
Thank you for your attention
thank you in advance for your oppinion
Milan Rusko