Hand & Mouth - Which One‘s First?

- On the temporal coordination

of gesture and speech -

Linguistic Theory: “Multimodal Communication”Universität BielefeldSoSe 2004

Data Sample 1 - Annotation

Data Sample II - Video Clip

[so our whole timetable will be /uhm/] changed again?

individualgesture item

onset of gesture

onset of affiliated word(s)

difference between onsets

in seconds

Temporal Displacement I

Relevant Time Relations

1. gesture contains speech2. speech contains gesture3. gesture overlaps speech4. speech overlaps gesture5. gesture precedes speech6. speech precedes gesture

Time Relations Evaluated

Temporal Displacement II

Summary

onset of gesture phrase anticipates onset of affiliated word(s)

however: onset of gesture stroke (most “meaningful” part of gesture) tends to coincide with onset of lexical affiliate (within a minimal temporal margin)