Analysing Discourse Markers:A Multi-Modal Approach

Svenja Adolphs and Dawn Knight

BAAL 200811th – 13th September 2008, Swansea University

Non-verbal, multi-modal behaviour plays an integral part indetermining meaning in language.

Current corpus methodologies in language data analysis(especially spoken language) need to be extended toinclude an integrated exploration of both verbal & non-verbalpatterns of social interaction.

Key components in gesture include head nods and handgestures and initial research indicates relationships betweenthem and key linguistic features such as backchannels anddiscourse markers.

Research foundations

DReSS

NMMC (Nottingham Multi-Modal Corpus) data:

250,000 words 125,000 words of 1-party data, 125,000 words of 2-party data Data in three different modes: textual, audio and video

Corpus tool-bench (DRS- The Digital Replay System):

A reusable corpus tool with the flexibility to be utilised for awide range of qualitative social science enquiry

Allows the analyst to search lexical, prosodic and gesturalfeatures of spoken discourse (and beyond…..)

Introducing the DReSS project

Working research question:

Are there specific gesture sequences that are associatedwith discourse markers that function to manage (e.g. right,so, anyway) the talk, as opposed to more interpersonalones (e.g. in a sense, I guess)?

Key challenges:

Marking and coding the gesture phase accurately,searching and locating the marked gestures andapproaching the analysis of these.

Key linguistic questions

Data

Plotting gesture shape and trajectory

For example:

13 = Left arm moves up and left, rightarm moves up and right

Tracking gestures

A coding scheme for gesture-in-talk

MOVEMENT CLASSIFICATION

Stage 1: Type

Stage 2: Shape / Trajectory /Intensity / Duration

Stage 3: Frequency

Combining movement and linguistic codes for analysis:

LINGUISTIC CLASSIFICATION

Stage 1: Discursive function

Stage 2: Form

Stage 3: Frequency

The interaction oflanguage and gesture-in-use for the generation of

meaning in discourse

Discourse Markers

Preliminary findings (based on corpus searches)

Discourse markers: Gesture Vs no-gestureGesture No Gesture

Pragmatic category in relation to linguistic form & function

DRS