Multimodal Methodologies Part I and II Jeff Bezemer, Carey Jewitt and Gemma Moss MODE multimodal methodologies FOR RESEARCHING DIGITAL DATA AND ENVIRONMENTS

Multimodal Methodologies Part I and II

Jeff Bezemer, Carey Jewitt and Gemma Moss

MO

DE

mul

timod

al m

etho

dolo

gies

FOR

RESE

ARCH

ING

DIG

ITAL

DAT

A AN

D E

NVI

RON

MEN

TS

On

e

http://mode.ioe.ac.uk

MODE multimodal methodologiesFOR RESEARCHING DIGITAL DATA AND ENVIRONMENTS

• An understanding of the origins and scope of multimodality

• Explications of multimodal frameworks and key concepts

• Knowledge of the different ways in which multimodality has been taken up by a range of perspectives

• An outline of the procedures of multimodal analysis

The aim of today is to provide:


• Why take a multimodal approach?• What concepts and methods can be used to

analyse interaction in digital environments?• How can we get at how the use of different

digital technologies shapes understandings and experiences?

• What challenges does multimodality raise for researching digital data and environments?

Focusing questions:


Part I • 9.15 Introduction • 9.30 Origins, Scope and Key concepts (Carey Jewitt)• 11.15 Multimodal perspectives: video data analysis

(Jeff Bezemer)

12.45 lunch

Part II • 16.00 Presentation and data workshop: Analyzing

multimodal digital texts (Gemma Moss)

Program:

Multimodality: origins, scope and key

concepts

MO

DE

mul

timod

al m

etho

dolo

gies

FOR

RESE

ARCH

ING

DIG

ITAL

DAT

A AN

D E

NVI

RON

MEN

TS


6

7

Concepts: the level of modeMODEMEANING FUNCTIONSSEMIOTIC RESOURCESEMIOTIC (MEANING) POTENTIALMATERIALITYAFFORDANCEPROVENANCEMODAL LOGICSFUNCTIONAL SPECIALISM


December 2011, Norway

Concepts: Level of modal-interaction MULTIMODAL ENSEMBEL INTER-MODAL RELATIONSCOHERENCEDISSONANCEMODAL DENSITYMODAL INTENSITYEQUAL-UNEQUALDOMINANT – SUBORINATEELABORATIONEXPANSIONREPETITION-EMPHASISSALIENCE

December 2011, Norway

Concepts: level of process of sign makingSIGN MAKERINTERESTMULTIMODAL ORCHESTRATIONMULTIMODAL DESIGNTRANSFORMATIONTRANSDUCTIONTRANSMODAL MOMENTTEXTUAL CYCLEMODAL ORIENTATION CONTEXT


• Social Semiotic Multimodal Analysis• A Systemic Functional Approach (Multimodal

Discourse Analysis)• Multimodal Interactional Analysis • Multimodal conversation analysis

Four main perspectives on multimodality:


• fine grained analysis: the need to understand the details of texts and interactions

• focus on meaning making as social and semiotic

• the demand to broaden conceptions of communication beyond language

• shared interests, and shared histories• externally material focus

Similarities


• Historical origins, influences and directions• Extent of emphasis on context• Degree of emphasis on modes and modal

systems• Attention to the agentive work• The scale of attention

Differences


• Organizational geography• Temporality of events• Recognizing events: patterns and customs• Detecting incongruities and breakdowns• Sequences of interaction, before, after and

immediately following one another

Conversation analysis & multimodality: analytical focus



Systemic Functional Linguistics (Anders Bjorkvall, 2009 in HMA)



Social Semiotic Multimodal Analysis


Analyzing Multimodal Interaction Norris (2004)


Norris (2009) Modal Denisity in HMA


• Sensory anthropology (e.g. David Howes)• Socio-cultural approaches and multimodality

(e.g. Barbara Rogoff, Roger Saljo)• Eye-tracking (Jana Holsvana)• New literacy Studies (Kate Phal, Jennifer Rowsell)• Historical analysis with a multimodal focus (e.g.

Grosvenor)

Other approaches

[email protected]


Documents

Multimodal Methodologies Part I and II Jeff Bezemer, Carey Jewitt and Gemma Moss MODE multimodal methodologies FOR RESEARCHING DIGITAL DATA AND ENVIRONMENTS