View
222
Download
0
Category
Tags:
Preview:
Citation preview
1
Multimodality
Dr. Charlotte van Hooijdonk
LCC Meeting, 10 september 2008
2
Program
Multimodality
Research done in my thesis
Multimodality research within LCC context
3
What is multimodality?
A multimodal information presentation can be classified on the basis of three criteria or perspectives:
Delivery medium: devices used to deliver the information
Presentation mode: format of the message or the sign system used
Sensory mode: the human senses employed to process information
4
Why is multimodality important?
Presenting information using multiple modalities is more effective than presenting information using a single modality (e.g., Mayer, 2001; Oviatt, 1999).
Moreover, recent developments in computer technology have led to new possibilities of presenting information and to a renewed interest in the effects of different presentation modes.
5
Thesis: research questions
Research questions
When and how do people present information in a multimodal way?
Which presentation modes are most effective for learning and executing procedural instructions?
6
Thesis: research questions
Research questions
When and how do people present information in a multimodal way?
Which presentation modes are most effective for learning and executing procedural instructions?
7
Answer modalities
Monitor and keyboard should be in line with the user. Adjust the height of your chair in such a way that when you sit, the lower arms are level with the keyboard, without you pulling up your shoulders. The latter can be achieved easily by ensuring that the chair's armrests just reach the elbows when you sit with your shoulders in a relaxed position. Adjust the height of the chair in such a way that the armrests are level with or slightly above the top of the desk. Use the mouse and the keyboard without stretching your arms and/or bending your wrist.(www.unimaas.nl)
How should my workspace be organized to prevent RSI?
8 http://www.geckil.com/~harvest/rsi/
How should my workspace be organized to prevent RSI?
Answer modalities
9
Answer lengthWhat does RSI stand for?
RSI stands for Repetitive Strain Injury.
RSI stands for Repetitive Strain Injury. This
disorder involves damage to muscles, tendons and
nerves caused by overuse or misuse, and affects
the hands,wrists, elbows, arms, shoulders, back,
or neck.
10
Research design
Participants 111 Students from Tilburg University participated for course credits (46
male and 65 female).
Task For 8 medical questions, provide two answers (one brief, one extended)
-as you would like to have them- using a PowerPoint template with whatever (combination of) media you feel is best for a given question.
Coding system In total 1775 answers were collected Each answer was coded on the presence of visual media (and on the
function of visual media.
11
Results: functions of visual media
Decorative visuals did not add any information to the answer.
12
Representational visuals represented parts of the textual answer. Their presence clarified the text.
Results: functions of visual media
13
Informative visuals added information to the answer.
Results: functions of visual media
14
Results: answer length
15
Results: question type
16
Thesis: research questions
Research questions
When and how do people present information in a multimodal way?
Which presentation modes are most effective for learning and executing procedural instructions?
17
Research design
We conducted an experiment studying a specific kind of procedural instructions, namely exercises for the prevention of RSI.
We varied the information modality (text versus static visual versus dynamic visual) and the difficulty degree of the exercises (easy versus difficult) into account.
Participants had to learn RSI exercises and were asked to execute them.
18
Instruction in a text
Place your right hand in front of your body
with the elbow straight. Place your left
hand on the palm of your right hand. Gently
push the your right hand towards you until
you feel a mild stretch.
19
Instruction in a static visual
20
Instruction in a dynamic visual
filmclip.avi
21
Results
None of the presentation formats was superior on all dimensions of interest, each had some disadvantages.
Certain RSI exercises seemed to be inherently easier to represent than others, which varied across different presentation formats.
22
Multimodality research within LCC context
What?Effectiveness of (combinations of) different
presentation formats in different genresProcessing of (combinations of) different
presentation formats Adapting presentation formats to different
user groups / needs How?
Quantitative Qualitative
Recommended