Vincent Aleven & Kirsten Butcher

Robust Learning in Visual/Verbal Problem Solving:

Contiguity, Integrated Hints, and Elaborated Explanations

Multiple Domains Involve Learning with Visual & Verbal Info

Steif (2004)

PHYSICS

Physics LearnLab: Andes Tutor

Multiple Domains Involve Learning with Visual & Verbal Info

CHEMISTRY

Chemistry LearnLab Buffer Tutorial, Davenport (2006)

Multiple Domains Involve Learning with Visual & Verbal Info

GEOMETRY

Geometry Cognitive Tutor: Angles and Circles Units.

Research Goals

To understand how coordination between & integration of visual and verbal knowledge influences robust learning

To explore the potential transfer of laboratory-identified multimedia principles to classroom context

To inform the design of effective educational multimedia for classroom use

Relevant Learning Research

Learning with Multimedia Contiguity Effect (e.g., Mayer, 2001) Diagrams support inference-generation & integration of

information (Butcher, 2006)

Self-explanations & Cognitive Tutors Self-explanations promote learning (e.g., Chi et al., 1994) Simple (menu-based) self-explanations support Geometry

Learning (Aleven & Koedinger, 2002)

Connections to PSLC Theory Sense-making

Coordinative Learning: Integrate results from multiple inputs & representations. Verbal information Visual information

Existing Tutor: Multiple Verbal Inputs

Existing Tutor: Multiple Visual Inputs

Connections to PSLC Theory Sense-making

Interactive Communication: Tutor prompts explanations Students “explain” geometry principles that justify

problem-solving steps Students receive feedback and hints on explanations

Existing Tutor: Explanations are verbal-only

Hypotheses: Visual Scaffolds to Improve Robust Learning

Contiguity Work & receive feedback in diagram

Elaborated Explanations Visual “explanations” to justify problem-solving

Integrated Hints Apply verbal hints to visual problem situation

(diagram)

Hypotheses: Sense-making Scaffolds

Contiguity Work & receive feedback in diagram

Elaborated Explanations Visual “explanations” to justify problem-solving

Integrated Hints Apply verbal hints to visual problem situation

(diagram)

Importance of PSLC LearnLab Access to ample participants

4 geometry teachers in 15 classes (190 students) High student attrition (50 of 70 students completed study #1)

Classroom context is meaningful & cooperative Tutor completion is part of normal classwork (graded!)

Study 1 -- 4 hours training, 1.5 hours testing Student motivation is realistic, learning context is stable

Teachers open to research, comfortable with research software

Research Support Carnegie Learning -- software QA, install, & support Math Curriculum Committee -- feedback, coordination of research

Methods: Contiguity (Study 1)

Geometry Cognitive Tutor: 2 conditions Table (noncontiguous) Diagram (contiguous)

Procedure Pretest (in class) Training (classroom use of tutor, grade-matched pairs

randomly assigned to conditions within classes) Posttest (in class)

Table Condition = Noncontiguous

Diagram Condition = Contiguous

Assessment: 3 types of items

Assessment: 3 types of items

Answers

Assessment: 3 types of items

Reasons

Assessment: 3 types of items

Transfer

Preliminary Results: AnswersHigher and Lower Ability Students'

Performance on Answers (Solvable)

0

10

20

30

40

50

60

Pretest Posttest

Test Time

% C

orr

ect

Table Low

Table High

Diagram Low

Diagram High

Main effect of test time: F (1, 38) = 29.5, p < .01

Preliminary Results: ReasonsHigher and Lower Ability Students'

Performance on Reasons

0

10

20

30

40

50

60

Pretest Posttest

Test Time

% C

orr

ect

Table Low

Table High

Diagram Low

Diagram High

Main effect of test time: F (1, 38) = 65.7, p < .01

Preliminary Results: TransferHigher and Lower Ability Students'

Performance on Transfer Items

0

5

10

15

20

25

30

35

40

Pretest Posttest

Test Time

% C

orr

ect

Table Low

Table High

Diagram Low

Diagram High

3-way interaction: Test Time * Condition * Ability: F (1, 38) = 4.3, p < .05

Preliminary Results: TransferHigher Ability Students: Transfer Performance

0

5

10

15

20

25

30

35

40

Pretest Posttest

Test Time

% C

orr

ect

Table HighDiagram High

Lower Ability Students: Transfer Performance

0

5

10

15

20

25

30

35

40

Pretest Posttest

Test Time

% C

orr

ect

Table LowDiagram Low

3-way interaction: Test Time * Condition * Ability: F (1, 38) = 4.3, p < .05

Preliminary Results: Process

Observational data (to be analyzed with log data) Longer latency of responses in table condition Order of solutions differ (table drives superficial order

decisions) Classroom Feedback

Teachers report student preference for diagram tutor Teachers report better engagement from low ability

students “I like the [diagram] better, because you can see the

answers in the diagram. Otherwise it’s easy to get confused with the table, you know, going back and forth and stuff.”

Research Team Vincent Aleven: Research Scientist, CMU HCII Kirsten Butcher: Research Postdoc, Pitt LRDC Shelley Evenson: Assoc Prof, CMU School of Design Octav Popescu: Research Programmer, CMU HCII Andy Tzou: Masters Student: CMU HCII Honors Program Carl Angiolillo: Masters Student: CMU HCII Honors Program Grace Leonard: Research Associate, CMU HCII Thomas Bolster: Research Associate, CMU HCII

Questions?

Extra Slides

Continued Work: Elaborated Explanations

Elaborated Explanations Interface

Elaborated Explanations Interface