Kirsten Butcher

Slide 1

Kirsten Butcher

Elaborated Explanations for Visual/Verbal Problem Solving:

Interactive Communication ClusterJuly 24, 2006

Slide 2

Visual & Verbal Information in Geometry

Geometry Cognitive Tutor: Angles and Circles Units.

Slide 3

Research Goals

To understand how coordination between & integration of visual and verbal knowledge influences robust learning

To explore the potential transfer of laboratory-identified multimedia principles to classroom context

To inform the design of effective educational multimedia for classroom use

Slide 4

Relevant Learning Research

Learning with Multimedia Contiguity Effect (e.g., Mayer, 2001) Diagrams support inference-generation & integration of

information (Butcher, 2006)

Self-explanations & Cognitive Tutors Self-explanations promote learning (e.g., Chi et al., 1994) Simple (menu-based) self-explanations support Geometry

Learning (Aleven & Koedinger, 2002)

Slide 5

Hypotheses: Sense-making Scaffolds

Contiguity Work & receive feedback in diagram

Integrated Hints Apply verbal hints to visual problem situation

(diagram) Elaborated Explanations

Visual “explanations” to justify problem-solving

Slide 6

Hypotheses: Sense-making Scaffolds

Contiguity Work & receive feedback in diagram

Integrated Hints Apply verbal hints to visual problem situation

(diagram) Elaborated Explanations

Visual “explanations” to justify problem-solving

Slide 7

Connections to PSLC Theory Sense-making

Coordinative Learning: Integrate results from multiple inputs & representations. Verbal information Visual information

Scaffolds change the format of the interface to promote coordinative learning. Contiguous representation: reduces mapping & supports

inferences made directly from diagram Integrated hints: reduce mapping & support recognition of

critical visual elements

Slide 8

Hypotheses: Sense-making Scaffolds

Contiguity Work & receive feedback in diagram

Integrated Hints Apply verbal hints to visual problem situation

(diagram) Elaborated Explanations

Visual “explanations” to justify problem-solving

Slide 9

Connections to PSLC Theory Sense-making

Interactive Communication: Tutor prompts explanations Students “explain” geometry principles that justify problem-

solving steps Students receive feedback and hints on explanations

Scaffold: Elaborated explanations require student to “explain” the application of geometry principles Rationale for explanations are visual in nature Diagram Condition: Visual format for explanation Table Condition: Verbal format for explanation

Slide 10

Existing Tutor: Explanations are verbal-only

Slide 11

Elaborated Explanations Tutor

Slide 12

Elaborated Explanations Tutor

Slide 13

Elaborated Explanations Tutor

Demo of the Geometry Cognitive Tutor with

Elaborated Explanations

New & Improved!

Now with more

explanations!

Slide 14

Connections to PSLC Theory What are the relevant knowledge components?

(Verbal) Geometry principles.

E.g., Inscribed Angle Theorem means that the measure of the

angle is half the measure of the intercepted arc.

(Visual) Geometry elements.

E.g., Recognizing angles, arcs, and their relationships.

(Integrated) Geometry inferences E.g., Recognizing that an arc, which is associated with a

known (or found) inscribed angle, can be found via the

Inscribed Angle Theorem

Slide 15

Knowledge Components vs. Overall Visual Match

Slide 16

Knowledge Components vs. Overall Visual Match

Slide 17

Mapping Given Information to Elements

Slide 18

Integration of Principles and Elements

Slide 19

Superficial Strategies of Integration: Close = Connected

Slide 20

Robust Knowledge: Relationships connect Elements via Principles

Slide 21

Difficulty Factors Analysis (DFA): Problem Format & Explanation Type

3 Problem Formats Diagram Quadrant Table

2 Explanation Types Simple Explanations (Reasons Only) Elaborated Explanations (Reasons + Application)

Slide 22

DFA: Diagram Problem Format with Simple Explanations

Slide 23

DFA: Diagram Problem Format with Elaborated Explanations

Slide 24

DFA: Quadrant Problem Format with Elaborated Explanations

Slide 25

DFA: Table Problem Format with Elaborated Explanations

Slide 26

DFA Results: Given Information

Linear trend for Explanation Type, F (1, 88) = 3.8, p = .055

Performance by Problem Format & Explanation Type

50

55

60

65

70

75

80

85

90

95

100

Diagram Quadrant Table

Problem Format

Perc

en

t C

orr

ect

Simple Explanations

Elaborated Explanations

Slide 27

DFA Results: Problem Solving

Performance by Problem Format & Explanation Type

0

5

10

15

20

25

30

35

40

45

50

Diagram Quadrant Table

Problem Format

Perc

en

t C

orr

ect

Simple Explanations

Elaborated Explanations

Linear trend for Explanation Type, F (1, 88) = 2.9, p = .09Quadratic effect for Problem Format, F (1, 88) = 3.8, p = .053Trend for interaction, F (1, 88) = 3.0, p =.088

Slide 28

Preliminary Results: Process

Observational pilot data Longer latency of responses in table condition BEFORE entering

quantities Longer latencies AFTER quantities entered when elaborated

explanations are required

Classroom Feedback Teachers report student preference for diagram tutor Students report no perceived differences in the “amount of work”

for the elaborated explanations Students adapt quickly to the elaborated explanations, but

performance far from ceiling even after successful completion of tutor with simple explanations.

Slide 29

Next Steps

Log files??????!!!! Think-aloud protocols with elaborated explanations

Summer 2006

Lab testing of elaborated explanations Summer 2006

In-vivo testing with the elaborated explanations & contiguous interface (2 X 2) Late Fall 2006

Slide 30

Research Team Vincent Aleven: Research Scientist, CMU HCII Kirsten Butcher: Research Postdoc, Pitt LRDC Shelley Evenson: Assoc Prof, CMU School of Design Octav Popescu: Research Programmer, CMU HCII Andy Tzou: Masters Student: CMU HCII Honors Program Carl Angiolillo: Masters Student: CMU HCII Honors Program Grace Leonard: Research Associate, CMU HCII Thomas Bolster: Research Associate, CMU HCII

Slide 31

Questions?

Slide 32

Extra Slides

Slide 33

Existing Tutor: Multiple Verbal Inputs

Slide 34

Existing Tutor: Multiple Visual Inputs

Slide 35

Table Condition = Noncontiguous

Slide 36

Diagram Condition = Contiguous

Slide 37

Methods: Contiguity (Study 1)

Geometry Cognitive Tutor: 2 conditions Table (noncontiguous) Diagram (contiguous)

Procedure Pretest (in class) Training (classroom use of tutor, grade-matched pairs

randomly assigned to conditions within classes) Posttest (in class)

Slide 38

Assessment: 3 types of items

Answers

Slide 39

Assessment: 3 types of items

Reasons

Slide 40

Assessment: 3 types of items

Transfer

Slide 41

Preliminary Results: AnswersHigher and Lower Ability Students'

Performance on Answers (Solvable)

0

10

20

30

40

50

60

Pretest Posttest

Test Time

% C

orr

ect

Table Low

Table High

Diagram Low

Diagram High

Main effect of test time: F (1, 38) = 29.5, p < .01

Slide 42

Preliminary Results: ReasonsHigher and Lower Ability Students'

Performance on Reasons

0

10

20

30

40

50

60

Pretest Posttest

Test Time

% C

orr

ect

Table Low

Table High

Diagram Low

Diagram High

Main effect of test time: F (1, 38) = 65.7, p < .01

Slide 43

Preliminary Results: TransferHigher Ability Students: Transfer Performance

0

5

10

15

20

25

30

35

40

Pretest Posttest

Test Time

% C

orr

ect

Table HighDiagram High

Lower Ability Students: Transfer Performance

0

5

10

15

20

25

30

35

40

Pretest Posttest

Test Time

% C

orr

ect

Table LowDiagram Low

3-way interaction: Test Time * Condition * Ability: F (1, 38) = 4.3, p < .05