Research-Based Principles for the Design of Effective Pedagogical Agents

Richard E. Mayer

University of California, Santa Barbara

Workshop on Empirical Research with Pedagogical Agents October 21, 2014

Thanks to collaborators: C. Scott DaPra Gayle Dow W. Lewis Johnson James Lester Sarah Mayer Roxana Moreno Hiller Spires Ning Wang

Thanks to funders: Mellon Foundation National Science Foundation Office of Naval Research

1. Introduction: Rationale, Theory, and Method

2. Herman-the-Bug: Modality, Personalization,

Redundancy, Image and Realism Principles

3. Virtual Factory Tutor: Politeness Principle

4. Dr. Phyz: Segmenting Principle

5. Solar Man: Embodiment, Voice, and Image Principles

6. Conclusion: What Works

Overview

Rationale for Studying the Instructional Effectiveness of Pedagogical Agents

On-screen pedagogical agents have promise for improving computer-based learning, but we are just beginning to test that potential in scientific research.

Players in Pedagogical Agent Research Who What they do How they view PAs Visionaries Inspire Positive Developers Dazzle Positive Educators Connect Positive Investigators Inform Open

Cognitive Theory of Multimedia Learning

Three Kinds of Cognitive Processes During Learning ______________________________________________________________ Process Definition Effects of pedagogical agents Extraneous Cognitive processing during Non-instructional features of learning that does not serve PAs can distract learners. the instructional objective. Essential Cognitive processing during PAs can direct learners’ learning required to mentally attention by highlighting represent the presented key material. material (e.g., selecting). Generative Cognitive processing during PAs can motivate learners learning required to make to engage more deeply by sense of the presented posing interesting tasks. material (e.g., selecting, organizing, integrating). ______________________________________________________________

Interact with PA

with feature X

Interact with PA without

feature X

Take transfer

test

Take transfer

test

Enhanced group

Control group

M = 12 SD = 5

M = 10 SD = 5

Compare learning with a PA who uses or does not use an instructional feature.

Measure learning outcome. (M & SD on transfer)

Compute effect size (d)

d = (12 – 10)/5 = 2/5 = .40

Value Added Method for Research on the Instructional Effectiveness of Pedagogical Agents

Modality Principle: Do people learn better from playing Design-A-Plant when the PA’s words are spoken rather than printed?

Base group: Play Design-A-Plant game. Herman-the-Bug communicates in printed text. Enhanced group: Play Design-a-Plant game. Herman-the-Bug communicates in spoken form. Transfer test: Determine best environment for various plants.

Modality Principle: People learn better when the PA’s words are spoken rather than printed. Source Effect size Moreno et al. (2001, Expt. 4a) 0.60 Moreno et al. (2001, Expt. 4b) 1.58 Moreno et al. (2001, Expt. 5a) 1.41 Moreno et al. (2001, Expt. 5b) 1.71 Moreno & Mayer (2002a, Expt. 1a) 0.93 Moreno & Mayer (2002a, Expt. 1b) 0.62 Moreno & Mayer (2002a, Expt. 1c) 2.79 Moreno & Mayer (2002a, Expt. 2a) 0.74 Moreno & Mayer (2002a, Expt. 2b) 2.24 Median 1.41

Personalization Principle: Do people learn better from playing Design-A-Plant when the PA’s words are in conversational style rather than formal style?

Base group: Play Design-A-Plant game. Herman-the-Bug communicates using formal style (e.g., “This program is about what type of plant survives on different planets”). Enhanced group: Play Design-a-Plant game. Herman-the-Bug communicates in conversational style (e.g., “You are about to begin on a journey, where you will visiting different planets”). Transfer test: Determine best environment for various plants.

Source Effect size Moreno & Mayer (2000, Expt. 3) 1.92 Moreno & Mayer (2000, Expt. 4) 1.49 Moreno & Mayer (2000, Expt. 5) 1.11 Moreno & Mayer (2004, Expt. 1a) 1.58 Moreno & Mayer (2004, Expt. 1b) 1.93 Median 1.58

Personalization Principle: People learn better when the PA’s words are in conversational style rather than formal style.

Redundancy Principle: Do people learn better from playing Design-A-Plant when the PA’s words are printed and spoken rather than spoken alone?

Base group: Play Design-A-Plant game. Herman-the-Bug communicates in spoken text. Enhanced group: Play Design-a-Plant game. Herman-the-Bug communicates in spoken form and with identical on-screen printed text. Transfer test: Determine best environment for various plants.

Redundancy Principle: People do not learn better when the PA’s words are printed and spoken rather than spoken alone.

Source Effect size Moreno & Mayer (2002b, Expt. 2a) 0.19 Moreno & Mayer (2002b, Expt. 2b) 0.25 Median 0.22

Image Principle: Do people learn better from playing Design-A-Plant when the pedagogical agent’s image is on the screen?

Base group: Play Design-A-Plant game. Herman-the-Bug’s image is not on the screen. Enhanced group: Play Design-a-Plant game. Herman-the-Bug’s image is on the screen. Transfer test: Determine best environment for various plants.

Source Effect size Moreno et al. (2001, Expt. 4a) -0.50 Moreno et al. (2001, Expt. 4b) 0.22 Moreno et al. (2001, Expt. 5a) 0.22 Moreno et al. (2001, Expt. 5b) 0.35 Median 0.22

Image Principle: People do not learn better in games when an agent’s image is on the screen.

Realism Principle: Do people learn better from playing Design-A-Plant when the PA in rendered in immersive virtual reality?

Base group: Play Design-A-Plant Game by interacting with Herman-the-Bug on a desktop computer. Enhanced group: Play Design-A-Plant Game by interacting with Herman-the-Bug in immersive virtual reality. Transfer test: Determine best environment for various plants.

Source Effect size Moreno & Mayer (2002a, Expt. 1a-voice) 0.60 Moreno & Mayer (2002a, Expt. 1b-text) -0.60 Moreno & Mayer (2002a, Expt. 2a-voice) -0.56 Moreno & Mayer (2002a, Expt. 2b-text) -0.18 Moreno & Mayer (2002a, Expt. 1a-personal) -0.05 Moreno & Mayer (2002a, Expt. 1b-impersonal) -0.66 Median -0.37

Realism Principle: People do not learn better from playing Design-A-Plant when the PA is rendered in immersive virtual reality.

Politeness Principle: Do people learn better from playing the Virtual Factory game when the pedagogical agent uses polite rather than direct wording?

Base group: Play Virtual Factory with onscreen agent who uses direct wording (e.g., “Press the ENTER key”). Enhanced group: Play Virtual Factory with onscreen agent who uses polite wording (e.g., “Shall we press the ENTER key?”). Transfer test: Answer comprehension questions.

Source Effect size Wang et al. (2008) 0.93

Politeness Principle: People learn better in games when an agent uses polite rather than direct wording.

Segmenting Principle: Do people learn better from an animated multimedia lesson when the PA explains part-by-part or as a continuous presentation?

Base group: Dr. Phyz provides an explanation of electric motors, with a separate segment for each step. Enhanced group: Dr. Phyz explains electric motors in a continuous presentation. Transfer test: Solve problems such as how to design a more powerful electric motor.

Source Effect size Mayer, Dow, & Mayer (2003) 0.82 Mayer, Dow, & Mayer (2003) 0.98 Median 0.90

Segmenting Principle: People learn better from an animated multimedia lesson when the PA explains part-by-part rather than as a continuous presentation.

- -

- -

- -

- -

- - -

-

- -

- - -

- - - -

- -

-

Sunlight

Embodiment Principle: Do people learn better from a multimedia lesson when the PA uses human-like gestures?

Base group: View slideshow on solar cells with motionless pedagogical agent. Enhanced group: View slideshow on solar cells with PA who gestures, maintains eye contact, exhibits facial expression, and moves. Transfer test: Solve problems about how to improve solar cells.

Source Effect size Mayer & DaPra (2012, Expt. 1-human voice) 0.92 Mayer & DaPra (2012, Expt. 2a-human voice) 1.10 Mayer & DaPra (2012, Expt. 2b-machine voice) 0.17 Mayer & DaPra (2012, Expt, 3a-human voice) 0.58 Mayer & DaPra (2012, Expt. 3b-human voice, choice) 0.35 Median 0.58

Embodiment Principle: People learn better from a multimedia lesson when the PA uses human-like gestures.

- -

- -

- -

- -

- - -

-

- -

- - -

- - - -

- -

-

Sunlight

Voice Principle: Do people learn better from a multimedia lesson when the PA speaks in a human voice rather than a machine voice?

Base group: View slideshow on solar cells with PA who speaks in machine voice. Enhanced group: View slideshow on solar cells with PA speaks in a human voice. Transfer test: Solve problems about how to improve solar cells.

Source Effect size Mayer & DaPra (2012, Expt. 2a-high embodiment) 0.63 Mayer & DaPra (2012, Expt. 2b-low embodiment) -0.17

Voice Principle: Do people learn better from a multimedia lesson when an embodied PA speaks in a human voice.

- -

- -

- -

- -

- - -

-

- -

- - -

- - - -

- -

-

Sunlight

Image Principle: Do people learn better from a multimedia lesson when the PA’s image is on the screen?

Base group: View slideshow on solar cells without PA’s image on the screen. Enhanced group: View slideshow on solar cells with motionless PA’s image on the screen. Transfer test: Solve problems about how to improve solar cells.

Source Effect size Mayer & DaPra (2012, Expt. 1) -0.31

Image Principle: People do not learn better from a multimedia lesson when the PA’s image is on the screen.

What Works: Evidence-Based Principles of PA Design Principle Description Lesson d Modality People learn better when the Design-a-Plant 1.41 PA’s words are spoken. Personalization People learn better when the Design-A-Plant 1.58 PA’s words are conversational. Politeness People learn better when the Virtual Factory 0.93 PA’s advice is in polite style. Segmenting People learn better when the Dr. Phyz 0.90 PA explains one part at a time. Embodiment People learn better when the Solar Cells 0.58 PA exhibits human-like gesture. Voice* People learn better when an Solar Cells 0.63 embodied PA speaks in a human voice.

Redundancy People do not learn better Design-a-Plant 0.22 when the PA’s words are printed and spoken rather than spoken alone. Image People do not learn better Design-a-Plant 0.22 when the PA’s static Solar Cell -0.31 image is on the screen. Realism People do not learn better Design-A-Plant -0.37 when the PA is rendered in immersive virtual reality.

What Doesn’t Work: Evidence-Based Principles of PA Design

Principle Description Lesson d

Design of Pedagogical Agents: Summary of Main Points

Rationale: Many strong claims are made for the educational effectiveness of pedagogical agents, but it is useful to take an evidence-based approach guided by cognitive theory.

Method: The research literature can be enhanced by using the

value added method for assessing the educational effectiveness of various features of pedagogical agents.

Results: Research on the educational effectiveness of

pedagogical agents is in its early stage, but several promising design principles are emerging.

Conclusion: When the goal is to help people learn with

pedagogical agents, their design should be informed by research evidence and grounded in cognitive theory.

Summary of Value Added Research on PA Design

Promising Principles Modality principle Personalization principle Politeness principle Segmenting principle Embodiment principle Voice principle*

Challenged Principles Redundancy principle Image principle Realism principle

Cited References Mayer, R. E., & DaPra, C. S. (2012). An embodiment effect in computer-based learning with an animated pedagogical agent. Journal of Experimental Psychology: Applied, 18, 239-252. Mayer, R. E., Dow, G., & Mayer, S. (2003). Multimedia learning in an interactive self-explaining environment: What works in the design of agent-based micro-worlds? Journal of Educational Psychology, 95, 806-813. Moreno, R. E., & Mayer, R. E. (2000).Engaging students in active learning: The case for personalized multimedia messages. Journal of Educational Psychology, 92, 724-733. Moreno, R. E. & Mayer, R. E. (2002a). Learning science in virtual reality environments: Role of methods and media. Journal of Educational Psychology, 94, 598-610. Moreno, R., & Mayer, R. E. (2002b). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology, 94, 156-163. Moreno, R., & Mayer, R. E. (2004). Personalized messages that promote science learning in virtual environments. Journal of Educational Psychology, 96,165-173. Moreno, R., Mayer, R. E., Spires, H., & Lester, J. (2001). The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19, 177-214. Wang, N., Johnson, W. L., Mayer, R. E., Rizzo, P., Shaw, E., & Collins, H. (2008). The politeness effect: Pedagogical agents and learning outcomes. International Journal of Human Computer Studies, 66, 96-112.

Core References Clark, R. C., & Mayer, R. E. (2011). E-Learning and the science of instruction (3rd ed). San Francisco: Pfeifer. Mayer, R. E. (2009). Multimedia learning (2nd ed). New York: Cambridge University Press. Mayer, R. E. (2014). Computer games for learning: An evidence-based approach. Cambridge, MA: MIT Press. Mayer, R. E. (Ed.). (2014). The Cambridge handbook of multimedia learning (2nd ed). New York: Cambridge University Press.

Thank You

Richard E. Mayer Department of Psychological and Brain Sciences

University of California, Santa Barbara Santa Barbara, CA 93106

mayer@psych.ucsb.edu