Desirable Difficulties in Science Learning: Taking What We Learn in

the College Classroom into the Middle School Classroom

CASL PI MeetingCASL PI MeetingWashington DCWashington DC

May 17-18, 2005May 17-18, 2005Marcia C. Linn and Britte H. Cheng, Marcia C. Linn and Britte H. Cheng,

University of California, BerkeleyUniversity of California, BerkeleyLindsey E. Richland, Robert A. Bjork, and Jason Lindsey E. Richland, Robert A. Bjork, and Jason

FinleyFinleyUniversity of California, Los AngelesUniversity of California, Los Angeles

IDDEASIDDEAS

Introducing Desirable Difficulties for Introducing Desirable Difficulties for Educational Applications in Science Educational Applications in Science

(IDDEAS) http://iddeas.psych.ucla.edu(IDDEAS) http://iddeas.psych.ucla.edu Extend results from laboratory studies Extend results from laboratory studies

to complex science materials to complex science materials Test promising findings from Test promising findings from

laboratory studies in science laboratory studies in science classrooms using a Technology-classrooms using a Technology-Enhanced Learning EnvironmentEnhanced Learning Environment

WISE EnvironmentWISE Environment

Web-based Inquiry Web-based Inquiry Science Science Environment (WISE) Environment (WISE) http://wise.berkeley.eduhttp://wise.berkeley.edu

Delivers instruction Delivers instruction in multiple formatsin multiple formats

Frees teacher to Frees teacher to tutor individualstutor individuals

Gathers embedded Gathers embedded assessments of assessments of student progressstudent progress

Desirable DifficultiesDesirable Difficulties Conditions that introduce difficulties Conditions that introduce difficulties

for the learner can enhance long-for the learner can enhance long-term retention and transfer term retention and transfer

Conditions that appear optimal Conditions that appear optimal during instruction can fail to support during instruction can fail to support long-term retention long-term retention

Desirable difficulties (Bjork, 1994, Desirable difficulties (Bjork, 1994, 1999) include:1999) include:– Today: generating rather than reading Today: generating rather than reading

responsesresponses

Role of GenerationRole of Generation When students respond by answering When students respond by answering

questions, explaining material, making questions, explaining material, making predictions or reflecting they engage in predictions or reflecting they engage in generationgeneration

Much of learning consists of listening to Much of learning consists of listening to lectures, reading texts, or watching lectures, reading texts, or watching demonstrationsdemonstrations

Many theories call for active, hands-on, or Many theories call for active, hands-on, or interactive learning consistent with generationinteractive learning consistent with generation

Generation research can clarify the nature of Generation research can clarify the nature of active learning and extend understanding of active learning and extend understanding of desirable difficultiesdesirable difficulties

Web-based Inquiry Learning Web-based Inquiry Learning EnvironmentEnvironment

An Awful Waste of An Awful Waste of SpaceSpace– Enables learners to Enables learners to

explore the variables explore the variables that determine that determine whether or not a whether or not a planet in another planet in another solar system might solar system might be habitablebe habitable

Generation featuresGeneration features– NotesNotes– JournalJournal– Prediction Prediction – Argument Argument

constructionconstruction

Middle School WISE Project

Generic and directed Generic and directed promptspromptsIntegrated ideas

Directed PromptCondition

Generic PromptCondition

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%Generic prompts require more generation than directed prompts; result in more integrated ideas.

(Davis, E. A. (1998). Scaffolding students' reflection for science learning. Unpublished doctoral dissertation, University of California at Berkeley,Berkeley, CA.)

Laboratories and Laboratories and classroomsclassrooms

Classroom learningClassroom learning– Students work in pairsStudents work in pairs– Students interact with Students interact with

the teacherthe teacher– Students motivated Students motivated

by personally relevant by personally relevant taskstasks

The booming, The booming, buzzing classroom buzzing classroom offers many offers many distractionsdistractions

Watch the videoWatch the video

Undergraduate & Classroom Undergraduate & Classroom StudiesStudies

Extend chain of evidence for generation Extend chain of evidence for generation from studies of phrases to concepts from studies of phrases to concepts

Compare Compare readread to to generategenerate with science with science conceptsconcepts

Compare Compare single conceptsingle concept generation to generation to concept integrationconcept integration about habitability about habitability

Laboratory studies:Laboratory studies: 1 hour of 1 hour of instruction on concepts, 48 hour instruction on concepts, 48 hour retention intervalretention interval

Classroom studies:Classroom studies: 5 class periods, 5 class periods, one week delayed posttestone week delayed posttest

Undergraduate Laboratory Undergraduate Laboratory Studies: Experimental Studies: Experimental

ConditionsConditions Experiment 1:Experiment 1: Sentence Sentence Completion Completion – Read: Read: JovianJovian-type planets -type planets

are mostly made up of gases.are mostly made up of gases. – Generate: Generate: ____-type planets ____-type planets

are mostly made up of gases.are mostly made up of gases. Experiment 2: Experiment 2: Sentence Sentence

level generation, more level generation, more educationally importanteducationally important– Single: [Mass] Describe in Single: [Mass] Describe in

a sentence how the size a sentence how the size of one planet's mass can of one planet's mass can affect another planet. affect another planet.

– Concept Integration: Concept Integration: [Mass + Distance][Mass + Distance]

Undergraduate Laboratory Undergraduate Laboratory Study: Read vs. GenerateStudy: Read vs. Generate

50

55

60

65

70

75

80

85

90

95

100

Instruction Memory Posttest

% Correct

Read Condition

Generate Condition

Undergraduate Laboratory Undergraduate Laboratory Study: Single vs. IntegratedStudy: Single vs. Integrated

0

10

20

30

40

50

60

Instruction Transfer to NewSingle Concept

Questions

Transfer to NewConcept Integration

Questions

Mean Performance

Single Concept Generation

Concept Integration Generation

Undergraduate Laboratory Undergraduate Laboratory Study Illustrative ResponsesStudy Illustrative Responses

Single idea Generation

Single concept generation

Concept integration generation

Prompt: The range of distances from the sun where the temperature allows water to be liquid are called the _____ _____.Student: “habitable zone”

Prompt: Scientists often use a single measurement to talk about a planet's distance from its sun, but why is this practice misleading? Use a distance listed in the table you saw to explain.Student: “The distance for Mars would be misleading because Mars travels in an elliptical orbit and is different distances from the sun at different times.”

Prompt: Using Jupiter's distance from the sun as an example, explain how the measure of an object's weight can shift when it is in different locations, even if that object is a planet.Student: “An object's weight can shift when its in different locations because it's weight depends upon the strongest pull of gravity. People weigh more on Jupiter than they do on Earth because Jupiter's gravitational pull is stronger. If the object is a planet, then changing the distance it is from the sun will change it's weight because it will either feel a strong gravitational pull (if close to the sun) or a weak gravitational pull (if further away from the sun).”

Undergraduate Laboratory Undergraduate Laboratory Studies ImplicationsStudies Implications

Chain of evidence–can generalize Chain of evidence–can generalize paradigm from typical recall studies paradigm from typical recall studies to investigations using typical to investigations using typical science conceptsscience concepts

Counterintuitive “desirable difficulty” Counterintuitive “desirable difficulty” can be used successfully to enhance can be used successfully to enhance instructioninstruction

Opportunities–how does generation Opportunities–how does generation work in typical middle school work in typical middle school classroom settings?classroom settings?

Middle School Classroom Middle School Classroom Study Read vs. GenerateStudy Read vs. Generate

0

2

4

6

8

10

12

14

16

18

20

Number Correct

Study 1

No Generation

Generation

Posttest Performance

Generation led to greater Generation led to greater recall of materialrecall of material

((FF (1,115) = 18.769 , (1,115) = 18.769 , pp =.000) =.000)

Middle School Classroom Middle School Classroom StudyStudy

Single vs. Integrated Single vs. Integrated Posttest Performance

0

5

10

15

20

25

Number Correct

Study 2

No Integration(No DesirableDifficulty)Integration(DesirableDifficulty)

Integrated generation led to Integrated generation led to more sophisticated more sophisticated understanding understanding ((FF (1,172) = 3.946 , (1,172) = 3.946 , pp = <.05) = <.05)

Middle School Classroom StudyMiddle School Classroom StudyIllustrative ResponsesIllustrative Responses

Single idea Generation

Single concept generation

Concept integration generation

Prompt: The range of distances from the sun where the temperature allows water to be liquid are called the _____ _____.Student: “habitable zone”

Prompt: Are planets always the same distance from their sun?Student: “Because planets' orbits are elliptical, scientists calculate the average number of AUs to describe how far a planet it from its sun.”

Prompt: On Jupiter, would your weight, your mass, or both your weight and your mass be more than it is on Earth? Why?Student: “If I were ever on Jupiter, my weight would change because Jupiter is such a large planet. Since it is a large planet it has a gigantic mass, it has a strong gravity pull. Weight is determined by amount of gravity pull on you, so you would weigh more on Jupiter because there has a greater gravity pull. Although, if I were on Jupiter, my mass would stay the same because mass is the amount of matter in an object and that doesn't change if you go to another planet.”

Laboratory and Classroom Laboratory and Classroom FindingsFindings

Generation instructions generally Generation instructions generally improve learningimprove learning– During learning, generation results in During learning, generation results in

more errorsmore errors– On posttest, generation yields better On posttest, generation yields better

understandingunderstanding Generation across topics intensifies Generation across topics intensifies

effectseffects– Single concept generation easier during Single concept generation easier during

learning but less effective that integrated learning but less effective that integrated concept generation on posttestconcept generation on posttest

Current Middle School Study–What Current Middle School Study–What forms of generation improve forms of generation improve

learning?learning?

Broad ComplexBroad Simple

Narrow ComplexNarrow SimpleNarrow

Broad

Scope of Scope of Ideas/Number Ideas/Number

of possible of possible connectionsconnections

Simple Complex

Complexity of Context/Problem Complexity of Context/Problem DifficultyDifficultyFour groups, either within

or across classrooms

Prompts Illustrating Simple, Prompts Illustrating Simple, Complex, Narrow, & Broad Complex, Narrow, & Broad

Generation Generation SIMPLE

(Identify Planets in Habitable Zone)

COMPLEX(Distinguish Planets in Habitable Zone from

Other Planets)

NARROW(Focus on the

Habitable Zone)

In the animation below of the inner four planets of our solar system, at which points is Mars in the Habitable Zone? Explain your answer.

In the animation below of the inner four planets of our solar system, which planets are in the habitable zone? Explain why you think the planets you chose are habitable and others are not.

BROAD (Connect to

factors beyond the habitable zone:

atmosphere, water, temp, etc)

Based on what you see here, What is the main reason Mars is not habitable while Earth is?

Based on what you know of the inner four planets of our solar system (represented below), what kinds of planets should scientists search for if they are looking for life in the universe?

0: Response is irrelevant0: Response is irrelevant 1: Response is incorrect1: Response is incorrect

2: Response is correct but connections not directly relevant 2: Response is correct but connections not directly relevant (non-target connections; not a broad response)(non-target connections; not a broad response)

3: Response links correct and incorrect ideas 3: Response links correct and incorrect ideas

4: Simplified or partial answer with no explanation/example4: Simplified or partial answer with no explanation/example 5: Correct with one step reasoning, partial answer with 5: Correct with one step reasoning, partial answer with

explanation, or full answer with no explanationexplanation, or full answer with no explanation 6: Ideal; includes using correct terminology, multi-step 6: Ideal; includes using correct terminology, multi-step

reasoning; full answer with explanation/sreasoning; full answer with explanation/s

7: Ideal response with elaborated explanation7: Ideal response with elaborated explanation 8: Ideal response with additional connections8: Ideal response with additional connections 9: Ideal response with elaborations and additional 9: Ideal response with elaborations and additional

connectionsconnections

Scoring Rubric: Emphasis on Scoring Rubric: Emphasis on making connectionsmaking connections

Normative Connections

Multiple Connections

Valid, Non NormativeConnections

Invalid, Non Normative

IdealIdeal Responses for Each Responses for Each Condition Condition SIMPLE

(Identify Planets in Habitable Zone)COMPLEX

(Generalization to other planets)

NARROW

(Focus on the

Habitable Zone)

In the animation of the inner four planets of our solar system, at which points is Mars in the Habitable Zone? Explain your answer.

Mars is habitable when it is in the Mars is habitable when it is in the northern most point of its orbit, northern most point of its orbit, because the distance from Mars because the distance from Mars to the sun gets larger as it orbits to the sun gets larger as it orbits away from the northern most away from the northern most point making it too cold for life to point making it too cold for life to survive.survive.

In the animation below of the inner four planets of our solar system, which planets are in the habitable zone? Explain why these planets are habitable and others are not.

The planet that is 1/2 of an AU and The planet that is 1/2 of an AU and the next one that is about 1 AU. We the next one that is about 1 AU. We chose these because they are the chose these because they are the best distance away from the sun. The best distance away from the sun. The other planets are either too close or other planets are either too close or to far from the sun to be in the to far from the sun to be in the habitable zone. habitable zone.

BROAD (Connect to

factors beyond the Habitable

zone: atmosphere,

water, temp, etc)

Based on the animation of the inner four planets, what is the main reason Mars is not habitable while Earth is?

Mars is too far away from the Sun Mars is too far away from the Sun during certain points of its orbit during certain points of its orbit so it is only habitable at certain so it is only habitable at certain points in time. {Earth, however is points in time. {Earth, however is in the habitable zone all year in the habitable zone all year long.}long.}

Based on what you know of the inner four planets, what kinds of planets should scientists search for to find life in the universe?

They should look for a planet within They should look for a planet within the habitable zone and with a climate the habitable zone and with a climate that won't either turn all the water that won't either turn all the water into gas because of so much heat or into gas because of so much heat or turn it all into ice because it's so cold.turn it all into ice because it's so cold.

Middle School Classroom Middle School Classroom StudyStudy

Embedded Notes Embedded Notes Performance on Note about Mass

0

1

2

3

4

5

6

Mean Score

SimpleNarrow

SimpleBroad

ComplexNarrow

ComplexBroad

Preliminary analyses, more Preliminary analyses, more responses still need scoringresponses still need scoring

During learning, simple During learning, simple generation is easiergeneration is easier

Will simple generation be Will simple generation be sufficient for learning?sufficient for learning?

Middle School Classroom Middle School Classroom StudyStudy

Embedded Notes Embedded Notes Performance on Note about Mutual Gravitation

0

1

2

3

4

5

6

Mean Score

SimpleNarrow

SimpleBroad

ComplexNarrow

ComplexBroad

Preliminary analyses, more Preliminary analyses, more responses still need scoringresponses still need scoring

During learning, simple During learning, simple generation is easiergeneration is easier

Will simple generation be Will simple generation be sufficient for learning?sufficient for learning?

Middle School Classroom Middle School Classroom StudyStudy

Embedded Notes Embedded Notes Performance on Note about Atmosphere

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Mean Score

SimpleNarrow

SimpleBroad

ComplexNarrow

ComplexBroad

Preliminary analyses, more Preliminary analyses, more responses still need scoringresponses still need scoring

During learning, simple During learning, simple generation is easiergeneration is easier

Will simple generation be Will simple generation be sufficient for learning?sufficient for learning?

Generation and science Generation and science learninglearning

Active, interactive, hands-on, & Active, interactive, hands-on, & autonomous learning all emphasize autonomous learning all emphasize generationgeneration

Related studies consistent with Related studies consistent with generationgeneration– Chi, Slotta–Generation and self-explanationChi, Slotta–Generation and self-explanation– Kintsch, McNamara, Songer–Organized vs. Kintsch, McNamara, Songer–Organized vs.

complex textcomplex text– Davis–Autonomous learning: Generic and Davis–Autonomous learning: Generic and

Directed promptsDirected prompts

ConclusionsConclusions

Research on generation in Research on generation in undergraduate laboratory studies undergraduate laboratory studies generalizes to complex science generalizes to complex science concepts-chain of evidenceconcepts-chain of evidence

Laboratory findings can generalize to Laboratory findings can generalize to buzzing, booming classroom context buzzing, booming classroom context – Classroom research can respect teacher Classroom research can respect teacher

goals, contribute to student learninggoals, contribute to student learning Research on generation helps clarify Research on generation helps clarify

calls for active, hands-on learning calls for active, hands-on learning opportunitiesopportunities

Make Thinking Visible –Make Thinking Visible – Chemical Reactions Chemical Reactions

QuickTime™ and aAnimation decompressor

are needed to see this picture.

Jennie Chiu

http://wise.berkeley.edu

http://TELSCenter.org

ImplicationsImplications

For designers, design principlesFor designers, design principles– Encourage generation across topics to Encourage generation across topics to

promote lifelong learningpromote lifelong learning– Select focus on generation based on goalsSelect focus on generation based on goals

For learnersFor learners– Test your ideas in multiple contextsTest your ideas in multiple contexts

For classroom teachersFor classroom teachers– Reward generation, use cumulative testsReward generation, use cumulative tests

For researchersFor researchers– Conduct research in complex settingsConduct research in complex settings