Upload
maxine
View
32
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Conducting Rigorous Evaluations of Interventions That (May) Improve Student Learning: A Researcher/Teacher Reflects. Jon R. Star Michigan State University Harvard University (as of July 2007). About me. Former middle and high school mathematics teacher - PowerPoint PPT Presentation
Citation preview
Conducting Rigorous Evaluations of Interventions That (May) Improve Student
Learning:A Researcher/Teacher Reflects
Jon R. StarMichigan State University
Harvard University (as of July 2007)
2
About me
• Former middle and high school mathematics teacher
• PhD in Educational Psychology, with emphasis on middle/high school students’ learning of mathematics
• Two grants from US Dept. of Ed (IES) exploring interventions designed to improve students’ learning of mathematics
3
Why I was asked to speak today
• Perhaps useful for me to share my experiences as a researcher who designs and conducts rigorous (experimental) evaluations of interventions designed to improve students’ learning of mathematics
• What are the challenges that I face in my work that I feel are endemic to the evaluation work that many of you are trying to do?
4
Research examples (1)
• The benefits of comparison for learning mathematics
• Better to solve problems by viewing worked examples sequentially or side by side?
Example 1
Example 2
p.1
p.2
Example 1 Example 2p.1
5
Research examples (2)
• Learning strategies for solving ratio and proportion problems using schema
• Is using schema is better than not using schema?
?
?= ?
Ratio Problem Schemata
? ?
? ?
If Then
Proportion Problem Schemata
6
Two types of challenges
• Tension between: my role as a researcher, and my heart and soul as a teacher
• Balancing between: the need to design the intervention to be very
‘clean’ to enable an experimental study, and the need to provide experiences for students
that are appreciative of good (and sometimes ‘messy’) instructional practices
7
Researcher vs. teacher
8
Flow of intervention
• Let intervention run its course Give help in standardized way Don’t deviate from the instructional
protocol or script
• If students don’t ‘get it’, spend extra time with them Focus on mastery of the material Spend extra time with struggling
students
9
Scorecard
Flow of intervention
=
10
Assessing learning
• Materials need to discriminate and have room for learning OK with pretest scores of 10% OK with posttest scores of 50%
• Materials should be challenging but students should be able to succeed ‘Failing’ tests is disheartening Average posttest grade of a B?
11
Scorecard
Assessing learning
Flow of intervention
=
12
Relations with schools
• Keep professional distance from schools and teachers Don’t want to ‘spill the beans’ so that
we can use the school in future work Don’t tell if study didn’t work
• Establish deep and long-lasting relationships with schools, teachers, and students Researchers shouldn’t zip in/out Go back and visit; share results
13
Scorecard
Relations with schools
Flow of intervention
Assessing learning
=
14
Ethics of study design
• Experimental design tells us the most about the intervention Control group is necessary There may be “winner” and “loser”
conditions
• All conditions should lead to student learning Design should allow non-treatment
students to experience treatment eventually (if successful)
15
Scorecard
Ethics of study design
Flow of intervention
Assessing learning
Relations with schools
=
16
Two types of challenges
• Tension between: my role as a researcher, and my heart and soul as a teacher
• Balancing between: the need to design the intervention to be very
‘clean’ to enable an experimental study, and the need to provide experiences for students
that are appreciative of good (and sometimes ‘messy’) instructional practices
17
Clean vs. messy
18
Instructional format
• Students working alone is an easier design Logistically Statistically
• Students working with partners is often better for learning Also can help teachers get more
comfortable with this instructional format
19
Scorecard
Instructional format
=
20
Instructional delivery
• Researchers providing instruction is easier: no PD consistency across schools
• Letting students’ regular teachers do the teaching is better: we are unfamiliar with norms we don’t know students’ names
21
Scorecard
Instructional delivery
Instructional format
=
22
Relations with teachers
• Better if teachers don’t know too much about the intervention might bias their interactions with
students
• The best way to have a long term impact on students is by working with teachers on instructional improvement
23
Scorecard
Relations with teachers
Instructional format
Instructional delivery
=