Testing in the classroom: Using tests to promote learning
Richard P. Phelps Universidad Finis Terrae, Santiago, Chile January
7, 2014
Slide 2
Q. What is a standardized test? A. An assessment with at least
one aspect in its content or administration standardized. Q. What
is the key advantage of standardized testing? A. It is
standardized.
Slide 3
2012, Richard P PHELPSWorld Association of Education Research,
17th Congress, Reims, June, 2012 3 Meta-analysis A method for
summarizing a large research literature, with a single, comparable
measure.
Slide 4
John Hatties meta-analyses of meta-analyses
Slide 5
John Hatties list Acceleration Classroom behavioral techniques
Vocabulary programs Repeated reading programs Creativity programs
Student prior achievement Self-questioning by students Study skills
Problem-solving teaching Not labeling students Student-centered
teaching Classroom cohesion Pre-term birth weight Peer influences
Classroom management techniques Outdoor-adventure programs Home
environment Socio-economic status 1. 11. 21. 31.
Slide 6
2012, Richard P PHELPS The effect of testing on student
achievement: 1910-2010 Richard P. PHELPS
Slide 7
2012, Richard P PHELPSWorld Association of Education Research,
17th Congress, Reims, June, 2012 7 The effect of testing on student
achievement 12-year long study analyzed close to 700 separate
studies, and more than 1,600 separate effects 2,000 other studies
were reviewed and found incomplete or inappropriate lacking
sufficient time and money, hundreds of other studies will not be
reviewed
Slide 8
2012, Richard P PHELPSWorld Association of Education Research,
17th Congress, Reims, June, 2012 8 Studies included in the
meta-analyses 2.when: a test is newly introduced, or newly removed
quantity of testing is increased or reduced test stakes are
introduced or increased, or removed or reduced
Slide 9
2012, Richard P PHELPSWorld Association of Education Research,
17th Congress, Reims, June, 2012 9 Number of studies of effects, by
methodology type Methodology type Number of studies Number of
effects Quantitative177640 Surveys and public opinion polls (US
& Canada) 247813 Qualitative245 TOTAL6691698
Slide 10
2012, Richard P PHELPSWorld Association of Education Research,
17th Congress, Reims, June, 2012 10 Effect size: Interpretation d
between 0.25 & 0.50 weak effect d between 0.50 et 0.75 medium
effect d more than 0.75 strong effect
Slide 11
2012, Richard P PHELPSWorld Association of Education Research,
17th Congress, Reims, June, 2012 11 Which predictors matter?
Treatment Group Mean Effect Size is made aware of performance, and
control group is not+0.98 receives targeted instruction (e.g.,
remediation)+0.96 is tested with higher stakes than control
group+0.87 is tested more frequently than control group+0.85
Slide 12
Why tests? Students tend to study more, and learn more, when:
they know they will be tested, but not precisely what will be
tested (e.g.) Experiment comparing gains of students with take-home
tests to those with in class tests -- the latter learned
substantially more. when there is reinforcement of material already
studied Mastery learning experiments of 1960s1980s: Students learn
more when asked to recall what they have learned. Up to a point,
the more students are made to actively process information, and
describe it to others, the better they learn.
Slide 13
2012, Richard P PHELPSWorld Association of Education Research,
17th Congress, Reims, June, 2012 13 Surveys and opinion polls:
Regular standardized tests, performance tests Regular tests (N 125)
Performance tests (N 50) Respondent opiniondd Achievement is
increased1.0 weighted by size of study population1.90.5 Instruction
is improved1.0 weighted by size of study population0.9 Tests help
align instruction1.0 weighted by size of study
population0.50.9
Slide 14
2012, Richard P PHELPSWorld Association of Education Research,
17th Congress, Reims, June, 2012 14 Qualitative studies: Effect on
student achievement Direction of effect Number of studiesPercent of
studies Percent without the inferred Positive2048493 Positive
inferred2410 Mixed522 No change834 Negative311 TOTAL244100 244
studies conducted in the past century in over 30 countries
Slide 15
Repeated retrieval during learning is the key to long-term
retention.
Slide 16
10 benefits of testing and their applications to education
Roediger, Putnam and Smith Direct effects of testing SOURCE:
Roediger, Putnam, & Smith, Ten benefits of testing and their
applications to educational practice, Psychology of Learning and
Motivation, 55, 2011. Retrieval practice during tests enhances
retention of the retrieved information (relative to not testing or
even to studying) -- the testing effect Repeated retrieval produces
knowledge that can be retrieved flexibly and transferred to other
situations On open-ended assessments (e.g., essay tests) retrieval
practice induced by tests helps students organize information into
a coherent knowledge base. Repeated retrieval leads to easier
retrieval of related information
Slide 17
10 benefits of testing and their applications to education
Roediger, Putnam and Smith Indirect effects of testing SOURCE:
Roediger, Putnam, & Smith, Ten benefits of testing and their
applications to educational practice, Psychology of Learning and
Motivation, 55, 2011. Students tested frequently study more and
with more regularity. Tests permit students to discover gaps in
their knowledge and adjust their study efforts to focus on
difficult material. Students who study after taking a test learn
more than if they had not taken a test. Students who self-test or
are tested more frequently in class learn more.
Slide 18
10 benefits of testing and their applications to education
Roediger, Putnam and Smith SOURCE: Roediger, Putnam, & Smith,
Ten benefits of testing and their applications to educational
practice, Psychology of Learning and Motivation, 55, 2011. Benefit
1: The Testing Effect: Retrieval Aids Later Retention Benefit 2:
Testing Identifies Gaps in Knowledge Benefit 3: Testing Causes
Students to Learn More from the Next Study Episode Benefit 4:
Testing Produces Better Organization of Knowledge Benefit 5:
Testing Improves Transfer of Knowledge to New Contexts Benefit 6:
Testing can Facilitate Retrieval of Material That was not Tested
Benefit 7: Testing Improves Metacognitive Monitoring Benefit 8:
Testing Prevents Interference from Prior Material when Learning New
Material Benefit 9: Testing Provides Feedback to Instructors
Benefit 10: Frequent Testing Encourages Students to Study
Slide 19
10 benefits of testing and their applications to education
Roediger, Putnam and Smith SOURCE: Roediger, Putnam, & Smith,
Ten benefits of testing and their applications to educational
practice, Psychology of Learning and Motivation, 55, 2011. Benefit
1: The Testing Effect: Retrieval Aids Later Retention Benefit 2:
Testing Identifies Gaps in Knowledge Benefit 3: Testing Causes
Students to Learn More from the Next Study Episode Benefit 4:
Testing Produces Better Organization of Knowledge Benefit 5:
Testing Improves Transfer of Knowledge to New Contexts Benefit 6:
Testing can Facilitate Retrieval of Material That was not Tested
Benefit 7: Testing Improves Metacognitive Monitoring Benefit 8:
Testing Prevents Interference from Prior Material when Learning New
Material Benefit 9: Testing Provides Feedback to Instructors
Benefit 10: Frequent Testing Encourages Students to Study
Slide 20
10 benefits of testing and their applications to education
Roediger, Putnam and Smith Most teachers should be testing much
more frequently, with smaller, shorter, less consequential tests.
Students learns more when they test. But learn best when the tests
are spaced. What is the optimal lapse of time between tests? The
best time to test again is just before students start forgetting
the information. This time lapse is shorter with discrete material,
like mathematics, than with other subjects. Some studies suggest
that math students should be tested at least once a week.
Slide 21
The more high-stakes decision points, the better the student
performance ? SOURCE: Phelps, Benchmarking to the best in
mathematics, Evaluation Review, 2001
Slide 22
Quality control has proportionally greater effect in poorer
countries SOURCE: Phelps, Benchmarking to the best in mathematics,
Evaluation Review, 2001
Slide 23
What testing skills do teachers need for interpreting
information from large-scale tests? Basic understanding of
statistics: - distributions, mean, median, skewness, kurtosis -
sampling error, measurement error - type 1 / type 2 error,
statistical power - sampling (size, representativeness) Protocols
to help them explain tests to others: - to students - to parents -
to the media
Slide 24
What testing skills do teachers need for developing and
administering classroom tests? Practice (with each other) in
writing items / prompts / rubrics : - unambiguous, relevant,
un-biased Learn the optimal frequency, spacing of tests for your
subject field and grade level. Understand that useful assessment
can be very simple: -e.g., save the last few minutes of each class
to assess by asking students to record 2-3 concepts they learned
that day
Slide 25
But, you can only know what students are learning if you
assess. It is easy to know what you are teaching.
