11
Classroom assessment:minute-by-minute and day-to-day
Session 1249T: ASCD Annual Conference 200717 March 2007; Anaheim, CA
Dylan Wiliam,Institute of Education, University of London
www.dylanwiliam.net
22
Overview of presentationWhy raising achievement is
importantWhy investing in teachers is the
answerWhy assessment for learning should
be the focusWhy teacher learning communities
should be the mechanismHow we can put this into practice
33
Raising achievement matters
For individuals Increased lifetime salary Improved health
For society Lower criminal justice costs Lower health-care costs Increased economic growth
44
Where’s the solution?
Structure Small high schools K-8 schools
Alignment Curriculum reform Textbook replacement
Governance Charter schools Vouchers
Technology
55
It’s the classroomVariability at the classroom level
is up to 4 times greater than at school level
It’s not class sizeIt’s not the between-class
grouping strategyIt’s not the within-class grouping
strategyIt’s the teacher
66
Teacher quality A labor force issue with 2 solutions
Replace existing teachers with better ones? No evidence that more pay brings in better teachers No evidence that there are better teachers out there deterred
by certification requirements Improve the effectiveness of existing teachers
The “love the one you’re with” strategy It can be done We know how to do it, but at scale? Quickly? Sustainably?
77
Why assessment for learning?
Several major reviews of the research Natriello (1987): grades K-12 Crooks (1988): grades K-12 Kluger & DeNisi (1996): grades K-16, work Black & Wiliam (1998): K-12 Nyquist (2003): grades 13-16
All find consistent, substantial effects
88
Cost/effect comparisons
Intervention Extra months of learning/yr
Cost/yr
Class-size reduction by 30% (e.g., from 30 to 20)
3 $30k
Increase teacher content knowledge from weak to strong (2 standard deviations)
1.5 ?
Formative assessment/Assessment for learning
6 to 9 $3k
99
Types of formative assessment
Long-cycle Span: across units, terms Length: four weeks to one year
Medium-cycle Span: within and between teaching units Length: one to four weeks
Short-cycle Span: within and between lessons Length:
day-by-day: 24 to 48 hours minute-by-minute: 5 seconds to 2 hours
1010
Effects of formative assessment
Long-cycle Student monitoring Curriculum alignment
Medium-cycle Improved, student-involved, assessment Improved teacher cognition about learning
Short-cycle Improved classroom practice Improved student engagement
1111
Five Key Strategies …
QuestioningEngineering effective classroom discussions, questions, and learning tasks
Feedback Moving learners forward with feedback
Sharing LearningExpectations
Clarifying and sharing learning intentions and criteria for success
Self AssessmentActivating students as the owners of their own learning
Peer AssessmentActivating students as instructional resources for one another
1212
…and one big idea
Use evidence about learning to adapt instruction to meet student needs
1313
Keeping Learning on Track (KLT)
A pilot guides a plane or boat toward its destination by taking constant readings and making careful adjustments in response to wind, currents, weather, etc.
A KLT teacher does the same: Plans a carefully chosen route ahead of time
(in essence building the track) Takes readings along the way Changes course as conditions dictate
1414
Eliciting evidence of student achievement by engineering effective classroom discussions, questions and
learning tasks
1515
Kinds of questions: IsraelWhich fraction is the smallest?
a) 16
, b) 23
, c) 13
, d) 12
.
Success rate 88%
Which fraction is the largest?
Success rate 46%; 39% chose (b)
a) 45
, b) 34
, c) 58
, d) 7
10.
[Vinner, PME conference, Lahti, Finland, 1997]
1616
Misconceptions
1717
Misconceptions
3a = 24
a + b = 16
1818
Molecular structure of water?
1919
Moving learners forward with feedback
2020[Butler(1988) Br. J. Educ. Psychol., 58 1-14]
Feedback Gain Attitudescores none top +ve
bottom -ve
comments 30% all +ve
Kinds of feedback: Israel 264 low and high ability grade 6 students in 12 classes in 4 schools;
analysis of 132 students at top and bottom of each class Same teaching, same aims, same teachers, same classwork Three kinds of feedback: scores, comments, scores+comments
2121[Butler(1988) Br. J. Educ. Psychol., 58 1-14]
Feedback Gain Attitudescores none top +ve
bottom -ve
comments 30% all +ve
What do you think happened for the students given both scores and comments:
A: Gain: 30%; Attitude: all +ve
B: Gain: 30%; Attitude: top +ve, bottom -ve
C: Gain: 0%; Attitude: all +ve
D: Gain: 0%; Attitude: top +ve, bottom -ve
E: Something else
Responses
2222[Butler (1987) J. Educ. Psychol. 79 474-482]
Kinds of feedback: Israel (2)
200 grade 5 and 6 Israeli students Divergent thinking tasks 4 matched groups
experimental group 1 (EG1); comments experimental group 2 (EG2); grades experimental group 3 (EG3); praise control group (CG); no feedback
Achievement EG1>(EG2≈EG3≈CG)
Ego-involvement (EG2≈EG3)>(EG1≈CG)
2323
Effects of feedback
Kluger & DeNisi (1996) Review of 3000 research reports Excluding those:
without adequate controls with poor design with fewer than 10 participants where performance was not measured without details of effect sizes
left 131 reports, 607 effect sizes, involving 12652 individuals Average effect of feedback substantial, but
Effect sizes very variable 40% of effect sizes were negative
2424
Sharing learning intentions and success criteria
Activating students as owners of their own learning
Activating students as instructional resources for one another
2525[White & Frederiksen, Cognition & Instruction, 16(1), 1998].
Student involvement in learning
3 teachers each teaching 4 year 8 science classes in two US schools
14 week experiment 7 two-week projects, scored 2-10 All teaching the same, except: For a part of each week
Two of each teacher’s classes discusses their likes and dislikes about the teaching (control)
The other two classes discusses how their work will be assessed
2626
Student involvement in learning (2)
7.47.26.7
6.65.94.6
Reflective assessment
Likes and dislikes
HighMiddleLowGroup
Iowa Test of Basic Skills
2727
Techniques: questioning Key idea: questioning should
cause thinking provide data that informs teaching
Improving teacher questioning generating questions with colleagues closed v open low-order v high-order appropriate wait-time
Getting away from I-R-E basketball rather than serial table-tennis ‘No hands up’ (except to ask a question) class polls to review current attitudes towards an issue ‘Hot Seat’ questioning
All-student response systems ABCD cards, Mini white-boards, Exit passes
2828
Questioning in math: discussion
Look at the following sequence:3, 7, 11, 15, 19, ….
Which is the best rule to describe the sequence?A. n + 4B. 3 + nC. 4n - 1D. 4n + 3
2929
Questioning in math: diagnosis
In which of these right triangles is a2 + b2 = c2 ?
A a
c
b
C b
c
a
E c
b
a
B a
b
c
D b
a
c
F c
a
b
3131
Questioning in science: discussion
Ice-cubes are added to a glass of water. What happens to the level of the water as the ice-cubes melt?
A. The level of the water dropsB. The level of the water stays the sameC. The level of the water increasesD. You need more information to be sure
3232
Wilson & Draney, 2004
Questioning in science: diagnosis
The ball sitting on the table is not moving. It is not moving because:
A. no forces are pushing or pulling on the ball. B. gravity is pulling down, but the table is in the way.C. the table pushes up with the same force that gravity pulls downD. gravity is holding it onto the table. E. there is a force inside the ball keeping it from rolling off the table
3333
Save the ozone layer
What can we do to preserve the ozone layer?A. Reduce the amount of carbon dioxide produced
by cars and factoriesB. Reduce the greenhouse effectC. Stop cutting down the rainforestsD. Limit the numbers of cars that can be used when
the level of ozone is highE. Properly dispose of air-conditioners and fridges
3434
Questioning in English: discussion
Macbeth: mad or bad?
3535
Questioning in English: diagnosis
Where is the verb in this sentence?
The dog ran across the road
A B C D
3636
A B C D
Questioning in English: diagnosis
Where does the subject end and the predicate begin in this sentence?
The dog ran across the road.
3737
Questioning in English: diagnosis
Which of these is a good thesis statement?A. The typical TV show has 9 violent incidentsB. There is a lot of violence on TVC. The amount of violence on TV should be reducedD. Some programs are more violent than othersE. Violence is included in programs to boost ratingsF. Violence on TV is interestingG. I don’t like the violence on TVH. The essay I am going to write is about violence on TV
3838
Questioning in history: discussion
In which year did World War II begin?A. 1919B. 1937C. 1938D. 1939E. 1941
3939
Questioning in HistoryWhy are historians concerned with bias when analyzing sources?A. People can never be trusted to tell the truthB. People deliberately leave out important detailsC. People are only able to provide meaningful
information if they experienced an event firsthand
D. People interpret the same event in different ways, according to their experience
E. People are unaware of the motivations for their actions
F. People get confused about sequences of events
4040
What’s wrong with this item?
There are two flights per day from Newtown to Oldtown. The first flight leaves Newtown each day at 9:05 and arrives in Oldtown at 10:45. The second flight from Newtown leaves at 2:15. At what time does the second flight arrive in Oldtown? Show your work.
4141
Figurative language
A. AlliterationB. HyperboleC. MetaphorD. OnomatopoeiaE. PersonificationF. None of the above
1. He was a bull in a china shop.2. May I have a drop of water?3. This backpack weighs a ton.4. The sweetly smiling sunshine…5. He honked his horn at the cyclist.6. I’ve told you a million times already.7. The Redcoats are coming!8. He was as tall as a house.
Triangle shirt waist factory fire, March 25th, 1911
4343
Triangle factory fire
Which of the following sources is biased?A. Photograph of the eventB. New York Times story on Mar 26, 1911C. Description of the fire in the textbookD. Transcript of talk by Frances Perkins, Sep 30 1964
4444
Techniques: feedback
Key idea: feedback should cause thinking provide guidance on how to improve
Comment-only grading Focused grading Explicit reference to mark-schemes and scoring guides Suggestions on how to improve
‘Strategy cards’ ideas for improvement Not giving complete solutions
Re-timing assessment (eg two-thirds-of-the-way-through-a-unit test)
4545
Techniques: sharing learning intentions
Explaining learning intentions at start of lesson/unit Learning intentions Success criteria
Intentions/criteria in students’ language Posters of key words to talk about learning
eg describe, explain, evaluate Planning/writing frames Annotated examples of different standards to ‘flesh out’
assessment rubrics (e.g. lab reports) Opportunities for students to design their own tests
4646
Practical techniques:peer and self-assessment Students assessing their own/peers’ work
with rubrics with exemplars “two stars and a wish”
Training students to pose questions/identifying group weaknesses
Self-assessment of understanding Traffic lights Red/green discs
End-of-lesson students’ review
4747
Putting it into practice
4848
+ –
Force-field analysis (Lewin, 1951)
What kinds of forces present in your school/district will promote or support the development of this kind of work?
What kinds of forces present in your school/district will oppose or constrain the development of this kind of work?
4949
A model for teacher learning
Content (what we want teachers to change) Evidence Ideas (strategies and techniques)
Process (how to go about change) Choice Flexibility Small steps Accountability Support
5050
Supporting Teachers and Schools to Change through Teacher Learning
Communities
5151
Implementing AfL requires changing habits, not adding knowledge Teachers “know” most of this already So the problem is not a lack of knowledge It’s a lack of understanding what it means to do AfL That’s why telling teachers what to do doesn’t work Experience alone is not enough—if it were, then the most
experienced teachers would be the best teachers—we know that’s not true (Hanushek, 2005)
People need to reflect on their experiences in systematic ways that build their accessible knowledge base, learn from mistakes, etc. (Bransford, Brown & Cocking, 1999)
5252
That’s what TLCs are for: TLCs contradict teacher isolation TLCs reprofessionalize teaching by valuing teacher
expertise TLCs deprivatize teaching so that teachers’ strengths and
struggles become known TLCs offer a steady source of support for struggling
teachers They grow expertise by providing a regular space, time,
and structure for that kind of systematic reflecting on practice
They facilitate sharing of untapped expertise residing in individual teachers
They build the collective knowledge base in a school
5353
The synergyContent: assessment for learningProcess: teacher learning communitiesComponents of a model
Initial workshops Support for TLC leaders Monthly TLC meetings Peer observations ‘Drip-feed’ resources
Web-siteWritingsNew ideas
5454
Summary Raising achievement is important Raising achievement requires improving teacher
quality Improving teacher quality requires teacher
professional development To be effective, teacher professional development
must address What teachers do in the classroom How teachers change what they do in the classroom
AfL + TLCs A point of (uniquely?) high leverage A “Trojan Horse” into wider issues of pedagogy,
psychology, and curriculum