11

Classroom assessment:minute-by-minute and day-to-day

Session 1249T: ASCD Annual Conference 200717 March 2007; Anaheim, CA

Dylan Wiliam,Institute of Education, University of London

www.dylanwiliam.net

22

Overview of presentationWhy raising achievement is

importantWhy investing in teachers is the

answerWhy assessment for learning should

be the focusWhy teacher learning communities

should be the mechanismHow we can put this into practice

33

Raising achievement matters

For individuals Increased lifetime salary Improved health

For society Lower criminal justice costs Lower health-care costs Increased economic growth

44

Where’s the solution?

Structure Small high schools K-8 schools

Alignment Curriculum reform Textbook replacement

Governance Charter schools Vouchers

Technology

55

It’s the classroomVariability at the classroom level

is up to 4 times greater than at school level

It’s not class sizeIt’s not the between-class

grouping strategyIt’s not the within-class grouping

strategyIt’s the teacher

66

Teacher quality A labor force issue with 2 solutions

Replace existing teachers with better ones? No evidence that more pay brings in better teachers No evidence that there are better teachers out there deterred

by certification requirements Improve the effectiveness of existing teachers

The “love the one you’re with” strategy It can be done We know how to do it, but at scale? Quickly? Sustainably?

77

Why assessment for learning?

Several major reviews of the research Natriello (1987): grades K-12 Crooks (1988): grades K-12 Kluger & DeNisi (1996): grades K-16, work Black & Wiliam (1998): K-12 Nyquist (2003): grades 13-16

All find consistent, substantial effects

88

Cost/effect comparisons

Intervention Extra months of learning/yr

Cost/yr

Class-size reduction by 30% (e.g., from 30 to 20)

3 $30k

Increase teacher content knowledge from weak to strong (2 standard deviations)

1.5 ?

Formative assessment/Assessment for learning

6 to 9 $3k

99

Types of formative assessment

Long-cycle Span: across units, terms Length: four weeks to one year

Medium-cycle Span: within and between teaching units Length: one to four weeks

Short-cycle Span: within and between lessons Length:

day-by-day: 24 to 48 hours minute-by-minute: 5 seconds to 2 hours

1010

Effects of formative assessment

Long-cycle Student monitoring Curriculum alignment

Medium-cycle Improved, student-involved, assessment Improved teacher cognition about learning

Short-cycle Improved classroom practice Improved student engagement

1111

Five Key Strategies …

QuestioningEngineering effective classroom discussions, questions, and learning tasks

Feedback Moving learners forward with feedback

Sharing LearningExpectations

Clarifying and sharing learning intentions and criteria for success

Self AssessmentActivating students as the owners of their own learning

Peer AssessmentActivating students as instructional resources for one another

1212

…and one big idea

Use evidence about learning to adapt instruction to meet student needs

1313

Keeping Learning on Track (KLT)

A pilot guides a plane or boat toward its destination by taking constant readings and making careful adjustments in response to wind, currents, weather, etc.

A KLT teacher does the same: Plans a carefully chosen route ahead of time

(in essence building the track) Takes readings along the way Changes course as conditions dictate

1414

Eliciting evidence of student achievement by engineering effective classroom discussions, questions and

learning tasks

1515

Kinds of questions: IsraelWhich fraction is the smallest?

a) 16

, b) 23

, c) 13

, d) 12

.

Success rate 88%

Which fraction is the largest?

Success rate 46%; 39% chose (b)

a) 45

, b) 34

, c) 58

, d) 7

10.

[Vinner, PME conference, Lahti, Finland, 1997]

1616

Misconceptions

1717

Misconceptions

3a = 24

a + b = 16

1818

Molecular structure of water?

1919

Moving learners forward with feedback

2020[Butler(1988) Br. J. Educ. Psychol., 58 1-14]

Feedback Gain Attitudescores none top +ve

bottom -ve

comments 30% all +ve

Kinds of feedback: Israel 264 low and high ability grade 6 students in 12 classes in 4 schools;

analysis of 132 students at top and bottom of each class Same teaching, same aims, same teachers, same classwork Three kinds of feedback: scores, comments, scores+comments

2121[Butler(1988) Br. J. Educ. Psychol., 58 1-14]

Feedback Gain Attitudescores none top +ve

bottom -ve

comments 30% all +ve

What do you think happened for the students given both scores and comments:

A: Gain: 30%; Attitude: all +ve

B: Gain: 30%; Attitude: top +ve, bottom -ve

C: Gain: 0%; Attitude: all +ve

D: Gain: 0%; Attitude: top +ve, bottom -ve

E: Something else

Responses

2222[Butler (1987) J. Educ. Psychol. 79 474-482]

Kinds of feedback: Israel (2)

200 grade 5 and 6 Israeli students Divergent thinking tasks 4 matched groups

experimental group 1 (EG1); comments experimental group 2 (EG2); grades experimental group 3 (EG3); praise control group (CG); no feedback

Achievement EG1>(EG2≈EG3≈CG)

Ego-involvement (EG2≈EG3)>(EG1≈CG)

2323

Effects of feedback

Kluger & DeNisi (1996) Review of 3000 research reports Excluding those:

without adequate controls with poor design with fewer than 10 participants where performance was not measured without details of effect sizes

left 131 reports, 607 effect sizes, involving 12652 individuals Average effect of feedback substantial, but

Effect sizes very variable 40% of effect sizes were negative

2424

Sharing learning intentions and success criteria

Activating students as owners of their own learning

Activating students as instructional resources for one another

2525[White & Frederiksen, Cognition & Instruction, 16(1), 1998].

Student involvement in learning

3 teachers each teaching 4 year 8 science classes in two US schools

14 week experiment 7 two-week projects, scored 2-10 All teaching the same, except: For a part of each week

Two of each teacher’s classes discusses their likes and dislikes about the teaching (control)

The other two classes discusses how their work will be assessed

2626

Student involvement in learning (2)

7.47.26.7

6.65.94.6

Reflective assessment

Likes and dislikes

HighMiddleLowGroup

Iowa Test of Basic Skills

2727

Techniques: questioning Key idea: questioning should

cause thinking provide data that informs teaching

Improving teacher questioning generating questions with colleagues closed v open low-order v high-order appropriate wait-time

Getting away from I-R-E basketball rather than serial table-tennis ‘No hands up’ (except to ask a question) class polls to review current attitudes towards an issue ‘Hot Seat’ questioning

All-student response systems ABCD cards, Mini white-boards, Exit passes

2828

Questioning in math: discussion

Look at the following sequence:3, 7, 11, 15, 19, ….

Which is the best rule to describe the sequence?A. n + 4B. 3 + nC. 4n - 1D. 4n + 3

2929

Questioning in math: diagnosis

In which of these right triangles is a2 + b2 = c2 ?

A a

c

b

C b

c

a

E c

b

a

B a

b

c

D b

a

c

F c

a

b

3131

Questioning in science: discussion

Ice-cubes are added to a glass of water. What happens to the level of the water as the ice-cubes melt?

A. The level of the water dropsB. The level of the water stays the sameC. The level of the water increasesD. You need more information to be sure

3232

Wilson & Draney, 2004

Questioning in science: diagnosis

The ball sitting on the table is not moving. It is not moving because:

A. no forces are pushing or pulling on the ball. B. gravity is pulling down, but the table is in the way.C. the table pushes up with the same force that gravity pulls downD. gravity is holding it onto the table. E. there is a force inside the ball keeping it from rolling off the table

3333

Save the ozone layer

What can we do to preserve the ozone layer?A. Reduce the amount of carbon dioxide produced

by cars and factoriesB. Reduce the greenhouse effectC. Stop cutting down the rainforestsD. Limit the numbers of cars that can be used when

the level of ozone is highE. Properly dispose of air-conditioners and fridges

3434

Questioning in English: discussion

Macbeth: mad or bad?

3535

Questioning in English: diagnosis

Where is the verb in this sentence?

The dog ran across the road

A B C D

3636

A B C D

Questioning in English: diagnosis

Where does the subject end and the predicate begin in this sentence?

The dog ran across the road.

3737

Questioning in English: diagnosis

Which of these is a good thesis statement?A. The typical TV show has 9 violent incidentsB. There is a lot of violence on TVC. The amount of violence on TV should be reducedD. Some programs are more violent than othersE. Violence is included in programs to boost ratingsF. Violence on TV is interestingG. I don’t like the violence on TVH. The essay I am going to write is about violence on TV

3838

Questioning in history: discussion

In which year did World War II begin?A. 1919B. 1937C. 1938D. 1939E. 1941

3939

Questioning in HistoryWhy are historians concerned with bias when analyzing sources?A. People can never be trusted to tell the truthB. People deliberately leave out important detailsC. People are only able to provide meaningful

information if they experienced an event firsthand

D. People interpret the same event in different ways, according to their experience

E. People are unaware of the motivations for their actions

F. People get confused about sequences of events

4040

What’s wrong with this item?

There are two flights per day from Newtown to Oldtown. The first flight leaves Newtown each day at 9:05 and arrives in Oldtown at 10:45. The second flight from Newtown leaves at 2:15. At what time does the second flight arrive in Oldtown? Show your work.

4141

Figurative language

A. AlliterationB. HyperboleC. MetaphorD. OnomatopoeiaE. PersonificationF. None of the above

1. He was a bull in a china shop.2. May I have a drop of water?3. This backpack weighs a ton.4. The sweetly smiling sunshine…5. He honked his horn at the cyclist.6. I’ve told you a million times already.7. The Redcoats are coming!8. He was as tall as a house.

Triangle shirt waist factory fire, March 25th, 1911

4343

Triangle factory fire

Which of the following sources is biased?A. Photograph of the eventB. New York Times story on Mar 26, 1911C. Description of the fire in the textbookD. Transcript of talk by Frances Perkins, Sep 30 1964

4444

Techniques: feedback

Key idea: feedback should cause thinking provide guidance on how to improve

Comment-only grading Focused grading Explicit reference to mark-schemes and scoring guides Suggestions on how to improve

‘Strategy cards’ ideas for improvement Not giving complete solutions

Re-timing assessment (eg two-thirds-of-the-way-through-a-unit test)

4545

Techniques: sharing learning intentions

Explaining learning intentions at start of lesson/unit Learning intentions Success criteria

Intentions/criteria in students’ language Posters of key words to talk about learning

eg describe, explain, evaluate Planning/writing frames Annotated examples of different standards to ‘flesh out’

assessment rubrics (e.g. lab reports) Opportunities for students to design their own tests

4646

Practical techniques:peer and self-assessment Students assessing their own/peers’ work

with rubrics with exemplars “two stars and a wish”

Training students to pose questions/identifying group weaknesses

Self-assessment of understanding Traffic lights Red/green discs

End-of-lesson students’ review

4747

Putting it into practice

4848

+ –

Force-field analysis (Lewin, 1951)

What kinds of forces present in your school/district will promote or support the development of this kind of work?

What kinds of forces present in your school/district will oppose or constrain the development of this kind of work?

4949

A model for teacher learning

Content (what we want teachers to change) Evidence Ideas (strategies and techniques)

Process (how to go about change) Choice Flexibility Small steps Accountability Support

5050

Supporting Teachers and Schools to Change through Teacher Learning

Communities

5151

Implementing AfL requires changing habits, not adding knowledge Teachers “know” most of this already So the problem is not a lack of knowledge It’s a lack of understanding what it means to do AfL That’s why telling teachers what to do doesn’t work Experience alone is not enough—if it were, then the most

experienced teachers would be the best teachers—we know that’s not true (Hanushek, 2005)

People need to reflect on their experiences in systematic ways that build their accessible knowledge base, learn from mistakes, etc. (Bransford, Brown & Cocking, 1999)

5252

That’s what TLCs are for: TLCs contradict teacher isolation TLCs reprofessionalize teaching by valuing teacher

expertise TLCs deprivatize teaching so that teachers’ strengths and

struggles become known TLCs offer a steady source of support for struggling

teachers They grow expertise by providing a regular space, time,

and structure for that kind of systematic reflecting on practice

They facilitate sharing of untapped expertise residing in individual teachers

They build the collective knowledge base in a school

5353

The synergyContent: assessment for learningProcess: teacher learning communitiesComponents of a model

Initial workshops Support for TLC leaders Monthly TLC meetings Peer observations ‘Drip-feed’ resources

Web-siteWritingsNew ideas

5454

Summary Raising achievement is important Raising achievement requires improving teacher

quality Improving teacher quality requires teacher

professional development To be effective, teacher professional development

must address What teachers do in the classroom How teachers change what they do in the classroom

AfL + TLCs A point of (uniquely?) high leverage A “Trojan Horse” into wider issues of pedagogy,

psychology, and curriculum