Dylan Wiliam

Deputy Director

Institute of Education, University of London

www.ioe.ac.uk

Leadership and learning in a changing world

Dylan Wiliam

Keynote address to the 2010 North of England Education Conference, January 2010: York, UK

www.dylanwiliam.net

Science

Improving education: science and designWe need to improve student achievement

This requires improving teacher quality

Improving the quality of entrants takes too long

So we have to help the teachers we have improve

Teachers can change in a range of ways

Some will benefit students, and some will not.

Those that do involve changes in teacher practice

Changing practice requires new kinds of teacher learning

And new models of professional development. Design

Raising achievement matters…For individuals Increased lifetime salary Improved healthLonger life

For societyLower criminal justice costsLower health-care costs Increased economic growth

…because the world of work is changing…Which of the following categories of skill is disappearing from the work-

place most rapidly?

1. Routine manual

2. Non-routine manual

3. Routine cognitive

4. Complex communication

5. Expert thinking/problem-solving

…in surprising ways.

Autor, Levy & Murnane, 2003

Beyond Leitch (Patel et al., 2009)

In fact low skill jobs are vanishing…

Over the last eight years, the UK economy has shed 400 no-qualification job every day

Beyond Leitch (Patel et al., 2009)

…and recessions accelerate the trend…

while the value of low skills is decreasing value…

$0.00

$5.00

$10.00

$15.00

$20.00

$25.00

$30.00

$35.00

1973 1975 1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003 2005

Dropout

HS Diploma

Some College

BA/BSc

Prof Degree

Source: Economic Policy Institute

There is only one 21st century skillSo the model that says learn while you’re at school, while you’re young, the skills that you will apply during your lifetime is no longer tenable. The skills that you can learn when you’re at school will not be applicable. They will be obsolete by the time you get into the workplace and need them, except for one skill. The one really competitive skill is the skill of being able to learn. It is the skill of being able not to give the right answer to questions about what you were taught in school, but to make the right response to situations that are outside the scope of what you were taught in school. We need to produce people who know how to act when they’re faced with situations for which they were not specifically prepared. (Papert, 1998)

The test of successful education is not the amount of knowledge that a pupil takes away from school, but his appetite to know and his capacity to learn. If the school sends out children with the desire for knowledge and some idea how to acquire it, it will have done its work. Too many leave school with the appetite killed and the mind loaded with undigested lumps of information. The good schoolmaster is known by the number of valuable subjects which he declines to teach.

The Future of Education (Livingstone, 1941 p. 28)

Where’s the solution?Structure

Smaller/larger high schools K-8 schools/”All-through” schools

Alignment Curriculum reform Textbook replacement

Governance Charter Schools Vouchers

Technology Computers Interactive white-boards

Workforce reforms Classroom assistants

Educational productivity 1996-2008

School effectivenessThree generations of school effectiveness researchRaw results approaches

Different schools get different results Conclusion: Schools make a difference

Demographic-based approaches Demographic factors account for most of the variation Conclusion: Schools don’t make a difference

Value-added approaches School-level differences in value-added are relatively small Classroom-level differences in value-added are large Conclusion: An effective school is a school full of effective classrooms

0.2

0.4

0.6

0.8

1.0

960 1000 1040 1080

% o

f co

hort

reach

ing p

rofici

ency

in 5

su

bje

cts

incl

udin

g E

nglis

h a

nd

Math

em

ati

cs

School contextualized value-added (CVA) score

Differences in CVA are often insignificant…

(Wilson & Piebalga, 2008)

Middle 50%: differences in CVA not significantly different from average

…are transient…

(Leckie & Goldstein, 2009)

Future school effects for the 2014 cohort based on 2007 data with 95% confidence intervals

…and are smallProportion of 16-year olds gaining 5 GCSE grades at grade C or higher 7% of the variability in the proportion achieving this is nothing to do with the

school, so93% of the variability in the proportion achieving this is nothing to do with

the school

So, if 15 students in a class get 5 A*-C in the average school:17 students will do so a “good” school (1sd above mean)13 students will reach proficiency at a “bad” school (1sd below mean)

Turkey . Hungary . Japan .Belgium .Italy .Germany .Austria .Netherlands .Czech Republic .Korea .Slovak Republic .Greece .Switzerland .Luxembourg .Portugal .Mexico .United States .Australia .New Zealand .Spain .Canada .Ireland .Denmark .Poland .Sweden .Norway .Finland .Iceland .

-80

-60

-40

-20

0

20

40

60

80

100

Within schoolsBetween schools explained by social background of schoolsBetween schools explained by social background of studentsBetween schools not explained by social background

Within schools

Between schools

OECD PISA data from McGaw, 2008

OECD

0 20 40 60 80 100

Luxembourg

JapanItaly

SwitzerlandFinland

DenmarkCzech Republic

SwedenHungary

AustriaPortugal

United States

NetherlandsSlovak Republic

KoreaIreland

SpainCanada

MexicoNew Zealand

Germany OECD

United Kingdom

Government schools

Government dependent private

Government independent private

-150 -100 -50 0 50 100

Observed performance difference

Difference after accounting for socio-economic background of students and schools

OECD

It’s the classroom…In the UK, variability at the classroom level is at least 4 times that at school level It doesn’t matter very much which school you go toBut it matters very much which classrooms you are in…

It’s not class size

It’s not the between-class grouping strategy

It’s not the within-class grouping strategy

… and specifically, it’s the teacher…

Barber & Mourshed, 2007

Teacher quality is often ignored…Because it is politically difficultFor teacher unions (who understandably resist performance-related pay)For politicians (who often prefer to focus on teacher supply, rather than

teacher quality)

And because it is hard to pin downAdvanced content matter knowledge 5%Pedagogical content knowledge 15%Further professional qualifications (MA, NBPTS) 5%Total “explained” 25%

But this can result in the pursuit of policies with poor benefit to cost

www.ioe.ac.uk

Some (very simple) analytics of teacher quality

Two important numbersThe correlation between teacher quality and student progressTypical values in the range 0.1 to 0.2

The effect of one year’s instruction, in population standard deviationsTypical values in the range 0.25 to 0.35

The consequence:Those taught by the most effective teachers learn at twice the average rateThose taught by the least effective teachers learn at half the average rate

And furthermore: In the classrooms of the most effective teachers, students from

disadvantaged backgrounds learn at the same rate as those from advantaged backgrounds (Hamre & Pianta, 2005)

Improving teacher quality takes time…A classic labor force issue with 2 (non-exclusive) solutionsReplace existing teachers with better onesHelp existing teachers become even more effectiveReplace existing teachers with better ones? Increasing the quality of entrants to exclude the lowest performing 30%

of teachers would in 30 years, increase average teacher quality by 0.5 standard deviations.

An increase of 0.5 standard deviations in teacher quality increases student achievement by (at most) 0.1 standard deviations

Across the system, this would be 1 standard deviation over R-12One extra student passing a test per class every three years…

…so we have to help the teachers we have improve…Improve the effectiveness of existing teachersThe “love the one you’re with” strategy

Teachers do improve, but slowly…

Leigh, A. (2007). Estimating teacher effectiveness from two-year changes in student test scores.

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0 5 10 15 20 25

Years in service

Extra months per year of learning

Literacy

Numeracy

.

Getting serious about professional developmentLeft to their own devices, teachers will improve, but slowlyThe average improvement in student value-added by a teacher over 20 years

is one-tenth of the difference between a good teacher and a weak teacher on the first day of their teaching career.

Because we have been doing the wrong kind of professional development100 “Baker days”Professional “updating”Recertification

Bigger improvements are possibleProvided we focus rigorously on the things that matterEven when they’re hard to do

Relevant studiesFuchs & Fuchs (1986)

Natriello (1987)

Crooks (1988)

Banger-Drowns, et al. (1991)

Kluger & DeNisi (1996)

Black & Wiliam (1998)

Nyquist (2003)

Dempster (1991, 1992)

Elshout-Mohr (1994)

Brookhart (2004)

Allal & Lopez (2005)

Köller (2005)

Brookhart (2007)

Wiliam (2007)

Hattie & Timperley (2007)

Shute (2008)

Cost/effect comparisons

Intervention Extra months of learning per year

Cost/classroom/yr

Class-size reduction (by 30%) 4 £20k

Increase teacher content knowledge from weak to strong

2 ?

Formative assessment/Assessment for learning

8 £2k

The formative assessment hi-jack…Long-cycle Span: across units, terms Length: four weeks to one year Impact: Student monitoring; curriculum alignmentMedium-cycle Span: within and between teaching units Length: one to four weeks Impact: Improved, student-involved, assessment; teacher cognition about learningShort-cycle Span: within and between lessons Length:

day-by-day: 24 to 48 hours minute-by-minute: 5 seconds to 2 hours

Impact: classroom practice; student engagement

Pareto analysisVilfredo Pareto (1848-1923)Economist and philosopher associated with the 80:20 rule

Pareto improvementA change that can make at least one person better off

without making anyone else worse off.

Pareto efficiency/Pareto optimalityAn allocation of resource is Pareto efficient or Pareto

optimal when there are no more Pareto improvements

Obstacles to Pareto improvementsThe political economy of reform It is very hard to stop people doing valuable things in order to give them

time to do even more valuable things

Creating a climate for improvementTeacher learning is just like any other learning in a highly complex area In the same way that teachers cannot do the learning for their learners,

leaders cannot do the learning for their teachers

What is needed from teachersA commitment to the continuous improvement of practice; andA focus on those things that make a difference to students

What is needed from leadersA commitment to engineer effective learning environments for teachers :

creating expectations for the continuous improvement of practice keeping the focus on the things that make a difference to students providing the time, space, dispensation and support for innovation supporting risk-taking

SummaryImproving student achievement is a national economic priority

Improving student achievement requires improving teacher quality

Improving the quality of entrants is important, but takes too long

So we have to help the teachers already in post to improve

Existing forms of professional development have negligible impact

So we have to concentrate on what has the best cost-benefit ratio

That means formative assessment

But the formative assessment with the biggest impact is hard to do

So we also need different models of professional development

Specifically school-based teacher learning communities

Unprecedented improvements in achievement are possible, if we focus