Linking Evidence to Practice: Data Systems That Can Be Used to

Improve Teaching and Learning

Dan Goldhaber

University of Washington and Urban Institute

2

Why Focus on Teachers?

• Teacher quality is the key to school success– Its impact is far larger than any other quantifiable

schooling input (Coleman et al., 1966; Goldhaber, 2002; Rivkin et al., 2005)

• The estimated impact of teacher quality is large– It can explain more than a full grade-level equivalent in

test performance (Hanushek, 1992)

3

Central Thesis

• K-12 education should be a continuous learning system, but it’s not– Key missing ingredients: data and analysis capacity

• Combined, these could be used to convincingly answer key policy questions

• In the absence of such a system decisions are made based on fads, institutional inertia, and politics

4

We’ve Actually Learned A Lot from Longitudinal Data in Select States

• Data Quality Campaign specifies key characteristics of state longitudinal systems

• The most important schooling research is going on based on data from a select number of states (see CALDER)

• Three lessons about schooling:– Teacher quality is the most important factor over which schools have

control– Teachers effectiveness varies considerably– Teacher credentials are weak, at best, predictors of teacher quality

5

40%

19%

41%

Teacher-level variables

School-level variables

Class-level variables

Teacher Quality Appears to be Primarily “Unobservable”

3%

97%

Less Tangible Aspects of Teacher Quality

Easily Measurable Aspects of Teacher Quality

back

6

Experience Levels

Estimated Teacher Effectiveness in Math by Experience Level

0

0.2

0.4

0.6

0.8

1

1.2

1.4

-1.5 -1 -0.5 0 0.5 1 1.5

Teacher EffectD

ensi

ty

1st year teachers

2nd year teachers

3rd year and beyond

Estimated Teacher Effectiveness in Reading by Experience Level

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

-1.5 -1 -0.5 0 0.5 1 1.5

Teacher Effect

Den

sity

1st year teachers

2nd year teachers

3rd year and beyond

1st year mean-2nd year mean: 0.059** sd2st year mean-3nd year plus mean: 0.026* sd

1st year mean-2nd year mean: 0.050* sd2st year mean-3nd year plus mean: 0.039** sd

back degrees NBCTs

7

Degree Levels

Estimated Teacher Effectiveness in Reading by Degree Status

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

-1.5 -1 -0.5 0 0.5 1 1.5Teacher Effect

Den

sity

Hold Advanced DegreeBachelor's Degree only

Estimated Teacher Effectiveness in Math by Degree Status

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

-1.5 -1 -0.5 0 0.5 1 1.5Teacher Effect

Den

sity

Hold Advanced DegreeBachelor's Degree Only

Difference in means: .005 sd Difference in means: .014 sd

back experience NBCTs

8

NBPTS Certification Status

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

-3 -2 -1 0 1 2 3

Estimated Teacher Effectiveness in Math

Den

sity

Does not holdcertification

Holds national boardcertification

Difference in means: 0.19** sd of teacher quality back degrees experience

9

Bad Data = Limited or Bad Research + Wrong Answers

• In the absence of this linkage one cannot assess most teacher-based policies– Teachers and students are not randomly matched to

schools or classrooms• Miss impacts on subgroups• Aggregation bias• Matching process produces erroneous estimates of the impact of

teacher credentials

10

Hurdles Associated with Student-Teacher Linkages

• FERPA

• Cost

• Politics

11

Conclusions

• We could learn a great deal more about schooling than we do based on the data that are already collected– Data systems need to be able to talk with one another

• We can’t afford not to create data systems that allow us to emprical test the efficacy of schooling policies and practices– The student achievement costs of not having K-12

education be a learning system are not readily apparent, but they are real