Embed Size (px)
Citation preview
Case Studies
Harry Anthony PatrinosWorld Bank
November 2009
1. Private Education Provision2. Comprehensive Education3. Using Assessments
Private Education Provision and Public Finance:
The Netherlands
Origins• 1917: ‘schools to the parents’• Segregation ended conflict• Freedom of education, religion, constitution• Today:
Less religious, unifiedBut schools retain independenceEase of entry
Secondary School Shares
0
10
20
30
40
50
60
70
80
90
100
1840 1860 1880 1900 1920 1940 1960 1980 2000
Public
Private
Primary Schools by Orientation (%)
0.5
0.9
3.2
4.0
27.1
29.3
35.0
Other
Free
Reformed
Private
Protestant
Catholic
Public
PISA 2006
Math
Science
Reading
1 Finland 548 Finland 563 Korea 556 2 Hong Kong 547 Hong Kong 542 Finland 547 3 Korea 547 Canada 534 Hong Kong 536 4 Netherlands 531 Estonia 531 Canada 527 5 Switzerland 530 Japan 531 New Zealand 521 6 Canada 527 New Zealand 530 Ireland 517 7 Macao, China 525 Australia 527 Australia 513 8 Liechtenstein 525 Netherlands 525 Liechtenstein 510 9 Japan 523 Liechtenstein 522 Poland 508
10 New Zealand 522 Korea 522 Sweden 507 11 Belgium 520 Slovenia 519 Netherlands 507
A Top Performer
But Need to Estimate Causal Impact
• Randomized trials are gold standard• Strategy: find variable that matters for
treatment (relevance) but not outcome (exclusion)
• Control for endogeneity in choice (enter private school) arising from selection on unobservables
• A good instrument predicts choice, but not achievement
Religion Important in Selection
• School choice and diversity of providers• Parents choosers• PISA 2006 asks if choice based on religion
50% choose based on religion (Teelken 1998) 91% of private schools religious 59% of all schools religious Private schools make up 65% of all schools
Real Impact of Private Schools Math Reading Science Private school attendance 17.4 (5.4)* 28.3 (5.8)* 17.9 (6.0)* N 3,840 3,840 3,840 R2 0.442 0.352 0.415 Partial R-sq. for 1st stage 0.166 0.166 0.166 F-test 762.64 762.64 762.64
[p-value] 0.0000 0.000 0.000 Anderson-Rubin test
F P-val
Chi-sq( 1) P-val
4.26 0.0390
4.29 0.0384
6.99 0.0082
7.03 0.0080
2.66 0.1033
2.67 0.1022
Source: PISA 2006 Controlling for school, individual, family characteristics, institutional factors
• School choice with public finance, with strong regulations
• Learning achievements with equity• Large private sector with public funding does
not mean weak central role• Rigorous analysis shows private schools have
causal impact on achievement
Summary
The Impact of the 1999 Education Reform in Poland:
Extending Comprehensive Schooling
Goals of 1999 Reform in Poland
• Raise level of education• Equalize education opportunities• Support quality improvements
Change in Structure of SystemOld Structure grade age grade
0 6 0I 7 III 8 IIIII 9 IIIIV 10 IVV 11 VVI 12 VIVII 13 IVIII 14 II
I 15 IIIII 16 IIII 17 IIIV 18 IIIV 19 IV
Basic vocational schools
Comprehensive lower secondary schools
Profiled general
secondary
Zero class (primary schools or kindergartens)
Comprehensive primary schools
General secondary
schools
Secondary vocational schools
Zero class (primary schools or kindergartens)
Comprehensive primary schools
General secondary
schools
Secondary vocational schools
Basic vocational schools
Matura
MaturaMatura Matura
Matura
Exam Exam
Exam
Exam Exam Exam Exam
New Structure
Impressive GainsPISA
460
465
470
475
480
485
490
495
500
505
510
515
2000 2003 2006
PIS
A M
ean
Sco
re
OECD average
Table 1: Top 10 Reading over Time, PISA 2000 2003 2006 1 Finland 549 Finland 543 Korea 556 2 Netherlands 537 Korea 534 Finland 547 3 Canada 535 Canada 528 Hong Kong 536 4 Hong Kong 532 Australia 525 Canada 527 5 Australia 528 Liechtenstein 525 New Zealand 521 6 Ireland 528 New Zealand 522 Ireland 517 7 New Zealand 526 Ireland 515 Australia 513 8 Japan 525 Sweden 514 Liechtenstein 510 9 United Kingdom 524 Netherlands 513 Poland 508
10 Korea 522 Hong Kong 510 Sweden 507
Empirical Method• Estimate distribution of scores change• Assess effects of factors of interest• Extension obligatory education by 1 year
– Postponing vocational education by 1 year• Specifically whether students who were in
vocational schools in 2000 would have similar scores in 2003 or 2006 in new lower secondary comprehensive schools
• Propensity score matching
Results of MatchingFactual and counterfactual scores of students in different upper secondary tracks
Reading achievement 9th grade 1st plausible value
individual scores
PISA 2000 factual weighted mean score (obs)
PISA 2003 factual weighted mean score (obs)
PISA 2003 matched counterfactual score (matched
obs)PISA 2006
factual weighted mean score (obs)
PISA 2006 matched counterfactual score (matched
obs)
Kernel matching 1-1 matching Kernel
matching 1-1 matching
(1) (2) (3) (4) (5) (6) (7)
All schools 480.0(3654)
501.7(4196)
495.6(4151)
499.3(2528)
513.8(5233)
517.3(5229)
514.6(3056)
ISCED 3C schools(vocational)
357.8(983) - 444.5
(4010)453.3(926) - 472.4
(5141)476.0(1090)
ISCED 3B schools(technical)
480.4(1491) - 487.0
(4150)487.9(1527) - 503.8
(5163)504.5(1823)
ISCED 3A schools(general)
543.7(1180) - 524.6
(4064)528.2(1233) - 545.8
(5221)553.2(1376)
ISCED 3A and 3B schools
514.6(2671) - 507.9
(4157)506.1(2206) - 527.5
(5233)525.9(2609)
Results of Matching
Relative score change (difference-in-differences) for students in vocational schools
Relative score change
from PISA 2000 to PISA 2003 from PISA 2000 to PISA 2006
Kernel matching 1-1 matching Kernel matching 1-1 matching
ISCED 3C versus ISCED 3A+3B 93.4 103.0 101.7 107.0
ISCED 3C versus ISCED 3A 105.9 109.8 112.5 108.8
ISCED 3C versus ISCED 3B 80.2 86.7 91.2 94.2
Summary
• Not all transition countries improved• Students might be accustomed to test taking,
but not reason for improvement• Delay of vocational led to change• Increase hours of relevant instruction
Assessments can be used to Inform Policy Decisions:
The Case of Jordan
Results can be used to Trigger Reform
• PISA/TIMSS provide opportunity for countries to:– Evaluate education systems– Establish benchmarks for future tracking– Inform policy responses
Case of Jordan
• Compares education system against others• Analyzes progress towards educational quality• Participates in World Education Indicators
– which benchmarks systems to OECD countries
IAEP II 1990
• Jordan first Arab country to participate in International Assessment of Educational Progress (IAEP II)
• IAEP II study launched simultaneously with Jordan’s review of education system, which led to comprehensive reform
Jordan Participated in TIMSS 1999
• The results of the study came as a shock• About 75% of students in mathematics and
67% of students in science scored lower than the international average
• Jordan ranked 3rd from the bottom in both subjects among the 20 participating countries
Education Reform
• Expert committee established to investigate causes of poor performance
• Item-by-item examination of the test and school curricula
• Jordan re-administered the entire TIMSS examination
• Results identical to those obtained during the first round of testing
• However, the results served to inform efforts to reform educational quality
Actions Taken
1. Establish benchmarks of 13-year-olds’ achievement2. Identify strength and weakness in each subject3. Compare performance of students4. Inform teacher training5. Analyze characteristics related to achievement6. Target negative and positive influences
Jordan, TIMSS Science
400
420
440
460
480
500
1999 2003 2007
Jordan Improves in TIMSS
Top 10 TIMSS Science Performance over Time 1999 2007 change Jordan 450 482 32 Lithuania 488 519 31 Turkey 433 454 21 Tunisia 430 445 15 Iran 448 459 11 Slovenia 533 538 5 United States 515 520 5 Japan 550 554 4 Korea 549 553 4 England 538 542 4
Summary
• Use of results provides significant returns• Cost of assessment is worthwhile, given the
significant benefits• Assessments are part of efforts to evaluate
education systems• Used to establish benchmarks• Most importantly, inform policy responses
