Embed Size (px)
Citation preview
1111A
mer
ican
Dip
lom
a P
roje
ct11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
rIn
tern
atio
nal
Ben
ch
mar
kin
g
International BenchmarkingWhat it means – what it takes
Washington, September 11, 2009
Andreas SchleicherHead, Indicators and Analysis Division
OECD Directorate for Education
Dimensions for educational benchmarking
National educ, social and economic context
Structures, resource alloc
and policies
Social & economic
outcomes of education
Community and school
characteristics
Student learning, teacher working
conditions
Socio-economic background of
learners
Antecedentscontextualise or
constrain ed policy
The learning environment at
school
Teaching, learning
practices and classroom
climate
Individ attitudes, engagement and
behaviour
Output and performance of
institutions
Quality of instructional
delivery
Quality and distribution of knowledge &
skills
Policy Leversshape educational
outcomes
Outputs and Outcomes
impact of learning
Individual learner
LevelA
Instructional settings
LevelB
Schools, other institutions
LevelC
Country or system
LevelD
Domain 3Domain 2Domain 1
National educ, social and economic context
Structures, resource alloc
and policies
Social & economic
outcomes of education
Community and school
characteristics
Student learning, teacher working
conditions
Socio-economic background of
learners
Antecedentscontextualise or
constrain ed policy
The learning environment at
school
Teaching, learning
practices and classroom
climate
Individ attitudes, engagement and
behaviour
Output and performance of
institutions
Quality of instructional
delivery
Quality and distribution of knowledge &
skills
Policy Leversshape educational
outcomes
Outputs and Outcomes
impact of learning
Individual learner
LevelA
Instructional settings
LevelB
Schools, other institutions
LevelC
Country or system
LevelD
Domain 3Domain 2Domain 1
Dimensions for educational benchmarking
Tertiary-type A graduation rate
A world of change – higher education
Graduate supply
Cost
per
stu
den
t
Tertiary-type A graduation rate
A world of change – higher education
United States
Finland
Graduate supply
Cost
per
stu
den
t
Tertiary-type A graduation rate
A world of change – higher education
Australia
FinlandUnited Kingdom
Tertiary-type A graduation rate
A world of change – higher education
Tertiary-type A graduation rate
A world of change – higher education
Tertiary-type A graduation rate
A world of change – higher education
Tertiary-type A graduation rate
A world of change – higher education
Tertiary-type A graduation rate
A world of change – higher education
Tertiary-type A graduation rate
A world of change – higher education
United States
Australia
Finland
National educ, social and economic context
Structures, resource alloc
and policies
Social & economic
outcomes of education
Community and school
characteristics
Student learning, teacher working
conditions
Socio-economic background of
learners
Antecedentscontextualise or
constrain ed policy
The learning environment at
school
Teaching, learning
practices and classroom
climate
Individ attitudes, engagement and
behaviour
Output and performance of
institutions
Quality of instructional
delivery
Quality and distribution of knowledge &
skills
Policy Leversshape educational
outcomes
Outputs and Outcomes
impact of learning
Individual learner
LevelA
Instructional settings
LevelB
Schools, other institutions
LevelC
Country or system
LevelD
Domain 3Domain 2Domain 1
Dimensions for educational benchmarking
14141414 E
duca
tion Indic
ato
rs
Pro
gra
mm
e20
09 e
dit
ion o
f Ed
uca
tion a
t a G
lance
Contribution of various factors to instructional cost per high school
student as a percentage of GDP per capita (2006)
Percentage points
B7.1
National educ, social and economic context
Structures, resource alloc
and policies
Social & economic
outcomes of education
Community and school
characteristics
Student learning, teacher working
conditions
Socio-economic background of
learners
Antecedentscontextualise or
constrain ed policy
The learning environment at
school
Teaching, learning
practices and classroom
climate
Individ attitudes, engagement and
behaviour
Output and performance of
institutions
Quality and distribution of knowledge &
skills
Policy Leversshape educational
outcomes
Outputs and Outcomes
impact of learning
Individual learner
LevelA
Instructional settings
LevelB
Schools, other institutions
LevelC
Country or system
LevelD
Domain 3Domain 2Domain 1
Dimensions for educational benchmarking
Quality of instructional
delivery
17171717In
tern
atio
nal A
sses
smen
t S
emin
ar11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
r
Le
sso
ns
fro
m P
ISA
for
dev
elo
pin
g a
sses
sm
en
ts
Emilia Romagna
VenetoFriuli Venezia Giulia
Autonoma of Bolzano
Trento
Lombardia
Liguria
Piemonte
445
465
485
505
525
545
565
616
Italy
Basque Country
Galicia
Catalonia
Andalusia
Asturias
Aragon
Castile and Leon
La Rioja
Navarre
Cantabria
445
465
485
505
525
545
565
616
Spain
Average performanceof 15-year-olds on PISA in science
High science performance
Low science performance
… 18 countries perform below this line
I srael
I talyPortugal Greece
Russian Federation
LuxembourgSlovak Republic,Spain,Iceland Latvia
Croatia
Sweden
DenmarkFrancePoland
Hungary
AustriaBelgiumIreland
Czech Republic SwitzerlandMacao- ChinaGermanyUnited Kingdom
Korea
J apanAustralia
Slovenia
NetherlandsLiechtenstein
New ZealandChinese Taipei
Hong Kong- China
Finland
CanadaEstonia
United States LithuaniaNorway
445
465
485
505
525
545
565
616
19191919In
tern
atio
nal A
sses
smen
t S
emin
ar11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
r
Le
sso
ns
fro
m P
ISA
for
dev
elo
pin
g a
sses
sm
en
ts
0
20
40
60
80
100
120
140
Tur
key
Hun
gary
Jap
an
Bel
gium
Ital
y
Ger
man
y
Aus
tria
Net
herl
ands
Cze
ch R
epub
lic
Kor
ea
Slo
vak
Rep
ublic
Gre
ece
Swit
zerl
and
Luxe
mbou
rg
Port
ugal
Mex
ico
Uni
ted
Sta
tes
Aus
tral
ia
New
Zea
land
Spa
in
Can
ada
Irel
and
Den
mar
k
Pola
nd
Swed
en
Nor
way
Fin
land
Icel
and
Consistency in quality standardsVariation in the performance of 15-year-olds in mathematics
20202020In
tern
atio
nal A
sses
smen
t S
emin
ar11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
r
Le
sso
ns
fro
m P
ISA
for
dev
elo
pin
g a
sses
sm
en
ts
- 80
- 60
- 40
- 20
0
20
40
60
80
100Tur
key
Hun
gary
Jap
an
Bel
gium
Ital
y
Ger
man
y
Aus
tria
Net
herl
ands
Cze
ch R
epub
lic
Kor
ea
Slo
vak
Rep
ublic
Gre
ece
Swit
zerl
and
Luxe
mbou
rg
Port
ugal
Mex
ico
Uni
ted
Sta
tes
Aus
tral
ia
New
Zea
land
Spa
in
Can
ada
Irel
and
Den
mar
k
Pola
nd
Swed
en
Nor
way
Fin
land
Icel
and
Variation of performance
between schools
Variation of performance within
schools
Consistency in quality standardsVariation in the performance of 15-year-olds in mathematics
OECD (2004), Learning for tomorrow’s world: First results from PISA 2003, Table 4.1a, p.383.
21212121In
tern
atio
nal A
sses
smen
t S
emin
ar11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
r
Le
sso
ns
fro
m P
ISA
for
dev
elo
pin
g a
sses
sm
en
ts
France=495
- 35 - 25 - 15 - 5 5 15 25 35
Overall science score
I dentifying scientific issues
Explaining phenomena scientifically
Using scientific evidence
Knowledge about science
Earth and space
Living systems
Physical systems
Strengths and weaknesses of countries in science relative to their overall performance
France
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 2.13
Science competencies
Science knowledge
22222222In
tern
atio
nal A
sses
smen
t S
emin
ar11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
r
Le
sso
ns
fro
m P
ISA
for
dev
elo
pin
g a
sses
sm
en
ts
France=495 Czech Republic=512
- 35 - 25 - 15 - 5 5 15 25 35
Overall science score
I dentifying scientific issues
Explaining phenomena scientifically
Using scientific evidence
Knowledge about science
Earth and space
Living systems
Physical systems
Strengths and weaknesses of countries in science relative to their overall performance
Czech Republic
OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Figure 2.13
Scientific competencies
Scientific knowledge
National educ, social and economic context
Structures, resource alloc
and policies
Social & economic
outcomes of education
Community and school
characteristics
Student learning, teacher working
conditions
Socio-economic background of
learners
Antecedentscontextualise or
constrain ed policy
The learning environment at
school
Teaching, learning
practices and classroom
climate
Individ attitudes, engagement and
behaviour
Output and performance of
institutions
Quality of instructional
delivery
Quality and distribution of knowledge &
skills
Policy Leversshape educational
outcomes
Outputs and Outcomes
impact of learning
Individual learner
LevelA
Instructional settings
LevelB
Schools, other institutions
LevelC
Country or system
LevelD
Domain 3Domain 2Domain 1
Dimensions for educational benchmarking
24242424In
tern
atio
nal A
sses
smen
t S
emin
ar11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
r
Le
sso
ns
fro
m P
ISA
for
dev
elo
pin
g a
sses
sm
en
ts
Low policy value
High policy value
Low feasibility
High feasibility
Money pits
Must haves
Low-hanging fruits
Quick wins
Examine individual, institutional and systemic
factors associated with high performance
Establish the relative standing of states on
international standards
Extending the range of competencies through which
outcomes are assessed
Measuring growth in learning
A real-time assessment environment that bridges the gap between formative and
summative assessment .
Monitor educational progress
Assuming that every new skill domain is orthogonal
to all others
25252525A
mer
ican
Dip
lom
a P
roje
ct11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
rIn
tern
atio
nal
Ben
ch
mar
kin
g
What it takes…
26262626A
mer
ican
Dip
lom
a P
roje
ct11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
rIn
tern
atio
nal
Ben
ch
mar
kin
g
Coverage of world economy 77%81%83%85%86%87%
OECD’s PISA assessment of the knowledge and skills of 15-year-olds
Subnational/regional PISA assessments in
Country Coordinated by Reported at Australia National authorities National
level Belgium Regions OECD level
Brazil Regions National level
Canada National authorities National level
Germany National authorities National level
Italy Regions OECD level Mexico National authorities National
level Spain Regions OECD level Switzerland Regions National
level United Kingdom Regions OECD level
27272727A
mer
ican
Dip
lom
a P
roje
ct11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
rIn
tern
atio
nal
Ben
ch
mar
kin
g What it takes Implementing PISA
volume of the tests, e.g.– 3½ hours of main assessment area– 1 hour for each of the minor assessment areas
each student– 2 hours on paper-and-pencil tasks (subset of all questions)– ½ hour for questionnaire on background, learning habits,
learning environment, engagement and motivationschool principals
– questionnaire (school demography, learning environment)
Alternatives Suitable if state performance is main interest
– Curriculum match / assessment match– Embedding PISA items in state tests– Embedding state items in PISA tests
Requires coherence between assessment frameworks .
28282828A
mer
ican
Dip
lom
a P
roje
ct11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
rIn
tern
atio
nal
Ben
ch
mar
kin
g Test Items
A unit structure Authentic stimuli/contexts High proportion of constructed response
items Multiple-choice Short constructed responses Open constructed responses .
29292929A
mer
ican
Dip
lom
a P
roje
ct11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
rIn
tern
atio
nal
Ben
ch
mar
kin
g Main products A set of basic indicators that provide
policy makers with a baseline profile of the knowledge, skills and competencies of students in their state relative to those in other countries
A set of contextual indicators that provide insight into how such skills relate to important demographic, social, economic and educational variables
Trend indicators that become available because of the on-going, cyclical nature of the data collections
A knowledge base for further focused policy analysis.
30303030A
mer
ican
Dip
lom
a P
roje
ct11
Sep
tem
ber
2009
An
dre
as
Sch
leic
he
rIn
tern
atio
nal
Ben
ch
mar
kin
g
Thank you !Thank you !
www.oecd.org; www.pisa.oecd.org– All national and international publications– The complete micro-level database
email: [email protected]
…and remember:
Without data, you are just another person with an opinion
