PISA for Development Initial Technical Meeting Overview Presentation 27 – 28 June 2013

PISA for Development

Initial Technical Meeting

Overview Presentation27 – 28 June 2013

Paris, France

EDU/DCD

PISA for DevelopmentInitial Technical Meeting

2

1. Shared understanding among participants and partners

2. A general agreement regarding the main technical challenges to be addressed

3. Framework established for the working methods and focus of the technical partnerships

4. Proposals presented for membership of the Steering Group, the Technical Advisory Group, and for the first set of technical papers to be commissioned

Expected Results from Meeting

3

• PISA for development: presentation of the project

• Roundtable on country and development partner perspectives

• The experience of Brazil in PISA• Stock-take of main technical

challenges

Overview of agenda and sessions: Day One

4

• Review of main technical challenges• Building on existing work• Framework for working methods

and technical partnerships• Next steps and meeting conclusion

Overview of agenda and sessions: Day Two

5

• Annotated agenda• Participants list• Final version of project document• Draft ToR for International Steering Group• Draft ToR for Technical Advisory Group• Draft Agreement for participation in PISA

for development• Draft roles and responsibilities expected

of National Centres

Key documents in folder

PISA for Development

• Erik Solheim – Chairman of the Development Assistance Committee

• Andreas Schleicher – Deputy Director, Directorate for Education and Skills

PISA for Development

Initial Technical Meeting

8

An ambition to make the benefits of PISA available to a broader group of countries

An opportunity to support the measurement of a post-2015 education goal that is focused on learning quality

• An aim to enable all of the countries of the global community to be on a single metric with regards to key educational outcomes

Motivation for PISA for development

9

• The international education community that aims to improve student outcomes globally through research;

• In-depth comparative analyses of factors that are strongly related to student outcomes;

• National and international debates about how to improve student outcomes;

Benefits of participation 1: being part of ..............

10

• Participants in PISA are exposed to world-class assessments and rigorous international standards.

• PISA allows countries to learn policy lessons from other countries, particularly from those that may share common challenges and conditions, thereby facilitating peer learning and the dissemination of good practices and knowledge of what works to improve student outcomes

Benefits of participation 2: good practices and peer learning

11

• Focus on student learning for policy reforms

• Focus on teacher professional development and training

• School leadership and school autonomy • Focus on standards that are congruent

and aligned • Focus on student competencies and

skills beyond (curricular) content

Benefits of participation 3: policy/system reform impacts

12

Increased numbers of developing countries use PISA assessments from 2016 onwards to: Monitor progress towards national improvement

targets

Comparatively analyse factors associated with student outcomes

For institutional capacity-building, and

For tracking international education targets within a post-2015 framework

PISA for DevelopmentEnhancing the relevance and use of PISA

… in support of participating countries

Expected impact

• So far, PISA has covered 74 countries, 28 of which are developing countries (ODA recipients)

• Our experience so far:• Relevance of the PISA approach and assessment• Adherence to technical standards• Quality of technical implementation matches that of OECD countries• Full compliance with confidentiality requirements

• Some open issues• Match between student ability distribution and item difficulties• Relevance of context questionnaires• Out-of-school populations

• How far can we go with expanding PISA as we know it geographically?

PISA for DevelopmentHow we arrived at this point – the journey

140 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000

300

350

400

450

500

550

600

Malta

Czech Rep.

Slovak Rep.

Trinidad and Tobago

Estonia

Croatia

Hungary

Shanghai-China

PolandLatvia

ChileLithuania

UruguayMiranda-Venezuela

Russia

Brazil

Turkey

Argentina

MexicoMalaysia

Kazakhstan

MauritiusRomania

Costa Rica

Panama

Montenegro

BulgariaColombia

Serbia

Azerbaijan

Peru

Chinese Taipei

Thailand

JordanTunisiaAlbania

Indonesia

GeorgiaMoldova

Tamil Nadu

Himachal Pradesh

Kyrgyzstan

Non-high income countries (GDP < 20 000)

GDP per capita (USD converted using purchasing power parity)

PISA

rea

ding

per

form

ance

Reading performance and national wealth in low income countries

PISA for DevelopmentBenchmarking for improvement

150 10000 20000 30000 40000 50000 60000 70000 80000 90000 100000

300

350

400

450

500

550

600

High income countries (GDP > 20 000)Linear (High income countries (GDP > 20 000))Linear (High income countries (GDP > 20 000))Non-high income countries (GDP < 20 000)Linear (Non-high income countries (GDP < 20 000))

GDP per capita (USD converted using purchasing power parity)

PISA

rea

ding

per

form

ance

Reading performance and national wealth

PISA for DevelopmentBenchmarking for improvement

16

• Initial dialogue with the largest donors to education and technical agencies, Summer 2012

• Meeting in November 2012 to discuss first draft of Project Document

• Second draft of Project Document circulated in March 2013 – initial commitments of support from development partners and technical agencies

• Dialogue with potential pilot countries

Discussion with Partners: timeline

17

• Cambodia• Ecuador• Guatemala• Mongolia• Punjab (Pakistan)• Senegal• Sri Lanka• Zambia

Dialogue with possible pilot countries

18

• France• Germany (BMZ/GIZ)• IADB• Korea• Norway• UK (DFID)• World Bank

Dialogue with development partners

19

• PASEC• SACMEQ• UNESCO• UIS• UNICEF• EFA GMR• GPE• CUE, Brookings Institution

Dialogue with other agencies

20

• Aims: to enhance the policy relevance of PISA for developing countries through:– the development of enhanced PISA

survey instruments and data collection methods…..

– that are more relevant for the contexts found in developing countries…..

– but which produce results on the same scales as the main PISA assessment.

PISA for DevelopmentEnhancing the relevance and use of PISA

21

PISA for DevelopmentPlans for the Pilot

… in support of developing countries

Five (to seven) partner countries

Five main outputs Five project phases

over 36 months

Main Phases and Governance

I. Design, Planning and CoordinationII. Technical DevelopmentIII. Field Trial and In-country Data

CollectionIV. Analysis and ReportingV. Post-Pilot Governance

36 Months of Implementation

PISA for DevelopmentEnhancing the relevance and use of PISA

23

Five Main Outputs

1. Contextual questionnaires and data-collection instruments enhanced (e.g. for students, parents, schools, etc.): Examples:• Socio-economic background of

students and schools• School climate• Resource availability and use• Autonomy and accountability• Governance

PISA for DevelopmentEnhancing the relevance and use of PISA

24

Five Main Outputs

2. The descriptive power of cognitive assessments enhanced in reading, mathematics and science

• Review, select, translate, validate and combine into test booklets

• Improved targeting at lower-end of proficiency

• Component Reading Skills assessment

PISA for DevelopmentEnhancing the relevance and use of PISA

25

Five Main Outputs

3. An approach developed, including a methodology and analytical framework, for including out-of-school 15 year-olds in the assessments

• Draw on existing work UNICEF, UNESCO

• World Bank STEP project• PIAAC and others

PISA for DevelopmentEnhancing the relevance and use of PISA

26

Five Main Outputs

4. Country capacity in assessment, analysis and use of results for monitoring and improvement strengthened among participating countries

• Planning process with each participant• Customised country-specific report for

participants• Working with development partners• Succession planning and knowledge

transfer

PISA for DevelopmentEnhancing the relevance and use of PISA

27

Five Main Outputs

5. Peer-to-peer learning opportunities identified related to participation in PISA

• Experience of previous and current PISA participants

• Seminar for pilot countries and others• Contribute to UN-led post-2015

discussions

PISA for DevelopmentEnhancing the relevance and use of PISA

28

Main Phases and Governance

International Steering Group

Technical Oversight and Coordination

International and National

Implementation

Partnership with and guidance from 5 (to 7) participating countries

PISA for DevelopmentManagement and budget

Overall budget (5 participating countries = Eur 2.9 million)

PISA for Development

Initial Technical Meeting

PISA for DevelopmentRoundtable countries and development partners

• Main expectations for the meeting of countries and development partners

• Perspectives of countries and development partners on the project, specifically: Country-specific opportunities and

challenges from participating in PISA Perspectives of development partners

on the opportunities and challenges

31

PanamaIndonesia

Costa RicaTurkey

AzerbaijanColombia

MexicoAlbania

UruguayBrazil

KyrgyzstanArgentina

BulgariaThailandMauritius

PeruGeorgia

Russian FederationTrinidad and Tobago

MalaysiaPartner averageOECD average

OECD average (excl. Turkey, Mexico, Chile)

The PISA InstrumentsEnrolment of 15-year-old students

These results from PISA 2009 for non-OECD countries (and Mexico and Turkey) show that among PISA participants, there are still large percentages of out-of-school youth