PISA 2009.International survey of reading literacy for 15 years old Several slides were used from PPT presentation “What Students know and can do” by Andreas

PISA 2009.International survey of reading literacy for 15 years old

Several slides were used from PPT presentation “What Students know and can do” by Andreas Schleicher.7 December 2010.www.oecd.org

Programme for International Student Assessment

In Kyrgyzstan PISA 2009 survey was undertaken by the CEATM under the order by Ministry of Education and

Science of the Kyrgyz Republic within the WB-financed Rural Education

Project

2

What PISA is?

PISA is an international comparative survey of functional literacy for 15 years old.

Which is executed by: • Organization for Economic Co-operation and

Development (OECD)

• Consortium comprising the world’s leading international testing organizations jointly with the national testing centers

Azerbaijan, Albania, Argentina, Bulgaria, Brazil, Shanghai (China), Hong Kong, Dubai (UAE), Indonesia, Jordan, Kazakhstan, Qatar, Columbia, Kyrgyzstan, Latvia, Lithuania, Lichtenstein, Macao (China), Panama, Peru, Russian Federation, Rumania, Serbia, Singapore, Thailand, Chinese Taipei, Trinidad and Tobago, Uruguay, Croatia, Tunis, Montenegro.

OECD countriesAustria, Australia, Belgium, Great Britain, Hungary, Germany, Greece, Denmark, Israel, Ireland, Spain, Island, Italy, Canada, Luxemburg, Mexico, Netherlands, New Zeeland, Norway, Poland, Portuguese, Slovakia, Slovenia, USA, Turkey, Finland, France, Czech Republic, Chili, Switzerland, Sweden, Estonia, South Korea, Japan.

Partner countries

PISA 2009

4

1998PISA countries 20002001200320062009World Economy 77%81%83%85%86%87%

OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World

Background: http://www.oecd.org/dataoecd/60/1/39727764.ppt

5

• The assessment takes place in three-yearly cycles. First survey was undertaken in 1998.

• Study areas:

- reading literacy (2000),

- mathematics literacy (2003),

- science literacy (2006),

- reading literacy (2009).

What PISA is?

6

PISA goal is…

…to assist various economies in…

…teaching their students to overcome the rapidly changing world…

…getting them ready to professions required by the modern society…

…training them to use modern technologies…

…giving them problem-solving skills…

7

• To show the countries possible educational achievements

• To support various governments in drafting modern education policy goals based achieved

• To compare the world education progress and country’s achievements

• To share their practical experience in education policy area

PISA goal is…

How the demand for skills has changedEconomy-wide measures of routine and non-routine task input (US)

(Levy and Murnane)

Mean t

ask

inp

ut

as

perc

enti

les

of

th

e 1

960 t

ask

dis

trib

uti

on

The dilemma of schools:The skills that are easiest to teach and test are also the ones that are easiest to digitise, automate and outsource

Andreas SchleicherWhat Students know and can do. 7 December 2010.www.oecd.org

9

Countries that joined PISA in 2009:

• Albania• Shanghai (China)• Kazakhstan• Panama• Peru• Trinidad and Tobago• Dubai (UAE)• Singapore

10

Everyone using survey results

should be confident that

the data reflect the actual

situation in education sector

Why the survey results are reliable?

Consortium controls the quality of survey administration following the developed quality standards

On this reason

11

Why the survey results are reliable?

• Sampling (65 countries, about 450 thousands students)

• Keeping cultural balance in test items

• Identical testing tools

• Thorough control over translation

• Administering Field Trial

• Involving 4 independent experts to analyze open questions and etc.

• Measuring student’s efforts spent to make test

12

What PISA assesses

• PISA assesses functional literacy acquired by boys and girls and their readiness for independent life in society.

13

Functional literacy

• Is the scope of knowledge and skills that enable a human being to act successfully and adopt to modern conditions, and his capability to a lifelong learning

• Modern definition of functional literacy is as follows - knowledge and skills necessary for a human being to become self realized in the modern society.

14

What’s new about PISA 2009?• Electronic Reading Assessment of e-texts

reading literacy (20 countries only)

• More detailed assessment of reading literacy for students with different readiness level (7 levels instead of 5)

• Special focus to education progress (comparing 2000 and 2009 results)

• New personnel student information (attitude towards reading, usage of library resources and notions about learning strategies)

15

PISA 2009

• PISA 2009 paid special attention to reading literacy

• More than two thirds of the testing time was dedicated to it

16

PISA 2009 in KyrgyzstanКарта RDesign©.

http://www.GoKyrgyzstan.net

mailto:[email protected]

http://www.gokyrgyzstan.net/

http://www.gokyrgyzstan.net/

17

PISA 2009 in Kyrgyzstan• 4986 15-years old from 173 secondary education

institutes:

• 116 rural education institutes;• 30 educational institutes from oblast centers and small

cities;• 28 education institutes from Bishkek city.

• All country’s regions were included into the sampling• Testing languages: Kyrgyz, Russian, Uzbek

18

• The 2006 was the first time when the Kyrgyz Republic took part in the survey

• The Kyrgyz Republic was one of the first Central Asian countries to take part in the survey

PISA 2009 in Kyrgyzstan

19

• In Kyrgyzstan the survey was conducted by the Center for Education Assessment and Teaching Methods (CEATM)

• The work was undertaken under the order by Ministry of Education and Science of the Kyrgyz Republic within the WB-financed Rural Education Project

PISA 2009 in Kyrgyzstan

20

What Kyrgyzstan needs this for?

• The goal is to assess literacy level of Kyrgyzstan students and, therefore, effectiveness of the education system of Kyrgyzstan with those of other countries’ education systems in both highly developed country and lowly progressing ones

21

• Search for the most successful education approaches used in the world practice

• Track changes in literacy level of the Kyrgyzstan students

• Assess factors effecting learning outcomes and effectiveness

• Elaborate recommendations and strategies to reform Kyrgyzstan’s education system

What Kyrgyzstan needs this for?

22

PISA assessment tools

Testing• Multiple-choice questions

• Open-constructed response

• Short response

• Closed constructed response

• The text lengthened 2 hours

23

Questioning

• Questions to student about learning process, family, personnel interests and etc.

• Questions to school administrations where PISA survey was administered

PISA assessment tools

24

Reporting results from PISA 2009

«Мыслитель» (фр. Le Penseur) — одна из самых известных скульптур Огюста Родена, созданная в период от 1880 до 1882.Image from: http://www.museum.ru/alb/image.asp?26801

25

Kyrgyzstan in PISA 2009

Just like in 2006, the average score demonstrated by Kyrgyzstan is the lowest among all survey targeted countries in all three subject areas:

• reading literacy;

• mathematic literacy;

• scientific literacy.

26

PISA standards

• The minimum international standard is considered to be level 2 out of 6 levels available in science literacy and mathematics literacy scale, while reading literacy scale anticipates 7 maximum levels of performance.

• It considers that students achieving level 2 starts to successfully demonstrate skills which let them effectively act in any vitally important situations.

27

In Kyrgyzstan the following percentage of students DID NOT achieve the minimum

international standard:

• 83,2% of students in reading literacy;

• 86,6% of students in mathematic literacy;

• 82% of students in scientific literacy.

28

What the difference in student scores means?

• 73 points correspond to 1 literacy level according to the PISA literacy scale (except for levels 5 and 6)

• Based on PISA’s definition, a 39 point disparity means 1 academic year in the PISA literacy scale.

• Average scores of 90% of countries varies between levels 2 and 3 .

29

How valuable is the difference is students’ scoring

• For example, difference between Kyrgyzstan and the highest ranked participant (Shanghai-China) in reading literacy area makes 242 points

• In Kyrgyzstan the scores vary from 155 (the weakest performers’ group) and the strongest performers proficient at 483 which means that the difference reports 328 points

30

A Profile of Student Performance in Reading

31

PISA 2009 defines reading literacy as

" understanding, using, reflecting on and engaging with written texts, in order to achieve one's goals, to develop one's knowledge and potential, and to participate in society."

Continue on the next slide

32

PISA. International comparative survey of reading literacy for 15 years old.Main Survey. May-April 2009.

Results by participant countries in reading literacy.Percent of students per each scale level of Reading Literacy.

33


Results by participant countries in reading literacy.Percent of students per each scale level of Reading Literacy.

34

The PISA reading literacy assessment is built on three major characteristics :

• Texts

• Aspects

• Situations

These three characteristics were used to build reading scale and subscale

35

Basic characteristics of text structure for the reading literacy

Texts

36

AspectsWhat is the reader’s purpose

and approach to the text?

Access andretrieve information

Integrate andinterpret

Reflect andevaluate

ASPECTS


37

SITUATION: What is the intended

use of the text,from the author’s

point of view?

Personal Public Occupational Educational

SITUATION


38

How proficient are students in reading? %Students at Level 5 can handle texts that are unfamiliar in either form or content. They can find information in such texts, demonstrate detailed understanding, and infer which information is relevant to the task. They are also able to critically evaluate such texts and build hypotheses about them, drawing on specialised knowledge and accommodating concepts that may be contrary to expectations.

Tasks at Level 6 require students to make multiple inferences, comparisons and contrasts that are both detailed and precise. They require demonstration of a full and detailed understanding of one or more texts and may involve integrating information from more than one text. Tasks may require the reader to deal with unfamiliar ideas, in the presence of prominent competing information, and to generate abstract categories for interpretations. Reflect and evaluate tasks may require the reader to hypothesise about or critically evaluate a complex text on an unfamiliar topic, taking into account multiple criteria or perspectives, and applying sophisticated understandings from beyond the text.

Some tasks at Level 2 require students to locate one or more pieces of information, which may need to be inferred and may need to meet several conditions. Others require recognising the main idea in a text, understanding relationships, or construing meaning within a limited part of the text when the information is not prominent and the reader must make low level inferences.

Tasks at Level 3 require students to locate, and in some cases recognise the relationship between, several pieces of information that must meet multiple conditions. Interpretative tasks at this level require the reader to integrate several parts of a text in order to identify a main idea, understand a relationship or construe the meaning of a word or phrase. Reflective tasks at this level may require connections, comparisons, and explanations, or they may require the reader to evaluate a feature of the text.

Tasks at Level 4 that involve retrieving information require the reader to locate and organise several pieces of embedded information. Some tasks at this level require interpreting the meaning of nuances of language in a section of text by taking into account the text as a whole. Other interpretative tasks require understanding and applying categories in an unfamiliar context. Reflective tasks at this level require readers to use formal or public knowledge to hypothesise about or critically evaluate a text. Readers must demonstrate an accurate understanding of long or complex texts whose content or form may be unfamiliar.

Tasks at Level 1a require students to locate one or more independent pieces of explicitly stated information; to recognise the main theme or author’s purpose in a text about a familiar topic, or to make a simple connection between information in the text and common, everyday knowledge

Tasks at Level 1b require students to locate a single piece of explicitly stated information in a prominent position in a short, syntactically simple text with a familiar context and text type, such as a narrative or a simple list. There is minimal competing information.

Andreas SchleicherWhat Students know and can do. 7 December 2010.www.oecd.org

40

• 16,8% of Kyrgyzstan students aged 15 can perform reading literacy tasks at the second level of complexity and above.

• Only few students can demonstrate high performance results (0,1%) (levels 5-6)

• The Kyrgyzstan’s mean score in reading literacy is 314 points or level 1b

Kyrgyzstan. Results in reading literacy

Kyrgyzstan:Percent of students who answered right: 40.0

Percent of students who MISSED the item: 18.9

Kyrgyzstan:Percent of students who answered right: 22.1

Percent of students who MISSED the item: 35.0

43

Level 6

Level 5

Level 4

Level 3

Level 2

Level 1а

Level 1b

Levels of Reading Literacy

0.8% 0 %

OECD Kyrgyzstan

OECD (2007), PISA 2006 – Science Competencies for Tomorrow’s World, Table 2.1a

6.8%

20.7%

28.9%

24.0%

13,1%

0.1 %

1.0 %

4.2 %

11.5%

23.8 %

Below level 1b

4.6%29.7

%

1.1%29.8

%

Russia0.3%

26.8 %

31.6 %

19 %

6.8 %

1.6 %

2.8 %

11.1 %

0 %

0.5 %

5.3 %

21.5%

36.9%

26.1 %

9.7 %

0 % 0 %

0.4 %

3.7 %

13.1 %

24.1 %

30.7 %

20.4 %

7.5 %

Azerbaijan Kazakhstan

Минимальный стандарт

44

Kyrgyz student performance in various reading areas

+13

-15

-14

+5

-21

-25 -20 -15 -10 -5 0 5 10 15

Integration and interpretation

Non-continuous texts

Reflect and evaluate

Access and retrieve information

Continuous texts

Kyrgyzstan’s mean score in reading literacy scale: 314

45

Kyrgyz student performance in various reading areas

• It is much harder for Kyrgyzstan students to execute the tasks related to info access and retrieving as well as thinking and evaluation.

• Tasks on Integration and interpretation were executed a bit better

• But nevertheless Kyrgyzstan held the very last position in the ranking list amongst 65 participants with regard to no single reading aspect

46

Gender gap between boys and girls in reading literacy

• Girls demonstrate much higher reading literacy results then boys do in any countries.

• But the gap in some countries is much stronger then in others• In Finland the greatest difference between scores by boys

and girls makes 55 points while in Columbia it equates to only 9 points.

In Kyrgyzstan:

• Score gap between boys and girls makes 53 points

• 19,1% of girls and 41,1% of boys demonstrate low performance results below level 1b

• 21,8% of girls and 11,5% of boys are scored at level 2 and above.

47

Kyrgyzstan. % of boys and girls per each reading literacy level

41.1

29.2

18.2

8.2

2.7 0.6 0 0

19.1

30.1 29

14.7

5.5

1.4 0.2 00

5

10

15

20

25

30

35

40

45

below

Level 1

Level

1b

Level

1а

Level

2

Level

3

Level

4

Level

5

Level

6

boysgirls

48

Science literacy

Continue on the next slide49


Results by participant countries in science literacy.Percent of students per each scale level of Science Literacy.

50


Results by participant countries in science literacy.Percent of students per each scale level of Science Literacy.

51

Science literacy

• About 18% of Kyrgyzstan students aged 15 can perform the science literacy tasks (level 2 and above)

• About 82% of students did not achieve the minimum international standard

• Only a few can demonstrate high performance results (level 5,6)

• Kyrgyzstan’s mean score in science literacy is 330 points (below level 1)

52Percent of Kyrgyzstan students that above the difficulty of item : 12.9

53

OECD and CIS countries students who achieved minimum standard and above in science literacy

17.922.0

55.4

70.0

82.082.178.0

44.6

30.0

18.0

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

OECD Russia Kazakhstan Azerbaijan Kyrgyzstan

Below minimum standard Minimum standard and above

Gender gap between boys and girls in science literacy

• On the whole, gender gap between boys and girls is insufficient in science literacy.

• In Jordan girls are greater than boys for 35 points

• In Columbia boys are greater than girls for 21 points• • In Kyrgyzstan the mean score of girls are greater than boys for

22 points

54

Kyrgyzstan. % of boys and girls per each science literacy level

55

boys

girls

56

Mathematics literacy

Continue on the next slide57


Results by participant countries in mathematics literacy.Percent of students per each scale level of Mathematics Literacy.

58


Results by participant countries in mathematics literacy.Percent of students per each scale level of Mathematics Literacy.

59

• About 13,4% of Kyrgyzstan students aged 15 can perform the mathematic literacy tasks (level 2 and above)

• About 86,6% of students did not achieve the minimum international standard

• Only a few can demonstrate high performance results (level 5,6)

• Kyrgyzstan’s mean score in mathematics literacy is 331 points (below level 1)

Mathematic literacy

60Percent of Kyrgyzstan students that above the difficulty of item : 1.6

61

OECD and CIS countries students who achieved minimum standard and above in mathematics

literacy

21.8

28.6

45.3

59.1

86.6

78.2

71.4

54.7

40.9

13.4

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

OECD Russia Azerbaijan Kazakhstan Kyrgyzstan

Below minimum standard Minimum standard and above

Gender gap between boys and girls in mathematics literacy

• On the whole, boys in any OECD country demonstrate higher math scores then girls. The gap makes 12 points.

• In Columbia and Lichtenstein gender gap between boys and girls reported 32 and 24 points accordingly.

• In Qatar, Lithuania Albania, Trinidad and Tobago girls are greater than boys in math literacy from 5 to 11 points.

• In Kyrgyzstan girls also are greater than boys for 6 points

62

Kyrgyzstan. % of boys and girls per each mathematics literacy level

63

boys

girls

64

Factors effecting student performance

What explains gaps and dispersions in student performance?

• were a girl,

• had better socio-economic background,

• visited private school and/or • visited the school with students with more

encouraging socio-economic background,

• were enrolled into a 1-year pre-school preparation program,

• were possessed of stronger and effective learning strategy,

• had positive attitude to reading,• willingly read various materials for at least

half an hour a day,

• visited a school with Russian language of instruction in Bishkek

+53 scores

You results could have been a bit higher in case you…(Some factors that effect student performance)

+40 scores

+103 scores

+62 scores

+47 scores

Total: 484

+87 scores

+24 scores

+15 scores

+53 scores

… Your literacy score could be higher then the mean score in top performing countries (approximately 484 scores) and higher then the scores of:

• boys,

• students with bad socio-economic background

• students with no prior experience with pre-school preparation class

• students from schools with no effective learning strategy

• those who have no positive attitude to reading

• students who visit a far distance rural state-run school.

66

67

Factors explaining low performance of Kyrgyz students

• Low GDP level• Poor costs allocated per one student• Relationship between school performance and schools’

socio-economic background• Lack of teaching staff• Lack of teaching and learning resources• Few students with pre-primary preparation• Students are not aware of effective learning strategies• Access to education materials at home• and etc.

68

GDP Level• Students representing countries with higher

GDP are used to having better literacy results

• GDP level explains a 6% performance gap between students from different countries

• Kyrgyzstan has the lowest real gross domestic product among all PISA countries – USD 1994 (as per 2007).

69

GDP level

• But it does not mean that is impossible to reach high performance with a low GDP

• For example, in Shanghai-China GDP makes only USD 5430, which is much lower then relevant results in most PISA 2009 countries

• Nevertheless, Shanghai-China is ranked first country in all three surveyed subject areas, this is, reading, mathematics and science literacy

70

Per capita education costs for 6-15 years old

• There is a positive correlation between a per capita expenses and demonstrated literacy results

• As a rule, the greater are per capita student expenses, the bigger are the average literacy results

• Per capita student expenses explains a 9% performance gap between students from different countries

71


• In Kyrgyzstan capitation expenses per one students aged 6-15 make USD 3000

• This is the least amount among all surveyed countries that provided the data (41 countries)

72


• In other countries similar capitation expenses vary from USD 106 000 in USA to USD 12 700 in Turkey.

• In Russian Federation capitation expenses make USD 17 500. No other CIS country provided required data.

73


• But the country should not necessarily have the lowest literacy results if its expenses are not the greatest ones

• For example, Estonia and Poland where capitation expenses reach USD 40 thousands demonstrated great literacy results almost similar to those by USA and Norway who spent USD 100 thousands per each student

74

Socio, economic and cultural student indices

• Social, economic and cultural aspects are the strongest that effect student performance results

Components of social, economic and cultural indices:

• Occupational status of parents, educational level of parents with regard to schooling period;

• Home educational resources relevant for culture

• Items measuring the existence of educational resources at home including a desk and a quiet place to study

• Computer that students can use for schoolwork, educational software, books to help with students’ school work, technical reference books and a dictionary

• Dishwasher, tape-recorder, DVD player, cell phone (s), TV set (s), computer(s), car (s), etc.

75

Socio, economic and cultural student indices

• In all countries, children from families with better socio and economic background usually receive high PISA scores

• In Kyrgyzstan, socio-economic gap explains almost a 14,5% differences in student scores

• In case socio-economic status of students increases for 1 index, one can forecast a 40 point increase in student performance

76

Socio, economic factors at country level

• Countries, where students have higher socio-economic status, usually demonstrate higher reading literacy results

• But in Kyrgyzstan and Shanghai with almost similar socio-economic status of students, Shanghai showed the highest scores, while Kyrgyzstan was the last one

77

Socio, economic factors at country level

78

Family structure• In the absolute majority of families students from two-

parent families usually demonstrate higher literacy results then the ones from incomplete families. In Estonia, Kazakhstan, Peru, Croatia children from incomplete families show higher literacy results

• In Kyrgyzstan the mean literacy score by children from incomplete families is 9 points higher then the one by students from two-parent families

79

80

• Direct relationship between socio-economic status of most students and literacy level

• In case socio-economic status of school increases for 1 index, one can forecast a 62 average point increase in student performance. This means that schools where enrolled students have better socio-economic background show better higher performance results

• As majority of Kyrgyz students have poor socio-economic status, enrollment of selected welfare students increases and strengthens education inequity and worsens the overall country’s literacy level

Impact of school intake index over literacy results

81

Pre-primary school preparation

• Students who have a prior experience of visiting a pre-school preparation class for more then one year usually show higher performance results at the age 15 then those with no prior experience

82


• In OECD countries, the average difference between students who have a prior experience of visiting a pre-school preparation class for more then one year usually show higher performance results (54 points) then those with no prior experience

• In Kyrgyzstan the difference between these two groups makes 72 points

83


0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

более года год или меньше

19.5% 17.8%

67%

11.0%

72%

19.5%

Кыргызстан Россия ОЭСР

More then one year A year and least

Kyrgyzstan Russia OECD

84

Attitude to reading• All countries where students noted their personnel desire in reading

showed much higher literacy results then in those countries where students do not enjoy reading

• IN OECD countries possible changes in scores that might have occurred due to “enjoying reading” index gives 39 additional points or 1 year of studies. Moreover the gap between students who enjoy reading and those who do not reaches almost 1,5 of studies on reading literacy scale.

• In Kyrgyzstan and Azerbaijan possible changes in scores that might have occurred due to “enjoying reading” index gives 24-23 additional points accordingly. In Russian this index reaches 49 points, while in Kazakhstan it has no statistical significance.

• The total score difference between students with personnel desire in reading and those who do not enjoy reading have the following values: 88 points in Russia; 43 points in Kyrgyzstan; 29 points in Azerbaijan and 10 points in Kazakhstan which the smallest difference

Attitude to reading• On the average, much more students in Kyrgyzstan, Kazakhstan

and Azerbaijan noted that they read for pleasure then those in OECD countries

• For example, in Russia and OECD countries 21,4% and 37,4% accordingly noted that they never read for pleasure, while only 11,7% of students in Kyrgyzstan and 7,2% in Kazakhstan gave similar answers

• Also only 4,5% of students in OECD countries and 6,9% of students in Russia answered that they read for more then 2 hours a day, while in Kyrgyzstan 14,5% of students noted that they for more then 2 hours a day and this is the highest value among all PISA countries except for Azerbaijan where almost 15,2% answered in this way

85

Attitude to reading• In 36 countries, students who enjoy reading for more then 30

minutes a day received 30 additional points more then students who do not enjoy reading at all

• Nevertheless in Kyrgyzstan the gap between students who enjoy reading for more then 30 minutes and those who do not enjoy reading at all makes only 15 points

• Thus, in Kyrgyzstan dependence between the time that students spent for reading for more then 30 minutes a day and those who never read for pleasure, is much poor as compared with other PISA countries

86

Attitude to reading• As “reading for pleasure” index is tightly connected with

literacy scores, students’ interest to reading in the school and beyond should be stimulated

• It is important to encourage a child to reading every day for pleasure

• Everyday reading during 30 minutes is enough to give a significant impact over literacy scores

87

Attitude to reading

• Students who read a scope of various materials also have higher reading literacy results

• Reading of books, novels, magazines and journals influences the score

• In Kyrgyzstan students who read magazines and journals regularly received 26 additional points in reading literacy scale

88

89

Knowledge of learning strategies

• Students that apply strategies to understand and remember the information, as well as summarise information strategies received mush higher scores then those that know no strategies and do not apply them in practice

Strategies to understand and remember

Most effective:• After I read the text I discuss it with other people• I underline most important text parts• I generalize the text in my own words

Less effective:• I focus on text parts that are easier to understand• I briefly look through the text two times• I read the text aloud to some other person

90

Strategies to summarise informationMost effective• I thoroughly check if the key text elements are

represented in the text summary• I briefly look through the text underlining key elements

and rewrite them with my own world as a summary

Less effective:• I write a summary of each paragraph and check is all

paragraph is included into the summary as it is critical to insert content of each paragraph

• Prior to drafting a summary I reread the text as many times as possible

Least effective:• I carefully rewrite as many sentences as possible

91

92


• OECD students that apply most effective strategies for remembering and understanding the facts received 70 additional points in literacy scale as compared with students who do not apply this strategies in practice. This score difference correspond to 1 level in reading literacy scale

• In Kyrgyzstan and the Russian Federation this gap makes 87 points and 91 points in Kazakhstan

93


• Kyrgyz students are less aware of learning strategies then the others

• In Kyrgyzstan and Azerbaijan index for remembering and understanding is the least among all PISA countries and makes -0,69. In the Russian Federation and Kazakhstan reported -0,11 and -0,33 accordingly

• In Shanghai which showed the highest reading literacy

results, remembering and understanding index reached 0,14


73

37

57

42

46

27

63

43

58

54

0 20 40 60 80 100

ОЭСР

Кыргызстан

Россия

Азербайджан

Казахстан

вдумчивые

поверхностные

% students who use effective learning strategies

Thoughtful students are those who use effective learning strategies

Superficial students are those who don’t know what learning strategies are most effective94

Thoughtful

Superficial

Kazakhstan

Azerbaijan

Russia

Kyrgyzstan

OECD

95

Kyrgyzstan. Impact of student questioners’ indices over reading literacy expressed in terms of regression numeric parameter

96

School administration characteristics

97

Selection and breakdown of students

• Relatively high rate of grade repetition

• Distribution of students by different education programmes up to age 15

• Transferring of students to another schools based on low performance results, behavior problems and to meet their specific education needs

• Practice of students’ distribution by their abilities by all subjects…

…. not only effect negatively on the average level of literacy but strengthens the impact of the family’s social and economic factor on literacy performance.

98

School autonomy in the area of curriculum development and

assessment• Higher literacy level is commonly observed in those

education systems where schools are granted a significant autonomy in drafting curriculums and executing assessment. Impact of social and economic factors in these systems is eased

• It becomes true just in case national standards are

available

99

School autonomy in the area of curriculum development and

assessment

• All CIS countries that took part in PISA assessment demonstrated weaker level of autonomy in learning process planning then the others OECD countries.

• Most dependent in the area of school learning planning is Kazakhstan with the index -0,98 and the most independent is Kyrgyzstan with the index -0,25

• Index volume in Russia and Azerbaijan makes -0,36 and -0,64 accordingly

Assessment• Even if take into account the per capita income, those

countries where national examinations are as a rule usually have higher literacy results

• Students received 19 additional points to their literacy scores in schools using mandatory national exams then those from schools that do not apply national exam system

• In 18 participant countries where school administration post student performance results in public access usually have higher literacy scores that school with restricted access to performance-related information.

• After we take into account all socio-economic factors, we will have only 5 countries left. But Kyrgyzstan is still among them 100

101

Teaching staff

• Teaching salaries directly effect the overall literacy level in the country

• In case countries have equal teachers’ remunerations, below average results will be achieved by the school that invests into the teachers’ salaries, not fullness of classes

102

Teaching staff

• When comparing answer sheets by school administrators from CIS countries it is obvious that in Kyrgyzstan teachers’ shortage is the most critical among all and reports 0,90

• In Russia corresponding figure makes 0,13, in Kazakhstan – 0,47. In Azerbaijan shortage of teaching staff is close to an average among OECD countries and makes -0,02

103

Material resources• No correlation is observed between availability of extra

educational resources and literacy scores

• But it does not mean that these resources have no impact on the literacy level

• It might be caused by the reason that there is no great variety in equipping all OECD schools

• We are confident that schools lacking adequate teaching staff, shortage of textbooks and other infrastructure problems are more likely to get the lowest literacy results

104

Material resources• As of 2006, material resources shortage index was the

highest among all participating countries and reported -1,72

• In other CIS countries is also a bit material resources shortage index then the OECD countries’ average

• In Kazakhstan index valued -0,76, in Russia and Azerbaijan it made -0,63 and -0,58 accordingly

• In Kyrgyzstan the material resources shortage index is relatively high throughout the country. CIS countries have single schools equipped much better then those from OECD countries (material resources index is almost one level higher then in standard OECD country )

105

Material resources

• It is meant that school performance does not depend on material resources provision, while in Kyrgyzstan impact of material resources is much higher then the one by school administration characteristics

• Before we take into account socio-economic factors, literacy score will mostly depend on educational resources, school administration, assessment system and student enrollment and distribution system last of all.

Values of factors effecting Kyrgyz performance

% deviation percentage that could be explained due to this factors

1

1.2

1.2

7.1

4.1

8.7

4.3

18.4

0 5 10 15 20 25 30

набор и распределение учащихся (перевод из школы в школу, оставление "на второй год" и др.)

школьное управление (автономия в области учебного плана)

система оценивания (национальные экзамены и опубликование показателей успеваемости)

образовательные ресурсы (педагогические кадры и материальные ресурсы в школе)

независимо от социально-экономических факторов

вместе с социально-экономическими факторами

106

Educational resources (teaching staff, school material resources etc.)

Assessment system (national exams and publication of performance results)

Administration (autonomy in drafting curriculum)

Enrollment and distribution of students (grade repetition, transferring to another school)

regardless of socio-economic factor

with regard to socio-economic factor

107

Kyrgyzstan. Comparing PISA 2006 and 2009 results

• An absolutely similar assessment approach in PISA is used once in 9 years only

• That is why the only true comparison of PISA results could be made in the years 2015, when PISA comes back to assess science literacy

108

Kyrgyzstan. Comparing PISA 2006 and 2009 results

• Regardless to the fact that just like in 2006 Kyrgyzstan took the last place among all ranked counties, one could observe slight positive changes as compared with 2006

• Both mean values and the number of students

achieving the minimum literacy standard has increased

109

Kyrgyzstan. Comparing PISA 2006 and 2009 mean scores

285

311

322

313

330 330

260

270

280

290

300

310

320

330

340

грамотность чтения математическая грамотность естественнонаучная грамотность

2006 2009

Reading literacy Mathematics literacy Science literacy

110

Kyrgyzstan. Comparing PISA 2006 and 2009

Процент учащихся, которые достигают минимального стандарта в чтении

16,6%

11,8%

0,0%

2,0%

4,0%

6,0%

8,0%

10,0%

12,0%

14,0%

16,0%

18,0%

2006 2009

Percentage of students who achieved the minimum literacy standard in reading

111

Процент учащихся, которые достигают минимального стандарта в математике

13,1%

10,6%

0,0%

2,0%

4,0%

6,0%

8,0%

10,0%

12,0%

14,0%

2006 2009


Percentage of students who achieved the minimum literacy standard in mathematics

112

Процент учащихся, которые достигают минимального стандарта в естественных науках

18,3%

13,6%

0,0%

2,0%

4,0%

6,0%

8,0%

10,0%

12,0%

14,0%

16,0%

18,0%

20,0%

2006 2009

Percentage of students who achieved the minimum literacy standard in science


Thank you for your attention!

Sources:1. OECD (2010), PISA 2009 Results: What Students Know and Can Do – Student Performance in Reading, Mathematics and Science (Volume I). http://dx.doi.org/10.1787/9789264091450-en2. OECD (2010), PISA 2009 Results: Overcoming Social Background – Equity in Learning Opportunities and Outcomes (Volume II). http://dx.doi.org/10.1787/9789264091504-en3. OECD (2010), PISA 2009 Results: Learning to Learn – Student Engagement, Strategies and Practices (Volume III. )http://dx.doi.org/10.1787/9789264083943-en4. OECD (2010), PISA 2009 Results: What Makes a School Successful? – Resources, Policies and Practices (Volume IV). http://dx.doi.org/10.1787/9789264091559-en5. PISA Data Analysis Manual: SPSS® SECOND EDITION. www.sourceoecd.org/education/97892640562686. OECD Database. http://pisa2009.acer.edu.au/7. PISA 2009. Evaluating systems to improve education. Presentation. Andreas Schleicher. Special advisor to the Secretary-General on Education Policy. Head of the Indicators and Analysis Division, EDU Some photos used in presentation were extracted from Microsoft Office Clipart online and should not be associated with PISA survey

113www.testing.kg

http://dx.doi.org/10.1787/9789264091450-en

http://dx.doi.org/10.1787/9789264091504-en

http://dx.doi.org/10.1787/9789264091559-en

http://www.sourceoecd.org/education/9789264056268

Documents

PISA 2009.International survey of reading literacy for 15 years old Several slides were used from PPT presentation “What Students know and can do” by Andreas