Macao PISA 2006 Study Report Number One - … PISA 2006 Study Report...Macao PISA 2006 Study Report Number One: Assessment of scientific, mathematical and reading literacy performance

Macao PISA 2006 Study ReportNumber One:

Assessment of scientific, mathematical and reading literacy performance of15-year-old students from an international comparison perspective

Kwok-cheung Cheung & Pou-seong Sit

Educational Testing and Assessment Research Center

University of Macau

Macao, People’s Republic of China

December, 2007

- 1 -

Acknowledgement

Fruitful completion of the Macao PISA 2006 Study was contingent on:

(1) Financial sponsorship and steering by Education and Youth Affairs Bureau of MacaoGovernment;

(2) Guidance and resource support by the University of Macau authority;

(3) Academic and technical support by the Educational Testing and AssessmentResearch Center, Faculty of Education, University of Macau, Macao;

(4) Cooperation of secondary schools participating in the PISA 2006 Study;

(5) Active participation of students and their parents in responding to tests andquestionnaires.

- 2 -

Table of Contents

Acknowledgement ............................................................................................................ 1

Table of Contents.............................................................................................................. 2

List of Tables .................................................................................................................... 4

List of Figures................................................................................................................... 6

Executive Summary.......................................................................................................... 7

Chapter 1 Conduct of Enquiry..................................................................................... 10

1.1 Brief introduction ............................................................................................. 10

1.2 Sample design....................................................................................................11

1.3 Scientific literacy assessment framework......................................................... 13

1.4 An example of scientific literacy item unit....................................................... 15

1.4.1 Sample scientific literacy test items – Acid Rain .................................. 16

1.5 Descriptions of proficiency levels of scientific, mathematical and reading

literacy scales.................................................................................................. 19

1.6 Examples of scientific attitudes assessed as outcomes of science learning .. 22

1.6.1 Sample attitude items ............................................................................ 22

Chapter 2 A Profile of Literacy Performance for 15-year-olds in Macao ................... 25

2.1 Macao 15-year-olds’literacy performance ....................................................... 25

2.2 Macao 15-year-olds’scientific literacy performance compared with other

participating countries/economies .................................................................. 29

2.3 Macao 15-year-olds’scientific attitude results ................................................. 31

2.4 Macao 15-year-olds’attitude results compared with OECD countries ............ 32

2.5 Gender differences in outcomes of science learning ........................................ 33

Chapter 3 Quality Science Education Indicators......................................................... 37

3.1 Macao 15-year-olds’quality science education indicators compared with

OECD countries.............................................................................................. 37

3.1.1 General interest in science scale............................................................ 38

3.1.2 General value of science scale............................................................... 39

3.1.3 Personal value of science scale.............................................................. 40

3.1.4 Self-efficacy in science scale................................................................. 41

3.1.5 Self-concept in science scale ................................................................. 42

- 3 -

3.1.6 Enjoyment of science scale ................................................................... 43

3.1.7 Instrumental motivation to learn science scale...................................... 44

3.1.8 Future-oriented motivation to learn science scale ................................. 45

3.1.9 Awareness of environmental issues scale .............................................. 46

3.1.10 Concern for environmental issues scale .............................................. 47

3.1.11 Optimism regarding environmental issues scale ................................. 48

3.1.12 Science-related activities scale ............................................................ 49

3.2 In search of quality science education indicators for Macao............................ 50

3.3 The ten quality science education indicators for Macao .................................. 51

Chapter 4 Literacy-ESCS Relationships for Macao Schools ...................................... 52

4.1 Plots of literacy performance with ESCS in the Macao sample....................... 52

4.2 Within-school correlations of scientific literacy performance with ESCS....... 55

4.3 Between-school scientific literacy performance with ESCS............................ 57

Chapter 5 International Comparison of Literacy Performance ................................... 59

5.1 Performance in the three literacy domains – An international comparison...... 59

5.2 Selection of countries/economies exemplary for Macao’s educational

improvement and curriculum reform.............................................................. 61

Chapter 6 Thematic Reports and Follow-up Studies................................................... 64

References ...................................................................................................................... 66

- 4 -

List of Tables

Table 1 Characteristics of schools in the PISA 2006 Macao sample … … … … . 11

Table 2 Number of 15-year-olds sampled and tested in Macao … … … … … … . 12

Table 3 Grade distribution of 15-year-olds tested … … … … … … … … … … … . 12

Table 4 Contexts for the PISA 2006 science assessment … … … … … … … … ... 13

Table 5 Blueprint used in the design of scientific literacy assessment itemunits (Example item unit: ACID RAIN) … … … … … … … … … … … … 16

Table 6 Description of the six scientific literacy proficiency levels … … … … .. 19

Table 7 Description of the six mathematical literacy proficiency levels … … ... 20

Table 8 Description of the five reading literacy proficiency levels … … … … ... 21

Table 9 Macao 15-year-olds’literacy performance results … … … … … … … … 25

Table 10 Distribution of Macao 15-year-olds’proficiency levels on the literacyscales … … … … … … … … … … … … … … … … … … … … … … … … … .. 27

Table 11 Macao 15-year-olds’scientific attitude results … … … … … … … … … .. 31

Table 12 A comparison of 15-year-olds’ responses to responsibility forsustainable development between Macao and OECD countries … … ... 32

Table 13 A comparison of 15-year-olds’ responses to general interest inscience between Macao and OECD countries … … … … … … … … … .. 38

Table 14 A comparison of 15-year-olds’responses to general value of sciencebetween Macao and OECD countries … … … … … … … … … … … … … 39

Table 15 A comparison of 15-year-olds’ responses to personal interest ofscience between Macao and OECD countries … … … … … … … … … .. 40

Table 16 A comparison of 15-year-olds’ responses to self-efficacy in sciencebetween Macao and OECD countries … … … … … … … … … … … … .. 41

Table 17 A comparison of 15-year-olds’ responses to self-concept in sciencebetween Macao and OECD countries … … … … … … … … … … … … .. 42

Table 18 A comparison of 15-year-olds’ responses to enjoyment of sciencebetween Macao and OECD countries … … … … … … … … … … … … ... 43

Table 19 A comparison of 15-year-olds’ responses to instrumental motivationto learn science between Macao and OECD countries … … … … … … . 44

Table 20 A comparison of 15-year-olds’ responses to future-orientedmotivation to learn science between Macao and OECD countries … ... 45

Table 21 A comparison of 15-year-olds’ responses to awareness ofenvironmental issues between Macao and OECD countries … … … … . 46

Table 22 A comparison of 15-year-olds’ responses to concern forenvironmental issues between Macao and OECD countries … … … … . 47

- 5 -

Table 23 A comparison of 15-year-olds’ responses to optimism regardingenvironmental issues between Macao and OECD countries … … … … . 48

Table 24 A comparison of 15-year-olds’responses to science-related activitiesbetween Macao and OECD countries … … … … … … … … … … … … ... 49

Table 25 Pearson correlation of quality science education indicators withscientific literacy performance scores … … … … … … … … … … … … .. 50

Table 26 Selected quality science education indicators for the Macao sample ... 51

Table 27 Relationships of scientific literacy and ESCS, broken down by school 56

Table 28 Performance of countries/economies in the three literacy domains … . 59

Table 29 Top five high-performing East Asian countries/economies … … … … .. 61

Table 30 Non-East Asian countries/economies having higher or comparableliteracy performance than Macao … … … … … … … … … … … … … … . 62

Table 31 Performance results of other countries/economies in the three literacydomains … … … … … … … … … … … … … … … … … … … … … … … … . 63

- 6 -

List of Figures

Figure 1 Framework for PISA 2006 science assessment … … … … … … … … … 15

Figure 2 Percentage of 15-year-olds at different scientific literacy proficiencylevels across grades in the Macao sample … … … … … … … … … … … 28

Figure 3 Percentage of 15-year-olds at different grade levels across scientificliteracy proficiency levels in the Macao sample … … … … … … … … .. 28

Figure 4 Literacy performance results by gender – Sampled school 1 … … … .. 34



Figure 7 Plots of literacy performance with ESCS in the Macao Sample … … . 53

Figure 8 Plots of scientific literacy subscale performance with ESCS in theMacao Sample … … … … … … … … … … … … … … … … … … … … … . 54

Figure 9 Plots between-school scientific literacy performance with ESCS … ... 57

- 7 -

Executive Summary

The main purpose of this report is to complement the PISA 2006 Study International

Report (entitled PISA 2006 Science competencies for tomorrow’s world, Vol. 1 & 2)

released on 4 December, 2007 in Paris. It is hoped that this report can serve as a good

starting point for any reading or systematic enquiry that makes use of Macao PISA 2006

Study data in the future. The following is an executive summary of this report.

1. Macao, special administrative region of People’s Republic of China, participated in

OECD’s Program for International Student Assessment (PISA) for the first time in

2003, and after 3 years, participated again in 2006. Macao will participate for the

third time in 2009. The followings are results of the PISA 2006 Study.

2. In each PISA study, three main kinds of literacy are assessed, namely, reading,

mathematical and scientific literacy. The target students assessed are all secondary

students who are aged between 15 years three months and 16 years two months at the

time of assessment. Most students are studying in the middle grade levels (i.e. grade

8, 9 and 10), whereas some students are studying in the lower or higher grade levels

(i.e. grade 7 and 11).

3. When comparing the literacy performance across schools, it is important to note that

the literacy assessed referred to the cumulative educational effects of all schools that

the students have previously attended. Therefore, a low-performing school identified

in the PISA Study may not be a poor school. Low-performing students dropped out

from one school may be subsequently enrolled in another school and thereby

lowering the school’s literacy performance level.

4. The focus of the PISA 2006 Study was on science. Amongst the 57 participating

countries/economies, Macao’s scientific literacy performance was statistically

significantly above the OECD average, and Macao ranked between 15 and 20 on the

combined science scale. In decreasing order of the mean of scientific literacy score,

countries/economies statistically significantly higher than Macao were: Finland,

Hong Kong, Canada, Chinese Taipei, Estonia, Japan, New Zealand, Australia,

Netherlands, Liechtenstein, Korea and Slovenia.

- 8 -

5.There are six scientific proficiency levels in the combined science scale. On one hand,

students who cannot reach the lowest level 1 are regarded as low-performing, and

they run the risk of being unable to function productively in the life-long learning

society in the 21st Century. Only 1.4% of the 15-year-olds performed below level 1

of the scientific literacy scale. On the other hand, students who can reach the highest

level 6 are regarded as high-performing, and they are cherished as valuable talents so

much needed for scientific advancement in nowadays knowledge society. Only 0.3%

of the 15-year-olds performed at this highest level on the combined science scale.

6. Amongst the three content areas of science, Macao’s 15-year-olds performed best in

“living systems”, moderately well in “physical systems” and less well in “earth and

space systems”. Amongst the three key scientific competencies, Macao’s

15-year-olds performed best in “explaining phenomena scientifically”, moderately

well in “using scientific evidence”, and less well in “identifying scientific issues”.

Generally speaking, Macao’s 15-year-olds performed better in “knowledge of

science” than “knowledge about science”. In Macao, males performed better than

females in “explaining phenomena scientifically”, whereas females performed better

than males in “identifying scientific issues”. Males and females were comparable in

literacy performance as far as “using scientific evidences” was concerned.

7. Ten quality science education indicators verified to affect scientific literacy

performance have been identified, namely: (1) General interest in science, (2)

General value of science, (3) Self-efficacy in science, (4) Self-concept in science, (5)

Enjoyment of science, (6) Instrumental motivation to learn science, (7) Awareness of

environmental issues, (8) Concern for environmental issues, (9) Optimism regarding

environmental issues, (10) Science-related activities. Guided by these indicators,

intervention studies can be designed and put to practice to help low-performing

students enhance scientific literacy performance.

8. Another minor focus of the PISA 2006 Study was on mathematics. Amongst the 57

participating countries/economies, Macao’s mathematical literacy performance was

statistically significantly above the OECD average, and Macao ranked between 7 and

11 on the mathematics scale. In decreasing order of the mean of mathematical

literacy score, countries/economies statistically significantly higher than Macao are:

Chinese Taipei, Finland, Hong Kong and Korea.

- 9 -

9. Another minor focus of the PISA 2006 Study was on reading. Amongst the 57

participating countries/economies, Macao’s reading literacy performance was not

statistically significantly different from the OECD average, and Macao ranked

between 18 and 22 on the reading scale. In decreasing order of the mean of reading

literacy score, countries/economies statistically significantly higher than Macao are:

Korea, Finland, Hong Kong, Canada, New Zealand, Ireland, Australia, Liechtenstein,

Poland, Sweden, Netherlands, Belgium, Estonia and Switzerland.

10. Amongst the 57 participating countries/economies, percentage of variance in

performance in science explained by the PISA index of economic, social and

cultural status of the home (ESCS) is the lowest. The overall effect and

between-school effect of ESCS on scientific literacy are also amongst the lowest of

all participating countries/economies. Hence, Macao’s basic educational system

succeeds in providing equitable schooling opportunities for the student body it

served.

- 10 -

Chapter 1Conduct of Enquiry

Abstract: This chapter recapitulates the conduct of enquiry of the PISA 2006 Scientific

Literacy Study undertaken in Macao from 21 April 2006 to 31 May 2006. It comprises

six sections: (1) Brief introduction; (2) Sample design; (3) Scientific literacy assessment

framework; (4) An example of scientific literacy item unit; (5) Description of

proficiency levels of scientific, mathematical and reading literacy scales; (6) Examples

of scientific attitudes assessed as outcomes of science learning.

1.1 Brief introduction

The PISA 2006 Study assessed 15-year-old students’literacy in three key subject areas:

(1) science, (2) mathematics, and (3) reading. In this third round of international

assessment, scientific literacy took its turn to be the main focus of international

assessment, whereas mathematical and reading literacy were assessed to a minor extent.

This assessment design allows researchers to chart changes since PISA 2000 Reading

Literacy Study and PISA 2003 Mathematical Literacy Study, in both studies scientific

literacy were likewise assessed to a minor extent. Literacy refers to the capacity of

students to apply knowledge and skills in key subject areas and to reason and

communicate effectively as they pose and solve problems in a variety of situations.

PISA 2006 Scientific Literacy Study sought to chart a profile of knowledge and skills,

i.e. a detailed profile of literacy for science, and an update for mathematics and reading.

For science, the emphasis was on the mastery of processes (i.e. identifying scientific

issues, explaining phenomena scientifically and using scientific evidence), the

understanding of concepts (i.e. knowledge of science such as knowledge of physical

systems, living systems, earth and space systems and technology systems; as well as

knowledge about science such as knowledge about science as scientific enquiry and

scientific explanations), and the ability to function in various situations (i.e. in personal,

social and global settings). In addition, how students respond to scientific issues (i.e.

interest in science, support for scientific enquiry and responsibility towards resources

and environments) were also examined.

- 11 -

Approximately 400,000 students were randomly sampled to participate in the PISA

2006 Scientific Literacy Study. The achieved sample represented about 32 millions

15-year-olds in the schools of the 57 participating countries/economies, of which 30

were OECD member countries and 27 were partner countries/economies. This breadth

of geographical coverage represented close to 90% of the world economy in 2006.

1.2 Sample design

Table 1 presents characteristics of the schools and the 15-year-olds sampled and tested

in the PISA 2006 Study, broken down by school type, study program, and language of

instruction.

Table 1Characteristics of schools in the PISA 2006 Macao sample

Stratifying Variable

Numberof

schoolsin

Macao

Numberof

studentsin

Macao

Numberof

schoolssampled

Numberof

schoolstested

Numberof

studentssampled

Numberof

studentstested

School Type

1. Government 4 266 2 2 247 2302. Private-In-Net 32 5,386 32 32 3,864 3,7343. Private 9 996 9 9 818 796

Study Program

1. Grammar-International 41 6,346 40 40 4,656 4,510

2.Technical-Prevocational 4 302 3 3 273 250

Language of Instruction

1. Chinese 33 5,494 31 31 3,908 3,7632. English 7 398 7 7 396 3913. Portuguese 1 42 1 1 46 454. Chinese & English 3 601 3 3 432 418

5. Chinese & Portuguese 1 113 1 1 147 143

Total 45 6,648 43 43 4,929 4,760

Note 1: All sampled schools offered basic education courses to 15-year-olds. Two government schools were excludedfrom the designed school sample, one offered senior secondary vocational education only to a few students. The otherwas a dancing school offering performing arts education.Note 2: Sampled students were all 15-year-olds born in 1990.

Table 2 presents the number of students (males/females) sampled and tested in the PISA

2006 Scientific Literacy Study. The response rates are very satisfactory, showing that

- 12 -

the achieved sample is highly representative of the Macao 15-year-old student

population.

Table 2

Number of 15-year-olds sampled and tested in Macao

Macao Sample

Number of 15-year-old students sampled4,929

(2,426 males and 2,503 females)

Number of 15-year-old students tested4,760

(2,320 males and 2,440 females)

Response rate (%) 96.6

Table 3 presents the grade distribution of Macao’s 15-year-old students tested in the

PISA 2006 Scientific Literacy Study. Despite 33.4% and 36.5% of the Macao sample

were studying at grade 9 and 10 respectively, there were 8.2% and 21.2% of sampled

students studying at grade 7 and 8 respectively. It is noteworthy that these students were

likely repeaters, new immigrants, or students receiving inclusive education. A sampled

school’s literacy performance is expected to be adversely affected when the proportions

of students studying at the lower grades (i.e. grade 7 and 8) are significantly higher than

the corresponding figures in the Macao sample (see Figure 2 and 3 in Section 2.1 for the

distribution literacy proficiency levels across grades).

Table 3

Grade distribution of 15-year-olds tested

Macao SampleGrade

N %

7 391 8.2

8 1,008 21.2

9 1,591 33.4

10 1,738 36.5

11 32 0.7

Total 4,760 100.0

- 13 -

1.3 Scientific literacy assessment framework

Unlike most school tests and examinations, PISA assessment is not curriculum-based.

PISA assessment items have to relate to life situations (i.e. health, natural resources,

environment, hazards, frontiers of science and technology) within contexts (i.e. personal,

social and global) that 15-year-olds are likely to encounter in daily life. Table 4 shows

the contexts for the PISA 2006 science assessment (OECD, 2007, p.36).

Table 4Contexts for the PISA 2006 science assessment

Contexts

Life

SituationsPersonal

(Self, family and peergroups)

Social

(The community)

Global

(Life across the world)

Health Maintenance of health,accidents, nutrition

Control of disease,social transmission,food choices,community health

Epidemics, spread ofinfectious diseases

NaturalResources

Personal consumptionof materials and energy

Maintenance of humanpopulations, quality oflife, security,production anddistribution of food,energy supply

Renewable andnon-renewable, naturalsystems, populationgrowth, sustainable useof species

Environment Environmentallyfriendly behavior, useand disposal ofmaterials

Population distribution,disposal of waste,environmental impact,local weather

Biodiversity, ecologicalsustainability, controlof pollution, productionand loss of soil

Hazards Natural andhuman-induced,decisions abouthousing

Rapid changes(earthquakes, severeweather), slow andprogressive changes(coastal erosion,sedimentation), riskassessment

Climate change, impactof modern warfare

Frontiers ofscience andtechnology

Interests in science’sexplanations of naturalphenomena,science-based hobbies,sport and leisure, musicand personaltechnology

New materials, devicesand processes, geneticmodification, weapontechnology, transport

Extinction of species,exploration of space,origin and structure ofthe universe

- 14 -

In PISA 2006 Study, scientific literacy refers to an individual’s:

Scientific knowledge and use of that knowledge to identify questions, acquire new

knowledge, explain scientific phenomena and draw evidence-based conclusions

about science-related issues.

Understanding of characteristic features of science as a form of human knowledge

and enquiry.

Awareness of how science and technology shape our material, intellectual, and

cultural environments.

Willingness to engage in science-related issues and with the ideas of science, as a

reflective citizen. (OECD, 2007, p.34-35)

In the scientific literacy assessment framework (OECD, 2007, p.35) shown in Figure 1,

context refers to engagement with science in a variety of life situations not limited to

life in school (i.e. personal, social and global situations within application areas such as

health, natural resources, environment, hazards, and frontiers of science and technology).

These applications require students to demonstrate three main competencies (i.e. identify

scientific issues, explain phenomena, and use scientific evidence), and how they do so is

influenced by their knowledge (i.e. knowledge of science and knowledge about science)

and attitudes (i.e. interest in science, support for scientific enquiry, and responsibility

toward resources and environments). Knowledge of science is classified in four

categories: (1) physical systems, (2) living systems, (3) earth and space systems, and (4)

technology systems, whereas knowledge about science in two categories: (1) scientific

enquiry, and (2) scientific explanations. Based on the test construction blueprint shown

in Table 5, there is a balance of items mapped to the various components of the

scientific literacy assessment framework.

- 15 -

Figure 1

Framework for PISA 2006 science assessment

1.4 An example of scientific literacy item unit

Table 5 serves as a blueprint when scientific literacy item units are designed (OECD,

2007, p.46). Using ACID RAIN as an example, this unit composes of three items, Q2,

Q3 and Q5 (OECD, 2007, p.104-107). All load on the three scientific competencies

respectively (see item unit below). In particular, Q2and Q3 is classified as knowledge of

science, whereas Q5 is on scientific enquiry and therefore classified as knowledge about

science. Embedded in the test item unit are attitude items probing into students’interests

as well as their attitudinal support for scientific enquiry in the science topics under

assessment.

Competencies

Knowledge

Attitudes

Require people to

Context

Identify scientific issues

Explain phenomenascientifically

Use scientific evidence

b) How they respond to science issues (interest,support for scientific enquiry, responsibility).

How they do so is influenced by

a) What they know:

about the natural world andtechnology (knowledge of science);

about science itself (knowledge aboutscience).

Life situations thatinvolve science andtechnology

- 16 -

Table 5Blueprint used in the design of scientific literacy assessment item units

(Example item unit: ACID RAIN)

Competencies

Identifyingscientific

issues

Explainingphenomena

scientifically

Usingscientificevidence

PhysicalSystems

Q2 Q3

LivingSystems

Earth andSpaceSystems

Knowledgeof science

TechnologySystems

ScientificEnquiry

Q5

Knowledge

Knowledgeabout science

ScientificExplanation

Interests in Science See section1.6.1.1 for an example (Q10N)embedded in ACID RAIN

Attitudes

Support for ScientificEnquiry

See section 1.6.1.2 for an example (Q10S)embedded in ACRD RAIN

1.4.1 Sample scientific literacy test items – Acid Rain

ACID RAIN

Below is a photo of statues called Caryatids that were built on the Acropolis in Athens

more than 2500 years ago. The statues are made of a type of rock called marble.

Marble is composed of calcium carbonate.

In 1980, the original statues were transferred inside the museum of the Acropolis and

were replaced by replicas. The original statues were being eaten away by acid rain.

- 17 -

Question 2: ACID RAIN S485Q02 – 0129

Normal rain is slightly acidic because it has absorbed some carbon dioxide from the air.

Acid rain is more acidic than normal rain because it has absorbed gases like sulphur

oxides and nitrogen oxides as well.

Where do these sulphur oxides and nitrogen oxides in the air come from?

… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …

… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …

The effect of acid rain on marble can be modeled by placing chips of marble in vinegar

overnight. Vinegar and acid rain have about the same acidity level. When a marble chip

is placed in vinegar, bubbles of gas form. The mass of the dry marble chip can be found

before and after the experiment.

- 18 -

Question 3: ACID RAIN S485Q03

A marble chip has a mass of 2.0 grams before being immersed in vinegar overnight. The

chip is removed and dried the next day. What will the mass of the dried marble chip be?

A Less than 2.0 grams

B Exactly 2.0 grams

C Between 2.0 and 2.4 grams

D More than 2.4 grams

Question 5: ACID RAIN S485Q05 – 0129

Students who did this experiment also placed marble chips in pure (distilled) water

overnight.

Explain why the students include this step in their experiment.

… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …

… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …

- 19 -

1.5 Description of proficiency levels of scientific, mathematical andreading literacy scales

In the PISA 2006 Study, students are required to select or construct their answers when

responding to the assessment items. Items were typically organized in units based on a

written passage or graphic, commonly of the kind students may encounter in real life

(see illustrative example ACID RAIN). Each student was randomly assigned one of

thirteen test booklets containing assessment items that take approximately 120 minutes

to complete. Table 6-8 presents description of the proficiency levels of the literacy

assessment scales, i.e. scientific, mathematical, and reading literacy. In all PISA literacy

scales that serve as a benchmark for future studies, the scales are calibrated against the

thirty OECD countries, with mean score set at 500 and standard deviation at 100 for

easy referencing and interpretation.

Table 6Description of the six scientific literacy proficiency levels (OECD, 2007, p.43)

Level What students can typically do at each level?

6 At Level 6, students can consistently identify, explain and apply scientific knowledge and knowledgeabout science in a variety of complex life situations. They can link different information sources andexplanations and use evidence from those sources to justify decisions. They clearly and consistentlydemonstrate advanced scientific thinking and reasoning, and they are willing to use their scientificunderstanding in support of solutions to unfamiliar scientific and technological situations. Students atthis level can use scientific knowledge and develop arguments in support of recommendations anddecisions that centre on personal, social, or global situations.

5 At Level 5, students can identify the scientific components of many complex life situations, apply bothscientific concepts and knowledge about science to these situations, and can compare, select andevaluate appropriate scientific evidence for responding to life situations. Students at this level can usewell-developed inquiry abilities, link knowledge appropriately and bring critical insights to situations.They can construct explanations based on evidence and arguments based on their critical analysis.

4 At Level 4, students can work effectively with situations and issues that may involve explicitphenomena requiring them to make inferences about the role of science or technology. They can selectand integrate explanations from different disciplines of science or technology and link thoseexplanations directly to aspects of life situations. Students at this level can reflect on their actions andthey can communicate decisions using scientific knowledge and evidence.

3 At Level 3, students can identify clearly described scientific issues in a range of contexts. They canselect facts and knowledge to explain phenomena and apply simple models or inquiry strategies.Students at this level can interpret and use scientific concepts from different disciplines and can applythem directly. They can develop short statements using facts and make decisions based on scientificknowledge.

2 At Level 2, students have adequate scientific knowledge to provide possible explanations in familiarcontexts or draw conclusions based on simple investigations. They are capable of direct reasoning andmaking literal interpretations of the results of scientific inquiry or technological problem solving.

1 At Level 1, students have such a limited scientific knowledge that it can only be applied to a fewfamiliar situations. They can present scientific explanations that are obvious and follow explicitly fromgiven evidence.

Note: There is an additional “below 1” level for those students who cannot attain at the minimum level.

- 20 -

Table 7Description of the six mathematical literacy proficiency levels (OECD, 2007, p.312)


6 At Level 6 students can conceptualize, generalize, and utilize information based on theirinvestigations and modeling of complex problem situations. They can link differentinformation sources and representations and flexibly translate among them. Students at thislevel are capable of advanced mathematical thinking and reasoning. These students can applythis insight and understandings along with a mastery of symbolic and formal mathematicaloperations and relationships to develop new approaches and strategies for attacking novelsituations. Student at this level can formulate and precisely communicate their actions andreflections regarding their findings, interpretations, arguments, and the appropriateness ofthese to the original situations.

5 At Level 5 students can develop and work with models for complex situations, identifyingconstraints and specifying assumptions. They can select, compare, and evaluate appropriateproblem solving strategies for dealing with complex problems related to these models.Students at this level can work strategically using broad, well-developed thinking andreasoning skills, appropriate linked representations, symbolic and formal characterizations,and insight pertaining to these situations. They can reflect on their actions and formulate andcommunicate their interpretations and reasoning.

4 At Level 4 students can work effectively with explicit models for complex concretesituations that may involve constraints or call for making assumptions. They can select andintegrate different representations, including symbolic, linking them directly to aspects ofreal-world situations. Students at this level can utilize well-developed skills and reasonflexibly, with some insight, in these contexts. They can construct and communicateexplanations and arguments based on their interpretations, arguments, and actions.

3 At Level 3 students can execute clearly described procedures, including those that requiresequential decisions. They can select and apply simple problem solving strategies. Studentsat this level can interpret and use representations based on different information sources andreason directly from them. They can develop short communications reporting theirinterpretations, results and reasoning.

2 At Level 2 students can interpret and recognize situations in contexts that require no morethan direct inference. They can extract relevant information from a single source and makeuse of a single representational mode. Students at this level can employ basic algorithms,formulae, procedures, or conventions. They are capable of direct reasoning and makingliteral interpretations of the results.

1 At Level 1 students can answer questions involving familiar contexts where all relevantinformation is present and the questions are clearly defined. They are able to identifyinformation and to carry out routine procedures according to direct instructions in explicitsituations. They can perform actions that are obvious and follow immediately from the givenstimuli.


- 21 -

Table 8Description of the five reading literacy proficiency levels (OECD, 2007, p.292-293)


5 Locate and possibly sequence or combine multiple pieces of deeply embedded information, some ofwhich may be outside the main body of the text. Infer which information in the text is relevant to thetask. Deal with highly plausible and/or extensive competing information. Either construe the meaningof nuanced language or demonstrate a full and detailed understanding of a text. Critically evaluate orhypothesize, drawing on specialized knowledge. Deal with concepts that are contrary to expectationsand draw on a deep understanding of long or complex texts. In continuous texts students can analyzetexts whose discourse structure is not obvious or clearly marked, in order to discern the relationship ofspecific parts of the text to its implicit theme or intention. In non-continuous texts, students can identifypatterns among many pieces of information presented in a display which may be long and detailed,sometimes by referring to information external to the display. The reader may need to realizeindependently that a full understanding of the section of text requires reference to a separate part of thesame document, such as a footnote.

4 Locate and possibly sequence or combine multiple pieces of embedded information, each of which mayneed to meet multiple criteria, in a text with familiar context or form. Infer which information in thetext is relevant to the task. Use a high level of text-based inference to understand and apply categoriesin an unfamiliar context, and to construe the meaning of a section of text by taking into account the textas a whole. Deal with ambiguities, ideas that are contrary to expectation and ideas that are negativelyworded. Use formal or public knowledge to hypothesize about or critically evaluate a text. Showaccurate understanding of long or complex texts. In continuous texts, students can follow linguistic orthematic links over several paragraphs, often in the absence of clear discourse markers, in order tolocate, interpret or evaluate embedded information or to infer psychological or metaphysical meaning.In non-continuous texts students can scan a long, detailed text in order to find relevant information,often with little or no assistance from organizers such as labels or special formatting, to locate severalpieces of information to be compared or combined.

3 Locate, and in some cases recognize the relationship between pieces of information, each of which mayneed to meet multiple criteria. Deal with prominent competing information. Integrate several parts of atext in order to identify a main idea, understand a relationship or construe the meaning of a word orphrase. Compare, contrast or categorize taking many criteria into account. Deal with competinginformation. Make connections or comparisons, give explanations, or evaluate a feature of text.Demonstrate a detailed understanding of the text in relation to familiar, everyday knowledge, or drawon less common knowledge. In continuous texts students use conventions of text organization, wherepresent, and follow implicit or explicit logical links such as cause and effect relationships acrosssentences or paragraphs in order to locate, interpret or evaluate information. In non-continuous textsstudents can consider one display in the light of a second, separate document or display, possibly in adifferent format, or combine several pieces of spatial, verbal and numeric information in a graph ormap to draw conclusions about the information represented.

2 Locate one or more pieces of information, each of which may be required to meet multiple criteria.Deal with competing information. Identify the main idea in a text, understand relationships, form orapply simple categories, or construe meaning within a limited part of the text when the information isnot prominent and low-level inferences are required. Make a comparison or connections between thetext and outside knowledge, or explain a feature of the text by drawing on personal experience andattitudes. In continuous texts students can follow logical and linguistic connections within a paragraphin order to locate or interpret information; or synthesize information across texts or parts of a text inorder to infer the author’s purpose. In non-continuous texts students can demonstrate a grasp of theunderlying structure of a visual display such as a simple tree diagram or table, or combine two pieces ofinformation from a graph or table.

1 Locate one or more independent pieces of explicitly stated information, typically meeting a singlecriterion, with little or no competing information in the text. Recognize the main theme or author'spurpose in a text about a familiar topic, when the required information in the text is prominent. Make asimple connection between information in the text and common, everyday knowledge. In continuoustexts students can use redundancy, paragraph headings or common print conventions to form animpression of the main idea of the text, or to locate information stated explicitly within a short sectionof text. In non-continuous texts students can focus on discrete pieces of information, usually within asingle display such as a simple map, a line graph or a bar graph that presents only a small amount ofinformation in a straightforward way and in which most of the verbal text is limited to a small numberof words or phrases.


- 22 -

1.6 Examples of scientific attitudes assessed as outcomes of science

learning

One prominent feature of the scientific literacy assessment framework is that not only

cognitive outcomes of science learning but also affective outcomes are also included.

The three attitudes assessed are: (1) Interest in science; (2) Support for scientific

enquiry; and (3) Responsibility towards resources and environments. The first two

attitudes are embedded in the test item units and therefore they are content-specific and

composed in the same way as the scientific literacy measures. Again, using ACID RAIN

as an example, the attitude scale statements are shown below for illustration. The third

attitude scale is a standard Likert scale commonly used in educational research. Seven

attitude statements have been validated with adequate psychometric properties.

1.6.1 Sample attitude items

1.6.1.1 Interest in science scale

Each student responded to one of the thirteen test booklets. In each booklet, some

science test units contain a set of attitude statements asking students to rate their

interests in the scientific topics suggested (see OECD, 2007, p.107, for an example from

the released item ACID RAIN shown below).

ACID RAIN (Q10N) S485Q10N

How much interest do you have in the following statements?

Tick only one box in each row.

High

Interest

Medium

Interest

Low

Interest

No

Interest

a) Knowing which human activities contributemost to acid rain 1 2 3 4

b) Learning about technologies that minimise theemission of gases that cause acid rain 1 2 3 4

c) Understanding the methods used to repairbuildings damaged by acid rain 1 2 3 4

- 23 -

1.6.1.2 Support for scientific enquiry scale

Each student responded to one of the thirteen test booklets. In each booklet, some

science test units contain a set of attitude statements asking students to rate their support

for scientific enquiry in the topics suggested (see OECD, 2007, p.107, for an example

from the released item ACID RAIN shown below).

ACID RAIN (Q10S) S485Q10S

How much do you agree with the following statements?

Tick only one box in each row.

StronglyAgree Agree Disagree

StronglyDisagree

a) Preservation of ancient ruins should be basedon scientific evidence concerning the causes ofdamage.

1 2 3 4

b) Statements about the causes of acid rain shouldbe based on scientific research. 1 2 3 4

1.6.1.3 Responsibility towards resources and environments

Contrary to the previous two attitude scales, all students responded to a questionnaire

and are asked to rate the following statements related to sustainable environmental

development, which is one key component of responsibility towards resources and

environment:

1. Industries should be required to prove that they safely dispose of dangerous

waste materials.

2. I am in favor of having laws that protect the habitats of engendered species.

3. It is important to carry out regular checks on the emissions from cars as a

condition of their use.

4. To reduce waste, the use of plastic packaging should be kept to a minimum.

5. Electricity should be produced from renewable sources as much as possible,

even if this increases the cost.

6. It disturbs me when energy is wasted through the unnecessary use of electrical

appliances.

7. I am in favor of having laws that regulate factory emissions even if this would

increase the price of products.

- 24 -

There are three other components of responsibility towards resources and environments,

namely, students’awareness of environmental issues, optimism regarding environmental

issues, and concern for environmental issues. These three will be examined and reported

in chapter 3.

- 25 -

Chapter 2A Profile of Literacy Performance for 15-year-olds in Macao

Abstract: This chapter recapitulates the key results, particularly those pertaining to

Macao, reported in the PISA 2006 Study International Report (OECD, 2007). It detailed

the profiles of student performance broken down by gender in science, mathematics and

reading. Apart from cognitive outcomes, affective outcomes of science education are

also documented to allow readers to evaluate strengths and weaknesses of educational

provisions in the three key domains of study (i.e. science, mathematics and reading) in

Macao. This chapter comprises five sections: (1) Macao 15-year-olds’ literacy

performance; (2) Macao 15-year-olds’ scientific literacy performance compared with

other participating countries/economies; (3) Macao 15-year-olds’ scientific attitude

results; (4) Macao 15-year-olds’attitude results compared with other OECD countries;

(5) Gender differences in outcomes of science learning.

2.1 Macao 15-year-olds’literacy performance

Table 9 presents Macao 15-year-olds’performance results in the three assessment area,

i.e. scientific, mathematical, and reading literacy, broken down by gender.

Table 9Macao 15-year-olds’literacy performance results

Scientific LiteracyDescriptive

Statistics CombinedScience

IdentifyingScientific

Issues

ExplainingPhenomena

Scientifically

UsingScientificEvidences

MathematicalLiteracy

ReadingLiteracy

Total = 4,760

Mean 510.8 490.0 520.0 511.5 525.0 492.3

StandardDeviation

78.2 78.8 82.7 84.4 84.3 76.6

Males = 2,320

Mean 512.7 482.5 526.9 511.7 530.3 479.4

StandardDeviation

82.2 82.1 86.4 88.7 87.7 80.3

Females = 2,440

Mean 508.9 497.7 512.9 511.4 519.6 505.5

StandardDeviation

73.9 74.6 78.0 79.7 80.4 70.2

Note1: Sample scientific literacy items are shown in chapter 1.Note2: Scientific literacy as measured by the combined scale is composed of three subscales, namely: (1)Identifying scientific issues; (2) Explaining phenomena scientifically; and (3) Using scientific evidences.

- 26 -

As seen in Table 9, Macao’s 15-year-olds performed very well in mathematics, quite

well in science, and only fair in reading. In mathematics, males outperform females,

whereas it is the other way round in reading. In science, males outperform females in

explaining phenomena scientifically, whereas females outperform males in identifying

scientific issues. In using scientific evidences, males and females perform equally well.

All standard deviations are low showing that students’ performances on the three

literacy scales and subscales are quite homogenous as compared with the average of the

thirty OECD countries, the standard deviation of which is set at 100.

Each student was assigned to the highest proficiency level for which he or she would be

expected to answer correctly the majority of the assessment items. Students classified as

below level 1 were unable to demonstrate competency in situations required by the

easiest PISA tasks, and therefore they were regarded as at a disadvantage for full

participation in society and economy. Table 10 presents frequency distributions of

Macao 15-year-olds’proficiency levels on scientific, mathematical, and reading literacy

scales, broken down by gender.

- 27 -

Table 10Distribution of Macao 15-year-olds’proficiency levels on the literacy scales

% of Students

Scientific LiteracyProficiency

Level

CombinedScience

IdentifyingScientific

Issues

ExplainingPhenomena

Scientifically

UsingScientificEvidence

MathematicalLiteracy

ReadingLiteracy

Total Sample = 4,760

6 0.3 0.1 0.8 0.5 3.8 -

5 5.1 2.7 7.5 6.4 13.6 3.0

4 22.9 17.3 25.1 23.2 24.4 18.5

3 35.7 34.0 33.9 33.6 27.3 36.6

2 25.9 30.3 23.3 24.7 20.0 28.9

1 8.8 12.9 7.9 9.3 8.3 10.1

Below 1 1.4 2.7 1.5 2.3 2.6 2.9

Males = 2,320

6 0.3 0.1 1.1 0.6 4.8 -

5 6.3 2.4 9.3 7.5 15.8 2.4

4 23.6 16.5 27.5 23.3 24.3 15.3

3 34.4 31.3 31.9 31.8 25.8 34.1

2 24.1 30.7 21.0 24.1 18.5 30.4

1 9.4 15.0 7.5 9.7 8.0 13.2

Below 1 1.8 3.9 1.7 3.0 2.9 4.5

Females = 2,440

6 0.2 0.1 0.4 0.4 2.7 -

5 3.8 3.0 5.7 5.4 11.5 3.7

4 22.2 18.1 22.6 23.0 24.5 21.8

3 37.0 36.8 36.0 35.4 28.8 39.1

2 27.7 29.8 25.7 25.2 21.5 27.4

1 8.0 10.8 8.3 8.9 8.7 6.8

Below 1 1.0 1.5 1.3 1.6 2.4 1.3

Note1: Description of the scientific, mathematical and reading proficiency levels are shown in chapter 1.Note2: Contrary to six proficiency levels calibrated in the mathematical literacy and scientific literacyscales, there are only five proficiency levels calibrated in the reading literacy scale.

As seen in Table 10, students’proficiency levels are mainly concentrated at levels 2, 3

and 4. Percentages of students with proficiency level below 1 for the three literacy

- 28 -

scales remain low (less than 5% of the Macao sample), showing that the number of

low-performing students who cannot function productively in society is small.

Unfortunately, the number of high-performing students with proficiency level above 5 is

not high and this is so for all the three literacy scales. Figure 2 and 3 further show the

relationships between scientific proficiency levels attained and grade levels enrolled by

the 15-year-olds in the Macao sample.

Figure 2

Percentage of 15-year-olds at different scientific literacy proficiency levels acrossgrades in the Macao sample

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

Grade 7 Grade 8 Grade 9 Grade 10 Grade 11

Below level 1

Level 1

Level 2

Level 3

Level 4

Level 5

Level 6

Figure 3

Percentage of 15-year-olds at different grade levels across scientific literacy proficiencylevels in the Macao sample

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0

Below level 1 Level 1 Level 2 Level 3 Level 4 Level 5 Level 6

Grade 7

Grade 8

Grade 9

Grade 10

Grade 11

- 29 -

2.2 Macao 15-year-olds’scientific literacy performance compared with

other participating countries/economies

Compared with other participating countries/economies, the following five

achievements deserve special announcement to the Macao educational community:

1. Macao ranks between 24 and 29 amongst the 57 participating

countries/economies on the identifying scientific issues scale. Country means

range from 321 to 555 on this scale, and Macao’s performance (mean = 490;

SE = 1.2) is statistically significantly below the OECD average. When ranked

in descending order of percentage of 15-year-olds in Levels 2, 3 4, 5 and 6,

the ranking rises to 17. Except five amongst the 57 countries/economies,

females outperform males statistically significantly on this scale. (OECD,

2007, p.66, p.69 & p.79)


countries/economies on the explaining phenomena scientifically scale.

Country means range from 334 to 566 on this scale, and Macao’s performance

(mean = 520; SE = 1.2) is statistically significantly above the OECD average.

When ranked in descending order of percentage of 15-year-olds in Levels 2, 3,

4, 5 and 6, the ranking rises to 4. Except sixteen amongst the 57

countries/economies, males outperform females statistically significantly on

this scale. (OECD, 2007, p.67, p.70 & p.88)


countries/economies on the using scientific evidence scale. Country means

range from 288 to 567 on this scale, and Macao’s performance (mean = 512;

SE = 1.2) is statistically significantly above the OECD average. When ranked

in descending order of percentage of 15-year-olds in Levels 2, 3, 4, 5 and 6,

the ranking rises to 6. Amongst all countries/economies, there are more cases

such that females outperform males statistically significantly on this scale.

(OECD, 2007, p.68, p.70 & p.103)


countries/economies on the combined science scale. Country means range

- 30 -

from 322 to 563 on this scale, and Macao’s performance (mean = 511; SE =

1.1) is statistically significantly above the OECD average. When ranked in

descending order of percentage of 15-year-olds in Levels 2, 3, 4, 5 and 6 the

ranking rises to 5. (OECD, 2007, p.49, p.56-58)

5. As evidenced in Table 10, students’ scientific proficiency levels are mainly

concentrated at middle levels 2, 3 and 4. Compared with the average

percentage or total number of sampled students at each of the six proficiency

levels of all participating countries/economies, there are lesser students at

either ends of the proficiency continuum, i.e. at levels below 1 and 1, as well

as at levels 5 and 6. This shows that Macao is successful to produce less

low-performing students to function productively in the scientific world, but

at the same time not so successful in producing students of higher caliber in

science identifying scientific issues, explaining phenomena scientifically, and

using scientific evidences.

If we analyze the Macao PISA data in greater depth across the various scientific

domains of study, the following results may be revealed:

1. Comparing the literacy performance difference between the combined science

scale (score = 511) and each of the three competency subscales, Macao’s 15

year-olds performed best in explaining phenomena scientifically (9.2 score

points higher), comparable in using scientific evidences (0.7 score points higher),

and not so well in identifying scientific issues (20.8 score points lower).

2. Comparing the literacy performance difference between the combined science

scale (score = 511) and each of the three knowledge of science subscales,

Macao’s 15 year-olds performed best in living systems (14.2 score points higher)

and quite good in physical systems (6.7 score points higher), and not so well in

earth and space systems (4.9 score points lower). In spite of this, knowledge of

science in the three key domains of study on average is still better than

knowledge about science (5.9 score points lower than the combined science

scale).

- 31 -

2.3 Macao 15-year-olds’scientific attitude results

As seen in Table 11, Macao 15-year-olds’scientific attitudes are very favorable. There is

a slight tendency for males to exhibit more favorable attitude than females in interest in

science and support for scientific enquiry, and for females to exhibit more favorable

attitude than males in responsibility towards sustainable development.

Table 11Macao 15-year-olds’scientific attitude results

Scientific Attitudes

DescriptiveStatistics

Interest in ScienceSupport for Scientific

Enquiry

Responsibility forSustainable

Development

Total Sample = 4,760

Mean 523.6 520.6 0.359

Standard Deviation 93.6 84.7 0.842

Males = 2,320

Mean 527.4 523.7 0.329


Females = 2,440

Mean 519.7 517.3 0.389


Note 1: Sample scientific attitude items are shown in Chapter 1.Note 2: Interest in science and support for scientific enquiry are two attitude scales with statementsembedded within scientific literacy test units, and therefore scale measures are constructed in the sameway as the combined science literacy scale. Responsibility for sustainable development is an attitude scalewith scale measure standardized across all the thirty OECD countries with mean equals to zero andstandard deviation equals to one.

- 32 -

2.4 Macao 15-year-olds’attitude results compared with OECD

countries

Compared with other OECD countries, percentages of students who agree or strongly

agree with the seven statements of responsibility for sustainable development scale are

summarized below:

Table 12

A comparison of 15-year-olds’responses to responsibility for sustainable development

between Macao and OECD countries

% of students whoagree or strongly agree

Attitude statements

Macao OECD

1. Industries should be required to prove that theysafely dispose of dangerous waste materials

98% 92%

2. I am in favor of having laws that protect thehabitats of engendered species

88% 92%

3. It is important to carry out regular checks on theemissions from cars as a condition of their use

98% 91%

4. To reduce waste, the use of plastic packagingshould be kept to a minimum

97% 82%

5. Electricity should be produced from renewablesources as much as possible, even if thisincreases the cost

97% 79%

6. It disturbs me when energy is wasted through theunnecessary use of electrical appliances

81% 69%

7. I am in favor of having laws that regulate factoryemissions even if this would increase the price ofproducts

82% 70%

Amongst the seven attitude statements in the responsibility for sustainable development

scale all except the one on laws that protect the habitats of engendered species Macao

15-year-olds’favorable responses are higher in percentage than that of the averages of

the thirty OECD countries.

- 33 -

It is beyond the scope of this report to provide comprehensive comparison results for the

other two attitude scales, i.e. interest in science and support for scientific enquiry. This

is because these attitude items are embedded in the test item units which mostly are kept

confidential for future rounds of international assessment.

2.5 Gender differences in outcomes of science learning

PISA 2006 Study provides educational researchers opportunities to rethink the

long-standing issue of gender differences in reading, mathematics and science from the

comparative education perspective.

Firstly, based on the afore-mentioned assessment results, it is evidenced that females

tend to outperform males to an appreciable extent in reading literacy, and males have a

greater chance to outperform females in mathematical literacy. However, gender

difference results across the three competences of scientific literacy are mixed,

stimulating educational researchers to think that preferred contexts of learning and

differential brain usage between the two sexes are at issue to uncover the secrets of

gender differences of literacy performance in the three key domains of school study.

Secondly, the patterns of within-school gender differences across the 43 sampled

schools in Macao are different. Figure 4-6 present some findings to illustrate the diverse

patterns existed in three sampled schools in Macao.

- 34 -

Figure 4

Literacy performance results by gender – Sampled school 1

200

250

300

350

400

450

500

550

600

650

Combined Science Identifying

Scientific Issues

Explaining

Phenomena

Scientifically

Using Scientific

Evidence

Mathematical

Literacy

Reading Literacy

Macao Sample Males Macao Sample Females

The Sampled School Males The Sampled School Females

As shown in Figure 4, pattern of gender differences in sampled school 1 across the

various literacy scales and subscales are similar to that of the Macao sample. Quite a

number of schools in the Macao sample are of this kind of pattern.

- 35 -

Figure 5


200

250

300

350

400

450

500

550

600

650


Scientific Issues

Explaining

Phenomena

Scientifically

Using Scientific

Evidence

Mathematical

Literacy

Reading Literacy




various literacy scales and subscales are not similar to that of the Macao sample. Except

reading literacy, males clearly outperform females to an appreciable extent on

mathematical literacy, as well as all subscales of the scientific literacy. Unlike the

Macao sample, females in this school do not outperform males in reading literacy. This

pattern of gender differences can be observed in a number of high-performing

co-educational schools in the Macao sample.

- 36 -

Figure 6


200

250

300

350

400

450

500

550

600

650


Scientific Issues

Explaining

Phenomena

Scientifically

Using Scientific

Evidence

Mathematical

Literacy

Reading Literacy




various literacy scales and subscales are not similar to that of the Macao sample. On all

literacy scales and subscales, males do not outperform females. This pattern of gender

differences can be observed in a number of low-performing co-educational schools in

the Macao sample.

In the light of the diverse patterns of gender differences across the scientific,

mathematical and reading literacy scales, multi-level analyses that take learning

contexts into account must be conducted for an adequate solution of this long-standing

issue in psychology of learning from the comparative education perspective.

- 37 -

Chapter 3

Quality Science Education Indicators

Abstract: This chapter seeks to search for effective quality science education indicators

amongst the scaled measures available in the PISA 2006 Study database so as to throw

lights on the design of intervention programs to elevate students’scientific literacy for

productive life-long learning. Apart from the three attitudinal outcomes analyzed in

chapter 2, out of twelve scaled measures constructed from student responses to

questions asked in the student questionnaire, ten were validated to have potential to

serve as quality science education indicators appropriate for use in Macao schooling

contexts.

3.1 Macao 15-year-olds’quality science education indicators compared

with OECD countries

In the PISA 2006 Study, students answered a questionnaire that took about 30 minutes

to complete. The questionnaire focuses on their personal background, motivation and

engagement, learning habits and perception of the living environments. Quality

indicators may be constructed to reflect dispositions and conditions facilitative of

science education in Macao in general and the individual sampled schools in particular.

There are altogether twelve quality indicators available for examination of their

appropriateness and they are described below:

- 38 -

3.1.1 General interest in science scale

There are altogether eight science study areas in this scale, the details of which are

shown in Table 13.

Table 13A comparison of 15-year-olds’responses to

general interest in science between Macao and OECD countries

% of students who showhigh interest or medium Interest

Science Study Areas

Macao OECD

1. Human biology 73% 68%

2. Topics in astronomy 58% 53%

3. Topics in chemistry 47% 50%

4. Topics in physics 49% 49%

5. The biology of plants 55% 47%

6. Ways scientists design experiments 53% 46%

7. Topics in geology 35% 41%

8. What is required for scientific explanations 37% 36%

The reported percentages for the above eight general interests in science, all except the

two topics on topics in chemistry and geology, are all higher than or equal to the

averages of the thirty OECD countries.

- 39 -

3.1.2 General value of science scale

There are altogether five general values of science in this scale, the details of which are

shown in Table 14.


general value of science between Macao and OECD countries


General values of science

Macao OECD

1. Science is important for helping us to understandthe natural world

99% 93%

2. Advances in science and technology usuallyimprove people’s living conditions

98% 92%

3. Science is valuable to society 95% 87%

4. Advances in science and technology usually helpto improve the economy

89% 80%

5. Advances in science and technology usuallybring social benefits

93% 75%

The reported percentages for the above five general values of science are all higher than

the averages of the thirty OECD countries.

- 40 -

3.1.3 Personal value of science scale

There are altogether five personal values of science in this scale, the details of which are

shown in Table 15.


personal value of science between Macao and OECD countries


Personal values of science

Macao OECD

1. I find that science helps me to understandthings around me

90% 75%

2. I will use science in many ways when I am anadult

73% 64%

3. Some concepts in science help me see how Irelate to other people

64% 61%

4. When I leave school there will be manyopportunities for me to use science

63% 59%

5. Science is very relevant to me 91% 57%

The reported percentages for the above five personal values of science are all higher

than the averages of the thirty OECD countries.

- 41 -

3.1.4 Self-efficacy in science scale

There are altogether eight tasks illustrative of self-efficacy in science in this scale, the

details of which are shown in Table 16.


self-efficacy in science between Macao and OECD countries

% of students who reply“I could do this easily” or

“I could do this with a little bit effort”Tasks

Macao OECD

1. Explain why earthquakes occur more frequentlyin some areas than in others

70% 76%

2. Recognize the science question that underlies anewspaper report on a health issue

70% 73%

3. Interpret the scientific information provided onthe labeling of food items

59% 64%

4. Predict how changes to an environment willaffect the survival of certain species

58% 64%

5. Identify the science question associated withthe disposal of garbage

68% 62%

6. Describe the role of antibiotics in the treatmentof disease

49% 59%

7. Identify the better of two explanations for theformation of acid rain

63% 58%

8. Discuss how new evidence can lead you tochange your understanding about thepossibility of life on Mars

37% 51%

Amongst the above eight tasks illustrative of self-efficacy in science all except two on

“disposal of garbage” and “formation of acid rain” are lower than that of the averages of


- 42 -

3.1.5 Self-concept in science scale

There are altogether six self-concepts in science in this scale, the details of which are

shown in Table 17.


self-concept in science between Macao and OECD countries


Self-concepts in science

Macao OECD

1. I can usually give good answers to testquestions on school science topics

60% 65%

2. When I am being taught school science I canunderstand the concepts very well

54% 59%

3. I learn school science topics quickly 49% 56%

4. I can easily understand new ideas in schoolscience

49% 55%

5. Learning advanced school science topics wouldbe easy for me

44% 47%

6. School science topics are easy for me 39% 47%

The reported percentages for the above six self-concepts in science are all lower than

the averages of the thirty OECD countries.

- 43 -

3.1.6 Enjoyment of science scale

There are altogether five situations illustrative of enjoyment of science in this scale, the



enjoyment of science between Macao and OECD countries

% of students who

agree or strongly agreeSituations

Macao OECD

1. I enjoy acquiring new knowledge in science 86% 67%

2. I generally have fun when I am learning sciencetopics

81% 63%

3. I am interested in learning about science 79% 63%

4. I like reading about science 72% 50%

5. I am happy doing science problems 56% 43%

The reported percentages for the above five situations of enjoyment of science are all

higher than the averages of the thirty OECD countries.

- 44 -

3.1.7 Instrumental motivation to learn science scale

There are altogether five situations of instrumental motivation to learn science in this

scale, the details of which are shown in Table 19.


instrumental motivation to learn science between Macao and OECD countries


Situations

Macao OECD

1. I study school science because I know it isuseful for me

85% 67%

2. Making an effort in my school science subjectis worth it because this will help me in thework I want to do later on

82% 63%

3. Studying my school science subject isworthwhile for me because what I learn willimprove my career prospects

79% 61%

4. I will learn many things in my science subjectsthat will help me get a job

76% 56%

5. What I learn in my science subject is importantfor me because I need this for what I want tostudy later on

80% 56%

The reported percentages for the above five situations of instrumental motivation are all

higher than the averages of the thirty OECD countries.

- 45 -

3.1.8 Future-oriented motivation to learn science scale

There are altogether four kinds of future-oriented motivation to learn science in this

scale, the details of which are shown in Table 20.


future-oriented motivation to learn science between Macao and OECD countries


Future-oriented motivation

Macao OECD

1. I would like to work in a career involvingscience

42% 37%

2. I would like to study science after secondaryschool

33% 31%

3. I would like to work on science projects as anadult

24% 27%

4. I would like to spend my life doing advancedscience

18% 21%

Except the first two on “work in a career involving science” and “study science after

secondary school”, the last two future-oriented motivations to learn science are lower

than that of the averages of the thirty OECD countries.

- 46 -

3.1.9 Awareness of environmental issues scale

There are altogether five cases of environmental issues in this scale, the details of which

are shown in Table 21.


awareness of environmental issues between Macao and OECD countries

% of students who“are familiar with this issue and wouldbe able to explain it well” or “I knowsomething about the issue and could

explain the general issue”Issues

Macao OECD

1. The consequences of clearing forests for otherland use

87% 73%

2. Acid rain 71% 60%

3. The increase of greenhouse gases in theatmosphere

65% 58%

4. Nuclear waste 32% 53%

5. The use of genetically modified organisms 37% 35%

Amongst the above five cases of awareness of environmental issues, all except one on

nuclear waste are higher than that of the averages of the thirty OECD countries.

- 47 -

3.1.10 Concern for environmental issues scale

There are altogether six cases of concern for environmental issues in this scale, the



concern for environmental issues between Macao and OECD countries

% of students who believe“this is a serious concern for me

personally as well as others” or “this isa serious concern for other people inmy country but not me personally”

Issues

Macao OECD

1. Air pollution 93% 92%

2. Extinction of plants and animals 81% 84%

3. Clearing of forests for other land use 81% 83%

4. Energy shortages 87% 82%


6. Water shortages 88% 76%

Amongst the above six concerns for environmental issues, all except three on air

pollution, energy shortages and water shortages are lower than that of the averages of


- 48 -

3.1.11 Optimism regarding environmental issues scale

There are altogether six cases of optimism regarding environmental issues in this scale,

the details of which are shown in Table 23.


optimism regarding environmental issues between Macao and OECD countries

% of students who believe “theproblem associated with the

environmental issues will improve overthe next 20 years”

Issues

Macao OECD

1. Energy shortages 26% 21%

2. Water shortages 27% 18%

3. Air pollution 28% 16%


5. Extinction of plants and animals 26% 14%

6. Clearing of forests for other land use 25% 13%

The reported percentages for the above six cases of optimism regarding environmental

issues are all higher than the averages of the thirty OECD countries.

- 49 -

3.1.12 Science-related activities scale

There are altogether six science-related activities in this scale, the details of which are

shown in Table 24.


science-related activities between Macao and OECD countries

% of students whodid “very often” or “regularly”

Science Activities

Macao OECD

1. Watch TV programs about science 21% 21%

2. Read science magazines or science articles innewspapers

20% 20%

3. Visit websites about science topics 11% 13%

4. Borrow or buy books on science topics 9% 8%

5. Listen to radio programmes about advances inscience

9% 7%

6. Attend a science club 4% 4%

The reported percentages for the above six science-related activities are comparable

with the averages of the thirty OECD countries.

- 50 -

3.2 In search of quality science education indicators for Macao

In the student questionnaire, there are questions allowing a number of scales to be

constructed that may be served as quality science education indicators and students’

responses to these scales have been reported and compared in section 3.1 above. A

total of twelve such scales are examined and Pearson correlations with scientific literacy

performance broken down by gender are reported in Table 25.

Table 25Pearson correlation of quality science education indicators

with scientific literacy performance scores

Pearson Correlation withScientific Literacy Score

Potential Quality Science Education Indicator

Male Female All

1. General interest in science 0.250 0.256 0.192

2. General value of science 0.193 0.140 0.168

3. Personal value of science 0.071 0.088 0.080

4. Self-efficacy in science 0.321 0.312 0.318

5. Self-concept in science 0.182 0.116 0.151

6. Enjoyment of science 0.289 0.252 0.273

7. Instrumental motivation to learn science 0.237 0.134 0.192

8. Future-oriented motivation to learn science 0.071 0.104 0.088

9. Awareness of environmental issues 0.417 0.389 0.405

10. Concern for environmental issues 0.131 0.093 0.111

11. Optimism regarding environmental issues -0.154 -0.167 -0.157

12. Science-related activities 0.156 0.178 0.168

After examining Table 25, it was decided to drop personal value of science and

future-oriented motivation to learn science as quality science education indicators. The

remaining ten indicators all have moderate to strong correlations (i.e. between 0.10-0.45)

with scientific literacy proficiency scores for the 15-year-olds in Macao.

- 51 -

3.3 The ten quality science education indicators for Macao

Ten scale scores are computed for each sampled student as quality science education

indicators, the mean values of which are broken down by gender because males and

females may exhibit their dispositions, attitudes and values differently. As shown in

Table 26, the first six indicators (i.e. 1-6) are psychological dispositions facilitative of

science learning, the next three indicators (i.e. 7-9) pertain to consciousness of the mind

on environments instrumental to an adequate understanding of the contexts of science

learning, and the last one (i.e. 10) is behavior-oriented in response to the dispositions

and consciousnesses.

Table 26Selected quality science education indicators for the Macao sample

Scale Score (Mean)Quality Science Education Indicators

Male Female All

1. General interest in science 0.20 0.00 0.10

2. General value of science 0.50 0.58 0.54

3. Self-efficacy in science -0.04 -0.17 -0.11

4. Self-concept in science 0.10 -0.36 -0.11

5. Enjoyment of science 0.52 0.31 0.41

6. Instrumental motivation to learn science 0.43 0.34 0.39

7. Awareness of environmental issues 0.08 0.03 0.06

8. Concern for environmental issues 0.01 0.22 0.11

9. Optimism regarding environmental issues 0.46 0.25 0.36

10. Science-related activities 0.38 0.16 0.27

Note: Scale measures are standardized across all the thirty OECD countries with mean equals to zero andstandard deviation equals to one.

- 52 -

Chapter 4

Literacy-ESCS Relationships for Macao Schools

Abstract: This chapter analyzes the intricate relationships between literacy performance

and ESCS (Index of Economic Social and Cultural Status of the Home) to throw light

on the excellence and equity dimensions of educational provisions for the 15-year-olds

in Macao. Plots of these relationships in the Macao sample, as well as within-school

Pearson correlations indicative of the linear components of these curvilinear

relationships for scientific literacy in individual sampled schools are presented to

illustrate the complicated analytical issues concerned.

4.1 Plots of literacy performance with ESCS in the Macao sample

As seen in Figure 7 and 8, there are non-linear relationships in the Macao sample

between scientific, mathematical and reading literacy performance with economic,

social and cultural status of the home. The same relationships are also observed for the

three scientific literacy subscales. Generally speaking, higher ESCS is associated with

moderately higher mathematical, scientific, and reading literacy performance. It is

noteworthy that this non-linear relationship may be different in the sampled school

depending on its admission policy and the student intake (see section 4.2).

- 53 -

Figure 7

Plots of literacy performance with ESCS in the Macao Sample

350

400

450

500

550

600

650

700

750

-4 -3 -2 -1 0 1 2 3

Index of Economic, Social and Cultural Status (ESCS)

Mathematical Literacy Scientific Literacy Reading Literacy

- 54 -

Figure 8

Plots of scientific literacy subscale performance with ESCS in the Macao Sample

350

400

450

500

550

600

650

700

750

-4 -3 -2 -1 0 1 2 3


Explaining Phenomena Scientifically Using Scientific Evidence Identifying Scientific Issues

Based on the afore-mentioned assessment results, there are three findings worthy of

concern by schools and the educational authority.

1. Although the impact of ESCS on mathematical, scientific and reading literacy are

not substantial, elevating homes of low ESCS to higher levels is still needed. This

is expected not only to bring about better educational opportunities and equitable

educational results, but also can enhance the mathematical, scientific and reading

literacy proficiency levels of the 15-year-olds in Macao.

2. In the PISA 2006 Study, Macao’s 15-year-olds achieve very well in mathematical,

scientific and reading literacy. Admittedly, there is still ample room for an increase

in the proportion of high-performing students.

- 55 -

3. It is heartening to learn that the proportion of low-performing students is low

compared with other participating countries/economies. This success may be

attributed to the instructional effectiveness of Macao’s teachers. In spite of this

success, it is noteworthy that quite a large proportion of Macao’s 15-year-olds are

still studying at grade 7 and 8. If those low-performing students studying at grade 7

and 8 can be arranged for academic guidance and counseling with reference to the

ten quality science education indicators examined in Chapter 3, then it is expected

their scientific literacy proficiency can be elevated accordingly.

4.2 Within-school correlations of scientific literacy performance with

ESCS

Figure 7 and 8 may not be drawn for each sampled school in order to examine the

within-school literacy-ESCS relationship because of two important reasons. First, the

number of sampled students within school is small, and therefore standard error of

estimates (SE) tends to be very large. Second, depending on school admission policy,

each sampled school covers a portion of the ESCS continuum only. Therefore,

within-school literacy-ESCS relationship may not be pronounced enough to decide

whether the relationship concerned is positively linear, negatively linear, or non-linear

as in the case of Macao sample. For scientific literacy in the PISA 2006 Study,

within-school Pearson correlations indicating the linear component of the literacy-ESCS

relationship range from -0.234 to 0.793 (see Table 27). Hierarchical linear modeling

(HLM) is needed to explain these varying slopes of relationships estimated with

different degrees of reliabilities.

- 56 -

Table 27Relationships of scientific literacy and ESCS, broken down by school

Scientific Literacy ESCS CorrelationSchoolNumber Mean SE Mean SE r SE

1 469.775 5.416 -1.364 0.053 0.099 0.087

2 453.722 7.440 -1.701 0.082 0.099 0.125

3 586.930 5.455 0.051 0.064 0.053 0.078

4 504.016 7.125 -0.134 0.068 -0.095 0.114

5 510.471 4.047 -0.836 0.048 -0.170 0.078

6 530.680 5.295 -0.730 0.059 0.038 0.083

7 509.971 4.738 -0.233 0.072 0.036 0.090

8 589.809 31.864 0.624 0.582 0.793 0.254

9 525.820 12.764 0.623 0.101 0.656 0.075

10 455.775 22.574 0.605 0.217 0.065 0.350

11 338.674 35.694 0.730 0.000 . .

12 516.115 4.687 -1.120 0.053 0.121 0.104

13 475.670 5.206 -1.215 0.069 -0.011 0.101

14 526.464 4.169 -0.994 0.059 0.231 0.085

15 427.989 6.655 -0.442 0.079 0.004 0.099

16 512.806 4.888 -1.204 0.057 0.057 0.100

17 510.484 5.260 -1.313 0.052 0.028 0.090

18 471.477 10.471 -0.947 0.117 0.088 0.176

19 537.867 5.010 -1.078 0.068 -0.013 0.089

20 560.449 7.451 -0.182 0.084 0.086 0.170

21 521.640 4.524 -0.809 0.061 -0.036 0.078

22 507.442 4.950 -1.141 0.063 0.000 0.089

23 534.951 4.608 -0.564 0.056 0.048 0.080

24 542.669 4.770 -1.072 0.057 0.115 0.093

25 434.136 7.652 -1.185 0.077 0.060 0.104

26 506.436 10.268 -1.346 0.118 -0.234 0.174

27 582.657 6.001 -0.621 0.063 0.033 0.096

28 407.376 29.854 -1.336 0.167 0.596 0.199

29 475.126 6.254 -0.837 0.062 0.151 0.095

30 574.690 4.831 -1.036 0.052 -0.001 0.085

31 466.526 6.693 -1.520 0.064 0.097 0.094

32 494.786 5.650 -1.335 0.061 -0.006 0.089

33 527.868 9.838 -0.342 0.155 -0.006 0.225

34 478.288 7.451 -1.335 0.107 0.126 0.099

35 485.910 5.200 -0.941 0.064 0.057 0.070

36 405.059 22.718 -1.027 0.222 0.123 0.241

37 507.890 4.881 -1.111 0.052 0.001 0.093

38 531.011 5.136 -0.266 0.065 0.003 0.091

39 511.117 5.776 -0.909 0.053 0.204 0.069

40 502.165 5.552 -1.345 0.057 0.015 0.078

41 426.345 6.930 -1.157 0.110 0.158 0.111

42 514.795 7.148 -1.210 0.111 0.227 0.151

43 461.036 19.229 -0.006 0.192 0.213 0.195

Total 510.803 1.058 -0.909 0.014 0.148 0.016

Note: There is only one sampled student for School 11. So no analysis is done for this school. Also,standard errors (SE) of estimates for schools with small number of sampled students are generally large.

- 57 -

4.3 Between-school scientific literacy performance with ESCS

Based on mean scientific literacy and ESCS in Table 27, the following figure showing

the between-school scientific literacy performance and ESCS relationships may be

drawn.

Figure 9

Plots between-school scientific literacy performance with ESCS

32

13

14

35

21

7

2

311

26

40

28

34

1742

16

25

41

12

37

19

24

30

36

18

39

29

5

6

27

23

15

33 38

20

4

43

3

10

9

8

11

300

350

400

450

500

550

600

-2 -1.5 -1 -0.5 0 0.5 1


▲High-performing school ＋Average-performing school ■Low-performing school

22

Sci

en

tifi

cL

itera

cy

If we classify the scientific literacy performance of the 43 schools into high-performing

(13 schools), average-performing (14 schools), and low-performing (13 schools), the

following three observations may be made without disclosing the identity of individual

schools:

1. Amongst the four schools classified as “International/Portuguese” in the PISA

2006 Study, the ESCS of these schools are the highest amongst the 43 sampled

schools (mean ESCS > 0.50). As far as scientific literacy performance is concerned,

- 58 -

two are high-performing (top one-third of the 43 schools) and two are

low-performing (bottom one-third of the 43 schools). Therefore, there is no

apparent relationship between the four school's scientific literacy performance with

school ESCS.

2. The thirteen high-performing schools in scientific literacy (except the two

International/Portuguese schools mentioned above) are generally associated with

homes of higher ESCS (i.e. -1.1 <ESCS< 0.5).

3. The thirteen low-performing schools in scientific literacy (except the two

International/Portuguese schools mentioned before) are generally associated with

homes of lower ESCS (i.e. -1.80 <ESCS< -0.90). There are two low-performing

schools (school number 15 and 43) having students coming from homes of higher

ESCS (i.e. ESCS = -0.44 and -0.01). Without making a serious investigation,

reasons for the two schools' low scientific literacy performance can only be

speculated.

- 59 -

Chapter 5

International Comparison of Literacy Performance

Abstract: From an international comparison perspective, this chapter analyzes Macao’s

scientific, mathematical, and reading literacy performance in the PISA 2006 Study. It

highlights a number of East Asian and western high-performing countries/economies

that may serve as exemplary models for Macao’s educational improvement and

curriculum reform.

5.1 Performance in the three literacy domains – An international

comparison

Table 28 displays literacy performance results allowing educational researchers and

policy makers to compose a league table that serves their international comparison

purposes. Macao’s literacy performance may not only be compared with East Asian

countries/economies (e.g. Chinese Taipei, Hong Kong, Japan, Korea, Indonesia, and

Thailand), but also contrasted with the other western high-performing

countries/economies (e.g. Australia, Canada, Estonia, Finland, Liechtenstein,

Netherlands, and New Zealand) in the three domains of literacy assessed in the PISA

2006 Study.

Table 28Performance of countries/economies in the three literacy domains

Scientific LiteracyMathematical

LiteracyReading Literacy

Country/Economy

Mean SE Mean SE Mean SE

1. Azerbaijan 382.236 2.747 475.769 2.211 351.959 3.105

2. Argentina 391.285 6.013 380.549 6.193 373.977 7.138

3. Australia 528.997 2.954 522.208 3.152 516.014 3.010

4. Austria 510.472 3.909 505.338 3.658 491.295 3.9495. Belgium 509.594 2.577 521.746 2.945 502.753 3.062

6. Brazil 390.229 2.788 369.515 2.933 392.885 3.743

7. Bulgaria 434.001 6.115 413.448 6.133 401.934 6.909

8. Canada 534.237 2.029 527.035 2.047 526.919 2.419

9. Chile 438.418 4.283 410.995 4.492 442.692 4.719

10. Chinese Taipei 532.449 3.570 549.357 4.102 496.235 3.37611. Colombia 387.934 3.377 369.979 3.783 385.306 5.083

- 60 -

12. Croatia 492.988 2.454 466.697 2.419 477.326 2.804

13. Czech Republic 512.825 3.480 509.859 3.551 482.715 4.181

14. Denmark 495.850 3.110 513.026 2.617 494.483 3.17815. Estonia 531.366 2.524 514.575 2.745 500.750 2.927

16. Finland 563.315 2.020 548.358 2.296 546.868 2.147

17. France 495.175 3.362 495.538 3.170 487.706 4.06418. Germany 515.502 3.867 504.322 3.950 496.267 4.37619. Greece 473.321 3.232 459.202 2.968 459.711 4.044

20. Hong Kong-China 542.192 2.474 547.461 2.670 536.066 2.422

21. Hungary 503.891 2.677 490.937 2.885 482.375 3.281

22. Iceland 491.457 1.501 506.907 1.457 486.076 1.426

23. Indonesia 393.346 5.748 389.654 5.617 392.901 5.562

24. Ireland 508.291 3.188 501.472 2.785 517.313 3.536

25. Israel 453.626 3.733 442.858 4.214 437.134 4.491

26. Italy 474.970 1.991 462.336 2.219 468.381 2.317

27. Japan 530.155 3.441 523.939 3.290 496.795 3.73828. Jordan 421.884 2.841 384.034 3.304 400.581 3.271

29. Korea 522.118 3.359 547.458 3.761 556.022 3.813

30. Kyrgyzstan 321.887 2.936 310.583 3.410 284.704 3.480

31. Latvia 489.496 2.973 486.166 3.027 479.492 3.730

32. Liechtenstein 523.408 4.244 525.321 4.638 510.793 4.488

33. Lithuania 488.597 2.777 485.990 2.823 470.374 3.118

34. Luxembourg 486.274 1.055 490.002 1.066 479.367 1.281

35. Macao-China 510.803 1.058 525.003 1.303 492.288 1.099

36. Mexico 409.765 2.663 405.027 2.890 409.457 3.167

37. Netherlands 524.833 2.743 530.654 2.586 506.747 2.925

38. New Zealand 530.547 2.737 521.769 2.484 521.381 3.000

39. Norway 486.478 3.113 489.846 2.640 484.293 3.182

40. Poland 498.046 2.368 494.995 2.354 507.534 2.781

41. Portugal 474.398 3.010 465.606 3.039 472.584 3.536

42. Qatar 349.188 0.856 317.956 1.022 312.213 1.196

43. Romania 418.593 4.210 414.286 4.251 396.989 4.897

44. Russian Federation 479.130 3.690 475.260 3.903 440.145 4.436

45. Slovak Republic 488.385 2.587 492.106 2.823 466.350 3.056

46. Slovenia 518.893 1.101 504.368 1.001 494.500 0.97347. Spain 488.598 2.460 479.840 2.338 460.391 2.301

48. Sweden 503.293 2.376 502.356 2.408 507.313 3.436

49. Switzerland 511.820 3.121 529.561 3.257 499.589 3.100

50. Thailand 420.566 2.183 416.663 2.311 416.750 2.572

51. Tunisia 385.402 2.959 365.479 3.959 380.335 4.020

52. Turkey 423.749 3.841 423.941 4.901 447.132 4.206

53. United Kingdom 514.738 2.290 495.444 2.142 495.077 2.25454. United States 488.907 4.224 474.352 4.020 - -

55. Uruguay 428.052 2.750 426.799 2.605 412.516 3.434

56. Serbia and/or Montenegro 433.298 2.746 431.959 3.178 400.170 3.134

All participatingcountries/economies

461.387 1.046 454.005 1.011 450.572 1.120

Note 1: Serbia and Montenegro student data are grouped together in the PISA database. Therefore, thereare 56 instead of 57 entries in the table.Note 2: Reading literacy results for United States were not available due to printing errors of the testbookletsNote 3: Literacy means which are: (i) statistically significantly higher than that of Macao-China areshaded for easy identification; (ii) comparable with that of Macao-China are printed in bold italic; (ii)statistically significantly lower than that of Macao-China are printed as usual without any change of fontor visual effect.

- 61 -

5.2 Selection of countries/economies exemplary for Macao’s

educational improvement and curriculum reform

Based on Table 29, one can see that Hong Kong, Chinese Taipei, Japan and Korea

performed very well in all three literacy domains and their ranks are amongst the top of

all participating countries/economies. On the whole, East Asian countries/economies

listed in Table 29 score highly in all three literacy domains. One hypothesis to explain

this finding pertains to the Confucian tradition of high parental emphases on their

children’s early education, and their high expectation they cast on their children during

the obligatory years of schooling.

Table 29Top five high-performing East Asian countries/economies

Scientific Literacy Mathematical Literacy Reading Literacy

East AsianCountry/Economy

MeanEast AsianCountry/Economy

MeanEast AsianCountry/Economy

Mean

Hong Kong-China 542.192 Chinese Taipei 549.357 Korea 556.022

Chinese Taipei 532.449 Hong Kong-China 547.461 Hong Kong-China 536.066

Japan 530.155 Korea 547.458 Japan 496.795

Korea 522.118 Macao-China 525.003 Chinese Taipei 496.235

Macao-China 510.803 Japan 523.939 Macao-China 492.288


461.387All participatingcountries/economies


450.572

However, it should be pointed out that the Confucian tradition hypothesis alone cannot

explain why some countries such as Australia, Finland, Canada, Liechtenstein, New

Zealand, Netherlands, and Switzerland perform so well in all three literacy domains (see

Table 30). There are countries/economies like Estonia and Ireland which perform highly

on individual literacy domains. Macao can learn a lot from these countries if factors

leading to their success can be revealed and exemplary educational practices can be

modeled upon in Macao schooling contexts.

- 62 -

Table 30Non-East Asian countries/economies

having higher or comparable literacy performance than Macao


Non-East AsianCountry/Economy

MeanNon-East AsianCountry/Economy

MeanNon-East AsianCountry/Economy

Mean

Finland 563.315 Finland 548.358 Finland 546.868

Canada 534.237 Netherlands 530.654 Canada 526.919

Estonia 531.366 Switzerland 529.561 New Zealand 521.381

New Zealand 530.547 Canada 527.035 Ireland 517.313

Australia 528.997 Liechtenstein 525.321 Australia 516.014

Netherlands 524.833 Australia 522.208 Liechtenstein 510.793

Liechtenstein 523.408 Belgium 521.746 Poland 507.534

Slovenia 518.893 New Zealand 521.769 Sweden 507.313

Germany 515.502 Netherlands 506.747

United Kingdom 514.738 Belgium 502.753

Czech Republic 512.825 Estonia 500.750

Switzerland 511.820 Switzerland 499.589

Austria 510.472 Germany 496.267

Belgium 509.594 United Kingdom 495.007

Ireland 508.291 Slovenia 494.500

Denmark 494.483

Austria 491.295




450.572

Note: Reading literacy results for United States were not available due to printing errors of the testbooklets

Last but not least, countries/economies like United States, United Kingdom and

Portugal have in the past influenced Macao’s basic education curriculum and

educational system. Hence, it may be a good idea to examine the strengths and

weaknesses of these educational systems to see whether there are exemplary educational

practices and provisions that can be modeled upon by Macao.

- 63 -

Table 31Performance results of other countries/economies in the three literacy domains


Country/Economy Mean Country/Economy Mean Country/Economy Mean

United States 488.907 United Kingdom 495.444 Portugal 472.584

Portugal 474.398 United States 474.352 Thailand 416.750

Thailand 420.566 Portugal 465.606 Indonesia 392.901

Indonesia 393.346 Thailand 416.663

Indonesia 389.654




450.572

Note: Reading literacy results for United States were not available due to printing errors of the testbooklets.

- 64 -

Chapter 6

Thematic Reports and Follow-up Studies

The present report serves as a valuable starting point for educational researchers and

policy makers to have an initial understanding of 15-year-old students’ literacy

performance in three key domain areas. More in-depth analyses are necessary to make

full use of the student, school and system data collected in the PISA studies.

Apart from the PISA 2006 Study database, the data analysis manual, technical report,

international report, as well as all relevant OECD PISA publications should be studied

for a full understanding of the findings reported in this Macao PISA 2006 Study Report

(see the full list of references documented in this report; see also www.pisa.oecd.org).

Evidently, it is not possible in this first report to address all kinds of issues and problems

thought significant by policy makers and teachers in Macao. However, this report does

point to key topics that should be addressed in thematic reports in the future. Three

thematic reports are worthy of preparation and publication:

1. By making full use of the student, school and system data, examine factors

affecting excellence and equity in literacy performance in the three key domains

for 15-year-olds in Macao. In particular, conditions, processes and contexts

facilitative of literacy acquisition for the different types of schools and different

kinds of students (e.g. students studying in single-sex versus co-educational

schools) should be examined.

2. By linking the literacy performance results with previous PISA 2003 Study data,

chart the change in literacy performance in the three key domains. Factors

affecting growth and decline of literacy performance should be identified so as to

guide educational improvement and curriculum reform.

3. By comparing selected high performing East Asian with western

countries/economies, attempt to understand the similarities and differences

pertaining to excellence in literacy performance for 15-year-olds in the three key

domains. This report revealed that East Asian countries/economies such as Chinese

- 65 -

Taipei, Hong Kong, Japan, and Korea may be compared with western countries

such as Australia, Estonia, Finland, Canada, Ireland, Liechtenstein, New Zealand

and Netherlands. It is hoped that such international comparisons can uncover the

secrets of academic excellence of 15-year-olds students of the top five

high-performing East Asian countries/economies.

Last but not the least, follow-up intervention programs should be designed to monitor

the learning progress of those low-performing students identified in the PISA 2006

Study. Given quite a large proportion of 15-year-old are repeaters and they are likely to

remain in the basic education system in the next few years, schools and educational

researchers should join efforts to explore ways of counseling them, and to this end the

ten quality science education indicators reported in chapter 3 are good starting points for

an initial understanding of their psychological dispositions and learning characteristics.

- 66 -

References

Lo, L.F. & Cheung, K.C. (2005). 2003 Macau-PISA Final Report: Assessment of

15-Year-Old Macau Students. Macao: The Macau-PISA Center.

OECD. (1999). Measuring Student Knowledge and Skills: A New Framework for

Assessment. Paris: OECD.

OECD. (2001). Knowledge and Skills for Life: First Results from the OECD

Programme for International Student Assessment (PISA) 2000. Paris: OECD.

OECD. (2002b). Reading for Change: Performance and Engagement across Countries

Paris: OECD.

OECD (2004a). Learning for Tomorrow’s World: First Results from PISA 2003. Paris:

OECD.

OECD (2004b). What Makes School Systems Perform? Seeing School Systems through

the Prism of PISA. Paris: OECD.

OECD (2004c). Problem-solving for Tomorrow’s World: First Measures of

Cross-curricular Competencies from PISA 2003. Paris: OECD.

OECD (2005a). The Definition and Selection of Key Competencies: Executive Summary.

Paris: OECD.

OECD (2005b). PISA 2003 Data Analysis Manual: SPSS Users. Paris: OECD.

OECD (2006). Assessing Scientific, Reading and Mathematical Literacy: A Framework

for PISA 2006. Paris: OECD.

OECD (2007). PISA 2006 -- Science Competencies for Tomorrow’s World, Vol. 1 & 2.

Paris: OECD.

Documents

Macao PISA 2006 Study Report Number One - … PISA 2006 Study Report...Macao PISA 2006 Study Report Number One: Assessment of scientific, mathematical and reading literacy performance