View
216
Download
2
Category
Preview:
Citation preview
Macao PISA 2006 Study ReportNumber One:
Assessment of scientific, mathematical and reading literacy performance of15-year-old students from an international comparison perspective
Kwok-cheung Cheung & Pou-seong Sit
Educational Testing and Assessment Research Center
University of Macau
Macao, People’s Republic of China
December, 2007
- 1 -
Acknowledgement
Fruitful completion of the Macao PISA 2006 Study was contingent on:
(1) Financial sponsorship and steering by Education and Youth Affairs Bureau of MacaoGovernment;
(2) Guidance and resource support by the University of Macau authority;
(3) Academic and technical support by the Educational Testing and AssessmentResearch Center, Faculty of Education, University of Macau, Macao;
(4) Cooperation of secondary schools participating in the PISA 2006 Study;
(5) Active participation of students and their parents in responding to tests andquestionnaires.
- 2 -
Table of Contents
Acknowledgement ............................................................................................................ 1
Table of Contents.............................................................................................................. 2
List of Tables .................................................................................................................... 4
List of Figures................................................................................................................... 6
Executive Summary.......................................................................................................... 7
Chapter 1 Conduct of Enquiry..................................................................................... 10
1.1 Brief introduction ............................................................................................. 10
1.2 Sample design....................................................................................................11
1.3 Scientific literacy assessment framework......................................................... 13
1.4 An example of scientific literacy item unit....................................................... 15
1.4.1 Sample scientific literacy test items – Acid Rain .................................. 16
1.5 Descriptions of proficiency levels of scientific, mathematical and reading
literacy scales.................................................................................................. 19
1.6 Examples of scientific attitudes assessed as outcomes of science learning .. 22
1.6.1 Sample attitude items ............................................................................ 22
Chapter 2 A Profile of Literacy Performance for 15-year-olds in Macao ................... 25
2.1 Macao 15-year-olds’literacy performance ....................................................... 25
2.2 Macao 15-year-olds’scientific literacy performance compared with other
participating countries/economies .................................................................. 29
2.3 Macao 15-year-olds’scientific attitude results ................................................. 31
2.4 Macao 15-year-olds’attitude results compared with OECD countries ............ 32
2.5 Gender differences in outcomes of science learning ........................................ 33
Chapter 3 Quality Science Education Indicators......................................................... 37
3.1 Macao 15-year-olds’quality science education indicators compared with
OECD countries.............................................................................................. 37
3.1.1 General interest in science scale............................................................ 38
3.1.2 General value of science scale............................................................... 39
3.1.3 Personal value of science scale.............................................................. 40
3.1.4 Self-efficacy in science scale................................................................. 41
3.1.5 Self-concept in science scale ................................................................. 42
- 3 -
3.1.6 Enjoyment of science scale ................................................................... 43
3.1.7 Instrumental motivation to learn science scale...................................... 44
3.1.8 Future-oriented motivation to learn science scale ................................. 45
3.1.9 Awareness of environmental issues scale .............................................. 46
3.1.10 Concern for environmental issues scale .............................................. 47
3.1.11 Optimism regarding environmental issues scale ................................. 48
3.1.12 Science-related activities scale ............................................................ 49
3.2 In search of quality science education indicators for Macao............................ 50
3.3 The ten quality science education indicators for Macao .................................. 51
Chapter 4 Literacy-ESCS Relationships for Macao Schools ...................................... 52
4.1 Plots of literacy performance with ESCS in the Macao sample....................... 52
4.2 Within-school correlations of scientific literacy performance with ESCS....... 55
4.3 Between-school scientific literacy performance with ESCS............................ 57
Chapter 5 International Comparison of Literacy Performance ................................... 59
5.1 Performance in the three literacy domains – An international comparison...... 59
5.2 Selection of countries/economies exemplary for Macao’s educational
improvement and curriculum reform.............................................................. 61
Chapter 6 Thematic Reports and Follow-up Studies................................................... 64
References ...................................................................................................................... 66
- 4 -
List of Tables
Table 1 Characteristics of schools in the PISA 2006 Macao sample … … … … . 11
Table 2 Number of 15-year-olds sampled and tested in Macao … … … … … … . 12
Table 3 Grade distribution of 15-year-olds tested … … … … … … … … … … … . 12
Table 4 Contexts for the PISA 2006 science assessment … … … … … … … … ... 13
Table 5 Blueprint used in the design of scientific literacy assessment itemunits (Example item unit: ACID RAIN) … … … … … … … … … … … … 16
Table 6 Description of the six scientific literacy proficiency levels … … … … .. 19
Table 7 Description of the six mathematical literacy proficiency levels … … ... 20
Table 8 Description of the five reading literacy proficiency levels … … … … ... 21
Table 9 Macao 15-year-olds’literacy performance results … … … … … … … … 25
Table 10 Distribution of Macao 15-year-olds’proficiency levels on the literacyscales … … … … … … … … … … … … … … … … … … … … … … … … … .. 27
Table 11 Macao 15-year-olds’scientific attitude results … … … … … … … … … .. 31
Table 12 A comparison of 15-year-olds’ responses to responsibility forsustainable development between Macao and OECD countries … … ... 32
Table 13 A comparison of 15-year-olds’ responses to general interest inscience between Macao and OECD countries … … … … … … … … … .. 38
Table 14 A comparison of 15-year-olds’responses to general value of sciencebetween Macao and OECD countries … … … … … … … … … … … … … 39
Table 15 A comparison of 15-year-olds’ responses to personal interest ofscience between Macao and OECD countries … … … … … … … … … .. 40
Table 16 A comparison of 15-year-olds’ responses to self-efficacy in sciencebetween Macao and OECD countries … … … … … … … … … … … … .. 41
Table 17 A comparison of 15-year-olds’ responses to self-concept in sciencebetween Macao and OECD countries … … … … … … … … … … … … .. 42
Table 18 A comparison of 15-year-olds’ responses to enjoyment of sciencebetween Macao and OECD countries … … … … … … … … … … … … ... 43
Table 19 A comparison of 15-year-olds’ responses to instrumental motivationto learn science between Macao and OECD countries … … … … … … . 44
Table 20 A comparison of 15-year-olds’ responses to future-orientedmotivation to learn science between Macao and OECD countries … ... 45
Table 21 A comparison of 15-year-olds’ responses to awareness ofenvironmental issues between Macao and OECD countries … … … … . 46
Table 22 A comparison of 15-year-olds’ responses to concern forenvironmental issues between Macao and OECD countries … … … … . 47
- 5 -
Table 23 A comparison of 15-year-olds’ responses to optimism regardingenvironmental issues between Macao and OECD countries … … … … . 48
Table 24 A comparison of 15-year-olds’responses to science-related activitiesbetween Macao and OECD countries … … … … … … … … … … … … ... 49
Table 25 Pearson correlation of quality science education indicators withscientific literacy performance scores … … … … … … … … … … … … .. 50
Table 26 Selected quality science education indicators for the Macao sample ... 51
Table 27 Relationships of scientific literacy and ESCS, broken down by school 56
Table 28 Performance of countries/economies in the three literacy domains … . 59
Table 29 Top five high-performing East Asian countries/economies … … … … .. 61
Table 30 Non-East Asian countries/economies having higher or comparableliteracy performance than Macao … … … … … … … … … … … … … … . 62
Table 31 Performance results of other countries/economies in the three literacydomains … … … … … … … … … … … … … … … … … … … … … … … … . 63
- 6 -
List of Figures
Figure 1 Framework for PISA 2006 science assessment … … … … … … … … … 15
Figure 2 Percentage of 15-year-olds at different scientific literacy proficiencylevels across grades in the Macao sample … … … … … … … … … … … 28
Figure 3 Percentage of 15-year-olds at different grade levels across scientificliteracy proficiency levels in the Macao sample … … … … … … … … .. 28
Figure 4 Literacy performance results by gender – Sampled school 1 … … … .. 34
Figure 5 Literacy performance results by gender – Sampled school 2 … … … .. 35
Figure 6 Literacy performance results by gender – Sampled school 3 … … … .. 36
Figure 7 Plots of literacy performance with ESCS in the Macao Sample … … . 53
Figure 8 Plots of scientific literacy subscale performance with ESCS in theMacao Sample … … … … … … … … … … … … … … … … … … … … … . 54
Figure 9 Plots between-school scientific literacy performance with ESCS … ... 57
- 7 -
Executive Summary
The main purpose of this report is to complement the PISA 2006 Study International
Report (entitled PISA 2006 Science competencies for tomorrow’s world, Vol. 1 & 2)
released on 4 December, 2007 in Paris. It is hoped that this report can serve as a good
starting point for any reading or systematic enquiry that makes use of Macao PISA 2006
Study data in the future. The following is an executive summary of this report.
1. Macao, special administrative region of People’s Republic of China, participated in
OECD’s Program for International Student Assessment (PISA) for the first time in
2003, and after 3 years, participated again in 2006. Macao will participate for the
third time in 2009. The followings are results of the PISA 2006 Study.
2. In each PISA study, three main kinds of literacy are assessed, namely, reading,
mathematical and scientific literacy. The target students assessed are all secondary
students who are aged between 15 years three months and 16 years two months at the
time of assessment. Most students are studying in the middle grade levels (i.e. grade
8, 9 and 10), whereas some students are studying in the lower or higher grade levels
(i.e. grade 7 and 11).
3. When comparing the literacy performance across schools, it is important to note that
the literacy assessed referred to the cumulative educational effects of all schools that
the students have previously attended. Therefore, a low-performing school identified
in the PISA Study may not be a poor school. Low-performing students dropped out
from one school may be subsequently enrolled in another school and thereby
lowering the school’s literacy performance level.
4. The focus of the PISA 2006 Study was on science. Amongst the 57 participating
countries/economies, Macao’s scientific literacy performance was statistically
significantly above the OECD average, and Macao ranked between 15 and 20 on the
combined science scale. In decreasing order of the mean of scientific literacy score,
countries/economies statistically significantly higher than Macao were: Finland,
Hong Kong, Canada, Chinese Taipei, Estonia, Japan, New Zealand, Australia,
Netherlands, Liechtenstein, Korea and Slovenia.
- 8 -
5.There are six scientific proficiency levels in the combined science scale. On one hand,
students who cannot reach the lowest level 1 are regarded as low-performing, and
they run the risk of being unable to function productively in the life-long learning
society in the 21st Century. Only 1.4% of the 15-year-olds performed below level 1
of the scientific literacy scale. On the other hand, students who can reach the highest
level 6 are regarded as high-performing, and they are cherished as valuable talents so
much needed for scientific advancement in nowadays knowledge society. Only 0.3%
of the 15-year-olds performed at this highest level on the combined science scale.
6. Amongst the three content areas of science, Macao’s 15-year-olds performed best in
“living systems”, moderately well in “physical systems” and less well in “earth and
space systems”. Amongst the three key scientific competencies, Macao’s
15-year-olds performed best in “explaining phenomena scientifically”, moderately
well in “using scientific evidence”, and less well in “identifying scientific issues”.
Generally speaking, Macao’s 15-year-olds performed better in “knowledge of
science” than “knowledge about science”. In Macao, males performed better than
females in “explaining phenomena scientifically”, whereas females performed better
than males in “identifying scientific issues”. Males and females were comparable in
literacy performance as far as “using scientific evidences” was concerned.
7. Ten quality science education indicators verified to affect scientific literacy
performance have been identified, namely: (1) General interest in science, (2)
General value of science, (3) Self-efficacy in science, (4) Self-concept in science, (5)
Enjoyment of science, (6) Instrumental motivation to learn science, (7) Awareness of
environmental issues, (8) Concern for environmental issues, (9) Optimism regarding
environmental issues, (10) Science-related activities. Guided by these indicators,
intervention studies can be designed and put to practice to help low-performing
students enhance scientific literacy performance.
8. Another minor focus of the PISA 2006 Study was on mathematics. Amongst the 57
participating countries/economies, Macao’s mathematical literacy performance was
statistically significantly above the OECD average, and Macao ranked between 7 and
11 on the mathematics scale. In decreasing order of the mean of mathematical
literacy score, countries/economies statistically significantly higher than Macao are:
Chinese Taipei, Finland, Hong Kong and Korea.
- 9 -
9. Another minor focus of the PISA 2006 Study was on reading. Amongst the 57
participating countries/economies, Macao’s reading literacy performance was not
statistically significantly different from the OECD average, and Macao ranked
between 18 and 22 on the reading scale. In decreasing order of the mean of reading
literacy score, countries/economies statistically significantly higher than Macao are:
Korea, Finland, Hong Kong, Canada, New Zealand, Ireland, Australia, Liechtenstein,
Poland, Sweden, Netherlands, Belgium, Estonia and Switzerland.
10. Amongst the 57 participating countries/economies, percentage of variance in
performance in science explained by the PISA index of economic, social and
cultural status of the home (ESCS) is the lowest. The overall effect and
between-school effect of ESCS on scientific literacy are also amongst the lowest of
all participating countries/economies. Hence, Macao’s basic educational system
succeeds in providing equitable schooling opportunities for the student body it
served.
- 10 -
Chapter 1Conduct of Enquiry
Abstract: This chapter recapitulates the conduct of enquiry of the PISA 2006 Scientific
Literacy Study undertaken in Macao from 21 April 2006 to 31 May 2006. It comprises
six sections: (1) Brief introduction; (2) Sample design; (3) Scientific literacy assessment
framework; (4) An example of scientific literacy item unit; (5) Description of
proficiency levels of scientific, mathematical and reading literacy scales; (6) Examples
of scientific attitudes assessed as outcomes of science learning.
1.1 Brief introduction
The PISA 2006 Study assessed 15-year-old students’literacy in three key subject areas:
(1) science, (2) mathematics, and (3) reading. In this third round of international
assessment, scientific literacy took its turn to be the main focus of international
assessment, whereas mathematical and reading literacy were assessed to a minor extent.
This assessment design allows researchers to chart changes since PISA 2000 Reading
Literacy Study and PISA 2003 Mathematical Literacy Study, in both studies scientific
literacy were likewise assessed to a minor extent. Literacy refers to the capacity of
students to apply knowledge and skills in key subject areas and to reason and
communicate effectively as they pose and solve problems in a variety of situations.
PISA 2006 Scientific Literacy Study sought to chart a profile of knowledge and skills,
i.e. a detailed profile of literacy for science, and an update for mathematics and reading.
For science, the emphasis was on the mastery of processes (i.e. identifying scientific
issues, explaining phenomena scientifically and using scientific evidence), the
understanding of concepts (i.e. knowledge of science such as knowledge of physical
systems, living systems, earth and space systems and technology systems; as well as
knowledge about science such as knowledge about science as scientific enquiry and
scientific explanations), and the ability to function in various situations (i.e. in personal,
social and global settings). In addition, how students respond to scientific issues (i.e.
interest in science, support for scientific enquiry and responsibility towards resources
and environments) were also examined.
- 11 -
Approximately 400,000 students were randomly sampled to participate in the PISA
2006 Scientific Literacy Study. The achieved sample represented about 32 millions
15-year-olds in the schools of the 57 participating countries/economies, of which 30
were OECD member countries and 27 were partner countries/economies. This breadth
of geographical coverage represented close to 90% of the world economy in 2006.
1.2 Sample design
Table 1 presents characteristics of the schools and the 15-year-olds sampled and tested
in the PISA 2006 Study, broken down by school type, study program, and language of
instruction.
Table 1Characteristics of schools in the PISA 2006 Macao sample
Stratifying Variable
Numberof
schoolsin
Macao
Numberof
studentsin
Macao
Numberof
schoolssampled
Numberof
schoolstested
Numberof
studentssampled
Numberof
studentstested
School Type
1. Government 4 266 2 2 247 2302. Private-In-Net 32 5,386 32 32 3,864 3,7343. Private 9 996 9 9 818 796
Study Program
1. Grammar-International 41 6,346 40 40 4,656 4,510
2.Technical-Prevocational 4 302 3 3 273 250
Language of Instruction
1. Chinese 33 5,494 31 31 3,908 3,7632. English 7 398 7 7 396 3913. Portuguese 1 42 1 1 46 454. Chinese & English 3 601 3 3 432 418
5. Chinese & Portuguese 1 113 1 1 147 143
Total 45 6,648 43 43 4,929 4,760
Note 1: All sampled schools offered basic education courses to 15-year-olds. Two government schools were excludedfrom the designed school sample, one offered senior secondary vocational education only to a few students. The otherwas a dancing school offering performing arts education.Note 2: Sampled students were all 15-year-olds born in 1990.
Table 2 presents the number of students (males/females) sampled and tested in the PISA
2006 Scientific Literacy Study. The response rates are very satisfactory, showing that
- 12 -
the achieved sample is highly representative of the Macao 15-year-old student
population.
Table 2
Number of 15-year-olds sampled and tested in Macao
Macao Sample
Number of 15-year-old students sampled4,929
(2,426 males and 2,503 females)
Number of 15-year-old students tested4,760
(2,320 males and 2,440 females)
Response rate (%) 96.6
Table 3 presents the grade distribution of Macao’s 15-year-old students tested in the
PISA 2006 Scientific Literacy Study. Despite 33.4% and 36.5% of the Macao sample
were studying at grade 9 and 10 respectively, there were 8.2% and 21.2% of sampled
students studying at grade 7 and 8 respectively. It is noteworthy that these students were
likely repeaters, new immigrants, or students receiving inclusive education. A sampled
school’s literacy performance is expected to be adversely affected when the proportions
of students studying at the lower grades (i.e. grade 7 and 8) are significantly higher than
the corresponding figures in the Macao sample (see Figure 2 and 3 in Section 2.1 for the
distribution literacy proficiency levels across grades).
Table 3
Grade distribution of 15-year-olds tested
Macao SampleGrade
N %
7 391 8.2
8 1,008 21.2
9 1,591 33.4
10 1,738 36.5
11 32 0.7
Total 4,760 100.0
- 13 -
1.3 Scientific literacy assessment framework
Unlike most school tests and examinations, PISA assessment is not curriculum-based.
PISA assessment items have to relate to life situations (i.e. health, natural resources,
environment, hazards, frontiers of science and technology) within contexts (i.e. personal,
social and global) that 15-year-olds are likely to encounter in daily life. Table 4 shows
the contexts for the PISA 2006 science assessment (OECD, 2007, p.36).
Table 4Contexts for the PISA 2006 science assessment
Contexts
Life
SituationsPersonal
(Self, family and peergroups)
Social
(The community)
Global
(Life across the world)
Health Maintenance of health,accidents, nutrition
Control of disease,social transmission,food choices,community health
Epidemics, spread ofinfectious diseases
NaturalResources
Personal consumptionof materials and energy
Maintenance of humanpopulations, quality oflife, security,production anddistribution of food,energy supply
Renewable andnon-renewable, naturalsystems, populationgrowth, sustainable useof species
Environment Environmentallyfriendly behavior, useand disposal ofmaterials
Population distribution,disposal of waste,environmental impact,local weather
Biodiversity, ecologicalsustainability, controlof pollution, productionand loss of soil
Hazards Natural andhuman-induced,decisions abouthousing
Rapid changes(earthquakes, severeweather), slow andprogressive changes(coastal erosion,sedimentation), riskassessment
Climate change, impactof modern warfare
Frontiers ofscience andtechnology
Interests in science’sexplanations of naturalphenomena,science-based hobbies,sport and leisure, musicand personaltechnology
New materials, devicesand processes, geneticmodification, weapontechnology, transport
Extinction of species,exploration of space,origin and structure ofthe universe
- 14 -
In PISA 2006 Study, scientific literacy refers to an individual’s:
Scientific knowledge and use of that knowledge to identify questions, acquire new
knowledge, explain scientific phenomena and draw evidence-based conclusions
about science-related issues.
Understanding of characteristic features of science as a form of human knowledge
and enquiry.
Awareness of how science and technology shape our material, intellectual, and
cultural environments.
Willingness to engage in science-related issues and with the ideas of science, as a
reflective citizen. (OECD, 2007, p.34-35)
In the scientific literacy assessment framework (OECD, 2007, p.35) shown in Figure 1,
context refers to engagement with science in a variety of life situations not limited to
life in school (i.e. personal, social and global situations within application areas such as
health, natural resources, environment, hazards, and frontiers of science and technology).
These applications require students to demonstrate three main competencies (i.e. identify
scientific issues, explain phenomena, and use scientific evidence), and how they do so is
influenced by their knowledge (i.e. knowledge of science and knowledge about science)
and attitudes (i.e. interest in science, support for scientific enquiry, and responsibility
toward resources and environments). Knowledge of science is classified in four
categories: (1) physical systems, (2) living systems, (3) earth and space systems, and (4)
technology systems, whereas knowledge about science in two categories: (1) scientific
enquiry, and (2) scientific explanations. Based on the test construction blueprint shown
in Table 5, there is a balance of items mapped to the various components of the
scientific literacy assessment framework.
- 15 -
Figure 1
Framework for PISA 2006 science assessment
1.4 An example of scientific literacy item unit
Table 5 serves as a blueprint when scientific literacy item units are designed (OECD,
2007, p.46). Using ACID RAIN as an example, this unit composes of three items, Q2,
Q3 and Q5 (OECD, 2007, p.104-107). All load on the three scientific competencies
respectively (see item unit below). In particular, Q2and Q3 is classified as knowledge of
science, whereas Q5 is on scientific enquiry and therefore classified as knowledge about
science. Embedded in the test item unit are attitude items probing into students’interests
as well as their attitudinal support for scientific enquiry in the science topics under
assessment.
Competencies
Knowledge
Attitudes
Require people to
Context
Identify scientific issues
Explain phenomenascientifically
Use scientific evidence
b) How they respond to science issues (interest,support for scientific enquiry, responsibility).
How they do so is influenced by
a) What they know:
about the natural world andtechnology (knowledge of science);
about science itself (knowledge aboutscience).
Life situations thatinvolve science andtechnology
- 16 -
Table 5Blueprint used in the design of scientific literacy assessment item units
(Example item unit: ACID RAIN)
Competencies
Identifyingscientific
issues
Explainingphenomena
scientifically
Usingscientificevidence
PhysicalSystems
Q2 Q3
LivingSystems
Earth andSpaceSystems
Knowledgeof science
TechnologySystems
ScientificEnquiry
Q5
Knowledge
Knowledgeabout science
ScientificExplanation
Interests in Science See section1.6.1.1 for an example (Q10N)embedded in ACID RAIN
Attitudes
Support for ScientificEnquiry
See section 1.6.1.2 for an example (Q10S)embedded in ACRD RAIN
1.4.1 Sample scientific literacy test items – Acid Rain
ACID RAIN
Below is a photo of statues called Caryatids that were built on the Acropolis in Athens
more than 2500 years ago. The statues are made of a type of rock called marble.
Marble is composed of calcium carbonate.
In 1980, the original statues were transferred inside the museum of the Acropolis and
were replaced by replicas. The original statues were being eaten away by acid rain.
- 17 -
Question 2: ACID RAIN S485Q02 – 0129
Normal rain is slightly acidic because it has absorbed some carbon dioxide from the air.
Acid rain is more acidic than normal rain because it has absorbed gases like sulphur
oxides and nitrogen oxides as well.
Where do these sulphur oxides and nitrogen oxides in the air come from?
… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …
… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …
The effect of acid rain on marble can be modeled by placing chips of marble in vinegar
overnight. Vinegar and acid rain have about the same acidity level. When a marble chip
is placed in vinegar, bubbles of gas form. The mass of the dry marble chip can be found
before and after the experiment.
- 18 -
Question 3: ACID RAIN S485Q03
A marble chip has a mass of 2.0 grams before being immersed in vinegar overnight. The
chip is removed and dried the next day. What will the mass of the dried marble chip be?
A Less than 2.0 grams
B Exactly 2.0 grams
C Between 2.0 and 2.4 grams
D More than 2.4 grams
Question 5: ACID RAIN S485Q05 – 0129
Students who did this experiment also placed marble chips in pure (distilled) water
overnight.
Explain why the students include this step in their experiment.
… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …
… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …
- 19 -
1.5 Description of proficiency levels of scientific, mathematical andreading literacy scales
In the PISA 2006 Study, students are required to select or construct their answers when
responding to the assessment items. Items were typically organized in units based on a
written passage or graphic, commonly of the kind students may encounter in real life
(see illustrative example ACID RAIN). Each student was randomly assigned one of
thirteen test booklets containing assessment items that take approximately 120 minutes
to complete. Table 6-8 presents description of the proficiency levels of the literacy
assessment scales, i.e. scientific, mathematical, and reading literacy. In all PISA literacy
scales that serve as a benchmark for future studies, the scales are calibrated against the
thirty OECD countries, with mean score set at 500 and standard deviation at 100 for
easy referencing and interpretation.
Table 6Description of the six scientific literacy proficiency levels (OECD, 2007, p.43)
Level What students can typically do at each level?
6 At Level 6, students can consistently identify, explain and apply scientific knowledge and knowledgeabout science in a variety of complex life situations. They can link different information sources andexplanations and use evidence from those sources to justify decisions. They clearly and consistentlydemonstrate advanced scientific thinking and reasoning, and they are willing to use their scientificunderstanding in support of solutions to unfamiliar scientific and technological situations. Students atthis level can use scientific knowledge and develop arguments in support of recommendations anddecisions that centre on personal, social, or global situations.
5 At Level 5, students can identify the scientific components of many complex life situations, apply bothscientific concepts and knowledge about science to these situations, and can compare, select andevaluate appropriate scientific evidence for responding to life situations. Students at this level can usewell-developed inquiry abilities, link knowledge appropriately and bring critical insights to situations.They can construct explanations based on evidence and arguments based on their critical analysis.
4 At Level 4, students can work effectively with situations and issues that may involve explicitphenomena requiring them to make inferences about the role of science or technology. They can selectand integrate explanations from different disciplines of science or technology and link thoseexplanations directly to aspects of life situations. Students at this level can reflect on their actions andthey can communicate decisions using scientific knowledge and evidence.
3 At Level 3, students can identify clearly described scientific issues in a range of contexts. They canselect facts and knowledge to explain phenomena and apply simple models or inquiry strategies.Students at this level can interpret and use scientific concepts from different disciplines and can applythem directly. They can develop short statements using facts and make decisions based on scientificknowledge.
2 At Level 2, students have adequate scientific knowledge to provide possible explanations in familiarcontexts or draw conclusions based on simple investigations. They are capable of direct reasoning andmaking literal interpretations of the results of scientific inquiry or technological problem solving.
1 At Level 1, students have such a limited scientific knowledge that it can only be applied to a fewfamiliar situations. They can present scientific explanations that are obvious and follow explicitly fromgiven evidence.
Note: There is an additional “below 1” level for those students who cannot attain at the minimum level.
- 20 -
Table 7Description of the six mathematical literacy proficiency levels (OECD, 2007, p.312)
Level What students can typically do at each level?
6 At Level 6 students can conceptualize, generalize, and utilize information based on theirinvestigations and modeling of complex problem situations. They can link differentinformation sources and representations and flexibly translate among them. Students at thislevel are capable of advanced mathematical thinking and reasoning. These students can applythis insight and understandings along with a mastery of symbolic and formal mathematicaloperations and relationships to develop new approaches and strategies for attacking novelsituations. Student at this level can formulate and precisely communicate their actions andreflections regarding their findings, interpretations, arguments, and the appropriateness ofthese to the original situations.
5 At Level 5 students can develop and work with models for complex situations, identifyingconstraints and specifying assumptions. They can select, compare, and evaluate appropriateproblem solving strategies for dealing with complex problems related to these models.Students at this level can work strategically using broad, well-developed thinking andreasoning skills, appropriate linked representations, symbolic and formal characterizations,and insight pertaining to these situations. They can reflect on their actions and formulate andcommunicate their interpretations and reasoning.
4 At Level 4 students can work effectively with explicit models for complex concretesituations that may involve constraints or call for making assumptions. They can select andintegrate different representations, including symbolic, linking them directly to aspects ofreal-world situations. Students at this level can utilize well-developed skills and reasonflexibly, with some insight, in these contexts. They can construct and communicateexplanations and arguments based on their interpretations, arguments, and actions.
3 At Level 3 students can execute clearly described procedures, including those that requiresequential decisions. They can select and apply simple problem solving strategies. Studentsat this level can interpret and use representations based on different information sources andreason directly from them. They can develop short communications reporting theirinterpretations, results and reasoning.
2 At Level 2 students can interpret and recognize situations in contexts that require no morethan direct inference. They can extract relevant information from a single source and makeuse of a single representational mode. Students at this level can employ basic algorithms,formulae, procedures, or conventions. They are capable of direct reasoning and makingliteral interpretations of the results.
1 At Level 1 students can answer questions involving familiar contexts where all relevantinformation is present and the questions are clearly defined. They are able to identifyinformation and to carry out routine procedures according to direct instructions in explicitsituations. They can perform actions that are obvious and follow immediately from the givenstimuli.
Note: There is an additional “below 1” level for those students who cannot attain at the minimum level.
- 21 -
Table 8Description of the five reading literacy proficiency levels (OECD, 2007, p.292-293)
Level What students can typically do at each level?
5 Locate and possibly sequence or combine multiple pieces of deeply embedded information, some ofwhich may be outside the main body of the text. Infer which information in the text is relevant to thetask. Deal with highly plausible and/or extensive competing information. Either construe the meaningof nuanced language or demonstrate a full and detailed understanding of a text. Critically evaluate orhypothesize, drawing on specialized knowledge. Deal with concepts that are contrary to expectationsand draw on a deep understanding of long or complex texts. In continuous texts students can analyzetexts whose discourse structure is not obvious or clearly marked, in order to discern the relationship ofspecific parts of the text to its implicit theme or intention. In non-continuous texts, students can identifypatterns among many pieces of information presented in a display which may be long and detailed,sometimes by referring to information external to the display. The reader may need to realizeindependently that a full understanding of the section of text requires reference to a separate part of thesame document, such as a footnote.
4 Locate and possibly sequence or combine multiple pieces of embedded information, each of which mayneed to meet multiple criteria, in a text with familiar context or form. Infer which information in thetext is relevant to the task. Use a high level of text-based inference to understand and apply categoriesin an unfamiliar context, and to construe the meaning of a section of text by taking into account the textas a whole. Deal with ambiguities, ideas that are contrary to expectation and ideas that are negativelyworded. Use formal or public knowledge to hypothesize about or critically evaluate a text. Showaccurate understanding of long or complex texts. In continuous texts, students can follow linguistic orthematic links over several paragraphs, often in the absence of clear discourse markers, in order tolocate, interpret or evaluate embedded information or to infer psychological or metaphysical meaning.In non-continuous texts students can scan a long, detailed text in order to find relevant information,often with little or no assistance from organizers such as labels or special formatting, to locate severalpieces of information to be compared or combined.
3 Locate, and in some cases recognize the relationship between pieces of information, each of which mayneed to meet multiple criteria. Deal with prominent competing information. Integrate several parts of atext in order to identify a main idea, understand a relationship or construe the meaning of a word orphrase. Compare, contrast or categorize taking many criteria into account. Deal with competinginformation. Make connections or comparisons, give explanations, or evaluate a feature of text.Demonstrate a detailed understanding of the text in relation to familiar, everyday knowledge, or drawon less common knowledge. In continuous texts students use conventions of text organization, wherepresent, and follow implicit or explicit logical links such as cause and effect relationships acrosssentences or paragraphs in order to locate, interpret or evaluate information. In non-continuous textsstudents can consider one display in the light of a second, separate document or display, possibly in adifferent format, or combine several pieces of spatial, verbal and numeric information in a graph ormap to draw conclusions about the information represented.
2 Locate one or more pieces of information, each of which may be required to meet multiple criteria.Deal with competing information. Identify the main idea in a text, understand relationships, form orapply simple categories, or construe meaning within a limited part of the text when the information isnot prominent and low-level inferences are required. Make a comparison or connections between thetext and outside knowledge, or explain a feature of the text by drawing on personal experience andattitudes. In continuous texts students can follow logical and linguistic connections within a paragraphin order to locate or interpret information; or synthesize information across texts or parts of a text inorder to infer the author’s purpose. In non-continuous texts students can demonstrate a grasp of theunderlying structure of a visual display such as a simple tree diagram or table, or combine two pieces ofinformation from a graph or table.
1 Locate one or more independent pieces of explicitly stated information, typically meeting a singlecriterion, with little or no competing information in the text. Recognize the main theme or author'spurpose in a text about a familiar topic, when the required information in the text is prominent. Make asimple connection between information in the text and common, everyday knowledge. In continuoustexts students can use redundancy, paragraph headings or common print conventions to form animpression of the main idea of the text, or to locate information stated explicitly within a short sectionof text. In non-continuous texts students can focus on discrete pieces of information, usually within asingle display such as a simple map, a line graph or a bar graph that presents only a small amount ofinformation in a straightforward way and in which most of the verbal text is limited to a small numberof words or phrases.
Note: There is an additional “below 1” level for those students who cannot attain at the minimum level.
- 22 -
1.6 Examples of scientific attitudes assessed as outcomes of science
learning
One prominent feature of the scientific literacy assessment framework is that not only
cognitive outcomes of science learning but also affective outcomes are also included.
The three attitudes assessed are: (1) Interest in science; (2) Support for scientific
enquiry; and (3) Responsibility towards resources and environments. The first two
attitudes are embedded in the test item units and therefore they are content-specific and
composed in the same way as the scientific literacy measures. Again, using ACID RAIN
as an example, the attitude scale statements are shown below for illustration. The third
attitude scale is a standard Likert scale commonly used in educational research. Seven
attitude statements have been validated with adequate psychometric properties.
1.6.1 Sample attitude items
1.6.1.1 Interest in science scale
Each student responded to one of the thirteen test booklets. In each booklet, some
science test units contain a set of attitude statements asking students to rate their
interests in the scientific topics suggested (see OECD, 2007, p.107, for an example from
the released item ACID RAIN shown below).
ACID RAIN (Q10N) S485Q10N
How much interest do you have in the following statements?
Tick only one box in each row.
High
Interest
Medium
Interest
Low
Interest
No
Interest
a) Knowing which human activities contributemost to acid rain 1 2 3 4
b) Learning about technologies that minimise theemission of gases that cause acid rain 1 2 3 4
c) Understanding the methods used to repairbuildings damaged by acid rain 1 2 3 4
- 23 -
1.6.1.2 Support for scientific enquiry scale
Each student responded to one of the thirteen test booklets. In each booklet, some
science test units contain a set of attitude statements asking students to rate their support
for scientific enquiry in the topics suggested (see OECD, 2007, p.107, for an example
from the released item ACID RAIN shown below).
ACID RAIN (Q10S) S485Q10S
How much do you agree with the following statements?
Tick only one box in each row.
StronglyAgree Agree Disagree
StronglyDisagree
a) Preservation of ancient ruins should be basedon scientific evidence concerning the causes ofdamage.
1 2 3 4
b) Statements about the causes of acid rain shouldbe based on scientific research. 1 2 3 4
1.6.1.3 Responsibility towards resources and environments
Contrary to the previous two attitude scales, all students responded to a questionnaire
and are asked to rate the following statements related to sustainable environmental
development, which is one key component of responsibility towards resources and
environment:
1. Industries should be required to prove that they safely dispose of dangerous
waste materials.
2. I am in favor of having laws that protect the habitats of engendered species.
3. It is important to carry out regular checks on the emissions from cars as a
condition of their use.
4. To reduce waste, the use of plastic packaging should be kept to a minimum.
5. Electricity should be produced from renewable sources as much as possible,
even if this increases the cost.
6. It disturbs me when energy is wasted through the unnecessary use of electrical
appliances.
7. I am in favor of having laws that regulate factory emissions even if this would
increase the price of products.
- 24 -
There are three other components of responsibility towards resources and environments,
namely, students’awareness of environmental issues, optimism regarding environmental
issues, and concern for environmental issues. These three will be examined and reported
in chapter 3.
- 25 -
Chapter 2A Profile of Literacy Performance for 15-year-olds in Macao
Abstract: This chapter recapitulates the key results, particularly those pertaining to
Macao, reported in the PISA 2006 Study International Report (OECD, 2007). It detailed
the profiles of student performance broken down by gender in science, mathematics and
reading. Apart from cognitive outcomes, affective outcomes of science education are
also documented to allow readers to evaluate strengths and weaknesses of educational
provisions in the three key domains of study (i.e. science, mathematics and reading) in
Macao. This chapter comprises five sections: (1) Macao 15-year-olds’ literacy
performance; (2) Macao 15-year-olds’ scientific literacy performance compared with
other participating countries/economies; (3) Macao 15-year-olds’ scientific attitude
results; (4) Macao 15-year-olds’attitude results compared with other OECD countries;
(5) Gender differences in outcomes of science learning.
2.1 Macao 15-year-olds’literacy performance
Table 9 presents Macao 15-year-olds’performance results in the three assessment area,
i.e. scientific, mathematical, and reading literacy, broken down by gender.
Table 9Macao 15-year-olds’literacy performance results
Scientific LiteracyDescriptive
Statistics CombinedScience
IdentifyingScientific
Issues
ExplainingPhenomena
Scientifically
UsingScientificEvidences
MathematicalLiteracy
ReadingLiteracy
Total = 4,760
Mean 510.8 490.0 520.0 511.5 525.0 492.3
StandardDeviation
78.2 78.8 82.7 84.4 84.3 76.6
Males = 2,320
Mean 512.7 482.5 526.9 511.7 530.3 479.4
StandardDeviation
82.2 82.1 86.4 88.7 87.7 80.3
Females = 2,440
Mean 508.9 497.7 512.9 511.4 519.6 505.5
StandardDeviation
73.9 74.6 78.0 79.7 80.4 70.2
Note1: Sample scientific literacy items are shown in chapter 1.Note2: Scientific literacy as measured by the combined scale is composed of three subscales, namely: (1)Identifying scientific issues; (2) Explaining phenomena scientifically; and (3) Using scientific evidences.
- 26 -
As seen in Table 9, Macao’s 15-year-olds performed very well in mathematics, quite
well in science, and only fair in reading. In mathematics, males outperform females,
whereas it is the other way round in reading. In science, males outperform females in
explaining phenomena scientifically, whereas females outperform males in identifying
scientific issues. In using scientific evidences, males and females perform equally well.
All standard deviations are low showing that students’ performances on the three
literacy scales and subscales are quite homogenous as compared with the average of the
thirty OECD countries, the standard deviation of which is set at 100.
Each student was assigned to the highest proficiency level for which he or she would be
expected to answer correctly the majority of the assessment items. Students classified as
below level 1 were unable to demonstrate competency in situations required by the
easiest PISA tasks, and therefore they were regarded as at a disadvantage for full
participation in society and economy. Table 10 presents frequency distributions of
Macao 15-year-olds’proficiency levels on scientific, mathematical, and reading literacy
scales, broken down by gender.
- 27 -
Table 10Distribution of Macao 15-year-olds’proficiency levels on the literacy scales
% of Students
Scientific LiteracyProficiency
Level
CombinedScience
IdentifyingScientific
Issues
ExplainingPhenomena
Scientifically
UsingScientificEvidence
MathematicalLiteracy
ReadingLiteracy
Total Sample = 4,760
6 0.3 0.1 0.8 0.5 3.8 -
5 5.1 2.7 7.5 6.4 13.6 3.0
4 22.9 17.3 25.1 23.2 24.4 18.5
3 35.7 34.0 33.9 33.6 27.3 36.6
2 25.9 30.3 23.3 24.7 20.0 28.9
1 8.8 12.9 7.9 9.3 8.3 10.1
Below 1 1.4 2.7 1.5 2.3 2.6 2.9
Males = 2,320
6 0.3 0.1 1.1 0.6 4.8 -
5 6.3 2.4 9.3 7.5 15.8 2.4
4 23.6 16.5 27.5 23.3 24.3 15.3
3 34.4 31.3 31.9 31.8 25.8 34.1
2 24.1 30.7 21.0 24.1 18.5 30.4
1 9.4 15.0 7.5 9.7 8.0 13.2
Below 1 1.8 3.9 1.7 3.0 2.9 4.5
Females = 2,440
6 0.2 0.1 0.4 0.4 2.7 -
5 3.8 3.0 5.7 5.4 11.5 3.7
4 22.2 18.1 22.6 23.0 24.5 21.8
3 37.0 36.8 36.0 35.4 28.8 39.1
2 27.7 29.8 25.7 25.2 21.5 27.4
1 8.0 10.8 8.3 8.9 8.7 6.8
Below 1 1.0 1.5 1.3 1.6 2.4 1.3
Note1: Description of the scientific, mathematical and reading proficiency levels are shown in chapter 1.Note2: Contrary to six proficiency levels calibrated in the mathematical literacy and scientific literacyscales, there are only five proficiency levels calibrated in the reading literacy scale.
As seen in Table 10, students’proficiency levels are mainly concentrated at levels 2, 3
and 4. Percentages of students with proficiency level below 1 for the three literacy
- 28 -
scales remain low (less than 5% of the Macao sample), showing that the number of
low-performing students who cannot function productively in society is small.
Unfortunately, the number of high-performing students with proficiency level above 5 is
not high and this is so for all the three literacy scales. Figure 2 and 3 further show the
relationships between scientific proficiency levels attained and grade levels enrolled by
the 15-year-olds in the Macao sample.
Figure 2
Percentage of 15-year-olds at different scientific literacy proficiency levels acrossgrades in the Macao sample
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
45.0
Grade 7 Grade 8 Grade 9 Grade 10 Grade 11
Below level 1
Level 1
Level 2
Level 3
Level 4
Level 5
Level 6
Figure 3
Percentage of 15-year-olds at different grade levels across scientific literacy proficiencylevels in the Macao sample
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0
45.0
Below level 1 Level 1 Level 2 Level 3 Level 4 Level 5 Level 6
Grade 7
Grade 8
Grade 9
Grade 10
Grade 11
- 29 -
2.2 Macao 15-year-olds’scientific literacy performance compared with
other participating countries/economies
Compared with other participating countries/economies, the following five
achievements deserve special announcement to the Macao educational community:
1. Macao ranks between 24 and 29 amongst the 57 participating
countries/economies on the identifying scientific issues scale. Country means
range from 321 to 555 on this scale, and Macao’s performance (mean = 490;
SE = 1.2) is statistically significantly below the OECD average. When ranked
in descending order of percentage of 15-year-olds in Levels 2, 3 4, 5 and 6,
the ranking rises to 17. Except five amongst the 57 countries/economies,
females outperform males statistically significantly on this scale. (OECD,
2007, p.66, p.69 & p.79)
2. Macao ranks between 9 and 15 amongst the 57 participating
countries/economies on the explaining phenomena scientifically scale.
Country means range from 334 to 566 on this scale, and Macao’s performance
(mean = 520; SE = 1.2) is statistically significantly above the OECD average.
When ranked in descending order of percentage of 15-year-olds in Levels 2, 3,
4, 5 and 6, the ranking rises to 4. Except sixteen amongst the 57
countries/economies, males outperform females statistically significantly on
this scale. (OECD, 2007, p.67, p.70 & p.88)
3. Macao ranks between 15 and 19 amongst the 57 participating
countries/economies on the using scientific evidence scale. Country means
range from 288 to 567 on this scale, and Macao’s performance (mean = 512;
SE = 1.2) is statistically significantly above the OECD average. When ranked
in descending order of percentage of 15-year-olds in Levels 2, 3, 4, 5 and 6,
the ranking rises to 6. Amongst all countries/economies, there are more cases
such that females outperform males statistically significantly on this scale.
(OECD, 2007, p.68, p.70 & p.103)
4. Macao ranks between 15 and 20 amongst the 57 participating
countries/economies on the combined science scale. Country means range
- 30 -
from 322 to 563 on this scale, and Macao’s performance (mean = 511; SE =
1.1) is statistically significantly above the OECD average. When ranked in
descending order of percentage of 15-year-olds in Levels 2, 3, 4, 5 and 6 the
ranking rises to 5. (OECD, 2007, p.49, p.56-58)
5. As evidenced in Table 10, students’ scientific proficiency levels are mainly
concentrated at middle levels 2, 3 and 4. Compared with the average
percentage or total number of sampled students at each of the six proficiency
levels of all participating countries/economies, there are lesser students at
either ends of the proficiency continuum, i.e. at levels below 1 and 1, as well
as at levels 5 and 6. This shows that Macao is successful to produce less
low-performing students to function productively in the scientific world, but
at the same time not so successful in producing students of higher caliber in
science identifying scientific issues, explaining phenomena scientifically, and
using scientific evidences.
If we analyze the Macao PISA data in greater depth across the various scientific
domains of study, the following results may be revealed:
1. Comparing the literacy performance difference between the combined science
scale (score = 511) and each of the three competency subscales, Macao’s 15
year-olds performed best in explaining phenomena scientifically (9.2 score
points higher), comparable in using scientific evidences (0.7 score points higher),
and not so well in identifying scientific issues (20.8 score points lower).
2. Comparing the literacy performance difference between the combined science
scale (score = 511) and each of the three knowledge of science subscales,
Macao’s 15 year-olds performed best in living systems (14.2 score points higher)
and quite good in physical systems (6.7 score points higher), and not so well in
earth and space systems (4.9 score points lower). In spite of this, knowledge of
science in the three key domains of study on average is still better than
knowledge about science (5.9 score points lower than the combined science
scale).
- 31 -
2.3 Macao 15-year-olds’scientific attitude results
As seen in Table 11, Macao 15-year-olds’scientific attitudes are very favorable. There is
a slight tendency for males to exhibit more favorable attitude than females in interest in
science and support for scientific enquiry, and for females to exhibit more favorable
attitude than males in responsibility towards sustainable development.
Table 11Macao 15-year-olds’scientific attitude results
Scientific Attitudes
DescriptiveStatistics
Interest in ScienceSupport for Scientific
Enquiry
Responsibility forSustainable
Development
Total Sample = 4,760
Mean 523.6 520.6 0.359
Standard Deviation 93.6 84.7 0.842
Males = 2,320
Mean 527.4 523.7 0.329
Standard Deviation 98.3 87.5 0.880
Females = 2,440
Mean 519.7 517.3 0.389
Standard Deviation 88.4 81.5 0.801
Note 1: Sample scientific attitude items are shown in Chapter 1.Note 2: Interest in science and support for scientific enquiry are two attitude scales with statementsembedded within scientific literacy test units, and therefore scale measures are constructed in the sameway as the combined science literacy scale. Responsibility for sustainable development is an attitude scalewith scale measure standardized across all the thirty OECD countries with mean equals to zero andstandard deviation equals to one.
- 32 -
2.4 Macao 15-year-olds’attitude results compared with OECD
countries
Compared with other OECD countries, percentages of students who agree or strongly
agree with the seven statements of responsibility for sustainable development scale are
summarized below:
Table 12
A comparison of 15-year-olds’responses to responsibility for sustainable development
between Macao and OECD countries
% of students whoagree or strongly agree
Attitude statements
Macao OECD
1. Industries should be required to prove that theysafely dispose of dangerous waste materials
98% 92%
2. I am in favor of having laws that protect thehabitats of engendered species
88% 92%
3. It is important to carry out regular checks on theemissions from cars as a condition of their use
98% 91%
4. To reduce waste, the use of plastic packagingshould be kept to a minimum
97% 82%
5. Electricity should be produced from renewablesources as much as possible, even if thisincreases the cost
97% 79%
6. It disturbs me when energy is wasted through theunnecessary use of electrical appliances
81% 69%
7. I am in favor of having laws that regulate factoryemissions even if this would increase the price ofproducts
82% 70%
Amongst the seven attitude statements in the responsibility for sustainable development
scale all except the one on laws that protect the habitats of engendered species Macao
15-year-olds’favorable responses are higher in percentage than that of the averages of
the thirty OECD countries.
- 33 -
It is beyond the scope of this report to provide comprehensive comparison results for the
other two attitude scales, i.e. interest in science and support for scientific enquiry. This
is because these attitude items are embedded in the test item units which mostly are kept
confidential for future rounds of international assessment.
2.5 Gender differences in outcomes of science learning
PISA 2006 Study provides educational researchers opportunities to rethink the
long-standing issue of gender differences in reading, mathematics and science from the
comparative education perspective.
Firstly, based on the afore-mentioned assessment results, it is evidenced that females
tend to outperform males to an appreciable extent in reading literacy, and males have a
greater chance to outperform females in mathematical literacy. However, gender
difference results across the three competences of scientific literacy are mixed,
stimulating educational researchers to think that preferred contexts of learning and
differential brain usage between the two sexes are at issue to uncover the secrets of
gender differences of literacy performance in the three key domains of school study.
Secondly, the patterns of within-school gender differences across the 43 sampled
schools in Macao are different. Figure 4-6 present some findings to illustrate the diverse
patterns existed in three sampled schools in Macao.
- 34 -
Figure 4
Literacy performance results by gender – Sampled school 1
200
250
300
350
400
450
500
550
600
650
Combined Science Identifying
Scientific Issues
Explaining
Phenomena
Scientifically
Using Scientific
Evidence
Mathematical
Literacy
Reading Literacy
Macao Sample Males Macao Sample Females
The Sampled School Males The Sampled School Females
As shown in Figure 4, pattern of gender differences in sampled school 1 across the
various literacy scales and subscales are similar to that of the Macao sample. Quite a
number of schools in the Macao sample are of this kind of pattern.
- 35 -
Figure 5
Literacy performance results by gender – Sampled school 2
200
250
300
350
400
450
500
550
600
650
Combined Science Identifying
Scientific Issues
Explaining
Phenomena
Scientifically
Using Scientific
Evidence
Mathematical
Literacy
Reading Literacy
Macao Sample Males Macao Sample Females
The Sampled School Males The Sampled School Females
As shown in Figure 5, pattern of gender differences in sampled school 2 across the
various literacy scales and subscales are not similar to that of the Macao sample. Except
reading literacy, males clearly outperform females to an appreciable extent on
mathematical literacy, as well as all subscales of the scientific literacy. Unlike the
Macao sample, females in this school do not outperform males in reading literacy. This
pattern of gender differences can be observed in a number of high-performing
co-educational schools in the Macao sample.
- 36 -
Figure 6
Literacy performance results by gender – Sampled school 3
200
250
300
350
400
450
500
550
600
650
Combined Science Identifying
Scientific Issues
Explaining
Phenomena
Scientifically
Using Scientific
Evidence
Mathematical
Literacy
Reading Literacy
Macao Sample Males Macao Sample Females
The Sampled School Males The Sampled School Females
As shown in Figure 6, pattern of gender differences in sampled school 3 across the
various literacy scales and subscales are not similar to that of the Macao sample. On all
literacy scales and subscales, males do not outperform females. This pattern of gender
differences can be observed in a number of low-performing co-educational schools in
the Macao sample.
In the light of the diverse patterns of gender differences across the scientific,
mathematical and reading literacy scales, multi-level analyses that take learning
contexts into account must be conducted for an adequate solution of this long-standing
issue in psychology of learning from the comparative education perspective.
- 37 -
Chapter 3
Quality Science Education Indicators
Abstract: This chapter seeks to search for effective quality science education indicators
amongst the scaled measures available in the PISA 2006 Study database so as to throw
lights on the design of intervention programs to elevate students’scientific literacy for
productive life-long learning. Apart from the three attitudinal outcomes analyzed in
chapter 2, out of twelve scaled measures constructed from student responses to
questions asked in the student questionnaire, ten were validated to have potential to
serve as quality science education indicators appropriate for use in Macao schooling
contexts.
3.1 Macao 15-year-olds’quality science education indicators compared
with OECD countries
In the PISA 2006 Study, students answered a questionnaire that took about 30 minutes
to complete. The questionnaire focuses on their personal background, motivation and
engagement, learning habits and perception of the living environments. Quality
indicators may be constructed to reflect dispositions and conditions facilitative of
science education in Macao in general and the individual sampled schools in particular.
There are altogether twelve quality indicators available for examination of their
appropriateness and they are described below:
- 38 -
3.1.1 General interest in science scale
There are altogether eight science study areas in this scale, the details of which are
shown in Table 13.
Table 13A comparison of 15-year-olds’responses to
general interest in science between Macao and OECD countries
% of students who showhigh interest or medium Interest
Science Study Areas
Macao OECD
1. Human biology 73% 68%
2. Topics in astronomy 58% 53%
3. Topics in chemistry 47% 50%
4. Topics in physics 49% 49%
5. The biology of plants 55% 47%
6. Ways scientists design experiments 53% 46%
7. Topics in geology 35% 41%
8. What is required for scientific explanations 37% 36%
The reported percentages for the above eight general interests in science, all except the
two topics on topics in chemistry and geology, are all higher than or equal to the
averages of the thirty OECD countries.
- 39 -
3.1.2 General value of science scale
There are altogether five general values of science in this scale, the details of which are
shown in Table 14.
Table 14A comparison of 15-year-olds’responses to
general value of science between Macao and OECD countries
% of students whoagree or strongly agree
General values of science
Macao OECD
1. Science is important for helping us to understandthe natural world
99% 93%
2. Advances in science and technology usuallyimprove people’s living conditions
98% 92%
3. Science is valuable to society 95% 87%
4. Advances in science and technology usually helpto improve the economy
89% 80%
5. Advances in science and technology usuallybring social benefits
93% 75%
The reported percentages for the above five general values of science are all higher than
the averages of the thirty OECD countries.
- 40 -
3.1.3 Personal value of science scale
There are altogether five personal values of science in this scale, the details of which are
shown in Table 15.
Table 15A comparison of 15-year-olds’responses to
personal value of science between Macao and OECD countries
% of students whoagree or strongly agree
Personal values of science
Macao OECD
1. I find that science helps me to understandthings around me
90% 75%
2. I will use science in many ways when I am anadult
73% 64%
3. Some concepts in science help me see how Irelate to other people
64% 61%
4. When I leave school there will be manyopportunities for me to use science
63% 59%
5. Science is very relevant to me 91% 57%
The reported percentages for the above five personal values of science are all higher
than the averages of the thirty OECD countries.
- 41 -
3.1.4 Self-efficacy in science scale
There are altogether eight tasks illustrative of self-efficacy in science in this scale, the
details of which are shown in Table 16.
Table 16A comparison of 15-year-olds’responses to
self-efficacy in science between Macao and OECD countries
% of students who reply“I could do this easily” or
“I could do this with a little bit effort”Tasks
Macao OECD
1. Explain why earthquakes occur more frequentlyin some areas than in others
70% 76%
2. Recognize the science question that underlies anewspaper report on a health issue
70% 73%
3. Interpret the scientific information provided onthe labeling of food items
59% 64%
4. Predict how changes to an environment willaffect the survival of certain species
58% 64%
5. Identify the science question associated withthe disposal of garbage
68% 62%
6. Describe the role of antibiotics in the treatmentof disease
49% 59%
7. Identify the better of two explanations for theformation of acid rain
63% 58%
8. Discuss how new evidence can lead you tochange your understanding about thepossibility of life on Mars
37% 51%
Amongst the above eight tasks illustrative of self-efficacy in science all except two on
“disposal of garbage” and “formation of acid rain” are lower than that of the averages of
the thirty OECD countries.
- 42 -
3.1.5 Self-concept in science scale
There are altogether six self-concepts in science in this scale, the details of which are
shown in Table 17.
Table 17A comparison of 15-year-olds’responses to
self-concept in science between Macao and OECD countries
% of students whoagree or strongly agree
Self-concepts in science
Macao OECD
1. I can usually give good answers to testquestions on school science topics
60% 65%
2. When I am being taught school science I canunderstand the concepts very well
54% 59%
3. I learn school science topics quickly 49% 56%
4. I can easily understand new ideas in schoolscience
49% 55%
5. Learning advanced school science topics wouldbe easy for me
44% 47%
6. School science topics are easy for me 39% 47%
The reported percentages for the above six self-concepts in science are all lower than
the averages of the thirty OECD countries.
- 43 -
3.1.6 Enjoyment of science scale
There are altogether five situations illustrative of enjoyment of science in this scale, the
details of which are shown in Table 18.
Table 18A comparison of 15-year-olds’responses to
enjoyment of science between Macao and OECD countries
% of students who
agree or strongly agreeSituations
Macao OECD
1. I enjoy acquiring new knowledge in science 86% 67%
2. I generally have fun when I am learning sciencetopics
81% 63%
3. I am interested in learning about science 79% 63%
4. I like reading about science 72% 50%
5. I am happy doing science problems 56% 43%
The reported percentages for the above five situations of enjoyment of science are all
higher than the averages of the thirty OECD countries.
- 44 -
3.1.7 Instrumental motivation to learn science scale
There are altogether five situations of instrumental motivation to learn science in this
scale, the details of which are shown in Table 19.
Table 19A comparison of 15-year-olds’responses to
instrumental motivation to learn science between Macao and OECD countries
% of students whoagree or strongly agree
Situations
Macao OECD
1. I study school science because I know it isuseful for me
85% 67%
2. Making an effort in my school science subjectis worth it because this will help me in thework I want to do later on
82% 63%
3. Studying my school science subject isworthwhile for me because what I learn willimprove my career prospects
79% 61%
4. I will learn many things in my science subjectsthat will help me get a job
76% 56%
5. What I learn in my science subject is importantfor me because I need this for what I want tostudy later on
80% 56%
The reported percentages for the above five situations of instrumental motivation are all
higher than the averages of the thirty OECD countries.
- 45 -
3.1.8 Future-oriented motivation to learn science scale
There are altogether four kinds of future-oriented motivation to learn science in this
scale, the details of which are shown in Table 20.
Table 20A comparison of 15-year-olds’responses to
future-oriented motivation to learn science between Macao and OECD countries
% of students whoagree or strongly agree
Future-oriented motivation
Macao OECD
1. I would like to work in a career involvingscience
42% 37%
2. I would like to study science after secondaryschool
33% 31%
3. I would like to work on science projects as anadult
24% 27%
4. I would like to spend my life doing advancedscience
18% 21%
Except the first two on “work in a career involving science” and “study science after
secondary school”, the last two future-oriented motivations to learn science are lower
than that of the averages of the thirty OECD countries.
- 46 -
3.1.9 Awareness of environmental issues scale
There are altogether five cases of environmental issues in this scale, the details of which
are shown in Table 21.
Table 21A comparison of 15-year-olds’responses to
awareness of environmental issues between Macao and OECD countries
% of students who“are familiar with this issue and wouldbe able to explain it well” or “I knowsomething about the issue and could
explain the general issue”Issues
Macao OECD
1. The consequences of clearing forests for otherland use
87% 73%
2. Acid rain 71% 60%
3. The increase of greenhouse gases in theatmosphere
65% 58%
4. Nuclear waste 32% 53%
5. The use of genetically modified organisms 37% 35%
Amongst the above five cases of awareness of environmental issues, all except one on
nuclear waste are higher than that of the averages of the thirty OECD countries.
- 47 -
3.1.10 Concern for environmental issues scale
There are altogether six cases of concern for environmental issues in this scale, the
details of which are shown in Table 22.
Table 22A comparison of 15-year-olds’responses to
concern for environmental issues between Macao and OECD countries
% of students who believe“this is a serious concern for me
personally as well as others” or “this isa serious concern for other people inmy country but not me personally”
Issues
Macao OECD
1. Air pollution 93% 92%
2. Extinction of plants and animals 81% 84%
3. Clearing of forests for other land use 81% 83%
4. Energy shortages 87% 82%
5. Nuclear waste 63% 78%
6. Water shortages 88% 76%
Amongst the above six concerns for environmental issues, all except three on air
pollution, energy shortages and water shortages are lower than that of the averages of
the thirty OECD countries.
- 48 -
3.1.11 Optimism regarding environmental issues scale
There are altogether six cases of optimism regarding environmental issues in this scale,
the details of which are shown in Table 23.
Table 23A comparison of 15-year-olds’responses to
optimism regarding environmental issues between Macao and OECD countries
% of students who believe “theproblem associated with the
environmental issues will improve overthe next 20 years”
Issues
Macao OECD
1. Energy shortages 26% 21%
2. Water shortages 27% 18%
3. Air pollution 28% 16%
4. Nuclear waste 21% 15%
5. Extinction of plants and animals 26% 14%
6. Clearing of forests for other land use 25% 13%
The reported percentages for the above six cases of optimism regarding environmental
issues are all higher than the averages of the thirty OECD countries.
- 49 -
3.1.12 Science-related activities scale
There are altogether six science-related activities in this scale, the details of which are
shown in Table 24.
Table 24A comparison of 15-year-olds’responses to
science-related activities between Macao and OECD countries
% of students whodid “very often” or “regularly”
Science Activities
Macao OECD
1. Watch TV programs about science 21% 21%
2. Read science magazines or science articles innewspapers
20% 20%
3. Visit websites about science topics 11% 13%
4. Borrow or buy books on science topics 9% 8%
5. Listen to radio programmes about advances inscience
9% 7%
6. Attend a science club 4% 4%
The reported percentages for the above six science-related activities are comparable
with the averages of the thirty OECD countries.
- 50 -
3.2 In search of quality science education indicators for Macao
In the student questionnaire, there are questions allowing a number of scales to be
constructed that may be served as quality science education indicators and students’
responses to these scales have been reported and compared in section 3.1 above. A
total of twelve such scales are examined and Pearson correlations with scientific literacy
performance broken down by gender are reported in Table 25.
Table 25Pearson correlation of quality science education indicators
with scientific literacy performance scores
Pearson Correlation withScientific Literacy Score
Potential Quality Science Education Indicator
Male Female All
1. General interest in science 0.250 0.256 0.192
2. General value of science 0.193 0.140 0.168
3. Personal value of science 0.071 0.088 0.080
4. Self-efficacy in science 0.321 0.312 0.318
5. Self-concept in science 0.182 0.116 0.151
6. Enjoyment of science 0.289 0.252 0.273
7. Instrumental motivation to learn science 0.237 0.134 0.192
8. Future-oriented motivation to learn science 0.071 0.104 0.088
9. Awareness of environmental issues 0.417 0.389 0.405
10. Concern for environmental issues 0.131 0.093 0.111
11. Optimism regarding environmental issues -0.154 -0.167 -0.157
12. Science-related activities 0.156 0.178 0.168
After examining Table 25, it was decided to drop personal value of science and
future-oriented motivation to learn science as quality science education indicators. The
remaining ten indicators all have moderate to strong correlations (i.e. between 0.10-0.45)
with scientific literacy proficiency scores for the 15-year-olds in Macao.
- 51 -
3.3 The ten quality science education indicators for Macao
Ten scale scores are computed for each sampled student as quality science education
indicators, the mean values of which are broken down by gender because males and
females may exhibit their dispositions, attitudes and values differently. As shown in
Table 26, the first six indicators (i.e. 1-6) are psychological dispositions facilitative of
science learning, the next three indicators (i.e. 7-9) pertain to consciousness of the mind
on environments instrumental to an adequate understanding of the contexts of science
learning, and the last one (i.e. 10) is behavior-oriented in response to the dispositions
and consciousnesses.
Table 26Selected quality science education indicators for the Macao sample
Scale Score (Mean)Quality Science Education Indicators
Male Female All
1. General interest in science 0.20 0.00 0.10
2. General value of science 0.50 0.58 0.54
3. Self-efficacy in science -0.04 -0.17 -0.11
4. Self-concept in science 0.10 -0.36 -0.11
5. Enjoyment of science 0.52 0.31 0.41
6. Instrumental motivation to learn science 0.43 0.34 0.39
7. Awareness of environmental issues 0.08 0.03 0.06
8. Concern for environmental issues 0.01 0.22 0.11
9. Optimism regarding environmental issues 0.46 0.25 0.36
10. Science-related activities 0.38 0.16 0.27
Note: Scale measures are standardized across all the thirty OECD countries with mean equals to zero andstandard deviation equals to one.
- 52 -
Chapter 4
Literacy-ESCS Relationships for Macao Schools
Abstract: This chapter analyzes the intricate relationships between literacy performance
and ESCS (Index of Economic Social and Cultural Status of the Home) to throw light
on the excellence and equity dimensions of educational provisions for the 15-year-olds
in Macao. Plots of these relationships in the Macao sample, as well as within-school
Pearson correlations indicative of the linear components of these curvilinear
relationships for scientific literacy in individual sampled schools are presented to
illustrate the complicated analytical issues concerned.
4.1 Plots of literacy performance with ESCS in the Macao sample
As seen in Figure 7 and 8, there are non-linear relationships in the Macao sample
between scientific, mathematical and reading literacy performance with economic,
social and cultural status of the home. The same relationships are also observed for the
three scientific literacy subscales. Generally speaking, higher ESCS is associated with
moderately higher mathematical, scientific, and reading literacy performance. It is
noteworthy that this non-linear relationship may be different in the sampled school
depending on its admission policy and the student intake (see section 4.2).
- 53 -
Figure 7
Plots of literacy performance with ESCS in the Macao Sample
350
400
450
500
550
600
650
700
750
-4 -3 -2 -1 0 1 2 3
Index of Economic, Social and Cultural Status (ESCS)
Mathematical Literacy Scientific Literacy Reading Literacy
- 54 -
Figure 8
Plots of scientific literacy subscale performance with ESCS in the Macao Sample
350
400
450
500
550
600
650
700
750
-4 -3 -2 -1 0 1 2 3
Index of Economic, Social and Cultural Status (ESCS)
Explaining Phenomena Scientifically Using Scientific Evidence Identifying Scientific Issues
Based on the afore-mentioned assessment results, there are three findings worthy of
concern by schools and the educational authority.
1. Although the impact of ESCS on mathematical, scientific and reading literacy are
not substantial, elevating homes of low ESCS to higher levels is still needed. This
is expected not only to bring about better educational opportunities and equitable
educational results, but also can enhance the mathematical, scientific and reading
literacy proficiency levels of the 15-year-olds in Macao.
2. In the PISA 2006 Study, Macao’s 15-year-olds achieve very well in mathematical,
scientific and reading literacy. Admittedly, there is still ample room for an increase
in the proportion of high-performing students.
- 55 -
3. It is heartening to learn that the proportion of low-performing students is low
compared with other participating countries/economies. This success may be
attributed to the instructional effectiveness of Macao’s teachers. In spite of this
success, it is noteworthy that quite a large proportion of Macao’s 15-year-olds are
still studying at grade 7 and 8. If those low-performing students studying at grade 7
and 8 can be arranged for academic guidance and counseling with reference to the
ten quality science education indicators examined in Chapter 3, then it is expected
their scientific literacy proficiency can be elevated accordingly.
4.2 Within-school correlations of scientific literacy performance with
ESCS
Figure 7 and 8 may not be drawn for each sampled school in order to examine the
within-school literacy-ESCS relationship because of two important reasons. First, the
number of sampled students within school is small, and therefore standard error of
estimates (SE) tends to be very large. Second, depending on school admission policy,
each sampled school covers a portion of the ESCS continuum only. Therefore,
within-school literacy-ESCS relationship may not be pronounced enough to decide
whether the relationship concerned is positively linear, negatively linear, or non-linear
as in the case of Macao sample. For scientific literacy in the PISA 2006 Study,
within-school Pearson correlations indicating the linear component of the literacy-ESCS
relationship range from -0.234 to 0.793 (see Table 27). Hierarchical linear modeling
(HLM) is needed to explain these varying slopes of relationships estimated with
different degrees of reliabilities.
- 56 -
Table 27Relationships of scientific literacy and ESCS, broken down by school
Scientific Literacy ESCS CorrelationSchoolNumber Mean SE Mean SE r SE
1 469.775 5.416 -1.364 0.053 0.099 0.087
2 453.722 7.440 -1.701 0.082 0.099 0.125
3 586.930 5.455 0.051 0.064 0.053 0.078
4 504.016 7.125 -0.134 0.068 -0.095 0.114
5 510.471 4.047 -0.836 0.048 -0.170 0.078
6 530.680 5.295 -0.730 0.059 0.038 0.083
7 509.971 4.738 -0.233 0.072 0.036 0.090
8 589.809 31.864 0.624 0.582 0.793 0.254
9 525.820 12.764 0.623 0.101 0.656 0.075
10 455.775 22.574 0.605 0.217 0.065 0.350
11 338.674 35.694 0.730 0.000 . .
12 516.115 4.687 -1.120 0.053 0.121 0.104
13 475.670 5.206 -1.215 0.069 -0.011 0.101
14 526.464 4.169 -0.994 0.059 0.231 0.085
15 427.989 6.655 -0.442 0.079 0.004 0.099
16 512.806 4.888 -1.204 0.057 0.057 0.100
17 510.484 5.260 -1.313 0.052 0.028 0.090
18 471.477 10.471 -0.947 0.117 0.088 0.176
19 537.867 5.010 -1.078 0.068 -0.013 0.089
20 560.449 7.451 -0.182 0.084 0.086 0.170
21 521.640 4.524 -0.809 0.061 -0.036 0.078
22 507.442 4.950 -1.141 0.063 0.000 0.089
23 534.951 4.608 -0.564 0.056 0.048 0.080
24 542.669 4.770 -1.072 0.057 0.115 0.093
25 434.136 7.652 -1.185 0.077 0.060 0.104
26 506.436 10.268 -1.346 0.118 -0.234 0.174
27 582.657 6.001 -0.621 0.063 0.033 0.096
28 407.376 29.854 -1.336 0.167 0.596 0.199
29 475.126 6.254 -0.837 0.062 0.151 0.095
30 574.690 4.831 -1.036 0.052 -0.001 0.085
31 466.526 6.693 -1.520 0.064 0.097 0.094
32 494.786 5.650 -1.335 0.061 -0.006 0.089
33 527.868 9.838 -0.342 0.155 -0.006 0.225
34 478.288 7.451 -1.335 0.107 0.126 0.099
35 485.910 5.200 -0.941 0.064 0.057 0.070
36 405.059 22.718 -1.027 0.222 0.123 0.241
37 507.890 4.881 -1.111 0.052 0.001 0.093
38 531.011 5.136 -0.266 0.065 0.003 0.091
39 511.117 5.776 -0.909 0.053 0.204 0.069
40 502.165 5.552 -1.345 0.057 0.015 0.078
41 426.345 6.930 -1.157 0.110 0.158 0.111
42 514.795 7.148 -1.210 0.111 0.227 0.151
43 461.036 19.229 -0.006 0.192 0.213 0.195
Total 510.803 1.058 -0.909 0.014 0.148 0.016
Note: There is only one sampled student for School 11. So no analysis is done for this school. Also,standard errors (SE) of estimates for schools with small number of sampled students are generally large.
- 57 -
4.3 Between-school scientific literacy performance with ESCS
Based on mean scientific literacy and ESCS in Table 27, the following figure showing
the between-school scientific literacy performance and ESCS relationships may be
drawn.
Figure 9
Plots between-school scientific literacy performance with ESCS
32
13
14
35
21
7
2
311
26
40
28
34
1742
16
25
41
12
37
19
24
30
36
18
39
29
5
6
27
23
15
33 38
20
4
43
3
10
9
8
11
300
350
400
450
500
550
600
-2 -1.5 -1 -0.5 0 0.5 1
Index of Economic, Social and Cultural Status (ESCS)
▲High-performing school +Average-performing school ■Low-performing school
22
Sci
en
tifi
cL
itera
cy
If we classify the scientific literacy performance of the 43 schools into high-performing
(13 schools), average-performing (14 schools), and low-performing (13 schools), the
following three observations may be made without disclosing the identity of individual
schools:
1. Amongst the four schools classified as “International/Portuguese” in the PISA
2006 Study, the ESCS of these schools are the highest amongst the 43 sampled
schools (mean ESCS > 0.50). As far as scientific literacy performance is concerned,
- 58 -
two are high-performing (top one-third of the 43 schools) and two are
low-performing (bottom one-third of the 43 schools). Therefore, there is no
apparent relationship between the four school's scientific literacy performance with
school ESCS.
2. The thirteen high-performing schools in scientific literacy (except the two
International/Portuguese schools mentioned above) are generally associated with
homes of higher ESCS (i.e. -1.1 <ESCS< 0.5).
3. The thirteen low-performing schools in scientific literacy (except the two
International/Portuguese schools mentioned before) are generally associated with
homes of lower ESCS (i.e. -1.80 <ESCS< -0.90). There are two low-performing
schools (school number 15 and 43) having students coming from homes of higher
ESCS (i.e. ESCS = -0.44 and -0.01). Without making a serious investigation,
reasons for the two schools' low scientific literacy performance can only be
speculated.
- 59 -
Chapter 5
International Comparison of Literacy Performance
Abstract: From an international comparison perspective, this chapter analyzes Macao’s
scientific, mathematical, and reading literacy performance in the PISA 2006 Study. It
highlights a number of East Asian and western high-performing countries/economies
that may serve as exemplary models for Macao’s educational improvement and
curriculum reform.
5.1 Performance in the three literacy domains – An international
comparison
Table 28 displays literacy performance results allowing educational researchers and
policy makers to compose a league table that serves their international comparison
purposes. Macao’s literacy performance may not only be compared with East Asian
countries/economies (e.g. Chinese Taipei, Hong Kong, Japan, Korea, Indonesia, and
Thailand), but also contrasted with the other western high-performing
countries/economies (e.g. Australia, Canada, Estonia, Finland, Liechtenstein,
Netherlands, and New Zealand) in the three domains of literacy assessed in the PISA
2006 Study.
Table 28Performance of countries/economies in the three literacy domains
Scientific LiteracyMathematical
LiteracyReading Literacy
Country/Economy
Mean SE Mean SE Mean SE
1. Azerbaijan 382.236 2.747 475.769 2.211 351.959 3.105
2. Argentina 391.285 6.013 380.549 6.193 373.977 7.138
3. Australia 528.997 2.954 522.208 3.152 516.014 3.010
4. Austria 510.472 3.909 505.338 3.658 491.295 3.9495. Belgium 509.594 2.577 521.746 2.945 502.753 3.062
6. Brazil 390.229 2.788 369.515 2.933 392.885 3.743
7. Bulgaria 434.001 6.115 413.448 6.133 401.934 6.909
8. Canada 534.237 2.029 527.035 2.047 526.919 2.419
9. Chile 438.418 4.283 410.995 4.492 442.692 4.719
10. Chinese Taipei 532.449 3.570 549.357 4.102 496.235 3.37611. Colombia 387.934 3.377 369.979 3.783 385.306 5.083
- 60 -
12. Croatia 492.988 2.454 466.697 2.419 477.326 2.804
13. Czech Republic 512.825 3.480 509.859 3.551 482.715 4.181
14. Denmark 495.850 3.110 513.026 2.617 494.483 3.17815. Estonia 531.366 2.524 514.575 2.745 500.750 2.927
16. Finland 563.315 2.020 548.358 2.296 546.868 2.147
17. France 495.175 3.362 495.538 3.170 487.706 4.06418. Germany 515.502 3.867 504.322 3.950 496.267 4.37619. Greece 473.321 3.232 459.202 2.968 459.711 4.044
20. Hong Kong-China 542.192 2.474 547.461 2.670 536.066 2.422
21. Hungary 503.891 2.677 490.937 2.885 482.375 3.281
22. Iceland 491.457 1.501 506.907 1.457 486.076 1.426
23. Indonesia 393.346 5.748 389.654 5.617 392.901 5.562
24. Ireland 508.291 3.188 501.472 2.785 517.313 3.536
25. Israel 453.626 3.733 442.858 4.214 437.134 4.491
26. Italy 474.970 1.991 462.336 2.219 468.381 2.317
27. Japan 530.155 3.441 523.939 3.290 496.795 3.73828. Jordan 421.884 2.841 384.034 3.304 400.581 3.271
29. Korea 522.118 3.359 547.458 3.761 556.022 3.813
30. Kyrgyzstan 321.887 2.936 310.583 3.410 284.704 3.480
31. Latvia 489.496 2.973 486.166 3.027 479.492 3.730
32. Liechtenstein 523.408 4.244 525.321 4.638 510.793 4.488
33. Lithuania 488.597 2.777 485.990 2.823 470.374 3.118
34. Luxembourg 486.274 1.055 490.002 1.066 479.367 1.281
35. Macao-China 510.803 1.058 525.003 1.303 492.288 1.099
36. Mexico 409.765 2.663 405.027 2.890 409.457 3.167
37. Netherlands 524.833 2.743 530.654 2.586 506.747 2.925
38. New Zealand 530.547 2.737 521.769 2.484 521.381 3.000
39. Norway 486.478 3.113 489.846 2.640 484.293 3.182
40. Poland 498.046 2.368 494.995 2.354 507.534 2.781
41. Portugal 474.398 3.010 465.606 3.039 472.584 3.536
42. Qatar 349.188 0.856 317.956 1.022 312.213 1.196
43. Romania 418.593 4.210 414.286 4.251 396.989 4.897
44. Russian Federation 479.130 3.690 475.260 3.903 440.145 4.436
45. Slovak Republic 488.385 2.587 492.106 2.823 466.350 3.056
46. Slovenia 518.893 1.101 504.368 1.001 494.500 0.97347. Spain 488.598 2.460 479.840 2.338 460.391 2.301
48. Sweden 503.293 2.376 502.356 2.408 507.313 3.436
49. Switzerland 511.820 3.121 529.561 3.257 499.589 3.100
50. Thailand 420.566 2.183 416.663 2.311 416.750 2.572
51. Tunisia 385.402 2.959 365.479 3.959 380.335 4.020
52. Turkey 423.749 3.841 423.941 4.901 447.132 4.206
53. United Kingdom 514.738 2.290 495.444 2.142 495.077 2.25454. United States 488.907 4.224 474.352 4.020 - -
55. Uruguay 428.052 2.750 426.799 2.605 412.516 3.434
56. Serbia and/or Montenegro 433.298 2.746 431.959 3.178 400.170 3.134
All participatingcountries/economies
461.387 1.046 454.005 1.011 450.572 1.120
Note 1: Serbia and Montenegro student data are grouped together in the PISA database. Therefore, thereare 56 instead of 57 entries in the table.Note 2: Reading literacy results for United States were not available due to printing errors of the testbookletsNote 3: Literacy means which are: (i) statistically significantly higher than that of Macao-China areshaded for easy identification; (ii) comparable with that of Macao-China are printed in bold italic; (ii)statistically significantly lower than that of Macao-China are printed as usual without any change of fontor visual effect.
- 61 -
5.2 Selection of countries/economies exemplary for Macao’s
educational improvement and curriculum reform
Based on Table 29, one can see that Hong Kong, Chinese Taipei, Japan and Korea
performed very well in all three literacy domains and their ranks are amongst the top of
all participating countries/economies. On the whole, East Asian countries/economies
listed in Table 29 score highly in all three literacy domains. One hypothesis to explain
this finding pertains to the Confucian tradition of high parental emphases on their
children’s early education, and their high expectation they cast on their children during
the obligatory years of schooling.
Table 29Top five high-performing East Asian countries/economies
Scientific Literacy Mathematical Literacy Reading Literacy
East AsianCountry/Economy
MeanEast AsianCountry/Economy
MeanEast AsianCountry/Economy
Mean
Hong Kong-China 542.192 Chinese Taipei 549.357 Korea 556.022
Chinese Taipei 532.449 Hong Kong-China 547.461 Hong Kong-China 536.066
Japan 530.155 Korea 547.458 Japan 496.795
Korea 522.118 Macao-China 525.003 Chinese Taipei 496.235
Macao-China 510.803 Japan 523.939 Macao-China 492.288
All participatingcountries/economies
461.387All participatingcountries/economies
454.005All participatingcountries/economies
450.572
However, it should be pointed out that the Confucian tradition hypothesis alone cannot
explain why some countries such as Australia, Finland, Canada, Liechtenstein, New
Zealand, Netherlands, and Switzerland perform so well in all three literacy domains (see
Table 30). There are countries/economies like Estonia and Ireland which perform highly
on individual literacy domains. Macao can learn a lot from these countries if factors
leading to their success can be revealed and exemplary educational practices can be
modeled upon in Macao schooling contexts.
- 62 -
Table 30Non-East Asian countries/economies
having higher or comparable literacy performance than Macao
Scientific Literacy Mathematical Literacy Reading Literacy
Non-East AsianCountry/Economy
MeanNon-East AsianCountry/Economy
MeanNon-East AsianCountry/Economy
Mean
Finland 563.315 Finland 548.358 Finland 546.868
Canada 534.237 Netherlands 530.654 Canada 526.919
Estonia 531.366 Switzerland 529.561 New Zealand 521.381
New Zealand 530.547 Canada 527.035 Ireland 517.313
Australia 528.997 Liechtenstein 525.321 Australia 516.014
Netherlands 524.833 Australia 522.208 Liechtenstein 510.793
Liechtenstein 523.408 Belgium 521.746 Poland 507.534
Slovenia 518.893 New Zealand 521.769 Sweden 507.313
Germany 515.502 Netherlands 506.747
United Kingdom 514.738 Belgium 502.753
Czech Republic 512.825 Estonia 500.750
Switzerland 511.820 Switzerland 499.589
Austria 510.472 Germany 496.267
Belgium 509.594 United Kingdom 495.007
Ireland 508.291 Slovenia 494.500
Denmark 494.483
Austria 491.295
All participatingcountries/economies
461.387All participatingcountries/economies
454.005All participatingcountries/economies
450.572
Note: Reading literacy results for United States were not available due to printing errors of the testbooklets
Last but not least, countries/economies like United States, United Kingdom and
Portugal have in the past influenced Macao’s basic education curriculum and
educational system. Hence, it may be a good idea to examine the strengths and
weaknesses of these educational systems to see whether there are exemplary educational
practices and provisions that can be modeled upon by Macao.
- 63 -
Table 31Performance results of other countries/economies in the three literacy domains
Scientific Literacy Mathematical Literacy Reading Literacy
Country/Economy Mean Country/Economy Mean Country/Economy Mean
United States 488.907 United Kingdom 495.444 Portugal 472.584
Portugal 474.398 United States 474.352 Thailand 416.750
Thailand 420.566 Portugal 465.606 Indonesia 392.901
Indonesia 393.346 Thailand 416.663
Indonesia 389.654
All participatingcountries/economies
461.387All participatingcountries/economies
454.005All participatingcountries/economies
450.572
Note: Reading literacy results for United States were not available due to printing errors of the testbooklets.
- 64 -
Chapter 6
Thematic Reports and Follow-up Studies
The present report serves as a valuable starting point for educational researchers and
policy makers to have an initial understanding of 15-year-old students’ literacy
performance in three key domain areas. More in-depth analyses are necessary to make
full use of the student, school and system data collected in the PISA studies.
Apart from the PISA 2006 Study database, the data analysis manual, technical report,
international report, as well as all relevant OECD PISA publications should be studied
for a full understanding of the findings reported in this Macao PISA 2006 Study Report
(see the full list of references documented in this report; see also www.pisa.oecd.org).
Evidently, it is not possible in this first report to address all kinds of issues and problems
thought significant by policy makers and teachers in Macao. However, this report does
point to key topics that should be addressed in thematic reports in the future. Three
thematic reports are worthy of preparation and publication:
1. By making full use of the student, school and system data, examine factors
affecting excellence and equity in literacy performance in the three key domains
for 15-year-olds in Macao. In particular, conditions, processes and contexts
facilitative of literacy acquisition for the different types of schools and different
kinds of students (e.g. students studying in single-sex versus co-educational
schools) should be examined.
2. By linking the literacy performance results with previous PISA 2003 Study data,
chart the change in literacy performance in the three key domains. Factors
affecting growth and decline of literacy performance should be identified so as to
guide educational improvement and curriculum reform.
3. By comparing selected high performing East Asian with western
countries/economies, attempt to understand the similarities and differences
pertaining to excellence in literacy performance for 15-year-olds in the three key
domains. This report revealed that East Asian countries/economies such as Chinese
- 65 -
Taipei, Hong Kong, Japan, and Korea may be compared with western countries
such as Australia, Estonia, Finland, Canada, Ireland, Liechtenstein, New Zealand
and Netherlands. It is hoped that such international comparisons can uncover the
secrets of academic excellence of 15-year-olds students of the top five
high-performing East Asian countries/economies.
Last but not the least, follow-up intervention programs should be designed to monitor
the learning progress of those low-performing students identified in the PISA 2006
Study. Given quite a large proportion of 15-year-old are repeaters and they are likely to
remain in the basic education system in the next few years, schools and educational
researchers should join efforts to explore ways of counseling them, and to this end the
ten quality science education indicators reported in chapter 3 are good starting points for
an initial understanding of their psychological dispositions and learning characteristics.
- 66 -
References
Lo, L.F. & Cheung, K.C. (2005). 2003 Macau-PISA Final Report: Assessment of
15-Year-Old Macau Students. Macao: The Macau-PISA Center.
OECD. (1999). Measuring Student Knowledge and Skills: A New Framework for
Assessment. Paris: OECD.
OECD. (2001). Knowledge and Skills for Life: First Results from the OECD
Programme for International Student Assessment (PISA) 2000. Paris: OECD.
OECD. (2002b). Reading for Change: Performance and Engagement across Countries
Paris: OECD.
OECD (2004a). Learning for Tomorrow’s World: First Results from PISA 2003. Paris:
OECD.
OECD (2004b). What Makes School Systems Perform? Seeing School Systems through
the Prism of PISA. Paris: OECD.
OECD (2004c). Problem-solving for Tomorrow’s World: First Measures of
Cross-curricular Competencies from PISA 2003. Paris: OECD.
OECD (2005a). The Definition and Selection of Key Competencies: Executive Summary.
Paris: OECD.
OECD (2005b). PISA 2003 Data Analysis Manual: SPSS Users. Paris: OECD.
OECD (2006). Assessing Scientific, Reading and Mathematical Literacy: A Framework
for PISA 2006. Paris: OECD.
OECD (2007). PISA 2006 -- Science Competencies for Tomorrow’s World, Vol. 1 & 2.
Paris: OECD.
Recommended