Studies in Educational Evaluation 34 (2008) 155–164

Contents lists available at ScienceDirect

Studies in Educational Evaluation

journal homepage: www.e lsev ier .com/stueduc

Examining differences in mathematics and reading achievement amongGrade 5 pupils in Vietnam

Njora Hungi

Flinders University Institute for International Education (FUIIE), Adelaide, Australia

A R T I C L E I N F O

Keywords:

Mathematics achievement

Reading achievement

Multilevel models

Vietnam education

Pupil achievement

A B S T R A C T

This study employed a multilevel technique to examine pupil-, school- and province-level factors that

influence achievement in mathematics and reading of Grade 5 primary school pupils in Vietnam. The data

for this study were collected as part of a major survey that sought to examine the quality of education

offered in Vietnam primary schools. Results show that pupil’s age, pupil’s home background and

correction of homework influence significantly achievement in mathematics and reading at grade 5 level

in Vietnam. Policy implications of the findings are discussed.

� 2008 Elsevier Ltd. All rights reserved.

Introduction

The purpose of this study was to identify the major pupil-,school- and province-level factors influencing achievement inmathematics and reading between Grade 5 pupils in Vietnam. Thisstudy utilized data from 72,660 pupils in 3635 schools in 61provinces in Vietnam. These data were collected as part of a majorstudy sponsored by the Vietnamese Ministry of Education thatsought to examine the quality of the education provided in primaryschools in Vietnam.

When dealing with multilevel data such as the data in thisstudy, the appropriate procedure is to formulate multilevelmodels, ‘‘which enable the testing of hypotheses about effectsoccurring within each level and the interrelations among them’’(Raudenbush & Bryk, 1994, p. 2590). Testing of hypotheses inmultilevel models can be carried out using multilevel data analysessoftware such as HLM5 for Windows (Raudenbush, Bryk, Cheong, &Congdon, 2000a). The HLM program was initially developed to finda solution for the methodological weakness of educationalresearch studies during the early 1980s, which was the failureof many analytical studies to attend to the hierarchical, multilevelcharacter of much of educational field research data (Bryk &Raudenbush, 1992). This failure came from the fact that ‘‘thetraditional linear models used by most researchers require theassumption that subjects respond independently to educationalprograms’’ (Raudenbush & Bryk, 1994, p. 2590). In practice, mosteducational research studies select students as a sample who arenested within classrooms, and the classrooms are in turn nested

E-mail address: hungi05@yahoo.com.

0191-491X/$ – see front matter � 2008 Elsevier Ltd. All rights reserved.

doi:10.1016/j.stueduc.2008.07.004

within schools, schools within geographical locations, states, orcountries. In this situation, the students selected in the study arenot independent, but rather nested within organizational units andignoring this fact results in the problems of ‘‘aggregation bias andmisestimated precision’’ (Raudenbush & Bryk, 1994, p. 2590).

The multilevel technique employed in this study has been usedin other studies to tease out factors influencing pupil achievementin several developing countries. For example, Willms and Somers(2001) used hierarchical models to examine socioeconomic factorsand other factors influencing mathematics and language achieve-ment among Grades 3 and 4 pupils from 13 Latin Americancountries.

The structure of this paper is as follows. A section is provided inwhich the data involved in this study are described followed by asection in which the construction of variables used in this study isdescribed. After that, two sections are provided in which thehypothesized multilevel models and the multilevel analyses aredescribed. Finally, sections are included in which results of theanalyses are presented and interpretations of the results and theirpossible implications for policy are discussed.

Data

The data for this study were collected by the Vietnam Ministryof Education and Training (MOET) with technical assistanceprovided by National Institute of Educational Sciences (NIES) staffin Hanoi, International Institute for Educational Planning (IIEP)staff in Paris and World Bank staff in Hanoi. These data werecollected using a stratified two-stage cluster sample design. At thefirst stage, schools were selected within provinces with probabilityproportional to the number of pupils in the defined target

N. Hungi / Studies in Educational Evaluation 34 (2008) 155–164156

population. At the second stage, a simple random sample of a fixednumber of pupils was selected within each selected school.

The outcome variables of interest in this study were pupilscores (on Rasch scales) in mathematics and reading tests at Grade5. These tests were developed after careful curriculum mapping bya panel of subject specialists mostly from the NIES in Vietnam toidentify those elements of curriculum outcomes that wereconsidered important and which were to be assessed in the tests.The subject specialists from the NIES also reviewed the test itemsto ensure that they conformed to the national syllabus. Inadditional, during the process of test development and beforethe tests were administered, they were field-tested in Thanh Hoaprovince and their psychometric characteristics examined usingclassical item analysis and Rasch analysis. The final mathematicsand reading tests consisted of 60 items each.

Apart from achievement scores in mathematics and reading, awide range of information about these pupils, their teachers, theirschool principals and characteristics of their schools werecollected. The variables examined in this study are those variablesidentified as potential predictors of academic achievementfollowing preliminary analyses, sound reasoning and researchfindings from studies in other countries. The variables examined inthis study are outlined in a separate section below together withthe hypothesized models.

An interesting aspect of this study is that the teachers were alsoassessed in mathematics and reading. The teacher and pupil testsused different sets of items but some common items (7 and 10items for mathematics and reading, respectively) were included inboth tests for purposes of equating. The teacher tests weredesigned to be more difficult than the pupil tests. In addition, likethe pupil tests, the teacher tests were also field-tested in ThanhHoa province and their measurement characteristics carefullyexamined using classical item analysis and Rasch analysis beforethey were administered. The final teacher mathematics andreading tests consisted of 45 items each. Teachers’ scores in thesetests are used as predictors of pupil achievement in mathematicsand reading in this study.

Construction of variables

In all, there were more than 50 separate questions in thequestionnaires. In some cases, one question (e.g. sex of pupil) wasused as an indicator. Sometimes, two or more variables were usedto form an indicator (for example, ‘Pupil–teacher ratio’). In yetother cases, a number of questions were combined to estimate, forinstance, the total school resources. Where questions arecombined in some way, the resultant variable is known as aconstruct or composite.

For example, in Vietnam poor parents tend to have fewpossessions at home, low educational levels, their homes do nothave private corners for children to study and they generally do notprovide their children with most learning materials such as pens,pencils, rulers and erasers. Consequently, a socioeconomic status(SES) factor (a principal component) was formed from these fourvariables with the following loadings:

Variable

Loading

Private corner

0.64

Total possessions at home

0.83

Parents’ education

0.67

Learning materialsa

0.65

a Number of learning materials that a pupil has (e.g. Pens, pencils,erasers and exercise books).

The correlation between this SES factor and mathematicsachievement was 0.31 while for reading achievement thecorrelation was 0.33. As expected, pupils with higher values onthe SES factor tended to obtain higher scores both in reading and inmathematics. The correlation between the two achievement testscores was 0.74.

Similarly, a further four composite variables (principal compo-nents) were constructed using the procedure employed to form thesocioeconomic variable above. These four composite variableswere: a pupils’ behaviour factor (SPBHP_2), a teacher behaviourfactor (STBHP_2), school resource factor (SREST_2), and a parentalcontribution factor (SPARC_2).

Hypothesized models

In the multilevel analyses reported in this paper, two separatethree-level models were hypothesized and examined, one forfactors influencing achievement in mathematics and the other forfactors influencing achievement in reading. The outcome variablesof interest in these models were pupils’ scores in mathematics(PMA500) and reading (PRD500) tests, both with a mean of 500 anda standard deviation of 100. The hierarchical structures of thesemodels were pupils at level-1, schools at level-2 and provinces atlevel-3. In other words, pupils were nested within schools andschools were nested within provinces.

In this three-level model, 16 variables were initiallyhypothesizedto influence directly achievement at the pupil-level and 27 variableswere initially hypothesized to influence directly achievement at theschool- and province-levels. The names and codes of all the predictorvariables tested (whether significant or not) for inclusion at eachlevel of the three-level hierarchical model have been provided inAppendix A. All variables for which data were available for testinghave been listed in Appendix A, in order to show the very extensiverange of possible effects that were examined, rather than to provideinformation only on those that were statistically significant. The lackof statistical significance can sometimes be of great interest in thedevelopment or modification of policy.

Analyses

A preliminary task in HLM analyses was to build a sufficientstatistics matrix (SSM) file. The original data involved in this studyhad 72,660 pupils in 3635 schools nested in 61 provinces.However, in the construction of the SSM file there were a fewschools that were dropped because of insufficient data. As a result,the Ns in the SSM file were 72,376 for pupils, 3620 for schools and61 for provinces. Weighting (with sampling weights calculated tocater for the design of this study) was undertaken in theconstruction of these SSM file.

The first step in HLM analyses was to run the so-called ‘nullmodels’ in order to obtain the amounts of variance available to beexplained at each level of the hierarchy (Bryk & Raudenbush, 1992).The null models contained only the dependent variables (PMA500for mathematics and PRD500 for reading) and no predictor variableswere specified at the pupil-, school- and province-levels.

The second step undertaken was to build up the pupil-levelmodel or the so-called ‘unconditional’ model at level-1. Thisinvolved adding pupil-level predictors to the model, but withoutentering predictors at any of the other levels of the hierarchy. Atthis stage, a step-up approach was followed to examine which ofthe 16 pupil-level variables (listed in Appendix A) had a significant(at p � 0.05) influence on the outcome variables, PMA500 andPRD500. Bryk and Raudenbush (1992) have recommended thestep-up approach for inclusion of variables into the model to thealternative approach often referred to as ‘working-backward’

N. Hungi / Studies in Educational Evaluation 34 (2008) 155–164 157

where all the possible predictors are included in the model andthen the non-significant variables are progressively eliminatedfrom the model.

In this study, all pupil-level predictor variables were grand-mean-centred in the HLM analyses so that the intercept termswould represent the mean mathematics and reading scores forschools (Kreft, De Leeuw, & Aiken, 1995).

The third step in the HLM analyses involved adding the level-2predictors into the model using the step-up strategy mentionedabove. The level-2 exploratory analysis sub-routine available inHLM5 was employed for examining the potentially significantlevel-2 predictors (as shown in the output) in successive HLM runs.The final step involved building up the model to the school-levelthrough adding the significant school-level predictor variables intothe model using the level-3 exploratory analysis sub-routine andthe step-up strategy.

In the analyses described above, coefficients of a variable werespecified as ‘fixed’ at a particular level of the hierarchy if thereliability estimate of the variable dropped below the value of 0.05at that level (see Raudenbush & Bryk, 2002, p.125). Specifying avariable as ‘fixed’ constrained its slope or intercept to be the sameacross all the level-2 and level-3 units. The alternative was tospecify a variable as ‘random’, which allowed the slopes andintercepts to vary among the level-2 and level-3 units (Rauden-bush, Bryk, Cheong, & Congdon, 2000b).

Results

In the next two sub-sections, results of the analyses describedabove have been presented. The results of fixed effects (path orregression coefficients) are presented in the first sub-section and

Table 1Final fixed effects for mathematics and reading

Level Variable name Variable included

Province Intercept (grand mean)

Teachers’ average score TMA500_3, TRD500_3a

School Average meals per day PMEAL_2

Average days absent PABSEN_2

School library SCHLI_2

Average textbook ownership PTXMS_2

Average homework corrected PMATH_2, PREAD_2a

Per cent full day PFULD_2

Average socioeconomic status PSES_2

Pupils’ behaviour problems SPBHP_2

Teachers’ average score TMA500_2, TRD500_2a

Pupil Pupil’s sex PSEX

Speaking Vietnamese PVIET

Ethnic background PETHNIC

Meals per day PMEAL

Travel time to school PTRAVEL

Interaction with school location SLOCA_2

Days absent PABSENT

Grade repetition PGREP

Sitting place PSIT

Age in months PAGEMON

Books at home PBOOKHM

Textbook ownership PTXMSHR, PTXVSHRa

Homework corrected PMATHHW

Socioeconomic status PSES

Interaction with average grade repetition PGREP_2

Interaction with school location SLOCA_2

Interaction with school resources SREST_2

Interaction with per cent female teachers SFEMT_2

Suffix _2 is used to indicate a school-level variable and suffix _3 is used to indicate provi

given are those of the metric coefficients.a Variable listed first was included in the mathematics model while the second vari

the results of variance partitioning and variance explained arepresented in the second sub-section.

Fixed effects

Estimates of fixed effects from the final three-level models formathematics and reading have been given in Table 1.

The metric coefficients given in Table 1 were obtained fromHLM runs using raw scores of the variables, while the standardizedcoefficients were obtained by applying the following formula tothe metric coefficients.

Std:Coeff : ¼ ðMetric Coeff :Þ � ðStd:Dev:Explanatory VariableÞStd:Dev:Dependent Variable

ðHox;1995;p: 25Þ

In interpreting the results in Table 1, the following three points areworth noting. First, all the coefficients displayed in Table 1 aresignificant at p � 0.05 because their values taken in absolute termswere more than twice their standard errors. The standard errorsgiven are those of the metric coefficients. Because weighting (usingsampling weights) was undertaken in the construction of the SSMfiles, the HLM program took into consideration the design of thisstudy in the computation of these standard errors.

Second, the signs of metric and standardized coefficientsindicate directions of effects and can be interpreted from thecoding. For example, negative coefficients for PAGEMON (age inmonths) indicate that younger pupils were estimated to achievebetter in mathematics and reading than older pupils. The positivecoefficients for PMEAL (meals per day) indicate that pupils who atemore meals per day were estimated to achieve better inmathematics than pupils who ate fewer meals per day.

Mathematics Reading

Standardized Metric S.E. Standardized Metric S.E.

493.82 2.77 490.94 2.33

0.08 0.19 0.06 0.07 0.21 0.06

0.03 10.66 4.79 0.06 18.47 4.89

�0.04 �6.30 2.36 �0.03 �5.09 2.29

0.06 16.55 4.16 0.05 12.72 3.40

0.03 53.38 20.50 0.05 99.38 26.69

0.10 17.98 2.60 0.07 11.99 1.97

0.03 9.06 2.65 0.03 7.68 2.43

0.10 15.06 2.34 0.12 17.06 2.26

�0.03 �14.97 7.32 �0.02 �14.25 6.24

0.16 0.17 0.01 0.11 0.12 0.01

�0.02 �3.91 0.59 0.04 8.51 0.63

0.03 5.07 0.92 0.04 7.15 1.15

�0.02 �5.24 1.79 �0.03 �7.47 1.93

0.02 3.32 0.71

�0.03 �0.27 0.03 �0.03 �0.25 0.03

0.00 �0.20 0.04 0.00 �0.12 0.05

�0.04 �2.58 0.20 �0.03 �2.33 0.22

�0.06 �10.89 0.80 �0.07 �13.68 0.95

�0.02 �6.03 1.52 �0.01 �4.53 1.68

�0.03 �0.22 0.03 �0.05 �0.39 0.04

0.03 0.12 0.02 0.03 0.13 0.02

0.01 6.62 1.14 0.03 17.16 2.14

0.06 5.41 0.27 0.05 5.10 0.45

0.11 10.99 0.57 0.11 11.44 0.55

�0.02 �8.63 1.94 �0.03 �14.69 1.83

0.01 2.28 0.68 0.02 4.22 0.82

0.03 0.57 0.11

0.03 14.63 2.50 0.03 13.53 2.55

nce-level variable. All coefficients are significant at p � 0.05. Standard errors (S.E.s)

able was included in the reading model.

Table 2Variance partitioning and variance explained

Pupil (N = 72,376) School (N = 3620) Province (N = 61) Total

Mathematics

(a) Null model 3609.83 5148.25 1356.67 10114.75

(b) Final model 3150.42 3640.44 389.30

(c) Variance available 35.7% 50.9% 13.4%

(d) Variance explained 12.7% 29.3% 71.3%

(e) Total variance explained 4.5% 14.9% 9.6% 29.0%

(f) Variance left unexplained 31.1% 36.0% 3.8% 71.0%

Reading

(a) Null model 4398.75 4625.20 1041.27 10065.22

(b) Final model 3888.84 3068.30 235.17

(c) Variance available 43.7% 46.0% 10.3%

(d) Variance explained 11.6% 33.7% 77.4%

(e) Total variance explained 5.1% 15.5% 8.0% 28.5%

(f) Variance left unexplained 38.6% 30.5% 2.3% 71.5%

N. Hungi / Studies in Educational Evaluation 34 (2008) 155–164158

Third, absolute values of standardized coefficients (called an‘effect size’) can be used to rank variables in terms of their relativedegree of influence on the outcome within the same sample, whilethose of metric coefficients can be used to compare differentsamples with each other (see Hox, 1995, p. 26). Generally, inresearch studies in education, a standardized regression coefficientis considered important if its magnitude taken in absolute terms is�0.10. Thus, based on the standardized coefficients given inTable 1, it would appear that the important predictors ofmathematics achievement among Grade 5 pupils in Vietnamwere: teachers’ average score (0.16), socioeconomic status atindividual-level (0.11), Socioeconomic status at school-level (0.10)and average homework corrected (0.10). For reading, theimportant predictors were average socioeconomic status atschool-level (0.12), socioeconomic status at individual-level(0.11) and teachers’ average score (0.11).

At the pupil-level, it can be seen from the results given inTable 1 that mathematics achievement was directly influenced by13 of the 15 pupil-level variables examined in this study. These 13variables were pupil’s sex, speaking Vietnamese, ethnic back-ground, travel time to school, days absent, grade repetition, sittingplace, age in months, books at home, textbook ownership,homework corrected and socioeconomic status. All but one (mealsper day) of these 13 pupil-level variables also had significantinfluences on achievement in reading. However, in the uncondi-tional models at level-1, meals per day also had a significantinfluence on achievement in reading but this variable waseventually deleted because it lost its significance when variablesat the subsequent levels of the hierarchical structure were added.

Out of the 27 school-level variables examined in the HLManalysis (listed in Appendix A), nine had significant influences onachievement in mathematics and reading. These nine school-levelvariables were average meals per day, Average days absent, schoollibrary, average textbook ownership, Average homework cor-rected, per cent full day, Average socioeconomic status, pupils’behaviour problems and teachers’ average score.

For both mathematics and reading, it can be seen from theresults in Table 1 that only one of the 27 variables examined at theprovince-level had a significant influence on achievement, namely,Teachers average score.

Variance components

The results of the final estimation of variance components forthe final mathematics and reading models and the results of theanalyses of the variance components obtained from the corre-

sponding null models have been presented in Table 2 in rows ‘a’and ‘b’, respectively. From the information in Table 2 rows ‘a’ and‘b’, the information presented in rows ‘c’ to ‘f’ were calculated. Adiscussion of the calculations involved here is to be found inRaudenbush and Bryk (2002, pp. 69–95).

Interpretation of the results

In the next three sub-sections, discussions of the effectsrecorded in Table 1 on achievement in mathematics and readingamong Grade 5 pupils in Vietnam at the various levels of hierarchyare outlined. In these sub-sections, it is assumed that pupilsdiffered only in the factor being considered and all other factorswere equal. The numbers given in parenthesis in these sub-sections are the metric coefficients and their standard errors.

The results of variance (presented in Table 2) are discussed in aseparate section below (under the heading ‘Variance partitioningand variance explained’).

Pupil-level model

The following effects on achievement in mathematics andreading were recorded among Grade 5 pupils in Vietnam whenother factors were equal.

(1) P

upil sex (0 = Boy; 1 = Girl): Boys were estimated to achievebetter than girls in mathematics (�3.91, 0.59) while girls wereestimated to achieve better than boys in reading (8.51, 0.63).This problem should be examined in depth by the primaryschool mathematics and reading specialists in Vietnam. Inaddition, the MOET should commission studies to examinethe reasons for gender differences in mathematics andreading achievement and to identify ways of correcting thisproblem.

(2) S

peaking Vietnamese: Pupils who always spoke Vietnamese(the language of the test) outside school were estimated toachieve better in mathematics (5.07, 0.92) and reading (7.15,1.15) than pupils who never spoke Vietnamese outside school.Schools and parents should therefore encourage pupils tospeak Vietnamese outside school. However, this should bedone with caution so as to ensure cultural survival of minoritygroups.

(3) E

thnic background (0 = Kinh; 1 = Others): Pupils who belongedto the Vietnamese ethnic majority group (Kinh) were likely toachieve better in mathematics (�5.24, 1.79) and reading

N. Hungi / Studies in Educational Evaluation 34 (2008) 155–164 159

(�7.47, 1.93) than pupils who belonged to other Vietnameseethnic groups. Thus, policy should concentrate in insuringethnic equality in access to quality education and resources.

(4) M

eals per day: Pupils who ate more meals per day wereestimated to achieve better in mathematics (3.32, 0.71) whencompared to pupils who ate fewer meals per day. Thus,parents should ensure children get enough meals per day sothat the children have adequate energy for learning. Educa-tion policy should focus on encouraging parents and schoolsto start School Feeding Programs to ensure that all childrenreceived enough meals per day for effective learning.

(5) T

ravel time: Pupils who took a shorter time to travel to school(or lived near the school) were likely to achieve better inmathematics (�0.27, 0.03) and reading (�0.25, 0.03) thanthose pupils who took a longer time to travel to school (orlived far from school). This implied that, in the future, any newschools should be located in convenient locations so as toallow as many as possible pupils to access schools in lesstravel time and consequently have more time and energy forlearning. Alternatively, the government could assist schoolslocated in isolated areas to purchase vehicles that would beused to bus pupils to school or assist schools to build boardingfacilities for pupils who lived far from the school.

(6) D

ays absent: Pupils who were never (or were rarely) absentfrom school were estimated to achieve better in mathematics(�2.58, 0.20) and reading (�2.33, 0.22) when compared topupils who were frequently absent from school. Obviously,pupils who were regular absentees received fewer hours ofinstruction and therefore were highly likely to achieve at alower level compared to the rest of their classmates.

Policy should concentrate on low absenteeism in schools.In addition, further analyses should be carried out on thesedata to examine the relationship between absenteeism andother factors among Grade 5 pupils in Vietnam with the aim ofequipping teachers, parents and policymakers with researchinformation that would assist them curb this problem.

(7) G

rade repetition: Pupils who had never repeated a grade wereestimated to achieve better in mathematics (�10.89, 0.80)and reading (�13.69, 0.95) when compared to pupils who hadrepeated a grade one or more times. This relationship was ofinterest because it could be observed even after the influenceof the variable age in months (9) had been controlled for in themodels.

Thus, education policy should emphasize low-graderepetition. In addition, further analyses should be carriedout on these data to examine factors associated with graderepetition in Vietnam with the aim of providing researchinformation that could be used to minimize grade repetition.

(8) S

itting place: Pupils who had their own sitting places in classwere estimated to achieve better in mathematics (�6.03,1.52) and reading (�4.53, 1.68) than pupils who shared sittingplaces or had no sitting places in class. Surely, pupils mighthave been less motivated to learn if they had to spend thewhole day in uncomfortable sitting places because of lack offurniture or over crowding in classrooms. This implied thatthe government should ensure that every class in Vietnam hassufficient sitting places.

(9) A

ge in months: Younger pupils were estimated to achievebetter in mathematics (�0.22, 0.03) and reading (�0.39, 0.04)than their older counterparts. Clearly, being older in Grade 5was a disadvantage. It was possible that this effect might havebeen a consequence of grade repetition by the less able pupils.However, this was unlikely because grade repetition wascontrolled for in the analysis after being found to be

statistically significant (see (7)). Therefore, parents inVietnam should ensure that all children enter school at theright age. In addition, the government should find ways ofensuring that all parents have their children enrolled in schoolat the designated age for beginning schooling.

(10) B

ooks at home: Pupils who had more books at home werelikely to achieve better in mathematics (0.12, 0.02) andreading (0.13, 0.02) compared with pupils who had hardly anybooks at home. It was possible that this effect might have beena consequence of socioeconomic status with pupils frompoorer backgrounds having fewer books at home compared topupils from richer backgrounds. However, this was unlikelybecause socioeconomic status was controlled for in theanalysis (see (13)). Thus, all parents (poor or rich) should beencouraged to provide more books at home to assist theirchildren in learning.

(11) T

extbook ownership: Pupils who had their own mathematicsand reading textbooks were estimated to achieve better inmathematics (6.62, 1.14) and reading (17.16, 2.14), respec-tively when compared to pupils who had no mathematics andreading textbooks of their own or compared to pupils who hadto share such textbooks. Again, it was possible that this effectmight have been a consequence of socioeconomic status withpupils from poorer homes not having their own textbooks andpupils from richer homes having their own textbooks.Nonetheless, this was unlikely because socioeconomic statuswas controlled for in the analysis (see (13)). This implied thatall parents (poor or rich) should be encouraged to buy at leastone mathematics textbook and at least one reading textbookto assist their children in learning.

(12) H

omework corrected: Pupils who were given homework (inreading and mathematics) more frequently and had itcorrected were estimated to achieve better in mathematics(5.41, 0.27) and reading (5.10, 0.45) when compared to pupilswho were given homework and had it corrected lessfrequently. The message was clear; all teachers should givehomework more frequently and make sure that they correctthe homework. In addition, the head teachers and local schoolinspectors should monitor the homework given to pupils andthe corrections by teachers of the homework given.

(13) S

ocioeconomic status: Pupils from homes with many posses-sions (rich), private corner for studying, more educatedparents and who had learning materials were estimated toachieve better in mathematics (10.99, 0.57) and reading(11.44, 0.55) than pupils from homes with few or nopossessions (poor), less educated parents and who had hardlyany of the learning materials.

Parents of high socioeconomic status are often welleducated and they show interest in their children learning inschool, encouraging children with their schoolwork. Suchparents often provide their children with basic learningmaterials such as pens and pencils. Clearly, it is importantfor pupils to have basic learning materials for motivation andacademic progress in general. In most cases, poor parentscannot afford to buy these learning materials for their children.

Educational policy should focus on finding ways toencourage low socioeconomic status parents to take aninterest in their children’s schoolwork and for the governmentto provide basic learning materials such as pencils andexercise books to all pupils in primary schools in Vietnam.

The results in Table 1 also show significant cross-levelinteraction effects between (a) travel time to school and schoollocation, (b) SES and average grade repetition, (c) SES and school

Fig. 1. Impact of the interaction effect of travel time to school with School location

on mathematics achievement.

Fig. 2. Impact of the interaction effect of SES with average grade repetition on

mathematics achievement.

N. Hungi / Studies in Educational Evaluation 34 (2008) 155–164160

location, (d) SES and school resources (for mathematics only), and(e) SES and per cent female teachers. Figs. 1 and 5 are graphicalrepresentations of these interaction effects, respectively, formathematics achievement. The co-ordinates for these two graphswere calculated using the procedure described by Aiken and West(1996) and Lietz (1996). It should be noted that the graphical plotsfor the interaction effects for reading achievement were found tobe basically identical to the corresponding plots for mathematicsachievement. Therefore, in order to avoid repetition, only the

Fig. 3. Impact of the interaction effect of SES with school location on mathematics

achievement.

graphs for mathematics achievement have been presented here,but it should be borne in mind that whatever conclusions drawnhere for mathematics achievement also applied for readingachievement as well.

When other factors are equal, Fig. 1 indicates that travel time toschool had a more negative impact in urban schools than in ruralschools while Fig. 2 shows that socioeconomic status had moreimpact in schools with low average grade repetition. Figs. 3–5show that socioeconomic status had more impact in schoolslocated in urban areas, schools with many resources, and in schoolswith high percentages of female teachers.

School-level model

The results in Table 1 indicate that the following school-leveleffects on achievement in mathematics and reading amongVietnamese Grade 5 pupils, all other things being equal.

(1) A

verage meals per day: Pupils in schools where a majority of thepupils ate more meals per day were likely to achieve better inmathematics (10.66, 4.79) and reading (18.47, 4.89) thanpupils in schools in which a majority of the pupils ate fewermeals per day. This would seem to emphasize the need forparents to start School Feeding Programs suggested above.

(2) A

verage days absent: Pupils in schools where a majority of thepupils were never (or rarely) absent from school were likely toachieve better in mathematics (�6.30, 2.36) and reading(�5.09, 2.29) than pupils in schools where a majority of thepupils were more often absent from school. This effect wasinteresting because it could be observed even after the variableDays absent had been included in the model at the pupil-level.This implied that a high rate of absenteeism at the school-levelaffected regular attendees within the school as well.

(3) S

chool library: Pupils in schools with a library or in schools inwhich pupils could borrow books were estimated to achievebetter in mathematics (16.55, 4.16) and reading (12.72, 3.40)when compared to pupils in schools without a library or inschools in which pupils could not borrow books. Thus, schoolsshould be encouraged to allow pupils to borrow books.

(4) A

verage textbook ownership: Pupils in schools where a majorityof the pupils had their own mathematics and reading textbookswere estimated to achieve better in mathematics (53.38, 20.50)and reading (99.38, 26.69) when compared to pupils in schoolswhere a majority of the pupils did not have their ownmathematics and reading textbooks. This clearly emphasizesthe importance of providing textbooks to pupils.

(5) A

verage homework corrected: Pupils in schools where morehomework was given and corrected more frequently were likelyto achieve better in mathematics (17.98, 2.60) and reading(11.99, 1.97) than pupils in schools where less homework wasgiven and corrected less frequently. Again, this seems toemphasize the need for school heads and inspectors to monitorthe homework given to pupils by teachers.

(6) P

er cent full day: Pupils in schools where pupils were attendingfull-day school were likely to achieve better in mathematics(9.06, 2.65) and reading (7.68, 2.43) than pupils in schools nothaving full-day schooling. Thus, pupils attending full-dayschool had a clear advantage because they had more contacthours with the teachers and therefore more time and moreopportunities to learning. Therefore, primary schools inVietnam should be encouraged to run full-day classes for theirGrade 5 pupils. In addition, education policy should specify theclass hours required to learn mathematics and reading skillsappropriately.

Fig. 4. Impact of the interaction effect of SES with school resources on mathematics

achievement.

Fig. 5. Impact of the interaction effect of SES with percentage of female teachers on

mathematics achievement.

N. Hungi / Studies in Educational Evaluation 34 (2008) 155–164 161

(7) A

verage socioeconomic status: Pupils in schools where amajority of the pupils were from homes with more possessions,more educated parents and had learning materials were likelyto achieve better in mathematics (15.06, 2.34) and reading(17.06, 2.26) than pupils in schools where a majority of thepupils were from homes with fewer possessions, less educatedparents and had no learning materials.

(8) P

upils’ behaviour problems: Pupils in schools in which theoccurrences of pupils’ behaviour problems were rare werelikely to achieve better in mathematics (�14.97, 7.32) andreading (�14.25, 6.24) when compared to pupils in schools inwhich the occurrences of such problems were very frequent.Thus, education policy should concentrate on minimizing pupilbehaviour problems in schools.

(9) T

eachers’ average score: Pupils in schools with teachers who hadhigher subject-matter scores were likely to achieve better inmathematics (0.17, 0.01) and reading (0.12, 0.01) than pupils inschools with teachers with lower subject-matter scores. Thus, ateacher-training and recruitment policy should be put in placeto ensure that all teachers have excellent subject matterknowledge.

Province-level model

When other factors were equal, the following statement can bemade regarding the effect of Teacher average score on mathe-matics and reading achievement at the province-level.

Teachers average score: Pupils in provinces with many teacherswho had higher subject-matter scores were estimated toperform better in mathematics (0.19, 0.06) and reading (0.21,0.06) than pupils in provinces with teachers with lower subject-matter scores.

This would have both short-term as well as long-termimplications for policy. In the short-term, policy shouldconcentrate on a uniform distribution of teachers accordingto their subject matter among the 61 provinces. In the long-term, a teacher-training policy should be put in place to ensurethat all teachers had excellent subject matter knowledge.

Variance partitioning and variance explained

The results in Table 2 show that the percentages of variancesavailable at pupil-, school- and province-levels were 35.7, 50.9 and13.4, respectively, for mathematics, and 43.7, 46.0 and 10.3,respectively, for reading. These percentages of variance of scores atthe various levels of the hierarchy are the maximum amounts ofvariance available at those levels that could be explained insubsequent analyses.

Generally, at the school-level, the variance of pupil scores formathematics (50.9%) followed closely the variance of pupil scorefor reading (46.0%). In these models, the percentages of varianceavailable at the school-level were markedly smaller compared tothe percentages of variance obtained from models developed byHungi (2004) analyzing these data with pupil, class and school asthe levels of analyses. Hungi found that the percentages of varianceavailable at the school-level were 67.8 and 59.9 based on two-levelmodels (pupil and school), and 63.6 and 55.8 based on three-levelmodels (pupil, class and school) for mathematics and reading,respectively.

Thus, from the results in Table 2, it would appear that some ofthe variance allocated to schools in the models developed by Hungi(2004) could be attributed to differences between provinces andthat the variance between schools in Vietnam was not soalarmingly large as implied earlier. Nevertheless, the results inTable 2 still indicate that the variance between schools in Vietnamwas larger compared to what was generally reported at similargrade levels in other developing countries. For example, Willmsand Somers (2001), utilizing data from Grades 3 and 5 pupils from13 Latin American countries, found that the variance betweenschools in mathematics achievement ranged from 19.5% to 41.2%.Therefore, there is need for further studies to look at the Vietnamschooling system to establish exactly what is happening to createthese differences between schools in the achievement of pupils inmathematics and reading.

It can be seen from the results in Table 2 that the predictorsincluded in the final mathematics model explained 12.7% of 35.7%variance available at the pupil-level and that is equal to 4.5% of thetotal variance explained at the pupil-level. Similarly, the predictorsincluded in the final mathematics model explained 14.9% (that is,29.3% of 50.9%) at the school-level and 9.6% (that is, 71.3% of 13.4%)at the province-level. Thus, the total variance explained by thepredictors included in the final mathematics model was4.5 + 14.9 + 9.6 = 29.0%, which left 71.0% of the total varianceunexplained in this model. Likewise, the predictors included in thefinal reading model explained 28.5% of the variance, which left71.5% unexplained. Generally, for both mathematics and reading,the predictors included in the final models explained most of thebetween-province (level-3) variance, about a third of the between-school variance (level-2) and about a tenth of the within-school(level-1) variance.

N. Hungi / Studies in Educational Evaluation 34 (2008) 155–164162

In these models, the total variances explained for mathematicsand reading were marginally larger compared to the totalvariances explained in the models that were developed by Hungi(2004) analyzing these same data at the pupil, classroom andschool-levels. The percentages of total variance explained in themodels that were developed by Hungi were 23.7 and 21.4 based ontwo-level models (pupil and school), and 25.2 and 24.4 based onthree-level models (pupil, class and school) for mathematics andreading, respectively. Thus, in terms of variance explained, it wouldappear that the models developed in the current study weremarginally better than the models developed earlier by Hungi.Nevertheless, the amounts of variances left unexplained were stillvery large (�71%) for both mathematics and reading. Clearly, thereis a need for a further study to examine what other importantfactors were left out of this study. Such factors would assist indevelopment models that were the most appropriate for explain-ing pupil achievement in Vietnam and which maximize the totalvariance explained.

Apart from examining what other important factors were leftout in this study, it would be important to remember that theanalyses carried out by Hungi (2004) revealed that some of thewithin-school variance could be attributed to the differencesbetween the classes that the pupils belonged to. In other words, theinclusion of a class-level in the analyses helped to explain some ofthe pupil-level variance. Consequently, it would be profitable torepeat these analyses based on a four-level model (pupil, class,school and province). Such analyses would be carried out usingMLwiN (Browne, Healy, Cameron, & Charlton, 2001) softwarebecause the current version of HLM program cannot handle morethan three levels of analyses.

Summary and discussions

The purposes of this study was to identify pupil-, school- andprovince-levels factors influencing achievement in mathematicsand reading among Grade 5 pupils in Vietnam. The study used datafrom the 2001 Grade 5 achievement study in Vietnam. This studycomprised 72,660 pupils attending 3635 schools in 61 provinces inVietnam.

In order to achieve the above purpose, a three-level model washypothesized and analyzed using HLM5 software for each of thetwo outcome measures (mathematics and reading). The results ofthe analyses showed that 13 of the 15 pupil-level variablesexamined in this study had some significant effects on mathe-matics achievement. These 13 variables were pupil’s sex, speakingVietnamese, ethnic background, travel time to school, days absent,Grade repetition, sitting place, age in months, books at home,textbook ownership, homework corrected and socioeconomicstatus. All but one (meals per day) of these 13 pupil-level variablesalso had significant influences on achievement in reading. Exceptfor socioeconomic status (with effect sizes of 0.11 in themathematics and reading models) all the other pupil-levelvariables had very small effect sizes (mostly �j0.07j).

At the school-level, the results of the analyses show that of the27 school-level variables examined in this study, nine variableshad significant (p � 0.05) effects on achievement in both mathe-matics and reading. These nine school-level variables were averagemeals per day, average days absent, school library, averagetextbook ownership, average homework corrected, per cent fullday, average socioeconomic status, pupils’ behaviour problems andteachers’ average score. Based on the effect size criterion, theimportant predictors of mathematics achievement were teachers’average score (0.16), average socioeconomic status (0.10) andaverage homework corrected (0.10), while the important pre-

dictors for reading achievement were average socioeconomicstatus (0.12) and teachers’ average score (0.11).

At the province-level, the results of the analyses show that onlyone (teachers’ average score) of the 27 variables examined in thisstudy had significant effects on achievement in mathematics andreading. The effect sizes of teachers’ average score were 0.08 and0.07 in the mathematics and reading models, respectively.

The results of analyses also show several interesting cross-levelinteraction effects between pupil-level variables and school-levelvariables. These interaction effects indicated that travel time toschool had a more negative impact in urban schools and thatsocioeconomic status had more positive impacts in schools withlow average grade repetition, schools in urban areas, schools withmany resources and school with high percentages of femaleteachers.

The results of variance partitioning showed that the percen-tages of variances available at the pupil-, school- and province-levels were 35.7, 50.9 and 13.4 for mathematics, respectively, and43.7, 46.0 and 10.3 for reading, respectively. At the school-level,these results of variance partitioning showed that the variancebetween schools in Vietnam was slightly larger compared to whatwas generally reported at similar grade levels in other developingcountries. However, these analyses revealed that the variancebetween school (while still large) was not as alarming as found in astudy by Hungi (2004) analyzing these data based on two-levelmodels (pupil and school) and three-level models (pupil, class andschool).

For both mathematics and reading, the multilevel modelsdeveloped in this study explained most of the between-province(level-3) variance but only explained about a third of the between-school variance (level-2) and about a tenth of the within-school(level-1) variance. The large percentages of levels 1 and 2 varianceleft unexplained strongly indicated that there were otherimportant pupil, class or school-level factors that were influencingpupils’ achievement in mathematics and reading, which were notincluded in the models developed in this study.

What then were the implications emanating from the results? First,the results revealed large variations (both in mathematics andreading achievement) between schools in Vietnam. This indicatessubstantial inequity between primary schools in Vietnam in termsof the quality of education they offer. Thus, the Vietnameseauthorities need to take measures to facilitate the improvement oflow performing schools with an aim of minimizing the variancebetween primary schools in Vietnam.

Second, from a methodological point of view, only 30% of theoverall variance was explained. This indicates that more thoughtneeds to be given to which other variables should be included inthe next Grade 5 survey at pupil-, school-, and province-levels.There is a need for further identification of other important factorsthat will maximize the total variance explained. It is also suggestedthat it would be profitable to repeat analyses of these data based ona four-level model (pupil, class, school and province) because someof the within-school (i.e. pupil-level) variance could be attributedto the differences between the classes that the pupils belong to (seeHungi, 2004).

Third, the two most important variables that emerged as strongpredictors of achievement were the socioeconomic status of thepupils and the teacher subject-matter knowledge. With the lattervariable, there is much that can be done both in pre-serviceeducation and in-service education. In the 1990s, the MOETabandoned the central examinations and each teacher trainingcollege set its own examinations. Clearly the standards of passingthe examination at the end of teacher training varied fromprovince to province. There was about one teacher training collegeper province and students from those colleges tended to teach in

N. Hungi / Studies in Educational Evaluation 34 (2008) 155–164 163

primary schools in the same provinces. Not only must there bemore subject-matter knowledge taught and better standardsexpected but the allocation of teachers to school could be acrossprovinces. The low socioeconomic pupils tended to come fromhomes of ethnic minorities. Special home intervention projectscould be begun whereby the teachers in these areas are trainedhow to change the parental behaviours in the home so that thechildren receive more encouragement and support for studying. Inthis way the achievement of the low socioeconomic pupils can beraised. This will also deal with the problem of pupil absenteeism.

Fourth, it is for the central MOET as well as the provincialeducational authorities to ensure action for those variables thatproved to be important in predicting the differences amongschools. Already the MOET has promulgated that there should bemore school changing to the full-day system and this will soon bespread across the country. There should be audits of the availabilityof textbooks and all efforts made to ensure that there be a

Appendix A. Variables tested on each level of the hierarchy

Level Variable of interest

Pupil Pupil’s sex (0 = Boy; 1 = Girl)

Speaking Vietnamese (0 = Never; 1 = Sometim

Ethnic background (0 = Kinh; 1 = Others)

Meals per day (0 = One meal; 1 = Two meals;

Travel time to school (0 = Shortest, . . ., 150 = L

Days absent (0 = Lowest, . . ., 30 = Highest)

Grade repetition (0 = Never; 1 = Once; 2 = Twi

Sitting place (0 = Floor; 1 = Bench; 2 = Chair)

Writing place (0 = None; 1 = Share; 2 = Alone)

Extra tuition (0 = 0 h/week, . . ., 20 = 20 h/wee

Age in months (100 = Youngest, . . ., 241 = Old

Number of books at home (0 = Smallest, . . ., 2

Textbook ownership (0 = No; 1 = Share; 2 = Ow

Homework corrected (Large values = More ho

Socioeconomic status (Large values = High SES

School Proportion of girls

Average speaking Vietnamese

Proportion of Kinh

Average meals per day

Average travel time

Average days absent

Average grade repetition

Average sitting place

Average writing place

Average extra tuition

Average age in months

Average number of books at home

Average textbook ownership

Average homework corrected

Average socioeconomic status

Per cent full day

School library (0 = No library; 1 = Has library)

Pupils’ behaviour problems

Teachers’ behaviour problems

Teachers’ average score

Management course (0 = No training; 1 = Has

School location (0 = Isolated; 1 = Rural; 2 = Ur

Average teacher training

School resources

Per cent female teachers

Parental contribution

Pupil–teacher ratio

Province Proportion of girls

Average speaking Vietnamese

Average ethnic background

Average meals per day

Average travel time

Average days absent

Average grade repetition

Average sitting place

maximum of two pupils per textbook and eventually one pertextbook. More reading materials for borrowing should be madeavailable either in classroom or school libraries. One majorpredictor was the average socioeconomic status level of the schooland there were differences. This is not easy to change but it isimportant that this factor was identified through this research andthat the MOET is aware of it. Only if the authorities are aware ofthis problem can they plan efforts to have more likeness in schoolintakes. Through teacher training and school management coursesmuch can be done to ensure that teachers correct homework, anddeal quickly with pupil misbehaviours.

It is not easy to reform systems of education. Vietnam has maderapid economic advances in the last 10 years and the share of thegovernment budget to education will surely increase. But, changemust be planned and must not be too rapid. It tales time to trainand retrain the school heads and teachers to cope with the plannedreforms.

Variables tested in HLM

PSEX

es; 2 = Always) PVIET

PETHNIC

2 = Three or more meals) PMEAL

ongest) PTRAVEL

PABSENT

ce; 3 = Thrice or more) PGREP

PSIT

PWRITE

k) PEXTUMAT, PEXTUVIEa

est) PAGEMON

50 = Largest) PBOOKHM

n) PTXMSHR, PTXVSHRa

mework) PMATHHW, PREADHWa

) PSESb

PSEX_2

PVIET_2

PETHNI_2

PMEAL_2

PTRAVE_2

PABSEN_2

PGREP_2

PSIT_2

PWRITE_2

PEXTUM_2, PEXTUV_2a

PAGEM_2

PBOOK_2

PTXMS_2, PTXVS_2a

PMATH_2, PREAD_2a

PSES_2

PFULD_2

SCHLI_2

SPBHP_2b

STBHP_2b

TMA500_2, TRD500_2a

training) SMANAG_2

ban) SLOCA_2

STT_1_2

SREST_2

SFEMT_2

SPARC_2b

SPUPT_2

PSEX_3

PVIET_3

PETHNI_3

PMEAL_3

PTRAVE_3

PABSEN_3

PGREP_3

PSIT_3

(Continued )

Level Variable of interest Variables tested in HLM

Average writing place PWRITE_3

Average extra tuition PEXTUM_3, PEXTUV_3

Average age in months PAGEM_3

Average number of books at home PBOOK_3

Average textbook ownership PTXMS_3, PTXVS_3a

Average homework corrected PMATH_3, PREAD_3a

Average socioeconomic status PSES_3

Average per cent full day PFULD_3

Proportion of school library SCHLI_3

Average pupils’ behaviour problems SPBHP_3

Average teachers’ behaviour problems STBHP_3

Teachers’ average score TMA500_3, TRD500_3a

Average management course SMANAG_3

Average school location SLOCA_3

Average teacher training STT_3_3

Average school resources SREST_3

Average per cent female teachers SFEMT_3

Average parental contribution SPARC_3

Average pupil–teacher ratio SPUPT_3

a Variable listed first was included in the mathematics model while the second variable was included in the reading model.b This is a composite variable suffix ‘_2’ is used to indicate a school-level variable and suffix ‘_3’ is used to indicate a province-level variable.

N. Hungi / Studies in Educational Evaluation 34 (2008) 155–164164

References

Aiken, L. S., & West, S. G. (1996). Multiple regression: Testing and interpreting interactions.Newbury Park, CA: Sage Publications.

Browne, W., Healy, M., Cameron, B., & Charlton, C. (2001). MLwiN (Version 1.10.0007).London: University of London, Institute of Education.

Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and dataanalysis methods. Newbury Park, CA: Sage Publications.

Hox, J. J. (1995). Applied multilevel analysis. Amsterdam: TT-Publikaties.Hungi, N. (2004). An HLM analysis of Grade 5 survey data. In Human Development

Sector Reports (Report No. 29787), East Asia and Pacific Region, Vietnam: Readingand mathematics assessment study (Vol. 2, pp. 260–298). Hanoi: The World Bank.

Kreft, I. G. G., De Leeuw, J., & Aiken, L. S. (1995). The effects of different formsof centering in hierarchical linear models. Multivariate Behavioral Research,30(1), 1–21.

Lietz, P. (1996). Changes in reading comprehension across cultures and over time.Minister: Waxman.

Raudenbush, S. W., & Bryk, A. S. (1994). Hierarchical linear models. In T. Husen & T. N.Postlethwaite (Eds.), International encyclopedia of education: Research and studies(2nd ed., pp. 2590–2596). Oxford: Pergamon.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and dataanalysis methods (2nd ed.). Thousand Oaks, CA: Sage Publications.

Raudenbush, S. W., Bryk, A. S., Cheong, Y. F., & Congdon, R. T. (2000a). HLM 5:Hierarchical linear and nonlinear modeling (Version 5.01.2067.1). Lincolnwood, IL:Scientific Software International.

Raudenbush, S. W., Bryk, A. S., Cheong, Y. F., & Congdon, R. T. (2000b). HLM 5:Hierarchical linear and nonlinear modeling (user guide). Lincolnwood, IL: ScientificSoftware International.

Willms, D. J., & Somers, M. A. (2001). Family, classroom, and school effects on children’seducational outcomes in Latin America. School Effectiveness and School Improve-ment, 12(4), 409–445.

Njora Hungi completed an honours degree in Agricultural Education and

Extension in Egerton University, Kenya, and taught in rural high schools before

undertaking masters and doctoral studies at Flinders University into the

estimation of value added effects in South Australian schools. He has recently

moved to work on large scale testing programs at the Australian Council for

Educational Research in the Sydney Office.