Literacy Boost Metro Manila - Resource Centre · 1 Literacy Boost Metro Manila Endline Report September 2013 Adrian Badiable, Jarret Guajardo, Rachael Fermin, and Elvie Jo C. Robis

1

Literacy Boost

Metro Manila

Endline Report September 2013

Adrian Badiable, Jarret Guajardo, Rachael Fermin, and Elvie Jo C. Robis

Acknowledgement to Richelle Gutierrez Dela Pena, Literacy Boost

community volunteers, and the teachers and students of Cielito Zamora

Memorial School for their hard work and excellent support

Special thanks to our team of enumerators:

Maria Luisa Lazaga, Monaliza Leonardo, Francis Masilang, Dave Mark Valerio,

Marly Guttierez, Anna Rose Pineda, Lea Llanita, Hanni Francisco, Francis

Macalinao, Jojimar Tabiolo, Mario Padida, Jr., Abigail Mallari, Ivy Gaviola, Geraldine

Rodriguez.

1

Table of Contents

Executive Summary .......................................................................................................................................... 1

Introduction ....................................................................................................................................................... 5

Context ........................................................................................................................................................... 6

Implementation History .............................................................................................................................. 6

Methodology .................................................................................................................................................. 7

Sample Selection .......................................................................................................................................... 7

Instruments and Measurement .................................................................................................................. 8

Analysis ......................................................................................................................................................... 8

Attrition Analysis .............................................................................................................................................. 8

Endline Descriptive Statistics Analysis ...................................................................................................... 10

Student Characteristics ............................................................................................................................ 10

Home Literacy Environment Data ......................................................................................................... 11

Literacy Boost Participation .................................................................................................................... 14

Sex Differences .......................................................................................................................................... 15

Impact on Emergent Literacy Skills ............................................................................................................ 15

Overall Impact ............................................................................................................................................ 15

Evidence from Focus Groups .................................................................................................................. 19

Equity Analysis ............................................................................................................................................ 21

Individual Skill Analysis.................................................................................................................................. 24

Concepts about Print .................................................................................................................................... 24

Letter Awareness .......................................................................................................................................... 25

Word Recognition: Most Used Words ....................................................................................................... 26

Readers and Nonreaders ............................................................................................................................ 27

Fluency and Accuracy ................................................................................................................................... 28

Reading Comprehension .............................................................................................................................. 30

One New Way to Look at Data: Readers with Comprehension ............................................................. 33

Trends in Reading Skill Data from Multilevel Regression Models ...................................................... 34

LB Reading Activities and Reading Achievement ................................................................................ 34

Literacy Boost Home Literacy Environment Channels for Reading Skill Improvement ............ 35

2

Conclusion ...................................................................................................................................................... 36

Appendix A. Literacy Boost Activities prior to Endline Assessment............................................. 38

Appendix B: Inter-rater Reliability ......................................................................................................... 40

Appendix C: Student Background Characteristics ............................................................................. 41

Appendix D: Multilevel Regression Analysis ....................................................................................... 43

Appendix E: Details of Focus Group Discussions (FGDs) with Primary Stakeholders ............. 45

1

Executive Summary

This report examines the results of a follow-up student background survey and reading

assessment conducted in June 2013 as part of Save the Children’s Literacy Boost program. The

Literacy Boost program includes teacher training, community reading activities, and age-

appropriate local language material creation to support emergent literacy skills among early-

grade children. In September 2012 a baseline survey and reading assessment was conducted

before the commencement of these activities, covering 261 grade 2 students throughout 14

sections of Cielito Zamora Memorial Elementary School in the Caloocan district of Manila in

the Philippines. The 14 grade 2 sections from which students were assessed were randomly

split into seven ‘full intervention’ sections designated to receive Literacy Boost, and seven

additional sections received no intervention at all to the extent possible. Following up with as

many of the original students as possible, the June 2013 survey and reading assessment of 283

grade 3 students serves to measure how students’ reading skills have changed over time as well

as to estimate the impact of eight months of the Literacy Boost intervention.

However, three issues undermined the ability to estimate program impact. First, randomization

of sections placed the two highest-performing sections in the LB group. Second, due to the fact

that both intervention and comparison groups were together in the same school, it is inevitable

that a degree of sample contamination occurred. Third and finally, by the time of the endline

assessment it was possible to find only 168 of the original 261 students assessed at baseline

(64%). Thus, an additional 115 students were randomly resampled who had been enrolled in

grade 2 with the cohort of students assessed at baseline in order to improve the sample size.

Attrition Analysis

The attrition rate is significantly higher among Literacy boost students (43%) than among

comparison students (28%). Using a form of transportation other than walking to arrive at

school and having exchanged books with others in the community in the past week were found

to be significantly associated with a higher likelihood of attrition. More information is necessary

to explain these findings. In addition, students with lower baseline Filipino most used word

scores and students who did not qualify readers in English at baseline were also significantly

more likely to attrit at endline. Students who struggle with their basic Filipino and

English reading skills may be more at-risk of not attending or dropping out of

school, and thus may need more special support to continue their stronger peers.

Background Characteristics and Home Literacy Environment

Literacy Boost and comparison children are remarkably similar on average, enough for a

difference-in-difference impact evaluation of the Literacy Boost intervention. Children’s self-

reported presence of various types of print in the household increased for most types of print

over time. However, many students still face a lack of variety of reading materials at home, and

also low levels of household members telling stories, singing, and playing games with students.

2

Literacy Boost should redouble its Parental Awareness Workshop efforts in order

to improve the household literacy environment for students.

In terms of student literacy habits, more children reported learning to read outside of school or

with family community members as well as having read to another person in the community in

the week prior to the assessment. A slightly lower proportion of students reported exchanging

books with others, similar to the proportion who reported borrowing books from the school

library at endline. No statistically significant differences exist between Literacy Boost and

comparison groups in terms of either student literacy habits or community activity

participation. It is likely that this is a product of both sample contamination as well as

small sample size.

Participation in Literacy Boost community activities is low among all students, and more

efforts should be made to expand Reading Camp coverage in terms of geography,

scheduling, and number of volunteers. In addition, more qualitative information

will need to be gathered to explain the even lower Book Bank borrowing rates

among students.

Impact Evaluation of Literacy Boost

With the exception of proportion of English readers with comprehension, no significant impact

is found overall between Literacy Boost and comparison groups. While results are not

statistically significant in the vast majority of skills, the fact that effect sizes are

either positive or practically zero when excluding the two high-performing Literacy

Boost sections indicates that the Literacy Boost program may have had a small

degree of impact overall. Beyond deficiencies in the program itself, sample

contamination may have undermined the ability to perceive a difference between

Literacy Boost and comparison groups.

No benchmarks set at baseline were met. While many benchmarks are reasonable,

those benchmarks set at 100% of items are unlikely to ever be reached and thus

letter knowledge and Filipino most-used words may be considered nearly-achieved.

However, more work remains to be done for the majority of other skills, especially

giving children enough exposure to actual books to increase their mastery of

concepts about print. It will be important to better understand why limited or no

impact was found. This can be done through combining evidence from this

quantitative analysis with qualitative evidence from focus group discussions. The

Literacy Boost program may need to be improved and lessons from the first year

of piloting the program applied so that Literacy Boost can demonstrate results for

students in its second phase with a more rigorous research design.

3

From focus group discussions conducted with students, teachers, community facilitators, and

parents, there is qualitative evidence that Literacy Boost has made a positive impact on

processes and learning outcomes. It does not appear that students attending Reading Camps

are either those with the highest or lowest baseline scores and thus the major challenge is that

not all children attend Reading Camps and therefore have the opportunity to benefit. From

analysis of focus group feedback, Literacy Boost should:

expand Reading Camp opportunities for all children by recruiting additional

volunteers and expanding coverage in terms of geography and scheduling

identify safe Reading Camp venues with adequate facilities

facilitate the provision of as many new titles as possible at Book Banks

better educate all stakeholders that the objective of the Reading Camps is to

provide a more informal and fun environment for children to practice their

reading skills, not an extension of classroom time

encourage parents to help their children with a variety of aspects of reading,

especially in terms of asking comprehension questions

further investigate whether or not messages related to literacy are getting

through to parents, and how these messages can be better conveyed

invest more in advertising Parental Awareness Workshops, and make

accommodations such as a young children’s play area at the workshop so

that those with young children to be able to attend

offer teachers refresher trainings and provide new titles for school libraries

Equity Analysis

The strongest baseline drivers of inequality in this context appear to be SES and HLE, followed

by work and chore load, although gender is also an issue with boys scoring significantly lower

than girls in a few skills. Literacy Boost helped boys catch up to girls in a few lower-order skills

such as concepts about print and Filipino and English most-used words. Suggested activities,

games, and materials should be provided to teachers and community volunteers to

engage disadvantaged groups of children, and perhaps extra remedial help can be

offered to students at-risk of falling behind.

Student Skill Profile

Based on analysis of students’ individual skills, Literacy Boost programming should:

include familiarization on parts of a story to participants of Reading Camps

train teachers and community reading activity facilitators to ensure all

students know all letters of the alphabet especially ñ, ng, l, q, j, & f.

highlight letter knowledge and phonemic awareness during Reading Camp

Sessions, ensuring active participation of all students, not just those that

know their letters

4

continue providing technical support and guidance to teachers to help

students learn and practice their decoding skills as well as provide ample

opportunity inside and outside of the classroom for students to practice

reading and memorizing basic, frequently used words such as these

give students ample time to practice reading through such initiatives as

reading buddies in order to build their automaticity

further enhance readers’ opportunity for practice of reading enjoyable and

relevant materials

facilitate the provision of mother-tongue based reading materials as a good

complement to the implementation of Mother-Tongue-Based Multilingual

Education program of DepEd and its schools

help teachers and community volunteers focus on asking different types of

questions before, during, and after stories so that students can better

develop their literal and deeper comprehension skills, especially

understanding the details of the story, making inferences based on the

information in the story, and summarizing the story

Looking at the relationship between background characteristics and reading skills, older

children and those from larger households did not improve their scores as much between

baseline and endline as their younger peers and those students from smaller households. Both

older children and those from larger households may need special remedial help in

order to keep pace with other children.

There were no significant relationships between reported participation and reading skill gains

over time. However, frequency of Reading Camp attendance was positively associated with

gains in letter knowledge and Filipino fluency. This is additional evidence that Literacy

Boost Reading Camps may have helped students improve some of their reading

skills.

Finally, there was no statistically significant association between Literacy Boost and potential

home literacy environment channels for impact. This may be due to potential sample

contamination between Literacy Boost and comparison groups, but students are

also not reporting improvements in their household literacy environment and

literacy habits in most measures. Literacy Boost will need to apply lessons from

focus groups held with parents to determine what more can be done to help

parents better engage in literacy acquisition promoting activities.

5

Introduction

This report examines the results of a follow-up student background survey and reading

assessment conducted in June 2013 as part of Save the Children’s Literacy Boost program. The

Literacy Boost program includes teacher training, community reading activities, and age-

appropriate local language material creation to support emergent literacy skills among early-

grade children. In September 2012 a baseline survey and reading assessment was conducted

before the commencement of these activities, covering 261 grade 2 students throughout 14

sections of Cielito Zamora Memorial Elementary School in the Caloocan district of Manila in

the Philippines. The 14 grade 2 sections from which students were assessed were randomly

split into seven ‘full intervention’ sections designated to receive the teacher training, community

activities, and material creation components of Literacy Boost, and seven additional sections

received no intervention at all to the extent possible. Following up with as many of the original

students as possible, the June 2013 survey and reading assessment of 283 grade 3 students

serves to measure how students’ reading skills have changed over time as well as to estimate

the impact of eight months of the Literacy Boost intervention.

Skills tested in this assessment include concepts about print (CAP), letter awareness, single

word reading of most used words (MUW), reading fluency, reading accuracy, reading

comprehension, and listening comprehension. The key research questions to be explored in

this report include the following:

1. How has the sample of students changed over time?

o Are the students who were able to be found at endline different than those who

were not able to be found? If so, how?

o Did the attrition rate differ between intervention and comparison sections?

2. Of the students who were able to be found at endline, how comparable are baseline

background characteristics and reading skills in intervention sections versus comparison

sections?

3. What can the endline assessment tell us about students’ reading skills?

o What does this mean for continuing Literacy Boost programming in this school?

4. Did the Literacy Boost program exhibit impact on students’ reading skills?

o For which types of students was impact the greatest/least?

o Does this impact result in more equitable outcomes for traditionally

disadvantaged groups?

5. How does students’ development of reading skills over time vary by student

background and community literacy environment?

o What does this mean for targeting Literacy Boost’s various intervention

components?

To investigate these questions, this report first describes the context and implementation

history of Literacy Boost in the Cielito Zamora Memorial School. Next, this report gives an

overview of the research methods used; including sampling, measurement, and analysis. The

report will then analyze the attrition of the sample over time and how intervention and

comparison groups have or have not remained statistically similar. The report will then present

results from impact analysis investigating the extent to which Literacy Boost appears to have

6

improved students’ reading skills. After this, students’ endline scores for each of the reading

skills will be analyzed to determine which skills students have mastered and which require

additional improvement. Finally, the report will investigate any correlations between baseline-

endline reading skill development and student background or community literacy environment

variables using multilevel regression analysis.

Context

The urban district of Caloocan North is located in the northern part of the Metro Manila area

in the Philippines. The vast majority of the population speaks Filipino (Tagalog)1 as their native

language. Caloocan is one of the more economically disadvantaged in the Metro Manila area;

and school was disrupted, property damaged, and residents killed in typhoon flooding in the

period immediately preceding the assessment. Other challenges faced by elementary school

students in this area include being in large classes, having short contact time with teachers

(approximately 4 hours per day), and exposure to violence in the community in addition to

extremes of pollution and a generally unsanitary environment.

Implementation History

All Literacy Boost and comparison sections in this school received the same School Health and

Nutrition interventions from Save the Children. See appendix A for a complete timeline of

Literacy Boost intervention components.

Country Office staff report the following challenges with respect to implementation:

- The original idea with the consultants was to train a core group who will serve as

trainers for the Reading Awareness Workshops for parents and the Reading Camp for

children. This core group was composed of child advocate volunteers from former

sponsorship impact areas in Paranaque, Taguig and Pateros, who have had the

experience implementing reading programs in their respective areas. Initially, they

served as support to the small group of trained volunteers from Barangay 177 who

were inexperienced and lacking in confidence in the first Reading Awareness

Workshops for parents. But because of the geographic distance of the core group’s

residence from the Caloocan impact area, this arrangement was not sustained. There

were only 53 adults and 33 youth who were trained.

- As the months went by, another challenge was the turnover of the volunteers, who

found regular jobs, or had to quit volunteering because of school.

- A perennial challenge for implementers of a program which includes parents is their

involvement in activities. They would allow their children to attend the Reading Camp,

but would not attend or commit to the Reading Awareness Workshops.

- Although Reading Camps began in November, there were not a lot of children then

attending then. Only 4 Reading Camps (RCs) were open in November 10 (2 RCs) and

Nov. 24 (2 more RCs), with only about 30 children in each RC site. More children

1 This report will use the term Filipino, as the reading assessment used is based on the grade 2 Filipino textbook. The differences between Filipino and Tagalog are minor, as Filipino is essentially a formalized version of Tagalog.

7

started to pour in after the Christmas vacation when more kids knew about the LB

program.

Methodology

Sample Selection

The sample for the September 2012 baseline assessment encompassed 261 grade 2 students,

divided between 14 sections in the same large urban school of Cielito Zamora Memorial

Elementary. In each of the 14 sections, 20 children in grade 2 were sampled. Ten boys and ten

girls were randomly selected where there were more than 20 children present in a section on

the day of assessment. Due to random assessor error in recording the correct student code, 19

observations from the baseline sample were dropped, yielding the final baseline sample size of

261 students. Of the 14 grade 2 sections, seven were randomly assigned to receive the full

Literacy Boost intervention (n = 133) and seven were likewise randomly assigned to receive no

Literacy Boost intervention (n = 128) as the comparison sections.

However, three issues with the research design merit discussion. First, randomization was

performed without taking into account the fact that students are effectively randomly

distributed among grade 2 sections except in the case of two sections. Students in these two

sections were assigned to these sections due to their high academic performance. During the

randomization, by chance both of these sections were assigned to the Literacy Boost group.

Therefore, during impact analysis tests will be conducted both including and

excluding these two high-performing sections to determine whether this affects the

results.

Second, due to the fact that both intervention and comparison groups were together in the

same school, it is inevitable that a degree of sample contamination occurred. With regard to

teacher training, only teachers from Literacy Boost sections were trained (the only anomaly

was that one Literacy Boost section teacher did not receive training due health issues).

However, teachers may have informally exchanged ideas and techniques related to Literacy

Boost. Furthermore, with regard to community activities an equally (small) proportion of

comparison students mentioned borrowing books from Literacy Boost Book Banks as Literacy

Boost students, and only a slightly lower proportion of comparison students reported having

attended a Reading Camp as Literacy Boost students. Impact analysis will attempt to

control for the community activity sample contamination, but cannot control for

any sample contamination from teacher training or other spillover effects.

Third and finally, by the time of the endline assessment it was possible to find only 168 of the

original 261 students assessed at baseline (64%). Save the Children Philippines staff believe that,

due to the mobile nature of families in the Caloocan area, a large proportion of students moved

to other areas (and perhaps schools) as their families searched for work. Thus, an additional

8

115 students were randomly resampled who had been enrolled in grade 2 with the cohort of

students assessed at baseline, in order to bring the total number of endline students to 20

assessed per each of the 14 grade 2 sections.

Instruments and Measurement

For the student assessment, all students in the sample were asked about their background

characteristics (age, household possessions, household building materials, health indicators,

etc.). Students were also asked about their family members’ reading habits (who they had seen

reading in the week prior to the assessment, who had read to them, etc) as well as about their

participation in Literacy Boost community activities such as Reading Camps attendance and

Book Bank borrowing.

After collecting this background data, all students were also given a literacy test composed of

six components administered through five sub-tests: concepts about print, letter awareness,

single word recognition (reading of most used words), reading fluency & accuracy (words per

minute read correctly and total percentage of passage read correctly; both within the same sub-

test), and a set of comprehension questions linked to the fluency & accuracy passage. The same

set of comprehension questions were administered for both those students who could read

independently (reading comprehension) and those who could not and thus had the assessor

read to them (listening comprehension). Children who could read independently (readers)

were defined as those who could read at least five words correct in the first 30 seconds of the

fluency & accuracy sub-test and finish the full passage. Those children who could not read

independently (nonreaders) were those who either read less than five words correct in the first

30 seconds of the fluency & accuracy passage or who could not finish reading the passage.

The most used words, fluency & accuracy, and comprehension sub-tests were conducted in

both Filipino and English, the former being the native language of nearly all of the students. All

assessment instructions were given in Filipino. For a discussion of inter-rater reliability, please

see Appendix B.

Analysis

This report will use comparison of means through clustered t-tests to assess the comparability

of intervention and comparison groups and consider differences between boys and girls scores.

Finally, this report will consider relationships between reading skills and background factors

using multilevel regression models that account for clustering in schools.

Attrition Analysis

This endline study assessed grade 3 students from the same seven Literacy Boost grade 2

sections and seven grade 2 comparison sections from Cielito Zamora Memorial School that

were assessed in the 2012 sample. At baseline 261 students were assessed, but only 168

9

students could be located at endline2. In order to determine whether or not the Literacy Boost

and comparison endline sample is still comparable, it is first necessary to test whether attrition

rates were different between the two sub-samples. Using clustered t-tests, the attrition rate is

significantly higher among Literacy boost students (43%) than among comparison students

(28%). This finding holds even excluding the two high-performing Literacy Boost sections, and

also remains marginally insignificant even controlling for a variety of baseline background

characteristics. Thus, it will be necessary to retest the comparability of the remaining

sample of Literacy Boost and comparison students.

Of all the variables collected about students’ socioeconomic status, home literacy environment,

and reading skills, two background variables and two reading outcome variables collected at

baseline robustly predict the type of student who might have attritted at endline. The technique

used to arrive at these findings was a multilevel logistic regression analysis, with students

clustered in schools.

Using a form of transportation other than walking to arrive at school was found to be

significantly positively associated with a higher likelihood of attrition. More information is

needed to understand the explanation for this, but not walking to school is significantly

correlated with socio-economic status (SES) and these students’ families may have the means to

relocate to other areas. Another explanation is that students not walking to school may live

farther from the school and may drop out if the distance becomes too much of a burden. The

other background variable robustly associated with higher likelihood of attrition is having

exchanged books with others in the community in the week prior to the assessment. More

information is necessary to explain this unintuitive finding as well. Finally, students with lower

baseline Filipino most used word scores and students who did not qualify readers in English at

baseline were also significantly more likely to attrit at endline. Note that these associations are

correlational only and causation cannot be established. However, students who struggle

with their basic Filipino and English reading skills may be more at-risk of not

attending or dropping out of school, and thus may need more special support to

continue their stronger peers.

Figure 1 below displays the relationship between two of these variables, walking to school and

Filipino most used word score, and the likelihood of attrition at endline.

2 Please note, this does not necessarily mean that students dropped out. There are many possible reasons for

student absence, and we can only note that the students were absent on the particular day of the assessment. The

analysis presented here does not establish why students were absent.

10

Endline Descriptive Statistics Analysis

Student Characteristics

Table 1 summarizes student background data, presenting the averages for all Literacy Boost and

comparison students as collected at baseline and endline. Of the sample of 168 students who

were assessed at both baseline and endline, excluding the 24 students from the two high-

performing sections, t-tests assuming unequal variances reveal that Literacy Boost and

comparison children are remarkably similar on average. Only in terms of chore frequency and

breakfast size are the two groups significantly different (see Appendix C for detailed statistics

on students’ time and health). It is possible that this small number of differences is due purely to

chance. Thus, children in Literacy Boost and comparison groups are similar enough

to perform a difference-in-difference impact evaluation of the Literacy Boost

intervention.

0%

20%

40%

60%

80%

100%

0 10 20

Pre

dic

ted

like

liho

od

of

attr

itio

n

Baseline Filipino MUW score

Figure 1: Likelihood of Attrition by How Student Arrives at School and

Filipino MUW Score

Does not walk toschool

Walks to school

11

Table 1: Student Background Characteristics by Sample Group (excluding sections 1 & 2)

Baseline Endline

Variable LB Control Sig. N LB Control Sig. N

% Female 44% 50%

144

Age in years 7.6 7.4

140 8.5 8.2

144

Walks to school 73% 75%

144

Previously attended ECD 81% 91%

139 76% 78%

141

Years of ECD 1.2 1.2

94 2.2 2.3

81

Has repeated a grade 14% 12%

144 14% 12%

142

# of household members 3.5 3.0

144 2.9 3.2

144

SES index 2.5 2.6

140 2.5 2.6

144

Chores/work index 1.7 1.7

144 2.5 2.4

144

Frequency of work1 1.0 1.1

36

Frequency of chores1 2.3 2.5 * 142

Frequency of chores and work1

3.7 3.3

144

Frequency of study1 2.7 2.7

142 3.4 3.4

144

Frequency of play1

2.7 2.4

144 * Statistically significant difference between Literacy Boost and comparison groups at p<0.05 1 Students were asked whether they spend no time (0), a little time (1), or a lot of time (2) on the activity in

question for each time period of before school, after school, and on the weekend. If necessary, it was clarified with

students that a little time meant less than two hours, and a lot of time meant two or more hours.

In terms of children’s time (see Appendix C for details), the only significant difference between

Literacy Boost and comparison students is that comparison students report spending less time

on average on work or chores after school than Literacy Boost students.

Home Literacy Environment Data

Just as important as investigating children’s school and classroom environment is examining

their home literacy environment (HLE). The HLE is not only about materials in the home, but

how those materials are used to engage the child in reading and learning. Hess and Holloway

(1984) identified five dimensions of the home literacy environment that are theoretically related

to reading achievement in children. The first is value placed on literacy, which the Literacy Boost

assessment operationalizes by asking children whether they see anyone reading at home. The

second is press for achievement, is operationalized as individuals telling the student to study. The

third is the availability and use of reading materials, operationalized as the amount of printed

materials at home. The fourth dimension is reading with children, operationalized by asking the

12

children whether anyone reads to them at home. The last is opportunities for verbal interaction,

operationalized as family members telling stories to learners.

In terms of the third dimension of availability and use of reading materials, Figure 2 below shows

that children’s self-reported presence of various types of print in the household increased for

most types of print over time (except for textbooks, for which prevalence approached 100% at

both baseline and endline). The increase was most marked for religious materials. Endline

prevalence of reading materials was similar between Literacy Boost and comparison groups

with the possible exception of presence of storybooks in the household. This difference is

marginally insignificant, meaning there is a greater chance that this difference was found by

chance. It is reported here because it indicates that for some reason more comparison students

reported having storybooks in their homes, which may fit with evidence displayed in Figure 3

that comparison students report a significantly larger percentage of household members seen

reading than in Literacy Boost households.

~Significant difference from control group at baseline at p<0.1

Figures 2 and 3 indicate that many students still face a lack of variety of reading materials at

home, and also low levels of household members telling stories, singing, and playing games with

students. Literacy Boost should redouble its Parental Awareness Workshop efforts

in order to improve the household literacy environment for students.

31%~ 33% 37%

23%

64%

46% 39% 39%

26%

53%

0%

10%

20%

30%

40%

50%

60%

70%

Storybooks Coloring books Newspapers Magazines Religiousmaterials

% o

f sa

mp

le

Figure 2: Types of Reading Materials Reported in the Household, by Sample Group (excluding

high-performing sections)

Baseline LB

Endline LB

Baseline Control

Endline Control

13

*Significant difference from control group at baseline at p<0.05

In terms of comparability between Literacy Boost and comparison samples, it

appears that comparison students may have a slight advantage in terms of

storybooks in the household and percentage of household members seen reading.

These differences did not exist at baseline. In fact, the finding about percentage of

household members seen reading comes about because that percentage actually

dropped over time among the Literacy Boost sample. More information is needed

to explain this finding, but household literacy environment will nevertheless be

controlled for during impact analysis.

Children were also asked about their own literacy habits, and Figure 4 below displays modest

change in the three indicators of this area over time. More children reported learning to read

outside of school or with family community members as well as having read to another person

in the community in the week prior to the assessment. A slightly lower proportion of students

reported exchanging books with others, similar to the proportion who reported borrowing

books from the school library at endline. No statistically significant differences exist between

Literacy Boost and comparison groups. This may be due to sample contamination or to

the indirect link between Literacy Boost programming and students’ literacy

habits.

48%*

79%

44% 42% 32%

56% 62%

82%

47% 42% 30%

59%

0%

20%

40%

60%

80%

100%

Seenreading

Encouragingstudent to

study

Reading tothe student

Telling storyto the

student

Singing tothe student

Playinggame withthe student

ave

rage

% o

f h

ou

seh

old

me

mb

ers

Figure 3: Proportion of Household Members

Engaging in Literacy Activities, by Sample Group (excluding high-performing sections)

Baseline LB

Endline LB

Baseline Control

Endline Control

14

Note that sharing books with others in the community in the past week was not collected at endline

Literacy Boost Participation

The last set of background questions asked children about their participation in various Literacy

Boost community activities. Based on the information presented in Table 2, participation in

Literacy Boost community activities is low, and more efforts should be made to expand

Reading Camp coverage in terms of geography, scheduling, and number of

volunteers. In addition, more qualitative information will need to be gathered to

explain the even lower Book Bank borrowing rates among students.

77%

17%

44%

86%

13%

51%

14%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Reports learning toread primarily

outside ofschool/withcommunitymembers

Exchanged bookswith others in

community in pastweek

Read to others incommunity in past

week

Borrows books fromschool library

% o

f sa

mp

le

Figure 4: Student Literacy Habits (excluding high-performing sections)

Baseline

Endline

15

Table 2: Literacy Boost Community Activity Participation by Sample Group (entire endline

sample)

Variable LB Control Sig. N

Has borrowed book from Book Banks 7% 7%

270

# of times borrowed Book Bank book per week (Book

Bank borrowers) 2.1 1.2

13

Has attended Reading Camp 22% 17%

273

Frequency of Reading Camp attendance (RC attendees)a 2.7 2.7

53

How many other children attended Reading Camp (RC

attendees) 27 21

18

Missed a Reading Camp session (RC attendees) 43% 78% ** 51

Did make and take at Reading Camp (RC attendees) 55% 61%

52

Frequency of make and take for (RC attendees) 0.9 0.8 52 ** Statistically significant difference between Literacy Boost and comparison groups at p<0.01 level a 1 = Once, 2 = Rarely, 3 = Many times, 4 = Always

Strangely, although participation is often higher among the Literacy Boost sample than the

comparison sample, there were no statistically significant differences between the two groups,

with the exception of the percentage of students who reported missing a Reading Camp

session. It is likely the findings of similar participation rates between Literacy Boost

and comparison students is a product of both sample contamination as well as

small sample size.

Sex Differences

There are no statistically significant differences between Literacy Boost and comparison

students for the majority of background variables. In terms of students’ time, it appears that

boys may be more likely to engage in play other than computer games than girls, and spend

more time playing before school and on the weekends. Girls may spend more time on work or

chores on the weekend, and spend more time studying overall.

With home literacy environment, girls may be more likely to have read to others in the

community in the week prior to the assessment. Finally, there are no statistically significant

differences between boys and girls regarding Literacy Boost community activity participation.

Impact on Emergent Literacy Skills

Overall Impact

In this section we examine gain scores in each literacy skill to assess the impact of Literacy

Boost on student learning outcomes. In Tables 3 and 4 below, baseline & endline data are

16

presented for both Literacy Boost and comparison students. Furthermore, the change from

baseline to endline is presented (gain scores), along with the initial benchmark3 set during the

baseline assessment and whether the benchmark was met. Clustered t-tests are were used to

test whether the findings are generalizable to the population, and standardized effect sizes4 are

also reported to give a comparable sense of magnitude of the difference in Literacy Boost and

comparison gain scores across all skills.

With the exception of proportion of English readers with comprehension,5 no significant impact

is found overall between Literacy Boost and comparison groups when excluding the two high-

performing sections from the analysis. The letter knowledge gain score difference between

Literacy Boost and comparison groups is marginally insignificant when not controlling for other

factors (see Appendix D for regression tables), but becomes completely insignificant once

controlling for these factors. All other overall results, whether in terms of statistical significance

or effect size (magnitude), do not change by much whether controlling for other factors. As

expected, including the high-performing sections in the analysis raises the effect sizes of the

different performance between Literacy Boost and comparison groups, but does not change

statistical significance.

3 A Note about Benchmarks: While Save the Children has used this approach to reading assessment in Bangladesh,

Ethiopia, Guatemala, Haiti, Malawi, Mali, Mozambique, Nepal, the Philippines, Sri Lanka, South Africa, Vietnam,

Uganda, Yemen, and Zimbabwe, comparison across countries and languages is less helpful than more detailed

contextual information for setting expectations of impact. For each measure used in these assessments, the

baseline established the upper end of the range of scores (the 75th percentile) as a benchmark, or a reasonable

estimate of what is currently possible among these children. 4 The effect size is a measure of the magnitude of an observed difference, expressed in standard deviations in order

to compare across different types of measures. So, as Literacy Boost exhibited a statistically significant effect size of

0.46 in RNGPS for fluency, this means that Literacy Boost children improved almost a half standard deviation more

than comparison children improved. 5 To calculate the definition of ‘readers with comprehension,’ the Literacy Boost research team standard definition

of at least 80% of reading comprehension questions correct (in this case, at least 8 of 10 questions correct) and

either at least one standard deviation of fluency lower than the average for children who scored 8 of 10

comprehension questions correct or at least one standard deviation of recognition lower than the average for

children who scored 8 of 10 comprehension questions correct, based on baseline scores. Thus, to meet this

threshold in Filipino children must score at least 8 of 10 comprehension question correct and read either at least

29.5 words per minute correct or with 91% recognition.

17

Table 3: Baseline/Endline Filipino Scores with Benchmarks, Significance, and Effect Sizes

(excluding high-performing sections)†

A B Ca Da E F G Hb Ib, c

Reading

Skill Group

Baseline

Scores

Endline

Scores

Benchmark

from

Baseline

Benchmark

met?

Change

from

Baseline

to

Endline

Sig. Diff.

of Gain

Scores

Between

Groups

Effect

Size of

Diff in

Gain

Scores

Concepts about

Print

LB 7 6.8 9 No

-0.2 None 0.08

Comparison 6.8 6.5 -0.3

Letter

Identification

(of 56)

LB 47.5 52.9

56 No

5.4

None 0.13 Comparison 50.9 52.4 1.5

Filipino Most

Used Words (of

20)

LB 12.8 18.9

20 No

6.1

None 0.08 Comparison 15 18.5 3.4

% Filipino

Readers

LB 79% 90% 100% No

12% None -0.01

Comparison 77% 88% 11%

Filipino Fluency

(Words Per

Minute)f

LB 35.9 51.2

62 No

15.7


Filipino Accuracy

(% text read

correctly)f

LB 68% 88%

99% No

20%

None 0.07 Comparison 69% 84% 16%

Filipino Reading

Comprehension

(of 10)f

LB 4 4.8 6d No

0.8 None 0.01

Comparison 3.7 4.4 0.8

% Filipino

Readers with

Comprehensione

LB 10% 19%

None N/A

10%


† Inclusion of the two high-performing sections does not significantly alter the results a Neither baseline nor endline scores exhibited any statistically significant difference between LB and comparison

students b Calculated through multilevel regression analysis accounting for clustering of students in schools c Widely cited statistician Jacob Cohen describes effect sizes of .2 as small, .5 as medium, and .8 as large. Cohen, J.:

Statistical Power Analysis for the Behavioral Sciences. (2nd ed.) 1988. d Recalculated to include baseline zero scores (nonreaders) e See footnote 5. f Averages include zero scores of nonreaders.

18

Table 4: Baseline/Endline English Scores with Benchmarks, Significance, and Effect Sizes

(excluding high-performing sections)†

A B Ca D E F G Hb Ib,c

Reading

Skill Group

Baseline

Scores

Endline

Scores

Benchmark

from

Baseline

Benchmark

met?

Change

from

Baseline

to

Endline

Sig. Diff.

of Gain

Scores

Between

Groups

Effect

Size of

Diff in

Gain

Scores

Concepts about

Print

LB 7 6.8 9 No

-0.2 None 0.08

Comparison 6.8 6.5 -0.3

Letter

Identification

(of 56)

LB 47.5 52.9

56 No

5.4


English Most

Used Words (of

20)

LB 13.9 16.7

19 No

2.8

None 0.07 Comparison 13.9 15.9 2

% English

Readers

LB 50% 75% 100% No

25% None 0.03

Comparison 54% 74% 20%

English Fluency

(Words Per

Minute)f

LB 29.8 48.7

68 No

18.9


English Accuracy

(% text read

correctly)f

LB 43% 67%

93% No

24%


English Reading

Comprehension

(of 10)f

LB 2.7 4 6d No

1.2 None 0.21

Comparison 2.9 3.1 0.2

% English

Readers with

Comprehensione

LB 8% 23%*

None N/A

15%

* 0.38 Comparison 14% 9% -5%

Significant difference from control group at *p<.05 † Inclusion of the two high-performing sections does not significantly alter the results a Baseline scores did not exibit any statistically significant difference between LB and comparison students b Calculated through multilevel regression analysis accounting for clustering of students in schools c Widely cited statistician Jacob Cohen describes effect sizes of .2 as small, .5 as medium, and .8 as large. Cohen, J.:

Statistical Power Analysis for the Behavioral Sciences. (2nd ed.) 1988. d Recalculated to include baseline zero scores (nonreaders) e See footnote 5. f Averages include zero scores of nonreaders.

While results are not statistically significant in the vast majority of skills, the fact

that effect sizes are either positive or practically zero when excluding the two high-

performing Literacy Boost sections indicates that the Literacy Boost program may

have had a small degree of impact overall. Beyond the program itself, two other

factors may contribute to this very limited impact. First, as evidence exists that

comparison children also received Literacy Boost community activities this may

19

undermine the ability to perceive a difference between the learning of Literacy

Boost and comparison groups. Second, as will be explored in the equity impact

analysis section, impact may have been concentrated among subsets of the school

population.

No benchmarks set at baseline were met. While many benchmarks are reasonable,

those benchmarks set at 100% of items are unlikely to ever be reached and thus

letter knowledge and Filipino most-used words may be considered nearly-achieved.

However, more work remains to be done for the majority of other skills, especially

giving children enough exposure to actual books to increase their mastery of

concepts about print.

It will be important to better understand why limited or no impact was found. This

can be done through combining evidence from this quantitative analysis with

qualitative evidence from focus group discussions. The Literacy Boost program

may need to be improved and lessons from the first year of piloting the program

applied so that Literacy Boost can demonstrate results for students in its second

phase with a more rigorous research design.

Evidence from Focus Groups

In addition to the quantitative assessment, a series of focus groups was held with students

(grade 2, 3, and 4), teachers, community volunteers, and parents. For more details on the focus

group discussions, see Appendix E.

The results of these focus groups indicate that reading is an enjoyable activity for students, who

associate it with learning and intelligence. However, they point to pronouncing and

understanding English words as a source of difficulty in reading. There appears to be a pattern

whereby younger students (grade 2 and 3) read mostly with others, especially family members,

and do so at home and using school books. Starting in grade 3, children begin to branch out

from the home and read in other places such as Reading Camps, and by grade 4 students are

reading alone and doing so in a wide variety of places, including public parks, and with a wider

variety of types of print. Literacy Boost should facilitate this process through

continuation of Reading Camps and increasing the variety of materials available in

Book Banks.

Students report that their family members encourage them to read, and when reading with

family members students mostly mention help with pronunciation, but also say that their

parents ask them questions about stories. Nevertheless, the focus appears to be on

pronunciation, and Literacy Boost should encourage parents to help their children

20

with a variety of aspects of reading, especially in terms of asking comprehension

questions.

The grade 2 and 3 students identified reading activities in school associated with Save the

Children, and said they enjoy reading itself and games. Many students were able to participate in

reading camps, and enjoy them because the facilitators are ‘nice,’ ‘smart,’ and ‘teach them well.’

Students who attend the Reading Camps say they learn a lot from the camps and like the

reading, drawing, playing, coloring activities and practicing writing. The grade 3 students

specifically mentioned improving not only their reading skills, but their writing skills. However,

students are sometimes annoyed and distracted by the presence of younger children at the

camps, and the grade 4 students said they are sometimes unable to attend due to other

commitments such as church. Reading Camps should be scheduled at times that are

most convenient for children, or sessions at different times during the week be

offered if resources exist, perhaps by recruiting additional volunteers.

Reading also appears to be a priority for parents, and many encourage their children to attend

Reading Camps and have a positive view of them. Their children enthusiastically share what

they did at the Reading Camp, and remind the parents when it is time to go again. Parents

report that their children are now more interested in reading and writing, more disciplined

about study, and read faster. Other positive spillovers include improvements in children’s

moods. Parents, along with Reading Camp Facilitators, suggest that venues may need to be

safer, better ventilated, have bathroom facilities, and have more new books for children.

Literacy Boost should work to identify adequate Reading Camp venues and

facilitate the provision of as many new titles as possible to the Book Banks at the

Reading Camps.

Parents attending the Parental Awareness Workshops appear to confuse the messages from

those workshops with other child rights/child protection workshops held by Save the Children.

Especially because household literacy practices do not appear to be changing much,

it will be important for Literacy Boost to understand whether or not messages

related to literacy are getting through to parents, and how these messages can be

better conveyed.

Parents report not attending the awareness workshops because they were not aware of them.

Those who do not attend are even willing to adjust their schedule to attend, but others

mention the presence of young children at home as a barrier to attendance. Literacy Boost

should invest more in advertising Parental Awareness Workshops, especially

through tapping the networks of existing participants to recommend others to

attend. Perhaps accommodations should also be made (such as another volunteer

21

organizing a children’s play area at the workshop) so that those with young

children to be able to attend.

Teachers reported the training sessions useful for them, although it appears much of the

material may not have been new but rather reinforced what they have learned through other

trainings. They appear to appreciate most the terminology and framework of the Components

of Reading as well as Read Aloud methods, but do not find activities that take a lot of class time

useful, such as role plays. Teachers report most frequently using Read Aloud methods and

spelling through sounds. They suggest refresher trainings and new titles as the best

way Literacy Boost can support their efforts.

Finally, the Reading Camp Facilitators also report improved reading skills, specifically that the

children have become more conscious with their vowel and consonant sounds. They report

that children are enthusiastic and many attend, but this is also a challenge for the facilitators

when there is an average of 50 to 80 children in their sessions every week. They also note that

there are many younger children being brought in by their parents even if the lessons will not

be appropriate for their age. From facilitator feedback, Literacy Boost should recruit and

train more Reading Camp Facilitators as well as provide new books, new activities

& materials, and more training for current facilitators on how to lead the camps as

well as fuller awareness of Save the Children programs.

Overall, there is qualitative evidence that Literacy Boost has made a positive impact on

processes and learning outcomes. However, important questions remain about which students

benefitted, and how many. From the LB community activity evidence presented earlier, it

appears that only about one-fifth of students are attending. Using multilevel regression analysis,

it does not appear that attending students are either those with the highest or lowest baseline

scores. Thus, the major challenge appears to be that not all children attend Reading Camps and

therefore have the opportunity to benefit. Additional challenges appear to be lack of new

materials, especially books, and the perceived need for more structured learning at the Reading

Camps. Literacy Boost should expand Reading Camp opportunities for all children,

facilitate the provision of new books, and should better educate all stakeholders

that the objective of the Reading Camps is to provide a more informal and fun

environment for children to practice their reading skills, not an extension of

classroom time.

Equity Analysis

In addition to standard impact evaluation, an equity impact analysis was performed. This analysis

looked at Literacy Boost impact specifically with respect to both students who struggled at

baseline as well as traditionally disadvantaged groups such as girls, the poorest of the poor, the

22

HLE-deprived, those children with large chore/workloads, and those children without previous

ECD experience. To conduct this analysis, endline data was used to construct indices to place

children into quintiles of socio-economic status (SES) and chore/work load, and baseline data

was used to construct an index to place children into quintiles of HLE.6

First, the series of multilevel regressions used for the impact analysis above was re-run to

examine Literacy Boost impact among students who struggled the most at baseline, defined as

either those who were classified as nonreaders in English or those who scored in the bottom

two quintiles of English most-used words.7 These results were similar to the overall impact

analysis, and thus it appears that Literacy Boost exhibited the same limited impact on students

who were struggling as it did for the entire population of grade 2 students.

Table 5 below presents an outline of which groups had significantly lower scores at baseline.

Boys scored significantly lower than girls in a few skills, and the poorest of the poor scored

lower than the less-poor in all of the lower-order skills. Those students from the most

deprived home literacy environments scored lower than their peers from richer home literacy

environments in many skills, and those students with larger work and chore loads (the busiest)

scored significantly lower than those with lighter work and chore loads in a few skills. The only

skill for which students lacking previous ECD experience scored lower than those with ECD

experience was proportion of English readers with comprehension. Thus, the strongest

drivers of inequality in this context appear to be SES and HLE, followed by work

and chore load.

6 Endline data was used for SES and chore/workload indices as it is assumed to be more accurate as the children have matured since baseline. Baseline data was used for the HLE index as it is assumed that the Literacy Boost intervention may change the HLE over time. 7 English most-used words was used as one definition of students struggling at baseline due to its high correlation with most of the other Filipino and English reading skills.

23

Table 5: At baseline, which groups are disadvantaged?

Sub-test Sex SES HLE Work ECD

CAP Boys Poorest

Least busy

Letters

Poorest

Filipino MUW

Poorest Most deprived Busiest

English MUW Boys Poorest

Filipino Reader

Busiest

English Reader

Filipino WPMC

English WPMC Most deprived

Filipino Accuracy

English Accuracy Most deprived

Filipino Reading Comp

Most deprived Busiest

English Reading Comp Boys

Filipino RWC

Most deprived

English RWC Poorest Without ECD

Multilevel regression analysis controlling for the same factors as the overall impact analysis

reveals little evidence that the Literacy Boost program had positive impacts on equity in this

school. In some cases, inequities widened as evidence suggests that girls and those from richer

home literacy environments benefitted disproportionately from Literacy Boost in some higher-

order skills. However, Literacy Boost did help boys catch up to girls in a few lower-order skills

such as concepts about print and Filipino and English most-used words. Figure 5 below displays

this dynamic for English most-used words.

Excludes two high-performing sections

3.4 1.8 1.4

3.4

0

5

10

15

20

Boys Girls Boys Girls

LB Control

MU

W c

orr

ect

Figure 5: English MUW Baseline and Gain Scores, by Sample Group and Sex

Gain

Baseline

24

During the second phase of Literacy Boost implementation, it will be necessary to

give even more attention to targeting existing inequalities. Suggested activities,

games, and materials can be provided to teachers and community volunteers to

engage disadvantaged groups of children, and perhaps extra remedial help can be

offered to students at-risk of falling behind. This will be very important in order to

make Literacy Boost a more effective program for helping all children in these

Caloocan schools learn to read with comprehension in the early grades of primary

school.

Individual Skill Analysis

This section takes an in-depth look at the results of each sub-test and its constituent items.

Concepts about Print

Average for entire baseline and endline samples shown due to lack of statistically significant differences between

groups. Magnitude of average gain shown at top of graph

The first sub-test of the reading assessment consisted of 10 ‘concepts about print’ (CAP)

questions. These questions concern familiarity with books – where to start, which way to read,

what is a letter, what is a word, etc. Just like the baseline results, students in the sample as a

whole still found CAP questions 10 and 4 the most difficult (‘where is the end of the story?’

And ‘where is the beginning of the story?’). Comparing the baseline and endline results,

students still performed best on CAP questions 2 and 5 (‘open the book’ and ‘show me a

word’). It appears students are generally familiar with the concept of words as well as basic

dynamics of books (where the cover is and how to open them). Students still appear to

experience difficulty in identifying the beginning and end of the story. Both the Literacy Boost

and comparison groups registered a slight decline from their baseline percentage. These

results may indicate that reading activities should include familiarization on parts

-0.4

-2

0

2

4

6

8

10

# o

f C

AP

qu

est

ion

s co

rre

ct

Figure 6: CAP Baseline Score and Gain

Gain

Baseline

25

of a story. The next phase of Literacy Boost implementation should give emphasize

to teaching the basic parts of a story to participants of Reading Camps.

Letter Awareness



The next sub-test examined students’ letter awareness. Students were shown a chart of all

uppercase and lowercase letters and asked to name the letter, pronounce the letter sound, or

give a word that begins with that letter. Students know the majority of their letters, but only

36% know all 56 upper and lowercase letters, nevertheless this percentage has increased from

its baseline figure of 28%. As shown in Figure 7, both the Literacy Boost and comparison groups

increased by about two and a half letters on average. The group also exemplified a noteworthy

increase in letter identification wherein 95% of the assessed student can recognize at least 80%

of both lower and upper case letters, this compared to the baseline result where one in five

students could not name more than four upper and four lower case letters.

Table 6 below presents the letters that were correctly identified only 90% or less of the time

by students. It appears confusion still exists between the uppercase ‘I’ and the lowercase ‘l’.

Likewise, students still struggle with consonants Ñ and NG in their upper and lowercase forms.

Nevertheless, when compared with the baseline result, the number of less recognized letters

was reduced from six upper case to just two upper case, and six lower case to just four lower

case letters. Finally, except for ‘NG’, all upper case and lower case letters have improved

ratings during endline assessment. The next phase of Literacy Boost implementation

should train teachers and community reading activity facilitators to ensure all

students know all letters of the alphabet specifically the identified letters in Table 6.

Further, reading camp sessions should highlight letter knowledge and phonemic

awareness, and ensure active participation of all students, not just those that know

their letters.

2.3

0

10

20

30

40

50

# o

f le

tte

rs c

orr

ect

Figure 7: Letters Baseline Score and

Gain

Gain

Baseline

26

Table 6: Upper and Lowercase Letters

Identified with 90% Frequency or Less by Students Uppercase Baseline

% Correct

Endline

% Correct

Lowercase Baseline

% Correct

Endline

% Correct

Ñ 54% 65% ñ 64% 67%

NG 83% 77% l 65% 77%

I 88% - q 72% 79%

Q 89% - ng 84% 77%

J 89% - j 88% -

W 90% - f 89% -

Word Recognition: Most Used Words

The most used words (MUW) sub-test consists of a chart of 20 words that the student is asked

to read. These 20 words were identified as ‘most used’ by tabulating the number of times a

word appeared in students’ language arts textbooks. Table 7 shows that among the 4 Filipino

words which students found difficult during the baseline assessment, only one remained below

90% during the endline assessment.

Table 7: Frequency of Correct Reading

of Filipino and English Most Used Words

Filipino Baseline

% Correct

Endline

% Correct

English Baseline

% Correct

Endline

% Correct

pook 82% 86% sentences 43% 40%

anu-ano 85% - these 56% 69%

mga 85% - read 61% 79%

ng 85% - think 62% 75%

words - 80%

For English, students struggled most with sentences, these, read and think for both assessments,

nevertheless as shown in the table, three out of the four words showed improved percentages

during the endline assessment. Furthermore, students were able to correctly read half of the

English words at least 85% of the time compared to none during the baseline assessment.

Finally, the average most used words read correctly improved to 83% from 75% for English and

94% from 70% for Filipino.

27



Figure 8 above shows that Literacy Boost and comparison students improved at about the same

rate, although they have yet to fully master their reading of words in isolation, especially in

English. Literacy Boost programming should continue providing technical support

and guidance to teachers to help students learn and practice their decoding skills as

well as provide ample opportunity inside and outside of the classroom for students

to practice reading and memorizing basic, frequently used words such as these.

Readers and Nonreaders

After the most used words sub-test, students are then asked to read aloud a passage of

connected text of about 100 words in length. This passage is based on the most used words

and passages found in students’ language arts textbooks. At this point in the assessment,

assessors classify students as either ‘readers’ or ‘nonreaders.’ Readers are defined as students

who were able to read at least 5 words correctly in the first 30 seconds of reading. Figure 9

shows the increase in percentage of Filipino and English Readers from baseline. All other

students were classified as nonreaders, and were read the passage by assessors.

4.8 2.1

0

5

10

15

20

Filipino English

# o

f M

UW

wo

rds

corr

ect

Figure 8: MUW Baseline Score and Gain

Gain

Baseline

28



Because some students have not yet fully mastered their letters and most-used words, it is not

surprising that all students cannot yet pass the 5 words correct in 30 seconds threshold to

qualify as readers. After addressing remaining gaps in letter knowledge and single-

word reading, it will be necessary to give students ample time to practice reading

through such initiatives as reading buddies in order to build their automaticity.

Fluency and Accuracy



Fluency (words per minute read correctly) and accuracy (percent of the passage read correctly)

are presented together here because they are measured together in the oral reading passage

11% 21%

0%

20%

40%

60%

80%

100%

Filipino English% o

f sa

mp

le r

ead

ing

at le

ast

5 w

ord

s co

rre

ct in

30

se

con

ds

Figure 9: Proportion of Sample Reading at Least 5 Words Correct in 30 Seconds

(Reader), Baseline and Gain

Gain

Baseline

6.3

8.3

0

10

20

30

40

50

60

70

80

Filipino English

Wo

rds

corr

ect

pe

r m

inu

te

Figure 10: Words Correct per Minute Among Readers, Baseline and Gain

Gain

Baseline

29

sub-test. For those students classified as readers, the number of words students read correctly

in a minute is tracked for fluency. As the student continues to read after the first minute, the

total number of words read correctly from the passage as a whole, no matter how long it takes

the student, is computed for accuracy. This section presents this data for readers only8 in order

to better understand the strengths and weaknesses of students who can read a degree of

connected text.



Figures 10 and 11 show the average fluency and accuracy for the subset of students defined as

readers (baseline n=205 and 146 of 261 for Filipino and English, respectively; endline n=251 and

217 of 284 for Filipino and English, respectively).9 Baseline and endline results provide similar

trend, wherein fluency and accuracy in reading Filipino and English words increased from its

baseline figures. From speeds of 52-62 words correctly per minute in baseline it increased to

58-71 words per minute. Likewise, the readers’ accuracy improved to 97-90% from its previous

rate of 90-85%. Students could read the English passage with greater speed, but with less

accuracy, than the Filipino passage. There are many possible reasons for this: Filipino words are

longer, students may rely more on sight-reading of English words and more slowly decode

Filipino; or are more familiar with English connected text and read it more confidently but

without native accuracy, or because they apply Filipino pronunciation rules to English, etc. The

next Literacy Boost Assessment should consider re-arranging the order of the

tool’s sections. Reading of English Passage should be administered first instead of

the Filipino passage. This is to test whether reading of relatively longer Filipino

words and passage at the start is a possible cause for fewer words read in a minute.

Nonetheless, continued implementation of Literacy Boost should further enhance 8 As opposed to including the zero scores of the nonreaders, which would lower the average scores for this data. 9 Assessors were instructed to stop the oral reading passage sub-test if the student had failed to read at least five words correctly in 30 seconds. The Tangerine electronic data collection software captures the student’s progress at 60 seconds, so this measure was not recorded for these students and thus their fluency measures were coded as 0. Accuracy scores for these students were deemed to be especially unreliable, and thus also coded as 0.

5% 6%

0%

20%

40%

60%

80%

100%

Filipino English

% o

f te

xt r

ead

co

rre

ctly

Figure 11: Accuracy among Readers, Baseline and Gain

Gain

Baseline

30

readers’ opportunity for practice of reading enjoyable and relevant materials.

Furthermore, the provision of mother-tongue based reading materials will be a

good complement to the implementation of Mother-Tongue-Based Multilingual

Education program of DepEd and its schools - pushing further the improvement of

Filipino fluency of readers in the succeeding years.

Reading Comprehension

After finishing the reading passage, readers were asked the full list of ten comprehension

questions linked to the passage (one summary question, six factual comprehension questions,

two inferential comprehension questions, and one evaluative question). Figure 12 presents

readers’ reading comprehension scores.



On average, Literacy Boost and comparison students answered around 50 percent of reading

comprehension questions correctly. Increases in scores from baseline to endline was marginal

or even nonexistent in the case of English, one the reason for which is the addition of new

readers (children who did not meet the 5 words correct in 30 seconds threshold at baseline

but did at endline). However, excluding these new readers does not change average endline

reading comprehension scores much – they rise from 5.5 to 5.7 in Filipino and from 5.3 to 5.6

in English.

The following section explores students’ scores by type of comprehension question to suggest

where students need the most help with their reading comprehension. Figures 13 and 14

present readers’ comprehension skills broken down by comprehension question type.10 A

10 For both nonreaders and readers, ten comprehension questions were asked: one summary question (scored correctly if the student recalled at least two of the four story details from the Filipino story or of the three story details from the English story), six non-yes-or-no literal questions, two non-yes-or-no inferential questions, and

0.5 -0.1

-2

0

2

4

6

8

10

Filipino English# o

f re

adin

g co

mp

reh

en

sio

n q

ue

stio

ns

corr

ect

Figure 12: Reading Comprehension among Readers, Baseline and Gains

Gain

Baseline

31

similar pattern presents itself for Filipino and English reading comprehension scores for readers.

Readers were able to improve their scores in at least 2 types of questions at the endline

assessment, nevertheless they still performed the worst in literal and inferential comprehension.

Similar to the baseline result, a larger portion of students still fail to summarize the English

passage than the Filipino passage.

Looking closely at Figure13, Filipino reading comprehension (by type of question) improved by

4 to 8 percentage points from its baseline except for the summary question which recorded a

decrease of 9 percentage points. The same tool was used for the baseline and endline

assessment. In any case, in Filipino it appears students need the most help

understanding the details of the story, and making inferences based on the

information in the story.


groups.

For the English reading comprehension (Figure 14), the endline assessment showed nearly the

same performance of students as at baseline. In English, students need help not only

understanding the details and making inferences, but also summarizing the story.

one evaluative question (scored correctly if the student provided information from the text in order to justify his/her opinon).

87%

45%

35%

74% 78%

51%

43%

78%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Summary Literal Inferential Evaluative

Figure 13: Frequency of Filipino Reading Comprehension Questions Answered Correctly

Baseline

Endline

32


groups.

Overall, it appears that readers still need to continue on improving their ability to understand

and express details from the text. For the result of literal and inferential questions (both Filipino

and English), when carefully analyzed, most of the students were able to easily answer the

‘what, when and where’ questions, the answers to which can be found directly from the text of

the story. In contrast, ‘how and why’ questions were more difficult for students, which would

require a good understanding of the entire story. Teachers and community volunteers

should focus on asking different types of questions before, during, and after stories

so that students can better develop their literal and deeper comprehension skills.

In terms of assessment, the next Literacy Boost assessment should replace terms

in some of the questions with more familiar words.

64%

48%

56%

77%

67%

45%

58%

75%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Summary Literal Inferential Evaluative

Figure 14: % of English Reading Comprehension Questions Answered Correctly

Baseline

Endline

33

One New Way to Look at Data: Readers with Comprehension

Significant difference from control group at *p<.05.

Students reading with comprehension, with understanding, is the ultimate goal of Literacy

Boost. As such, a new composite measure to focus attention on this goal as well as to track

progress in terms of equity, in terms of all children reading with comprehension, is displayed

above in Figure 15. ‘Reading with comprehension’ is defined as reading a grade-level passage

with such skilled comprehension that the child correctly answers at least 80% of

comprehension questions. To ensure that children who correctly answer this number of

questions have done so as a result of sufficiently skilled reading (as opposed to guessing), the

threshold of ‘reading with comprehension’ is not only defined as the reading with at least 80%

comprehension in addition to scoring either at least one standard deviation below the average

fluency corresponding to 80% comprehension or at least one standard deviation below the

average accuracy corresponding to 80% comprehension.

Figure 15 shows that very few children qualified as ‘readers with comprehension’ at baseline.

However, by endline there was a large increase in readers with comprehension, such that about

a quarter of Literacy Boost students meet this definition in Filipino and about a third do in

English. In English, the LB group gained statistically more readers with comprehension than the

comparison group. We may attribute the Literacy Boost program interventions in

improving the English Reading Comprehension of students.

49%

63%

35%

50%

65%

73%

42%

62%

36%

27%

41%

37%

25%

15%

26%

25%

16%

9%

24%

13%

11%

12%

33%*

14%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Endline

Baseline

Endline

Baseline

Endline

Baseline

Endline

Baseline

Co

mp

aris

on

LBC

om

par

iso

nLB

Filip

ino

Engl

ish

Figure 15: Filipino and English Readers with Comprehension Tiers

Emergent Beginner Reader with Comprehension

34

While this represents an improvement over the baseline, more grade 3 children

should be able to read a grade 2 level text with at least 80% comprehension. Thus,

it will be necessary to modify the Literacy Boost program so that the large number

of children who are able to read with decent speed and accuracy also understand

what they are reading and can interact with the text on a deeper level to be truly

able to read to learn.

Trends in Reading Skill Data from Multilevel Regression Models

This section examines results from a series of standard t-tests and regressions with clustered

standard errors to account for the clustering of children within schools in order to investigate

relationships between reading outcomes on one hand and student background, channels for

reading skill improvement, and intensity of LB participation on the other hand. Appendix D

presents the results of multilevel regression analysis of the impact of Literacy Boost on gain

scores, controlling for a range of variables thought to be unaffected by the Literacy Boost

intervention and clustering standard errors to account for the clustering of children within

schools.

First, in terms of associations with student background, age and number of household members

were negatively correlated with three or more reading skill gains. This means that older

children and those from larger households did not improve their scores as much between

baseline and endline as their younger peers and those students from smaller households. Both

older children and those from larger households may need special remedial help in

order to keep pace with other children.

Additionally, girls often gained more in their reading skills over time than boys, and students

with larger work and chore loads gained more than those with lighter work and chore loads.

For a couple of skills, students with previous ECD experience gained more than those without

such experience, and those with greater body mass index (BMI) gained more than those with

lower BMI. Strangely, there is also a consistent negative association between self-reported

amount of time spent studying and reading skills gains over time. That is, the more time

students report spending on study, the lower their reading skill gains are predicted to be. More

information is required to explain this latter unintuitive finding. It should be noted that these

results are for the sample as a whole, meaning that these findings apply to both Literacy Boost

and comparison students.

LB Reading Activities and Reading Achievement

Another set of multilevel regressions examined reading skill gains and endline scores and the

intensity of participation in Literacy Boost community activities among Literacy Boost students

only. The variables of interest used in these regressions include self-reported frequency of

35

borrowing from Book Banks (# of times usually borrowed from Book Bank per week) and self-

reported frequency of Reading Camp attendance (never, once, rarely, many times, or always).

Whether students created make-and-take materials at reading camps ‘sometimes’ or ‘always’

was also examined.

There were no significant relationships between reported participation and reading skill gains

over time. However, frequency of Reading Camp attendance was positively associated with

gains in letter knowledge and Filipino fluency, as shown in Figure 16 below. As the regressions

used to generate Figure 16 control for a variety of factors including baseline scores, this is

some evidence that Literacy Boost Reading Camps may have helped students

improve some of their reading skills.

Relationship shown for both LB and comparison students

Literacy Boost Home Literacy Environment Channels for Reading Skill

Improvement

Finally, a series of t-tests were run on the baseline-endline sample of children (n=168). These

tests investigated possible ‘channels’ for reading skill improvement – channels such as changes in

children’s literacy habits and HLE. It should be noted that there was no statistically significant

association between Literacy Boost and these potential channels, as shown in Table 8. This

may be due to potential sample contamination between Literacy Boost and

comparison groups, but students are also not reporting improvements in their

household literacy environment and literacy habits in most measures. Literacy

Boost will need to apply lessons from focus groups held with parents to determine

0

5

10

15

20

25

30

Never Once Rarely Many Times Always

Gai

n

Frequency of Reading Camp Participation

Figure 16: Predicted Endline Endline Scores by Reported Reading Camp

Frequency of Participation

Letters

Filipino WPMC

36

what more can be done to help parents better engage in literacy acquisition

promoting activities.

Table 8: Change in Potential Community Channels for Literacy Boost Impact, by Sample

Group

Variable

Change

in LB

Change in

Control Significance

Frequency of study 0.6 0.7

# of types of reading materials at home 0.8 0.4

Learns to read primarily outside school 9% 5%

Exchanged books with others in community in

past week -8% -5%

Read to others in community in past week -1% 8%

% hhold members seen reading -6% -2%

% hhold members encouraging student to study -6% 0%

% hhold members reading to the student -8% -14%

% hhold members telling story to the student 5% -4%

% hhold members singing to the student -4% -8%

% hhold members playing game with the student -4% 2%

Conclusion

This study served to evaluate Save the Children’s first implementation of Literacy Boost in an

urban context, taking the first deep look at children’s home literacy environment, literacy skills,

and the relationship between the two in a large school in the city of Manila. The lessons learned

from this study will serve as an important resource for adapting the Literacy Boost program to

other urban areas in the Philippines and other countries. By analyzing the challenges of a

problematic research design, high attrition, and low community activity participation present in

this pilot study the Literacy Boost program can be enhanced to more effectively help all

students in Manila and other cities learn to read with comprehension.

Recommendations

Based on background statistics and focus group discussions, Literacy Boost programming

should:

redouble its Parental Awareness Workshop efforts in order to improve the

household literacy environment for students

gather more qualitative information to explain the even lower Book Bank

borrowing rates among students

37

expand Reading Camp coverage in terms of geography, scheduling, and

number of volunteers

identify safe Reading Camp venues with adequate facilities

facilitate the provision of as many new titles as possible at Book Banks

better educate all stakeholders that the objective of the Reading Camps is to

provide a more informal and fun environment for children to practice their

reading skills, not an extension of classroom time

encourage parents to help their children with a variety of aspects of reading,

especially in terms of asking comprehension questions

further investigate whether or not messages related to literacy are getting

through to parents, and how these messages can be better conveyed

invest more in advertising Parental Awareness Workshops, and make

accommodations such as a young children’s play area at the workshop so

that those with young children to be able to attend

offer teachers refresher trainings and provide new titles for school libraries

Based on analysis of students’ individual skills, Literacy Boost programming should:

include familiarization on parts of a story to participants of Reading Camps

train teachers and community reading activity facilitators to ensure all

students know all letters of the alphabet especially ñ, ng, l, q, j, & f.

highlight letter knowledge and phonemic awareness during Reading Camp

Sessions, ensuring active participation of all students, not just those that

know their letters

continue providing technical support and guidance to teachers to help

students learn and practice their decoding skills as well as provide ample

opportunity inside and outside of the classroom for students to practice

reading and memorizing basic, frequently used words such as these

give students ample time to practice reading through such initiatives as

reading buddies in order to build their automaticity

further enhance readers’ opportunity for practice of reading enjoyable and

relevant materials

facilitate the provision of mother-tongue based reading materials as a good

complement to the implementation of Mother-Tongue-Based Multilingual

Education program of DepEd and its schools

help teachers and community volunteers focus on asking different types of

questions before, during, and after stories so that students can better

develop their literal and deeper comprehension skills, especially

understanding the details of the story, making inferences based on the

information in the story, and summarizing the story

38

Appendix A. Literacy Boost Activities prior to Endline Assessment

Date of

Intervention / Start

Intervention Number

of days

I. LB Teacher Training

October 27, 2012 1st Teachers training

Topics covered:

1. Introduction to Save the Children

2. What is Literacy Boost?

3. Introduction to Reading Development & Instruction for

Young Children’

4. Vocabulary

5. Comprehension

2 days

November 12, 2012 Read Aloud workshop for all teachers of CZMS as part

of their In-service training

1 whole

day

November 17, 2012

2nd teachers’ training

Review of Vocabulary and Comprehension modules

Formative Assessment

1

December 8, 2012 3rd Teachers’ Training

Phonemic Awareness

Letter Knowledge

1

March 18 Demo-teaching 3 (non-

consecutiv

e;

duration is

2 hours

per

session

April 29-30 4th Teachers’ Training

1. Review of past modules

2. Formative Assessment part 2

3. Reading Fluency

4. Addressing language issues in the literacy classroom

2

II. Community Action

1. Enhancing the Literacy Environment

October 30, 2012 Soft launch of Literacy Boost in CZMS: Read Aloud for all

students

Turn-over of Book Banks to CZMS

Months of October -

November

Book Banks installed all-around barangay 177

2. Reading Camp for children

October 13, 2012 Training of Reading Camp facilitators

Topic: Phonemic Awareness, Read-Aloud

1 day

November 5, 2012 Reading Camp facilitators training 1 whole

day

November 10, 2013 1st Reading Camp held in 5 reading camp sites: 1) Blessed

39

Date of

Intervention / Start

Intervention Number

of days

Mary Day Care Center, 2) Little Star DCC, 3) The Light

DCC, 4) Maria Luisa DCC, 5) Franville DCC

November 17, 2012 Reading Camp facilitators training to cover remaining

Reading Sessions

1 whole

day

November 21, 2012 Reading Month celebration: Reading Camp conducted in

2 separate sections in CZMS

Half day

March 20-21, 2013 Training of new batch of Reading Camp facilitators 2 full days

3. Reading Awareness Workshops for Parents

September 27, 2012 TOT: Reading Awareness Workshops for Parents* 2 full days

October 1-2, 2012 Training of community volunteers for the Reading

Awareness workshops for parents

2 full days

October 6, 2012 Orientation of Day Care Workers on the Literacy Boost

program

- The orientation was more about an introduction

to the program and what their role would be in

the Reading Camps. The DC workers now serve

as the adult support to the youth facilitators of

Reading Camp and they also supervise the lending

of books in the Book Banks.

3 hours

October 30, 2012 Soft launching of Literacy Boost program in CZMS: 3 hours

November 10, 2012 1st Reading Awareness Workshop for parents 2 hours

November 27-28,

2012

Refresher sessions for Parent volunteers of Reading

Awareness Workshops

2 full days

December 15 Year-end review for community volunteers of LB Half day

*Participants of the TOT were former Child Advocate Volunteers of “older” impact areas like Paranaque, Taguig

and Pateros. Ideally, they will eventually train the succeeding volunteers for the Reading Camp and Reading

Awareness workshops for Parents. Unfortunately, this initial design did not happen because of many factors, such

as the distance of the new site (Caloocan) and the time and availability of the trained trainors.

40

Appendix B: Inter-rater Reliability

To test inter-rater reliability, or the consistency with which children’s responses were

recorded the same way between different assessors, 10% of students (n=28) were assessed by

two assessors simultaneously. Long one-way analysis of variance (ANOVA) techniques were

used to calculate the intra-class correlation within pairs of assessors for a measure of reliability.

Table B presents the results below. Using Fleiss’ benchmarks for excellent (ICC>0.75), good or

fair (0.75>=ICCA>0.4), and poor (0.4>=ICC).11

Table B: Inter-rater reliability by sub-test

Sub-test Inter-rater

Reliability (ICC)

Reliability Rating

Concepts about Print .90 Excellent

Letters .99 Excellent

Filipino MUW .24 Poor

English MUW .99 Excellent

Filipino Reader or Nonreader 1.0 Excellent

English Reader or Nonreader 1.0 Excellent

Filipino Fluency .33 Poor

English Fluency .65 Good/Fair

Filipino Accuracy .88 Excellent

English Accuracy .92 Excellent

Filipino Reading Comprehension .93 Excellent

English Reading Comprehension .96 Excellent

Filipino Listening Comprehension .96 Excellent

English Listening Comprehension .90 Excellent

The inter-rater reliability for most sub-tests falls within the excellent range, with

the exception of English fluency (good/fair) and Filipino fluency and most-used

words (poor). Thus, while assessors were consistent in their interpretation of

students’ answers overall, there may be added noise in the data for Filipino fluency

and most-used words and English fluency, which will make it more difficult to

detect relationships with other variables. However, as assessed students were

randomly allocated among assessors, the distribution of this noise should not be

systematically related to students’ scores and thus not bias the average score for

these sub-tests. In the future, more inter-rater reliability exercises should be

undertaken for scoring Filipino most-used words and fluency in both languages.

11 Fleiss JL. The Design and Analysis of Clinical Experiments. New York, NY: John Wiley & Sons Inc;1986 .

41

Appendix C: Student Background Characteristics

Table C.1: Relative Amount of Time Reported Spent on Work/Chores,

Study, and Play

Before school After school On weekends Altogether N

Work 1.1 1.1 1.4 3.6 283

Study 1.0 1.2 1.2 3.4 283

Play 0.6 0.7 1.2 2.5 283

Averages for entire endline sample shown due to lack of significant differences between LB

and comparison groups

Scale: 0 = no time, 1 = a little time/less than 2 hours, 2 = a long time/2 hours or more

Table C.2: How Children Report Spending Their Time

Outside of School

Proportion N

Housework 88% 283

Caring for children 47% 283

Working for money 4% 283

Playing computer games 20% 283

Other types of play 71% 283

Homework 76% 283

Other types of study 86% 283

Averages for entire endline sample shown due to lack of

significant differences between LB and comparison groups

Table C.3: Heath Indicators by Sample Group (excluding high-performing sections)

Baseline Endline

Variable LB Control Sig. N LB Control Sig. N

Size of breakfast 1.8 2.1 * 142 2.2 2.4

144

Had eaten within last 4 hours 86% 81%

123 74% 70%

128

Ill on day of assessment 9% 13%

97 1% 7%

113

In pain on day of assessment 14% 8%

140 22% 11% ~ 140

Reports safe water purification method 35% 26%

144 19% 17%

144

Height in centimeters 119.1 120.1

141 121.9 121.5

144

Weight in kilograms 20.7 21.3

141 23.9 23.8

143

42

BMI 14.5 14.6 141 16.0 16.1 143

43

Appendix D: Multilevel Regression Analysis

(1) (2) (3) (4) (5) (6) (7) (8)

VARIABLES CAP Gain Letter Gain

Filipino MUW Gain

English MUW Gain

Filipino Reader Gain

English Reader Gain

Filipino Fluency Gain

English Fluency Gain

Literacy Boost 0.207 0.995 0.609 0.371 -0.00550 0.0142 2.943 1.230

Sex (1=Female) 0.468 2.045* 0.965 0.932 0.108* 0.0627 5.043 2.325

Endline SES quintiles 0.273 0.0552 -0.220 -0.132 -0.000136 0.0231 0.248 2.451

Baseline HLE quintiles 0.0670 -0.364 -0.134 0.0926 -0.0190 0.0266 -0.981 0.662

Endline chore/work load 0.481* 0.246 0.0305 0.237 0.00223 0.0518** 2.454 2.801~

Attended ECD/Preschool -0.072 0.0580 -0.618 0.547 -0.0907 0.135* -5.947 3.172

Age in Years -0.263~ -0.410 -1.021* -0.787* -0.0122 -0.0344 -0.796 -2.111

Number of Household Members -0.470* -0.179 -0.253 -0.350 0.00950 -0.0397~ -1.593 -1.962

Has Repeated a Grade 0.167 -0.905 0.379 0.341 -0.0613 -0.106 -8.831 -6.814

Frequency of study -0.5*** -0.436 -0.127 -0.491 -0.0210 -0.0717* -4.947* -5.307*

Frequency of play 0.0398 -0.0278 0.133 0.159 -0.00699 -0.0131 -0.837 -0.890

Baseline Child Literacy Habits -0.213 -0.0890 -0.125 -0.302 -0.0397 0.00733 0.197 3.453

Size of Breakfast 0.534** -0.0076 -0.478 -0.343 -0.0482 0.00495 0.0423 1.831

Body Mass Index 0.0400 0.320* 0.0965 0.0439 0.0182~ -0.00686 -0.312 -1.404

Baseline Score -0.7*** -0.68*** -0.899*** -0.579*** -0.604*** -0.520*** -0.494*** -0.283***

Constant 5.914* 36.3*** 26.83*** 18.14** 0.654* 0.882* 64.27* 68.67**

Observations 138 138 138 138 137 138 137 138

R-squared 0.536 0.817 0.821 0.494 0.506 0.430 0.334 0.243

Robust standard errors in parentheses *** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1

44

Appendix D: Multilevel Regression Analysis (Continued)

(9) (10) (11) (12) (13) (14)

VARIABLES Filipino

Accuracy Gain English

Accuracy Gain Filipino Reading

Comprehension Gain English Reading

Comprehension Gain Filipino

RWC Gain English

RWC Gain

Literacy Boost 0.0241 0.00479 0.0177 0.729 0.0441 0.168*

Sex (1=Female) 0.0976~ 0.0712 1.026* 0.678~ 0.0626 0.0723

Endline SES quintiles -0.00328 0.0253 0.0899 0.147 -0.0123 0.0227

Baseline HLE quintiles -0.0107 0.0213 0.0706 0.107 0.0364 -0.00175

Endline chore/work load 0.000452 0.0469** 0.262 0.347* -0.00674 -0.000721

Attended ECD/Preschool -0.0289 0.107* -0.00318 0.260 0.0305 0.0676

Age in Years -0.0141 -0.0362 -0.275 -0.289 -0.0211 -0.0517

Number of Household Members -0.00475 -0.0392~ -0.353* -0.714** -0.0619~ -0.0651*

Has Repeated a Grade -0.0714 -0.0802 0.443 -0.447 0.0233 0.120

Frequency of study -0.0151 -0.0635* -0.721** -0.750** -0.108** -0.0658**

Frequency of play -0.0108 -0.000325 0.249~ -0.323* 0.0111 -0.0328

Baseline Child Literacy Habits -0.0464 0.0103 -0.478 -0.112 -0.0824 -0.00156

Size of Breakfast -0.0344 0.0252 -0.0650 -0.215 -0.00762 -0.00896

Body Mass Index 0.0156 -0.00409 0.00121 0.0390 -0.00224 -0.00333

Baseline Score -0.639*** -0.466*** -0.524*** -0.640*** -0.703*** -0.900***

Constant 0.685* 0.722** 6.788* 8.079* 0.825 0.930~

Observations 138 138 138 138 138 138

R-squared 0.522 0.397 0.400 0.526 0.379 0.529

Robust standard errors in parentheses *** p<0.001, ** p<0.01, * p<0.05, ~ p<0.1

45

Appendix E: Details of Focus Group Discussions (FGDs) with Primary

Stakeholders

Literacy Boost Program

Period Covered: 05 to 09 August 2013

Area: Barangay 177, Caloocan City

INTRODUCTION

Following its first year of implementation, Save the Children’s Literacy Boost Program conducted an Assessment that

included a series of FGDs carried out from 05 to 09 August 2013. Target groups for this activity included partner school

Cielito Zamora Memorial School pupils from Grades 2 to 4 and teachers who participated in the Literacy Boost training

for teachers. Likewise, community members, specifically Reading Camp Facilitators and parents of children

participating in Reading Camps were made part of this undertaking.

A total of 7 FGDs, with a minimum of 5 and a maximum of 8 participants, and a key informant interview (KII) with the

Literacy Boost Focal Person from Cielito Zamora Memorial School were completed. Number of participants per FGD are

listed below:

Date Conducted FGD Groups # of

Participants

05 Aug; AM 1. Grade 2 Pupils 5

06 Aug; PM 2. Grade 3 Pupils 8

05 Aug; AM 3. Grade 4 Pupils 6

06 Aug; AM 4. Teachers who participated in the Literacy Boost training

sessions

Grade 1 4

Grade 3 4

09 Aug; PM 5. Parents 2

09 Aug; PM 6. Reading Camp Facilitators (children and youth) 6

09 Aug; AM 7. Reading Camp Facilitators (adults) 7

Since there were only two (2) parents who took part in the FGD intended for them, another session is being organized

to complete the information that they can share with the Program. Difficulties were encountered at setting a date as

there are classes during the weekdays where parents are busy with their tasks at home. More so, the nearest Saturday,

17 August 2013, was celebration of the barangay fiesta. However, this FGD will still be scheduled and completed.

Conducted by:

ELVIE JO C. ROBIS

Date: 21 August 2013

Documents

Literacy Boost Metro Manila - Resource Centre · 1 Literacy Boost Metro Manila Endline Report September 2013 Adrian Badiable, Jarret Guajardo, Rachael Fermin, and Elvie Jo C. Robis