Global TIES for Children:Transforming Intervention Effectiveness and Scale

Experimental Impacts of the ‘Quality Preschool for Ghana’ Intervention: Implications for System-level Reform

Sharon Wolf, J. Lawrence Aber & Jere Behrman

OutlinePartI:Background

o Earlychildhoodeducationo TheGhanaiancontextoQualityPreschoolforGhana(QP4G)oResearchdesignandTheoryofchange

PartII:ImpactsofQP4GoClassroomqualityandteacherwell-beingoChildren’sschoolreadinessoDifferencesinpublicandprivateschoolsandchildcharacteristics

PartIII:Conclusions&NextSteps

2

Early childhood education (ECE)

• Earlybehavioralandsocialskillshavelifelongconsequencesforchildren.

• PotentialreturnonECEinvestmentislarge.

• EmotionalclimateofECEclassrooms:importantforearlyadjustmentandlearning.

3

The policy context in Ghana

• The2012GESreportthatthe2004KGcurriculumissound,butthatteacherbehaviorhasnotadaptedtoreflectnewpedagogy.

• Toppriority:Train27,000untrainedteachers inKG-specificpedagogy.

• Anotherpriority:engagingparents inschoolsandraisingtheirawarenessofKG-specificpedagogy.

• Privateschoolsmustcomplywiththenationalcurriculumandstandards.

4

Quality Preschool for Ghana (QP4G)

• InpartnershipwithGhanaEducationService, NationalNurseryTeacherTrainingCenter, UniversityofPennsylvania,NYUand InnovationsforPovertyAction:

• DevelopandtestanationallyscalablemodelforteachersandparentswiththegoalofimprovingKGqualityandchildren’sschoolreadiness.

5

The In-Service Teacher Training Program

• 5daysinSeptember,followedbyrefreshertrainingsinJanuary(2days)andMay(1day)implementedbyNNTTCtrainers.• Classroomvisitspairedwithmonitoring/feedbackfromdistrictcoordinators.

6

5areas:(1) Howchildrenlearn—developingachild-friendlyenvironment(2) Classroommanagement(3) Integratingplayintolanguageandliteracyinstruction(4) Integratingplayintoearlynumeracyinstruction(5) Assessmentandplanning

The Parental Awareness Program

• 3sessions(1/term)heldatschoolPTAmeeting,opentoallparentswithKGchildren.ParentalawarenesstrainingaboutKGlearning.

• Videoscreeningfollowedbydiscussion,ledbydistrictcoordinators,focusedon(1)play-basedlearning,(2)parents’roleinchildlearning,and(3)encouragingparent-teacherandparent-schoolcommunication.

7

Summerof2015

QP4G: Research design

8

240KGschools(108publicand132private)

79(35public,44private)

Controlgroup

Randomization

82(36public,46private)

T1

Teachertrainingandcoachingprogram

79(37public,42private)

T2

Teachertrainingandcoachingprogram

ParentalawarenessaboutKGlearning

6disadvantageddistrictsintheGreaterAccraRegion

Stratification

Teachertraining

+Monitoring/support

Intervention Classroom-levelmediators Childoutcomes

Teacherprofessionalwell-being

QP4G Theory of Change

Parentalintervention

9

Schoolreadiness

ClassroomQuality

September2015(baseline),June2016(followup1),June2017(followup2)2015-16academicyear

+Part II: Impacts of QP4G

10

Measures

1. Classroomquality– implementationandquality2. Teacherprofessionalwell-being3. Teacherattrition4. Childschoolreadiness

Impactsareassessed:• Endofimplementationyear• Oneyearlater

11

Teachertraining

+Monitoring/support

Intervention Classroom-levelmediators Childoutcomes

ClassroomQuality

Teacherprofessionalwell-being

Schoolreadiness

QP4G Theory of Change 15

Implementation: Are teachers integrating practices from the training in their classroom?

12

Teacherpraiseschildrenforpositivebehavior

Teacherthreatenschildrenwithorusesacaneonchildrenatleastonce

Teacherexplicitlyremindschildrenoftheclassrules

Teacherusesoneormultiplesongstofacilitatelearning

Thelessonconsistsofagamethatfacilitatedthelessonobjectives

Teacherasksstudentsatleasttwoopen-endedquestionsduringtheclass

TeacherincorporatesfounditemsasLearningMaterials(e.g.,bottlecaps,milkcartons)

Checklistwith15teachingpracticesthatwereinthetraining.Forexample:

Teachers integrate training practices in their classrooms13

3.24.6*** 4.8***

0.0

1.0

2.0

3.0

4.0

5.0

6.0

Num

bero

factivitiesobserved

Control TT TTPA

Teacherswerevideotapedteachingfor30-45minutes.

Onaverage,teachersinbothtreatmentconditionsimplemented1.5additional“developmentallyappropriate”activitiesduringtheobservedperiodofteachingpractice.

TT=Teachertraining;TTPA=Teachertraining+Parentalawarenesstraining

Emotionalsupport

• Positiveclimate• Negativeclimate• Teachersensitivity• Regardforstudent

perspectives

Classroomorganization

• Behaviormanagement• Productivity• Instructionallearning

formats

Emotionalsupport&behaviormanagement

• Positiveclimate• Negativeclimate• Teachersensitivity/tone• Behaviormanagement• ConsistentRoutine

Supportingstudentexpression

• Studentideasconsidered• Reasoning/problemsolve• Connectionstolife• Languagemodeling

Facilitatingdeeperlearning

• Scaffolding(conceptdevelopment)

• Qualityoffeedback• Objectivesexplicit

14

Developedbasedexploratoryandconfirmatoryfactoranalysis,weassessimpactsonthreedimensionsofclassroomquality.

Classroom quality: Does QP4G improve the quality of teacher-child interactions?

Emotionalsupport

• Positiveclimate• Negativeclimate• Teachersensitivity• Regardforstudent

perspectives

Classroomorganization

• Behaviormanagement• Productivity• Instructionallearning

formats

Emotionalsupport&behaviormanagement

• Positiveclimate• Negativeclimate• Teachersensitivity/tone• Behaviormanagement• ConsistentRoutine

Supportingstudentexpression

• Studentideasconsidered• Reasoning/problemsolve• Connectionstolife• Languagemodeling

Facilitatingdeeperlearning

• Scaffolding(conceptdevelopment)

• Qualityoffeedback• Objectivesexplicit

15

Developedbasedexploratoryandconfirmatoryfactoranalysis,weassessimpactsonthreedimensionsofclassroomquality.

QP4G improves the quality of some teacher-child interactions

-0.15

0.62*0.48*

-0.20

0.64*

0.21

-1.00

-0.80

-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

Facilitatingdeeperlearning

Emotionalsupport/behaviormanagement

Supportingstudentexpression

EffectSize

(dwt)

TT TTPA

Teacher professional well-being: Does QP4G improve teacher well-being?

16

Construct Measures Sample items

Motivation(5items,α =.64)

AdaptedfromBennell &Akyeampong (2007)

I’mhighlymotivatedto:• …helpchildrenlearntoreadandwrite• …helpchildrendevelopwellsocially.

Burnout(11items,α =.0.85)

Maschlach BurnoutInventory(Maschlach etal.,1996)

• Howoftenhavefeltmentallydrainedfromyourwork.

• Howoftendoyoufeelfatiguedwhenyouwakeupinthemorning.

Jobdissatisfaction(4items,α =.72)

AdaptedfromBennell &Akyeampong (2007)

• Iwanttotransfertoanotherschool• Iwanttoleavetheteaching• profession

Developedbasedonbothnewandpreviouslyvalidatedsurveymeasures,andanalyzedusingfactoranalysis:

0.35

-0.40*

0.220.12

-0.55***

-0.13

-1.00

-0.80

-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

Motivation Burnout JobSatisfaction

EffectSize

(dwt)

TT TTPA

Teacher attrition: Does QP4G reduce the likelihood that teachers’ leave the school mid-year?

17

YESNotably,thisoccurredentirelyintheprivatesector.

Theprobabilityofateacherleavingtheschoolintheprivatesectorwasreducedby82%.

0.435

0.123** 0.174*

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Pred

ictedprob

abilityofattrition

Control TT TTPA

PrivateSchools

QP4G improves children’s school readiness, primarily social-emotional development

18

Construct Measures Sample items

Overallschoolreadiness(compositescore)

InternationalDevelopmentandEarlyLearningAssessment(IDELA);SavetheChildren,2015.

Earlyliteracy,Earlynumeracy,Social-emotional,andExecutiveFunctionskills

Earlyliteracy(6itemsets,α =.73)

IDELA PrintawarenessOralcomprehensionLetteridentificationPhonologicalawareness

EarlyNumeracy(6itemsets,α =.71)

IDELA ShapeidentificationOnetoonecorrespondenceNumberidentificationSize/lengthdifferentiation

Social-emotional(5itemsets,α =.69)

IDELA EmotionidentificationConflictresolutionEmpathyPersonalawareness

Executivefunction(2itemsets,r=.27)

IDELA WorkingmemoryInhibitorycontrol

0.15* 0.14*0.11*

0.17*

0.05

0.000.02

-0.03

0.11

0.07

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

Schoolreadiness Earlyliteracy Earlynumeracy Social-emotional Executivefunction

Effectsize(d

wt)

TT TTPA

Differences by public and private sectors

• Inthe9outcomesassessed,wefindtwosignificantdifferenceinpublicandprivatesectorschools.• Impactsonreducedteacherburnoutarelargerinprivateschools.• Impactsonreducedteacherattritionoccurinprivateschoolsonly.

• NomajordifferencesinhowQP4Gimpactedclassroomqualityandchildren’soutcomesinpublicorprivateschools.• Butsignificantdifferencesinimprovingteacherwell-being.

19

One year later: Teachers are still using some of the training practices, but other impacts have faded out or become negative

20

0.21*

-0.26

0.58*

-0.40

0.22*

-0.33

0.32*

-0.72-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Fidelitychecklist Facilitatingdeeperlearning

Emotionalsupport/behaviormanagement

Supportingstudentexpression

TT TTPA

One year later: Impacts on social-emotional outcomes sustained21

0.11+

0.040.09

0.14*0.11+

-0.060 -0.06 -0.04

0.01

-0.060

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Schoolreadiness Earlynumeracy Earlyliteracy Social-emotional Executivefunction

TT TTPA

Conclusions• QP4Gisoneofthefirstimpactevaluationsinsub-SaharanAfricatoshowanin-serviceteachertrainingcanimproveKGqualityandschoolreadiness.

• Theimportantroleofrefreshertrainingsandcoachingvisits isconsistentwithresearchinhigh-incomecountrycontexts.

• Findingsareconsistentwithrelatedresearch,whichfindmediumtolargeeffectsonclassroomprocessqualityanda smalleffectsonchildoutcomes.

• Itwouldbeimportanttoassessiftheseeffectspersisttosupportchildren’stransitiontoprimaryschool.

• QP4Gtookplaceintheperi-urbancommunities.Itiscriticaltoconsiderifandhowdifferentcontextswouldrequireadaptationstoensureprogramsuccess.

22

Next steps

• Followchildrenforonemoreyearastheytransitionintoprimaryschool(ongoingthisacademicyear).

• Understandthecounteractingeffectsoftheparent-awarenessintervention.

• Completeacostanalysisoftheprogramandassessitsimplicationsforpolicy.Currentresourcecostestimatesare$841.66,$402.10,and$16.08perschool,teacher,andchildrespectively.

• ConsiderneedsofandadapatationsforruralareasinGhana,whereteachingqualityandlearningisgenerallylowerthanperi-urbanandurbanareas.

23

24Implications for Policy 1.Recommendationsforin-serviceteachertrainingsingeneral:Brief,affordablein-service,teachertrainingcanbesuccessful ifteachersaregivenongoingsupportandmessagesarereinforcedthroughouttheyear

2.Teacherscontinuetousesometrainingactivitiesoneyearlater,butsomeaspectsofclassroomqualitydecrease.• Thishighlightstheimportanceofthemonitoringandcoachingvisitsinsupportingimprovedclassroomquality,inadditiontotheneedforongoingannualprofessionaldevelopment.

3.Parentalengagementcounter-actedtheimpactoftheteachertraining• Parentengagementisvaluable.However,amoreintensiveapproachmaybeneeded.• Carefulattentionshouldbepaidtohowparentsinterpretmessagesand whowouldbestdeliverthem.Positiveimpactsshouldnotbeassumed.

Thank you• GhanaEducationService• DistrictCoordinators• NationalNurseryTeacherTrainingCenter(NNTTC)• UBSOptimusFoundationandWorldBankSIEF

• ThankyoutoalloftheKGteachersandHeadTeachers,andKGchildrenandtheircaregiversthatparticipatedinthisstudy.Noneofthiswouldbepossiblewithoutyou!

25

+ Thank You

26