Effective strategies for improving student learning: results from the low SES NP … · 2018-03-08 · NP Teacher Survey; and • Case studies of Low SES NP schools. The report develops

Effectivestrategiesforimproving

studentlearning:resultsfromthelowSESNP

evaluation

ShuyanHuoStephenLamb

Researchreport

UniversityofCanberraVictoriaUniversity

i

ExecutiveSummaryThis reportwas prepared by the Centre for International Research on Education Systems(CIRES) at Victoria University and is part of an evaluation of National Partnership reforminitiatives in low-SES schools. The evaluation is being undertaken by the University ofCanberrainpartnershipwithVictoriaUniversityonbehalfoftheStateofNewSouthWales.Inthisreport,theauthorsfocusonstudentperformanceintheLowSESschoolsinNSWthatparticipatedintheNationalPartnershipAgreements,theextenttowhichchangesinstudentperformance are related toNP initiatives or other factors and the initiatives thatmaybeidentifiedasmakingthestrongestcontributionstochangesinstudentoutcomes.

Previous findings associatedwith this study have already identified positive relationshipsbetweenLowSESNationalPartnershipparticipationand improvedstudentoutcomesoverthestudyperiod.

Theyinclude:

• significantimpactonstudentNAPLANresultsover3yearsbetween2011and2013inLowSESNPschools.

• scoresincreasingforeachadditionalyearofparticipationintheprogram.• reduction ingainscoregapsbetweenLowSESschoolsandothermoreadvantaged

settings.

In investigatinghow the reforms implemented inNP schools have led to improvement instudent learning outcomes and which reforms may be identified as contributing toimprovementthisreportdrawsonawidevarietyofdatasources,including:

• student levelNAPLANdata in2011and2013providedby theNSWDepartmentofEducationandTraining;

• NAPLANschoolprofiledatain2011sourcedfromACARA;• school leveldatafromthe2014LowSESNPPrincipalSurveyandthe2013lowSES

NPTeacherSurvey;and• CasestudiesofLowSESNPschools.

Thereportdevelopsthreekeysetsofanalyses: (1)estimatingaggregateschooleffectsonstudent achievement, (2) testing the relationship between aggregate effects and effortacross the broad NP reform strategy areas, and (3) identifying individual strategies orinitiativesthatwereassociatedwithstrongerstudentoutcomes.


ii

AggregatelowSESNPeffects

Effects forNP schools on achievement scores in Year 5 and Year 9NAPLANReading andNumeracy were estimated using multi-level regression taking into account a number ofstudent and school level factors that influence student achievement, such as studentbackground,priorNAPLANachievement (two-yearsearlier),genderandtheconcentrationofstudentsbyfamilySES,concentrationofAboriginalstudents,schoolsizeandschooltype.The analysis measures how much each school ‘value-added’ to their student NAPLANachievementandhowwellaschoolperformscomparedtosimilarschools.

Schools participating in the low SES NP exhibit overall a positive effect on studentachievement,albeitwithvariationsforReadingandNumeracyandbetweendifferentyearsofNP implementation.TheNPeffect ismoreconsistentor strongeronNumeracy than inReading andmore consistent for the Year 9 cohort than for the Year 5 cohort. Schoolscommencingtheir lowSESNP in2011showonaveragethestrongestNPeffectamongallNPschools.

ComparedtostudentsinlowSESschoolswhodidnotparticipateintheNP,studentsinthelowSESNPschoolsperform,onaverage,3pointshigherinYear5Numeracy,3pointshigherinYear9Readingand6pointshigherinYear9Numeracy.

LowSESreformareaslinkedtobetterstudentoutcomes

ToexaminewhetherornotparticularaggregateeffectswereassociatedwitheffortplacedinparticularlowSESreformareas,atabroadlevel,datawereusedfromthe2013Teachersurvey, and the 2014 Principal survey to examine the aggregate NP effects. Teachersresponded in the 2013 Teacher Survey to a series of questions about changes in teacherpracticeandschoolmanagementasaresultofNPimplementation.Principalswereaskedinthe2014surveyfivegroupsofquestionsthatarebroadlyrelatedtothesixNPreformareas.Foreachgroupofquestions,principals responded to threeaspectsofNP implementationincluding:

• the intensity of NP implementation (i.e., the total number of NP initiativesimplemented),

• thelevelofeffectiveness(from“Notalleffective”to“Highlyeffective”),and• thetotalnumberofNPinitiativesthatwerecontinuingattheschool.

Asetofmeasureswereconstructedusingteacherresponsestoaseriesofquestionsinthesurvey, includingthe levelofpositivechanges inteacherpracticeandthe levelofpositivechanges in school management. These were correlated with the low SES NP effects onNumeracyandReading.Afurtheranalysisexaminestherelationshipbetweenthesizeoftheimpact of the low SES NP on student Reading and Numeracy achievement and principalviewsonwhatreformareascontributedmosttoimpact.


iii

Theresultssuggestthatforprimaryschools(Year5)thereisapositiverelationshipbetweenNP school effect scores and principal-reported level of effectiveness for initiativesimplemented as part of Reform Area 4, providing innovative and tailored learningopportunities.Thehigherlevelofeffectivenessasperceivedbytheprincipal,themorelikelytheschoolshowsahigherNPeffect.TheresultsalsoshowforprimaryschoolsarelationshipbetweenthenumberofinitiativesimplementedfromReformArea4andNPschooleffects.It suggests that schools which placedmore effort in this reform areawasmore likely toshowahighereffectonstudentlearningoutcomes.

Individualinitiativeslinkedtobetterstudentoutcomes

NPschoolsmaybefacedwithsomecommonchallenges,suchas lowattendanceand lowachievement, however other conditions or circumstances they operate with may vary.Although government schools were required to address all six areas of the NP reform,schools could prioritise their efforts or resources on particular initiatives or activities toaddresstheiruniquechallengesorconditions.Informationprovidedbytheprincipals(inthe2014survey)enablesanalyses to identify the impact, ifany,of individualNP initiativesorstrategies.

To identifywhat initiativesorstrategiesthatmayhavedifferent influencesonthe levelofNPeffectamongNPschools,schoolsweregroupedbasedontherankingderivedfromthesizeoftheirNPeffectvalues.Threegroupswereidentified:(1)schoolswith lowNPeffectvalues, suggesting negligible or little effect on Reading and Numeracy achievement, (2)schoolswithsmallormodesteffects,and(3)schoolsatthetopendoftherankdisplayinglargereffects.

InlookingatwhatseparatedschoolsonthebasisofwhethertheirimprovementinNAPLANachievement had been ranked relatively higher or lower, the analysis revealed that highimpactschoolswerecharacterisedbythefollowingfeatures:

• They had placed greater emphasis than low impact schools on innovations inteachingpracticeandaddressingstudentneeds

• Theyhadprovidedstrongerwelfareandlearningsupportfordisadvantagedstudentsthrough providing teachers with training around individual learning needs, andstrengtheningtheuseofIndividualLearningPlans(ILPs)forstudents

• Strongeruseofevidencetoidentifystudentlearningneeds• Stronger emphasis on developing staff through providing opportunities for

professionallearninganddevelopment

ForschoolswheretheNAPLANachievementeffecthadbeen lowornegative,theanalysisrevealedthatschoolstendedtobecharacterisedbythefollowing:


iv

• Some had a stronger emphasis on strategies to attract and retain teachers,particularlythroughleadershippositions,whichmayitselfreflectgreaterneedinthisareaandschoolsstrugglingtorecruitandretainqualityteachersandleaders

• Less focuson innovation in teaching, addressing studentneeds, and strengtheningschoolaccountabilityandmoreonstaffingandperformancemanagement

• Pursued strategies around teacher and leadership recruitment, retention anddevelopmentsuggestingthatstaffstabilitywasamajorissueforthem.

Thedifferences in focus and emphasis amongNP schoolsmaywell reflect the challengesthat they faceand towhich theyare responding, rather than theeffectsof thestrategiesthemselves.Forexample,schoolsthatshowlowNPimpactonstudentNAPLANresultshaveplacedgreateremphasisonattractinghighperformingteachersandimprovingthestaffingandperformancemanagement roleof theprincipal. Theymayhave issues in staffingandmanagement practices that need (and the schools recognise that need) to be addressed.High impact schools, on the other hand,may havemore stable staffing profiles, and it istheir staffing andmanagement stability thathasenabled these schools to target areasofimprovement in teachingpracticeand inaddressingstudentneedsthathasdeliveredrealtractionfromtheNPinitiatives.


v

ContentsContents...............................................................................................................................................v

Listoftables........................................................................................................................................vi

Listoffigures......................................................................................................................................vii

Glossary............................................................................................................................................viii

1. Introduction....................................................................................................................................1

1.1 Previousfindings.....................................................................................................................................3

1.2 Thisreport...............................................................................................................................................4

2. Datasources....................................................................................................................................6

2.1 Limitationsofthestudy...........................................................................................................................9

3. MethodologyandDefinitions........................................................................................................11

3.1 Measuringstudentlearning..................................................................................................................12

3.2 Measuringschoolleveleffectsonstudentprogress.............................................................................13

4. AggregatelowSESNPeffectsonstudentlearning.........................................................................15

4.1 Influentialstudentbackgroundandschoolfactors...............................................................................15

4.1.1 Significantstudentlevelfactors...................................................................................................16

4.1.2 Significantschoollevelfactors.....................................................................................................18

4.1.3 NPschoolleveleffects..................................................................................................................18

4.2 Estimatingthesizeofschooleffects.....................................................................................................18

4.3 ComparingschooleffectsbetweenlowSESNPschoolsandlowSESnon-NPschools.........................21

5. Initiativeslinkedtobetterstudentoutcomes................................................................................23

5.1 KeyNPreforms......................................................................................................................................23

5.2 Overalllevelsofchangelinkedtoreformareasasperceivedbyprincipals..........................................245.3 NPeffectsassociatedwithchangesinteacherandschoolpractice.....................................................28

5.4 IndividualNPinitiativesorpracticesassociatedwithNPeffect...........................................................30

5.4.1 GroupinglowSESschoolsbasedonestimatedlevelofimpact....................................................30

5.4.2 GreatestimpactNPinitiatives......................................................................................................33

6. Casestudyschools.........................................................................................................................37

6.1ValleyPublicSchool......................................................................................................................................39

6.2SouthCoastHighSchool...............................................................................................................................416.3WesternPublicSchool..................................................................................................................................43

6.4WesternHighSchool....................................................................................................................................45

7. Conclusion.....................................................................................................................................50

References..........................................................................................................................................52

Appendix............................................................................................................................................55


vi

ListoftablesTable2-1 Profiles ofmatched cohorts for student NAPLAN records 2011-2013: Year 3-5

andYear7-9,studentsinNSWgovernmentschoolsandstudentsinparticipatinglowSESNPCatholicschools 7

Table2-2 Profilesofschoolswithmatchedcohorts 8

Table4-1 Estimates of predictors of NAPLAN Reading and Numeracy achievement(standardisedscores)in2013:Year5andYear9students 17

Table5-1 TestsofSpearmanCorrelationbetweenschoolNPeffectsandtheeffectivenessofNPimplementationasperceivedbytheprincipal 27

Table5-2 TestofSpearmanCorrelationbetweenNPschooleffectandtheeffectivenessofNPimplementationasperceivedbyteachers 29

Table5-3 LowSESNPschoolsbyschooleffectcategory 31

Table5-4 OrdinallogisticregressionmodelresultsidentifyingsignificantpredictorsofsizeofNPschooleffectonNAPLANachievement 32

Table5-5 NPstrategiesassociatedwithNPeffect impactcategory(low,medium,high) inReadingandNumeracy 34

Table6-1 Profilesofcasestudyschools 38

TableA-1 Schoollevelcovariancestatistics(Intra-classCorrelationCoefficient) 55


vii

ListoffiguresFigure3-1 Flowchartoutliningapproachtotriangulationofdata 11

Figure3-2 VariousmeasuresofstudentachievementsinNAPLAN(2011-2013cohorts) 13

Figure4-1 Distributions of estimated school level effects (school level residuals) by YearlevelsandDomains 20

Figure4-2 School effect on student NAPLAN scores, comparing low SES NP and low SESnon-NPschools 21

Figure5-1 DistributionofNPinitiativesimplementedinschoolsbyreformareas 26


viii

Glossary

ACARA TheAustralian Curriculum, Assessment and Reporting Authority. Has oversight ofNAPLAN.

AECG AboriginalEducationConsultativeGroup

ARIA Accessibility/Remoteness Index of Australia—an index that underpins manylocationclassifications.

COAG Council of Australian Governments, an organisation consisting of the federalgovernment,thegovernmentsoftheeightstatesandterritoriesandtheAustralianLocalGovernmentAssociation.NPsareimplementedundertheauspicesofCOAG.

CIRES CentreforInternationalResearchonEducationSystems,VictoriaInstitute,VU.

DEC DepartmentofEducationandCommunities,NSW(fromApril2011).

DET DepartmentofEducationandTraining,NSW(untilApril2011).

FTE FullTimeEquivalent.HAT HighlyAccomplishedTeacher (governmentsector)orequivalent (non-government

sector). An initiative within the NP. A HAT models good teaching practice andmentorsotherteachersthroughsupervision,demonstrationandteamteaching.AHAT usually has half the teaching load of a regular classroom teacher and is amemberoftheschoolexecutive.

ICSEA

IndexofCommunitySocio-EducationalAdvantage,ascalethatrepresentslevelsofeducational advantage associated with the educational and occupationalbackgroundofparentsofstudents.Aschool'sICSEAvalueistheaverageleveloftheeducational advantage of its students. Developed by ACARA to assist with theinterpretationofNAPLANresults.

LowSESNP LowSocio-EconomicStatusSchoolCommunitiesNationalPartnership.MCEETYA TheMinisterial Council on Education, Employment, Training and Youth Affairs, (a

MinisterialCouncilofCOAG)whichwasreplacedfromJuly2009bytheMinisterialCouncilforEducation,EarlyChildhoodDevelopmentandYouthAffairs(MCEECDYA)andtheMinisterialCouncilforTertiaryEducationandEmployment(MCTEE).

NAPLAN National Assessment Program - Literacy and Numeracy. An annual nationalstandardisedliteracyandNumeracytestingprogramforstudentsYears3,5,7and9.

NP National Partnership, agreements between the Commonwealth and state andterritorygovernmentsmadeundertheauspicesofCOAGoutliningfunding.

ISP IndividualStudentPlan.PD/PL Professionaldevelopment/Professionallearning.UoC UniversityofCanberra

UniversityofCanberra VictoriaUniversity

1

1. Introduction

An ongoing interest among educational policymakers and practitioners is to identifyeffective school practices that lead to improved student learning outcomes. This isparticularlyimportantinhelpingschoolsandsystemsworkouthowbesttoinvestfundsandresources to gain improvements and maximise returns on investment. The NationalPartnerships initiatives implemented in New South Wales schools provided an idealopportunitytocontributetothisgoalbecauseofthestructuredandtargetedapproachtoreformsandtheavailabilityofdata,bothadministrativeandsurvey,withwhichtoestimateandmeasureimpactandchange.

TheNationalPartnershipAgreementswereaseriesofpolicy interventions initiatedbytheCouncilofAustralianGovernments(COAG)andimplementedacrossallstatesandterritoriesin Australia. There were three Smarter Schools National Partnerships - the Literacy andNumeracy, Improving Teacher Quality and Low Socio-Economic School CommunitiesNationalPartnerships. All threeNationalPartnershipsweredesignedto improvestudentlearning in schools. Although each National Partnership functioned independently, theywere conceived as a mutually complementary reform package. The Low SES SchoolsNational Partnership targeted schools serving disadvantaged communities and providedthemwithextra resources and targetedpolicy initiatives tohelp thembuild capacity andimprovestudentlearningoutcomes.

The Low SES NP featured six reform areas that encompassed a variety of initiativesimplementedbyschools:

ReformArea1: Incentivestoattracthigh-performingteachersandprincipals

ReformArea2: Adoption of best practice performance management and staffingarrangementsthatarticulateaclearroleforprincipals

ReformArea3: School operational arrangements that encourage innovation andflexibility

ReformArea4: Provideinnovativeandtailoredlearningopportunities

ReformArea5: Strengthenschoolaccountability

ReformArea6: External partnerships with parents, other schools, businesses andcommunitiesandtheprovisionofaccess


2

Initiativesassociatedwithstaffing,managementandaccountabilityfellunder5outofthe6reform areas associatedwith the Low SES NP. Over the life of the Low SES NP, schoolsimplemented a variety of interconnecting and reinforcing strategies associated with thevariousreformareas.

ThisreportpresentsresultsofafurtherevaluationofthelowSESNPreforminitiatives.ItispartoftheevaluationthatinitiallywasdescribedastheEvaluationServicesAgreementforDETSSNP1024:EvaluationofSchoolexternalpartnerships,betweentheStateofNewSouthWales and the University of Canberra. That evaluation was revised to focus on studentperformance in low-SES schools that had participated in the National PartnershipAgreements and in particular to examine the extent to which changes in studentperformancearerelatedtoNPinitiativesorotherfactors.Therevisionisconsistentwiththeoriginal aims of the research project and builds on the findings of previous stages of theevaluation, which suggest a more thorough investigation of student performance in thecontextofNPinitiatives.

Partoftheaimofthiswork,consistentwiththeoriginalevaluationgoals,istoevaluatetheeffectivenessandlongtermimpactofschoolexternalpartnershipsontheschoolandpost-schooloutcomesofstudentsindisadvantagedschools,aswellasassesstheextenttowhichsupportingschoolexternalpartnershipsrepresentsvalueformoneyinachievingimprovedstudent outcomes. The two key research questions to be addressed in that evaluationwere:

(1)Towhatextenthaveschoolexternalpartnerships ledto improvedoutcomes inLowSESNPschoolsaftercontrollingforotheractivitiesintroducedundertheNP?

(2) What activities are associated with the greatest improvements in outcomes forstudents?

Theworkundertakenforthecurrentreportaimstoaddressthesequestionsaswellas toexaminetheimpactmorebroadlyofthedifferentreformsthatmadeupthelowSESNP.So,afurtherquestionofimportancetothisstudyis:

(3)Whichreformshavecontributedmosttoimprovedstudentoutcomes,andwhy?

These research questions are prompted by a policy interest in how student performancechangedbetween2008and2013and theextent towhichany changesare related toNPinitiatives.AkeyissueforpolicyistounderstandwhichNPinitiatives,ifany,hadapositiveimpactonstudentperformanceinlow-SESschoolcommunitiesandwhy.


3

1.1 PreviousfindingsPrevious findingsassociatedwiththisstudyhave identifiedpositiverelationshipsbetweenLowSESNationalPartnershipparticipationandimprovedstudentoutcomesoverthestudyperiod.

Intheevaluationofthestaffing,managementandaccountability initiativesofthe lowSESNP, comprehensive analyseswere undertaken to evaluate the impact of the initiatives inimproving NSW student outcomes across a range ofmeasures including NAPLAN results,Year 12 attainment, HSC achievement, ATAR scores, and attendance (CIRES, 2015). Thereport identified positive effects of somemeasures at an aggregate or system level as aresultofLowSESNP.Inparticular,thepreliminaryanalysisusingNAPLANmatchedcohortdata suggested that low SES NP schools on average saw a significant impact on studentNAPLANresultsover3yearsbetween2011and2013.

The NAPLAN trend analysis employed an approximation of a multiple error componentmodel using OLS with student level fixed effects, to measure effects of Low SES NPparticipationonNAPLANscores. Estimationwasperformedonapproximatelyonemillionstudentscoresbetween2008and2013.ResultssuggestedthattheLowSESNPhashadasignificantpositiveeffectonstudentNAPLANachievement,withthelargestimprovementinwritingscores.Theanalysisalsoindicatedtheimportanceofconsideringthedurationthataprogram has been in place, with Reading, Spelling, Grammar and Numeracy scoresincreasingby1to1.5pointsperadditionalyearofparticipationintheprogram.

Anumberofextensionstotheanalysiswerealsoconsidered, includingcomparingschoolswithcommonICSEAscores,examiningheterogeneityinthetreatmenteffectandestimationonCatholicschools.Thecomparisonsampleestimatesaresimilarinsignandmagnitudetothe main results, suggesting modest positive effects on student NAPLAN scores ofparticipationinLowSESNP.

The Low SES NP overlaps with several different programs, and an analytical model wasdeveloped to consider the individual effect of each program and its duration. Programsincluded in the models, in addition to the broad Low SES NP, include the Literacy andNumeracyNP, the ImprovingTeacherQualityNP (including theNP for the teacherqualityenhanceddecisionmakingpilot,andtheNPforteacherqualityforschoolsparticipatingasa‘spoke’ofaCentreforExcellenceHub).ResultssuggestthatparticipationinthebroadLowSESNPprogramonlyhadthelargestpositiveeffectsonNAPLANscores.

Ananalysisofachievementgaincomparinggainscoresagainstthestateaverageandwithinschool variations shows there is a reduction in the gap between the primary schoolsparticipating intheLowSESNPandthestateaverage inReadinggain.Thereductionsaresmall,butconsistent.


4

1.2 ThisreportPrevious resultspoint topositiveaggregateeffectsof the lowSESNP reforms.What theydon’tshowisiftheeffectisuniformorvariesacrossschools,andifsowhatmightexplainvariations in impact.While it is important to know if on average the Low SESNP helpedimprovetheeducationandlifeopportunitiesofstudentsfromlowSESbackgroundsthroughimprovementsinstudentoutcomes,itisalsoimportanttoknowiftheeffectsvaryandwhatcontributes to variation in impact. To examine this aspect, a range of analyses wereundertakenusingavailabledata toevaluate thevariations ineffectivenessof theLowSESNPinimprovingNSWstudentoutcomesusingNAPLANachievementdata.

Drawing on quantitative evidence by cross-matching administrative and survey datasources, this report provides analyses on how and to what the extent the NP effect onstudentNAPLANresultsvaryacrossschools.By linking individual studentNAPLANrecordsanddataonoveralllevelofNPrelatedpracticesandfeaturesinNPschools(asreportedbyprincipals and teachers), this report identifies the keyNP initiatives or interventions thatmaypromoteordrivestudentlearningimprovement.Italsoprovidesanalysesandinsightsonotherschool-levelfactorsthatmayassistinoptimisingprogrameffects.This report builds on the findingsof previous stagesof the lowSESNPevaluation,whichsuggest a more thorough investigation of student performance in the context of NPinitiativeswouldbeworthwhile.An important goal of the evaluation is to measure the impact of NP interventions byexamining student and school performance differences using a multi-level modelling or‘value-added’methodology1.Valueadded(VA)analysisprovidesarobustwayofestimatingpupilandschoolperformance.ItwillbeusedtoidentifywhichelementsoftheNPinitiativesmakeadifference,intermsofimprovedresultsinlow-SESNPschools.BylinkingindividualstudentNAPLANrecordsbetween2011and2013, this report isable to track the learningprogressof2cohortsofstudents(i.e.,Year3-5andYear7-9)inReadingandNumeracyoverthe3yearsfrom2011to2013.TheVAeffectofindividuallow-SESNPschoolsonNAPLANReadingandNumeracyarethenestimated.TheVAmeasuresadjustforstudentandschoolfactorsthatareknowntohaveanimpactonstudentlearningoutcomesincluding,atthestudentlevel,priorperformance,individuallow-SES student background, gender and ATSI status. The contextual characteristics of theschool—SES measures, type, location, total enrolments and enrolments by gender,AboriginalityandlanguagebackgroundsotherthanEnglish(LBOTE)—werealsoapplied.The

1Lu,LucyandRickard,Karen(2014)Value-addedmodelsforNSWgovernmentschools.TechnicalPaper.CentreforEducationStatisticsandEvaluation,NSWDepartmentofEducationandCommunities,OfficeofEducation.


5

purpose of defining the VA measures is to ensure that the impact of a students’ priorachievementlevel,socio-economicstatusbackgroundandthatoftheirpeersareaccountedforwhenmeasuringstudentachievement.Thisenables identificationofwhich low-SESNPschools, if any, contributeadditionally to studentachievement relative to theaverageNPschoolovertheperiod.

Atamoredetailed level, theanalysisofVAmeasureshelpsdeterminewhich schoolsandwhichNP initiativeshavebeensuccessful in raisingstudentperformance forstudentsandhowlargetheimpacthasbeen.

This focus ispromptedbyapolicy interest inhowstudentperformancechangedbetween2008and2013and theextent towhichanychangeswere related toNP initiatives.AkeyissueforpolicyistounderstandwhichNPinitiatives,ifany,hadapositiveimpactonstudentperformanceinlow-SESschoolcommunitiesandwhy.

Itshouldbenotedthatprivatenon-Catholicschoolswerenotincludedintheanalyses.


6

2. Datasources

Thisreportdrawsonawidevarietyofdatasources,including:

• studentlevelNAPLANdatain2011and2013;• NAPLANschoolprofiledatain2011sourcedfromACARA• schoolleveldatafromthe2014lowSESNPPrincipalSurvey,and• datafromthe2013lowSESNPTeacherSurvey.

Amajor goal of theNP initiativeswas to improve theeducationand lifeopportunitiesofstudents from low SES background through targeting teacher quality, better use of theassessment data, strengthened school leadership and strengthened partnerships withcommunity agencies and members including parents. One major question to be asked,therefore, iswhetherthereformsimplementedinNPschoolshave ledto improvement instudentlearningoutcomes.

NAPLAN is the only available source that provides standardised and calibrated tests ofstudent learning achievements administrated at regular intervals. It provides an annualassessmentforallstudentsinYears3,5,7and9invariousliteracydomainsandNumeracy.AlthoughNAPLANisnotadirectmeasureofcurriculumcontentcoveredinschoolprograms,theydoprovideanassessmentofessentialskillsthatwemightexpectchildrentopossesstoprogressthroughschoolandlife.

BylinkingindividualstudentNAPLANrecordsbetween2011and2013,weareabletotracktheNAPLANachievementof2cohortsofstudents:(1)Year3studentsin2011and(2)Year7studentsin2011.AchievementwasmeasuredinReadingandNumeracyin2013whenthetwocohortswere inYear5andYear9, respectively.Over52,000studentrecords in2011were successfully matched to the 2013 records for both cohorts. This accounts forrespectivelyaround66per centofYear3 studentsand70per centofYear7 students in2011.TheprofilesofthetwocohortsarepresentedinTable2.1.

TheresultsinTable2-1showthataround4percentofstudentsdonothaveachievementresultsduetoabsence,withdrawalorexemption.Exemptionsmaybegrantedbyschoolsforstudentswith a disability orwith a language background other than Englishwho arrivedfromoverseasandhavebeenattendingschoolforlessthanayearbeforethetest.Dataforstudentswithoutassessmentscoresaretreatedasmissingvaluesandareexcluded intheanalyses.

The results also show the loss of students due to the matching and linking of studentrecordsbetweenYears3to5andYears7to9.Studentsmayhavemissingdataatonepointrather than both because they changed schools orwere absent or exempt at one of the


7

measurementpoints.ToensurethatthechangesinresultsorNPeffectsarelinkedtotherole of programs in individual schools, only students who stayed in the same schoolbetweenthetwoNAPLANassessmentperiodswereincludedintheanalyses.Differencesinlevelsofabsenceandexemptionacrossschools,anddifferencesinproportionsofstudentsstayinginthesameschoolacrossthetwotimepoints,maybeaninfluenceontheresults.However,onethingtonotefromTable2-1isthatatanaggregatelevelthematchedsampleis sufficiently similar to the total sample to give us confidence that the sample isrepresentativeofthepopulationofstudents.Theoverall ratesofmatchingfor indigenousstudents,forLBOTEstudents,formalesandfemales,andforthosenotassessedbecauseofexemptionorabsenceare roughly similar. There stillmaybedifferencesata school levelthatneedtobeconsidered.Futureworkwillbeundertakentoexaminewhatimpact,ifany,thishasonestimatedvalue-addedscores.

Table2-1 ProfilesofmatchedcohortsforstudentNAPLANrecords2011-2013:Year3-5

and Year 7-9, students in NSW government schools and students inparticipatinglowSESNPCatholicschools

NAPLANmatchedcohort NAPLANCohortin2011

Years3-5 Year7-9 Year3 Year7 Number

Sex Male 26,716 27,153 40,238 38,529Female 25,439 25,532 38,209 36,395

Indigenousstatus

Non-Indigenous 49,306 49,694 74,363 70,788Indigenous 2,849 2,991 4,084 4,136

LBOTEStatusNon-LBOTE 36,807 36,444 55,742 53,821LBOTE 15,135 15,595 22,265 20,379Unknown 213 646 440 724

NotassessedReading 1,996 2,047 2,884 2,782Numeracy 2,109 2,360 3,039 3,172

Total 52,155 52,685 78,447 74,924 Percentage

Sex Male 66.4 70.5 100 100Female 66.6 70.2 100 100

Indigenousstatus

Non-Indigenous 66.3 70.2 100 100Indigenous 69.8 72.3 100 100

LBOTEStatusNon-LBOTE 66.0 67.7 100 100LBOTE 68.0 76.5 100 100Unknown 48.4 89.2 100 100

NotassessedReading 69.2 73.6 100 100Numeracy 69.4 74.4 100 100

Total 66.5 70.3 100 100


8

Table2-2 Profilesofschoolswithmatchedcohorts Primary Secondary Combined Total* Numberofschoolswithmatchedcohorts

SchoolSector

Catholic 72 58 9 139Government 1,524 370 67 1,961Total 1,596 428 79 2,112

SchoolLocation

Metropolitan 956 290 6 1,252Provincial 616 133 54 803Remote 20 4 12 36VeryRemote 4 1 4 9Total 1,596 428 79 2,112

NPschools:YearofNPparticipation

2009 70 22 25 1172010 105 33 8 1462011 121 23 1 1452012 50 16 13 79Total 346 94 47 487

Surveyparticipation

TeacherSurvey(2013) 217

PrincipalSurvey(2014) 188 Percentageofschoolswithmatchedcohorts

SchoolSector

Catholic 17% 54% 64% 26%Government 94% 93% 99% 94%Total 78% 85% 96% 80%

SchoolLocation

Metropolitan 78% 76% 4% 70%Provincial 74% 80% 46% 72%Remote 57% 100% 100% 71%VeryRemote 40% 50% 67% 50%Total 76% 77% 26% 71%

NPschools:YearofNPparticipation

2009 80% 96% 100% 87%2010 79% 89% 53% 79%2011 81% 82% 25% 80%2012 76% 70% 81% 75%Total 79% 85% 78% 80%

Surveyparticipation

TeacherSurvey(2013) 75%PrincipalSurvey(2014) 70%

*Totalsincludeschoolswherethetypeofschoolisunknown.NOTES:Specialschoolsareexcluded.

Other important sources of data have been developed over the course of the evaluationthrough a series of school-based surveys. Surveys of school principals and teachers haveassisted in obtaining detailed information about the implementation and effectiveness oflow SES NP initiatives or interventions. In this report, data on the low SES NP


9

implementationandeffectivenesscollectedthroughthe2014PrincipalSurveyandthe2013Teacher Survey have also been linked to their student NAPLAN records using schoolidentifiers.

Data on the low SES NP implementation and effectiveness collected through the 2014Principal Survey and the 2013 Teacher Survey have been linked to the student NAPLANrecordsusingschoolidentifiers.

ProfilesofschoolswithmatchingcohortsareprovidedinTable2-2.Thenumbersofschoolsmatched to the NAPLAN cohort data are provided in the first panel of the table. Eachpercentagereportedinthesecondpanelofthetablereportswhatthenumberofmatchedschoolsisasapercentageofallschoolsforthatcategory. Forexample,1524governmentprimaryschoolswerematchedtotheNAPLANcohortdataandthisrepresented94percentofallpossiblegovernmentprimaryschools.

ThemajorityoflowSESNPschools—487outof607or80percent—havebeenmatchedtotheNAPLANcohortdataforthevariousanalysesinthisreport.

About75percentofschoolsthatparticipatedinthe2013TeacherSurveyand70percentofschoolsthatparticipatedinthe2014PrincipalSurveywerematchedtostudentNAPLANcohort data. While not covering all schools, the matching provides sufficient samples ofstudentsandschoolstoprovideconfidenceintheanalyticalresultspresentedinthereport.

2.1 Limitationsofthestudy

Datamatchingacrossmultiplesources,andusingdatafromsamplesurveys (teachersandprincipals),appliessomelimitationsthatneedtobeconsidered inusingtheresults inthiswork.

NAPLAN datawere available for government schools and for a small number of Catholicschools(thoseparticipatinginthelowSESNP).Duetothelackofacommonschoollinkagekey,someschoolswereexcludedfromthematcheddataset.ThismostlyaffectedCatholicschools.

Improvement inNAPLANresultsmaynotbe thebestor themost idealwayofmeasuringNP-related effects. Changes may occur in student behaviour, engagement, skills andoutlooks and in ways which could have lasting and beneficial effects in a lifelongperspective.Studentlearningandskillsmaymarkedlyimproveinvariousclassroomsubjectareasbutnotnecessarily inNAPLANtestdomains,oronothermeasures.Byvirtueof theprogram, theNP initiativesmayhave contributed to lowSES schoolsbecoming farbetterplacesinnurturingstudentsandaddressingtheirneeds,andpromotingabetterandmoreengaged and active community as a result of participation without it leading directly to


10

improved studentoutcomes. Thedataused for this studydonot allowus toexplore thisissue.

Data fromthe2013TeacherSurveyandthe2014PrincipalSurveyprovidevery importantsources of information and insights about the implementation and effectiveness of NPstrategies and initiatives in schools. Although a sufficient number of schools haveparticipated inboth surveys, theoverall sample size forboth surveys are relatively small.The small sample sizemay impacton the statisticalpowerof someanalysespresented inthe report. To overcome this problem to some degree, results using data from the twosurveys have been cross validated and compared using different methods. Only the keythemesorpatternsthatareconsistentlyshownacrossallanalyseshavebeenreported.

Administrative data with a wider coverage of schools andmore detailed information ontheir NP implementation have now become available, and will be examined in the nextphaseofwork,butwewerenotabletoincorporatethedataintothisreport.


11

3. MethodologyandDefinitions

Todeveloprich,robustandcomprehensiveevidenceinevaluatingthelowSESNPeffectonNAPLAN outcomes across NP schools, this report draws on data and information from awidevarietyof sources.Thedatavaryconsiderablyas to referenceperiods,definitionsofmeasures, and types or scale of data. Because of this, for the analyses in this report atriangulation approach has been adopted in recognition of the various limitations of thedataandtotestthevalidityoffindingsfromdifferentsources.This issetoutinschematicform in Figure 3.1. Using this approach, we are able to draw on evidence from bothqualitativeandquantitativedataandexaminetheconsistencyoffindingsbasedondifferentdatasources.

Figure3-1 Flowchartoutliningapproachtotriangulationofdata

Es^mateschoolleveleffecton

studentNAPLANachievement

Mulo-levelmodelling

2011-2013NAPLANmatchedcohortdataat

bothstudentandschoollevels

Testrela^onshipbetweenNPeffectandkey

schoola`ributes

Spearmancorrelaoonanalysis

NAPLANschooleffectand2013TeacherSurvey&2014

PrincipalSurvey

Iden^fykeyNPStategiesrelated

tohigh/lowimpactschools

OrdinalLogisocregression

NAPLANschooleffectand2014

PrincipalSurvey

CaseStudiesofNPschools

CIRESReports(2012&2013)

Method

Stages

Data


12

Various statistical methods have been applied in analyses at different stages inconsiderationofthetypeorlimitationofeachdataset,including:

• multi-levelmodellingtoderivemeasuresofschooleffectsonNAPLANachievement,• Spearman's rank-order correlation test to identify the relationship between the

estimated level of NP effect for a school and the levels of effectiveness of NPinitiatives perceived by teachers or the principal. The Spearman's rank-ordercorrelationcoefficientsarecommonlyusedtomeasurethestrengthofassociationbetweentwoordinalvariables.

• Ordinal logistic regression models to identify the key NP initiatives or strategies

associatedwithlowandhighimpactschools.Sections 3.1 and 3.2 below describe how the key measures required for the multi-levelmodelshavebeendefinedandconstructedinthereport.

3.1 MeasuringstudentlearningIndividual studentperformance inNAPLAN is assessedonanational assessment scale foreachtest.Thesinglescaleallowsstudents,teachersandparentstomonitorprogressacrossthe years and compare results to those in previous years as students advance throughschool.Thecommonscaleforeachdomainisalsodividedinto10achievementbands(Band1–Band10)withsixbandsreported foreachyear level.Anationalminimumbenchmarkstandardissetbasedonbandscoreforeachyearlevel, i.e.,Band2forYear3,Band4forYear5,Band5forYear7andBand6forYear9.

While the scale for each of the five domains appears to be similar the results cannot bedirectly comparedacrossdomains. For this report,only resultsof studentperformance intwo key domains, Reading and Numeracy, have been used for the various analyses. Thisreport does not cover the full set of literacy and numeracy skills of students whoparticipatedinNAPLAN.

Figure3-2showsthatstudentsinNPschoolsperformconsistentlylowerthanthoseinnon-NP schools in both Reading and Numeracy, across different year-levels and across broadstagesof schooling.Students inNPschoolsperformonaverageatabout40pointsbelowstudentsinnon-NPschools.StudentsinNPschoolsarealsotwiceaslikelytoperformatorbelowthenationalminimumstandard(NMS)inbothReadingandNumeracy,andacrossallyearlevels.


13

Figure3-2 VariousmeasuresofstudentachievementsinNAPLAN(2011-2013cohorts)

StudentNAPLANachievement(MeanScore)byschooltype

StudentNAPLANachievementbyNationalMinimumBenchmarkStandard

3.2 Measuringschoolleveleffectsonstudentprogress

Childrenlearnanddevelopthroughtheirinteractionswithdifferentenvironments,includingtheir families, peers, schools and a wider community and society. Student learningoutcomes in schools often reflect a compounded effect from both individual and schoolfactors.

426

509545

586

408

497

552

597

378

471506

553

372

458

512

557

300

350

400

450

500

550

600

650

AtYR3 AtYR5 AtYR7 AtYR9 AtYR3 AtYR5 AtYR7 AtYR9

Reading Numeracy

Non-NPSchool NPSchool

13% 10% 19% 21%28% 23% 37% 39% 13% 19% 19% 26% 28% 36% 37% 42%

AtYR3

AtYR5

AtYR7

AtYR9

AtYR3

AtYR5

AtYR7

AtYR9

AtYR3

AtYR5

AtYR7

AtYR9

AtYR3

AtYR5

AtYR7

AtYR9

Non-NPschool NPschool Non-NPschool NPschool

Reading Numeracy

AtorbelowNMS AboveNMS


14

School level effects (or between school variability)measure the extent to which studentoutcomes differ across schools, after taking account of student individual factors. Forexample, students from a particular background attending one school might displayacademic outcomes that are stronger or weaker compared to students from the samebackgrounds attending other schools. Within school effects (or within school variability)measure the extent to which student outcomes differ within the same school based onstudentindividualcircumstances/attributes,suchasfamilybackgroundandmotivation.

Mixedeffectsormulti-levelmodellingmethodsprovidea robust approach for addressingchallenges associatedwithhierarchical data, suchasNAPLANdata for individual studentsnestedwithinschools.Usingamulti-levelmodellingapproach,weareabletopartitionandestimatethelevelsofeffectinstudentoutcomeswithinaschoolandbetweenschools.

To estimate to what extent NP implementation in a school has contributed to theimprovement of student learning outcomes, school effects were calculated for studentNAPLAN scores in Reading andNumeracy separately. The effectswere calculated using amultilevellinearmodel(LM)forcontinuousoutcomevariables.

In termsofequations, a simple linearmixedmodel (withone student levelpredictor andoneschoollevelpredictor)canbeexpressedas:

𝑦!" = 𝐵!! + 𝐵!!𝑥!" + 𝜀!"

𝐵!! = 𝛼!! + 𝛼!"𝑧! + 𝜇!

𝜀!"~ 0,𝜎!

𝜇!~ 0,𝜎!!!

Where Y is a continuous variable for student i in school j; β0j is the overallmean acrossschools,Xijrepresentsastudentlevelpredictorincludedinthemodel;Zjrepresentsaschoollevelpredictorincludedinthemodel;theschoollevelresidualµjisdefinedastheeffectofschooljontheirstudentoutcomes;andƐijistherandomerrorassociatedwithstudentiinschoolj.


15

4. AggregatelowSESNPeffectsonstudentlearning

The lowSESNPeffect foreachschoolcanbeestimatedbymodellingachievementscoresaftertakingaccountofboththepriorachievementofstudentsandthefactorsknownfromresearch and analysis to make the biggest differences to the variations in outcomesbetween schools. Using multilevel modelling to control for student and school levelinfluences on achievement, and including an indicator for low SES NP, it is possible tomeasurethe independentcontributionof lowSESNPschoolstostudentachievementandtomeasurehowwellaschoolperformscomparedtosimilarschools.

This section examines the levels of impact of the low SES NP initiatives on NAPLANachievementthroughthefollowingsteps:

1. identifyingthestudentandschoolfactorsthatinfluenceNAPLANresults,2. modelling school effects on NAPLAN achievement, controlling for the influential

factors,3. examiningthesizeanddistributionoftheschooleffects,and4. identify the NP effect by comparing the size of the school effects of the low SES

schoolsthatparticipatedinNPcomparedtothosenotparticipatinginNP.

4.1 Influentialstudentbackgroundandschoolfactors

To identifykey influencesonNAPLANachievement, two typesofmulti-levelmodelswereconducted. The first, a multilevel linear model for continuous variables, was based onReadingandNumeracyachievementscoresinYear5andYear9andincludedthefollowingstudentbackgroundcontrols:

• studentpriorperformance(measuredbyNAPLANscoresinYear3orYear7)

• student SES (basedona scale scorederivedusinganequallyweighted indexofparentaleducationandoccupation)

• ATSIstatus

• gender,and

• whetherthestudentwasfromalanguagebackgroundotherthanEnglish(LBOTE)ornot.

Thefollowingschool-levelpredictorswereusedfortheanalysis:

• schoolLocation(ARIAareas)

• schoolsector


16

• schoolSES(medianofstudentfamilySESindexwithintheschool)

• proportionofLBOTEstudents

• proportionofAboriginalstudents

• proportionofboys

• sizeofschool(enrolments)

• selectiveentryschoolornot(secondaryschoolsonly)

The model also included a flag for low SES NP schools by participation year to assesswhether there were independent effects for participation or not, and whether thiscontributedtoexplainingvariationinNAPLANReadingandNumeracyachievement.

Regressiondiagnosticswereundertakentoexaminethevalidityofthemodel.ThisincludedplottingofresidualsandrunningaKolmogorov-Smirnovtestfornormality.Foreachofthemodels in this section, the results showed that conditional residuals followed a normaldistribution and the results from the Kolmogorov-Smirnov tests for normality were notsignificant,suggestingnormalityofresiduals.

Table4.1showstheresultsofthelinearmixedmodels.ItreportstheimpactofarangeofstudentandschoolvariablesonstudentNAPLANscoresinReadingandNumeracyin2013.Resultsfromthetwomodelsaregenerallyconsistent.

4.1.1 Significantstudentlevelfactors

Previousstudentachievement,measuredinYear3in2011forYear5studentsandinYear7in 2011 for Year 9 students, appears to have the strongest predictive power in NAPLANoutcomesin2013forbothYear5andYear9cohorts.However,beingfromalowSESfamily,and fromanATSI backgroundexert independenteffectsonachievement inbothReadingandNumeracyforbothYear5andYear9cohorts.BeingfromalanguagebackgroundotherthanEnglishisassociatedwithgainsinNumeracyatbothyearlevels,andinReadingatYear9, independently of other factors. Being male is associated with a reduced achievementscoreinReadinginYear9andahigherscore,allelseequal,inNumeracyinbothYear5andYear9.

LBOTE appears to increase in effect in Reading fromprimary school to secondary school.Thismaysuggestthatalthoughbeingfromanon-Englishlanguagebackgroundisassociatedwith little advantage in Reading in the earlier years of schooling, LBOTE students inaggregatetendtoexcelinsecondaryschool.ThisisnotthecaseinnumeracywherethereisalreadyaclearadvantageinYear5.

There may be limitations or deficiencies in using LBOTE as a measure of educationaldisadvantage. LBOTE students cover potentially very diverse groups of students. Analysis


17

undertakenbytheNewSouthWalesDepartmentofEducationandCommunities(NSWDEC,2011)showthat inNewSouthWales,aftercontrollingfortheeffectofstudents’parentalbackground and school-level effects, LBOTE students with limited English languageproficiencyexperiencetwicethelevelofdisadvantageasthosewhoarerefugeesorwhoareIndigenous.However,byfarthemostdisadvantagedgrouparerefugeestudentswhohavelimitedEnglish languageproficiencyandhavebeen inanAustralian school formore thanoneyear.

Table4-1 Estimates of predictors of NAPLAN Reading and Numeracy achievement(standardisedscores)in2013:Year5andYear9students

Year5Score Year9Score Reading Numeracy Reading Numeracy Intercept -0.333 -0.492 (***) -0.046 0.55 (***) StudentFactor

Priorperformance(Score) 0.711 (***) 0.688 (***) 0.784 (***) 0.81 (***)FamilySES 0.005 (***) 0.004 (***) 0.003 (***) 0.00 (***)

Aboriginalstatus -0.053 (***) -0.068 (***) -0.090 (***) -0.05 (***)LBOTEstatus 0.007 0.134 (***) 0.036 (***) 0.09 (***)

Gender(Male) -0.002 0.076 (***) -0.056 (***) 0.03 (***) Schoollevelfactor

SchoolSES 0.009 (***) 0.008 (***) 0.006 (***) 0.00 (***)Aboriginalconcentration(%) -0.006 (**) -0.011 (***) -0.015 (***) -0.02 (***)

Schoolsize(enrolments) -0.020 (***) -0.001 -0.002 -0.02 (**)CatholicvsGov. -0.016 -0.096 (***) 0.012 0.01

Non-selectivevsFullyselective -0.221 -0.75

PartiallyselectivevsFullyselective -0.160 -0.70

2009NPschoolvsNon-NPschool -0.043 (**) 0.026 0.017 0.08 (***)2010NPschoolvsNon-NPschool 0.009 -0.005 0.029 (*) 0.04 (**)2011NPschoolvsNon-NPschool 0.017 0.055 (***) 0.081 (***) 0.08 (***)2012NPschoolvsNon-NPschool -0.038 -0.035 -0.025 -0.02

***Indicatesresultsarestatisticallysignificantat1%level;**at5%leveland*at10%level;Allcontinuousvariableshavebeenstandardisedforcomparisonofcoefficients.

Theresults inTable4-1alsosuggestthatthesexofstudentsappearstohaveanoppositeeffect for Reading compared to Numeracy, with male students progressing better inNumeracyandfemalestudentsdoingbetterinReading.

AnotherinterestingcontrastisthattheeffectforpriorachievementwhichforNumeracyinYear9isstrongerthanforReading,whereastheoppositeistrueinprimaryschoolatYear5wheretheeffectofpriorachievementisstrongerforReadingthanNumeracy.


18

4.1.2 Significantschoollevelfactors

After controlling for student background characteristics, school-level attributes contributeindependently toNAPLANachievement scores (Table 4-1).A small, but significant part ofthe variation in student progress is attributed to school environmental factors, such asschool level SES, and concentration of ATSI students. For achievement scores in bothReadingandNumeracy,schoolswithahigherSESstudentintakeandalowerconcentrationofATSIstudentsgainmorepositiveresults,allelseequal.

School size (measured by school full time enrolments in 2011) appears to have a smallnegativeeffectonstudentprogressinReadingforYear5andNumeracyforYear9.

4.1.3 NPschoolleveleffects

SchoolsparticipatinginthelowSESNPexhibitoverallapositiveeffectonstudentprogress,albeit with variations in Reading and Numeracy and between different years of NPimplementation.

SchoolscommencingthelowSESNPin2011showonaveragethestrongestNPeffectwhileschools participating from 2012 show the least effect. The patterns associatedwith timespentintheprogrammayreflectseveralfactors,includinga‘lag’effectand‘halo’effectforthe2009NPschools.TheremaynotbesufficienttimefortheNPinitiativetotakeeffectonthe2011-2013cohortsinschoolsthatjoinedlowSESNPin2012.

TheNPeffectappearstobemoreconsistentorstrongeronNumeracythaninReadingandmore consistent for the Year 9 cohort than for the Year 5 cohort. This warrants furtherinvestigations into the difference (if any) in school practices that contribute to thesedisparities,e.g.,whether it isdue to teachers/schoolsdevotingmore timeor resources inNumeracy related activities in secondary schools compared to primary schools. Someresearchersbelievethatmathachievementismoredependenton“withinschoolactivities”andrespondsmorequicklytocurriculumchange.PreviousstudiesbothwithinAustraliaandoverseas have found that Reading intervention programs have more difficulty producingdemonstrablegainsthatmathprograms(Loveless2013;Helal2014).

4.2 EstimatingthesizeofschooleffectsIn order to produce school effect measures that are comparable across NP and non-NPschools,school level residuals foreachof the2,112schools forwhichdatawereavailablewere derived using the multilevel models. The models were re-estimated including onlyvariablesforbothstudentandschoollevelfactorsthatwereshowntoexhibitastatisticallysignificanteffectonstudentperformance.NPidentifyingvariableswereexcludedfromthe


19

modelestimationinordertoidentifyandestimatetheeffectofNPimplementationamongNPschools(seeTable4-2foralistofincludedvariables).Table4-2 Studentandschoollevelfactorsusedforadjustingschoolleveleffect

Linearmixedmodel

Schooleffectmeasure

StudentNAPLANscoreinReadingandNumeracy

Studentlevelvariablesandvariabletype

Priorperformance(Continuous)FamilySES(Continuous)

Aboriginalstatus(Binary)LBOTEstatus(Binary)

Gender(Male)(Binary)

Schoollevelvariablesandvariabletype

SchoolSES(Continuous)ATSIconcentration(Continuous)

Schoolsize(Continuous)

Selectiveschool(Binary,Year7-9cohortonly)

Regression results fromtheNPeffectsmodel showthat therearesignificantvariationsatthe school level in studentNAPLAN achievements regardless of year levels and domains,althoughtheyaresignificantlyreducedafteradjustingforstudentandschoollevelfactors.Theestimated school level variances and Intra-classCorrelationCoefficient (ICC) statisticsareprovidedinTableA1intheAppendix.

Theestimatedschooleffectforeachschooltakesintoaccountanumberofkeystudentandschoollevelfactorsthatinfluencestudentachievement,suchasstudentpriorachievement,the concentration of students by family SES, concentration of Aboriginal students, schoolsize and school type. Itmeasures howmuch each school ‘value-added’ to their studentNAPLANachievementandhowwellaschoolperformscomparedtosimilarschools.

Theschoollevelresidualsderivedoriginallyfromthemodelsarenormallydistributed(withameanofzeroandconstantvariance/standarddeviation).Theyareessentiallyarelativeorrankingmeasureonschooleffect.Notallschoolsexperiencedordisplayedthesamelevelofeffects.Themajorityofschoolsshowanaverageleveleffect,withsomeschoolsshowinglargerthanaverageeffectwhilesomeschoolsshowasmallerthanaverageeffect.

Figure4-1showsthedistributionsof school level residualsderived fromthemixedmodelmeasuringtheeffectsonstudentNAPLANaveragescorewithineachschool.


20

Figure4-1 Distributionsofestimatedschool leveleffects(school levelresiduals)byYearlevelsandDomains

SchooleffectonstudentNAPLANscores

Year5Reading

Year5Numeracy

Year9Reading

Year9Numeracy


21

4.3 ComparingschooleffectsbetweenlowSESNPschoolsandlowSESnon-NPschools

The regression analyses in Section 4.1 show that schools participating in the low SES NPexhibitoverall apositiveeffecton studentprogress, albeitwithvariations inReadingandNumeracyandbetweendifferentyearsofNP implementation. SchoolscommencingtheirlowSESNPin2011showonaveragethestrongestNPeffectamongallNPschools.

In this section,bycomparing theestimatedschooleffectsamong lowSESNPschoolsandlowSESnon-NPschools,weareabletofurtheridentifyandquantifythesizeofNPeffectson student NAPLAN achievement. The low SES schools are defined as schools that arerankedatthelower50thpercentilebasedontheirschoolSESindex(measuredasmedianofstudent family SES index within the school). To make the school residual results moremeaningful, the standardised school residuals were transformed back to the scale ofNAPLANscore(Figure4-2).

Figure4-2 SchooleffectonstudentNAPLANscores, comparing lowSESNPand lowSESnon-NPschools

low SES NP 2011 low SES Non-NP schools

Year5Reading

Year5Numeracy

Year9Reading

Year9Numeracy


22

Figure4-2 compares thedistributionsof schooleffectson studentNAPLANscoresamonglowSESschoolsbyNPparticipationstatus.ResultsshowapositivegaporadvantageforlowSESNPschoolscomparedtothe lowSESnon-NPschoolsaftercontrollingfortherangeofother influential student and school influences. The advantage is more consistent orstrongeronNumeracythanReadingandmoreconsistentfortheYear9cohortthanforYear5. This is consistent with the regression results presented previously in section 4.1. Thenarrow spread of school effect sizes in Reading for both Year 5 and Year 9 cohortsmayreflect that Reading intervention programs have more difficulty producing demonstrablegains thanmathprograms, or they aremoredifficult tomeasure inNAPLAN instruments(seesection4.1formoredetaileddiscussion).

ResultsforeffectsonstudentNAPLANscores(Figure4-2)showthatstudentsinthelowSESNPschoolsperformonaverage3pointshigherinYear5Numeracy,3pointshigherinYear9Readingand6pointshigherinYear9Numeracy,comparedtothesamecohortinlowSESschoolsthatdidnotparticipateinNP.

NotallNP schoolsexperiencedordisplayed the same levelofeffectson student learningoutcomes after introducing the low SES NP reforms. Some NP schools show little or noeffectatall,despiteteachersandprincipals intheirsurveyresponsesreportingchangesinpracticesandsomeevenexpressingaview that the reformshadhadamarked impactattheirschool(CIRES,2015).OtherschoolsdisplaymarkedeffectswithstrongimprovementinNAPLANachievement.ApartfromtheschoollevelfactorsincludedinthelowSESNPeffectestimation models, there are other school level factors that may have an influence onstudent NAPLAN achievement, such as strategic and operational differences in NPimplementation and varying conditions or circumstances in which schools are operated.Section5willfocusonvariationsofschooleffectsamongallNPschoolsandthevariationsintheirNPinitiativesandpracticesthatmayhaveinfluencedtheirschooloutcomes.


23

5. Initiativeslinkedtobetterstudentoutcomes

This chapter presents the results of analyses attempting to identify the practices andfeaturesofNPschoolsthatcontributedtothedifferentialimpactofNPreformsonstudentoutcomes.Whatarethereforminitiativesthathavecontributedtosomeschoolsrecordingstrong impact and improvement, and what characterises schools that show no or littleimpact?

The chapter begins looking at the broadNP school reformareas andwhether a focus byschoolsonparticularreformarealedtogreatergains.Toexaminethis,datafromprincipalsontheimpactofreformsareusedinconjunctionwithNAPLANresultstoassesstheimpactofbroadreformareas.Wecommencewithananalysisbasedontheinformationreportedbytheprincipalsonwhatinitiativesandschoolpracticeswereimplementedattheirschoolsandwhichreformstheyfelthadthegreatestimpact.

Following this, attention turns to examine the information provided by teachers on thepositivechangesthatoccurredintheirschoolsafterNPimplementationlinkedtochangesinteacherpracticesandchangesinschoolmanagement.

Thechapterconcludesbylookinginafarmoredetailedwayatindividualinitiativeswithinreform areas to identify individual initiatives and practices that accounted for strongerschooleffects.Analyseshavebeenundertakentoidentifywhetherandhowtheseinitiativesorschoolpracticesmayhavecontributedtotheimprovementinstudentlearningoutcomes.

The school effectsmeasure used in this section is based on the results of themultilevellinearmodeloutlinedinsection4.

5.1 KeyNPreforms

To improve student learning outcomes, the Low SES NP sought to build capacity inparticipatingschools.TheLowSESNPfeaturedsixreformareasthatencompassedavarietyofinitiativesimplementedbyschools:

ReformArea1: Incentivestoattracthigh-performingteachersandprincipals

ReformArea2: Adoption of best practice performance management and staffingarrangementsthatarticulateaclearroleforprincipals

ReformArea3: School operational arrangements that encourage innovation andflexibility


24

ReformArea4: Provideinnovativeandtailoredlearningopportunities

ReformArea5: Strengthenschoolaccountability

ReformArea6: External partnerships with parents, other schools, businesses andcommunitiesandtheprovisionofaccess.

Allschoolswererequiredtoimplementactivitiesfromoneormoreofthesixreformareasover the duration of their four-year participation in the Partnership.Government schoolswere required to address all areas in each year of their participation. Schools could alsochoose to introduce strategies to address activities from the two other Smarter SchoolsNational Partnerships—Literacy and Numeracy and Improving Teacher Quality—wherethese also addressed a specific reform area. Schools were able to choose strategiesassociatedwithotherreformareasaspartoftheirschoolconsultationprocess.

5.2 Overall levels of change linked to reform areas as perceived byprincipals

InSeptember2014,principalsfrom267NPschoolsrespondedtoasurveyaddressingtheirexperiencesofandreflectionsontheimpactoftheLowSESNPreforms.ThissurveysoughtperceptionsfromtheprincipalofthelowSESNPprogramoverall,togetherwithinformationontheirownschool’sprocessesand initiatives.ComparedtoanearlierPrincipalSurvey in2012, principals in this survey were able to draw on a more sustained experience andunderstanding of the reform areas, and to project a picture of the sustainability of theirschoolinitiatives.

Consistentwiththeearliersurvey,principalswereverypositiveabouttheeffectivenessofmostof theNP initiatives, including indeveloping their existing staff capacity, developingoperationalarrangementsthatencouragedinnovationandflexibilityinstaffing,andbuildingteacherandstudentaccesstomoreinnovativeandtailoredlearningopportunities.

At the same time, principalswere least likely to express ‘strong’ support for propositionsthat they had been able to either attract or retain quality staff. In their open-endedresponsesanumberofprincipalsclarifiedsomeofthecomplexitiesassociatedwiththeLowSESNPprogramtoassistinunderstandingthispicture.Theymadereference,forexample,to the enhanced mobility of quality teachers whose development had been supportedthrough Low SES NP initiatives – such teachers were in demand on leadership teamselsewhereandforatleastsomeschoolswerethereforedifficulttoholdonto.

In relation to student learning outcomes, principals seemed to bemore reserved in theirsenseofimpact.Almostallprincipals(97percent)feltthattheprogramhadallowedtheir


25

studentstoaccess‘moreinnovativeandtailoredlearningopportunities’andacomparableproportion agreed that as a result of their participation in the program ‘student learningoutcomes improved’. Only a third, however, ‘strongly agreed’ that the program hadachievedimpactonstudentoutcomes(CIRES2015).

The interest in thecurrentanalysis is toexaminetherelationshipbetweenthesizeof theimpactofthelowSESNPasmeasuredbyschooleffects(residuals)relatedtoReadingandNumeracy achievement and principal views on what reform areas contributed most toimpact.Principalswereaskedinthe2014surveyfivegroupsofquestionsthatarebroadlyrelated to the six NP reform areas. For each group of questions, principals responded tothreeaspectsofNPimplementationincluding:

• the intensity of NP implementation (i.e., the total number of NP initiativesimplemented),

• thelevelofeffectiveness(from“Notatalleffective”to“Highlyeffective”),and• thetotalnumberofNPinitiativesthatwerecontinuingattheschool.

ThreemeasureshavebeenconstructedforanalysesoftheseaspectsofNPimplementationinaschool.The firstandthirdmeasuresaredirectaggregationmeasuresbycounting thetotalnumberofvalid responseswithineachgroupofquestions.For thesecondmeasure,the categorical responses have been converted to ordinal scores (i.e., 0 for Not At AllEffective, 1 for Somewhat Effective, 2 for Effective and 3 for Highly Effective) and thensummed toestimate foreachschool theoverall levelofNPeffectivenessasperceivedbytheprincipal.

As all three measures derived from the survey are ordinal in scale, the Spearman rankcorrelation method was used to measure the association between school NP effects (asmeasuredbyVAeffectscores)andtheeffectivenessofNPimplementationasreportedbyprincipalsinthesurvey.

Figure5-1shows thevaryingnumberof initiatives schoolshave implementedwithineachreformareasacrossallNPschools.

Table 5.1 presents the results of the correlation analysis between the three measures(numberofinitiativesineachreformarea,principalperceptionsoflevelofeffectivenessofreforms,andwhethertheinitiativeshavecontinuedpostlowSESNP)andNPschooleffectVAscores.

Theresultssuggestthatforprimaryschools(Year5)thereisapositiverelationshipbetweenNP school effect scores and principal-reported level of effectiveness for initiativesimplemented as part of Reform Area 4, providing innovative and tailored learningopportunities.Thehigherlevelofeffectivenessasperceivedbytheprincipal,themorelikelytheschoolshowsahigherNPeffect.Theresultsalsoshowforprimaryschoolsarelationship


26

betweenthenumberofinitiativesimplementedfromReformArea4andNPschooleffects.Themore initiatives implemented in a school, the higher theNP impact in the school. Itsuggeststhatschoolsthatplacedmoreeffortinthisreformareaweremorelikelytoshowastrongereffectonstudentlearningoutcomes.

Figure5-1 DistributionofNPinitiativesimplementedinschoolsbyreformareas

RF1:High-performingteachersand

principals

RF2-3:PerformanceandStaffingmanagement

RF4:Studentlearningopportunities

FR5:Schoolaccountability

RF6:Externalpartnershipswithparents,otherschools,businessesandcommunities


27

Table5-1 Tests of Spearman Correlation between school NP effects and theeffectivenessofNPimplementationasperceivedbytheprincipal

Reading Numeracy

Coefficients Prob>|r=0| Coefficients Prob>|r=0| Primary(Year5)NumberofNPinitiativesimplemented

Attracthighperformanceteachers(RF01) -0.05 0.59

0.03 0.76Staffing&performancemanagement(RF02-03) -0.03 0.71

0.06 0.51

StudentLearningopportunity(RF04) 0.17 0.04 ***

0.18 0.04 ***Continuousimprovement(RF05) 0.10 0.25

0.15 0.07

Engagementwithparents/carers(RF06) 0.12 0.17 0.12 0.16 LevelofeffectivenessofNPinitiatives


0.03 0.76Staffing&performancemanagement(RF02-03 -0.03 0.71

0.06 0.51

StudentLearningopportunity(RF04) 0.17 0.04 *** 0.18 0.04 ***Continuousimprovement(RF05) 0.10 0.25

0.15 0.07

Engagementwithparents/carers(RF06) 0.12 0.17

0.12 0.16NumberofNPinitiativesretainedatschoolafterNP

Attracthighperformanceteachers(RF01) 0.17 0.05 ***

0.05 0.53Staffing&performancemanagement(RF02-03 0.04 0.71

0.00 0.97

StudentLearningopportunity(RF04) 0.09 0.33

0.09 0.36Continuousimprovement(RF05) 0.21 0.03 ***

0.09 0.36

Engagementwithparents/carers(RF06) 0.12 0.23

0.14 0.15 * Secondary(Year9)NumberofNPinitiativesimplemented


-0.22 0.10 **Staffing&performancemanagement(RF02-03) -0.17 0.19

-0.08 0.55

StudentLearningopportunity(RF04) -0.09 0.49

-0.03 0.85Continuousimprovement(RF05) -0.01 0.92 0.10 0.44

Engagementwithparents/carers(RF06) 0.03 0.81 -0.17 0.20 LevelofeffectivenessofNPinitiatives

Attracthighperformanceteachers(RF01) 0.08 0.56 -0.19 0.16

Staffing&performancemanagement(RF02-03 0.04 0.75

-0.08 0.54StudentLearningopportunity(RF04) 0.10 0.45 -0.02 0.87

Continuousimprovement(RF05) 0.17 0.21

0.11 0.40Engagementwithparents/carers(RF06) 0.20 0.14 * -0.06 0.67

NumberofNPinitiativesretainedatschoolafterNP

Attracthighperformanceteachers(RF01) 0.09 0.58

-0.20 0.20

Staffing&performancemanagement(RF02-03 0.05 0.73

-0.10 0.49StudentLearningopportunity(RF04) 0.04 0.78

0.05 0.73

Continuousimprovement(RF05) 0.07 0.63 0.07 0.65

Engagementwithparents/carers(RF06) -0.16 0.27 -0.28 0.05 ******Indicatesthatresultsarestatisticallysignificantat5%level;**indicatesstatisticalsignificanceat10%leveland*at15%level.

The results suggest that primary schools that placed more effort on initiatives linked toproviding innovative learningopportunitiesweremore likely to showastrongereffectonstudentlearningoutcomes.


28

The relationships appear to be stronger ormore consistent for primary schools than forsecondary schools. This may be due to the small number of secondary schools (i.e., 46secondary vs 128 primary schools) included in the analysis, which reduces the statisticalpowerinthistest.

5.3 NPeffectsassociatedwithchangesinteacherandschoolpractice

Teachersinschoolsareanimportantsourceofinformationabouttheimplementationandeffectiveness of National Partnership strategies. Their views and reflections on the NPimplementationandeffectivenessintheirownschoolswerecollectedinthe2013TeacherSurvey.Atotalof2,408teachersfrom290NPschoolsrespondedtothesurvey.

ThelowSESNPevaluation(CIRES,2015)reportedthattheoverallviewsofteachersabouttheLowSESNPinitiativeswerestronglyfavourable,withthemajorityofteachersindicatingthatthelowSESNPinitiativeshadencouragedthekindsofchangesintended.

Inthisreport,asetofmeasureshasbeenconstructedusingteacherresponsestoaseriesofquestionsinthesurvey,includingthelevelofperceivedpositivechangeinteacherpracticesand the perceived level of positive change in school management as a result of NPimplementation.Bothmeasuresarecalculatedasthepercentageofteacherswhoprovidedavalidresponsefortherelevantquestions.

The Spearman rank order correlation method has been applied in testing the extent towhichthereisanassociationbetweenNPschooleffects(theNPeffectonstudentNAPLANscores)andpositivechangesinaschoolresultingfromNPimplementationasperceivedbyteachers. The Spearman correlation coefficients are calculated based on the ranking ofvariablesinthetest.Apositivecoefficientindicatesapositiverelationship,whileanegativeindicatesareversedrelationship.Forexample,thepositivecoefficientsforquestionsrelatedtochangesinteacherpracticeindicatesthatthemoreaspectsofchangeinteacherpracticeinaschool,themorelikelytheschoolshowsahighlevelofNPeffectonNAPLAN.

Table5.2presentstheresultsforNPeffectsonstudentNAPLANscores.Thereisapositiverelationship between the level of NP effect and the level of positive changes related toparents and community engagement in school. This suggests that the higher the level ofeffectivenessinparentsandcommunityengagementinaschool,themorelikelytheschoolshowsahighlevelofNPeffectoutcomeonNAPLANachievement.


29

Table5-2TestofSpearmanCorrelationbetweenNPschooleffectandtheeffectivenessofNPimplementationasperceivedbyteachers

Reading Numeracy Coefficient Prob>|ρ| Coefficient Prob>|ρ|

Primary(Year5)

Changesinteacherpractice Meetingstudentindividuallearningneeds 0.08 0.31 0.13 0.11 *

Communicatingwithparents/carers 0.16 0.05 *** 0.17 0.04 ***Managingclassroombehaviours 0.08 0.34 0.11 0.18 Involvingparents/carersinstudentlearning 0.16 0.04 *** 0.04 0.64 Beingsupportedinclassroom 0.10 0.20 0.10 0.21 Abletoexplainschoolgoals 0.15 0.06 ** 0.16 0.04 ***

Teachingingeneral 0.12 0.12 0.06 0.44 Levelofpositiveviewsonschoolmanagement Theschoolrunsmoresmoothly 0.13 0.11 * 0.05 0.56 Teachersusebetterstrategiestosupportlearning 0.08 0.28 0.02 0.82 Improvementinthewayteachersrelatetostudents 0.06 0.44 0.04 0.58 Thisschoolusesmoreeffectivemethodstodetermine

howwellteachersareperforming 0.14 0.10 ** 0.12 0.15 *

Thereisamorestrategicapproachtoschoolplanning 0.09 0.27 0.11 0.15 *Schoolcommunicatesbetterwithparentsandcarers 0.10 0.20 0.17 0.03 ***

Schoolismoreeffectiveinengagingparentsandcarersfromdiversesocialandculturalgroups 0.12 0.15 * 0.19 0.02 ***

Schoolismoreengagedwithitswidercommunity 0.16 0.05 *** 0.15 0.06 **Schoolhasbecomeabetterplaceinwhichtoteach 0.15 0.06 ** 0.10 0.23 Schoolhasbecomeabetterplaceforstudentstolearn 0.13 0.10 ** 0.06 0.43

Secondary(Year9)Changesinteacherpractice Meetingstudentindividuallearningneeds 0.12 0.38 0.10 0.44 Communicatingwithparents/carers 0.07 0.61 0.24 0.08 **

Managingclassroombehaviours -0.07 0.61 0.13 0.37 Involvingparents/carersinstudentlearning 0.08 0.56 0.19 0.17 Beingsupportedinclassroom 0.13 0.35 0.15 0.29 Abletoexplainschoolgoals -0.05 0.71 0.17 0.23 Teachingingeneral 0.17 0.21 0.19 0.16 Levelofpositiveviewsonschoolmanagement Theschoolrunsmoresmoothly 0.02 0.89 0.04 0.78 Teachersusebetterstrategiestosupportlearning -0.03 0.80 0.12 0.36 Improvementinthewayteachersrelatetostudents 0.04 0.75 0.15 0.27 Thisschoolusesmoreeffectivemethodstodeterminehowwellteachersareperforming 0.01 0.97 0.04 0.77

Thereisamorestrategicapproachtoschoolplanning 0.09 0.52 0.19 0.16 Schoolcommunicatesbetterwithparentsandcarers 0.22 0.10 ** 0.14 0.31 Schoolismoreeffectiveinengagingparentsandcarersfromdiversesocialandculturalgroups 0.24 0.09 ** 0.26 0.06 **

Schoolismoreengagedwithitswidercommunity 0.22 0.10 ** 0.25 0.06 **Schoolhasbecomeabetterplaceinwhichtoteach 0.12 0.37 0.13 0.34 Schoolhasbecomeabetterplaceforstudentstolearn 0.05 0.71 0.13 0.33

***Indicatesresultsarestatisticallysignificantat5%level;**at10%leveland*at15%level.


30

5.4 IndividualNPinitiativesorpracticesassociatedwithNPeffect

AllNPschoolsmaybefacedwithsomecommonchallenges,suchaslowattendanceandlowachievement, however some conditions or circumstances they operate under may vary.Although government schools were required to address all six areas of the NP reform,schools could prioritise their efforts or resources on particular initiatives or activities toaddresstheiruniquechallengesorconditions.

In this section, based on information provided by the principals (in the 2014 survey),wefocusontheimpactof individualNPinitiativesorstrategiesandattempttoidentifywhichparticularinitiativesorstrategiesmayindividuallyhavecontributedtotheimprovementofstudentNAPLANoutcomes.

5.4.1 GroupinglowSESschoolsbasedonestimatedlevelofimpactUsing the results of the multilevel linear model estimating Reading and Numeracyachievement to derive school effects (school level residuals), all schools (NP andnon-NP)weregroupedintothreecategories:

1. schools displaying higher NP effects on student NAPLAN scores, defined asschoolsrankedtowardsthetopofthedistribution(top25percent)

2. schoolsdisplayingpositive,butmoremodesteffectsonNAPLANresults,defined

asschoolsrankedinthemiddle50percent,and3. schoolsdisplayingnoeffectsorweakeffects,definedasschoolsrankedtowards

thebottomofthedistribution(bottom25percent).

Schoolswere rankedbasedon the values of their school residuals separately onReadingandNumeracyandseparatelyfortheirYear5andYear9cohorts.SchoolswithbothYear5andYear9resultsweregroupedaccordingtothehigherrankingamongthetworesults.

Table 5.3 shows the distributions of low SES NP schools according to their school effectcategory—low,middleorhigh—andbyschooltype.

ForReadingandNumeracy,about30percentoflowSESschoolsweretowardsthetopofthe performance distribution displaying higher levels of school effects on studentachievement,afurther50percentwereinthemiddlegroup,whileabout20percentwerein the bottom group exhibiting low effects on student achievement. In primary schools,nearlyeightoutoftenlowSESNPschoolsdisplayedatleastmodesteffectsonReadingandNumeracy performance with over 30 per cent in the top group of schools. Secondaryschoolsontheotherhandsawaround85percentinthetoptwogroups,butamuchhigher


31

proportion—nearly40ofthisgroup—witheffectscoresplacingtheminthetopgroup.Onlyaverysmallproportionofcombinedschoolswereinthebottomgroup.

Table5-3 LowSESNPschoolsbyschooleffectcategory

Schooltype Schoolgroup Reading Numeracy

Primary Low 76 22% 80 23%

Middle 167 48% 163 47%

High 103 30% 103 30%

Total 346 100% 346 100%

Secondary Low 15 16% 13 14%

Middle 41 44% 47 50%

High 37 40% 34 36%

Total 93 100% 94 100%

Combined Low 2 4% 0 0%

Middle 37 79% 32 68%

High 8 17% 15 32%

Total 47 100% 47 100%

AllNPschools Low 93 19% 93 19%

Middle 245 50% 242 50%

High 148 30% 152 31%

Total 486 100% 487 100%

ItisrelevanttonotethatgroupinglowSESNPschoolsbasedontherankingoftheirschooleffectvalues is insomerespectsarbitrary,andtheremaynotbeanymaterialdifferencesamong some schools with residual values close to the cut off points separating groups.However,ourprimary interest istoexaminetherelationshipsbetweenthe levelofschooleffectsandtheimplementationofindividualinitiativesorstrategies,andtofurtheridentifywhatinitiativesorstrategiesthatmayseparateschoolsatthetopandatthebottomoftheschoolrankingbasedoneffectvalues.

Ordinallogisticregressionwasusedtotestthestatisticalsignificanceofschoolattributesinpredictingaschool’slikelyNPeffectgroup:low,mediumorhigh.TheresultsarepresentedinTable5.4.Theestimatesforeachpredictorcanbeinterpretedasmeaningthatforaoneunitincreaseinthepredictor(suchasattendance,forexample)weexpectanincreaseintheorderedlogoddsprovidedinthesizeoftheestimate(0.041)ofbeinginahigherNPeffectcategory,givenalloftheothervariablesinthemodelareheldconstant.

Theresultsshowthat:


32

• Variationsinstaffexperience,ageandmale-femalestaffratiodonotappeartobepredictiveofaschool’sNPeffectoutcome.However,staffmobility(measuredasproportionofstaffturnoveroverin2010)appearstobeasignificantpredictorforNPeffectinNumeracy,takingintoaccountotherschoolfactors.Schoolswithlowerturnovertendedtobeinthehigherimpactcategory,allelseequal.

• TheconcentrationofLBOTEstudents inschoolappearstobeassociatedwitha

school’s likely effect groupoutcome forNumeracy, albeitwith aweak level ofsignificance.SchoolsinhighercategoriesofNPeffectforNumeracytendtohavehigherpercentageofLBOTEstudents,allelseequal.

• Theattendancerate ina lowSESNPschoolalsoappears tobeassociatedwith

theschool’slikelyeffectgroup.SchoolsinhighercategoriesofNPeffecttendtohavehigherattendancerates,allelseequal.ForNumeracy,attendancehadthestrongestindependenteffectofNPeffectcategorylocation.

Table5-4 Ordinal logistic regression model results identifying significant predictors ofsizeofNPschooleffectonNAPLANachievement

NPEffectonReading NPEffectonNumeracy

Estimate p-value Estimate p-value

SchoolSES-EOIindex 0.003 0.01 -0.001 0.58

PercentofLBOTEstudents -0.005 0.19

0.006* 0.15

SchoolLocation 0.144 0.54 0.077 0.74

Attendancerate 0.047* 0.15 0.064*** 0.03

Staffprofile

StaffMobility(turnoverrate) -0.001 0.79

-0.007 0.10**

AgeprofileIndex 0.008 0.61 -0.009 0.57Maletofemalestaffratio 0.002 0.71

0.007 0.21

Lengthofcurrentposition 0.012 0.38

0.011 0.43

***Indicatesresultsarestatisticallysignificantat5%level;**at10%leveland*at15%level.


33

5.4.2 GreatestimpactNPinitiativesTo identify the sorts of strategies that high impact schools have implemented, and thatappeartomakeadifference,anordinallogisticregressionmodelwasappliedestimatingtherelationship between implementation of NP initiatives and predicted school NP outcomecategory(low,medium,highimpact).Theanalysiswasundertakenintwostages.First,allofthestrategiesschoolshadimplementedaspartoflowSESNP(asreportedbytheprincipal)were included as explanatory variables (in binary values, 1= Yes and 0=No). From thisanalysis,becauseof the largenumberof initiativesand the limitednumberof schools forwhichprincipal responseswereavailable, initiativeswere identifiedwhichhadprobabilityvalueslessthan0.4oneitherReadingorNumeracy.Second,ananalysiswasconductedontheshorterlistofinitiativeswhichweremorelikelytobeassociatedwithimpact.

Table 5-5presents the results of theordinal regressionprocedurewhere the groupingofschoolswasbasedonthelevelofNPeffect(low,medium,high).Selectedinitiativeswereincluded,thosesuggestingahigherinitialestimationofimpact.

Theresultsshowtheeffect foreach initiativeestimatedasan increase in theordered logoddsofbeinginahigherNPeffectcategory,withalloftheothervariablesinthemodelheldconstant. TheWald values and probabilities are also reported to identify if the effect isstatisticallysignificant.ResultsarepresentedforReadingandthenNumeracy.

For Reading, schools that have implemented the following initiativesweremore likely toshowhigherimpact:

• Providedopportunitiesforprofessionallearninganddevelopment(RF1)• ProvidedIndividualLearningPlans(ILPs)forstudentsneedingassistance(RF4)• Providedprofessionallearningforteachersonmeetingindividuallearningneeds(RF4)• Usedassessmentandotherdatatoidentifystudentneeds(RF4)• Engagementwithemployers(RF6)• Partnershipswithprimaryschools(RF6)

ForNumeracy,schoolsthathaveimplementedthefollowinginitiativesweremorelikelytoshowhigherimpact:

• Providedopportunitiesforprofessionallearninganddevelopment(RF1)• ProvidedIndividualLearningPlans(ILPs)forstudentsneedingassistance(RF4)• Providedprofessionallearningforteachersonmeetingindividuallearningneeds(RF4)• Usedassessmentandotherdatatoidentifystudentneeds(RF4)• Madeplanningand/or reportingprocesseswithin theschoolmorepubliclyavailable

(RF5)• Engagementwithemployers(RF6).


34

Table5-5 NPstrategiesassociatedwithNPeffectimpactcategory(low,medium,high)in

ReadingandNumeracy Reading Numeracy

Estimate Wald Prob Estimate Wald Prob

ReformArea1:Attracthighperformanceteachers

Establishedleadershipandstrategicpositions -0.829 3.684 0.055 ** 0.072 0.027 0.869

Providedmentoringsupporttoteachers -0.120 0.088 0.766

-1.319 9.099 0.003 ***Providedattractiveterms/conditionsoutsidestandard

entitlements -0.306 0.223 0.637

-1.643 5.977 0.014 ***Providedopportunitiesforprofessionallearningand

development 0.371 0.773 0.379

0.471 1.152 0.283

ReformAreas2-3:Staffingandperformancemanagement Employedadditionalparaprofessionalsinlearningsupportroles 0.314 0.564 0.453

0.285 0.439 0.507

Employedadditionalparaprofessionalsinothersupportroles -0.521 1.622 0.203

0.520 1.519 0.218

Adoptedteam-basedapproachestoteachingandplanning 0.338 0.795 0.373

-0.571 2.155 0.142 *Introducedincreasedflexibilityintimetablingand/orschool’s

hours -1.078 8.557 0.003 *** -0.076 0.042 0.838 Introducedgreatercooperationwithotherschoolstoshare

resources 0.174 0.205 0.651

0.381 0.949 0.330Providedqualityprofessionallearningforschool-basedteams

ofstaff 0.267 0.474 0.491

-0.279 0.488 0.485 Implementedrelevantandappropriateprofessionallearning

forstaff -0.500 0.870 0.351

-0.184 0.112 0.738 Providedprofessionaldevelopmentonarangeofbehaviour

managementtheoriesandapproaches -0.696 2.669 0.102 * -0.666 2.410 0.121 *Engagedstaffthroughprofessionaldialogueonbehaviour

management 0.303 0.489 0.485

0.511 1.367 0.242

ReformArea4:StudentLearningopportunity

Usedassessmentandotherdatatoidentifystudentneeds 0.708 2.761 0.097 ** 0.699 2.121 0.145 *Implementeddifferentiatedteachingmethodstobettermeet

theneedsofallstudents -0.414 1.022 0.312

-0.403 0.904 0.342 Providedprofessionallearningforteachersonmeeting

individuallearningneeds 0.551 2.194 0.139 * 0.569 2.222 0.136 *

Providedstudentswithaccesstolearningsupportservices -0.469 1.348 0.246

-1.073 6.343 0.012 ***

ProvidedILPsforstudentsneedingassistance 0.675 2.418 0.120 * 1.633 12.750 0.000 ***

ReformArea5:Strengtheningschoolaccountability

Madeplanningand/orreportingprocesseswithintheschool

morepubliclyavailable -0.006 0.000 0.986

0.691 3.677 0.055 **

Expandedtherangeofschoolactivitiesthatareevaluated -0.050 0.017 0.897

0.432 1.181 0.277ReformArea6:Externalpartnerships

Parentsandcarersgenerallyintheschoolcommunity 0.103 0.062 0.803

-0.372 0.783 0.376

Employers 1.334 6.200 0.013 *** 0.777 2.111 0.146 *

OneormoreTAFEsandtrainingproviders 0.586 1.403 0.236

-0.692 1.812 0.178

Oneormoresecondaryschools -0.583 2.182 0.140 * -0.204 0.256 0.613

Oneormoreprimaryschools 0.678 3.245 0.072 ** -0.118 0.095 0.758 Note***P<.05**p<.10*p<.15


35

ForReading, schools that implemented the following initiativesweremore likely to showlowerimpact:

• Establishedleadershipandstrategicpositions(RF1)• Introducedincreasedflexibilityintimetablingand/orschool’shours(RF2/3)• Providedprofessionaldevelopmentona rangeofbehaviourmanagement theories

andapproaches(RF2/3)• Partnershipswithoneormoresecondaryschools(RF6)

ForNumeracy,schoolsthatimplementedthefollowinginitiativesweremorelikelytoshowlowerimpact:

• Providedmentoringsupporttoteachers(RF1)• Providedattractiveterms/conditionsoutsidestandardentitlements(RF1)• Adoptedteam-basedapproachestoteachingandplanning(RF2/3)• Providedprofessionaldevelopmentona rangeofbehaviourmanagement theories

andapproaches(RF2/3)• Providedstudentswithaccesstolearningsupportservices(RF4)

In lookingat the initiativesnegativelyassociatedwithNP impact, there is forNumeracyasignificantdifferencelinkedtostrategiesattemptingtoattractordevelophighperformingteachers.Possiblyreflectinggreaterstabilityinstaffing,highimpactschoolswerelesslikelyto provide mentoring support or establish attractive terms and conditions to recruitteachers. Higher impact schools did, however, have a significantly higher likelihood ofimplementingstrategiestoprovideopportunitiesforprofessionallearninganddevelopmentofstaff.

While in terms of Reading scores, higher impact schools were more likely to haveimplementedstrategiesaroundpartnershipswithprimaryschools, theywere less likelytoimplement strategies associated with staffing and performancemanagement. Again, thismayreflectthechallengesfacedbylowimpactschoolsaroundstaffingandmanagement.

Thedifferences in focus and emphasis amongNP schoolsmaywell reflect the challengesthat they faceand towhich theyare responding, rather than theeffectsof thestrategiesthemselves.Forexample,schoolsthatshowlowNPimpactonstudentNAPLANresultshaveplacedgreateremphasisonattractinghighperformingteachersandimprovingthestaffingandperformancemanagement roleof theprincipal. Theymayhave issues in staffingandmanagement practices that need (and the schools recognise that need) to be addressed.High impact schools, on the other hand,may havemore stable staffing profiles, and it istheir staffing andmanagement stability thathasenabled these schools to target areasof


36

improvement in teachingpracticeand inaddressingstudentneedsthathasdeliveredrealtractionfromtheNPinitiatives.

Finally, many other contextual factors that are not captured by the data may have aninfluence on the effectiveness of NP implementation in NP schools. For example, thereseems to be difference in specific approaches and operational practices in NPimplementation across NP schools. In the following section we present case studies ofschools that provide further contextual insights and understanding of results from thequantitativeanalyses.


37

6. CasestudyschoolsPreviously, as part of the agreed work program for the evaluation of NP initiatives, anumberofcasestudiesofschoolswereundertakentoexaminemorein-depththeadoptionand operation of low SES reform initiatives. Thesewere presented in a series of reports(CIRES,2012;CIRES,2013;UoC,2012,UoC,2013)focusingontwodifferentNPreformareasrespectively:evaluationofSchoolStaffing,ManagementandAccountabilityandevaluationof school External Partnerships. The case studies provide a snapshot of what specificstrategies/actionshavebeenimplementedinschoolsandhowthesestrategies/actionshaveworked in different school settings. They are useful to re-examine in the context of thisreport because some are high impact schools in terms ofNP effects onNAPLAN studentlearningoutcomes,somearemediumimpact,andotherslowimpact.Subsequentanalyseswill draw further on case studydata to illustrate the complexity of schools’ planning andimplementationconsiderationsinarrivingatstrategiesthatwillbebothlocally-appropriateandeffective. What canwe learn fromthese schools that canhelpourunderstandingofdifferentialeffects?

Inthisreport,wehavelinkedsomeofthecasestudyschoolswiththeirNPeffectdataand,therefore, areable toprovide further contextual insightsonwhat initiativeshaveworkedwelland inwhatcontext theyareeffectiveornoteffective in improvingstudentNAPLANperformance.

In this section, we present four schools as examples of differences in context and effectoutcomes.Theseschoolsdiffer in termsof their school settings, thechallenges they face,andthefocusoftheNPstrategiesandinitiativesthattheyimplemented.

Table6.1providesageneralprofileof the schools.As inearlier reports thenamesof theschools have been changed to remain consistent with the original guarantees ofconfidentialitygiventoschools.

TheprofilesinTable6.1revealthatbothValleyPublicSchoolandSouthCoastHighSchoolshowahigh levelNPeffectonNAPLANoutcomesandahigh levelofeffectiveness in theimpactoftheirNPstrategiesandinitiatives,withonebeingaprimaryschoolandtheotherasecondary school. No schoolwith low impact on both Reading andNumeracy have beenmatchedtothecasestudyschools.

WesternPublicSchoolshowsanaverageresultonNPeffectinbothReadingandNumeracy.TheschoolenteredtheLowSESNationalPartnershipsprogramatapointofinstabilityinitsleadership arrangements. Essentially the Low SESNP funding and processes provided theschoolwithopportunitiesandspacetofocusonestablishing,implementingandcontinuallyreviewingitsimmediateorganisationalchangemanagementmodel.Interviewswithschoolleadershipandteachersrevealedarangeofreformsandinitiativesappliedbytheschoolas


38

partofitsparticipationinNationalPartnerships.Howeversurveyresponsesandotherdatasuggest that these processes and strategies have not been effective and have not beentranslatedintoimprovedNAPLANoutcomes.

Table6-1 Profilesofcasestudyschools

WesternHigh SouthCoast

HighValleyPublic WesternPublic

Schoolprofile

SchoolType Secondary Secondary Primary PrimarySchoolLocation Metropolitan Provincial Provincial MetropolitanSchoolSES(SESEOIIndex) Low(Q4) Low(Q4) Medium(Q3) Medium(Q2)

Fundingprograms

NPparticipation 2010 2010 2010 2010PrioritySchoolProgram2010 Yes No Yes Yes

Staffprofile(2010)

Malestaff(proportion) Medium(35%) High(65%) Medium(16%) Medium(7%)Ageindex Medium(Q3) High(Q4) High(Q4) Medium(Q2)

<30 13% 2% 4% 21%30-49 57% 23% 60% 57%50+ 30% 75% 36% 21%

StaffMobility High(32%) Medium(8%) Medium(10%) Low(0%)

Studentprofile

AboriginalConcentration High(13%) High(10%) High(25%) Medium(4%)LBOTEConcentration High(44%) Medium(3%) Low(1%) High(40%)Attendancerate2011 Low(82%) Low(89%) Low(90%) Medium(94%)Attendance2013 Low(83%) Medium(89%) Low(92%) Medium(94%)

NPEffectonNAPLAN

Numeracy Medium High High Medium

Reading Low High High Medium

WesternHighSchoolshowsalowlevelofNPeffectinReadingandanaverageNPeffectinNumeracy. Western High School provides a good example of the complexity of thechallenges confrontingmany schools in planning and acting on school improvement. TheSchool’splannedresponsedevelopedundertheNPinitiativefocusedonliftingtheschool’sperformance.Asuiteofstrategiesandinitiativeswereimplementedcoveringabroadrangeof NP reform areas, including building leadership and staff capacity, utilising quality datamanagement systems and learning methods for students and building community andparentpartnerships.However,evidencesuggeststhattheplannedstrategiesandactivitiesfailed to translate through to substantial improvements on student NAPLAN outcomeswithin the timeframe of the evaluation. As discussed below the case demonstrates thechallenges and the time required to shift school outcomes in a very challengingenvironment.


39

6.1ValleyPublicSchool

NPeffectonNAPLANachievement:HighonbothReadingandNumeracy

SchoolProfileValley Public School is a provincial Government public school that caters for over 400kindergarten to Year 6 students. Located in the Hunter Central Coast, the school has ateachingteamofover20staffandboastsasmallbutactiveParentandCitizenAssociation(P&C)andactiveinterestfromlocalEldersandtheAboriginalEducationConsultativeGroup(AECG).In2013theschool’sICSEAvaluewas855,whichismarkedlybelowthestatemeanand similar to the school’s ICSEA value in 2010. In 2013 the school was made up of 26percent indigenous students, and four percent of students were from a languagebackgroundotherthanEnglish(LBOTE).ValleyPublicSchoolhadanattendancerateof92percentandtheschooloffersanumberofprogramsforgiftedstudentsaswellasstudentswithlearningdisabilities.

MainChallengesandareasforimprovementValleyPublicSchoolfacedanumberofchallengesduetoalevelofparentaldissatisfactioninhomeandschoolcommunication;accesstoteachers;inputintodecisionmaking;aswellasthetransitionprogramtosecondaryschool.Theschoolhadbecomeawareofanumberofconcerns raised by the Parents and Citizens Committee (P&C), including perceivedinconsistency in the provision and approach to homework, poor attitudes to teacherstowardsparents,andinadequatecommunicationaroundwhattheschooloffersandhowitmakesdecisions.

EffectiveActions/InitiativesValleyPublicSchoolpursuedanumberofstrategiestoaddressthechallengeshighlightedbyparentsandthecommunity.AspartoftheNPreformeffort,theschool:

1. Establisheda strategic focusonparentpartnerships through theemploymentof aCommunityEducationOfficer(CEO)

ThispositionwasfundedbytheLowSESNPProgram,andsupportedteachers,parentsandstudentstoestablishrelationshipswithintheschoolandcommunity.TheCEOhelpedcreateaculturalchangewithintheschoolbymediatingandaddressingmattersbetweenparentsand teachers. This was accomplished by the CEO’s implementation of regular parentsurveys, parent focus group meetings, attendance to P&C meetings and participation inactionresearchprojects.Staffreportedthatthesepositivelinkswithparentsbenefitedtheteacher-parent relationship,which led tobettercollaborationonstudent learning.Due tothesuccessoftheCEO,theP&Chasofferedtoraisefundstomaintainthepositionbeyonditscurrentfunding.

2. Tried to establish a better process of communicating with parents through activeengagementindialoguewithparents


40

Anumberofinitiativessuccessfullyimprovedthedialoguebetweentheschoolandparents.ThepresenceofthePrincipalintheplaygroundduringmorningsandafterschoolcreatedaninformal space forparents to interactwith thePrincipaland school staff.Additionally theschool’s newsletterwasused as a communication tool topromotepositive conversationswith a new format that included student achievements, learning journeys and individualstories.Overall,parent’sappraisaloftheschoolshiftedtoapositiveone,evidentinsurveyresults.

3. IncreasedparentparticipationthroughmeasuresthatimprovedtheeffectivenessoftheP&Candencouragedparentstoprovidefeedbackandtomakesuggestions

TheschoolreportedthattheP&Coperatedmoreeffectivelyandnewpeoplejoinedastheculturalchangeencouragedmoreparentparticipation.AroundeightparentsattendedeachPandCmeetingin2012comparedwithsubstantiallymorescantorunreliableattendancesin earlier years. Additionally, a number of parents volunteered in the school canteen.Furthermore,promptedbytheschool’seffortstocreateabetterdialogue,parentsprovidedfeedbackandmadesuggestionsvia informalcontactwithstaff,attendancetofocusgroupmeetingsandteacherforumsorparticipationinschoolsurveys.Parentsurveysundertakenat the end of 2012, indicated that 67 per cent of parents reported that the provision ofregularcommunicationisadequatecomparedto50percentin2011.

4. Developedparentknowledgeaboutstudentlearningthroughtheprovisionofparentinformationsessionsandworkshops

Theschoolimplementedparentinformationsessionsandworkshops.Theseweredesignedtofacilitateparent’sknowledgeabouttheirchild’slearning.Fromtheseexperiencesparentshighlightedthedesire foradditionalopportunities toobserveteachingand learning intheclassroom.

5. Facilitated home-school collaboration through the improvement of Three-WayConferences and engagement of parents in student discipline, as well as theirtransitiontohighschool

The implementationof theThree-WayConferencebetween teacher, parents and studentwas successful. After altering it so that parents and teacher meet prior to meeting thestudent, the Three-Way Conference became a tool to create constructive dialogue,acknowledge achievements, find solutions, make commitments and plans for the future.Additionally,theschoolranmeetingsforparentstodiscussstrategiesforimprovingstudentbehaviours.Alongwith increasedparent attendance to assemblies that celebrate studentachievementsandawards,theschoolnotedadropinthesuspensionrate.Furthermore,theschoolranmeetingsforparentswithchildrenwhoaretransitioningtoYear7inanefforttodiscusstransitionstrategiesandhowboththeschoolandparentscouldbettersupporttheirchildren.


41

ReflectiononresultsTheValleyPublicSchool’sefforttocreateasustainable,opendialoguewithparentscreateda positive cultural change that benefitted the students, staff and community at large.Parents respondedwell to the school’s incorporationof parentpartnerships to its overalloperation,evidentinthepositivefeedbackfromparentsurveys, increasedcommunicationwithstaffandattendancetoforum,meetingsandworkshops.Additionally,theworkoftheCEOpavedthewayforfurtherdevelopmentofrelationshipsbetweenparents,theP&C,thePrincipal,schoolleadersandteachers.TheschoolandP&CacknowledgedtheimportanceoftheCEO’sworkandexploredstrategiestocontinuetheworkaftertheLowSESNPProgram.The CEO has been the cornerstone to the positive cultural changes that have led toimprovedparentpartnershipswithinValleyPublicSchool.

6.2SouthCoastHighSchoolNPeffectonNAPLANachievement:HighonbothReadingandNumeracy

SchoolProfileSouthCoastHighSchool isaco-educationalgovernmentsecondaryschool inthe IllawarraandSouthEastRegionwith justunder690enrolments in2013(thekeyyearforanalysis).Studentnumbershaveremainedfairlystablesincetheearly2000s.

ThreedistincttypesorgroupsoflargelyEnglish-speakingfamilieswereidentifiedbyschoolleadersasmakinguptheschool.While themajorityof familiesare low-incomeandoftensingleparenthouseholds,thereisarelativelysmallgroupoffamiliesinwhichbothparentsare employed. In addition a significant Aboriginal community within the town and itssurroundingsmeansthattheproportionofAboriginalstudentswithintheschoolhassatataround12percentformanyyears.

KeyChallenges

Improved parental engagement was viewed as a key area of need, particularly for theindigenous population using the school. Critical to this was working out how to betteraccommodatetheneedsofAboriginalfamilies.Therewerespecificchallengesindoingthis,challenges associated with better communicating with these families, in particular withregard to the role of the home in students’ education. Linked to this, the school faced anumberofchallengesinincreasingthenumberofparentsinvolvedintheschoolgenerally.Initialstrategiesexplicitlyaddressedoutreachapproachesto“hardtoreach”parents.

Effectiveactions/initiatives

South Coast High School’s Low SES NP plan aimed to lift the attendance, retention andperformance of students through a whole-of-school and community engagement reform


42

approach. The approach reflected the priorities of a regional comprehensive secondaryschoolcateringtotheneedsofadiverseschoolcommunity.Partnershipswithfamilieswerefundamental to its strategic focusandaparentengagementstrategywasconsideredasaleadingdrivertoimprovementsinschool-homerelationships.Thestrategyincluded:

• Enhanced communicationwithparents including regularnewsletters, regular emailcontacts with parents, changed use of school website (updated regularly andincreasinglytheinformationconduitforteachers,parents,studentsandcommunity).As a result of the extensive promotion of the parent and teacher meetings innewslettersandSMSmessaging,theschoolreportedthatin2013therewasamorethan 30 per cent increase in the number of parents attending parent and teachermeetingscomparedtopreviousyears.

• Toassistparents totakeonastrongerrole in theirchildren’s learning, informationsessions were provided to offer parents opportunities to learn about the schoolsystemandtheschool’sownstrategiesaimedatsupportingchildrentoimprovetheirperformance. Regular workshop sessions were scheduled for this purpose andactivelypromoted.Parentswereexplicitlyencouragedtovolunteerattheschoolandinvitedinfollowupcontactstoidentifyareaswheretheycouldbebestcalledontoassist–suchas inReadingsupport, languageteachingsupportandassistancewithexcursions.Thesesessions,designedtoassistparentstoengagemorefullywiththeirchildren’slearningatschoolandathomewereincreasinglywell-attended.Sessionsand workshops covered themes such as: family Reading; writing and Numeracy;positiveparenting;understanding reportsandstudentdata; studentpathwaysandcareeroptions;andstudentwellbeingandmentalhealth.

• EffectivelyengagingaprioritygroupbyworkingdirectlywithAboriginal families inrevitalisation of the Personalised Learning Plan (PLP) processes. The school’s newapproachtoPLPs,whichaimedtoincreasethelevelofstudentengagement,enlisteddirectsupportfromparentsandfamilies.AlocalcoachwasemployedtoredevelopthePLPstobuildtheirutilityasaplanforfamiliesandstudentsthat“enabledthemtolearn”.Thenewapproachshiftedemphasisfromateacher-directedapproachtoone that highlighted student and parent ownership of the process. Aboriginalparents were encouraged to engage with the school in a more positive way.Accordingtofeedbackformtheschool,formanyparentsitmarkedadeparturefromcoming into the school only as a result of misbehaviours or expulsions. The PLPmeetingwithparentswasdesignedtogiveparentsa“positivereasontocometotheschool”,andenabledthemtoprovideinputintotheirchild’slearningplans.

Reflectiononresults

An important early outcome from improved and targeted communication between theschoolandthehome,accordingtotheschool,wasasignificantimprovementinthestudent


43

attendance rate. A head teacher (welfare) was employed to coordinate a whole- of-communitywelfare support service, andNP fundingalso supported theemploymentof aspecialist school administrative support (SAS) staff member to provide administrativeassistanceforNPinitiativesassociatedwithstudentengagementandattendance.

Withintwoyearsofinitialimplementationtheschoolwasabletoidentifysomeindicationsthat the parent engagement strategy had led to changes in parent expectations of theschoolandintheirconnectionwiththeirchildren’slearning.

School personnel reported that parent engagement had shifted from a limited group ofparentsofhighperformingstudentstoamorerepresentativebodywithamore informedandfocussedinterestonhowtheschoolviewedstudent learningandachievement:“theirinterest was evident through their questions about student learning and schoolexpectations”(principal).

The school reported that strengthened parent and community engagement enabled theschool to take amore rigorous approach toworkingwith families in an effort to supportproductivestudentlearningandextendedopportunitiesavailabletostudents.Forexample,engagementofparents intheSchoolBasedApprenticeshipandTraineeshipsprogramwascitedasanexampleofsomethingthathadincreasedthesuccessoftheprogram.Parentssupportedtheestablishment, implementationandreviewofprogram. Inparticular,manyparents were instrumental in assisting students to locate suitable employers andworkplaces.

6.3WesternPublicSchoolNPeffectonNAPLANachievement:MediuminbothReadingandNumeracy

SchoolProfile

Western Public School is a preschool to Year 6 school with around 210 students in theprimary school and 40 pre-school students. The school is located in theWestern SydneyRegion. Its 2013 enrolment of 223 and attendance rate of 94 per cent are both slightlyhigherthanthoseofearlieryears.Theschool’sAboriginalpopulationisgrowing(8percentin 2013) and 53 per cent of students are reported to be from families of a languagebackgroundotherthanEnglish.

KeyChallenges

WesternPublic entered the LowSESNPprogramat apointof instability in its leadershiparrangements.Between2007and2012theschoolhadseenfourprincipals;atthetimeofinitialinterviewin2012thethenprincipalhadbeenintheschoollessthanaterm.


44

KeyStrategiesandinitiatives

Partly as a result of this extendedperiod of instability, and in linewith the school’s ownassessmentofimmediateneeds,itsNPplanandstrategiesfocussedonimprovementofthequality of its teaching, school planning and accountability processes. In consequence, theschoolestablishedanewexecutiveleadershipmodeltoimproveleadershipeffectivenessinthe school. The executive team was expanded to include stage and learning areacoordinators.Members of thenewexecutive team, including theprincipal, had access tomentors and also had opportunities to participate in team based leadership trainingprograms.

Afurtherkeygoaloftheschoolwastobuildteachercapacitybyimplementingawholeofschool approach to professional learning. Data associated with student achievementoutcomeswereusedtoinformthedesignofstaffprofessionallearningactivities.Itlinkeditsstaffprofessionallearningwithschoolplandevelopment,withafocusontheimprovementofstudentprograms.Forexample,teacherswererequiredtoensurethattheyusereflectivejournals or learning logs which incorporate relevant Low SES SSNP professional learningactivities.

Reflectionontheresults

Essentially, theLowSESNP fundingandprocessesprovidedtheschoolwithopportunitiesandspaceto focusonestablishing, implementingandcontinually reviewing its immediateorganisational changemanagementmodel. Thesewere subject to changeover the life oftheprogram.Forexample,teachersandleadersregardedtheirreviewprocessesasinitiallynot usefully linked to the broader context of the school, restricting the capacity of theprocess to contribute strategically to achieving school goals. A more explicit link to theschoolplanasrequiredwassubsequentlyimplemented.

These elements – consolidating effective leadership and implementation ofwhole schoolchangeinteachingpractice—servedasimmediateandpressingprioritiesforthisschoolinitsengagementwiththeLowSESNP.

At thepointof interview in2012therewasaperceptionfromschool leadershipandstaffthattheprofessionaldialogueintheschoolhadimproved.Staffmembersweredescribedasmorewillingtodiscussanddebateandinsodoingabletobetteraddressissues.

WesternPublicSchool,whereplannedactivityfailedtotranslatethroughtotheobjectiveofsubstantialimprovementsonstudenttestscores,demonstratestheneedforunderstandingofthetimerequiredtomaketheculturalshiftsthatunderpinenhancedschooloutcomes.


45

6.4WesternHighSchoolNPeffectonNAPLANachievements:LowinReadingandMediuminNumeracy

SchoolProfile

WesternHighSchool isacoeducationalgovernmentsecondaryschool(Years7-10)withanenrolmentof575in2013,downfrom630in2012.ItservesthecurricularandsocialneedsofaverydisadvantagedandhighneedspopulationintheWesternRegionofSydney.Nearlyhalf its student population report language backgrounds other than English andmany ofthese are from Pacific Islander families. A further 15 per cent of the student populationreportAboriginalorTorresStrait Islanderbackgrounds.Parentemployment isprimarily inunskilledorlowskilledwork;unemploymentisalsohighandsignificantnumbersofparents,including many single parents, are not in the labour force. In the first year of NAPLANtesting, the school reported 94 per cent of its student population to be drawn from thelowestSESquartile.

AttendanceininitialyearsoftheLowSESNPwasreportedat81percentandtheschool’sinitiatives includedasignificantdirectionofNP funding toward improving theserates.MySchoolreportsreflectrisingratesofattendance,to85percentin2014:

Year 2010 2011 2012 2013 2014

StudentAttendance Rate(%)

81 83 82 83 85

Source:MySchool

Challenges

Theschool’sLowSESNPProgramrespondedtoaninitialsituationalanalysisundertakenin2009thatidentifiedanumberofdistinctconcernstobeaddressedthroughNPinitiatives.

• First,students’learningandengagementoutcomeswerelowerthanstateaveragesona rangeof keymeasures suchasNAPLAN scores, attendancedata and studentretention.

• Second,staffing issuespresentedfurtherconcerns—attritionamongteacherswashighwithacomparativelyhighturnoverofstaffandtheproportionofcomparativelyinexperiencedstaffwasalsoaconcern.Theschoolwantedtoholdenergeticyoungstaff and better support all staff in working more effectively with their peers;sustainedprofessionalsupportanddevelopmentwasneededtohelpteachersbettermeetstudents’complexneeds.


46

• Third, the school identified community perceptions of the school as problematic;parents’engagementwith theschoolwasvery lowandparticularly lowforcertaingroups(suchasAboriginalstudents)withintheschoolcommunity.

KeyStrategies/initiatives

The planned response developed under the initiative therefore focused on lifting theschool’sperformanceonseveral linkedbutdistinctfronts.Nothingshortofaculturalshiftwithin theschool– inareasofbuilding leadershipandstaffcapacity,utilisingqualitydatamanagement systems and learning methods for students and building community andparentpartnerships–wasregardedasrequiredtoeffect thedesiredchanges instudents’literacyandnumeracyachievementlevels.

ActivitiesassociatedwiththeNPwereaccordinglyspreadacrossseveralkeyareasofactivitywithastrongfocusonleadershipandstaffimprovement.AnAssistantPrincipalpositionwascreated to oversee the complex program associated with the NP strategies. At theleadershipandstafflevel,forexample,severalkeyinitiativeswerereportedtobeintegraltotheschool’splansforimprovedoutcomes.In2012theseinitiativesweredescribedas:

• Focusonperformancemanagementandstaffmentoring,whichhadalready led to“positive change” in the school’s staffing profile. Long-time staff who had beenresistanttoreformshadlefttheschoolwhileothers,energisedbythenewactivities,notonlywantedtoremainbuthadtakenoneducationalleadershipchallengesandwere actively encouraging like-minded colleagues to join the school community.Youngstaffinparticularhadservedas“attractors”toothersimilarly-motivatedstaffwithin theirnetworks. Being regardedasan“employerof choice”wasanewandwelcomeexperienceforthisschoolandagreatmoralebooster.

• Sharingofpractice:thebuildingoflearningteamsandprofessionalrelationshipswasakeyfocusoftheNPinitiativesandfundamentaltoreformswithintheschoolandresourcesoftimeandprogramfundingwereallocatedtosupportingthisinitiative.

• Strategic use of technology: a classroom-based digital recording system allowedteacherstoshareandcommentontheirteachingpractice,buildingamorecollegialand team based approach to practice. The system was described as “constantly”utilisedandverypopularwithstaff.Otherschoolsdrewonthisschool’sexampleinsettingup similar systems.A factor in itsacceptancehadbeen involvementof thewhole staff in development of professional protocols and safeguards surroundingthe program – leadership emphasised from the outset that the purpose of theprogram was not oversight and control but the sharing and encouragement ofinnovativepractice.

• Teachersasresearchersundertookprojectsonresearchtopicsassociatedwiththeirpractice –these projects were regularly reviewed by peers and awards given for


47

strong practice. The process played an important role in changing staff culturethroughopeningupclassrooms,collaboratingonactivitiesandsharinginformation.

• Ateacher-basedreviewcommitteewasestablishedtooverseenewstaffingappraisalarrangements and students and parents had regular opportunities to providefeedback on teachers (three times a year). This information and associatedreflections played a role in Teacher Annual Assessment reviews. These newprocesses were described as “evaluative and reflective processes” of greatimportanceincontributingtochangesinteachers’professionalapproaches.

As noted a significant focus ofWHS’ initiative also involved improvement of attendancerates through amix of engagement strategies and substantial teacher allocations for themonitoringandfollowingupofstudentabsences.Workstudiesclasses,forexample,provedsuccessful in turning around attendance and completion rates for at-risk Year 9 and 10students.Whileattendanceratesremainedlowerthanstateaveragesimprovementswereseenannuallysincetheintroductionofthesemeasures:

Year 2012 2013 2014

Student AttendanceRate

82 83 85

Source:MySchool

AnabidingconcernforWHSwasthatparentengagement intheschoolwasvery low.Theconcerntoenhanceparentandcommunitypartnershipsconstitutedafinalstrategicfocusfortheschool’sLowSESNPplan:Iftherewasonethingwecouldfixthatwouldbeit,theparentengagement(Communityengagementcoordinator).

• The school allocated Low SES NP funds to develop and implement a parent andcommunityengagement strategy. Twopart-timepositionswerecreated in2009:aParent Education coordinator; and a Community Education Officer. Parents wereencouraged to channel their requests for assistance on specific concerns orquestions to these staff in the first instance to promote a “triaging” that wouldensuretheirissueswereaddressedinwaysthatweretimelyandeffective.Anotherteacher was allocated a fractional load to improve content and delivery of thenewsletter as a means of communicating more effectively with the schoolcommunity.

• Theschoolexperimentedwith itsorganisationofparentand teachermeetingsandforums,always verypoorlyattended in thepast. Itwas successful in staging somecommunity evenings attended by many parents and community members andsoughttobuildonthesuccessofsuchpositiveevents.Moretargetedactivitiesforspecific community groups were also organised, such as administration of surveyinterviewsandfocusgroupmeetingsforAboriginalfamilies.Inconsequencearange


48

ofopportunitieswereidentifiedforteacherstofacilitatehome-schoolpartnerships,especially inmaking direct contactwith parents to encourage their attendance atassemblies, award ceremonies, other celebratory community events, and studentextra-curricularactivities.Teachersreceivedprofessionaldevelopmenttobuildtheirconfidenceandskillsinworkingwithallparents,inparticularwithregardtostudentattendance.

Reflectiononresults

Western High School provides a good example of the complexity of the challengesconfrontingmany schools in planning and acting on a school improvement plan. In thiscase,theplannedandimplementedactivitiesandinterventionsfailedtotranslatethroughtotheschool’sobjectiveofsubstantialimprovementonstudenttestscores.Theexperienceof WHS highlights a point made by a number of research studies on improvementinterventions in school settings (Lamb and Rice 2009, CDRP 2015): that the interactionsbetween school culture (characterised in terms of staff and student stability, orderlyenvironmentsandconsistencyof leadership)andthequalityandfocusof interventionwillimpact on effect. The case demonstrates the need for understanding of the time or pre-conditionsrequiredtomakeimprovementinschooloutcomes.

Leaders in this school made reference to “lining up the ducks” for improvement andbelieved that their NP approaches –which included targeting 1. student attendance andengagement, 2. teacher and leadership capacity and staff renewal and 3. School-parentrelations–weresettinguptheconditionsforimprovementstobemade.

By 2012, teachers and parentswere able tomake some cautious assessments about thesuccess of this strategy: “there is an increase because itwas just so low before. Studentattendancehasdefinitelyincreasedalbeitremaininglowerthanstateaverages.Therehavebeenincreasesinawardshandedoutforacademics,sporting,citizenship,andattendance”.Therewasasense,too,thatstaffdevelopmentstrategieshadbroughtanewenergytotheschoolandsupportednewandmoreproductivewaysofworking.

ForWHS,however,thesuiteofstrategiessopainstakinglydevelopedandimplementedaspartoftheschool’sNPplanshowedverylittleoutcomeinstudentperformanceinNAPLAN.The intractability of poor test outcomes, especially in NAPLAN results, was a source offrustration,disappointmentanddisbeliefamongschoolstaff,giventhatschoolplansandarange of pedagogical and staff development initiatives had been introduced with theexpresspurposeofaddressingandturningaroundNAPLANperformances.

Essentially the school’s planning and activity, especially in its targeting of studentengagementandstaffdevelopment,hadcentredonculturalchange.Thestrategiesfocusedonthetransformationofstaffasprovidingtheclearestandmostdirectwaytosecurethe“runson theboard” required fora stronger,more stable school communitywithpositive


49

community profiles and a strong interfacewithparent communities. Itwas felt that thisculturalshiftwouldinturnwouldliftperformanceoutcomesinlongrun.

The challenges faced by this school weremany. Interim indicators thatmay have betterreflectedincrementalgainsmadeovertheperiodofthestudycouldhavebeenlinkedmoredirectly to the factors associatedwith the preconditions for success rather thanwith thestudent outcomes themselves. These could have linked for example to attendance, staffdevelopment or parental engagement outcomes, all factors associatedwith a productiveanddynamic school culture. Indicators of change, especially in the context of communityexperiences and perceptions, could be the quality and incidence of contactsmadewithparents outside the contexts of conventional parent involvement – such as the parent-teacher evenings, school forums or other contexts that tend to define parental roles inschool. Strategies that enlist parent support in building attendance rates or developingindividuallearningandpathwayplansforyoungpeopleatriskinvolveparentsinwaysthatconnect closelywith students’ experiencesof school; the impactofprograms suchas theschool’sYear10workstudiesprograminengagingparentsinstudents’programsandthusimprovingattendanceandretentionratesisimportant.Similarly,theinitiativesofyearlevelcasemanagers incontacting familiesasamatterofcoursehasbuiltconnectionsbetweenschool and the home. Steps such as these provide a stronger platform for furtherengagement and their role in shifting parent-teacher relationships to stronger andmorepositive foundations may be found in parent and teacher perceptions of incrementalchange.


50

7. Conclusion

ThisstudywascommissionedbytheCentreforEducationStatisticsandEvaluation(CESE)toexamine student performance in low-SES schools which participated in the NationalPartnership Agreements, and the extent to which changes in student performance arerelatedtoNPinitiativesorotherfactors. Ofparticular interestwasidentifyingwhetherornot schoolexternalpartnershipsandassociatedNP initiatives lead to improvedoutcomesfor schools and students. Toexamine thisquestion, a rangeof analyseswereundertakenusing available data to evaluate the effectiveness of the Low SES NP in improving NSWstudentoutcomesacrossarangeofmeasuresincludingNAPLANresults.

Thereportexplorestheissueofgoodpracticeusingdataderivedfromavarietyofsourcesincludingfromadministrativedata,NAPLANachievementdata,surveyresultsfromteacherandprincipalsurveys,andcasestudymaterialscollectedinearlieryearsfromsitevisitstoselected schools. Drawing on information from a variety of data sources helps to gain abetterunderstandingoftheeffectoftheinitiativesimplementedbyschools.Importantinthis are several earlier reports,many of which provided highly detailed accounts of howspecificschoolshaveworkedwiththeNPorhowschoolsandschoolpersonnelacrossNSWhaveexperiencedthe initiativesovertime.Thesearevaluablepiecesofwork intheirownright and their findings inform this report. But themain focusof this report lieswith thebroadquestionofoutcomeandimpact.

Previous work, documented in the low SES NP staffing, management and accountabilityevaluation report, has identified positive effects of the NP program at an aggregate orsystemlevel. Inparticular,theanalysisusingNAPLANmatchedcohortdatasuggestedthatlowSESNPschoolsonaveragehaveasignificantimpactonstudentNAPLANresultsover3yearsbetween2011and2013.Whatthecurrentreportaimedtodowastounpackfurtherthe results of the earlier work to identify variations in impact across schools in order tobetterunderstandwhereandinwhichcontextsparticularstrategieshaveworkedwell,andwheretheyseemtohavehadlittleeffect.

Among NP schools there is a positive relationship between the level of their NP effectoutcomeandtheoveralllevelofeffectiveimplementationintwoNPreformareas:

• ReformArea4—Provideinnovativeandtailoredlearningopportunities,and

• Reform Area 6 — Promote external partnerships with parents, other schools,businessesandcommunities.

Themoreinitiativesrelatedtothesetwoareasimplementedinaschool,thehigherimpactof NP in the school. The higher level of effectiveness as perceived by the principal and


51

teachers,themorelikelytheschoolshowsahigherNPeffect.Itsuggeststhatschoolsthatplacedmoreeffort inthesetworeformareasweremorelikelytoshowahighereffectonstudentlearningoutcomes.

In looking at what separated high impact schools from low impact schools, the analysisrevealed that schools with the following characteristics weremore likely to show a highimpactontheirstudentNAPLANimprovement:

• They had placed greater emphasis than other schools on innovations in teachingpracticeandaddressingstudentneeds

• Theyhadprovidedstrongerwelfareandlearningsupportfordisadvantagedstudentsthrough providing teachers with training around individual learning needs, andstrengtheningtheuseofILPsforstudents

• Strongeruseofevidencetoidentifystudentlearningneeds• Stronger emphasis on developing staff through providing opportunities for

professionallearninganddevelopment

ForschoolswheretheNAPLANachievementeffecthadbeenlowornegligible,theanalysisrevealedthatschoolstendedtobecharacterisedbythefollowing:

• Some had a stronger emphasis on strategies to attract and retain teachers,particularlythroughleadershippositions,whichmayitselfreflectgreaterneedinthisareaandschoolsstrugglingtorecruitandretainqualityteachersandleaders

• Lessfocusonuseofevidenceforstrategicplanninganddecision-making• Less focuson innovation in teaching, addressing studentneeds, and strengthening

schoolaccountabilityandmoreonstaffingandperformancemanagement• Pursued strategies around teacher and leadership recruitment, retention and

developmentsuggestingthatstaffstabilitywasamajorissueforthem.

Inmeasuring variations inNAPLAN impact across schools, this report has shown howsomeschoolshaveimprovedstudentoutcomesasaresultoftheNP,whileothershavebeen less effective in doing so. The next phase ofwork for this projectwill include acloserstudyofanumberofcasestudyschoolsthathavedisplayedhighNPimpact,andanumberoflowimpactschoolstotrytoteaseoutwhathascontributedtodifferentialimpact. Thiswill be complemented through amore extensive analysis based on newandmorecomprehensivedataoftheinitiativesthathavebeenemployedinschoolsthathaveaddedsignificantlymorevaluetostudentachievementrelativetootherschools,andthe initiatives of schools that have held ground but not necessarily improved relative tootherschools,andtheinitiativesofschoolswithdecliningornegligibleperformance.


52

References

AustralianCouncilforEducationalResearch(ACER)(2013).PISA2012:HowAustraliaMeasuresUp

AustralianCouncilforEducationalResearch(ACER)(2012).HighlightsfromTIMSS&PIRLS2011

fromAustralia’sperspective.

Centre for Research on Education Systems (CRES) (2012). Evaluation of the School Staffing,Management and Accountability Initiatives. Progress Report No. 1. Prepared for: the NewSouthWalesNationalPartnershipsEvaluationCommittee.

COAG Reform Council (2010). Education 2010: Comparing performance across Australia. Sydney:COAGReformCouncil.Retrievedfrom:

www.coagreformcouncil.gov.au/reports/docs/education/2010compare/education_2010_report.pdf

CouncilofAustralianGovernments(2008).NationalPartnershipAgreementonLowSocio-EconomicStatusSchoolCommunities.Retrievedfrom:

www.smarterschools.gov.au/nationalpartnerships/Documents/SSNatPartnerAgreem.pdf

CouncilofAustralianGovernments (2009). "NationalEducationAgreement: Fact Sheet."Retrievedfrom:

www.coag.gov.au/coag_meeting_outcomes/2008-11-29/docs/20081129_national_education_agreement_factsheet.pdf

Darling-Hammond, L. (2000). "TeacherQualityandStudentAchievement:AReviewofStatePolicyEvidence."EducationPolicyAnalysisArchives8(1).

Department of Education and Training (2011). Low Socio-economic Status School CommunitiesNationalPartnership.InformationPackageforSchools.Retrievedfrom:

www.lowsesschools.nsw.edu.au/wcb-content/uploads/psp/file/2_percent20LOW_SES_InfoPackage.pdf

Elmore,R.(2000).BuildingaNewStructureforSchoolLeadership.Washington:TheAlbertShankerInstitute.

Elmore,R.(2007)."EducationalImprovementinVictoria."Retrievedfrom: www.eduweb.vic.gov.au/edulibrary/public/staffdev/schlead/Richard_Elmore-wps-v1-

20070817.pdf

Field, S.,M. Kuczera& B. Pont (2007).NoMore Failures: Ten Steps to Equity in Education. Paris:OECD.Retrievedfrom:

www.education.vic.gov.au/edulibrary/public/publ/research/publ/policy_and_provision-research_rpt.pdf

Friedlaender,D.&L.Darling-Hammond(2007).HighSchoolsforEquity:PolicySupportsforStudentLearning inCommunitiesofColour.SchoolRedesignNetwork,StanfordUniversity.Retrievedfrom:www.srnleads.org/press/pdfs/hsfe_report.pdf

Fullan,M.(2001).LeadinginaCultureofChange.SanFrancisco:Jossey-Bass.

Gustafsson, J. (2003). “What do we know about the effects of school resources on educationalresults?”SwedishEconomicPolicyReview10.

Harris,A.,T.Allen,&J.Goodall(2008).“CapturingTransformation:Howschoolssecureandsustainimprovement”.Retrievedfrom:

www.almaharris.co.uk/files/capturing_transformation.pdf


53

Helal, M. (2014). Evaluating the Impact of the Smarter Schools National Partnerships on StudentAchievement,MelbourneInstituteWorkingPaperSeries:2014.

Kellock,P.,G.Burke,etal. (2007).TheUseofEquityFundingto ImproveOutcomes,CentrefortheEconomicsof EducationandTraining,MonashUniversity.Reportprepared for theVictorianDepartmentofEducationandEarlyChildhoodDevelopment.

Lamb,S.(2007).“SchoolReformandInequalityinUrbanAustralia:ACaseofResidualizingthePoor”.SchoolReform&Inequality.R.TeeseandS.Lamb,Dordrecht:Springer.

Lamb, S.&B.McGaw (2007). TheperformanceofAustralian schools, internationalbenchmarking.Federalist Paper 2: The Future of Schooling in Australia. Retrieved from:www.caf.gov.au/Documents/TheFutureofSchoolinginAustralia.pdf

Lamb,S.&R.Teese(2005).EquityprogramsforgovernmentschoolsinNewSouthWales:aReview,CentreforPost-compulsoryEducationandLifelongLearning.

Loveless,T. (2013)The2013BrownCenterReportonAmericanEducation:HowWellAreAmericanStudentsLearning,Volume3,Number3.

McKinsey & Company (2007).How the world's best-performing school systems come out on top.Retrievedfrom:

mckinseyonsociety.com/downloads/reports/Education/Worlds_School_Systems_Final.pdf

Ministerial Council on Education Employment Training and Youth Affairs (2008). MelbourneDeclaration on Educational Goals for Young Australians. Ministerial Council on EducationEmploymentTrainingandYouthAffairs.

National Smarter Schools Partnerships (2011).New SouthWales Progress Report 2011. Retrievedfrom:www.nationalpartnerships.nsw.edu.au/resources/documents/2011-ProgressReport.pdf

NewSouthWalesDepartmentofEducationandTraining.IncentivesMenu.SmarterSchoolsNationalPartnerships on Improving Teacher Quality and Low Socioeconomic Status SchoolCommunities.Retrievedfrom:

www.nationalpartnerships.nsw.edu.au/resources/documents/ITQ-LSES-DETIncentivesMenu.pdf

NSW Department of Education & Training.National Partnership of Low SES School Communities:ResearchUnderpinningtheReforms.Retrievedfrom:

www.lowsesschools.nsw.edu.au/wcb-content/uploads/psp/file/Resources/Reforms_paper.pdf

NSWNationalPartnershipsEvaluationCommittee(2011).ProjectBrief:EvaluationofSchoolStaffing,ManagementandAccountabilityInitiatives,AttachmentA:AnOverviewofInitiatives.TenderdocumentprovidedtoCRES.

NewSouthWalesDepartmentofEducationandCommunities(NSWDEC)2011),NewSouth

WalesDepartmentofEducationandCommunitiesdiscussionpaper:Australianschoolfunding

arrangements,papersubmittedtotheReviewofFundingforSchooling.

OECD. (2005). Teachers Matter. Attracting, Developing and Retaining Effective Teachers.OrganisationforEconomicCo-OperationandDevelopment:Paris.

ProductivityCommission.(2012).SchoolsWorkforce.ResearchReport,Canberra.

Robinson,V.(2007).“Theimpactofleadershiponstudentoutcomes:Makingsenseoftheevidence”.Theimpactofleadershiponstudentoutcomes:MakingsenseoftheEvidence.

Smarter Schools National Partnership (2011). New South Wales Smarter Schools NationalPartnershipsImplementationPlan.PreparedbytheAustralianGovernment,NewSouthWales


54

Government, Association of Independent Schools NSW and NSW Catholic EducationCommission.Retrievedfrom:

www.nationalpartnerships.nsw.edu.au/resources/documents/2011-NSW-Implementation-Plan.pdf

Teese, R. and S. Lamb (2009). Low achievement and social background: patterns processes andinterventions; a Discussion paper. Centre for Post-Compulsory Education and LifelongLearning,TheUniversityofMelbourne.

Welch,A.,S.Helme,etal.(2007).“RuralityandInequalityinEducationtheAustralianExperience”.InR.Teese,S.LambandM.Duru-Bellat, InternationalStudies inEducational Inequality,TheoryandPolicy,Springer:Netherlands:271-293.

Zbar,V.,R.Kimber,etal.(2008).Howourbestperformingschoolscomeoutontop:anexaminationofeighthighperformingschools,DataandEvaluationDivision;DepartmentofEducationandEarlyChildhoodDevelopment.


55

AppendixTableA-1 Schoollevelcovariancestatistics(Intra-classCorrelationCoefficient)* Linearmixedmodel

Schoolvariance ICC

Notadjustedforstudentandschoollevelfactors

Year5Reading 0.18 18%

Year5Numeracy 0.19 19%



Adjustedforstudentandschoollevelfactors





*Allresultsonschoollevelvariancearestatisticallysignificantat1%level.