A RCT evaluation of The Letterbox Club in Northern Ireland Dr Karen Winter and Jennifer Mooney...

A RCT evaluation of The Letterbox Club in Northern Ireland

Dr Karen Winter and Jennifer Mooneyk.winter@qub.ac.uk

jmooney06@qub.ac.uk

• Poor educational outcomes

• Measures: attendance rates; suspensions; exclusions; attainment in tests, exams and formal qualifications

• Contributory factors: interplay of individual child; familial and care related characteristics

• Disability; abuse; trauma; physical/emotional ill health; poor familial relationships; multiple transitions; poor attachment; stigma; labelling; lack of support for carers

Context

• Legislation

• Policy and practice initiatives

• Macro level - structures and processes

• Micro level – direct to child interventions

• Concern with evidence, what works, cost effectiveness

Initiatives

The Letterbox Club

• Once monthly parcel for 6 months • May to October• Books, stationery, number games• Addressed to the child• Personalised letter• Interest level, not ability level• Aims • www.letterboxclub.org.uk

Contents

• Letterbox Club began 2002• 2003-2006 pilot work• 2007 – national pilot funded by government• 2008; 2010 evaluations by programme designers

in England and Northern Ireland• Gains in reading and number skills • 2011 independent evaluation data in Northern

Ireland – gains reported• www.qub.ac.uk/cee

Available research

Can the gains in reading and number skills be attributed directly to the Letterbox Club

intervention?

The gap

Comparison using standardised scores - limits

Importance of control group – identical, random allocation, evenly balanced, variations accounted for

Essential if we are serious about addressing inequality Next slides consider 3 elements to study design: the RCT, logic model and the process evaluation.

The gap

• Does not account for previous research around reading

• Assumes a linear movement • Hinges on feelings of ownership• Assumes children do not have access to

materials• Middle class deficit view of children in care

Logic modelMismatch between inputs and anticipated outcomes?

Current studyThe RCT study

Flow chartRCT study participants

Measures• Neale Analysis of reading ability• Elementary Reading Attitude Survey (Garfield)

Other data• Age• Gender• Type of placement• Trust area

Logic modelData collected

• Placement moves• Carer information• Length of time in care• Siblings/ foster siblings• Letterbox fun days

• Attitudes• Ethics• Collaborators• Fieldworkers• Results • Responses to results

RCT challenges

• Interviews foster children and their carers• Exploring:• what actually happens at time of receipt of

the parcel; • engagement with parcels;• views on parcel content;• and ongoing support with the materials.

Logic modelProcess evaluation

• Further Studies- nurture groups and school based interventions

• Collaborations- with colleagues in Canada and REES centre (Oxford)

• Development of the programme

Many Thanks!

Logic modelThe Future