Session 3: Trial management Sarah Miller (Queens, Belfast) Laura Dunne (Queens, Belfast)

Session 3: Trial management

Sarah Miller (Queens, Belfast) Laura Dunne (Queens, Belfast)

RCT MANAGEMENT: KEY ISSUES AND LESSONS LEARNED…

Dr Laura Dunne & Dr Sarah Miller

Centre for Effective Education | SCHOOL OF Education

“An ounce of prevention is worth a pound of cure”

Related issues:

• Recruitment

– achieving required sample and power

• Retention

– Treatment attrition

– Measurement attrition

• Communicating with schools and developers

Introduction:


• RCT evaluation of Barnardo’s Ready To Learn afterschool programme: 16 schools and 8 data collection sweeps over three years

• RCT evaluation of a classroom based social emotional learning programme: 74 schools and 5 data collection sweeps over four years

Two examples…


The programme:

• Aims to raise achievement by focusing on children’s early literacy skills and enhance parental engagement in children’s learning

• Programme runs during term time for one hour after school on Tuesday, Wednesday and Thursday afternoons

• Activities in the programme include story telling, drawing, singing, puppets, talking and listening

RCT Evaluation of Barnardo’s Ready To Learn Afterschool Programme


The evaluation:

• A cluster randomised design combining outcome data with experiences of participants (over three years, 8 data sweeps)

• 16 schools, 564 Year 1 pupils (9 intervention; 7 control)

Measures

• Battery of age-appropriate tests , focus groups and parental interviews

• Principal interviews

RCT Evaluation of Barnardo’s Ready To Learn Afterschool Programme


The Programme:

• A universal programme delivered on a whole-class basis for one academic year

• 27 lessons based around 9 themes delivered by a trained RoE instructor

• Lessons revolve around a monthly classroom visit by an infant and parent

• The parent-infant visit serves as a springboard for discussions about understanding feelings and infant development and effective parenting practices

RCT evaluation of the Roots of Empathy programme


The Evaluation:

• Cluster randomized controlled trial: 74 primary schools randomly allocated to control and intervention groups

• Schools in the intervention group receive the programme in their selected Year 5 class for one academic year

• Schools in the control group do not receive the programme but receive it the following year

RCT evaluation of the Roots of Empathy programme


School recruitment

• Actively engage with schools re: both the programme and the methodology e.g. attend information sessions, meetings with service and Principals.

• Make it clear to schools what their participation will involve (MoU)

• Meetings with service and Principals

• Consider a wait list control

• Financially compensate for teacher time

• Consent prior to randomisation

Key issues


Retention

• Regular communication: newsletters each term, study updates, Christmas cards, annual principal interviews, school visits

• Minimise disruption and be flexible - pilot measures, testing P1-3, stickers

• Document reasons for dropout and monitor attrition between intervention and control groups

• If a school drops out of the intervention, continue to collect data (ITT)

• Process evaluation: observation and field notes

Key issues


Communication

• Make clear to schools who in which team is responsible for each element of the study (MoU), provide a timetable of events

• Identify a key contact to liaise with in the school

• Minimise the number of different people the school has to deal with

• Agree a research plan and be clear who is responsible for what

• Regular (and efficient) communication between programme developer(s) and research team (Skype, email etc)

• Set realistic expectations of the possible outcomes of the evaluation

Key issues

Documents

Session 3: Trial management Sarah Miller (Queens, Belfast) Laura Dunne (Queens, Belfast)