FROM TRAINING TO PRACTICE TRANSFORMATION

Implementing a public health parenting program

Sally Gaven, Evaluation Project ManagerJanet Schorer, NSW Department of Family & Community Services

Nexus Evaluation Team: Greg Masters, Sally Gaven, Dr Louise Askew, Ashley Pennington

Professor Sven Silburn of Menzies School of Health Research in Darwin,Professor Sven Silburn of Menzies School of Health Research in Darwin,

A/Prof Stephen Jan of The George Institute for International Health, University of Sydney

Agenda 1. Since when did parenting

become public health?

2. Evaluating the population impact of a parenting programprogram

3. What changed? • In the system• In the system• For families

4. The 21st century challengey g

5. 5 steps to practice transformation

6. Q&A

A public issue

Childhood conduct problems

Increased risk for:• Crimeconduct problems• Substance abuse• Mental health

bl d i idproblems and suicide• Domestic and family

violenceviolence• Poor physical health• Early parenthood

No other common childhood condition with such far-hi

y p

reaching consequences.Christchurch Health & Development Study (Fergusson 2010)

Triple P

Parenting intervention aimed at reducing the risk factors for poor developmental outcomes in children.

By: Professor Matthew Sanders and colleagues at University of Queensland’s Parenting and Family Support Centre -Bandura’s cognitive social learning theory 1977, 1986

Improves: and prevents behavioural, emotional and developmental problems in children – by improving knowledge and skills of parents

Evidence: By far the most successful intervention for childhood conduct problems.

The Triple P system of flexibleThe Triple P system of flexible intervention and reach

Breadth of Reach

Flexible Delivery FormatsIncreasing Intensity

Flexible Delivery Formats

Individual Self-directed Group

Low Parenting Information Campaigns

LEVEL 1Brief Parenting Advice

LEVEL 2Narrow Focus Parent Skills Training

LEVEL 3Broad Focus Parent Skills Training

LEVEL 4Behavioural Family Intervention

Source: Sanders, M.R. & Prinz, R.J. (2005) The Triple P System. The Register Report Spring: 42-46.

High LEVEL 5

Population-level, public health approach - a coordinated system of training and accreditation for

titipractitioners:

• Over 62,000 practitioners trained

• From Australia, New Zealand, Singapore, Hong Kong, Canada, United States, England, Scotland, Belgium, The Netherlands, Curacao, Republic of Ireland, Japan, Germany, Switzerland, Sweden and Iran

• Across various fields of health, education, childcare, general practice and social welfare.

Triple P in NSW

Families NSW: range of prevention and early intervention initiatives to improve the health and well-being of families

ith hild f bi th t i htwith young children from birth to eight years.

jointly delivered by:NSW Health

Department of Family & Community Services

Department of Education and Communities

In partnership with families communityIn partnership with families, community organisations and local government.

Long term aims: children’s developmentLong term aims: children s development

Levels 2 and 4: seminars and groups

Reduce risk factors:• The prevalence of early onset behavioural and emotional

problems in childrenproblems in children • Coercive, harmful & ineffective parenting • Parents’ emotional distress and conflict

Increase protective factors:• Parental confidence and efficacy• Positive parenting practicesp g p• Participation in evidence-based parenting programs

And to build the capacity of communities to support parents:And to build the capacity of communities to support parents: • Capacity and confidence of service providers

• Interagency collaboration and referral pathways

Implementation of Triple P via Families NSW

To offer free parenting training to all NSW parents with children 3-8 years.

Led by Community Services - $5m in Government funding over four years to:to:

• Governance via Triple P Working Groups

• Train 1,100 practitioners from government and non-governmentTrain 1,100 practitioners from government and non government agencies 2008-2010

• Support delivery by each accredited practitioner (2 x Seminar Series & 2 G o ps pe ea ongoing)& 2 x Groups per year, ongoing)

• Collect data on delivery for program management and evaluation

• Support networks of practitioners• Support networks of practitioners.

Governance Training Delivery Data NetworksGovernance Training Delivery atacollection Networks

The tri-fold evaluationquantity and quality of outputs• Program data – practitioners, courses, attendees, demographics• Focus groups – partners & over 60 practitioners

P titi

PROCESS:

• Practitioner survey

quantity and quality of changes in child behaviour

• Non-randomised controlled trial• Non randomised controlled trial• Pre and post outcome data • Family survey

quantity and quality of changes in parenting practices

OUTCOME

quantity and quality of changes in parenting practices

• Family survey • Attendee Satisfaction Survey

costs and benefits of creating these changes

• Costing analyses – head offices, regional offices, providersECONOMIC

longer-term population impacts

• Literature review, emerging implementation science

ECONOMIC

Outcome evaluation methodsQuestion MethodQuestion Method

OverallWhat are the short and longer term effects of Triple P on parents and children?

Level 2: quasi-experiment

Level 4: pre and post scores analysis

Family interviews

Attendee satisfaction data analysisWhat are the outcomes for nominated target groups? Level 4: pre and post scores analysis

Family interviewsParentsWas there an increase in use of and confidence in positive parenting behaviours?

Family interviewspositive parenting behaviours?Was there a decrease in coercive, harmful or ineffective parenting behaviours?

Family interviews

Was there a decrease in emotional distress over parenting?

Family interviewsparenting?Was there a decrease in parenting conflict? Family interviewsDid the program meet participants’ needs? Attendee satisfaction data analysis

Family interviewsyWere there unintended consequences? Family interviewsPopulationWhat were the population-level impacts? Literature review

Quasi experiment study design

A1 A2Triple P group

( 104)A3

(n=104)

Comparison group (n=78)

B1 B2 B3

Prescore

Postscore

6 month follow-up

Section 5ChallengesChallenges

Section 4 Section 6Outcomes Impacts

Section 7Section 3Output

Section 7Strengths for Development

Section 2Overview

Section 8Future Lessons

FNSW Triple P Evaluation: Final Report

What was done? Outputs

Program and client demographic and satisfaction data to examine the quantity and quality of:

• The practitioner training

Th T i l P• The Triple P courses delivered to parents by practitionerspractitioners

• The support for practitioners available through Familiesavailable through Families NSW.

Practitioner Training

How much?

• In just over two years, over 1,000 practitioners were trained to deliver Triple P.

• Two thirds are from 250 non-government organisations.g g• Of the Government practitioners, only 19% are from the Department of

Education and Communities.• Most practitioners are child and family workers, caseworkers or managers.

100 d li T i l i l th th E li h• 100 can deliver Triple in languages other than English.

How well?

• 86% of trained practitioners achieved accreditation.N l ll titi f lt fid t b t d li i T i l P ft

How well?

• Nearly all practitioners felt confident about delivering Triple P after training.

Course delivery to parents

1 257 T i l P d li d b th d f 2010 92 i th

How much?

• 1,257 Triple P courses were delivered by the end of 2010 - 92 in other languages.

• Only 60% of trained practitioners had started delivering courses to families.

• Only a third are delivering the expected number of courses.• An estimated 12,500 attendees came to Triple P by end 2010.• These attendees are generally more disadvantaged than the general

pop lationpopulation.• Their children experience more emotional and behavioural difficulties.

H ll?

• High client satisfaction• Program fidelity tools not consistently used – although changes mainly

How well?

• Program fidelity tools not consistently used although changes mainly suit client needs

• Broad engagement strategies not widely used.

Practitioner Support

Implementation science shows that high-fidelity, effective and efficient practitioner delivery needs implementation d i th t fdrivers that focus on:

• Targeted practitioner selection

• Training

• Ongoing coaching consultation and supportOngoing coaching, consultation and support

• Data systems to support decision-making

( ll b d• System interventions (collaboration and peer support –international experience).

Focus of the 2010 issues paperCommunity Services initiated several specific actions to support practitioners:

• Triple P Working Groups – patchy

• Peer Support Groups – mentoring, co-facilitation

• Annual Practitioner Development Day - 300 practitioners from NSWacross NSW

• Scoring application to improve attendee demographic and outcome data entry by practitionersy y p

• Practitioner website promoting practitioner networking and collaboration

• Promotional resources - to broaden, streamline and standardise engagement strategies

A i t f di t t T i l P titi ( i l• Assistance funding to support Triple P practitioners (regional practitioner forums, AV equipment, childcare, refreshments and venue hire).

What changed? Outcomes

Level 2 – quasi experiment

• Children of sampled families attending Triple P showed C d e o sa p ed a es a e d g p e s o edsignificant improvements in behaviour (in contrast to children in the non-treatment comparison group).

• Quasi-experiment: only the Triple P group showed a Q p y p g pmarked improvement six months after doing a Seminar Series compared with the no-treatment

f b h b d lcomparison group – for both boys and girls.

• 10% net reduction in children in the clinical range.

Q iQuasi:SDQ Total Problems Score - Means and 95% Confidence Intervals

Sc

ore

s

6 month follow-up

6 month follow-up

Pro

ble

ms S

SD

QTo

tal P

S

Triple P Comparison

NOTE: the blue, green and red lines refer to the 95% confidence interval and the dot in the middle shows the mean

Triple P group clinical status:Triple P group clinical status: SDQ total problems scorepre score6 month follow-up

88.2100

78.575

% SDQ scores

21.525

50% SDQ scores

clinical/ normal

11.8

0

25

Pre-intervention 6 month follow-up

Clinical range Normal range

Level 4: Pre and Post scores

Level 4 attendees reported significant improvements in:

• Parenting behaviours (Parenting Scale)

• Child behaviours and emotional difficulties (SDQ)

Over 11% net reduction in the proportion of children in the SDQ clinical range.

Level 4: Clinical status on SDQ total problems score pre and post-intervention

100

p ob e s sco e p e a d post te e t o

64.775.2

75

% SDQ scores

35.324.8

25

50% SDQ scores clinical/ non-

clinical

0

25

P P tPre-program Post-program

Clinical range Non-clinical range

Family surveys

Most respondents felt, following Triple P:

• Their child’s behaviour had got better

• They had changed their parenting practices

• More confident in their parenting.

Challenges

However, the evaluation articulates two key challenges in d li i l ti b ddelivering a population-based parenting program:

• Translating training into delivery of the program to familiesfamilies

• Achieving reach into the population by engaging apopulation by engaging a sufficiently broad range of families.families.

The evaluation has demonstrated thatThe evaluation has demonstrated that the implementation has, so far:• Successfully trained a multi-disciplinary workforce

• Enabled high quality delivery

• Achieved positive results for the clients it has reached

However:• Too few of those trained are delivering coursesToo few of those trained are delivering courses

• Only a fraction of the planned courses are being delivered

• Fewer families than expected are attending• Families attending are not representative of the• Families attending are not representative of the

general NSW population.

1,180

TARGET

on trackpractitioners trained:

ACTUAL

ll i 2 i & 2 b /3 hi iFull-time: 2 seminars & 2 groups Part-time: 1 seminar & 1 group

expected to deliver:

about 1/3 achieving expected delivery

to achieve a total of:

1,674 Seminars per year1,674 Groups per year

In 2010:301 Groups

287 Seminars

to reach: 300,000 familiesBy end 2010:

12,580 familiesto reach: 300,000 families 12,580 families14% of expected reach

to result in:improved family and population outcomes

improved family outcomes

ImpactsCosts

• Direct investment $5M - leveraged $8M value

• Cost per child $641 – high because early in implementation?

$ %

Estimated total cost of implementation of Triple P in NSW

Head office 4,744,820 59.2

Partner agency 86,000 1.1

Regional co-ordination 663,408 8.3

Delivery 2,520,625 31.4

Total overall cost 8,014,853 100.0

ImpactsPractice impacts

• Doing a better job

• 90% would recommend Triple P

• 90% said it enhanced client

• collaboration benefits

• But – at the major cost of time pressure, for time-poor practitioners!ut at t e ajo cost o t e p essu e, o t e poo p act t o e s

Population impacts

• Evidence of longer-term social benefits and reduced costs from• Evidence of longer term social benefits and reduced costs from Triple P

• Extrapolating the outcome data, Triple P in NSW has already moved p g , p y1,150 children from the clinical to the non-clinical range.

Wh h Th t hTraining a pool of

practitioners

Who have delivered high quality, well-i d

That have achieved good

results for families d hildreceived programs and children

H th i l t ti i t t hiHowever, the implementation is yet to achieve:

• Expected delivery rates

• A viable population reach.

Practice transformationThe transition from science to service

The science of implementation reveals the magic ingredients that power d b d d l levidence-based service delivery to population impact.

Program: Population impact:

Evidence based Implementation drivers that transform practice:

High fidelity Targeted participation

Effective Ongoing support

Efficient Data-driven decisions

Collaboration(Fixen & Blasé 2009)

Fergusson et al 2011

Apply the prevention science paradigm to program implementation:

• Select evidence-based programs

• Pilot - program acceptability and fidelityPilot program acceptability and fidelity

• Randomised trial - program efficacy at the new site

T k t l i l it• Take to scale progressively - monitor program effectiveness.

The 21st Century Challenge:

Strong body of evidence for effective treatment of conduct disorders BUT implementation barriers:

• Policy makers Researchers

• Resistance to change• Resistance to change

• RCT phobia

Triple P in NSW has cleared most of these barriers:

• Trained over 1000 practitioners in an evidence-based a ed o e 000 p a o e s a e de e basedprogram

• Tested the early implementation: quasi-experimentTested the early implementation: quasi experiment

• Policy-research engagement.

5 steps to practice transformation

The hurdles ahead: establishing the infrastructure to support delivery and achieve population reach.

Strengths to be developed:

• Universal entry pointsUniversal entry points

• Broad and strategic program promotion

I t t d d t ll ti ( t t d li t• Integrated data collection (program output and client outcome data)

l d l• Active central delivery management

• Concerted and consistent practitioner support at all levels.

Strength 1: Universal engagement

Ron Prinz (US Triple P Population Trials) identifies four key features of the population approach to implementing Triple P d f di blP – and four corresponding enablers:

Features of the population Universal engagement eatu es o t e popu at oapproach

U e sa e gage e tstrategies

1. Aiming for population impact Mass media campaigns

2. Using existing multi-disciplinary workforce Interagency collaboration

3. Broad availability and accessibility Universal entry points

4. Sources of efficiency Inclusivenessy

Population-level impact enabler = mass media campaign

Recommendation: mass media campaign

N t j t i i

Key messages

• All parents need information d t t b th b t th• Not just awareness-raising -

depict practical parenting solutions

and support to be the best they can for their children

• Going to parenting courses can• Address stigma (‘listen up bad

parents’)

• Going to parenting courses can make a huge difference to children’s development and i f il lif• Create positive norms

• Drive program engagement.

improve family life

• Parenting information and support is helpful and easy to

Strengths

support is helpful and easy to access at the FNSW website.

• Triple P has a mass promotional component• Families NSW website.

g

M lti di i li kf bl i tMulti-disciplinary workforce enabler = interagency collaboration

Recommendation: active interagency working groups – with funds

Without genuine, proactive engagement at the regional Working Group level, driven by the FNSW senior officers group, the d li d h f T i l i lik l t idelivery and reach of Triple is unlikely to improve.

Involving Working Groups in the expenditure of the annual f d dassistance funding, to support practitioners and promote

collaborative delivery, would help engage these groups.

Strengths

• Families NSW structures promote interagency collaboration• Community Services is raising Triple P’s profile as a central Families NSW

intervention.

B d il bilit d ibilit bl i l tBroad availability and accessibility enabler = universal entry points

The literature focuses on using universal entry points –specifically:

Recommendations to:• Integrate Triple P with transition

to school programsspecifically:• An education sector that

embraces and funds parenting t i h l tti d

to school programs• Promote Triple P internally with

Department of Education & support in school settings, and

• Healthcare settings that offer parenting support.

Communities• Make direct approaches to

principals and P&Cs to promotepa e t g suppo t

However, less than one fifth are from the Department of Ed ti d C iti

principals and P&Cs to promote Triple P

• Neutral venues - deliver through local government services andEducation and Communities.

Strength

local government services and libraries.

• Good take-up of Triple P by health practitioners, delivering Triple P in community health settings.

g

Sources of efficiency enabler = inclusiveness

Creating a context where the population approach belongs to all of the participating professionals, the parents, the entire communitycommunity.

Recommendations to broaden engagement strategies beyond the welfare sector:welfare sector:

• Service specifications in funding agreements to support broad community engagementy g g

• Promotional activities with a universal reach – beyond service clients.

St th

• Very high level of enthusiasm for Triple P from delivering practitioners.

Strength

Strength 2: Practitioner support andStrength 2: Practitioner support and collaborationOngoing practitioner engagement relies on:

• Completing the full training process, including accreditation p g g p , g

• Practitioner’s confidence following training

• Time between training and delivery• Time between training and delivery

• Organisational support for practitioners Seng, Prinz, & Sanders, 2006; Sanders, 2008.

We identified 4 sources of support in NSW: Recommendations for:

• Community Services (and Families NSW) Ongoing funding

• Service providers Delivery accountability

• Peers Active Peer Support Groups

• Collaboration Practice networks for co-delivery

Strength 3: Data collectionImplementation is optimised when:

D t di d ti

Recommendations:• Ongoing funding dependent

d t ll ti• Data recording and reporting is embedded

• Data covers the input-

on data collection • Track delivery and enforce

accountabilityData covers the inputoutput-outcome spectrum

• Data is used to track delivery

accountability• Annual practitioners survey • Longitudinal study of Triple

• Evaluation can increase delivery rates and reach.

g y pP cohort – tracking use of child protection, health, social and justice services

Strength

social and justice services.

• Custom-built scoring application to capture a range of client data.

Strength 4: Delivery management

Focus on selecting and training practitioners

Governance bodies need to extend their involvement to the management

R d ti f ti t l d li t

their involvement to the management of program delivery to clients.

Recommendations for active central delivery management:

• Memoranda of Understanding with service provider organisations

• Locking in delivery requirements

• Systems to track actual delivery.

Lessons for the futureThe emerging implementation science literature emphasises the importance of systems and practice transformation and is starting to explore alternatives to the training model in the science to serviceexplore alternatives to the training model in the science to service bridge.

In summary, large scale universal prevention and early intervention programs require:

i. Universal entry points soft, accessible, stigma-free

ii. Program promotion light, direct population-wide touch

iii. Data collection systems use input, output, outcome data -accountability

iv. Involvement in delivery management active and central delivery trackingdelivery tracking

v. Practitioner support early, consistent, collaborative.

Thank youa you

W t ?Want more?http://www.families.nsw.gov.au/assets/triplep_eval_report.pdf

sally@sallygaven.com.au