National Child Nutrition Program Cluster Evaluation: Preliminary Findings Evaluation team Colin Bell Jan Garrard Dionne Holland Boyd Swinburn

National Child Nutrition Program Cluster Evaluation: Preliminary Findings

Evaluation team

Colin Bell

Jan Garrard

Dionne Holland

Boyd Swinburn

Deakin University 2003

Background: National Child Nutrition Program

• A community grants program established in 1999 by Commonwealth Department of Health and Ageing

• Aimed at improving nutrition among children aged 0 to 12 years and pregnant women

• Focus on priority communities:• Rural and remote communities• Aboriginal and Torres Strait Islander communities• Lower socio-economic communities

• 110 community-based projects funded nationally • 18 projects in Victoria - $2,190,300 over three year

period


Evaluation: Background

• Evaluation not built into overall program planning • No specific evaluation budget• Importance of evaluation recognised by Victorian

office of Dept of Health & Ageing• 9 projects committed $3,000 each to fund a

‘cluster’ evaluation• Additional $60,000 from Dept of Health & Ageing• Deakin University contracted to conduct the

evaluation


Evaluation: Preview

• A ‘retrofitted’ evaluation strategy• Optimum balance of common measures while

acknowledging the uniqueness of projects, populations and settings

• Evidence base for effective community-based child nutrition programs is poor – good practice often not documented, disseminated and acknowledged

• A practical, feasible evaluation strategy designed to meet the needs of funding body, project management agencies and practitioners


Evaluation: Objectives

• To assess intended and unintended impacts of individual projects

• To gain an understanding of key implementation issues impacting on project success

• To draw upon collective learnings from the 9 projects to guide future efforts in community-based child nutrition


Evaluation: Design & Methods

• Multiple case study design (9 ‘cases’)• Process measures (ie. what have they done?) and • Impact measures (ie. what have they achieved?) • At individual project level - evaluation team assisted

each project to conduct process and impact evaluation• Across projects, evaluation team conducted a total of

53 key informant interviews – combination of core questions and project-specific questions

• Enabled ‘detection’ (but not quantitative measurement) of impacts – useful for unintended impacts

• Provided data on important how? and why? questions


Evaluation: Lessons Learned

• Build evaluation planning into project planning• Provide adequate resources for evaluation

(approx 10-15% of project budget)• Recognise that responsive, needs-based,

individually tailored community-based projects will rarely meet the requirements for rigorous quantitative assessment

• Such projects rarely find their way into conventional evidence-based reviews.


Lessons Learned (cont)

• Focus instead on what can be achieved (measured impacts) in what settings with what population groups under what conditions – recognise this will vary in different priority settings

• Distil common achievements and learnings if possible

• Provide a mechanism for sharing experiences via a ‘community-based projects’ evidence-base.


Cluster Evaluation Projects

• ‘High Five’ School project – Warrnambool• ‘Eat Well, Grow Well’ – Hamilton, Southern

Grampians Shire• H2O for Moonee Valley Kids – North Melbourne• ‘Breast is Best’ – Brimbank, West Melbourne• ‘Healthy Eating for Country Kids’ – Swan Hill• ‘Who’s Eating Gilbert Grape and Vicki Vegetable’ –

Murray Valley Aboriginal Cooperative, Robinvale• ‘A healthy family is built on a good diet of nutritious

food’ - Robinvale• ‘Child nutrition in the West’ – West Melbourne• ‘Food Facts for preschoolers’ - KPV


Aim: To increase the fruit and vegetable intake in order to improve the overall nutrition of prep to year 4 children in the South West of Victoria

Children with fruit in lunchbox

0%

20%

40%

60%

80%

100%

School1 School2 School3 AVERAGE

PRE

POST 1

POST 2

• ↑largest in yr 2&3

• Also ↑ in serves

• Vegetable intake was very low (~5%) & only increased at one of the schools

• Supporting curriculum changes and parent involvement

South West Health Care


Aim:To increase the proportion and duration of breastfed babies in the City of Brimbank, particularly in low SES and CALD communities

• Distributed BF promotion kits to GPs & MCHNs

• Prenatal BF promotion checklist adopted by Sunshine Hospital

• Breastfeeding guide for new mothers (Vietnamese)

• Specific support and promotion of BF in Vietnamese community

• 66 BF-friendly venues• Adoption of Lactation policy by

ISIS

Change in Brimbank BF rates 00/01 to 01/02

0

1

2

3

4

5

Discharge 2 weeks 3 months 6 months

Per

cent

Cha

nge

01/02 75% 69% 51% 34%

ISIS Primary Care


Murray Valley Aboriginal Co-Op

• Murray Valley River Quest

• PAN messages at ‘stops’ on route

• Reached 516 children

• Teachers said it was easy to implement and raised awareness

• In 2003 hours have ↑

Murray Valley Aboriginal Cooperative

Hours walked as a class on MRQ 2002

0

5

10

15

20

25

30

35

School1 School2 School3

Hou

rs w

alke

d Average

Prep

Grade 1/2

Grade 3/4

Grade 5/6

48 towns53 towns100 towns

Aim: Improve nutritional knowledge and healthy eating patterns of pregnant women, young mothers/caregivers of children aged 0-12 years and children within local Aboriginal and Torres Strait Islanders


Western District Health Service

Increases in knowledge after education sessions

0 20 40 60 80 100

Toddler SM tour

Toddler food talk

Intro solids

Healthy eating & BF

Healthy eating & preg

Percent

Pre

Post

• Education sessions effective in improving knowledge of participants

• Supermarket tours and visits to new mums also beneficial

• ↑awareness of conditions which may require dietetic referral among WDHS staff

Eat Well – Grow Well

Aim: To improve the nutritional health of children aged 0-5 years in the Southern Grampians Shire


Swan Hill District Hospital

• ↑access to nutrition education for pregnant/new mums

• Good readership of newsletter and newspaper column

• These were helpful for making healthy food choices

Evaluation of newsletter & newspaper column (n=125)

0 20 40 60 80 100

Agree/SA news helps with healthy food choices

Occ/Always read newspaper column

Agree/SA tips help with healthy food choices

Agree/SA tips useful

Read tips in newsletter

Percent

Aim: To improve the eating habits of all children and pregnant women in Swan Hill and surrounding district, providing GPs with nutrition information kit


Doutta Galla Community Health Service

• ‘H2O for Moonee Valley Kids’• Drink water rather than

sweet drinks

• ‘Food and Families’• 4 week course on

nutrition information and parents skills

• 10-14 Somali women

• Breastfeeding, bottle, solids, vitamin D

Sweet Drink Consumption

0

20

40

60

80

100

Moonee Valley Colac

Aim: To improve the healthy food intake of infants and children aged 0-12 years with Kensington, Flemington, North Melbourne and Ascot Vale


Western Region Health Centre

Fruit in Lunchboxes

0

20

40

60

80

100

Prep Grade 1/2 Grade 3/4 Grade 5/6

Mar-01 Nov-02 Mar-03

• ‘Eating altogether-5 times better’• Fruit, vegetable & water

promotion at 2 primary schools

• Bilingual child education program• In 1 year, 700 people from LD

communities participated in 75 nutrition education sessions

Western Region Health Centre

Aim: Increase nutritional knowledge and capacity of pregnant women, parents and children ages 0-12 with CALD and recently arrived communities, to enhance healthy eating patterns


Robinvale District Health Service

At the MCCD I Learnt How To ...

0

20

40

60

SA S Neither

A/DA

DA SD NR

Cook healthy food Give healthy food to family

Make favorite food healthier Make food from different cultures

• Parent cooking classes• Multi-cultural food day

• Attended by 120-180 people

• Nutrition education sessions

• Large Polynesian population

• Breakfast program at schools

• Linked with MVAC

Aim: Improve nutritional knowledge and healthy eating patterns of parents and children in Robinvale


Kindergarten Parents Victoria

• Staff training sessions• 9 locations throughout Victoria with over 170 participants• ↑ knowledge of Aust. Dietary Guidelines for children, role of nutrition

policy, value of partnerships b/w parents and early childhood staff

• Parent workshops• At sessional preschools, long day care, other services• ↑ understanding of childhood nutrition

• Resources• Staff training sessions, ‘parent workshop’ package, Food Safety

Information Sheet, nutrition built into existing KPV resources (eg Preschool Matters’

Aim: To increase the capacity of early childhood services to deliver consistent, relevant and current nutrition information to preschool children and their parents


Cluster Evaluation

• 53 semi-structured key informant interviews completed• 11 project officer interviews• 39 key informant interviews (single and focus groups)• 2 Steering Committee member interviews

• Combination of core and project specific questions

• Presenting preliminary findings today• Report available early 2004


Reported Impacts

Increased collaborative links

Human services NGOs Govt Depts

• Community

Health Centres

• PCPs

• Royal Children’s

Hospital

• Breast Feeding

Association

• Ecumenical

Migration Centre

• Dept. of Health

and Ageing

• LGAs

• City West Water


Reported Impacts (cont)

Diffusion of nutrition messages• Reached target groups

• Children, parents, carers, pre-schools, kindergartens, primary schools

• And beyond…..• Program participants ‘spread the message’

• Fathers read brochures on breastfeeding aimed at mothers

• Program participants ‘modelled’ healthy eating behaviours


Reported Impacts (cont)

Spin-offs• Children

• Increased concentration in class• Likely reduction in dental carries

• Families• Budget savings (eg. not purchasing soft drink)

• Bilingual workers• Future work• Empowerment


Capacity Building Framework

Organisational Development

Workforce development

Resource allocation

Partnerships

Leadership

Context

Build Capacity

• Infrastructure

• Program sustainability

• Problem solving

Source: Hawe 1999


Organisational Development

• Management support• Projects more effective when management

understood the complexity of community work and supported it

• Recognition of work• Projects better supported within the organisation

when encouraged to showcase their achievements• Additional support from co-workers

• Policy development


Workforce Development

• Project officer training and previous experience in community work

• Needed a sound knowledge of community work and how it differs from clinical work

• Generally, new graduates who lacked field experience struggled unless provided with additional support within the workplace

• Projects easier to implement when working in a team environment

• Important to share learnings with co-workers to gain support and increase organisational capacity to carry out community work


Resource Allocation

• Control of project resources (incl budget)

• Advice on how to get the ‘best bang for your buck’• Project officers need to be aware existing resources

• ‘Cost-effectiveness’ of different strategies

• Continual planning towards obtaining additional resources, especially into the future


Partnerships

• Can be extremely effective• Working towards a common goal, but from varied view

points• Provided additional ‘sponsorship’• Often opened up opportunities for shared access to experts

and resources

• Enhanced opportunities for sustainability as multiple organisations/individuals were able to share the role lessening the burden (time and money) on one person

• Often enabled strategic responses across multiple settings


Leadership

• Essential

• Creative problem solving skills• Needs to be able to take on the view points and the

needs of the local community and develop and re-develop strategies where needed

• Continuity• Leadership role best held by the same person

throughout the project

• Empowerment of others


Context

• Physical, economic, political, organisational and cultural environments of the project• Currently high profile of nutrition• Organisations with community-based and

population focus


What does the literature say about success factors for community-based interventions?

• Limited literature around community-based child nutrition programs

• Community-based smoking programs have been at it a lot longer!

• Systematic review of 32 studies covering 30 years of community-based interventions to reduce smoking (Secker-Walker et al. 2003)


Lessons for child nutrition promotion(Adapted from Secker-Walker et al. 2003)

• Child nutrition needs to be recognised by the community as an important issue

• Community organisation, assessment of community capacity, and the identification of individuals and organisations interested in supporting nutrition interventions are important during project development.

• Community members and staff need skills in working with diverse groups and in health education

• Coalitions need several months to form, and a year or more to become effective change agents in their community


• Interventions to improve child nutrition need to continue for several years

• Resources for promoting good nutrition and support for healthy eating should be readily available throughout the community

• It is important to involve, train, and provide referral services and resources for health professionals

• The use of mass media (print, radio, television) is especially useful for modelling behaviour change, and for changing community norms about nutrition

Lessons for child nutrition promotion (cont)


Conclusion

• Projects have had some good impacts at the local level

• Contributed to the knowledge base about effective implementation

• Important to disseminate these findings

Documents

National Child Nutrition Program Cluster Evaluation: Preliminary Findings Evaluation team Colin Bell Jan Garrard Dionne Holland Boyd Swinburn