Evaluation and Organisational Change: Managing Complexity-

Guiding Solutions

anzea 2013 Conference 22–24 July 2013

Alexandra Park, Epsom, Auckland

Michael Blewden and Sarah GreenawayTe Pou o te Whakaaro Nui

Session Overview and Purpose• Background to Te Pou

• Introduce the change embarked on

• Evaluative tools/processes used

• Reflect on utility and value, progress and challenges

Te Pou

• NGO funded by MoH through Health Workforce NZ

• Workforce/service development support

• Mental health, addictions, and disability sectors

Community development

Workforce development

Organisational development

System development

Policy development

Historically Opportunity

Research and Evaluation Programme

Outcomes Information Programme

Workforce Development Programme

Integrated/coordinated delivery

Collaborative approach

Organisational focus

Responsiveness (need, priorities, context, readiness)

Strengths-based/capacity building/ sustained outcomes

Ability to demonstrate success

This way of working became known as the:

Support Effective Practice (SEP) approach

Client management

approach

Engagement

Partnering

Understanding

Agreement Planning /Develop

Delivery

Review

Simple Complex

Partners need and context (e.g. readiness or capacity for change, priorities)

Coexistence Communication Cooperation Co-ordination and Collaboration Partnership

Self Reliance SharedInformation

SharedResources

SharedWork

SharedResponsibility

e.g. information dissemination

e.g. training programme

e.g. service redesign, comprehensive change strategy

Problem/Need Causes and Contributing Factors

Consequences Opportunity

What is the evidence?

Nature and extent

For whom?

History?

Future projections?

Why does problem/need occur?

Are some causes more important or influential?

Are there known casual pathways?

Is anything already known about what is/isn’t effective in response?

What are the consequences of problem/need?

Who is affected – directly and indirectly

What opportunities are presented?

Why are these worth pursuing?

What benefits will be delivered?

Understanding the need

Funnell, S., & Rogers, P. (2011). Purposeful Program Theory. Jossey-Bass, San Francisco, CA

Situational Analysis Issue Causes Consequences Opportunities

Stakeholder engagement

Relationships and engagement often ‘individual’ rather than ‘organisational’

Necessary leaders/leadership not engaged

Services not always delivered efficiently or strategically (e.g. when ready, in right order)

Engagement that builds leadership, commitment and ownership

Delivery more coordinated, needs based, timely, project managed

Community/Society

Organisation

Team

Individual

Work practices

Outcomes for service

users

Factors impacting effective practice

Enhanced services

Enhanced service/workforce development strategy

Enhanced service/workforce development approach

 

 Clarifying the programme theory

Least ability to attribute outcomes to Te Pou

Greatest ability to attribute outcomes to

Te Pou

Enhanced service/workforce development interventions

 

Enhanced outcomes for service users

More about our practice

More about our partnerships

Planning, evaluation, monitoring framework

How much did we do?

WHAT Outputs (Quantity)

How well did we do it?

HOW and WHY Process (Quality)

Is anyone better off?

RESULT Outcomes (Success)

Focus and scope of evaluation

Te Pou programmes

Outcomes Impact

Planning, development and delivery of service

and workforce development

Sector engagement

and participation

Improved outcomes for service

users

Service development

Quality

effective services

Sphere of Control

Sphere of Influence

Sphere of Interest

‘What’/’how well’ ‘What’/’how well’ ‘Is anyone better off?’

Utility and value• Tools and models- translation devices - engagement/communication- shared understanding - connecting to the evidence

• Emergence of evaluative thinking, outcomes focus, critical engagement

Implementation• SEP implemented in a variety of forms

- formalised MoU - complex service change- co-ordination of work at service level - NGO district or regional level development- planning and delivery of training

Challenges/Issues• Environment of constant change

• Conceptual understanding to operational responses

• Strategic developments vs day to day needs