Upload
business-connectors
View
36
Download
0
Tags:
Embed Size (px)
Citation preview
vMEASURING IMPACTAnne Kazimirski, Head of Measurement & Evaluation, NPC
Connect@Evolve, Monday 15th June 2015
NPC: TRANSFORMING THE CHARITY SECTOR
2
NPC works at the nexus between charities and
funders
Charity
SectorFunder
Increasing the impact of charities
eg, impact-focused theories of change
Strengthening the partnership
Eg, collaboration towards shared
goals
Increasing the impact of funders
eg, effective commissioning
Consultancy & Think tank
THE FOUR PILLARS APPROACH TO
MEASURING IMPACT
3
Map your
theory of
change
Prioritise
what you
measure
Choose
your level
of evidence
Select your
sources and
tools
Effective measurement
framework developed
Strategic vision
Leadership
Case for impact measurement
DEVELOPING AND USING
A THEORY OF CHANGE
WHAT IS A THEORY OF CHANGE?
Links activities intermediate outcomes final outcomes
A description of how activities lead to outcomes
5
- Clarifies what the activities aim to achieve and how
- Provides the case for why achieving intermediate outcomes is important
- Provides a structure for identifying what can be measured
HOW TO REPRESENT A THEORY OF CHANGE
6
Planning Triangle Logic Model Outcomes Chain
However you represent your theory of change, it should be supported by a written description.
7
Children’s emotional
resourcefulness
improves
Counselling
Clients’ ability to support
their children's healthy
development improves
Clients’ emotional or
psychological
difficulties decrease
Mother / Child
interaction
improves
Clients’ capacity
for self care
increases
EXAMPLE (SIMPLIFIED) THEORY OF CHANGE
MOTHERS’ COUNSELLING SERVICE
Activities Intermediate Outcomes Final outcome
PRIORITISING OUTCOMES
PRIORITISE OUTCOMES THAT:
• Are directly influenced (rather than indirectly supported)
• are important / material to the mission
• are not too costly to measure
• will produce credible data
9
CHOOSING THE RIGHT
LEVEL OF EVIDENCE
COUNTERFACTUAL
• Comparing the world with your
organisation in it with what the
world would be like without it.
• Control group
• Attribution
11
LEVELS OF EVIDENCE
12
Randomised
control trial
Anecdotes /
quotes
Before and
after survey
Self-reported
change
Case
studies
Control
groups
Credibility
Basic Advanced
SELECTING DATA
SOURCES AND TOOLS
• Quantitative data (numbers)
• Statistical estimates
• Prevalence of views, attitudes and experiences
• Admin data or questionnaires (paper, web, etc.)
• Qualitative data (words)
• Detailed understanding
• In-depth interviewing (telephone or face-to-face)
• Observation
• Stakeholders’ views
Proportion of beneficiaries
whose outcomes have
improved, and by how
much.
What did beneficiaries
think, did it make a
difference to them? How?
How could it have been
better?
DIFFERENT TYPES OF EVIDENCE
EVALUATION TRAPS
15
Collect data that matters, and work together
Don’t force
squares into circles
& don’t collect
arbitrary data
INSPIRING IMPACT: MEASURING UP! AND
THE IMPACT HUBMeasuring Up!
• online, step-by-step self-assessment tool
• looks at the way you plan, evidence, communicate and
learn from the difference your work makes
The Inspiring Impact Hub
• pulls together resources relevant to improving impact
practice
• enables users to search and filter results according to their
needs
inspiringimpact.org
16
ACTIVITY – BENEFITS AND CHALLENGES
17
Two questions to discuss:
• What are the benefits of measuring impact?
• What are the challenges of measuring impact?