Measuring and Reporting your Social Value

The CheckUP Forum14th September 2018

About us• The Incus Group

• Purpose-led consultancy, founded in 2015 by professionals with substantial experience and expertise in social, environmental and economic impact measurement.

• We work with organisations to help measure what matters and convert best intentions into successful outcomes

• Jerry Marston • 18 years working for listed companies in senior corporate social responsibility

roles in the UK and France• 7 years in overseas development • 12 years consulting in Australia, specialising in social impact measurement• Co-chair the Social Impact Measurement Network Australia (SIMNA) QLD

committee

2

SIMNA

3

• SIMNA is a knowledge sharing network for social impact measurement. Its purpose is to help foster the emerging practice of social impact measurement in Australia. SIMNA supports competency development by individuals and capacity building by organisations through sharing knowledge of, and promoting education and training in social impact measurement.

• The SIMNA Queensland Committee is led by a committed group of professionals from a range of organisations – not for profit, academic, public sector and corporate.

• We organise 4-6 member events a year on topics of interest suggested by them, with a typical attendance of 45-60 from across all sectors and disciplines.

• It’s easy to join SIMNA. Simply click here and fill in the form and we’ll keep you up to date with SIMNA national news and events in Queensland. For more information, visit our website

Outline for the session

1. Why do you need to measure and evaluate 2. How to develop your story (or theory) of change 3. How to measure your social value4. How to report and communicate your social value

4

Financial vs social performance

5

• The 2015 Pro Bono Australia Sector Survey revealed a disconnect between how well organisations felt they measured their financial performance and social outcomes created

Financial vs social performance

6

• The disconnect remained when considering the size of organisations (number of employees)

Quick glossary

Short term outcomes: The short term change that occurs as a result of a project

• Improved interview skills

• Access to regular meals

Longer-term outcomes: The long term changes that occurs as a result of a project – the intended results

• Traineeship

• Improved physical health

Value: The ultimate changes for stakeholders to which the project contributes

• A meaningful, sustained job for a long term unemployed person

• Improved quality of life for someone experiencing food poverty

7

8

Why do you need to measure and evaluate

Why measure your outcomes and value?

Beneficiaries – Clients

Help clients make better choices and achieve their goals

Demonstrate your alignment with their values

Funders – Government, foundations, donors and corporates

Recognised limitations of funding activities and focusing on outputs not outcomes

New types of funding models (Social Impact/Benefit Bonds)

Mission and strategy

Gaining clarity and consensus around the purpose of your work

Mission statements usually focus on outcomes

External factors Internal factors

Performance measurement

Measurement drives where resources are allocated

Identifying effective practices & areas for improvement

10

Why do you need to measure value?

• What are your needs?• To fulfil reporting requirements• To understand whether and how you create social value• To continuously improve your programs• To improve your fundraising• To communicate achievements to your stakeholders (internal and external)• To advocate for change

11

Why do you need to measure value?

• Now ask yourself:1. At what scale do you want/need to know (one program vs systemic issue)?2. Is it possible to complete the process in stages (qualitative quantitative picture)?3. What resources do you have available (staff time, volunteers, budget for external support)?4. What existing data and research can you use?5. What opportunities do you have for collecting more data? (end of year event, annual staff

day)6. How rigorous do you need it to be and why?

What’s the best approach for measuring social value?

• There is no ‘best’ approach • Most Significant Change, Social Return on Investment, Results Based Accountability, Social

accounting and audit, etc…• Each methodology has its strengths and weaknesses

• Apply Principles

12

Principles for measuring social value*

Principle 1: Involve your stakeholders • Measure what matters from the perspective of your stakeholders – the people or organisations that have

experienced change as a result of your activities.

Principle 2: Understand and measure your story of change • Apply appropriate methods and resources to understand and measure outcomes (positive and negative,

intended and unintended)

Principle 3: Be credible and transparent• Be accurate and credible by taking account of:

• How much did other factors contribute to the change? (Attribution/Contribution)• What would have happened anyway? (Counterfactual)

• Report/share your findings and methods

13*Principles derived from list here: http://socialvalueint.org/our-work/principles-of-social-value/

14

Develop your story of change

Involving stakeholders

• Two stage process to involve stakeholders:1. Map your stakeholders2. Engage with stakeholders through qualitative research, e.g.

• Focus groups• Interviews • Questionnaires• Social media • Diaries

15

Mapping stakeholders - example

16

Community housing

customers

Families and

friends

Partner agencies (e.g. Health, Education,

Legal, Employment)

Community groups

Local councils and

residents Grant providers

Staff

Community Housing Provider

Executive / BoardVolunteers

Department of Families

and Community

Services Corporate partners

Local schools

Employers and educators involved with customers

Primary beneficiary and related groups

Delivery partners

Interested parties

Funding partners

Internal stakeholders

17

Community housing

customers

Families and

friends

Partner agencies (e.g. Health, Education,

Legal, Employment)

Community groups

Local councils and

residents Grant providers

Staff

Community Housing Provider

Executive / BoardVolunteers

Department of Families

and Community

Services Corporate partners

Local schools

Employers and educators involved with customers

Mapping stakeholders - example

Engaging stakeholders

• Key questions: • What has changed (both positively and negatively) as a result of being involved with this

program? • What aspects of the program contributed to those changes?• What do you think would have happened if you had not been involved with this program?• How can the program be improved?

• If you don’t have resources to directly engage with your stakeholders, look at existing studies on your stakeholder group or similar programs to yours

18

Story of change

• A story, or theory, of change depicts how a program is intended to achieve meaningful, positive changes for stakeholders

19

Inputs Activities Outputs Outcomes

What resources will be used to pursue project

goals?

What actions will be

undertaken to achieve project

goals?

What will be produced

through these resources and

actions?

What longer-term changes will

result from this project?

Value

The ultimate changes for

stakeholders to which the project

contributes

Key elements

• An expanded logic model that:• Depicts pathways to change • Identifies measures of success

• Useful for Planning, Ongoing monitoring and Evaluating

20

Example: Community housing for disadvantaged clients

21

Arrange local events and activities

What is being done (activities)…

Provide access to newly built housing

What will be produced (outputs)…

Conduct wellbeing visits

Responsive property maintenance

What resources are being mobilised (inputs)…

80 units built

Staff time / salaries

Operating expenditure

80 new social and affordable housing units

# of wellbeing visits

% of requests completed on time

# of events and activities arranged

Reduced stress over housing

situation

Know someone is interested in their welfare

Increased sense of security

Tenancy issues addressed

swiftly

Increased housing stability

Sense of belonging and

inclusion

Peace of mind

Feel part of a community

More social interactions

Positive outlook for the future

More confident to plan and set

goals

What changes will occur (outcomes)…

Depicts pathways to change

Identifies measures of success

22

Exercise: Identifying change

• Creating our own stories of change1. List your main activities on the butcher’s paper 2. Develop outcomes pathways for 1 stakeholder group using the post-it notes

– write only 1 outcome per post-it note

24

Activity Output Short term outcome

Long term outcome

25

Measure the value you create

Quantifying outcomes

• Measurement drives where resources are allocated and where accountabilities are established.

• We are interested in measuring material outcomes for stakeholders, not just outputs or those outcomes that are easy to measure. Find out what is important, then seek to measure Find out what is easy to collect, so measure it

• Key questions: • How many people have experienced this outcome or change?• How do we know the extent to which this change has taken place?

26

Measuring outcomes - example

27

• Measure outcomes using indicators – ways of knowing a change has occurred

• Healthy eating and cooking classes (activity) aim to make people healthier (outcome). As a result of our classes:

• Some people have become healthier • 25 people have become healthier• 25 people have reduced their BMI by 10%

• What about for more subjective changes?

Quantifying outcomes – outcomes star

28

Quantifying outcomes – bespoke optionOutcome

Achievement Level Indicators

Improved money management

Very Low (1) - I am always short of money for basic expenses- I am unable to pay any of my regularly scheduled bills

Low (2) - I often find myself short of money for basic expenses- I often default on my regularly scheduled bills

Medium (3) - I have tried using a budget but found it difficult to follow - I occasionally fall behind on my regularly scheduled bills

High (4) - I am using a budget regularly but sometimes tend to lose track of my expenses- I am making some changes in my habits to be able to afford my regularly scheduled bills

Very High (5) - I am able to purchase my basic requirements and stay within a budget- I never default on my regularly scheduled bills

29

Example: Ian Potter Foundation

• http://www.ianpotter.org.au/what-we-support/education/

• http://www.ianpotter.org.au/what-we-support/community-wellbeing/

30

Exercise: Measuring success

• Using the A3 template provided, 1. In Column 1, write down your end outcomes2. In Column 2, list potential signs of change (indicators) for each outcome

31

• Swap with someone and rank High/Medium/Low on Relevance:• i.e. is the indicator relevant for the outcome/change described?

Understanding your added value

• Not all of the impact (or lack of impact!) is due to your programs/services

• Important to consider:• How much did other organisations contribute to the change?

(Attribution/Contribution)• What would have happened anyway? (Counterfactual)

32

Attribution/contribution

Time (years)

Attr

ibut

ion

• How many social issues can be resolved by single organisations working alone?

• There are other influences and factors inhibiting and enabling the achievement of outcomes

• Considering who/what else contributes helps understand added value and what else may be required to support your stakeholders

Attribution stages1. No account of attribution levels

Effectively attribution = 100%

2. Estimate of attribution levelsKnowing about what else our participants are doing

(e.g. our participants involved in 2 other programs so attribution to us = 33%)

3. Quantitative researchQuantitative stakeholder research gives average stakeholder’s view

(e.g. participants attributed 50% of Outcome A and 75% of Outcome B to us)

4. Academic researchExisting research can help refine the values (e.g. longitudinal studies of similar programs)

Counterfactual example

8-2 = 6Number of outcomes we

can claim credit for

2 = The number of outcomes that would’ve

happened anyway

8 people in the program got a job

Start of program

End of program

2 would have gotten a job without your support

• We run an employment readiness program with 10 newly arrived migrants and have an 80% “success rate”

36

Report and communicate success

Utilising and reporting outcomes data

• What are your needs and objectives?• To fulfil reporting requirements• To understand whether and how you create social value• To continuously improve your programs• To improve your fundraising• To communicate achievements to your stakeholders and supporters• To advocate for change

• Be transparent and utilise a mix of case studies/stories and numbers

37

Utilising and reporting outcomes data

• What are your needs and objectives?• To fulfil reporting requirements• To understand whether and how you create social value• To continuously improve your programs• To improve your fundraising• To communicate achievements to your stakeholders and supporters• To advocate for change

• Be transparent and utilise a mix of case studies/stories and numbers

38

• 10x10 events are run by volunteer committees who select three innovative charities to receive $100 each from 100 guests in a ‘crowdpitching’ evening.

• At the end of the event there is a call for volunteers to form the next 10x10 committee.

• Using existing survey data from volunteers and charities plus 10 in-depth interviews, we developed a theory of change

Example for communicating your achievements: 10x10 Philanthropy

39

https://10x10philanthropy.com/

40

“The best part was getting to know some great people from across different industries and backgrounds. Then knowing that our small investment of time went towards making a real difference.” – Committee member

“I think the biggest thing for us was the network, the connections and the people we met through the process. Connecting with other charity partners, all of the people who were there on the day, getting to connect with young people, talk to people about what we’re doing and spread the word the work that we’re doing” – Charity

Example for how social value is created: Milk Crate Theatre

• Partner with professional arts facilitators and community service providers to deliver workshops, theatre productions and films to build confidence, skills and connections for people experiencing homelessness

• Conducted 3 x 2 hour focus groups with arts facilitators, participants and MCT staff

• What have you seen change for participants in MCT as a result of their involvement?

• What is it about the program that enables these changes to happen?

• Is there anything you’ve been able to achieve through MCT that you wouldn’t have achieved elsewhere?

41

https://www.milkcratetheatre.com/

Theory of change for MCT

42

Short-term outcomes Medium term outcomes

• Participants feel valued and heard • Able to have a voice and express themselves• Being proactive

• Sense of pride in achieving something• Sense of pride in self

• Participants able to access needed support• Participants feel supported – there is someone they

can go to for help

• Maintains focus• Participants feel valued • Sense of community

• Participants see Artist Facilitator taking risks and exposing themselves

• See other participants dealing with similar issues• Willingness to work with one another

• Increased creative expression and risk taking• Increased consideration / reflection• Ability to see things from different perspectives

• Given tools to explore emotions and express themselves

• Setting goals for themselves

• See different possibilities and how to make decisions

• Take up new opportunities – study, work, grant applications

Increased confidence

Interpersonal skills

Social connections

Increased aspirations

Improved decision-making

Sense of self-worth

Measuring the outcomes

Outcome Quotes Potential measures on a before / after scale (1-5)

Data source

Increased social connections

“I wouldn’t have found another way of connecting and meeting new people”

“I have made new friends” “I feel connected to the Milk

Crate Theatre community”

Participant self-report

Improved decision-making

“Recently a participant has gotten a corporate job and had negative responses from people from her old job. She said drama made her understand she could make different choices in her life and those negative responses were not facts, just their thoughts, which helped her imagine her way out of things”

“I know how to consider different options in making a decision”

“Participant demonstrated a willingness to try new ways of approaching things and resolving issues”

Participant self-report

Facilitator/ social worker observation

43

X% of participants with increased social connectionsY% of participants who experienced improved decision

making

Example to advocate and improve fundraising: Lifeline

• The MLC Community Foundation and the Lifeline Research Foundation undertook a Social Return on Investment (SROI) study to measure the social value created by Lifeline Crisis Support Chat Service.

• Three main social outcomes were identified for the service users:• Reduced suicidality/self harming potential • Improved resourcefulness• Enhanced belonging

• Two areas of cost re-allocation were identified for public services:• Reduced use of hospital services• Reduced call outs for police and emergency services

• For every $1 to run the program, $8 of value was being created for the users and the government

Source: https://www.lifeline.org.au/static/uploads/files/lifeline-online-crisis-support-chat-sroi-report-final-wfegdzzgtacg.pdf

Example to advocate and improve fundraising: Lifeline

Key steps to takeaway

1. Understand your organisational needs2. Map and engage your stakeholders3. Understand your story of change 4. Measure the outcomes and account for added value 5. Report (and celebrate!) your social value with a mix of stories and numbers

46

47

Thank you

www.theincusgroup.com