Measuring the Impact and ROI of community/social investment programs

Measuring the social impact

and return on investment of

development programmes Reana Rossouw

Next Generation Consultants

Introduction / Basics

2016/04/15Next Generation Consultants

2

Definitions

� What is impact?

�Measures ‘the difference being made by an intervention’

�Impact measurement is the process of providing evidence that

your organisation/program - is doing something that provides a

real and tangible benefit.

� What is return?

�Measures ‘the funder benefit/gain’

� What is an impact measurement framework?

� The activities an organisation carries out can have long-term effects

on beneficiaries, beneficiaries’ families and the broader community.

An impact measurement framework seeks to identify and

quantify/qualify this impact/change.

� An impact measurement framework provides the structure for

assessing all aspects of impact.


3

Why is measuring impact and return

important?

� Measuring impact can help to

� Build on the things you are doing well and to learn from the challenges you have

faced/experienced.

� This feeds into good practice and means the organisation will learn and improve continuously.

� Just as financial accounts prove the viability of a business, impact measurement

can show a robust and rigorous approach to providing community or economic or environmental benefits.


4

Why the pressure to measure?� The debate of impact and return on investment are playing out in three

arenas:

� In private/corporate foundations and CSI divisions

� Aiming to be more strategic about their social/community investments and

development programmes

� In nonprofit organisations in response to pressures from investors, corporates,

foundations and government

� To be more accountable for the resources received and programme outcomes

expected

� Among all other role-players such as government agencies, intermediary

organisations, and international development agencies

� Seeking to improve development effectiveness and lessen dependency on

development aid2016/04/15Next Generation Consultants

5

Why now?

� Funders are increasingly asking for verifiable results – to understand the difference they make, directly and indirectly in the economy, society and the environment

� This trend is accelerating, and the sector is increasingly looking to pay for/by results – and to learn from what they, and those they fund, do

� It is not just donors that care about impact. In a age of competition, transparency, and recessionary economies there is a growing competition

for resources


6

Impact Assessment - Variety of purposes

� One can and should use impact data to make programme decisions acrossinvestment and development portfolios

� One can and should use impact data to make funding decisions within investment and development portfolios

� It is about building a unifying measurement standard, as well as a conceptual framework for understanding the biggest impact across a Rand value unit. So - it is not just about comparing one investment or one development portfolio to another or one organisation to another, but to also determine which programme/organisation/investment yields the highest return/impact for the most effective use of resources

� Therefore, we require a combination of the breadth of quantitative with the depth of qualitative evaluation/assessment methods


7

DifferencesDefinition Detail

MONITORING:

Regular systematic collection and

analysis of information to track the

progress of programme implementation

against pre-set targets and objectives.

DID WE DELIVER?

• Clarifies programme objectives

• Links activities and their resources to objectives

• Translates objectives into performance indicators and sets targets

• Routinely collects data on these indicators and compares actual results with

targets

• Reports progress to managers and alerts them to problemsEVALUATION:

Objective assessment of an on-going or

recently completed project, programme

or policy, its design, implementation and

results.

WHAT HAS HAPPENED AS A RESULT?

• Analyses why intended results were or were not achieved

• Assesses specific casual contributions of activities to results

• Examines unintended results

• Provides lessons, highlights significant accomplishments or programme potential and offers recommendations for improvement

IMPACT ASSESSMENT:

Assesses what has happened as a result

of the intervention and what may have

happened without it, from a future point

of time.

HAVE WE MADE A DIFFERENCE AND

ACHIEVED OUR GOAL?

• Seeks to capture and isolate the outcomes that are attributable (or caused

by) the programme

• Will review all fore-going monitoring and evaluation activities, processes, reports and analysis

• Provides an in-depth understanding of the various causal relationships and the

mechanisms through which they operate

• May seek to synthesize, compare, contrast a range of interventions in a region, timeframe, sector or reform area

8

The impact value chain


9

Input

Resources that are deployed in

service of a certain (set of) activities

Cash, Products, Services, Hours, Non-cash –

buildings, time, books, resources (wheelchairs,

books)

Invested resources or capital deployed

Activity

Actions, or tasks that are performed

in support of specific impact

objectives

Actions by an organisation to

facilitate a change

Activities undertaken to

deliver on program objectives goals

Output

Tangible, immediate practices,

products and services that result from the activities

that are undertaken

Number of beneficiaries served by an organisation/ program

Services rendered through the capital

provided

Outcome

Changes, or effects on

individuals/communities that follow from the delivery of products and

services

Changes among beneficiaries i.e.

increased capacity to earn a

living

Impact

Changes or effects on society or the environment that

follow from outcomes that have been achieved

Changes in broader

environment for instance reduced incidences of

crime

Impact on society due to capital deployed

Return

Benefit gained/achieved by the funder as a result of the input

resources

Changes in the business or operating

environment for instance better stakeholder relationships

Benefit by the funder due to the impact capital

invested

Guidelines/process for impact measurement

Goal setting

Framework development and indicator selection

Data Collection

Data Validation

Data Analysis

Data Reporting

Data Driven Management

� Set goals

� Articulate the desired impact of the investment – establish a theory of change/value creation process to form the basis of strategic planning and decision-making to serve as a reference point for investment performance

� Develop Framework and Select indicators

� Determine metrics/indicatos to be used for assessing the performance of the investment - the framework integrates indicators and outlines how specific data are captured and used, it uses indicators that align with existing standards to measure progress/impact

� Collect data

� Ensure that the information, technology, tools and resources, human capital and methods used to obtain and track data for the anticipated impact are effectively utilised

� Validate Data

� Validate data to ensure sufficient quality – verify that impact data is complete and transparent by checking assumptions and calculations against known data sources, where applicable (baselines)

� Analyse Data

� Distil insights from the data collected – review and analyse data to understand how investments are progressing against impact goals/objectives

� Report Data

� Share progress with stakeholders – distribute impact data coherently, credibly, and reliably to effectively inform decisions by all stakeholders

� Make Data-driven management decisions

� Identify and implement mechanisms to strengthen the investment/development process and outcomes – assess stakeholder feedback on reported data and address recommendations to make changes to the investment or Theory of Change for sustainable development

2016/04/15Next Generation Consultants 10

Indicators� Indicators define/measure and demonstrate the outcome of an objective

� Indicators should be easily measurable (within reason) and meaningful to stakeholders

� In selecting indicators the following factors may be influential:

� Who — the target population to be measured/assessed

� How much — the degree of change that is expected

� How many — the amount of change among the target population that would indicate a successful level of achievement

� When — the time frame in which this change should occur

� Indicators can be quantitative or qualitative or both

� Quantitative indicators measure changes expressed as numbers or statistics, e.g. frequency of behaviours, number of attendees

� Quantitative data can also measure changes in attitudes, beliefs, and perceptions when respondents express opinions in the form of ratings or rankings

� An advantage of a quantitative data collection approach is that data can be easier to collect and analyzethan qualitative information. However, data that is easy to gather may not be the most illuminating or insightful

� Qualitative indicators relies on observations of changes with the intent of understanding patterns and relationships. Data are often in the form of descriptions, narratives, and open-ended responses

� Qualitative data might be used to capture individual and collective stories of social change, including personal perceptions

� Sometimes qualitative evidence is the only way to document changes in relationships and contexts


11

Important indicator selection issues

� Involve stakeholders in defining indicators

� Consider both quantitative and qualitative indicators

� What is most important to a program or initiative may not be easiest thing to measure. In

collecting data that is easily accessible or purely quantitative, stakeholders run the risk of being dismissive of the true and deeper impact of a program

� Conversely, a cursory anecdotal review of a program done by those who are predisposed to view it as successful would not likely hold up to scrutiny from outsiders

� Determine if your outcomes—and therefore the indicators you define—are short-,

intermediate- or long-term

� Measurement of short-term change will most likely focus on the implementation of a project (How did it work? How effective was the design?) or on establishing a baseline from which to assess change down the road.

� Intermediate change can be measured by defining benchmarks. A benchmark is a type of indicator that refers to the level of change an organisation expects to make from its baseline.

� Most evaluators and researchers agree that impact takes time to occur and that outcomes are only measurable in the intermediate- or long-term. The longer a program runs, the more complex

its selection of indicators and the more likely, or even necessary, it becomes to use benchmarks.2016/04/15Next Generation Consultants

12

Important indicator selection issues

� Consider a program’s context and its complexity as this will affect results and the selection of indicators. Select indicators at the beginning of a program or project in order to plan for data collection, but also allow for indicators to be defined or revised as new outcomes emerge during program implementation.

� Decide on a modest range of realistic indicators. It is nearly impossible for a few indicators to capture complex social change, particularly since change tends to happen incrementally and can be attributed to numerous inputs.

� Avoid common pitfalls.

� One frequent mistake is that people choose outcomes that they believe they can easily achieve and measure based on the desire to show success. Conversely, they may ignore outcomes that

are difficult to achieve and measure.

� Another mistake is to disregard negative outcomes or not be alert to unexpected outcomes that, if recorded and assessed, can be just as important in terms of understanding impact.

� Yet another pitfall is the tendency to attribute causality for social change to one modest program

when that change is caused by multiple factors.


13

What Impact Assessment is all about

� Impact

�Identify, interpret, improve, investigate, involve, inform

� To provide evidence

� To demonstrate performance

� To prove accountability

� To show programme/investment effectiveness

� To demonstrate shared/blended value

� To empower and capacitate all stakeholders

� Ultimately - to alleviate, reduce and eradicate poverty and contribute to sustainable development


14

Conducting an impact assessmentMethodology: Impact Investment Index™

Developed by Next Generation Consultants


15

Impact Assessment – WHAT?


16

Calculating Impact – The Process


17 Information Sources (Strategies, applications, contracts, evaluation reports, site inspections,

engagement)

Primary, secondary and tertiary assessment

Data score sheet –identifying and

calculating impact and return

Impact per stakeholder (QL & QN)

Return for investorDimension of impact

and returnAnalysis, interpretation, triangulation of data

Impact per programme, per focus area per stakeholder group

Return per programme per focus area

Cost benefit analysis Shared Value X:YRecommendation

strategic, operational and programmatic

How much was spent – where – on

what?


18

Case Study 1 - Multichoice

Total Impact – Per programme Total Impact – Per Focus Area


19

Case Study 1: Multichoice

Total Return per Brand Total Return by Investment


20

Case Study 2: Pioneer Foods

Spend per focus area Impacts per focus area


21


Impact across the triple bottom line

4

3 3 3 3 3 3 3

2 2

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

Economic Impact

7

5

4 4

0

1

2

3

4

5

6

7

8

F&TFA-

Limani

Aqua Noir Heart MOT

Social Impact


22

8 8 8

3

2 2

0

1

2

3

4

5

6

7

8

9

Environmental

Impact


Impact over time

6

3

2 2 2 2 2

0

1

2

3

4

5

6

7

Short-Term Impact

4

3

2 2 2 2 2

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

Medium Term

Impact


23

7

4

3 3 3 3 3

0

1

2

3

4

5

6

7

8

Long-Term Impact

Case Study 2: Impact per focus area

97

91

Paardeberg WWF

Impact Scores for SED

Focus Area -

Environment

10097

91

79 78

60 60

50

38

Impact Scores for SED

Focus Area - Food

Security


24

103

61

MOT PFECT Bursary

Scheme

Impact Scores for PFECT

Focus Area - Education

71

25

Aqua Noir Katmakoep

Impact Scores for Focus

Area - Enterprise

Development

Case Study 3 – BHP Billiton Metalloys –Programme performance

Top five

94

7669

64 62

Bottom five

34

4446 46 47


25

60

54

52 52

Middle four

Case Study 3: BHP Billiton Metalloys -Impact across focus areas per programme

Commuity Impact High

Developmental

Business Impact High

Strategic

HIV and Me 94:7

Community Impact Low

Charitable

Inkululeko 52:1

Cansa 44:2

Reach for a Dream 34:3

Business Impact Low

Commercial

Health

Average 56:3

Community Impact

High

Developmental

Eureka School 76:2

Meyerton High School

69:1


Strategic

Dr Malan School 64:3

PC Literacy for Educators

52:4

Community Impact

Low

Charitable

Springfield Primary 46:1

Sibongile School 46:2

Business Impact Low

Commercial

Wings for Life 54:3

CASME 47:3

Education

Average 57:2


26

Commuity Impact High

Developmental


Strategic

Kotulong Community

Centre 62:4

Raizcorp Business

Incubation 60:4

Community Impact Low

Charitable

MCD Training 52:3

Business Impact Low

Commercial

Poverty

Alleviation

Average 58:4

Impact vs. Return: Programmes vs.

Focus Areas

Commuity Impact Above Average

Eureka School 76:2

Meyerton High School 69:1

Dr Malan School 64:3

Business Impact Above Average

HIV & ME 94:7

Kotulong CC 62:4

Raizcorp Business Incubation 60:4

Community Impact Below Average

Wings for life 54:3

MCD Training Centre 52:3

Adv. of Science and Maths 47:3

Reach for a Dream 34:3

Sibonile School 46:2

CANSA 44:2

Inkululeko Project 52:1

Springfield Primary 46:1

Business Impact Below Average

PC Literacy for Educators 52:4

Total Impact 852

Total Return 43

Overall Averages

56:3

Commuity Impact Above Average

Business Impact Above Average

Education 454:19

Community Impact Below Average

Health 224: 13

Poverty Alleviation 174:11

Business Impact Below Average

Total Impact 852

Total Return 43

Overall Averages

284:14


27

Case Study 3: BHP Billiton Metalloys –

Impact vs. Cost


28

Individual cost per impact and return on investment for the community investment and development programmes

Total Spend Programmes Total Impact Total Return

R 62 678 764,00 15 852 43

Cost per Impact R 4 933. 39

Cost per Return R 78 841. 21

Individual cost per impact per Focus Area

Focus Area Spend Programmes Total Impacts Cost per Impact

Health R 1 364 500.00 3 217 R 2 096.00

Education R 36 314 264.10 8 472 R 9 617.00

Poverty Alleviation R 25 000 000.00 3 158 R 52 742.60

Individual cost per return per Focus Area

Focus Area Spend Programmes Total Returns Cost per Return

Health R 1 364 500.00 3 15 R 30 322.22

Education R 36 314 264.10 8 27 R 168 121.59

Poverty Alleviation R 25 000 000.00 3 11 R 757 575.75

The table below indicates that the Poverty Alleviation portfolio/focus area delivers the least community impact,

whilst the health and educational portfolios delivered more cost effective impact for communities.

The table below highlights the fact that to achieve a single impact in the community R4 933 needs to be spend per programme.

Similarly, R78 800 needs to be spent to achieve a single return on investment for BHP Billiton Metalloys South Africa.

The table below clearly indicates that the health portfolio not only had the highest return on investment for BHP Billiton Metalloys, but

was also the most cost effective portfolio.

The Educational portfolio delivered the second highest return on investment and the Poverty Alleviation was not only the mostexpensive, had the least community impact, but also the least return on investment.

Return on Investment


29

Business value of determining ROI

KnowledgeDeep understanding of value and

impact as well as risks

Comparative data –industry/sector

Insights into and across impact

dimensions

Insights into stakeholder groups affected

ActionNew or enhanced business

decisions, practices and

behaviours

Develop new

products/services/markets

Changes policies, strategies and

practices to increase impact and return

Report in a more credible,

integrated and useful way

ResultsImproved performance –profitability/competitiveness

Reduce potential risks –community activism/licencing

Save costs – of court cases/mitigation of risks

Enhanced stakeholder relationships

Improved licence to operate conditions

Improve trust and transparency


30

Return on investment – Indicators

Strategic AspectsSupport of corporate values and

strategies

Support of sustainability strategy/programs

Support of future growth,

development and market access

Investor /

Shareholder AspectsShare price not affected when

industry or sector are targeted by

activists

Rated as industry leader in

Sustainability Indices

Increased investment from socially

responsible investment funds

Inclusion and high ratings in

awards programs

Reputation AspectsRecognition/awards

Media coverage

Increased brand awareness


31

ROI Impact

Profit Aspects

Sales generated from programs

linked to products

Value of new products and services generated from CI/CSI

programs

Increased worker productivity

Increased share price (e.g. from attention of socially-screened

investment funds)

Environmental

AspectsCosts mitigated from rehabilitation

Costs saved from waste

management/recycling

Carbon emissions sequestrated

Costs of fines

Sector Specific AspectsFinancial Sector

• Economic trends and demographics and expanding workforce needs

• Increasing regulatory activity (e.g. CRA, PRI, CRESA, JSE, investment screening)

• Increasing equality/disparity between haves/have-nots – financial inclusivity

• Globalization

• Opportunities to brand company through community involvement

Mining

• Intensity of opposition

• Previous negative incidents

• Regulators’ sensitivities

• Compatibility with existing development objectives

• Reputation of company

• Level of community involvement

• Involvement of external advocates


32

ROI Impact

Stakeholder Aspects

Increased community/government

awareness/positive

relationships/stakeholder relations

Decreased complaints/grievances/

activism/strikes/boycotts/negative

press coverage

Cost savings/avoidance

Prevention of operational

stoppages/delays

Reducing/decreasing legal costs/law

suits

Support for market entry/expansion

plans

Savings Aspects

Tax rebates received from

philanthropic/

charity/social/community contributions

Saved costs of free advertising space

received from media coverage of the

CI/CSI programs

Legal fees averted (includes legal

department staff time and projected

billable hours from contracted firms)

Savings Aspects Continue

Crisis PR efforts averted (includes PR staff time

and projected billable hours from contracted

firms)

Costs of avoided down-time from failure to

receive building approval, work stoppages,

etc.

Reduced employee recruitment costs, reduced turnover costs, and/or reduced absenteeism

Reduced employee training costs (e.g.,

through community service learning initiatives)

Reduced customer turnover

Other staff management hours saved


33

ROI Impact

Customer AspectsSurveys indicating improved customer perceptions and impacts on shopping decisions

Sales leads generated in specific geographic or demographic markets

Development/increased sales of specific products/services in targeted geographic or demographic markets

Annual brand tracking surveys indicating higher scores

Collaboration/participation/co-design of new product/service development

Greater participation/involvement/ contribution in community investment and development programs

Increased brand awareness

Increased customer acquisition/retention

Operational AspectsMitigation of operational risks

(health/environment/safety)

Support and enhancement of business operational requirements

(integration, skills development,

etc.)

Compliance Aspects

• B-BBEEE

• Licence to operate

• SLP Mandate/Strategy

• DMR/King III/ICMM/IPIECA

• Approval rates/new explorations/extensions

• Rehabilitation

• Drop in complaints/grievances

• Global Compliance


34

ROI Impact

Employee Aspects� Positive response to utilizing volunteerism for professional development/skills

development and team building

� Employee surveys demonstrating that volunteer activities contribute to leadership development

� Voted one of the best companies to work for

� Surveys showing increased employee morale from participation and increased numbers of employee volunteers, volunteer hours, and the number of company-sponsored volunteer projects

� Satisfaction surveys indicating positive impact and anecdotal evidence

� Employee training programs designed to use volunteers and products with most donations

� Employees learning to use products to that they are more equipped to sell/market them

� CSI/CI projects used for team building or during orientation/induction or other training

� Recruitment from communities where CSI/CI projects are run

� Internal surveys showing an increase in employee pride, morale and commitment as a result of employee involvement in volunteer activities

Social Aspects

� Improvement of quality of life

� Community job creation / empowerment

� Improved stakeholder relations within the community

� Poverty reduction


35

IN CLOSING


36


37

Impact: Thinking beyond evaluations:

Decision

•Now what?

Synthesis

•So what?

Analysis

•What impact was achieved?

•What does the evidence show?

Method

•How will we determine impact or gather evidence?

Information

•How will we know?

•What evidence do we need?

Standard

•What would indicate impact?

Criteria for impact/ value of impact

•What matters?

Key Question

•What do you want to know?

Thank You

� Reana Rossouw - Next Generation Consultants

� Next Generation Consultants are internationally recognized and have published extensively and spoken at local and international conferences. Copies of these articles, research papers, presentations, whitepapers and awards are available on:

� Website: www.nextgeneration.co.za

� Linkedin: https://www.linkedin.com/company/next-generation-consultants

� Google+: https://plus.google.com/+reana rossouw

� Pinterest: https://www.pinterest.com/reanarossouw/

� Facebook: https://www.facebook.com/ nextgenerationconsultants/

� Slideshare: http://www.slideshare.net/Reana1

� Please Note:

� The material is this presentation is copyrighted

� Permission must be obtained for using the material/content of this presentation.


38