Reasonable ROI Tracking – Measuring Reasonable ROI Tracking – Measuring Internal and External ImpactInternal and External Impact

Marie Beason

Indiana Grantmakers Alliance

Section I: IntroductionSection I: Introduction

Thesis of the Presentation

Expectations for companies as responsible corporate citizens are rising, and stakeholders are demanding results. Clearly defined goals with strong social impact makes for a more compelling story to tell your stakeholders, resulting in greater impact for the business.

Telling a Compelling Story

All of your stakeholders need some level of information.. the “story.”

Section II: Key IssuesSection II: Key Issues

Why Measure?

Increasing Expectations• Internal and external stakeholders are demanding more “results”

• Companies are expected to help address issues, not just give money

• Employees expect their company to be a leading corporate citizen

• Community Leaders expect a commitment to the community

• Customers/Clients expect a company to meet standards

Competitive Advantage• Most companies measure only outputs, not outcomes

• Most companies are unable to articulate how CI affects their business (reputation and/or employee recruitment, retention, morale, etc.)

• Trend is toward defining specific goals for major programs; opportunity still exists to be a leader in an industry or geographic area

Program Improvement• Findings can be relevant, timely and useful

Which Programs Need to be Measured?

Program Content Resources

Signature Program: Teacher Training in Grand Ville

Targeted support of teachers in 3 public schools in poor communities

$100,000 annually

50 volunteers

Employee contributions

Major Focus Areas: Education, Social Services

Support of critical needs in education and social services in Grand Ville

$75,000 annually

Business / Client-DrivenSupport for events, organizations based on client requests to sales reps around the country

$25,000 annually

Employee Engagement (Volunteerism, Dollars for Doers, Matching Gifts, etc.)

Support for matching gifts and dollars for doers

$50,000 annually

APC focuses primarily on its headquarters community and where its employees live and work

Multiple Stakeholders withVarying Needs for Results

Companies need to identify priority stakeholders, issues of concern to those stakeholders and then determine level of measurement needed for each stakeholder

Barriers to Measurement

• Different programs have different expectations of “results”; lack of internal consensus on what to measure

• Portfolio of programs – major initiatives require more rigorous outcomes measurement

• Limited budgets or support for deep measurement and evaluation

• Most companies have not engaged senior management effectively to gain support for increased measurement

• Limited internal knowledge/capacity regarding measurement and evaluation

• Most companies have limited staff and do not have the capacity or knowledge to measure their programs

Section III: Defining MeasurementSection III: Defining Measurement

vs. ReactiveProactive

How Can Measurement Data Be Used?

What Can Be Measured?

InputsInputs

Monitoring:What is the quantity

and quality of the resources we have

invested in this program? Are they

aligned with our program goals?

Performance measurement:

What is the quantity and quality of the products or

services we have delivered?

OutputsOutputs

Outcome measurement:What are the

results for participants,

grantees, employees, the business, and

other stakeholders?

OutcomesOutcomes

Impact study:What are the

social and larger business impacts attributable to the

program/ company?

ImpactImpact

Methods of Measurement

Monitoring Performance measurement

Outcome measurement Impact study

Documenting Research

Tracking investment and progress Determining causality

Minimum evaluation expertise required High level evaluation expertise required

Low cost High cost

Strategies for Measurement

Low-Cost Strategies

•Develop a program model with clear and measurable goals

•Ensure that program vision, goals and objectives align

•Focus on measuring inputs and outputs

•Create a tracking system to regularly monitor and report progress (inputs and outputs)

Medium-Cost Strategies

•Develop a program model with clear and measurable goals

•Create a measurement framework that lays out key measurement questions, indicators, data sources, data collection methods

•Focus on measuring inputs, outputs and short-term outcomes

•Utilize data collection tools that have been developed or could be developed easily

•Create a tracking system to regularly monitor and report progress

Higher Cost Strategies

•Develop a program model with clear and measurable goals

•Hire an external measurement expert to design data collection plans, systems and tools

•Provide PD for grantee’s internal staff around data collection and measurement

•Fund peer-to-peer exchanges re: measurement

Section III: Section III: Defining Your Approach Defining Your Approach

1. What Can Be Measured?

InputsInputs

All the resources needed to support a program:•People•Dollars•Marketing•Lines of business

The specific activities and/or programs that serve a particular target audience:•Signature programs•Focused giving•Employee engagement activities

OutputsOutputs

Affects on target audience:•Community awareness, perception of company•Grantee knowledge, attitudes, behaviors•Employee morale, retention, engagement, team work, leadership, productivity

OutcomesOutcomes

The long-term and aggregate effect of the program on overall target audience and the company:•Social impact•Business impact (Reputation among key stakeholders)

ImpactImpact

3. What Level of Information Is Needed?

Signature

Major Issue or Focus Area

Giving

Business-Driven Giving

Employee Programs

Pro

gra

ms

Inputs Outputs Short-term Outcomes Long-term Outcomes Impact

Levels of Information Needed by Stakeholders

Program ContentKey Stakeholder(s)

ReachedLevel of Information

Needed

Signature Focused AllImpact; Long-term

outcomes

Major Issue or Focus Area Giving

Focused on several issues

Regional Stakeholders; Community Partners;

Employees

Outputs; short-term outcomes

Local Area GivingBroad: based on

local needsLocal community leaders,

employeesInputs and outputs, some

short-term outcomes

Client-Driven Giving

Broad: based on client requests

Clients, Business Partners

Inputs

Employee Programs

Broad: based on employee

interests and company priorities

Employees, Community Partners

Varies by program; primarily inputs and

outputs, some short-term outcomes

3. What Tools Can Be Used to Measure?

In House:• Financial databases/reporting• HR surveys and data (recruitment,

retention, morale, productivity, performance)

• Gifts databases• Grantee reports• Line of Business systems and

reports• Existing data (consider marketing

and sales information)• Media Hits• Consumer data

External:• Document/File Review• Surveys• Interviews• Focus Groups• Core Capacity Assessment Tool

(CCAT)

Program Content ResourcesLevel of

MeasuresMeasurement

Tools

Signature Program: Teacher Training in Grand Ville

Targeted support of teachers in 3 public schools in poor communities

$100,000 annually

50 volunteers

Employee contributions

Outcomes

• Teacher survey• Employee survey• Media hits

Major Focus Areas: Education, Social Services

Support of critical needs in education & social services in HQ community

$75,000 annually Outputs

• Financial databases

• Gifts database• Grantee reports

Client-DrivenSupport for events, organizations based on client requests

$25,000 annually Inputs• Financial

databases

Employee Engagement (Volunteerism, Dollars for Doers, Matching Gifts, etc.)

Support for matching gifts and dollars for doers

$50,000 annuallyOutputs - Outcomes

• Financial databases

• HR, survey• Media hits

How to Measure… An Example: APC

4. What Resources Do You Currently Haveor Can You Leverage?

Skills and resources(Also consider other departments)

• People (time and expertise)• Money• Technology• Partners• Leadership

Existing data/ tools(Also consider other departments)

• Marketing reports• Demographic analyses• Community data, reports

Company culture• HR data• Department data

Broader context

5. Who Participates in Measurement?

MeasurementMeasurement

Community Stakeholders,

Grantees, Volunteers

C-Suite Executives

Senior Leaders and Managers

Community Involvement

Staff

Implementer / Coordinator

ReviewerReviewer

Recipients of Findings

6. How Much Will It Cost?

Monitoring

1 – 2%

Performance measurement

3 – 5%

Outcome measurement

5 – 10%

Impact study

10% +

OutcomesOutcomesStrategiesStrategiesInputsInputs

The key “learning-based” question

7. Evaluating Your Program

Which resources were most important for providing “high-quality” service?

Which strategies (program qualities) were most

important for achieving the desired outcomes (including

business outcomes)?

What was the quantity and quality of the

resources we used to implement our

programs?

How much service did we provide and what was the quality of that service?

How were our employees involved (to what extent, in

what ways)?

Did our business or employees change?

Did our service recipients change?

If so, how much and in what ways?

The key “learning-based” question

Key takeaways

Maximize the resources you already have• People (time and expertise)• Money• Technology• Partners• Leadership• Existing data/ tools (Program participation, Program Reach, Marketing data,

Company data, local data about the region and community)

Leverage the resources of other departments• HR• Marketing

Select your measurement level based on your resources, the program you want to measure, and the type of information you need for your targeted audiences

Resources

CEP website (foundations and strategy reports): www.effectivephilanthropy.org/index.php?page=publications

DHFS Evaluation Resource Guide. Wisconsin Department of Health and Family Services. Office of Strategic Finance Evaluation Section. Updated June 2006. http://dhs.wisconsin.gov/aboutDHS/OPIB/policyresearch/EvaluationResourceGuide06-06.pdf

Diversity in Philanthropy website: www.diversityinphilanthropy.org/news/case_studies/pdfs/DPP-Evaluation-Case-Study2.pdf

Davidson, Jane. ‘Evaluation Methodology Basics: The Nuts and Bolts of Sound Evaluation’: www.sagepub.com/authorDetails.nav?contribId=532276

GEO website (see report on Foundations and Evaluation and the Change Agent Project): www.geofunders.org/publications.aspx

Joint Committee on National Standards: www.jcsee.org Kellogg Foundation Program Evaluation and Logic Model Manuals:

www.wkkf.org/~/media/475A9C21974D416C90877A268DF38A15.ashx Mertens, Donna; and Ginsberg, Pauline. The Handbook of Social Research Ethics:

www.sagepub.com/refbooksProdDesc.nav?prodId=Book230293 RWJF Evaluation Series: http://www.rwjf.org/files/research/50349.quality.checklist.final.pdf