View
170
Download
0
Category
Preview:
Citation preview
Next Generation Impact Measurement
Measuring What Matters to Nonprofits
Why Do We NOT Know If We Are Making a Difference?
What are Nonprofits Being Asked to Measure?
10% Have Some Output Data
Disconnected datasets<500 cases per dataset
No ”true” outcomes
1% have a Little Bit of Outcome Data
Disconnected datasets <250 cases per dataset
30%-50% aren’t using valid and reliable metrics
100% of 501c3s Have 990s
Shared Dataset600,000 cases
No program quality or outcome data
600,000
60,000 ….
$400M spent on filing 990s $12B $30B
What’s Stopping Us?
The MetricsChallenge #1
Lots of Outputs Little Outcomes
The Balance of What We Measure
The Unfair Economics of the Nonprofit Sector
Consumer Buyers “Bulk” Proxy Buyers
A Consumer Buys Personal Outcomes A Funder Buys Population Impact
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for a Population
The Act of “Bulk” Proxy Buying Creates Unrealistic Accountability
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for You
Outcome Metrics aren’t Realistic
Lower Stress Reduction in Child Abuse
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for You
Outcome Metrics aren’t Realistic
Better Infant Care Skills and
Knowledge
Achievement of
Developmental Milestones
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for You
Outcome Metrics aren’t Realistic
Improved Maternal Physical
Recovery
Reduction in Subsequent Pregnancies
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for You
Outcome Metrics aren’t Realistic
Better Sibling Adjustment
Reduction in Risky
Behaviors
Postpartum Doula ServicesBought by Me, for Me
Postpartum Home Visitation ServicesBought by Others, for You
Outcome Metrics aren’t Realistic
Better Communication with Providers
Kindergarten Readiness
Show Us Don’t Show Us
As a Result, Our Current Impact Metrics…
How MuchWe Are Doing
Direct Results
What Works
The MethodChallenge #2
Does a Program Work for the Average Participant?
How Does a Program Work for Targeted Segments?
Means versus Me
Did the Program Recipe Move the Average?
Were Always Above Average
Still Below Average
The Average Moved
Which Program Ingredients Work for You?
You are like me! Can you tell me how you did it?
Data AnalysisChallenge #3
Analyses Describe What Just Happened
The Unresolved Challenges of Impact MeasurementChallenge #3: Data Analysis
Analyses Describe What Just Happened
Amount, Frequencies,
Averages, Trends
Statistical Significance of
Outcomes
Predict What Might Happen Provide Insight on What Might Work
Very Few Existing Analyses…
Describes Never
As a Result, Our Current Impact Metrics…
Often, What We Just Did
Sometimes, Our Results
Diagnoses ProblemsPredicts What Might
HappenProvides Insights About
What To Do
Measuring in IsolationChallenge #4
We Don’t Share Stories, Insights or LessonsData Analytics Findings
The Solution
Step One: Come Together w/in Fields of Practice
Come Together w/in Fields of PracticeThere’s no one-size-fits-all ideal business
performance and impact model for ALL for-profit service companies
Each ”industry” has it’s own standards, and when it comes to evaluating their services, each business within an industry competes on a set of common
market-driven (customer-driven need and demand) outcomes.
Academic Achievement Consulting/Capacity Building
Employment Career Advancement
It’s time to do the same within our
sectorYouth Development
Nonprofit Capacity Building
Senior Services
Workforce Development
Environmental Policy
Etc.
Step Two: Agree on Shared Metrics
Develop Shared Metrics
Target Population Field Practices Achievable Field Outcomes
Step Three: Real-World Methods
Natural ExperimentControlled Experiment
Conduct Natural Experiments
Add Predictive Analytics
“Predictive analytics tells you what will [likely] happen”
Jeff Bertolucci, Information Week
Add Prescriptive Analytics
“…prescriptive analytics tells you what to do about
it.”Jeff Bertolucci, Information Week
Find the Positive Deviants!
Step Four: Make Meaning, Together
Everyone Gathering Stories, Finding Positive Deviants and Sharing How They Did It
Data-Driven Decision Making for Impact
Recommended