Susan Hoff-Capstone Thesis Project

NONPROFIT MEASUREMENTS:

THE SEARCH FOR THE MIDDLE GROUND

CAPSTONE THESIS PROJECT

Submitted in partial fulfillment of the requirements for the degree of Master of Science

in Philanthropy and Fundraising from the George H. Heyman, Jr. Center for

Philanthropy and Fundraising, New York University

Professor Marian Stern

Susan Hoff

May 6, 2013

Nonprofit Measurements: The Search for the Middle Ground - Susan Hoff 1 | P a g e

TABLE OF CONTENTS

INTRODUCTION .............................................................................................................................. 3

DEDICATION .................................................................................................................................... 5

ACKNOWLEDGEMENTS ............................................................................................................... 5

THESIS STATEMENT ...................................................................................................................... 7

A BRIEF HISTORY ............................................................................................................................ 7

Philanthropies and the Demand for New Practices .................................................................... 8

MEASUREMENT AND EVALUATION ...................................................................................... 12

CURRENT CHALLENGES ............................................................................................................. 15

Access to Management Information .......................................................................................... 15

Public Perception ........................................................................................................................ 15

Donations as a Means to an End ................................................................................................ 18

Choosing the Right Methodology ............................................................................................... 19

Is the Demand for Data Limiting Innovation? .......................................................................... 21

Performance Measurements & Risk Management .................................................................... 22

Leadership: Are Nonprofits Too Nice? .................................................................................... 24

Re-Aligning the Culture of the Industry ..................................................................................... 25

ARGUMENTS FOR MEASUREMENT ......................................................................................... 27

Accountability and Transparency ............................................................................................... 28

Competitive Advantage for Funding .......................................................................................... 30

More Effective Marketing and Social Media Strategies ............................................................. 30

Wealthy Young Donors Demanding Results ............................................................................ 32

Enabling a Performance Oriented Culture ................................................................................ 33

ARGUMENTS AGAINST MEASUREMENT .............................................................................. 34

One Size Does Not Fit All ......................................................................................................... 34


Foundations Demand Metrics, But Will Not Fund Them ....................................................... 35

Short Timeframes May Diminish Ambition and Idealism ....................................................... 37

Improved Reporting Does Not Improve Outcomes ................................................................. 39

Impact Measurement Is Misunderstood, Is Not New, and Will Continue to Fail in Its

Attempt to Right All Wrongs ..................................................................................................... 39

WHEN PUSH HAS COME TO SHOVE ...................................................................................... 41

CURRENT MODELS AND METHODOLOGIES ...................................................................... 41

Theory of Change ....................................................................................................................... 42

Logic Models ............................................................................................................................... 46

Your Planned Work ................................................................................................................... 46

Your Intended Results ................................................................................................................ 46

Dashboards ................................................................................................................................. 48

DEVELOPING A COMMON LANGUAGE ................................................................................. 49

BUILD SECTOR KNOWLEDGE .................................................................................................. 52

PerformWell ............................................................................................................................... 52

McKinsey & Company’s Social Sector Office: Learning for Social Impact ............................. 53

RECOMMENDATIONS .................................................................................................................. 53

Books........................................................................................................................................... 53

Reports & Guides ........................................................................................................................ 53

Performance Management Software (Examples) ...................................................................... 54

WHAT MEASUREMENTS HAVE BROUGHT THE FIELD THUS FAR ............................. 54

CONCLUSION .................................................................................................................................. 55

WORKS CITED ................................................................................................................................ 58


INTRODUCTION

Performance and impact measurements are not new to the nonprofit industry.

Interestingly, what I discovered through the process of researching this thesis was that performance

and impact measurements are widely misunderstood. My own knowledge prior to this capstone

project was no more than a basic familiarity with measurements. It came from working in the arts

back in the 1990’s where I raised money through foundation and government grants. Back then

the required measurements focused mainly on inputs and outputs and not so much on outcomes

and impact. Since then the nonprofit sector has seen tremendous growth overall with

approximately $300 billion raised in 2012 alone. So with that growth came the need for better

analytics, better oversight and better reporting. The reduced availability of resources that resulted

from the economic recession put a microscope on outcomes and whether or not funds were being

used effectively. The demand for nonprofits to develop a measurement-driven system increased.

But the financial and educational support for establishing measurement systems did not

match the demand. This has left many nonprofits, large and small, without the wherewithal to

produce the metrics now required by granters and donors. A divide has formed between those

who advocate for performance and impact measurement and those who do not. On the one side,

government agencies press nonprofits to produce more with less, and donors who made their

fortunes in business expect results expressed in terms of a bottom line. On the other side there is a

diverse group of nonprofits including but not limited to those that support the arts and those that

address massive and intractable issues of human concern. They stand in opposition because it is

hard for them to quantify short-term results and difficult to qualify ambiguous outcomes.


I chose this subject because it is passionately debated and I wanted to have a better

understanding of how measurements have evolved and how they will impact the future of the

nonprofit sector. What follows is an unbiased overview of the current landscape that attempts to

bring into focus the principle issues and concludes with some recommendations drawn from the

research. This exploration has provided me with a great deal of insight about our industry and it is

my hope that it will serve others as well.


DEDICATION

To Knut, my compass and to Aidan, my light


ACKNOWLEDGEMENTS

Thank you to the following for their generous participation, which made this thesis possible:

Michael Balin, former President, Edna McConnell Clark Foundation

John Bliss, Principal, NonProfit Solutions, LLC

Scott Cargle, Nonprofit Management Consultant

Mirele Goldsmith, PhD, Founder, Green Strides Consulting

William Josephson, former Assistant Attorney General-in-Charge of the Charities Bureau

Brian Kinsella, CTO and Co-Founder, The Virtue Group

Nello McDaniel, Founder & Director, Arts Action Research

Charlie Murphy, Consultant and Special Advisor, Hudson Valley Pattern for Progress

Barron “Buzz” Tenny, former President, Ford Foundation

Rodney Trapp, Director of Institutional Giving, Dance Theatre of Harlem

Michael M. Weinstein, Senior Vice President, Robin Hood Foundation

SPECIAL ACKNOWLEDGEMENTS

Mrs. Naomi Levine, Executive Director, NYU Heyman Center for Philanthropy & Fundraising

Marian Stern, Adjunct Professor, NYU Heyman Center for Philanthropy & Fundraising

Timothy Higdon, Adjunct Professor, NYU Heyman Center for Philanthropy & Fundraising


THESIS STATEMENT

While critics argue that impact measurements for nonprofit organizations are not adequate

for obtaining a comprehensive view of mission outcomes, realigning an organization’s focus toward

performance measurement is not just a trend for nonprofit scholars and bloggers to pontificate

about. It is a significant movement within this industry and it is here for the long haul. The ways in

which sustainable outcomes are realized in philanthropy are an art because of the many nuanced

influences on effectiveness, even though we might like it to be more of a science. Therein lies a

significant challenge. (Karoff 2012) Philanthropy translates as the “love of mankind”. (The Free

Dictionary) The human element of the equation is not one that can or should be reduced to a

metric. In the business sector there are uniform reporting requirements that the philanthropic

sector does not have. It is the Wild, Wild West out there in terms of the diversity of reporting

methodologies with no central place to call home. This paper will present arguments for and

against using outcome measurements in the nonprofit sector and look to see if there is a middle

ground being articulated amidst the diversity of opinion and practice.

A BRIEF HISTORY

Contrary to what some may think, measurements are not new to the sector; they have been

around for 100 years beginning with John D. Rockefeller and Andrew Carnegie who instituted the

“scientific method” of philanthropy which has grown to become what some now term “the

effectiveness movement”. (Karoff 2012)

The establishment of more formalized measurements began primarily in health and

human services. In 1960’s and 70’s guidelines such as Standards of Accounting and Financial

Reporting for Voluntary Health and Welfare Organizations (National Health Council and


National Assembly for Social Policy and Development 1964) and Accounting and Financial

Reporting (United Way of America 1974) were developed to determine how funds were being

spent. Later in the 70’s the focus on measurements expanded to include what the funds generated

and this was determined within two categories: products delivered and people served.

The late 70’s brought more standards of measurement in terms of quality of service

delivery. There were concerns with issues such as staff qualifications, staff-to-client ratios, specific

service delivery practices, record keeping, confidentiality protections and condition of facilities.

These concerns led to the establishment of accreditation and certification groups such as The

Council on Accreditation of Services for Families and Children.

The 1980’s brought additional concerns from funders that nonprofits were not serving

those who were deemed the neediest. Participant-related measures were established for the

collection and reporting of data such as demographic characteristics. Public accounting firms began

employing key performance indicators for inputs, services, outputs and total costs. Later in the 80’s

participant satisfaction measurements were added to track quality assurance.

By the 1990’s these measurement standards had succeeded in collecting critical

information about the services nonprofits were providing, but what was still lacking was something

to indicate whether or not the participants were better off as a result of the services. This

encouraged the development of outcome measurements to better understand what kind of impact

nonprofits are having, shifting the focus from activities to results.

Philanthropies and the Demand for New Practices

By the end of 2010 there were more than 1.6 million nonprofits registered with the United

States Internal Revenue Service, an increase from a base in the year 2000 of about 1.3 million.


(National Center for Charitable Statistics 2012) While the wealth of the nation increased during

that time, the common wisdom was that nonprofits were in danger of overwhelming available

funding sources. It seemed that there were more and more organizations chasing fewer dollars.

This drove the more energetic organizations to look for ways to present a more efficient and

trustworthy profile. To project efficiency, they looked to their donor base. Much of the funding

they sought would come from people who were successful in business, and it was felt that efficiency

would communicate seriousness of purpose to this demographic. As for trust, a series of highly

publicized scandals over the course of twenty years or more had tarnished the industry in the eyes

of the public.

Two of the highest profile scandals involved two of the most familiar names in American

philanthropy, the United Way of America and the American Red Cross. In the early 1990’s it

came to light that the long-standing president of United Way had embezzled funds to woo and win

a young woman forty-three years his junior. The fact that his actions came to light through the

testimony of one of his mistresses, all of whom were employed by the organization, didn’t make

the revelation sit any better with the public and their elected representatives. The sense that

corruption could invade the philanthropic sector caught on with the public, some of whom were

only too happy to feel vindicated having previously felt coerced by the management in their

workplaces to contribute to the United Way while it in turn wined and dined their senior

management. The scandal involving the American Red Cross was perhaps worse in the eye of the

public, since it involved money collected for victims of the 9/11 attack on the World Trade

Center. Dependent as it is on hundreds of local chapters to collect donations and provide services,

the reputation of the Red Cross is at the mercy of hundreds of local managers. When the Red

Cross headquarters moved 9/11 funds to an account that the local chapters could not access, the


rebellion drew public attention, causing it to be revealed that the financial management practiced

by the Red Cross was extremely ineffective. Funds collected locally for a national cause were not

always being forwarded to Red Cross headquarters. Funds that were forwarded were placed into a

single account that the local chapters could withdraw from largely at their own discretion. And the

accounting for these funds at the local as well as the national level could not be reconciled.

(Broeckling 2010)

It seemed obvious to many that the fault should be laid at the feet of an industry that lacked

oversight and willfully avoided transparency (Broeckling 2010). To regain public trust, many of the

larger foundations began looking for some way to separate themselves from the past and associate

themselves with a better future. It so happened that there was a model they could adopt from a

source very close at hand, their donors. Two industries had exploded in size beginning at almost

the same time. They were the financial services industry and the electronics industry, the latter led

largely by the worldwide adoption of the personal computer. The financial services revolution

began first when toward the end of October 1979 the Federal Reserve Bank of the United States

officially stopped managing the economy by setting interest rates. No longer constrained, interest

rates could now move freely based on supply of and demand for interest-bearing securities, and

with that there sprang up a bond market. Over a very short time, more and more people were

attracted to the banking and financial service industries as a career choice and many of them went

on to make significant amounts of money, enough money to persuade some of them to become

involved in charitable giving and philanthropy. But their donations came with certain expectations.

For one thing, they were all trained in accounting, having been required to study it for a year or

more to qualify for the degrees that were the gateway to the jobs that brought them their wealth.

They also worked in organizations that were subject to oversight by one federal agency or another.


They were not only used to the careful oversight of money, they expected it, and that meant that

they expected it of the nonprofit philanthropic organization that they chose to engage.

The electronic revolution got traction only a couple of years later in 1983 when the first

IBM personal computer went on sale. In the frenzy that followed, new computer makers

proliferated, as did manufacturers of peripherals from printers to special desks for working at your

computer, and the developers and publishers of software to run on the new computers. Skilled

people were drawn to the myriad of new opportunities this represented and many came away

wealthy. Many of them turned around and reinvested their money in the industry, either in their

own startups or in the startups of others. This was the entrepreneurial spirit that became identified

with the period, a time when the venture capitalist was transformed from a small-time risk taker to

a super hero of the financial world and an example to be followed.

These were the people that the nonprofits now wanted to attract, and they realized that they

were more likely to attract them if they were to behave in ways that were familiar to and approved

by these newly wealthy people. It meant they would have to start accounting for the outcomes

produced by their grants, or as their new funders would put it, the return on their investments.

People in the financial services industry were used to marking positions to market every day and

never going home until the books were closed. Owners of technology startups were used to

developing business cases to sell their ideas to investors, and if successful to have the investors

appear regularly for meetings, sometimes weekly, to ask about the use of their funds, the results

achieved, and progress to the established goals. Should the entrepreneur fail in any regard, the

investors might take over the business and leave the entrepreneur unemployed. For nonprofits,

this was not how they had performed in the past and there was more than a learning curve

required, investment was required.


There would have to be changes made to the management structures of the nonprofits

adapting to this new model. There would have to be investment made to track the information

needed to produce the reports that these new funders would require. New structures, new skills,

and new costs were required, all in the name of a new practice: performance and impact

measurement. It seemed to be a brave new world with the potential of rewards for those that

adopted measurement systems. But an obstacle raised itself early on. Nonprofits didn’t function in

the financial services sector and they didn’t participate in the technology sector. For those who did

decide to bring new reporting processes to their organization, the realization soon dawned that they

were not just adopting a new way of functioning they were inventing a new way of functioning

because, after all, for a nonprofit, there were no standard practices for performance and impact

measurement.

MEASUREMENT AND EVALUATION

This paper addresses various attitudes about and approaches to the measurement and

reporting of impacts and outcomes in the nonprofit sector. Measurement in this context is

sometimes referred to as impact assessment and is defined by the International Association for

Impact Assessment.

“Social impact assessment includes the processes of analyzing, monitoring and managing

the intended and unintended social consequences, both positive and negative, of planned

interventions (policies, programs, plans, projects) and any social change processes invoked

by those interventions. Its primary purpose is to bring about a more sustainable and

equitable biophysical and human environment."


To be clear, there is a very real difference between measurement and another term,

performance evaluation, though the two are sometimes used interchangeably. “Conceptually,

evaluation and performance measurement are still being conflated in the minds of nonprofit

boards and managers.” (Saul 2003) Put simply, the measurement of outcomes is an ongoing

management task that tracks the progress of a nonprofit organization toward its goals.

Measurement reports come out at regular intervals, whether annually, quarterly or monthly, and

because of the regularity of production and the consistency of reporting, the results are comparable

from period to period.

The United States Government Accountability Office’s (GOA) 2011 report titled

Performance Measurement and Evaluation: Definitions and Relationships, explains it this way.

“Performance measurement is the ongoing monitoring and reporting of program

accomplishments, particularly progress toward pre-established goals. It is typically conducted by

program or agency management. Performance measures may address the type or level of program

activities conducted (process), the direct products and services delivered by a program (outputs), or

the results of those products and services (outcomes). A “program” may be any activity, project,

function, or policy that has an identifiable purpose or set of objectives. Program evaluations are

individual systematic studies conducted periodically or on an ad hoc basis to assess how well a

program is working. They are often conducted by experts external to the program, either inside or

outside the agency, as well as by program managers. A program evaluation typically examines

achievement of program objectives in the context of other aspects of program performance or in

the context in which it occurs.” (Kingsbury 2011)

This GOA report then goes on to explain the relationship between performance

measurement and program evaluation in terms of differences in focus and use. Different focus


“Performance measurement focuses on whether a program has achieved its objectives, expressed

as measurable performance standards. Program evaluations typically examine a broader range of

information on program performance and its context than is feasible to monitor on an ongoing

basis.” Different use “Both forms of assessment aim to support resource allocation and other

policy decisions to improve service delivery and program effectiveness. But performance

measurement, because of its ongoing nature, can serve as an early warning system to management

and as a vehicle for improving accountability to the public. A program evaluation’s typically more

in-depth examination of program performance and context allows for an overall assessment of

whether the program works and identification of adjustments that may improve its results.”

(Kingsbury 2011)

The debate within the nonprofit industry is that the preference for evaluation has skewed

the field , creating a kind of tunnel vision that focuses on “proving whether a program or initiative

works, rather than on improving programs” (W.K. Kellogg Foundation 2004). Instead, some

practitioners advocate using “performance data as business intelligence in order to systematically

improve results.” (Saul 2003)

Understanding the similarities and differences between measurement and evaluation can

help nonprofits to decide which performance methodologies to apply. They can be selective,

focusing implementation to benchmarks and logic models that will futher the mission and avoid

methods that conceal qualitative results under quantitative excess. By defining a suitable reporting

strategy and following through, they can showcase the impact, accountability, and sustainability of

their programs and advance their relationships with donors and funders alike.


CURRENT CHALLENGES

Access to Management Information

Many nonprofit professionals, board members, and donors are not familiar with

measurements and how to use them. As discussed in the preceding section, many organizations

have not grasped the distinction between impact assessment and performance evaluation. “Despite

all the right intentions, the vast majorities of nonprofits do not have the benefit of good

information and tools to determine where they’re headed, chart a logical course, and course-

correct when they’re off….Only a fortunate few have a reliable way to know whether they’re doing

meaningful, measurable good for those they serve.” (Morino 2011) Without access to the

appropriate management information, some nonprofit organizations basically do without.

Public Perception

Corruption Some in the industry feel that the push for outcome measurements is a judgment cast

upon the sector due to the actions of a few bad apples. New York’s Governor Cuomo has weighed

in on the topic. “Not-for-profits that provide services to the poor and the needy have a special

obligation to the taxpayers that support them. Executives at these not-for-profits should be using

the taxpayer dollars they receive to help New Yorkers, not to line their own pockets.” (Cuomo

2011) Clearly, corruption is not exclusive to any one industry. But the prevailing opinion seems to

be that the nonprofit sector should be held to a higher standard than the for-profit and government

sectors. Perhaps it is because of the special tax treatment afforded nonprofits, or maybe it is the

thought that good deeds should be reward enough in themselves. Either way, impact and

performance measurement is now one of the tools by which some in our society wish to enforce

that standard.


Charity Watchdogs There are a number of charity watchdog organizations that proactively

examine nonprofit organizations, applying their own standards. Some examples are Guidestar.org,

the Better Business Bureau, Charity Watch, GreatNonprofits.org and GiveWell.org. Though

some philanthropists consider their methods controversial, they are making a significant impact on

how the public perceives the level of financial health, accountability and transparency a nonprofit

organization is achieving. Charity Navigator, which was established in 2002 by businessman John

Dugan, is one of the largest online sources for evaluating nonprofit groups. (Strom 2010) In

response to the growing demand for measurements, in January 2013 they released Charity

Navigator 3.0: The Third Dimension of Intelligent Giving. This expanded version of their rating

system includes what they term Results Reporting. “Charity Navigator 3.0 will include a dimension

focused on the quality of reporting of results that charities provide to the public. We believe this

will highlight those charities that are high performing and results oriented, as well as encouraging

other charities to become so.” (Charity Navigator 2013) Their plans to implement this new

evaluation category are as follows:

For the next several years, our professional analysts will be researching the applicability of this tool to each of the 34 charity “cause areas” that we evaluate, making appropriate modifications as needed, and applying the criteria to the 10,000 charities we plan to rate as of 2016.

To give donors immediate access to our research during this process, we will post our findings on each charity’s page month by month, as the new data is gathered.

However, this new analysis will not impact any charity’s star rating until we have gathered the data for all 10,000. Given the complexity and variety of charities we are evaluating, we will need this time to continue our research on each of the 34 cause areas to determine how all 5 rating elements apply and what modifications are needed for some cause areas. We also need this time to compile an ample amount of data to determine the appropriate weighting of this information in our rating system. (Charity Navigator 2013)

http://www.charitynavigator.org/index.cfm?bay=content.view&cpid=34


If in fact charity watchdog organizations end up directing donors and funders to certain

organizations and eventually away from others, they then become arbiters in the nonprofit sector

wielding a power based on measurements that are not universally calibrated. Besides, each self-

appointed participant in the oversight space asserts its own standards, methods and point of view.

In the end it is not clear who is watching the watchdogs.

The Double Standard A recent TED Talks video, The Way We Think about Charity is Dead

Wrong, was posted sometime during March 2013 and has gone viral. (1,579,422 views as of May

3, 2013) It features a speech given by activist and fundraiser Dan Pallotta whose goal is to

transform the way society thinks about charity and giving and change. In it he says that “everything

the donating public has been taught about giving is dysfunctional”. He calls out the “double

standard that drives our broken relationship to charities”, the notion that the business sector is

encouraged and then rewarded for spending as much as is needed, no matter the amount, to

achieve the highest level of profit possible. While on the other hand, too many nonprofits are

rewarded only for how “little they spend -- not for what they get done”. The public, he feels, is

confusing “frugality with morality”. There are essentially “two rulebooks”, one applied to for-profit

organizations and the other to nonprofits, and this he claims has kept the nonprofit industry at a

standstill for more than 40 years, limiting its contribution to the GDP and denying it the resources

needed if the industry is to be recognized for the serious role it plays in changing the world.

(Pallotta 2013)

We’ve all been taught that charities should spend as little as possible on overhead things like fundraising under the theory that, well, the less money you spend on fundraising, the more money there is available for the cause. Well, that’s true if it’s a depressing world in which (the) pie cannot be made any bigger. But if it’s a logical world in which investment in fundraising actually raises more funds and makes the pie bigger, then we have it precisely backwards, and we should be investing more money, not less, in fundraising, because fundraising is the one thing that has the potential to multiply the amount of money available for the cause that we care about so deeply. (Pallotta 2013)


Donations as a Means to an End

Because overhead, that now has to budget for assessment, is not a priority for funders to

support, the majority of nonprofits have to divert funding from other areas in order to produce

information on performance and the impact of programs. For an organization constrained in terms

of the size and makeup of its donor base, the funds would have to come from existing donations,

draining resources and curtailing the organization’s ability to support its mission. “There is no

uniformly accepted way to measure social impact, and no single repository for information about

nonprofit activities and results. This information-poor environment makes it difficult to have

honest conversations about performance, limiting opportunities for learning and improvement.”

(O'Flanagan, Harold and Best 2008) In comparison to high-performing, high-participation markets

such as the stock exchanges, commodity markets, or Amazon.com, the nonprofit marketplace is at

a disadvantage. Unlike the financial and consumer markets, the nonprofit’s commitment is not

directed to market participants who invest or spend money, but to people whose welfare the

organization is dedicated to support. Donors are the equivalent of investors in the other markets

and are therefore expected to share the same focus as the nonprofit organization. To confuse or

conflate the for-profits with nonprofits in this respect is to do a disservice to the nonprofit for

which the money is a means to an end and not an end in itself. It is not clear how many donors

would be content to think that a substantial portion of their gift would be spent on in-depth

quantitative analysis and reporting when a qualitative accounting would be sufficient.

Still, there is a gap between the nonprofit and the for-profit when it comes to accounting for

funds. In order to bridge this gap, the sector needs to find a more efficient way to capture, analyze,

distribute, and use information on social impact and organizational performance instead of hoping


to impress by adopting goals designed to be easily analyzed and reported on. (O'Flanagan, Harold

and Best 2008)

Choosing the Right Methodology

Currently there is a mix of methodologies, analytical models, and industry consultants, each

claiming to offer an advantage in supporting performance measurement. But there are pitfalls

awaiting organizations that feel the need to adopt a solution in a box. A reporting method that

doesn’t complement the organization, its structure and its mission will produce information that is

likely to miss the point at best and divert the organization from its mission at worst. Either way, the

effort will cost resources that might be better spent. “In today's climate of scarce economic

resources, the pressure for nonprofits to show quantifiable results is greater than ever; as a result,

an organization without a strong sense of strategic direction and the internal data to understand its

own strengths and weaknesses can be overly influenced by outside demands for metrics that may

not always be relevant to its ultimate success.” (Hanna 2011)

Ideally, impact assessments and performance measurements should support comparisons

between reporting organizations, contrasting differences in efficiency and operational competence,

and highlighting effectiveness, but within the context of the challenges faced by an organization

given its mission and goals. “A basic problem that plagues philanthropic decisions is that there is

no natural yardstick by which to measure, and therefore compare, different philanthropic

outcomes.” (Weinstein and Bradburd 2013) For Michael Weinstein, an economist and Senior

Vice President at the Robin Hood Foundation, counterfactuals are the most important thing they

do. “Counterfactuals” is a contraction of a philosophical term, “counterfactual theories of

causation”. As explained in the Standford Encyclopedia of Philosophy, “The basic idea of

counterfactual theories of causation is that the meaning of causal claims can be explained in terms


of counterfactual conditionals of the form “If A had not occurred, C would not have occurred”.

(Menzies 2009) Counterfactuals make up one part of the Foundation’s continual effort to develop

more effective ways of reporting on every benefit related to their mission. (Weinstein, Senior Vice

President and Chief Program Officer 2013) In finance terms, counterfactuals are very much like

the practice of calculating incremental ROI, or the value of the next dollar spent, and it suits the

Robin Hood Foundation and the high-net-worth donors it works with.

A funder gives money to a job training group, ten of the trainees get jobs. So how much good did the funder do? How much good did that training program do? If you answer ten, you’re measuring it in a very different way from the way Robin Hood goes about its business. Because we need to ask, how many of those trainees would have gotten jobs even if they had never entered the program. And that’s what is called a counterfactual. How many of these trainees would have gotten jobs without our help? And that’s what Robin Hood’s staff spends a lot of time trying to estimate…We fund programs that cater to New Yorkers who we think are going to fail without our help.” (M. Weinstein)

The Foundation uses other reporting methods as well. Along with his co-author Ralph M.

Bradburd, Mr. Weinstein recently published a book, The Robin Hood Rules for Smart Giving.

In it he describes the framework he developed for Robin Hood called Relentless Monetization

(RM). The Foundation defines “monetization” as the assignment of dollar values to philanthropic

outcomes, and “relentless” in this context means making assignments even when the benefits

associated with those outcomes are hard to measure and the evidentiary basis for assigning dollar

values to specific outcomes is slim. (Weinstein and Bradburd, The Robin Hood Rules for Smart

Giving 2013) Relentless Monetization methodically applies the workhorse of modern economics,

benefit/cost analysis, to the task of making effective philanthropic decisions.

The power of Relentless Monetization (RM) lies in its consistent and persistent application of

benefit/cost analysis. Implemented carefully, the strategy takes full account of the funder’s philanthropic mission, the preferences and values of nonprofit actors (funders, donors, policy makers, academics, service deliverers) and resources. RM also takes full account of the best available evidence about the impact of philanthropic interventions on the outcomes that are relevant to the missions of funders and donors. And the strategy does all this in a manner that leaves a tangible trail of accountability, thereby exposing philanthropic decisions to challenge and revision. (Weinstein and Bradburd, The Robin Hood Rules for Smart Giving 2013)


This type of in-depth, multi-employee strategy takes an enormous amount of time and

dedicated resources. The Robin Hood Foundation is fortunate to have a board of directors whose

members are successful hedge fund managers and celebrities who supply 100% of the funds

needed for general operating costs. That allows the Foundation to apply 100% of its donor

proceeds directly to the cause of fighting poverty in New York City. But not every organization is

funded like the Robin Hood Foundation. For the rest, challenges remain. Not the least of which is

a shifting consensus within the measurement movement; the strategies favored for measuring

impact keep changing as the market turns from less productive methodologies to new ones. The

Robin Hood Foundation, given its mission and its donor base, can invest in researching and

developing the reporting models that best suit their needs. This is not something that is available to

the majority of organizations in the industry.

Is the Demand for Data Limiting Innovation?

Funders are demanding more and more data. The common frustration among nonprofits is

that even though funders demand more data, they are unwilling to fund or supply the resources

needed to collect that data. William Josephson, retired partner of the New York City law firm

Fried, Frank, Harris, Shriver & Jacobson LLP, and Assistant Attorney General-in-Charge of the

Charities Bureau under former Attorney General Eliot Spitzer, feels that the demands for

measurements have damaged philanthropy’s innovative purpose in such a way that it may no

longer be rectifiable. “Reliable social science research is very expensive and often inconclusive.

Such research also often takes a long time to yield any results. Foundations have not done badly

without such research in the past. Demanding it now, particularly of start-ups, will diminish

innovation.” However, a long-time friend and colleague of Mr. Josephson, Michael Bailin, does

not agree with Mr. Josephson’s criticism. Mr. Bailin is the former President of the Edna


McConnell Clark Foundation and during his time there was responsible for a strategic initiative

that moved the foundation away from traditional foundation practices to a performance-based

approach. In an interview, Mr. Bailin argued that when measurements are designed with the right

focus, they are an invaluable tool for producing “just communications” by which he means

substantive, mission-based reporting on foundation performance. (Bailin 2013) “The approach we

have chosen arose because it addressed a problem that had been plaguing us for some time: We

were committed to changing huge systems with small resources, we believed we were highly likely

to fail at that mission, and we needed a different way of looking at our goals and our relationship

with grantees.” (Bailin, Philanthropy in Practice: Great Expectations Versus Getting the Job Done

2004) In other words, if performance-based reporting will indeed force change within an

organization, use it, and let it direct change to the good.

For all the benefits ascribed to impact assessments and performance measurements, the

demand for data remains suspect even among some funders. Abigail E. Disney, founder and

president of the Daphne Foundation, who supports community-based groups in New York, among

other causes, warned donors against allowing their focus on metrics and measurement to prevent

them from supporting nonprofit leaders who are tackling society’s most intractable issues. “I

wonder if Martin Luther King Jr. showed up at my door today, if any of us would fund him,” she

said. “I think we would have kicked him to the curb in such a hurry.” (Preston, Philanthropy Must

Do More to Influence Policy, Say Government Officials 2013)

Performance Measurements & Risk Management

Lehn M. Benjamin is an assistant professor with the Nonprofit Management Studies Program

in the Department of Public and International Affairs at George Mason University. Her research


examines the intersection between nonprofit organizations, accountability, and democratic

practice. Ms. Benjamin wrote an article advocating that risk management deserves more direct

attention in discussions about nonprofit accountability generally and performance accountability

specifically. The article is based upon a study of performance measurement frameworks in

nonprofit relational work. It concludes that choosing the appropriate method of reporting is

important because an inappropriate reporting method can work at cross purposes with the needs

of the beneficiaries. The organizations in the study raise funds that they in turn distribute to

neighborhood organizations dedicated to community development. The study employs risk

analysis to demonstrate that an inappropriate stress on efficiencies can lead to unintended

consequences “Performance accountability systems require nonprofits to bear more risk for

achieving results. Although a growing body of work has examined nonprofit accountability, less

attention has been given to the concept of risk. (There is) a potential conflict between performance

accountability frameworks and nonprofit work. This conflict can be best understood as one

between managing risk in task-driven relationships, in which relationships are formed simply to

achieve desirable results, and managing risk in developmentally driven relationships, in which

performing a task is intended not only to achieve desirable results but also to build enduring

capacity to take action on common problems.” (Benjamin 2008)

Relationships are the backbone of nonprofit work and vital to long-term sustainability.

Preserving relationships among organizations and with beneficiaries often requires trust, especially

when beneficiaries feel socially marginalized because they may have learned to become distrustful

of anyone promoting change, no matter how constructive. “Public administration scholars have

given much attention to the advantages and disadvantages of ensuring accountability through

hierarchy or by incentives coupled with competition, but perhaps nonprofit organizations require

an accountability approach that recognizes the distinct work they do in building relationships.”


(Benjamin 2008) The accountability model adopted in the study linked levels of funding to the

achievement of certain goals. But threatening to reduce levels of funding if goals were not met,

proved counterproductive. The unintended result was the transfer of risk from the funding

organization to the more dependent and less resilient community organizations, and a reversal of

the traditional responsibilities of the benefactor and the beneficiary.

Leadership: Are Nonprofits Too Nice?

When a nonprofit is faced with the dual challenges of first establishing and then managing

performance and impact measures, the net effect on the culture of an organization that is not

completely committed to this new approach can threaten the success of the measurement plan. As

with all oversight and management, the responsibility for creating and sustaining an effective

performance-driven culture lies with the leadership. “Without a leader, who is committed to

measurement as a top priority, articulates how it enhances impact, and identifies someone within

the organization to lead the charge, organizations will not overcome the natural reluctance among

staff to embrace what seems like such an overwhelming enterprise.” (Forti 2012)

There is a sense that because the nonprofit industry is sacrificing for the common good, it is

unfair to criticize them. Unlike business, with which it is often compared when speaking of

performance and reporting, the nonprofit sector is an area that many choose to be involved in

because the emphasis is on creating a better world, as opposed to accumulating wealth. To some,

criticizing a philanthropic organization by saying it has not achieved its mission effectively is like

judging a nun for not being holy enough. But because of the economic challenges that continue to

exert pressure on funding, many nonprofit professionals feel that without honest conversations

about performance and impact, the industry overall is likely to be diminished in its effectiveness

and blamed for unwillingness to address issues of leadership, board governance and self-regulation.


Vignetta Charles, senior vice president of AIDS United made this point recently in the Chronicle

of Philanthropy. “It’s harder for us to talk about this in philanthropy than in the corporate world

because we’ve had this genteel attitude. We’ve been too nice to each other for too long,” she says.

“It’s especially timely for us to think through the ways we communicate, how transparent we are,

and who needs to potentially fade gracefully into the night.” (Preston, Some Nonprofit Leaders

Ask: Is Philanthropy Killing Itself with Kindness? 2013)

In his book, Leap of Reason: Managing to Outcomes in an Era of Scarcity, Mario Morino

makes a similar point:

In my business life we once brought in a speaker to inspire our team and get everyone on the same page. He gave great examples of getting folks involved and buying into mission, the normal song and dance of inspirational speakers. But he wrapped up the session with a pithy statement that is indelibly etched into my memory: “Catch the vision or catch the bus!” Harsh? For sure, and it’s unlikely that you’ll use it at your next all-hands meeting. On point? Very much so. (Morino 2011)

Morino believes that what commonly prompts a shift in favor of metrics is the recognition that

while measurements might be necessary to report results to funders, the greatest power lies in using

them as a tool to improve the nonprofit’s ability to fulfill its mission. But without internal buy-in,

there may not be external acceptance. Identifying what needs to be measured and how to measure

it requires education, internal commitment, financial support, planning, and openness to

experimentation and change. Inaugurating the program and obtaining the commitment of staff and

the support of funders takes leadership.

Re-Aligning the Culture of the Industry

If the natural evolution of the nonprofit sector will lead to universal acceptance of impact

assessment and performance measurement, it is not to be without a struggle. “There’s an ongoing,

pitched battle for the soul of philanthropy.” (Preston, Some Nonprofit Leaders Ask: Is


Philanthropy Killing Itself with Kindness? 2013) And if it is a battle, how will it end? Battles

generally end when someone wins and someone loses, wherein lie the fears of some who oppose

reporting. For others, there is a less traumatic and more pragmatic question, do performance and

impact measurements really lead to success? Which is to say, will they keep an organization

focused on its mission, will they add value by helping an organization function more efficiently, and

will they help to attract and retain a loyal base of donors and funders? As yet, there is no sure

answer to these questions. Evaluator of winners and losers, effective management tool or strain on

resources, impact assessments and performance measurements do represent one thing, though:

change. And if nonprofits and the nonprofit sector do not evolve to accept and embrace this

change, the concern is that they may be forced to instead.

Some nonprofits, those who apply for grants from certain funders or who contract with

government agencies, must already submit to reporting requirements that they would not otherwise

adopt by themselves. But the others, who are the majority, have and do perform their roles and

meet expectations without allocating the resources that reporting would require of them. To do

otherwise might require significant changes in function, and perhaps changes in mission. To do

otherwise might require significant changes in organizational culture. Realigning the culture of an

organization is a tough, uphill climb requiring change that begins at the top, but what we are talking

about probably requires a re-alignment of the culture of an entire industry, and that is an even

more formidable task. Even now, success does not come easily. “What we want when we donate

money is to feel good,” Susan Davis, President of BRAC USA says. “It’s much harder to engage

the public in something more complicated, but the truth is, as our world has become more

interconnected, people are becoming more sophisticated, and what we’re trying to do is invite

people on a learning journey.” (Preston, Some Nonprofit Leaders Ask: Is Philanthropy Killing


Itself with Kindness? 2013) Few will willingly give up the methods that have served them in the past

when there is no assurance of a benefit for the organization or its beneficiaries to do otherwise.

ARGUMENTS FOR MEASUREMENT

According to a 2010 paper by Alnoor Ebrahim and V. Kasturi Rangan of the Social Enterprise

Initiative at the Harvard Business School, the debates on measuring outcomes and impacts are

playing out in three arenas:

Private foundations aiming to be more strategic about their philanthropy

U.S. nonprofit organizations in response to pressures from foundations and government

International development organizations such as bilateral government agencies and

nongovernmental organizations (NGOs) seeking to improve development effectiveness

Economic conditions play a part as well. “The pressures to demonstrate impact are likely to

increase across all of these players in times of economic crisis, as public and private resources

diminish and as competition for existing resources heightens.” (Ebrahim and Rangan 2010) This is

not to suggest that the impulse to adopt performance and impact measurements is simply a

reaction to the worldwide recession that began in 2008 with the collapse of financial markets

following the bursting of the housing bubble.

For one thing, the arguments in support of performance and impact measurements for

nonprofits are well thought out, and bear more of the weight of logic than one would expect from a

faddish reaction to economic stress. Instead, the authors foresee an imminent risk to the sector as

a whole. A risk that if the industry is unable to acknowledge and embrace the importance of

measuring and communicating mission impact, it may see an erosion of public trust and eventual

marginalization as a functioning sector of the economy.


Accountability and Transparency

At its core, the demonstration of accountability in a market participant is a way to establish

trust. For the nonprofit sector of the market, trust plays a more significant role in achieving success

than in business or government since the customers or investors, represented by the donor base

are in some cases not direct recipients of the benefits produced by the organization. Even though

there are situations where a donor is the recipient of benefits, as in hospitals, universities, religious

institutions, to name a few, often the recipients of the benefits are separate from the donor,

separate by location, economic stratum, physical location, and any number of other possible

differences that make it unlikely that the donor will otherwise be able to support them directly.

Accountability can provide the proof needed by the donor to ensure that the donor’s wishes were

fulfilled by the nonprofit organization as intermediary and thus deserve the trust put in the

organization by the donor.

Beyond the nonprofit’s donor base, the public at large expects accountability because

nonprofits are by definition of the IRS tax exempt organizations. It is understood that a tax exempt

status means that every tax-exempt organization is in fact being supported indirectly by the

taxpayers of the nation. Furthermore the tax deductibility of donations to tax-exempt organizations

represents foregone government revenue and further implies that the donors to these organizations

are being supported by the taxpayer as well. When considered in this light, it is not a stretch to see

why people can conclude that nonprofits are working with the public’s money, no matter what the

source of their funding is, whether public or private. “The broader definition of accountability,

including the requirement to produce results, also has become more pronounced with the growing

impact of donors who view their giving as social investment and demand specific evidence of the

impact of their support.” (Worth 2012)


Form 990 The recently revised Form 990 has allowed for more information to be

accessible about the work of nonprofit organizations including specifics concerning governance

practices. Even more recently, President Obama released a budget asking legislators to pass a law

mandating electronic filing for the Form 990. (McRay, White House Advocates for Mandatory

Form 990 E-File 2013) There has been a call for the IRS to disclose more data from Form 990’s

and to do so in a way that makes it easier to search. Tom Pollak, program director of the National

Center of Charitable Statistics at Urban Institute remarked, “We really appreciate the IRS taking

this major step forward to make a lot of data available on a timely basis, and we are very hopeful

that in future years the IRS will be able to expand on what they include in the file.” (McRay 2013)

But the purpose and structure of the Form 990 in its current incarnation has limited value for

measuring organizational performance, and the information provided is not adequate for readers

to have the ability to conceptualize the data. Yet the Form 990 is the only publicly available source

of data for all nonprofits. “In the absence of better information, individual donors, foundations,

advisors, and the news media tend to fixate on operating expenses and fundraising ratios or reports

of financial abuses or scandals at unrelated nonprofits to form their perceptions of nonprofit fiscal

responsibility.” (McKinsey & Company 2008) In a 2005 final report to Congress and the nonprofit

sector, the Panel on the Nonprofit Sector concluded that “The (Form 990 is) not useful as a tool to

communicate complex information about program goals, accomplishments, failures, and changes

that have affected an organization’s overall performance or the performance of a particular

program. Each organization is therefore encouraged, as a recommended practice, to share more

detailed information about its programs through an annual report or other appropriate document

that is available to the public on the same basis as its Form 990. Organizations are also encouraged

to post such information on their websites.” (Strengthening Transparency, Governance, and

Accountability of Charitable Organizations 2005)


If prepared properly, performance and impact measurements can play a significant role in

filling the information gaps in Form 990. They could provide more transparent and mission driven

communications for public viewing that will simultaneously create a road map for accountability

and efficiency. If nonprofits prove unwilling to take this step on their own, it is safe to assume that

given public sentiment, the character and content of Form 990 will change. It may behoove the

nonprofit to ensure that the information they now provide is sufficient to satisfy the burgeoning

watchdog groups, public and private, that will be analyzing and reporting on this information.

Competitive Advantage for Funding

As the nonprofit sector has grown, so has its need for funding. In the not so distant past, a

nonprofit in pursuit of support spent most of its time crafting the perfect “elevator pitch” that

would capture the spirit and essence of its mission as well as the eyes of donors and grantmakers.

Now more than ever, with the state of the economy and the vast numbers of nonprofits in business

today, many organizations whose missions barely differentiate them from other similar

organizations, need to provide data that will allow a nonprofit to obtain singular recognition and

make a positive impression on funders’ minds. “As needs increase and resources dwindle, the

community organization which can demonstrate impact of resources more substantially than an

ally (or competitor?) organization may end up with more resources than those unable to show a

direct impact.” (Boland 2012)

More Effective Marketing and Social Media Strategies

It is believed that communication of successful performance measurements can serve as a

powerful marketing tool. According to the Stanford Social Innovation Review, measurement and

transparency is the number one technology trend to watch in 2013. “What gets measured gets


improved…Nonprofits should not only capture data about their own performance for reflection

internally, but also consider how sharing that data with people who use their services can positively

affect their area of work—whether it’s health care, education, energy, or another issue.” (Tobias

2013) Nonprofits are marketing to more than the individual donors and the foundations they

solicit for funding. They are also marketing themselves to their future board members, major

donors and up-and-coming, self-selected, nonprofit professionals whose financial and

organizational training and sophistication will be accompanied by higher expectations of

professionalism and purpose. In this context, nonprofits with successfully implemented

performance measurement systems will be more attractive to the top talent and the wealthiest

donors. Marketing built on the results of performance measurement can also serve as a tool for

educating the public on public policy agendas. Framed within professionally managed marketing

campaigns, measurements of success are made to order for influencing public perceptions as to

those organizations best prepared to handle public issues efficiently and effectively. (Sawhill and

Williamson 2001)

Social media will become more important to a nonprofit’s ability to market. Though

currently not the most reliable resource for communication, social media is none-the-less a

medium to which people turn when they want information. Performance results are the kind of

hard data that can make a difference in this environment. Here an organization can also engage

those who seek them out and not just as for support but to obtain data in service to their mission

goals. “Social media and that whole realm of crowdsourcing are going to be more and more

helpful in getting data.” (Berger 2012) In the crowded nonprofit sector, a presence in social media

that tonally reflects the organization and quantitatively defines the organization has the potential of

creating an advantage for the organization. A presence that can engage and encourage participation


in this venue, could improve the standing of an organization substantially. In the social media

environment, the nonprofit presence that wants to demonstrate professionalism and sophistication

will need to explain itself with hard data in the form of performance data that reinforces the spirit

and substance of its mission.

Wealthy Young Donors Demanding Results

It is no longer so much about the Baby Boomers as it is about their children. The current

projection for the transfer of wealth that is set to begin in this decade is approximately 41 trillion

dollars, the largest transfer of wealth ever. The difference between the Boomers and their kids is

that the kids are more active in the many aspects of their philanthropy and they have begun

making mega gifts much earlier than their parents. That increase in involvement includes higher

expectations for results. “Affluent young donors say they are more focused than their parents and

grandparents on producing a measurable impact with their giving…What’s more, some of them say

they care more about advancing a cause than helping an institution, the reverse of how they see the

philanthropy of their elders.” (Preston 2013) So as a result of this heightened involvement and

demand for performance and impact measurements, the future of this industry will possibly see

some of the most profound philanthropy that it has seen since Rockefeller and Carnegie. “The

next generations of major philanthropists, who fit into “Gen X” (born 1964-1980) or “Gen

Y/Millennial” (born 1981-2000) generational cohorts, will wield more philanthropic power than

any previous generation. With an unprecedented amount of wealth, these donors hold the future

of philanthropy in their hands, yet, until now, there has been little previous research on the

powerful but very private group of young people who stand to become the major donors of the

future.” (21/64 and the Dorothy A. Johnson Center 2013)


Enabling a Performance Oriented Culture

The absence of a standard accountability system within the industry has let nonprofits delay

taking responsibility for establishing best practices in the management of performance and impact.

When asked in an interview, “Where do you stand on the trend that is mandating impact

measurements for nonprofits?” John Bliss, Principal, NonProfit Solutions, LLC and Board

President, Hudson Highlands Nature Museum, Cornwall NY, simply replied, “If you can’t

measure it, you can’t manage it.” (Bliss 2013) For a nonprofit, performance and impact

measurements present opportunities to establish strategic, mission specific goals while creating a

manageable, rational approach to cultivating and sustaining an internal culture of accountability

and purpose. The culture of an organization informs not only practices and attitudes that frame its

internal life, culture is also outward facing. An organization’s culture is a broad concept, but it is

simultaneously an intimate thing. It is disclosed through every transaction, every exchange,

between a representative of the organization with other representatives, and with every outsider

from prospective donor, to funder, to grantee. To reveal the culture of an organization is ultimately

to reveal the best and the worst of the organization. An organization that does not find the

resources to engage in periodic self-evaluation risks losing touch with itself and its mission, and that

is difficult to conceal. It will be communicated. Self-evaluation can and should take many forms,

but performance management in some form is a necessary introspective element, besides being

one of the best forms of communication to the public. These points can be found in the literature.

The U.S. Office of Personnel Management defines performance management as: the

systematic process by which an agency involves its employees, as individuals and members of a

group, in improving organizational effectiveness in the accomplishment of agency mission and

goals. (Walker Ph.D. and Anderson Moore Ph.D. 2011) So understanding that “Missions are


typically better at providing inspiration than direction” (Bradach, Tierney and Stone 2008), it is the

responsibility of the leaders to embrace a method or system that will enable the organization as a

whole to align itself toward a more unified set of standards for reaching the impact their mission

denotes. “The absence of processes for setting employees’ goals and obtaining feedback, for

instance, disconnects individuals and their performance from the organization’s strategy.”

(Bradach, Tierney and Stone 2008)

ARGUMENTS AGAINST MEASUREMENT

In the Bill Moyers PBS series, The Power of Myth, Joseph Campbell recalls the quote by

Carl Jung, “Religion is a defense against a religious experience.” For Campbell, religion is a

bureaucratic impediment to the “experience of deep mystery that one has to regard as the ultimate

religious experience.” (Campbell 1988) Similarly, the process of establishing measurement metrics

for social impact has been blamed for creating an impediment to innovation. There is a fear of

unmanageable bureaucracy undermining the value of individual aspirations, and taking away the

purpose and meaning of the work of nonprofit organizations. “Too much emphasis on measuring

performance could create a “Dilbert world,” in which the passion and commitment of the

professional staff and volunteers are replaced by caution, even skepticism, which might undermine

the nonprofit culture and its traditional strengths.” (Worth 2012)

One Size Does Not Fit All

One of the main criticisms of performance and impact measurements is that nonprofits do

not focus individually on the same or even similar issues. Their methods are not uniform either.

The story was once told about someone putting three flies in one narrow-necked glass bottle and

three bees in another. The bottles were placed on their sides, uncapped, in a position where


daylight shone through the bottom of the bottle. The three bees bumbled and fumbled at the glass

at the bottom of the bottle. They never found their way out. The three flies, so the story goes, kept

zinging around randomly in their bottle until finally each of the three found the open end and

escaped. (Peters and Waterman 1984) Though it is not a flattering comparison, some think of the

nonprofit sector as acting like the flies, facing intractable problems and somehow finding answers,

each in its own way, by working tirelessly and perhaps seemingly randomly, to find a solution, or to

come up with an expedient solution that serves a need, improves a bad situation, or gives someone

an opportunity to get up and try again tomorrow, without trumpeting that the solution has been

found, the problem has been solved, and from now on everything will be better.

Nor is their uniformity in the demand for reporting from the government or foundations

that are the lifeline for many nonprofits that survive on grants. There is no standard for the

grantees to adhere to, nor is there a standard underlying the requests made by the grantees. As any

nonprofit leader can tell you, she may have to report outcomes in as many different ways as the

number of grants she receives from foundations and government, for there is no generally

accepted, uniform way for such outcomes to be reported. (Schambra 2011)

Foundations Demand Metrics, But Will Not Fund Them

It’s a fact of nonprofit life, overhead is rarely funded. Some foundations will offer grants for

administrative purposes, but providing funding for office work is not generally the kind of ambition

that calls someone to philanthropy, and for most people supporting the sector, analysis and

progress reporting is a kind of office work, and not the good kind. The good kind of office work is

performed in a second-story office in a rundown building in a city in a third-world country where

one or two people supported by concerned funders thousands of miles away minister to the needs


of a population for whom some of the most basic needs to support health and wellbeing can only

be supplied from outside.

One of the reasons that this latter kind of office work is supportable is that it requires

relatively little payment of overhead. In the eyes of the public and many funders, overhead is to be

kept to a minimum. Somehow it is in and of itself a bad thing, even if the downside is evident in

the very country where the philanthropic effort is focused, where there is no such thing as public

health since the government there cannot or will not pay for the overhead it represents.

“Investing in an IT system that can track program results is good; paying excessive rent for

opulent office space is bad. Attempts to limit all overhead blur this distinction and severely

undermine nonprofits’ ability to invest in the people and HR processes necessary to deliver great

results year after year.” (Bradach, Teirney, Stone, Delivering on the Promise of Nonprofits,

Harvard Business Review, December, 2008) It comes back to a lack a standards, a diversity of

expectations, and a diversity of effort as each foundation and nonprofit stakes out an area of

concern and goes about addressing the shortfalls that now call for intervention in its own way based

on its own insights and abilities. For the critics of performance measurement, there are many

things that measurement does not do. For one thing, the publishing and distribution of reports

does not in and of itself eliminate the need for the funder to monitor resources, activities, and

outputs. The results gleaned from a report may show spectacular results and unprecedented

achievement, but is the program measuring and reporting on the right outcomes? Can the

explanations provided in the report explain why the program achieved the level of outcome it did?

Is there any proof offered that the program itself caused the observed outcomes, or did a decade-

long drought suddenly end in a monsoon that not only made farming possible again but washed

away the miscreants who had been living up in the hills and terrorizing the inhabitants below. At


base, the critics ask, can essentially ad hoc reports confirm whether or not the outcome being

described is one in which resources should be invested.

Short Timeframes May Diminish Ambition and Idealism

There is an expression used by business managers, “Nobody wants a story.” In other words

don’t tell me why you didn’t succeed; don’t tell me why you don’t know the answer. If you didn’t

succeed, you have failed. If you don’t have the answer, you have failed. This is the attitude, one

most associated with the commercial business world, which worries some critics of measurement

reporting in the nonprofit world. The fear is that in a short-term reporting cycle, where

measurements are expressed only as success or failure, longer-term commitments will end up

looking like failures in the short-term.

It is a keenly felt concern. “Assessments don’t include the role nonprofits play in averting

problems, such as keeping kids from delinquent crimes that land them in jail or providing

preventive health counseling and nutrition services so adults won’t later need costly medical care.

Nonprofits contribute to a strengthening of civil society and democracy, to increasing civic good

and social capital…Unless the full range of nonprofit purposes is included in assessing the value of

all charitable programs, everything but the most fundamental functions will be discounted and

marginalized…we will turn away from the long-term investments…in favor of a narrowly defined

short-term individual payout.” (Rosenman 2013)

Once organizations begin to manage reporting cycles, commitments to efforts affording

long-term results may no longer seem attractive. In his 1973 book, Management: Tasks,

Responsibilities, Practices, Peter Drucker wrote, “It is meaningless to speak of short-range and

long-range plans. There are plans that lead to action today…and there are plans that talk about

action tomorrow-they are dreams…The essence of planning is to make present decisions with


knowledge of their futurity. It is the futurity that determines the time span, and not vice versa.”

(Drucker 1973) This is the modern business approach that began in the 1950’s and has remained

as a part of our commercial culture. But in this case Mr. Drucker is referring to planning for profit.

Can this blanket approach apply to nonprofits? Can a nonprofit be managed the same way as a for-

profit? It is well known that innovation is the pulse that drives nonprofit work. It takes time and

money to produce results. And sometimes those results are that the plan is not working. As

Thomas Edison said, “I have not failed. I’ve just found 10,000 ways that won’t work.” (Edison) In

many cases, failure in the nonprofit world is not necessarily a bad thing. “It is appropriate for

foundations to risk their funds on new ideas/programs, even with the possibility that some of them

will fail, because it ultimately saves taxpayer dollars, which might otherwise be appropriated to the

same measures that don't work.” (Stern 2013)

Openness and readiness to support ideas from outside an organization may be restricted by

a report driven culture. Nonprofits must be open, open to needs, open to ideas, open to change.

They cannot afford to be closed to the outside. They cannot afford to be exclusively inward

looking. Critics see in the imposition of ill-defined reporting requirements the drawing of a blind,

where the only view into the organization will be through the distorted lenses of measurement

reports. Specific promises will have to be made and then evidence provided that they were

achieved. Because of the risk of the consequences of failure that would accompany the inability to

deliver on a projected outcome, the management of the description of results will have to be

paramount. No information will be permitted to circulate that has not been reviewed, filtered and

packaged in a report. According to critics of reporting, what will follow will be a farcical imitation

of philanthropy as we now know and practice it.


Improved Reporting Does Not Improve Outcomes

Critics of reporting say that most programs do not serve enough individuals to affect

community-wide statistics, and so whether the program can be deemed successful or not can’t be

relegated to a report designed to be reviewed by someone removed from the community being

served. They point to the myriad problems that can consort to create conditions that may not be

resolvable in generations: economic conditions, environmental conditions, demographics, public

policies, and the local culture, its norms and expectations. In spite of these daunting facts,

nonprofits have traditionally taken on the task of trying to alleviate misery and offer hope to the

individual, even if the community itself is too large or two despairing to be helped as a whole. In

this kind of environment, refining the nonprofit’s reporting process will not produce any advantage

or improvement of conditions for the community being served.

Impact Measurement Is Misunderstood, Is Not New, and Will Continue to Fail in Its Attempt to

Right All Wrongs

Impact measurement is not a recent trend the critics tell us. “Lost in today’s measurement

mania…is one disturbing fact: This devotion to measurable outcomes is hardly new. Indeed, it is at

least a century old. More to the point, it has itself apparently had so little measurable impact on the

way we do business that a full century later, we’re talking about measuring impact as if we’ve just

discovered the concept”. (Schambra 2011) And not only are there no standards, but we don’t even

understand what we are talking about when we talk of measurements and reporting on impact.

“Many organizations do not yet grasp the all-important distinction between impact assessment and

performance measurement.” (McGill 2011) In other words, why do we keep trying to implement

something that has never worked and that we don’t understand anyway?


Maybe we are trying to be something we are not. “Impact assessment is not the end goal of

foundation and nonprofit work.” (McGill 2011) Or maybe we are not equipped as a sector to use

the reporting tools employed in other sectors. “All those numbers are being gathered without any

way to make meaningful comparisons among them.” (Schambra 2011) But then the numbers we

do gather are suspect after all. “A bare handful of programs—again, this is a century after we began

counting—can claim to have been scientifically validated according to the “gold standard” of

measurement, using randomized control groups.”(Schambra 2011) But then how could we achieve

the necessary scientific validation of the effort after all? We are at a distinct disadvantage. “Social

investment does not take place in a controlled laboratory setting.” (McGill 2011) For that reason,

“measurement error creeps in everywhere.” (McGill 2011) We can only ever hope to come to

grips with the issue of reporting when we are able to have “a thorough discussion about the

measurement challenges in the field of philanthropy in order to be able to talk meaningfully about

the possibility of "social impact assessment." (McGill 2011) And we will know that day has come

when we can “decide jointly on a simple, coherent, user-friendly system to which we can both pay

attention, which will prevail over bureaucratic inertia and political connections, and which will feed

into a serious body of knowledge.”(Schambra 2011)

Back in 2007, a speech given by the former president of the Ford Foundation, Susan

Beresford addressed what she referred to as the recent popular dichotomy of “old” and “new”

philanthropy. She asked "what truly is new"? Is today's new philanthropy "new" and "better" because

it is strategically aimed at root causes, results oriented, global, influenced by the business model

and driven by donor engagement?… Hardly…"Old" philanthropies have been doing this work for

decades…This approach has been common sense in philanthropy as well as in business for many

years, and is increasingly standard practice.” She warned her fellow industry colleagues to tread


carefully not to “discredit the special capacities the nonprofit sector brings to the table” and that the

business and venture capital approach is “leading some donors to distort philanthropic practices

that served society well”. Ms. Beresford stated that she is a 40 year veteran in the field and that this

notion of “old” versus “new” philanthropy “makes good copy”, but, “it does not fit the reality I

know”. From her point of view, “it has the potential to damage our field”. (Beresford 2007)

WHEN PUSH HAS COME TO SHOVE

Amid this clamor for accountability, transparency and impact it is hard not to be deeply

concerned about how the industry will evolve as the result of the demand for increased

measurements. Many, if not all arguments for and against measurements are valid and should

continue to be thoughtfully considered. Because of the industry’s tremendous growth in the past

15-20 years, it has no choice but to change. Push has come to shove. (Roumain 1947) “The

challenge for leadership and management is to prioritize among competing accountability

demands. This involves deciding both to whom and for what they owe accountability.” (Ebrahim,

The Many Faces of Nonprofit Accountability 2010) How we will make these changes as individual

professionals, organizations and as an industry is as yet unknown.

CURRENT MODELS AND METHODOLOGIES

According to Theodore H. Poister’s book, Measuring Performance in Public and Nonprofit

Organizations, 2003:

A measurement system must be designed to serve the needs of the particular management

process it is intended to support. Performance measurement systems are used to support a variety

of management functions, including the following:

Monitoring and reporting

Strategic planning

Budgeting and financial management


Program management

Program evaluation

Performance management

Quality improvement, process improvement

Contract management

External benchmarking

Communication with the public

Each of these functions can be carried out in ways that facilitate results oriented

management, and in each case performance measures are critical to provide the feedback

that allows it to focus on results. Yet these are very different, though often complementary,

management actions that serve different purposes. Each of these functions represents a

distinct use of and a specific set of requirements for performance measures. Thus, any

given measurement system needs to be tailored to its purpose and developed very

deliberately to support its intended use. (Poister 2003)

Theory of Change

Theory of Change is a term of art used to describe the use of maps to guide organizations

and communities through a transformation effort, maps that lay out the steps needed to arrive at a

desired outcome. “A theory of change offers a picture of important destinations and guides you on

what to look for on the journey to ensure you are on the right pathway.” (Organized Research

Services 2004) Like flow-charting, theories of change can come in many forms from simple linear

progressions from one point to the next to maps used to coordinate simultaneous and overlapping

processes. Like any good map, it will show you how to reach your destination, and show you how

to get back on your way if you have made a wrong turn or the highway is washed out. Well-

developed theories of change can be used as blueprints for achieving specific results in well-defined

domains and to make a nonprofit’s strategic vision operational. (Morino 2011)

Outcome Map One approach is to develop an outcome map, a visual diagram that

illustrates relationships between initiative strategies and intended results. The results will typically


include outcomes, both short and long term, and reflect changes at individual, organizational,

systems and community levels.

List of Assumptions about Change Maps need keys to interpret them. Geographic maps

may show roads and highways, elevations, ecological information, or population concentrations. A

key helps the reader to understand the markings. An outcome map needs a key as well, and it

comes in the form of a list of assumptions about change that accompanies the map and should

include the “philosophies, principles or values; ways to work together; community context and

other assumptions on which you have based your change effort.” (Organized Research Services

2004)

There is no right or wrong way to draw an outcome map; each map will look different,

depending on the organization’s unique needs and preferences. Within the theory of change, the

short and intermediate term outcomes for separate strategies or programs can be clearly identified.

This is important because many projects must produce change at the individual level if they are to

be able to instigate the change that is their ultimate goal and the outcome map will show the way to

the ultimate goal through these intermediate goals. Labeling these outcomes in the map will not

only bring a focus to evaluation and reporting but will also help show how the shorter term changes

contribute to the long term vision of change as illustrated in the table below.


Impact: Individual and Family Outcome Areas and Sample Outcome Statements

Outcome Area Sample Outcome Statements

Changes in attitudes, e.g.

perceptions and beliefs

Increased view among parents that local schools are positive "hubs" for families.

Increased desire among neighborhood residents to become engaged in community change efforts.

Increased feeling of safety among residents.

Increased desire of parents to create a personal savings plan.

Youth have increased belief that they will have a positive future.

Changes in knowledge Increased knowledge among neighborhood residents of community resources.

Increased knowledge of parents and caregivers about child development milestones.

Changes in awareness Increased awareness among neighborhood residents of a neighborhood's history.

Increased awareness of U.S. Laws regarding acceptable child discipline among immigrants and refugees.

Changes in skills Increased parents' employment skills.

Increased parent ability to locate child care.

Changes in behavior Neighbors more frequently call on one another for assistance.

Increased parent involement in their child's education.

Residents vote more frequently.

Changes in health Decreased blood pressure.

Families have access to a "medical home"

Changes in family stability Families' children attend the same school for all of their elementary grades.

Families maintain a stable residence.

Changes in financial status Increased family income.

Increased family savings.

The final map can incorporate many elements initially developed in table form into a visual

rendering, from short term changes to final outcomes, and including “So That” chains, core

capacities, influences and available leverage. A finished map might follow any flow-charting

protocol, standard or nonstandard, to illustrate the steps and directions the project needs to follow

to arrive at its outcome. The key to the map, the List of Assumptions, can accompany it in tabular

or narrative form.

(Organized Research Services 2004)


Collaborative grantmaking (pooled,

aligned and matching grants)

Coordinating, convening and

networking among

private/public funders

Funding priority issues

Increased ability to

influence community

goals

Increased ability to

achieve better outcomes

for the community

Increased amount

of funding

directed to

priority issues

Increased

networking

dedicated to

priority issues

Increased

stability of

funding and

consistency in

approach to

priority issues

Improved outcomes for

“aligned fund” goals

Improved “outcomes for family,

friends and neighbor care” goals

Improved outcomes for “school

readiness” goals

Improved effectiveness

as grantmakers

All children have an equal opportunity to succeed in school and as adults

Increased

intentionality

in the

approach to

priority

issues

Increased ability to leverage

resources to support

community goals

Increased

clout and

pressure to

tackle

priority

issues

Increased

visibility of

and

community

attention to

priority issues

Increased

willingness

to “tackle”

tough

priority

issues

Example of a completed outcome map

(Organized Research Services 2004)


Logic Models

Logic Models are a “systematic and visual way to present and share your understanding of

the relationships among the resources you have to operate your program, the activities you plan,

and the changes or results you hope to achieve.” (W.K. Kellogg Foundation 2004) Creating a

written logic model of program inputs, activities, outputs, outcomes and impact is a useful process

for thinking through the changes participants experience during and after a program. It also allows

for an analysis of the logic of the “if/then” influences the program intends to set in motion. (Plantz,

Greenway and Hendricks 1997) A logic model employs five basic components designed to

exemplify the connection between your planned work and your intended results.

Your Planned Work The first two components address your planned work, identifying the

resources needed for program implementation and the activities it will pursue.

1. Resources/Inputs include the human, financial, organizational, and community resources a

program has available to direct toward doing the work.

2. Program Activities are what the program does with the resources. Activities are the processes,

tools, events, technology, and actions that are an intentional part of the program implementation.

These interventions are used to bring about the intended program changes or results.

Your Intended Results include all of the program’s desired results: outputs, outcomes, and impact.

3. Outputs are the direct products of program activities and may include types, levels and targets of

services to be delivered by the program.

4. Outcomes are the specific changes in program participants’ behavior, knowledge, skills, status

and level of functioning. Short-term outcomes should be attainable within 1 to 3 years, while


longer-term outcomes should be achievable within a 4 to 6 year timeframe. Longer-term outcomes

should result in impacts that begin to be felt in 7 to 10 years.

5. Impact is the fundamental intended or unintended change occurring in organizations,

communities or systems within 7 to 10 years of the project’s inception. In the current W. K.

Foundation model, grantmaking and evaluation, impact often occurs after the conclusion of project

funding.

The term logic model is frequently used interchangeably with the term “program theory” in the

evaluation field. Logic models are alternatively referred to as theory because they describe how a

program works and to what end. (W.K. Kellogg Foundation 2004)

Sample Logic Model

(Pell Institute n.d.)

)


Dashboards

Performance dashboards are not “one size fits all”. Every organization needs to track metrics

that are relevant to its own mission, activities, output and outcomes. However, effective dashboards

share the following characteristics:

They contain meaningful, timely, accurate data

They are rich with information but not overwhelming

They help nonprofit organizations make short- to medium-term operational decisions with

a clear eye toward improving long-term outcomes

They flag issues or problems as well as accomplishments. (O'Flanagan, Harold, Best 2008)

The key factor in designing a dashboard is deciding what kind of data to collect and how to use

it. According to Jason Saul, author of Benchmarking for Nonprofits: How to Measure, Manage,

and Improve Performance: “A dashboard could just be a fancy report that says nothing if the data

isn’t meaningful. It’s easy to create a database and plug in numbers. It’s much harder to create

performance metrics that can be measured against desired results and used to make real-time

programmatic, financial, or managerial decisions.” (Saul 2003)

Indiana Museum of Art Dashboard

(Indianapolis Museum of Art)


One thing worth noting in the dashboard illustrated above, it is not an end in itself. Even

though the data shown is summary data, each data item is accompanied by a link that allows the

viewer to drill down to a greater level of detail. That next level might be another dashboard, or it

might be a tabular display of data. While the term dashboard itself might suggest a single layer of

information, in practice it should allow the user/visitor to follow lines of inquiry as desired. In fact,

a well designed dashboard should draw the user to information that the organization wants to

emphasize.

DEVELOPING A COMMON LANGUAGE

In 2011, the Center on Nonprofits and Philanthropy sponsored a symposium that

addressed the barriers to and opportunities for making performance management more common

in the social sector. Mindy Tarlow from the Center for Employment Opportunities addressed the

lack of a common language for measurements:

“Around performance management time and again, you hear (that) it is somehow

antithetical to passion and mission. Or that no one is ever going to understand the impact

of what you do if you talk about numbers. They are only going to understand it if you talk

about stories. And I would submit that that is just not true. You have to really be able to

talk about both….I think it is not so much about numbers; it is about facts. If you actually

just use that word—instead of saying ‘numbers,’ ‘performance,’ ‘outcomes,’ you just used

the word ‘facts’—I think most people would agree that you would rather know facts than

fiction. That you would rather push yourself toward something that you actually know than

something that somebody just told you and so you are running with it. So, I think that is a


language issue that we would be well served to think about.” (Tough Times, Creative

Measures 2011)

In 2006, The Urban Institute and the Center for What Works created a draft taxonomy of

candidate quality indicators for nonprofit outcomes to assist nonprofits in collecting outcome data.

As important as the taxonomy may prove to be, it is instructive to read how the Urban Institute has

gone about assembling it.

While there is no shortage of outcomes and their indicators in some program areas, there

is no centralized grouping of them or assessment of their quality that could serve as a

resource for organizations that wish to develop outcome measurement systems. And

because of the vast range of programs in the voluntary sector, major gaps exist in the

coverage of indicators that have been developed. The taxonomy attempts to provide a way

to help reduce this gap – for those programs for which indicators are not yet available.

We collected information from a wide range of sources, from national nonprofit umbrella

groups in the US, national accreditation agencies in specific fields, and from national

nonprofits with local affiliates. They were assessed with thought about which ones were

useful, relevant, and feasible. We also considered outcome indicators that were seldom

currently used but appear to be very appropriate for inclusion.

The most useful taxonomies tend to reflect the manner in which the sector itself organizes,

collects and reports the information. Although essential taxonomic principles of

comprehensiveness, mutual exclusivity of elements, and logical consistency must be

followed, a grounding is needed in what is actually in use by practitioners and what has

worked for the specific program areas. Thus, testing by stakeholders (including nonprofit


staff; funders, both public and private; clients, participants, and service users; and even the

public, where appropriate) is essential.

Outcomes and indicators were collected for fourteen different program areas to help

inform the development of the taxonomy. Lists of quality outcomes and their indicators

were selected for program areas ranging from emergency shelter to youth mentoring to

health risk reduction programs. The outcomes for the various programs were reviewed for

common elements, which then became part of the taxonomy.

The development and refinement of the taxonomy will continue to be an iterative process,

as outcomes and indicators are collected for even more programs. (The Urban Institute

and the Center for What Works 2006)

If nothing else, the extent of the effort expended suggests the extent of the problem. In the

diverse sector that is the nonprofit industry agreement on a common vocabulary will be difficult.

For that reason, a more basic approach may be more successful.

In Mark Friedman’s 2005 book, Trying Hard is Not Good Enough: How to Produce

Measureable Improvements for Customers and Communities, he addresses the need for a

common language to be established within an organization. His Results Accountability framework

approach looks at the conceptual building blocks that get your organization from talk to action,

“Whether it's English, Spanish or another language, we often use words in confusing ways that no

one really understands. People who work together need a common language to be successful.

Results Accountability uses three common sense performance measures: How much did we do?

How well did we do it? And is anyone better off? (Friedman 2005) ; And uses just a few key words

to deliver the message.


Results: The conditions of well-being we want for our children, families and the community

as a whole.

Indicators: How we measure these conditions.

Baselines: What the measures show about where we've been and where we're headed.

Turning the curve: What success looks like if we do better than the baseline.

Strategies: What works to improve these conditions.

Performance measures: How we know if programs and agencies are working.

This may be a more practical approach, at least for the near term.

BUILD SECTOR KNOWLEDGE

There are two initiatives currently underway to assist nonprofits in accessing information on

performance and impact measurement. The descriptions that follow are in their own words.

PerformWell

PerformWell is a collaborative effort initiated by Urban Institute, Child Trends, and Social

Solutions. PerformWell provides measurement tools and practical knowledge that human

services professionals can use to manage their programs’ day-to-day performance.

Information in PerformWell leverages research-based findings that have been synthesized

and simplified by experts in the field. By providing information and tools to measure

program quality and outcomes, PerformWell helps human services practitioners deliver

more effective social programs. (PerformWell.org n.d.)


McKinsey & Company’s Social Sector Office: Learning for Social Impact

The Learning for Social Impact site, part of McKinsey's Social Sector Office has developed

a site to help funders, their grantees, and other essential partners achieve social change by

offering best practices, guidelines, tools, insights, and practical help in developing

assessment plans that drive social impact. (Learning for Social Impact n.d.)

RECOMMENDATIONS

The following are recommendations based on the research conducted for this paper.

Books

Leap of Reason: Managing to Outcomes in an Era of Scarcity by Mario Morino, Venture

Philanthropy Partners, 2011

Free download: http://www.vppartners.org/leapofreason/getit

Working Well & Working Hard: A Practical Guide to Performance Management by

David E. K. Hunter, Venture Philanthropy Partners, 2013

Free download: http://www.vppartners.org/leapofreason/workingwellbook

Trying Hard is not Good Enough by Mark Friedman, Trafford Publishing, 2005

The Robin Hood Rules for Smart Giving by Michael M. Weinstein and Ralph M.

Bradburd, Columbia University Press, 2013

Reports & Guides

Outcomes Indicators Project, a joint project of the Urban Institute and the Center for

What Works: The Outcome Indicators Project provides a framework for tracking nonprofit performance. It suggests candidate outcomes and outcome indicators to assist nonprofit organizations that seek to develop new outcome monitoring processes or improve their existing systems. http://www.urban.org/center/cnp/Projects/outcomeindicators.cfm

Building a Common Outcome Framework to Measure Nonprofit Performance, The

Urban Institute and The Center for What Works, 2006

http://www.vppartners.org/leapofreason/getit

http://www.vppartners.org/leapofreason/workingwellbook

http://www.urban.org/center/cnp/Projects/outcomeindicators.cfm


PDF download: http://www.urban.org/publications/411404.html

The Nonprofit Taxonomy of Outcomes: Creating a Common Language for the Sector,

The Urban Institute and The Center for What Works, 2006.

PDF download:

http://www.urban.org/center/met/projects/upload/taxonomy_of_outcomes.pdf

W. K. Kellogg Foundation Logic Model Development Guide, 2004

PDF download: http://www.wkkf.org/knowledge-center/resources/2006/02/wk-kellogg-

foundation-logic-model-development-guide.aspx

Performance Management Software (Examples)

Company/Software Website Speciality

Cityspan http://www.cityspan.com/default.asp Social Services

CTK (Apricot) http://www.communitytech.net All Nonprofits

EZReports http://ezreports.org After-School Programs

nFocus http://www.nfocus.com All Nonprofits

Social Solutions http://www.socialsolutions.com Human Services

Cayen Systems http://www.cayen.net All Nonprofits

Doview http://www.doview.com All Nonprofits

WHAT MEASUREMENTS HAVE BROUGHT THE FIELD THUS FAR

Besides the passionate articles and heated debates, the theories and the methods, the focus

on oversight and the lack thereof, what measurements have brought the field thus far is the

realization that this sector has become a rising force in the economy of this and other countries

around the world and it deserves to be recognized and respected. The subject has also brought

articles, essays and books from the pens of some of the world’s most eminent nonprofit scholars

and studies published by the most prestigious and esteemed institutions, including Harvard

Business School, Indiana University, the United Way of America and the Urban Institute. They

have shown us how buoyant our nonprofit institutions are and how important and lasting they will

continue to be.

http://www.urban.org/publications/411404.html

http://www.urban.org/center/met/projects/upload/taxonomy_of_outcomes.pdf

http://www.wkkf.org/knowledge-center/resources/2006/02/wk-kellogg-foundation-logic-model-development-guide.aspx

http://www.wkkf.org/knowledge-center/resources/2006/02/wk-kellogg-foundation-logic-model-development-guide.aspx


CONCLUSION

In 1997, the United Way of America published an article, Outcome Measurement:

Showing Results in the Nonprofit Sector. It described outcome measurement in five important

areas, provided lessons learned and discussed key challenges to overcome. The article concluded

in this way:

Nonprofit agencies often ask if outcome measurement is just a fad that, if ignored long

enough, will go away. The consensus of the field is a resounding "No." Like earlier aspects

of performance measurement that built upon each other to strengthen the management

and delivery of nonprofit services, the careful measurement of outcomes is another

essential building block that informs us in new and necessary ways about how to improve

programs. It is not the last brick to be added. Nor has it developed as far as it can and will.

However, outcome measurement is here to stay, and the nonprofit sector and the

individuals it serves will be better for it. (Plantz, Greenway and Hendricks 1997)

Over 15 years later, this statement still sounds contemporary because the issue of reporting

outcomes is still not settled and there are still two sides staking their positions. Has this thesis

identified a middle ground? Is there a middle ground? They jury may remain out on that for a

long time because this is still the crest of the wave for the measurement movement. But while

searching for a middle ground, what became clear was that, like the composition of a painting, in

addition to a middle ground, there needs to be a background and there needs to be a foreground.

At a conference in 2007, Susan Beresford, former President of the Ford Foundation put it most

eloquently:

I hope you now understand why I believe it would be good to discard the "old/new"

effectiveness dichotomy. It is false and divisive at a time when we need to understand our

field's history, apply knowledge gained from decades of donors' experience and inject new

and exciting ideas and capacities. We need all kinds of donors and philanthropies, all kinds

of skills and ambitions. We need people with near-term goals as well as long-range aims.

We need business management skills but also moral leadership skills, and much more.

Great leaders such as Dr. Martin Luther King Jr., Nelson Mandela and Mahatma Gandhi


combined powerful moral vision, expressive power and extraordinary dedication and

courage. For enduring social change, for humankind, we need it all. (Beresford 2007)

In the end, the same will probably be said of reporting. If we cannot now formulate a single

best practice for all nonprofits to adopt for reporting, then we will probably discover we are best

served when the different types of nonprofits each adopt a method best matched by structure, size,

location, mission, beneficiaries and funding sources. Some organizations will occupy a middle

ground, others will work in the foreground, while others will occupy the background. Each will

report in its own way. But all participants should realize that the message has been sent: nonprofits

should report on outcomes. The sector has become too large a part of the economy. It will reach

the point, if it hasn’t already, when economic events in the nonprofit world will affect the rest of the

economy. The special tax status afforded nonprofits and their donors extend the support they

receive beyond their own sources of funding to funding provided by all taxpayers. As the sector

grows, the demands for an accounting for those funds will become louder.

There is already a requirement to report sources of funds and uses of funds. Funds are

tangible. What is at question here is how to report on the intangible, the benefits that result for the

beneficiaries of the nonprofit organization - the benefits that result even when a project has failed.

That is where the struggle is now and that is where we still wait for an answer. In the meantime,

there is perhaps a minimum that a nonprofit should be prepared to report: how the outcomes

achieved support the purpose set out in the mission statement. It has been said that mission

statements are more an expressions of an aspiration than a guide for an organization. There has

probably been no time when a nonprofit’s mission statement has meant more than it means now.

Not every organization can be micro-focused on single issues and back up the effort with

substantial donations. Some of the world’s most intractable problems are being addressed by small

organizations of great dedication that sometimes must feel they take one step back for every two


steps forward. Yet they can still justify every result, short-term or long-term, positive or negative, if

they can hold up their mission statement and show exactly how the actions they take, the money

they spend, and the results they achieve are in keeping with the letter and the spirit of the mission

statement that defines their purpose. After that, everything else is just numbers.


WORKS CITED

uniteforsight.org. n.d. http://www.uniteforsight.org/evaluation-

course/module13_clip_image002.jpg?h=364&w=623.

21/64 and the Dorothy A. Johnson Center. "#NEXTGENDONORS Respecting Legacy,

Revolutionizing Philanthropy." Research Project, 2013.

Bailin, Michael A., interview by Susan Hoff. Former President, Edna McConnell Clark

Foundation (March 27, 2013).

Bailin, Michael A. "Philanthropy in Practice: Great Expectations Versus Getting the Job Done."

State of Philanthropy, 2004.

Benjamin, Lehn M. "Bearing More Risk for Results: Performance Accountability and Nonprofit

Relational Work." Administration & Society, Sage Publications, 2008.

Berger, Ken. "Measuring the Impact of Your Charitable Donations." NPR Talk to the Nation.

2012.

Beresford, Susan. "Philanthropy: What's New, What's Not." www.fordfoundation.org. February 9,

2007. http://www.fordfoundation.org/newsroom/speeches/192.

Bliss, John. Principal at NonProfit Solutions, LLC (February 22, 2013).

Boland, Steve. "Correlation, Causation, and Communication." nonprofitsassistancefund.org.

September 28, 2012. http://www.nonprofitsassistancefund.org/blog/2012/09/28/correlation-

causation-and-communication/

Bradach, Jeffrey L., Thomas J. Tierney, and Nan Stone. "Delivering on the Promise of

Nonprofits." Harvard Business Review, December 2008.

Broeckling, Jerica. "Performance Measurement Systems: Theory and Practice." Masters Thesis,

2010.

Campbell, Joseph, interview by Bill Moyers. The Power of Myth (1988).

Charity Navigator. Where We Are Headed (2013 and Beyond). 2013.

http://www.charitynavigator.org/index.cfm?bay=content.view&cpid=1193.

Cuomo, The Honorable Andrew M. www.governor.ny.gov. August 3, 2011.

http://www.governor.ny.gov/press/08032011taxpayersupportednot-for-profits.

Drucker, Peter. Management: Tasks, Responsibilities, Practices. New York City: Harper & Row,

Publishers, Inc., 1973.

http://www.nonprofitsassistancefund.org/blog/2012/09/28/correlation-causation-and-communication/

http://www.nonprofitsassistancefund.org/blog/2012/09/28/correlation-causation-and-communication/


Ebrahim, Alnoor. "The Many Faces of Nonprofit Accountability." Jossey-Bass Handbook of

Nonprofit Leadership and Management, 2010.

Ebrahim, Alnoor, and V. Kasturi Rangan. The Limits of Nonprofit Impact: A Contingency

Framework for Measuring Social Performance. Working Paper, Harvard Business School,

2010.

Edison, Thomas. Goodreads.com. n.d.

http://www.goodreads.com/author/quotes/3091287.Thomas_A_Edison?auto_login_attemp

ted=true.

Forti, Matthew and Yazbak, Kathleen. Building Capacity to Measure and Manage Performance.

The Bridgespan Group, 2012.

Foundation, W.K. Kellogg. Logic Model Development Guide. W.K. Kellogg Foundation, 2004.

Friedman, Mark. Trying Hard is Not Good Enough. Trafford Publishing, 2005.

Frostenson, Sarah. "Number of Charities and Foundations Rose Slightly in 2012." The Chronicle

of Philanthropy, April 4, 2013.

GuideStar. n.d. http://www.guidestar.org.

Hanna, Julia. "The New Measures for Improving Nonprofit Performance." Harvard Business

School, Working Knowledge, December 14, 2011.

"Ibid." n.d.

Karoff, Peter. Stanford Social Review, Novemeber 27, 2012.

Kingsbury, Nancy R. Performance Measurement and Evaluation: Definitions and Relationships.

United States Government Accountability Office, 2011.

Learning for Social Impact. n.d. http://lsi.mckinsey.com/.

McGill, Larry. "Why Measuring Impact Remains an Elusive Goal." Philantopic, 2011.

McKinsey & Company. "The Nonprofit Marketplace: Bridging the Information Gap in

Philanthropy." 2008. http://www.hewlett.org/library?page=2.

McRay, Greg. "IRS Improves Nonprofit Data Availability." www.501c3.org. April 9, 2013.

http://www.501c3.org/blog/irs-improves-nonprofit-data-availability/.

—. "White House Advocates for Mandatory Form 990 E-File." www.501c3.org. April 11, 2013.

http://www.501c3.org/blog/white-house-advocates-for-mandatory-form-990-e-file/.


Menzies, Peter. "Counterfactual Theories of Causation." The Stanford Encyclopedia of

Philosophy. 2009. http://plato.stanford.edu/cgi-

bin/encyclopedia/archinfo.cgi?entry=causation-counterfactual.

Morino, Mario. Leap of Reason: Managing to Outcomes in an Era of Scarcity. Venture

Philanthropy Partners, 2011.

National Center for Charitable Statistics. The Nonprofit Sector in Brief: Public Charities, Giving,

and Volunteering in 2012. Brief, Urban Institute Press, 2012.

National Health Council and National Assembly for Social Policy and Development. Standards of

Accounting and Financial Reporting for Voluntary Health and Welfare Organizations.

New York: National Health Council and National Assembly for Social , 1964.

O'Flanagan, Maisie, Jacob Harold, and Paul Best. The Nonprofit Marketplace: Bridging the

Information Gap in Philanthropy. McKinsey & Company, 2008.

Organized Research Services. Theory of Change: A Practical Tool for Action, Results and

Learning. Manual, Organized Research Services, 2004.

Pallotta, Dan. "The Way We Think about Charity is Dead Wrong." Ted.com. March 2013.

http://www.ted.com/talks/dan_pallotta_the_way_we_think_about_charity_is_dead_wrong.h

tml.

Paton, Rob. Managing and Measuring Social Enterprises. SAGE Publications, 2003.

Pell Institute. "Evaluation Toolkit." toolkit.pellinstitute.org. n.d.

http://toolkit.pellinstitute.org/evaluation-guide/plan-budget/using-a-logic-model/.

PerformWell.org. n.d. http://www.performwell.org/.

Plantz, Margaret C., Martha Taylor Greenway, and Michael Hendricks. "Outcome Measurement:

Showing Results in the Nonprofit Sector." New Directions for Evaluation, 1997: 15-30.

Poister, Theodore H. Measuring Performance in Public and Nonprofit Organizations. San

Francisco: John Wiley & Sons, Inc., 2003.

Preston, Caroline. "Philanthropy Must Do More to Influence Policy, Say Government Officials."

The Chronicle of Philanthropy, 2013.

—. "Some Nonprofit Leaders Ask: Is Philanthropy Killing Itself with Kindness?" The Chronicle of

Philanthropy, February 24, 2013.

—. "Wealthy Young Donors Push Charities to Show Results." The Chronicle of Philanthropy,

January 25, 2013.


Rosenman, Mark. "Obama's College Scorecard is a Bad Idea for All Nonprofits." The Chronicle of

Philanthropy, 2013.

Roumain, Jacques. Masters of the Dew (English Translation). Berkshire, Great Britain: Cox &

Wyman Ltd, 1947.

Saul, Jason A. "Nonprofit Business Intelligence-How to Measure, Manage and Improve

Performance." www.orgwise.ca. 2003.

http://www.orgwise.ca/sites/osi.ocasi.org.stage/files/resources/Nonprofit%20Business%20Int

elligence,%20How%20to%20Measure%20and%20Improve%20Nonprofit%20Performance

.pdf.

Sawhill, John C., and David Williamson. "Mission Impossible? Measuring Success in Nonprofit

Organizations." Nonprofit Management & Leadership, Spring 2001.

Schambra, William. "Measurement is a Futile Way to Approach Grant Making." The Chronicle of

Philanthropy, 2011.

Stern, Marian. "Adjunct Professor, NYU Heyman Center for Philanthropy & Fundraising." May 1,

2013.

Strengthening Transparency, Governance, and Accountability of Charitable Organizations. Panel

on the Nonprofit Sector, 2005.

Strom, Stephanie. "To Help Donors Choose, Web Site Alters How It Sizes Up Charities." The

New York Times, November 26, 2010.

The Free Dictionary. n.d. http://www.thefreedictionary.com/philanthropy.

The Urban Institute and The Center for What Works. Building a Common Outcome Framework

to Measure Nonprofit Performance. The Urban Institute, 2006.

The Urban Institute and The Center for What Works. The Nonprofit Taxonomy of Outcomes:

Creating a Common Language for the Sector. The Urban Institute, 2006.

Tobias, Mark. "Ten Technology Trends to Watch in 2013." Stanford Social Innovation Review,

December 18, 2013.

"Tough Times, Creative Measures." Tough Times, Creative Measures: What Will it Take to Help

the Social Sector Embrace an Outcomes Culture? The Urban Institute and Venture

Philanthropy Partners, 2011.

United Way of America. Accounting and Financial Reporting: A Guide for United Ways and Not-

for-Profit Human Service Organizations. Alexandria, VA: United Way of America, 1974.


W.K. Kellogg Foundation. "W.K. Kellogg Foundation Evaluation Handbook." 2004.

http://www.wkkf.org/knowledge-center/resources/2010/w-k-kellogg-foundation-evaluation-

handbook.aspx.

Walker Ph.D, Karen E., and Kristin Anderson Moore Ph.D. "Performance Management and

Evaluation: What's the Difference?" Child Trends, 2011.

Weinstein, Michael M., interview by Susan Hoff. Senior Vice President and Cheif Program

Officer (March 28, 2013).

Weinstein, Michael M., and Ralph M. Bradburd. The Robin Hood Rules for Smart Giving. New

York City: Columbia Business School Publishing, 2013.

Weinstein, Michael. Robinhood.org. n.d. http://www.robinhood.org/metrics.

Wikipedia.org. n.d. http://en.wikipedia.org/wiki/Performance_measurement.

Worth, Michael J. Nonprofit Managment: Principles and Practice. Sage Publications, 2012.

Documents

Susan Hoff-Capstone Thesis Project