5 Simple Steps to Evaluating Your Program

8/3/2019 5 Simple Steps to Evaluating Your Program

1/4

5 Simple Steps to Evaluating Your ProgramBy Jenelle Montoya and contributing author Marcie L. Wagner

January 2012

For a startup nonprofit, the idea of program evaluation can be overwhelming, and yet funders are

increasingly asking grantees to provide not only numbers served, but demographic information and

outcomes-based information. As more nonprofits compete for a

reduced funding pool, its important that they know and understand

the difference their investment made in your ability to carry out the

organizational mission. Lets say your mission is to provide a safe

place for youth. Sure, you may have given Jim a place to sleep for

the night, but how did that positively impact his life and his future?

What did you do for Jim to prevent him from having to use your

services in the future? How many Jims did you help in 2011? How

many did you turn away? What are their ages? Where did they come

from? Why did they come? How many are high school drop outs?

How many have their GED? How many are immigrants? And so on.

The secret to getting started is to keep program evaluation simple and include board members, staff,

and anyone else involved in data collection and use. Its easy to end up with analysis paralysis if you

try to do too much too soon. Realize that program evaluation is a process, not an event. It can take

many months to implement, and years to refine. Whats important is that you get started now. If you

receive grant funding, you really have no choice. We are here to help!

STEP ONE: Decide What You Will Measure. First you need to know what outcomes your funders want atyears end. Second, you must track the information your organizations leadership needs in order to

make knowledgeable decisions that either impact the course of your organizations future or ensure

your organization is tracking with your strategic plan (if you have one). This process can be as simple as

reviewing different reports required by existing funders and the grant guidelines of your prospects. Sit

down with your board president or create an ad-hoc Program Evaluation committee to take on the task.

There are several levels of knowledge you might need when it comes to collecting program data:

y Data including age, residence, income, race, ethnicity(demographics)

y Numbers served, goods distributed, or other units of service youdefine (outputs)

y The impact of your services on the lives of your clients (outcomes)Demographic data is easy to collect. It can be as simple as handing each

client a form to fill out. If confidentiality is important, your form doesnt


2/4

require a name. In order to count numbers served or units of service, staff must be trained to record

the people they serve each day and the specific activities they engage in with the client.

Measuring the impact your services make on clients is a bit difficult, but it can be the most valuable and

rewarding data you collect. If you decide to measure impact, it will benefit you to explore creating a

logic model. A logic model illustrates the

relationships between inputs, activities, outputs,

outcomes and goals that create the structure and

flow of your organization. Briefly, inputs are the

resources that go into making your program

activities possible, such as technology, training,

people and funding. The activities you undertake

as part of your program are what you will do to

accomplish your goals. Outputsare the product ofyour program, as in the number of people served

or the number of items distributed or made. Finally, outcomes are the impact of your work on those you

serve, or the change affected in their lives. Logic models are a complex topic and there are many

different approaches that can be taken to developing one for your program. Resources are available for

assistance in creating a logic model, including the United Way of Americas book Measuring Program

Outcomes, which is available at www.unitedwaystore.com for a mere $5. The W.K. Kellogg Foundation

has also created a more complex logic model guide which can be downloaded free from their

Knowledge Center at www.wkkf.org.

Finally, set quantitative goals around the items youve decided to measure. If you exclusively measure

demographics, create a goal that 80% of clients served are classified homeless by federal definition,

which helps you measure the impact of organizational outreach, or marketing and advocacy work. If you

measure numbers served or other units of service, your goal could be to provide 1,000 children withshoes in a 12-month period, or provide 800 hours of community outreach to areas with high youth

crime rates. An example of an outcome to measure impact would be that 75% of clients are able to

provide for their food and shelter needs upon exit from your program, or that 60% of individuals

receiving parenting education report a greater sense of harmony and wellbeing in their households after

they complete a course.

STEP TWO: Determine the Tools You Will Need. Assess your

current evaluation tools, such as client intake forms or reporting

already required by current funders, refining what youre already

doing and preventing duplication. The tools you need and the

cost of collecting data depends on what youve decided to

measure and the complexity of your plan. If youve decided to

collect demographic data, you can use current intake forms byincluding questions which add depth. If you collect numbers

served or units of service involve your staff in deciding how to do it efficiently. Use simple ticker sheets

compiled by hand, or use electronic spreadsheets with embedded formulas to automatically tabulate


3/4

numbers. Google Docs offers a free way to create electronic forms in an online database which allows

you to distribute to a number of users via email. The software automatically compiles the numbers

submitted on the forms into a single spreadsheet which you can use for reporting. A more costly option

is to purchase database management software. There are several companies offering tailor-made

solutions.

When collecting outcome-based data to measure impact, it can be difficult to determine the best

method. Lets go back to Jim. After providing Jim with a place to sleep for the night, your outcome is to

link him with community resources and increase his knowledge and utilization of such services, and to

ensure he follows through and uses your referrals. In order to accurately measure the outcome, you

could provide him with an exit survey allowing him to rate your services and the likelihood that he will

follow through with the referrals on a numerical scale. Lets say Jim does return; during the intake rate

his level of improvement on a scale from 1 to 5 since his last visit. At year-end calculate how many

clients returned once, twice, or never. If you come up with a logical way of quantifying this data, you caneasily record it in a database and tabulate it. Remember that you are the expert when it comes to your

work and the data collection methods most applicable and reasonable for your particular service(s).

STEP THREE: Assign Responsibility. When possible, put this responsibility in the appropriate staff

person(s) job description. When you designate a person to manage the data collection process, including

compilation and reporting, you are 90% on your way to success. Make sure you choose someone who

has the skills and understands the value in evaluation, and then provide them with the tools and

training. In our hypothetical safe place for youth scenario, it makes sense to designate staff in charge

of client intake and discharge to collect and manage data collection. If data collection is new to your

organization and the responsibility was not in a job description upon

hire, you may encounter resistance from program staff to take on

additional tasks. We suggest you dont act flip about adding to their

already long list of responsibilities Instead, introduce data collection inthe form of a PowerPoint presentation to the entire staff. Begin with

your logic model, as it will best convey that program evaluation

requires everyones full support and assistance. It will also clearly

explain who is best positioned to collect the data: program staff. Place

a quantitative value on the importance of data collection and how it

will directly and positively impact their job and their work with clients by, for instance, sharing the

amount of support received from funders who require the data. Well done evaluation opens the door to

new and increased funding; it gathers emotional and financial support from your community as you tell

your story through qualified and quantified data which proves youre making a difference.

STEP FOUR: Implement. Youre ready to implement the project

evaluation plan once youve decided what to measure, obtained

and prepared the necessary tools, and determined who isresponsible. Implementation should begin by creating a clear and

concise procedure manual. This manual will define those

responsible, the timing and frequency of data collection, and

Comment [MLW1]: Public Benefits is an

explosive phrase for obvious reasons.


4/4

when and how often reports are produced and reviewed. It will also serve as an invaluable resource for

training new staff. Your designated data manager(s) must be vigilant in the beginning, checking in on

staff to ensure they follow the procedure. In an effort to make your oversight seem less invasive, solicit

suggestions on how the system is working and whether there are ways to improve the system and make

it more efficient. Ask all staff responsible to track the amount of time data collection is requiring and

watch closely to see if the time allotted begins to wane as they grow accustomed to the process. When

you secure new funding, double check your current data collection to ensure your existing system will

provide the evaluation data they require; if not, implement their unique specifications as seamlessly as

possible and update the staff, forms and procedure manual.

STEP FIVE: Utilize your Data. This is the step where many nonprofits tend to fall short; conversely it can

be where the process of collecting data and evaluating comes full circle via new, renewed or upgraded

gifts.

Most likely your organization has a strategic plan. Every

decision your board makes should relate directly to that

plan. A strategic plan primarily takes into account all things

related to successfully and sustainably carrying out your

mission over the long haul. This information boils down to

finance and program impact to make certain your

organization or program is carrying out its mission. You feed

your board this information as a result of successful data collection. If your data reveals problems or

issues, leadership can make informed decisions and act accordingly. If your data reflects positive results,

send out a press release and publicize your successful outcomes in your newsletter(s), annual report,

and website.

All donors (whether corporate, foundation or individual) want and need to know their return on

investment. Their return on investment can be given qualitatively through client testimonials, and

quantitatively by showing impact in numbers. For example, maybe youve helped 1,000 people increase

their knowledge of how to gain employment by 80%. Thats something to celebrate!

A program evaluation system is beneficial to nonprofits on many

levels. Developing the system requires organizational investment,but it doesnt have to be cumbersome or costly. Some orga-

nizations may feel they cant afford to spare any amount of time

for such a project. We say organizations cant afford to ignore such

a project. If nonprofits want to survive and thrive, they must

acknowledge their duty to supporters and clients by measuring the

impact (or lack of impact) of their mission.

Documents

5 Simple Steps to Evaluating Your Program