B.Quilitch 4-4-14

EVIDENCE-BASED PRACTICES

Dr. Bob QuilitchNevada Licensed Psychologist

B.Quilitch 4-4-14

Which Evidence & What Practices?

• Evidence? data, facts, information which is a credible reflection of what it is you are doing

• Practices? Your daily work, what you are doing.

• Example: a school collecting and using outcome data in such a way as to track and improve it’s teaching of long division

B.Quilitch 4-4-14

Another Example: Honda

• Collecting, reviewing and analyzing the defect rates of newly built automobiles in such a way as to improve their production techniques and begin building new cars which had almost no defects.

• Important Point: these data were not being collected to send to the government; they were being collected and formally reviewed to gradually improve their products.

B.Quilitch 4-4-14

Crucial requirements for evaluation

• Mission: what are you trying to do? What is your goal?

• Measurement: how might you measure the results of your Mission, of what you are doing?

• Questions: how well are you doing? How effective is your program? Is your program worth my financial investment?

B.Quilitch 4-4-14

We Love Staying Busy

• As individuals: running breathlessly through our To Do Lists, going to bed realizing we did nothing that really mattered today;

• Agencies: raising funds, hiring staff, building facilities, staging events with too little consideration to “What are we trying to do?”

• Busy is not Effective

B.Quilitch 4-4-14

The Target = The Mission

B.Quilitch 4-4-14

Robert F. Mager on “What is it you are trying to do?”

B.Quilitch 4-4-14

Goal Analysis

• “if your goals-your visions-are important to achieve, then it is essential that you do more than just talk about them in “fuzzy” terms. …Broad statements of intent can be achieved only to the degree that their meaning is understood, to the degree that you can recognize achievement of the goals when you see it” Robert Mager

B.Quilitch 4-4-14

The Program

B.Quilitch 4-4-14

The Program Ready to Use Feedback to Improve Its Performance

B.Quilitch 4-4-14

The Feedback Loop is Complete: Ready to Do Better Next Time

B.Quilitch 4-4-14

The Solace Tree

• The Solace Tree• The mission of the Solace Tree, Inc. is that all

children, teens and their family members have the freedom to express their feelings associated with death in a safe and loving environment. Our peer support programs and educational opportunities help children and teens learn to cope and adjust to the changes in their lives.

B.Quilitch 4-4-14

The Note Ables

• Mission…• To create lasting change in the lives of people of all abilities through music.• Vision…• We envision a community where every person can access, experience, and connect through music.• Purpose…• To help people of all ages and abilities achieve personal goals through music.• Philosophy…• All of our programs are grounded in the philosophy that participating in music – expressing oneself through

making sound – is both liberating and empowering for those who traditionally have had no voice in our society.• Pride…• We are proud to say that Note-Able Music Therapy Services enhances the lives of those who face so many

barriers to community inclusion – and improves the quality of life for all of us.• • Music is the difference• •

B.Quilitch 4-4-14

Volunteers of America

•Mission Statement

• Volunteers of America is a movement organized to reach and uplift all people and bring them to the knowledge and active service of God.

Volunteers of America, illustrating the presence of God through all that we do, serves people and communities in need and creates opportunities for people to experience the joy of serving others.

Volunteers of America measures its success in positive change in the lives of individuals and communities we serve.

Vision •

A world where all people live in safety, with social, emotional and physical well being, spiritual fulfillment, justice and hope. - See more at: http://www.voa.org/About-Us/Our-Mission#sthash.NnQRanDG.dpuf

B.Quilitch 4-4-14

Boys and Girls Club• Our Mission• To enable all young people, especially those who need us most,

to reach their full potential as productive, caring, responsible citizens.

• Core Beliefs• A Boys & Girls Club Provides:• A safe place to learn and grow...• Ongoing relationships with caring, adult professionals...• Life-enhancing programs and character development

experiences...• Hope and

B.Quilitch 4-4-14

Unintended Consequences Corrected With Data

• Washington, DC: a program to reduce domestic violence in Hispanic neighborhoods belatedly discovered its program seemed to be increasing domestic violence.

B.Quilitch 4-4-14

Back to Honda

• Honda did not achieve building near-perfect cars with a goal such as

• “We are dedicated to building really fine automobiles”

B.Quilitch 4-4-14

Goals Are “Political” Not “Scientific”

• Stakeholders must discuss, argue and eventually, agree on the desired goals.

• Imagine coming up with goals for children’s music education in – Bali– Detroit– Berlin– Reno

B.Quilitch 4-4-14

Mario Morino

B.Quilitch 4-4-14

Mario Morino

• “the vast majority of nonprofits have no reliable way to know whether they are on track to deliver what they promise to those they serve”.

• “managing to outcomes means investing in continuous and use of information to guide the organization’s decisions and operations”

B.Quilitch 4-4-14

B.Quilitch 4-4-14

Give Smart

• “Every donor wants his or her money to make a difference and nobody wants to see hard-earned wealth go to waste…..Outstanding philanthropy is distinguished by what it accomplishes….It requires you to (have) a disciplined consideration of what you hope to accomplish: the results that will define success”.

B.Quilitch 4-4-14

B.Quilitch 4-4-14

Ken Stern

• “We live in a society that is obsessed with results, from government, from sporting events…..Yet in this results-obsessed country, the public rarely demands measures of how effective charities are in implementing their services and meeting their service goals….This book’s story of charitable ineffectiveness is not one of greed or incompetence..-but one of misguided incentives and failed market structures”.

B.Quilitch 4-4-14

Pat Lawlor and Youth Villages

• A children’s treatment program throughout the southeast which can show that– 80% of clients are reintegrated into their own

homes within 6 months;– 82% of clients are free of legal problems within 6

months;– Those numbers remain constant at 12 and 24

months after treatment ends.

B.Quilitch 4-4-14

Pat Lawlor says

• “The state should not be buying beds. They should be buying outcomes, successful outcomes”.

B.Quilitch 4-4-14

Program Evaluation

• Initially, does not require a computer, statistics, a university education or any theories about life. All it requires are people committed to asking two questions of a non-profit CEO:– What are you trying to do here?– How’s it coming?And not accepting vague, abstract answers.