Embed Size (px)
Citation preview
Counseling & Psychological Services
MORE THAN SATISFACTION:ASSESSING CLINICAL
OUTCOMES IN COUNSELING CENTERS
International Assessment and Retention Conference
New Orleans, LAJune 13, 2009
Counseling & Psychological Services
Today’s PresentersJacqueline Alvarez, Ph.D.
Director of Counseling and Psychological Services, Oregon State University
CAPS Assessment Committee Member
Beth Dyer, B.A.
Information Technology Expert at Counseling and Psychological Services, Oregon State University
CAPS Assessment Committee Chair. Student Affairs Assessment Council Member
Michele Ribeiro, Ed.D.
Staff Psychologist at Counseling and Psychological Services, Oregon State University.
CAPS Assessment Committee Member. Student Affairs Assessment Council Member
Counseling & Psychological Services
Other Contributors
Shailagh Clarke, Ph.D.
Staff Psychologist at Counseling and Psychological Services, Oregon State University
CAPS Assessment Committee Member
Kimberly Miller, Ph.D.
Pre-doctoral Intern at Counseling and Psychological Services, Oregon State University
CAPS Assessment Committee Member
Today’s Objectives1. Explore how assessment can be used for learning
outcomes for clinical services.2. Identify ways to overcome barriers to clinical
assessment common to counseling centers.3. Become familiar with a successful model of
clinical outcome assessment that produces data that can influence client treatment.
4. Identify at least one learning outcome that you can assess at your home institution.
5. Identify strategies for implementing a comprehensive clinical outcomes assessment program.
How We Got StartedThe University Began to Require
Effective Assessment
It became important to demonstrate:
1. How our work aligned with the mission of the university
2. How counseling helps retain students
3. How counseling helps student matriculate
How did CAPS respond to the push to assess our services? Two staff members began developing assessment
instruments for Counseling and Psychological Services. We began with simple client satisfaction surveys.
OSU Student Affairs Assessment Council was established six years ago, offering a place for all offices to learn together, try new tools, and to work collaboratively.
CAPS staff members joined the Student Affairs Assessment Council, and established an Assessment Committee within CAPS.
This year, three new members joined the CAPS Assessment Committee, infusing new energy and ideas.
Barriers to Doing Clinical Assessment Well
Staff Resistance to Assessment
Lack of Expertise within Staff regarding Program Evaluation and the Assessment of Learning Outcomes
Limited Time and Resources
Overcoming Barriers to Clinical Assessment
Staff Resistance• Make assessment data meaningful to staff
members
• Link assessment measures to agreed upon mission and goals
• Facilitate staff participation
• Reward participation
Overcoming Barriers to Clinical Assessment, Continued
Lack of Expertise in Program Evaluation and Identification of Learning Outcomes• Use a committee model• Provide training and continuing education
opportunities• Invite members outside the agency who have
more expertise (e.g. a faculty member in your math/statistics department)
• Recognize that assessment is an ongoing process and that the tools will develop over time
Overcoming Barriers to Clinical Assessment, Continued
Limited Time and Resources• Create a standing time to do
assessment work• Add assessment responsibilities to
some staff members’ job descriptions, and take away other responsibilities
So, What Have We Done?
Methodology
Administered a 43 item self-report survey to students using CAPS services in the 6th and 7th weeks of each 10-week term for three terms.
Revised and updated survey regularly. Hence, the method does not change but the content does.
Information About Our Sample
Data from one term (Winter 2009), sample of convenience
N=126 with a 51% return rate
In which session did you start to see improvement/change occur?
Count Percentage
First/Intake 12 13%
2-5 60 67%
6-10 7 8%
11 or more 3 3%
No change occurred/No change yet
8 9%
How the Content of Our Assessment Has Developed
Began with satisfaction surveys
Moved to assessing learning outcomes
Now we assess the relationship among:o Student Goalso Student Learning Outcomeso Content of Counseling Sessions
Three Areas of Assessment
Stress and Mood Management
Personal and Relationship Skills
Academic Success
Stress and Mood Management
• Overall self-care• Positive sleep habits• Eating habits• Regular exercise• Overall mood• Coping with mental and emotional
difficulties
Personal and Relationship Skills
• Communication skills• Relationships with others• Identifying and expressing what I need• Relationships with my partner• Making relationship decisions• Insight into my personality• Self-esteem• My level of connection to the OSU
community
Academic Success
• Time management• Motivation• Test anxiety• Attending classes• Study skills• Concentration• Procrastination• Effectiveness as a student
The assessment revolves around three
questions we ask about the previously
identified areas
First Question
“It was important to me to experience change in…”
Rated on a 5-point Likert scale, ranging from “Very True” to “Very Untrue”
Second Question
“Please describe changes or growth, if any, since you began
counseling in…”
Rated on a 5-point Likert Scale, ranging from “Much Improved” to “Much Worse”
Third Question
“Which of the following issues did you discuss in counseling?”
(Check all that apply)
Rated as endorsed or not endorsed
Results
Slide 1: Graph illustrating…% of clients who expressed a desire to
experience change in that particular areaAND
% of clients who said the area of focus was discussed in counseling
Slide 2: Graph illustrating…% of clients reporting each level of improvement
Slide 3: Results summary statements
Stress and Mood Management:Coping with Mental and Emotional Difficulties
Are topics that clients say are important to change being discussed in counseling?
Important to Experience
Change
Discussed in Counseling
0
40
80
94 90
Per-
cen
tag
e
Coping with Mental and Emotional Difficulties
Overall, how much improvement did clients report?
0
20
40
60
21
53
19
5 2
Per-
cen
tag
e
Results for Coping with Mental and Emotional Difficulties
94% indicated that it was important to experience change
74% indicated it was important to experience change in this area and reported improvement
20% wanted change, but did not experience change
Personal and Relationship Skills: Communication Skills
Are topics that clients say are important to change being discussed in
counseling?
Important to Ex-perience Change
Discussed in Counseling
0
20
40
60
80
10067
49
Perc
en
tag
e
Communication Skills
Overall, how much improvement did clients report?
0
20
40
60
13
4837
2 0
Per-
cen
tag
e
Results for Concentration67% indicated that it was important
to experience change54% indicated it was important to
experience change in this area and reported improvement
13% wanted change but did not experience change
7% did not indicate that it was important to change, but did experience positive change
Academic Success: Concentration
Are topics that clients say are important to change being discussed in
counseling?
Important to Ex-perience Change
Discussed in Counseling
0
20
40
60
80
10067
30
Perc
en
tag
e
Concentration
Overall, how much improvement did clients report?
-10
10
30
50
70
12
36 38
122
Perce
nta
ge
Results for Communication67% indicated that it was important
to experience change45% indicated it was important to
experience change in this area and reported improvement
22% wanted change but did not experience change
3% did not indicate that it was important to change, but did experience positive change
Important to Change as related to Discussed in Counseling
Change desired and discussed Change desired, not discussed
Relationships with others
Coping with mental/emotional difficulties
Expressing needsCommunication skillsOverall mood
Self-careSelf-esteemSelf-motivationEffectiveness as a
studentConcentration
General Conclusions
By looking at the relationships among the variables (client goals, counseling content, and client change), we get more meaningful information than simply asking, “Did you improve?” or “Did counseling help you stay in school?”
Some specific things we’ve learned through our assessment processStudents come to counseling with all kinds of hopes,
expectations and goals
Students’ personal and relationship skills and ability to manage mood improve at higher frequencies than academics
More students report greater change in mood management than academic skills
Students report that they talk about mood, personal and relationship skills at higher frequencies than academics
Clinical Uses of Assessment Data
The thing that makes assessment compelling is that, when done well, it can give us good information about our work.
If the data are meaningful, we are often moved to begin exploring how we can do things more effectively to better meet students’ needs.
Clinical Implications
We’re doing very good work in helping students positively change their mood, personal skills and relationship skills.
Clinical Implications Continued
Counselors may need to assess students’ goals and expectations more completely, especially regarding students’ hopes for academic change.
Clinical ImplicationsContinued
Counselors may need to more overtly discuss academic performance throughout counseling sessions.
Clinical ImplicationsContinued
Counselors may need to make explicit to student the relationships among the variables.
Next Steps
Clinical Service Assessment
Outreach and Program Assessment
Training of pre-doctoral interns and other trainees
How to improve your own clinical assessment
• Work with other people!
• Begin to identify learning objectives.
• Explore alternative factors that affect learning and may warrant assessment.
Let’s design one learning outcome that you could effectively assess at your institution.
Questions?
Contact Information
Oregon State UniversityCounseling and Psychological
Services500 Snell HallCorvallis, OR 97331541.737.2131http://oregonstate.edu/counsel/
