ALUMNI VOICES &

OUTCOMES-BASED

ASSESSMENT

WASC ARC APR IL 2014

Occidental College 2

Founded in 1887 Mission: Excellence, Equity, Community,

and Service Eagle Rock, CA Small, Private, Highly Selective,

Residential Liberal Arts College (FTE=2123)

31 Majors Faculty-Student Ratio: 10 to 1 Assessment

Why Alumni?3

Alumni Engagement

Institutional Learning Outcomes (post-Oxy)

Post-Graduation Success Time to Job Graduate School Lifelong Learning Connecting

Alumni and College-Level Assessment

College-Wide Assessment5

Direct & Indirect Evidence Direct Evidence

Senior Capstones & Comprehensives Course Level Student Work

Indirect Surveys Student Surveys (CIRP: TFS, YFCY, CSS, DLE) Faculty Surveys Alumni Surveys Focus Groups

Alumni Survey Schedule6

Summer 2013In-

House1 & 5 Year

Summer 2014HEDS

Survey1, 5, 10 Years

Summer 2015In-

House1, 5, 10, 15 Years

Summer 2016In-

House1, 5, 10, 15 Years

Summer 2017In-

House1, 5, 10, 15 Years

Summer 2018HEDS

Survey1, 5, 10 Years

Results 7

2013 Alumni Survey Findings In-House: 1 & 5 Years 50% response rates

Presented to Senior Staff & BoT

Shared and Presented to Department Chairs

ALUMNI SURVEY: Initial FindingsWhat is current (primary) status?

8

0%

20%

40%

60%

80%

32%

73%

0% 4% 2%

30%

73%

1% 5% 2%

Class of 2008 (n=207) Class of 2012 (n=180)

ALUMNI SURVEY: Initial FindingsIf employed, what is your employment?

9

0%10%20%30%40%50%60%70%80%90%

9% 10%

81%

3%30%

67%

Class of 2008 (n=137) Class of 2012 (n=118)

10

ALUMNI SURVEY: Initial FindingsIf enrolled in Grad School, What is your highest degree planned?

JD Masters MBA MD PhD Other/Misc

0%

10%

20%

30%

40%

50%

60%

9%

33%

4% 5%

49%

12%

47%

6%16% 18%

Class of 2008 (n=57) Class of 2012 (n=49)

ALUMNI SURVEY: Initial FindingsCLASS OF 2012 (Recent Graduates)

11

40% (31) had a job lined up before graduation 80% (62) had a job within 3 months of graduation 97% had a job within 6 months of graduation

Economic downturn (e.g. lack of entry-level jobs, competition with skilled applicants, etc.

Uncertain of career goals/focus

Limited professional network

Low response to applications (i.e. poor yield in securing interviews)

Lack of job search skills (e.g. unsure of how to launch productive job search, difficulty finding employment of interest, difficulty interpreting job qualifications, unsure of professional skills, etc.)

Insufficient relevant opportunities within geographic area of interest

Limited technical skills for position of interest

Offered employment, but not in area of interest

Insufficient interview skills (i.e. secured interviews but less successful in securing offers)

0% 20% 40% 60%

What, if any, challenges did you face in your job search?

Are you doing what you thought you’d be doing after graduation?

12

55%45%

Yes NoWhen I imagined post-graduate life, I imagined an entry-level (underpaid or volunteer) job in a psychology-related field while continuing to apply for graduate school and more education. In the job I have now, I work at one of the top children's inpatient psychiatric units in the country and I have more responsibility with a Bachelor's degree than I thought would be possible.

Currently, I'm just paying off student debt thanks to secure full-time employment at a "mom'n'pop" retail store. I hoped to stay in Los Angeles to pursue a career in music, but I didn't have the money to live independently out there.

I had no clue what I wanted to do.

I hoped to be working in a museum or other cultural history institution. I have realized that to secure a job in a museum I will need further schooling so plan to return to graduate school Fall 2014.

Question Acquired

Skills at Oxy

Applied Skills Post Oxy

TECHNICAL & HARD SKILLS (the ability to perform on-the-job & industry specific skills including languages, finance, software programs, design skills, lab skills, etc.)

57% 77%

MANAGEMENT & ADMINISTRATION SKILLS (the ability to analyze tasks, identify resource materials, delegate responsibility for task completion, motivate/lead people, organize people & tasks to achieve specific goals, etc.)

63% 69%

HUMAN RELATIONS & INTERPERSONAL SKILLS (the ability to maintain group cooperation & support, delegate tasks & responsibilities, persuade others, teach a concept or method, work across difference, work under time & environment pressures, etc.)

83% 63%

PERSONAL DEVELOPMENT & LEARNING SKILLS (the ability to develop personal growth goals, identify one's strengths & weaknesses, accept & learn from criticism, take risks, accept consequences of one's actions, etc.)

86% 59%

COMMUNICATION SKILLS (the ability to use various form of written communication, speak effectively to individuals & groups, use media to present ideas, effectively communicate value judgments, etc.)

92% 58%

INFORMATION MANAGEMENT SKILLS (the ability to sort data & objects, compile & rank information, apply information creatively to specific problems or tasks, evaluate information against appropriate standards, etc.)

78% 57%

DESIGN & PLANNING SKILLS (the ability to identify alternative courses of action, set realistic goals, follow-through with a plan/decision, manage time effectively, predict future trends/patterns, set priorities, etc.)

82% 57%

CRITICAL THINKING SKILLS (the ability to identify critical issues when making a decision or solving a problem, define the parameters of a problem, create innovative solutions to complex problems, etc.)

94% 50%

RESEARCH & INVESTIGATION SKILLS (the ability to identify problems & needs, design an experiment that systematically defines a problem, formulate relevant questions, identify appropriate information sources, etc.)

95% 41%

College-wide Alumni Engagement

14

Connect the Dots

Changing perceptions for Alumni

Investment Engagement Protect their meaning and quality of their degree

Support Outcomes-Based Assessment

Alumni & Program Review

16

Self-Study Components17

Program Overview Goal Achievement Curriculum Relevance Student Experience Program Resources Supporting Documentation

Exhibits: CVs, Course Syllabi, etc.

EXAMPLE: Psychology Self-Study for External Review Team

Example: Self-Study Psychology

18

Alumni Survey: Qualtrics & Facebook

Sample Questions

Results included in the self-study

Response Rate: 45%

Findings

Psychology Alumni Survey19

Psychology Alumni Survey Questions20

General Experience Questions Specific PLOs How well do you think the

department met these goals? Which specific experience (e.g., course,

research, lab, etc.) in the psychology major has had the most positive impact on you? Why?

If you had to select an aspect of your psychology major that needed the most improvement, which would it be and why?

Psychology Alumni Survey & PLOs21

Questions Mean

Emphasis on leadership skills 3.58

Emphasis on contributing to your community 3.95

Emphasis on oral communication skills 3.98

Structured opportunities for community service and for research 4.30

Opportunity for faculty mentoring 4.51

Emphasis on writing skills 4.65

Small class sizes 4.65

High availability of faculty for academic consultation (course work, advising) 4.65

High level of personal interaction between faculty and students 4.65

Opportunity for classroom discussion 4.70

High quality of courses in the major 4.77

High-quality teaching 4.91

Example: Education Department22

Lost Accreditation from CA Commission on Teacher Credentialing

Self-Study Triangulation of Data

Expert Panel called together by Education Dept Expert Panel called together by Administration Expert Panel called together by ALOED

Affected Future Programming Loss of credential Attempting to get a major Climbing “uphill”

Example: Program Review- English 23

Self-Study: Alumni in Graduate SchoolAlumni Survey and Focus Group

Graduating Seniors AlumniJobs, Graduate School, National Awards

Closing the Loop Align requirements for graduate programs

with senior capstone Honors Requirement Changes

Next Steps24

Connect with the Alumni Office Create opportunities for the Alumni

Office and Faculty/Departments/Colleges to collaborate in assessment

Create a culture of sharing results and collaboratively “close the loop”

Continue the momentum for the engaged alumni

Get Creative!