1

To: Seattle University Student Development Administration Program Faculty and Advisory Board Program Faculty

Dr. Jeremy Stringer, Program Director and Associate Professor, Seattle University Erin Swezey, Core Faculty and Internship Director, Seattle University

Advisory Board Bernie Liang, Director of Student Activities, Seattle University (Chair) Art Costantino, VP of Student Affairs, Evergreen State College (Member) Ana Blackstad, Director of Educational Planning and Services, Cascadia Community

College (Member) From: Kaitlin Ehlers, Nikki Robison, and Stephanie Weiskopf Date: December 1, 2011 Re: Assessment of SDA Program Learning Outcomes

Description and scope of project

We were charged with the task of assessing and evaluating the Student Development

Administration (SDA) program based on the eight learning outcomes (LOs): demonstrating

mastery of program knowledge base; understanding of students and student issues; responding to

ethical dilemmas utilizing appropriate judgment; understanding and fostering diversity;

communicating effectively, orally and in writing; working successfully in teams; demonstrating

commitment to profession and professional development; and adapting student services to

specific environments and cultures. These LOs were developed in 1994 to compliment the

Seattle University Graduate Programs LOs and have not been formally updated since 2004

(Jeremy Stringer, personal communication, October 24, 2011). As a group, we considered

materials given to us by faculty and feedback from both current and past SDA students about

their academic experience and the presence of the LOs throughout the SDA courses and

2

program. From our assessment and analysis we were able to identify some strengths,

weaknesses, and areas for further evaluation of the SDA LOs established by the program.

Methodology

The stakeholders identified for this project are current students of the SDA program,

alumni of the SDA program, SDA core faculty, and the SDA advisory board. These groups are

considered stakeholders because each group is connected to and interested in the SDA LOs as a

tool to guide and measure learning within the program. Current students are experiencing the

SDA program and courses in present time and are supposed to be exploring and experiencing

each LO through their studies. Those who have graduated from the program can give a historical

perspective of the program and the success at which the program has previously addressed LOs.

The faculty design and facilitate courses that should address certain areas of knowledge within

the identified LOs. Finally, the SDA advisory board will be conducting a SWOT (Strengths,

Weaknesses, Opportunities, Threats) analysis this year and will be able to use this information to

more thoroughly complete their analysis.

To assess the SDA LOs, we implemented a variety of methods. First, we reviewed the

course syllabi from required SDA courses that were provided by SDA faculty and the College of

Education (COE) faculty. We examined the portfolio element of the program using considered

judgment. We also compared the SDA LOs and skill clusters with the American College

Personnel Association (ACPA) and the National Association of Student Personnel

Administrators (NASPA) professional competencies. Lastly, we utilized web-based assessments

to collect data from current students and alumni of the SDA program.

3

Information Review

Syllabi Review. In review of the course syllabi from all SDA required courses we quickly

discovered that very few syllabi indicated which SDA LOs would be addressed in the course.

The only SDA courses that listed the LOs were those taught by Jeremy Stringer and SDAD 580

Higher Ed Law. We also determined that of the three required courses taken through the College

of Education, none of them included LOs of any kind. Overall, there was very limited insight in

to the effectiveness of the LOs based on the course syllabi. Our recommendation is that all SDA

course syllabi include the LOs addressed and that the course assignments show how they support

the desired LOs.

Portfolio Review. In our review of where students in the SDA program experience the

eight identified LOs, it became evident that portfolio is an element of the program that excels in

serving as an avenue for students to demonstrate their learning around all outcomes. By its

nature, portfolio requires students to examine and reflect on their learning and identify specific

examples that illustrate their understanding of each LO. As a student commented in the survey,

“learning outcomes aren’t addressed enough until portfolio.” This comment confirms the

purpose of portfolio in that it is an element of the program designed to evoke students’

understanding of all LOs and successfully does so. Portfolio is transitioning in fall 2012 from an

optional graduation requirement to a mandatory graduation requirement for every student,

ultimately strengthening SDA students’ experience with LOs.

ACPA/NASPA competencies and alignment with SDA Learning Outcomes. Before

portfolio is required for every SDA student, evaluation of the LOs for possible revision should

occur to ensure that the program is preparing students to meet and exceed expectations of

professionals in the field of student development. For this reason and because the LOs have not

4

been reviewed sine 2004, the SDA LOs and skill clusters were compared to the ACPA/NASPA

professional competencies in a document entitled Professional Competency Areas for Student

Affairs Practitioners ("Professional competency areas," 2010) to identify shared professional

expectations and any gaps present between the identified SDA LOs and ACPA/NASPA

competencies.

The ten competencies are: advising and helping; assessment, evaluation, and research;

equity diversity, and inclusion; ethical professional practice; history, philosophy, and values;

human and organizational resources; law, policy, and governance; leadership, personal

foundations; and student learning and development ("Professional competency areas," 2010). In

the document, each competency is described and provides three skill levels within each one:

beginning, intermediate, and advanced. Some graduate programs use these competencies as a

guide for determining learning outcomes and professional development within their program.

The comparison between SDA and ACPA/NASPA was mapped in a chart (see Appendix

C). The map was created with considered judgment as to where the professional competencies

intersect with the SDA LOs and skill clusters. The results show that the professional competency

with the most overlap is human and organizational resources. The competencies that did not have

any apparent overlap with SDA LOs or skill clusters are: assessment, evaluation, and research;

and law, policy, and governance. The SDA LOs and skill clusters that did not have any apparent

overlap with the competencies are learning LO7 (demonstrating commitment to profession and

professional development), LO8 (adapting student services to specific environments and

cultures) and skill cluster 3 (critical analysis and problem solving).

Based on this review, some recommendations have been identified. First, the SDA LOs

could be more clearly defined. It is difficult to assume what is meant by and included in the SDA

5

LOs. Current students support this concern as seen in the survey results with the comment, “I

hesitate to check learning outcomes because I don’t fully understand their meaning”. Second, it

would be helpful if the SDA LOs and skill clusters were combined into one synthesized

document. A great example of this is the ACPA/NASPA professional competency document

where every competency includes three skill levels: beginning, intermediate, and advanced. It

would be easy for SDA students to identify where their skill level is and establish goals to

advance their skills as a way to measure their learning in the competency area throughout their

time in the program. The third and last recommendation would be to better integrate the

following competencies into the SDA LOs and skill clusters: assessment, evaluation, and

research; leadership; and personal foundations. Assessment is a growing and sought after skill set

in the field of student development and there is no data to show that at any point in the program a

student encounters this competency unless they are a Master of Arts student or completing a

graduate project. The leadership and personal foundations competencies are Jesuit in nature,

capturing the holistic and individualistic needs of student development professionals. Better

reflecting on these competencies in the SDA LOs and skill clusters would serve students well.

Web-Based Survey. In order to truly evaluate the learning outcomes, our group felt that it

was necessary to survey two of our stakeholder groups: alumni and current students. Current

students were invited to participate in a web-based survey distributed via email. 41 students

responded to the survey. Additionally, we contributed two questions to a web-based survey that

was distributed to alumni of the SDA program. 43 alumni responded to the survey.

Current students: The survey (see Appendix E) asked students to provide feedback on

every required course of the SDA program in relation to the LOs and to submit open-ended

comments or suggestions about the SDA LOs. For each course, all eight LOs were listed and

6

students indicated which LO(s) they experienced. The survey allowed students to indicate if they

had not taken the course or if they had taken the course, but felt that no LOs were addressed.

When analyzing the collected data (see Appendix A), our group determined that percentages of

70% or more were to be considered practically significant. Essentially, LOs that proved to be

addressed at a “C average” were determined to be experienced at a significant level.

Appendix A reviews the survey results. Results show that LO5 (communicating

effectively, orally and in writing) is the LO most experienced in the classroom. In regards to

specific courses, those with the most LOs experienced are SDAD 578: Student Development

Theory (6/8 LOs), SDAD 576: Leadership and Governance (5/8 LOs), and SDAD 564-6

Internship in SDA (5/8 LOs). The courses with no significant experience of any LO are EDUC

500: Educational Research, EDAD 571: Leadership in Education I, and SDAD 580: Higher

Education Law. These are also the courses with the highest percentages of zero LOs experienced.

Other courses that had one or more respondents experience zero LOs are SDAD 559:

Community College, EDUC 501: Social Justice, and EDUC 515: Multicultural Perspectives. An

additional observation relating to courses is that every respondent that has fulfilled the EDAD

572: Leadership in Education II requirement has done so by substituting it for another course.

Alumni: The two questions in the alumni survey asked alumni to indicate which LOs

(LO1-8 - check all that apply) they experienced as a result of being in the SDA program and

where (coursework, internships, and professional experience - check all that apply) they

experienced each LO. Appendix B illustrates the results from the survey. The results showed that

LO2 (understanding students and student issues) is the most experienced and LO8 (adapting

student services to environment and culture) is the least experienced. The data regarding where

the LOs are experienced is comparable. Coursework scored the highest for 5/8 LOs, professional

7

experience scored the highest for 1/8 LO, one LO tied with coursework and professional

experience, and another LO tied with internship and professional experience. It is clear that LOs

are present for SDA students both in and out of the classroom and that all of these experiences

have an impact on students’ understanding of the LOs.

Further Recommendations and Examination

Our group is invested in the success, growth, and development of the SDA program.

Because of our investment, we have identified some recommendations and suggestions for

further examination in regards to the LOs.

Building upon a recommendation mentioned above, we recommend that the SDA

program consolidate the program’s LOs, skill clusters, and themes into one exhaustive document

for students. Having one location for students to visit for direction and guidance of desired

learning in the SDA program, as well as more thorough definitions of what the desired learning

is for each outcome, would be helpful and provide synthesis to students’ education. A great

model for this document is the ACPA/NASPA competencies. Encouraging learning that is

expected of student development professionals by ACPA and NASPA will make SDA students

more competitive in the job search and will make the SDA program more competitive nationally.

This will require a revision of the current SDA LOs.

With that, we think that any changes to the LOs, skill clusters, and themes need to happen

quickly in hopes of creating the most cohesive learning process for students before the next

academic year when portfolio is a requirement for all students.

For further examination, we believe that deeper and more evaluation is needed in regards

to the elective courses to ensure that elective courses are achieving the SDA LOs. Core faculty of

the SDA program should identify what LOs should be addressed in elective courses and prior to

8

approval of a student taking an elective course, the course plan or syllabus should be able to

describe the learning that will occur for the student in that course. These descriptions do not

need to be directly stated, but it should be clearly identified what the student will learn and how

it will support one or more of the SDA LOs.

Further work is needed in evaluating the survey data. The conclusions we made from the

data are not the only relevant findings. There are gaps and questions still to be identified and

answered. One example is identifying where the LOs are not showing up in courses that seem to

be a natural fit in providing specific learning to students. Additional questions may be, can we

re-evaluate courses to find opportunities to add LOs to the course content? Where do the LOs fit

naturally that do not appear in the data?

It should be noted that a limitation in our data collection is apparent in that we asked

current students and alumni to report on similar, but different questions. Current students

provided feedback on the LOs in relation to current and past coursework only. Alumni reported

on the more complete view of the SDA experience and LOs, including all courses and

professional experience. Clearly alumni have a relationship with the LOs through professional

experience and we did not ask current students to indicate their learning through professional

experience. While the surveys are comparable, they do not report on the exact same questions.

Further work could be done to collect data according to the identified limitation.

Elaborating on another recommendation previously mentioned, we suggest that every

SDA course syllabi include the SDA LOs and which LOs are intended for each course. However,

there is further work to be done to complete this recommendation. Although faculty are an

identified stakeholder group, we have not communicated with them to get their perspective and

feedback on the SDA LOs. Because the faculty members design the courses, they should be

9

brought in to the evaluation of the program LOs. It is recommended that faculty members

provide feedback as to what LOs they feel are addressed in their courses and also what LOs they

desire to address in the future.

It would be ideal if all SDA required courses included the SDA LOs in the syllabi and

design, even COE courses and electives. Implications and/or further examination of this project

pose the questions, what would collaboration look like with COE in regards to LOs? When

would it happen and how?

Group Reflection & Integration of Capstone Reading

As a result of this project, we learned a lot and found the information that we gathered

about the SDA LOs to be very interesting. It became evident to us why this project is so

important and we were excited to be producing a product that will contribute to the development

and improvement of the SDA program. Founding our initial interest came from An Assessment

of Skills and Competencies Necessary for Entry-Level Student Affairs Work by Jeffery Waple

(2006):

The most emphasized area of professional preparation deals with building skills

and competencies necessary to be successful in the field. Designing preparation

programs that will result in students’ possessing the knowledge and skills that

entry-level student affairs staff will need to carry out their respective jobs is the

charge of academic program faculty and student affairs practitioners. (Waple,

2006, p. 2)

Additionally, part of our approach to this project was guided by the ACPA and NASPA

Professional Competency Areas for Student Affairs Practitioners (2010) article. As we began

thinking about recommendations, we used the ACPA and NASPA identified competencies as a

10

reference point, “graduate student affairs program faculty may choose to use this document to

develop or refine their curriculum to better address the competencies expected of practitioners in

the field” (Bresciani & Todd, 2010, p. 5).

Working together as a group was a very rewarding experience. We have built trust and

camaraderie over the past year in the program, which demonstrates LO6 (working successfully in

teams). This project allowed us to connect our work to the LOs of the program including LO5

(communicating effectively, orally and in writing). Our communication skills have developed

throughout the program and were highlighted in this project. We were able to communicate

successfully as a team.

On a more personal level, as a team, we found that throughout this process our

investment in the project deepened. Each of us truly values our experiences both inside and

outside the classroom through the SDA program and hope that our work on this project becomes

a legacy for this program. We became more invested in the success of our program and would

love to see it grow to be a more notable and recognized program in the field of student

development.

However, it was difficult to become aware that our individual frustrations as students in

the program were also felt by some of our classmates. Hopefully now that this data has been

collected and presented, strides will be made to rectify some of the gaps in the LOs and

consideration will be given to establishing a presence of intended LOs in SDA courses.

As a group we spent quite a bit of time reflecting throughout the course of this project

and it would not be an honest reflection if we did not mention our frustration at times with this

project simply because we were continually reminded that none of us have a background in the

CAS standards and how to conduct relevant assessments. Although this project stretched us and

11

gave us the opportunity to engage with assessment, we felt at times, aside from the assigned

reading in Capstone, that we were lacking a significant and foundational understanding of how

best to gather and report on data. Separate of the recommendations we made about learning

outcomes, we recommend that there be a focus to some degree within the SDA program on

assessment because of its growing importance within student affairs as professionals continue to

cement the field’s importance in higher education.

Overall, our group worked successfully together. We all contributed equally to the

project and are proud of the completed product.

12

Appendices

A: Current Student Survey Results

B. Alumni Survey Results

C. ACPA/NASPA Chart

D. Syllabus Review

E. Survey questions

13

A. Current Student Survey Results

14

15

B. Alumni Survey Results

16

C. ACPA/NASPA Chart

17

D. Syllabus Review

18

E. Survey questions

See attached printed version.

19

References

American College Personnel Association and National Association of Student Personnel

Administrators, (2010). Professional competency areas for student affairs practitioners.

Retrieved from website: http://www.naspa.org/programs/prodev/Professional_

Competencies.pdf

Bresciani, M. J., & Todd, D. K. (2010). Acpa and naspa professional competency areas for

student affairs practitioners.

Waple, J. N. (2006). An assessment of skills and competencies necessary for

entry-level student affairs work. NASPA Journal, 43(1), 1-18.