SLO & SAO WorkshopSLO & SAO Workshop

Dr. Deborah DiThomas

Interim Vice Chancellor

Student Services & Operations

Cuesta College Student Services Managers

Dr. Edward C. Bush

Vice President, Student Services

SLO 10 Commandments

1. Acknowledge the SLO/SSO and keep it holy.2. Thou shall not do a SLO/SSO in isolation.3. Thou shalt make SLOs/SSOs relevant. 4. Thou shall not use the word data in vain.5. Thou shall not bite off more than one can chew. 6. Thou shall not covet your neighbor’s SLO/SSO. 7. Thou shall not keep your SLO/SSO on the shelf.8. Thou shall not ever become complacent. 9. Thou shall not kill, hurt, or injure anyone during the

SLO/SSO process, 10. but cursing is okay.

History of RCCD Program Review 05-06 :

Spring 05: New Standards/SLO TrainingDraft Program Reviews

Spring 05: Student Services Retreat Review Accreditation StandardsRevise SLO’sRevise SAO’sWrite Evaluation Plans

Summer 05/Fall 05: Finalize Program Reviews including SAO’s, SLO’s and Assessment Plans

Spring 06: Implement “Assessment Plans” for SAO’s & SLO’s

Summer 06: Begin ‘06 Program Reviews

Better understanding of campus programs

Make our SLO’s and SAO’s more singular—don’t bundle

Pick data that can be measured

We were too broad

Chose one or two specific things to work on

Challenged in serving students at the same time we were assessing students

Write more measurable objectives

See things through the eyes of students

Get staff involved

Follow the template it gives direction

Focus and prioritize

Focus of how we state SLO’s and SAO’s

Need a way to capture data

What We Learned:

Keeping good records so at end it was easy—use timelines

Organization of how we did things—using timelines

Worked well together—shared responsibilities

To truly serve students it takes one on one

Education and dissemination of safety information to college community

Documented needs

Completing the entire process makes us better for next year

Comfort level is higher

Ahead of academics and using us as example

Teamwork

Need to communicate changes to students

Get staff involved

Instruments to measure

Increased communication

What We Did Well:

We will do differently:

Get more input from hourly staff

Don’t wait until end to do evaluation

Better timelines and stick to the timelines (monthly calendars…)

Get all members of department on board

Improve communications with students

Identify specific people to do certain things

Include communication with academic side

More campus specific SLO’s SAO’s

Change format of last year

Defining SLOs & SSOs

Student Learning Outcome - Defines what students should be able to do, think, or know.

Service Area Outcome - Defines how the service area performs.

Defining Outcomes

Detailed, specific, measurable or identifiable, and personally meaningful statements that arederived from the goals and articulate what the end result of a unit, program, course, activity, or process is.

Outcomes should be:• Measurable (not necessarily, countable), or

observable,• Manageable, and• Meaningful.

Standard IIB – WASC

The institution recruits and admits diverse students who are able to benefit from its programs, consistent with its mission. Student support services addresses the identified needs of students and create a supportive learning environment. The entire student pathway through the institutional experience is characterized by a concern for student access, progress, learning, and success. The institution systematically assesses student support services using student learning outcomes, faculty and staff input, and other appropriate measures in order to improve the effectiveness of these services.

What Does This Mean?

The new standards promote the values of Student Services.

Student Service professionals are leaders in providing responsive services to a diverse and changing student population.

The learning outcomes challenge to Student Services is an opportunity to improve effectiveness and to integrate with instruction.

Standard IIB – continued

……The institution systematically The institution systematically assesses student support servicesassesses student support services using student learning outcomes, using student learning outcomes, faculty and staff input, and other faculty and staff input, and other appropriate measures appropriate measures in order to in order to improve the effectiveness of these improve the effectiveness of these services…services…

Effectiveness is the Ultimate Goal

Learning outcomes focus on what students know and can do rather than what we do to support them.

Shift the paradigm from Student Satisfaction to Student Learning.

Student affairs needs to demonstrate effectiveness thru data.

Campus staff learn how student affairs contributes to student learning.

Perhaps the key question to be asked of student affairs personnel might be expressed as follows…

“Of those students who use the services, programs and facilities, is there any effect on their learning, development, academic success or other intended outcomes, particularly when compared to non-users?”

It is important to note that not all outcomes of interest to student services are learning outcomes; many pertain to the timely, efficient and effective delivery of services.

Source: Upcraft, M. Lee and Schuh, John. Assessment in Student Affairs: A Guide for Practitioners. Jossey-Bass, San Francisco, 2000. ISBN 0-7879-0212-8

Service Area Outcomes Based onCurrently Existing Services

Name of Unit will provide Name of Current Services

improve

decrease

provide

Client will be satisfied with Name of Current Service

Formulating

Service Area Outcomes (SAOs)

SSA Service Verb + Objective

Tutoring Improve…

Students attending Academic Advising will Increase…

Workshops Understand…

Counseling Sessions Know….

“Students participating in the development of a student educational plan will understand the educational requirements for their ed goal and major.”

Formulating Student Learning Outcomes (SLOs)

Student Service Areas

Checklist: Student Learning Outcomes Consistent with Mission Statement. Agreement ~ were staff involved in the

development? Reasonable ~ for the ability of the students. Measurable ~ can be observed, tested. Key Concepts ~ important to the service area,

program. Clarity ~ precise in description. Singular ~ not “bundled” (shown by use of

conjunctions, commas).

Critique & Correct RCC Outcomes

Group Task: Let’s Look at SLOs and SAOs

R I V E R S I D E C O M M U N I T Y C O L L E G E Moreno Valley Campus Norco Campus Riverside City Campus

Manager: ____________________ Current Service Area Outcome:

Checklist: Service Area Outcomes (SAO’s)

Consistent with Unit Mission Statement. Agreement ~ were staff involved in the development? Reasonable ~ for the ability/staffing of the service area. States what “will” be ~ provided, improved, increased etc. Measurable ~ can be observed, tested. Key Concepts ~ important to the program/service area. Clarity ~ precise in description. Singular ~ not “bundled” (shown by use of conjunctions, commas). Revised Service Area Outcome:

R I V E R S I D E C O M M U N I T Y C O L L E G E Moreno Valley Campus Norco Campus Riverside City Campus

Manager: Current Student Learning Outcome: Checklist: Student Learning Outcomes (SLO’s)

Consistent with Mission Statement. Agreement ~ were staff involved in the development? Reasonable ~ for the ability of the students. Measurable ~ can be observed, tested. Key Concepts ~ important to the course, program. Clarity ~ precise in description. Singular ~ not “bundled” (shown by use of conjunctions,

commas). Revised Student Learning Outcome:

Part II:Navigating

Methods for Assessing Student Learning Outcomes in Student Services

The Assessment Process

4. Utilize Resultsfor Improvement

3. Interpret Findings 2. Gather Evidence

1. Articulate Goals

Good Assessment Techniques

ValidReliableActionableEfficient and cost-effectiveEngaging to students and others Interesting to faculty and other

stakeholdersTriangulated

Types of Assessment

Direct assessments- provide students the opportunity to show what they know. They “…prompt students to represent or demonstrate their learning or produce work so that observers can assess how well students’ texts or responses fit institutional or program-level expectations.” (Maki, p. 88)

Indirect assessments -“…capture students’ perceptions of their learning and the educational environment that supports that learning, such as access to and the quality of services, programs, or educational offerings that support their learning…By themselves, results of indirect methods cannot substitute for the evidence of learning that direct methods provide. They can, however, contribute to interpreting the results of direct

methods…” (Maki, p. 88, 89)

Types of Assessment (cont)

Formative Assessment-is designed to assist the learning process by providing feedback to the learner, which can be used to identify strengths and weakness and hence improve future performance. Formative assessment is most appropriate where the results are to be used internally by those involved in the learning process (students, teachers, curriculum developers).

Summative Assessment-is used primarily to make decisions for grading or determine readiness for progression. Typically summative assessment occurs at the end of an educational activity and is designed to judge the learner’s overall performance. In addition to providing the basis for grade assignment, summative assessment is used to communicate students’ abilities to external stakeholders, e.g., administrators and employers.

Types of Assessment (cont)Qualitative research - uses open-ended questions to gain an in-depth

understanding of the questions being explored. Common qualitative techniques include focus group discussions, mini-groups, and in-depth interviews either face-to-face or via telephone.

Quantitative methods - are distinguished by their emphasis on numbers, measurement, experimental design, and statistical analysis. Researchers typically work with a small number of predetermined response categories to capture various experiences and perspectives of individuals. Often emphasis is on analyzing a large number of cases using carefully constructed instruments that have been evaluated for their reliability and validity (Patton, 1990). Techniques include questionnaires, structured interviews, and tests.” (Palomba and Banta, p. 337)

Both qualitative and quantitative methods bring valuable information to light, but they yield different types of results. Review the type of information needed when selecting a method, and also consider whether qualitative and quantitative research could be used together.

Common Assessment Methods

Performances and Staff Observations

– Presentation, or demonstration Surveys

– Attitudes and perceptions of students and staff Narrative

– Staff and student journals, interviews, focus groups Cumulative

– Portfolios and capstone projects Database-Tracked Academic Behavior

– Grades, graduation, lab usage, persistence

Your Charge: Lets Practice Assessment

Your Task…

Work with the members of your service area to:

Develop an assessment plan:– Consider types of evidence.– Utilize existing assessment tools if appropriate.– Consider partnering with other student service areas.– Close assessment loop.

Create an annual timeline for assessment.

Delineate people responsible for the various segments of the assessment plan.

RCCD Student Services Assessment of SLO’s and SAO’s

1. Decide on how to assess your SLO’s & SAO’s Can have more than one assessment for any of the

SLO’s or SAO’s

Consider assessment tools that are easy for your area to use

Create own assessment tools, if necessary

Decide where the assessment will happen

Decide who will conduct the assessment

Decide when it will be conducted

Checklist: Means of Assessment

Specific means of measurement identified. Overall score, rating or response level set. Sub-score minimum of a lower scale in any

given area established. Number of students/clients who meet score

given. Administration when and by whom defined. Results judged on what basis stated. Results judged by whom stated.

R I V E R S I D E C O M M U N I T Y C O L L E G E Moreno Valley Campus Norco Campus Riverside City Campus

Manager: Means of Assessment: Checklist: Means of Assessment

Specific means of measurement identified. Overall score, rating or response level set. Sub-score minimum of a lower scale in any given area established. Number of students/clients who meet score given. Administration when and by whom defined. Results judged on what basis stated. Results judged by whom stated. Revised Means of Assessment:

ASSESSMENT WORKSHEET

STUDENT SERVICE AREA: OBJECTIVE:

Student Learning Outcome/ Service Area Outcome Audience

Circumstance/Condition Intervention

Benchmark data (if applicable)

Significant Degree (target and performance level)

Data Collection: Assessment methods and tools

Persons (s) Responsible

Timeline for data collection

SLO/SAO Data Analysis Recommended Improvement

1.1)

Student Learning Outcome/ Service Area Outcome Audience

Circumstance/Condition Intervention

Benchmark data (if applicable)

Significant Degree (target and performance level)

Data Collection Assessment methods and tools

Person (s) Responsible

Timeline for data collection

SLO/SAO Data Analysis Recommended Improvement

Definitions

• Student Learning Outcome - Defines what students should be able to do, think, or know.

• Service Area Outcome - Defines how the service area performs.• Audience - Who is your program/service targeting?• Circumstance/Condition Intervention - Where or under what

circumstances/ conditions will the learning occur?• Benchmark data - Data that serves as a standard by which others may be

measured that serves as a basis for evaluation or comparison• Significant Degree - Target performance level that states how much

learning will be enough?• Data Collection – What assessment methods and tools will be used to

measure learning or performance?• Person (s) Responsible – Identify titles of persons responsible for carrying

out SLO/SAO.• Timeline for data collection – Set a goal for data collection.• SLO/SAO Data Analysis – Summarize the outcome(s) of the study.• Recommended Improvement – How will you use the outcome(s) for

service area improvement?

Spring

Implement Assessment Plans (for SAO’s & SLO’s)

Implementing Assessment Results

Discuss and disseminate

Make appropriate program improvements

Integrate with program review and strategic planning process

RCCD Student Services Assessment of SLO’s and SAO’s

Assess the objectives and collect the data: Conduct the Assessment Organize the data Summarize your findings Disseminate results

Decide what to do next based on the findings Happy with the results?

You congratulate yourself and select another SLO/SAO to measure…

Not happy with the results? What can you do in your area to make changes?

Thank YouThank You&&

Good Good Luck!Luck!