Tia A. Minnis Assistant Director for Assessment Office of Planning Assessment and Research

2013 TAM | OPAR

A faculty Driven Process

Student Learning Outcomes (SLOs) Are and Are NOT!

Learning Outcomes at Different Levels. Defining Assessment of SLOs.

Why Do We Assess Student Learning? Faculty Role in Developing SLOs. Steps in Developing & Assessing SLOs.

Group Activity. Questions, Comments, Conversation.

2013 TAM | OPAR

Statements that define the goals of the learning experiences.

What a student should be able to do, know, or value/think as a result of engaging in that learning experience. Expressed as knowledge, skills, and attitudes or values.

Define impact – how the student has changed.

2013 TAM | OPAR

The same as satisfaction with education, professors, facilities, activities, or experiences

The same as outputs, such as grades, GPA, retention rate, graduation rate, etc.

2013 TAM | OPAR

Why are grades not enough? Grades can be subjective

Grades may reflect student behaviors (class participation, attendance, cooperation, etc.)

Grades may be inconsistent between instructors

Grades are not enough to say the we know that the students learned

Student Achievement is not the same as student learning

2013 TAM | OPAR

2013 TAM | OPAR

•Students who complete this course can calculate and interpret a variety of descriptive and inferential statistics

Course Level

(CLOs)

• Students who complete the Psychology program can use statistical tools to analyze and interpret data from psychological studies

Program Level

(PLOs) • Students who graduate

in business can apply quantitative methods to business-related problems

School Level (SLOs)

•Graduates from our campus can apply quantitative reasoning to real-world problems.

Institutional Level (ILOs)

The systematic collection, review and use of information about educational programs undertaken for the purpose of improving student learning and development.

-Ted Marchese

Assessment is a systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions that affect student learning.

-Barbra E. Walvoord, 2010 2013 TAM | OPAR

Assessment is the ongoing process of: Establishing clear, measurable expected outcomes of

student learning Ensuring that students have sufficient opportunities to

achieve those outcomes Systematically gathering, analyzing, and interpreting

evidence to determine how well student learning matches our expectations

Using the resulting information to understand and improve student learning

-Linda Suskie, 2009

2013 TAM | OPAR

Establishing measurable SLO

Provide Sufficient Learning

opportunities

Systematically Gather, Analyze, & Interpret Evidence

of Student Learning

Using The Resulting to

Improve Student Learning

2013 TAM | OPAR | CAU

On-going Process

Regional Accreditation (SACS COC) Core Requirements: 2.5 and 2.12 Comprehensive Standards: 3.3.1.1 and 3.5.1 Federal Requirements: 4.1

School and Program Accreditation SAS: NASPA, ABET, ACEJMC, APA SOB: AACSB, SOE: NCATE, CACREP, SSW: CSWE

2013 TAM | OPAR

Student Success is our Highest

Priority

2013 TAM | OPAR

Improve the quality of student learning experience Curriculum assessment and change Evaluation of instruction Effective use of resources Improved Communication Advising Tools Improving promotional materials Program Validation and Credibility Provide accountability to stakeholders Targets for assessment and accreditation Institutional Effectiveness

2013 TAM | OPAR

We assess SLOs for two primary reasons: Improvement - Assessment can improve the quality on

teaching learning, program and services, and planning and decision making.

Accountability – Assessment can validate current programs, services, and teaching and learning efforts and thereby secure continued support for them by demonstrating their effectiveness to concerned audiences.

Source: adapted from Linda Suskie (Assessing Student Learning, 2009)

2013 TAM | OPAR

Instructor-focused Teaching

to Student-focused

Learning

2013 TAM | OPAR

Collaborative Faculty-Driven Process Faculty Role: Identify student learning outcomes Identify the Means of Assessment (Tool, Method,

& Criteria) Analyze results, determine appropriate

improvements, and write annual assessment reports.

2013 TAM | OPAR

Faculty Role Hindrance to Participation

Insufficient Time Compromised Autonomy Meaningless – Data gather that no one will use Complicated Process

Utilization-Focused Evaluation (Paton, 2008) Evaluation judged by utility and actually use (Useful and Used) Focus on intended use by intended users Collaborative effort by the users for their intended uses Evidence need to identify program strengths and areas of

concern

2013 TAM | OPAR

Faculty Benefits: Shared understanding and commitment

Inform Pedagogical Practices Changes to the curriculum, requirements, programmatic

structure, etc.

Secure Needed Resources Changes to funding, planning, and policies that support

student learning

Faculty Development

2013 TAM | OPAR

WHEN DEVELOPING YOUR STUDENT LEARNING OUTCOMES REMEMBER:

Student learning outcomes (SLOs) are statements that specify what students will know, be able to do or demonstrate and think or value, after successfully completing the program/curriculum. Focus on Mastery of Program Content.

Ask yourself: what are the most important things a student should know, be able to do or demonstrate and think or value after completing my program?

Relate the SLO to the overall goal and objectives of the academic program.

SLO are expressed as knowledge, skills, and attitudes or values.

2013 TAM | OPAR

WHEN DEVELOPING YOUR STUDENT LEARNING OUTCOMES REMEMBER:

SLOs specify an action by the student that must be observable (able to be demonstrated) and measurable!

SLOs serve as benchmark assessments for completion of your

program! SLOs are related to the mission of the program or department,

school, and institution. Identify 5-8 student learning outcomes.

2013 TAM | OPAR

THE SLO’S SHOULD INCLUDE THE FOLLOWING:

An Action Verb that results in overt behaviors that can be observed and measured.

Sample action verbs are:

Analyze, Apply, Argue, Arrange, Assemble, Assess, Calculate, Categorize, Choose, Classify, Compare, Compile, Compute, Create, Criticize, Critique, Defend, Define, Demonstrate, Describe, Design, Develop, Differentiate, Discuss, Distinguish, Estimate, Examine, Explain, Formulate, Identify, Illustrate, Indicate, Interpret, Label, List, Locate, Manage, Memorize, Order, Operate, Organize, Plan, Practice, Predict, Prepare, Propose, Question, Rate, Recognize, Repeat, Report, Reproduce, Review, Revise, Schedule, Select, Solve, State, Translate, Use, Utilize, Write

Avoid verbs that are unclear and call for covert, internal behavior which cannot

be observed or measured. Appreciate, Know, Learn, Understand, Become aware of, Become familiar with….

2013 TAM | OPAR

The verb used may vary depending on the domain of learning and the appropriate level of thinking

2013 TAM | OPAR

Bloom’s Taxonomy 1956 Anderson and Krathwohl’s

Taxonomy 2000

2013 TAM | OPAR

THE SLO’S SHOULD INCLUDE THE FOLLOWING:

Language that indicate the BIG PICTURE rather than nuts and bolts. Does the language describe what students can do, know, think or

value as a result of their matriculation (observable/demonstrable)?

Does it ask students to apply what they’ve learned by producing something?

Does it address student competency rather than content coverage?

2013 TAM | OPAR

Each SLO should be S.M.A.R.T • Specific • Measurable (Result-Oriented) • Attainable (Realistic) • Relate to the mission (Relevant) • Time-bound

2013 TAM | OPAR

Each SLO should be S.M.A.R.T

The SLO should be Specific to the program The SLO’s should be specific to the program. Include in clear and

definite terms the expected abilities, knowledge, values and attitudes a student who graduates from the program is expected to have.

The SLO should be Measurable and result oriented

The intended outcome should be one for which it is feasible to collect accurate and reliable data.

The intended outcome should produce results that can identify problems and assist with program improvement.

2013 TAM | OPAR

Each SLO should be S.M.A.R.T

The SLO should be Attainable and realistic Consideration was given to the available resources (e.g., staff,

technology, assessment support, institutional level surveys, etc.) in determining whether the collection of data for each student learning outcome is a realistic expectation.

There are a realistic proportion of students able to achieve the stated specific level (e.g., 80% of the students will pass the written portion of the comprehensive exam with a minimum of 75 points on the assessment rubric).

2013 TAM | OPAR

Each SLO should be S.M.A.R.T

The SLO should be Related and relevant to the department’s/schools’ mission It should focus on intended outcomes that are critical to the

program and its mission statement.

The SLO should be Time-bound When defining the outcomes, it is important to indicate when the

outcome will be assessed (e.g. each semester, spring, annually, etc.), and for what period of time.

2013 TAM | OPAR

WHEN DEVELOPING YOUR STUDENT LEARNING OUTCOMES USE:

Appropriate Sources for SLOs Mission statements Written descriptions of the program Employers Your “ideal” graduate Teaching goals inventory Accreditation agencies Professional/certification exams

Established Baseline Data Set goal that are attainable/realistic

2013 TAM | OPAR

When writing student learning outcomes try using templates similar to these: Students will be able to ___________ (ACTION VERB- know, do,

value) _______________ (CONTENT AREA).

As a result of students participating in ________ (PROGRAM), they

will be able to _____________ (ACTION VERB- know, do, value) _________________ (CONTENT AREA).

______________ (PROGRAM) seniors will be able to

_______________ (ACTION VERB- know, do, value) _____________ (CONTENT AREA).

2013 TAM | OPAR

WHEN ASSESSING YOUR STUDENT LEARNING OUTCOMES:

Use Multiple Measures or Means of Assessment: Direct Assessment Methods as the primary MOA Indirect Assessment Methods as the secondary MOA

At least 1 direct measure must be used Qualitative/Quantitative Formative/Summative

2013 TAM | OPAR

Direct Measures (all students)

– Capstone projects/senior projects

– Samples of student work (Portfolios)

– Examinations – Oral Presentation – Observations of student

behavior (Internships) – Theses/Dissertations – Pre- and post-tests

Indirect Measures (sample)

– Alumni, employer, and student surveys

– Questionnaires – Focus groups – Interviews

2013 TAM | OPAR

Appropriate measures of Learning Outcomes include:

WHEN ASSESSING YOUR STUDENT LEARNING OUTCOMES :

Develop Scoring Guide/Rubric Identify what competencies are assessed Identify how scores are assigned to each competency Ensure that appropriate assessment measures are used to assess

the degree to which learning is attained

Identify the Criteria for Success Overall Passing Score (NOT Grade) % of students expected to achieve passing score

2013 TAM | OPAR

WHEN ASSESSING YOUR STUDENT LEARNING OUTCOMES :

Identify the course/s where learning and/or assessment will occur Identify how the course relates to the SLO

Introduced; Reinforced; Assessed Curriculum Mapping

Assessment reports should: Indicate how the results will be used for improvement Include follow-up action plans when the criteria are not met Be generated annually and evaluated by appropriate faculty/staff members Used in periodic program reviews – Longitudinal Assessment Stored centrally

2013 TAM | OPAR | CAU

Developing Student Learning Outcomes is: Collaborative Faculty-Driven Process

Student Learning Outcomes should: Be expressed as knowledge, skills, and attitudes or values that students acquire as a

result of engaging in that learning experience. Focus on program content mastery (not individual course learning outcomes) Define impact and specify an action by the student that is observable (able to be

demonstrated) and measurable Be assessed using appropriate direct/primary MOA and when applicable,

indirect/secondary MOA

Assessment of SLOs should: Be on-going, not episodic Address the degree to which learning has occurred and be distinct from grades Should indicate how the results will be used for continuous improvements

2013 TAM | OPAR | CAU

Instructions: o Divide into groups with members of your program/department o Use the SLO exercise sheet provided to develop 3 outcomes (one for

each domain). o Use your check sheet to review your SLO o Select a spokesperson from the group to share your SLO

2013 TAM | OPAR

Questions? Comments!

Conversation. 2013 TAM | OPAR

2013 TAM | OPAR

Tia A. Minnis Assistant Director for Assessment

Office of Planning Assessment and Research tminnis@cau.edu

404-880-6294 (Tel) 404-880-8737 (Fax)