Using Learning Outcomes Assessment to Build a Transition Program for Incoming Freshmen

with Disabilities

Postsecondary Disability Training Institute – 2014Presented byJustin Lozano & Becca TerryDisability Center - University of Missouri

What is Bridge to MU?• 2 day transition program for incoming freshmen with

disabilities

• Started in 2012

• The first year was REALLY good! (in a not at all sort of way)

• Cover expenses for early move in and food

• More in depth education regarding resources which are available

Bridge Participant Make-Up

• 7 students total– 2-Physical disability– 2-Learning disability– 1-Cognitive– 1-LD & Psychological– 1-Autism spectrum

Program Sessions

• Disability. What is it?• Advocacy in the Classroom• Academic Advising• Study Skills & The “3-Part Study Plan”• Stress and Time Management Strategies• Campus Scavenger Hunt

Background - The Foundation• “All programs, services and even

facilities should be structured to facilitate learning and enhance the Mizzou student experience.”

oVice-Chancellor Dr. Cathy Scroggs Building a Culture of Assessment

Background – About MU

• 34,658 students o 26,965 undergraduates

• Over 800 student organizations, 23 residence halls

• Students registered with the Disability Center: 1,130

Background – About MU

40%

17%16%

10%6%

4% 3% 2% 1% 1% 0%

Demographics (2012-2013)

Cognitive (448)Psychological (193)Learning (185)Health Related (111)Temporary (73)Physical (43)Blind/Low-Vision (29)Deaf/Hard of Hearing (22)Developmental (16)Neurological (8)Speech/Language (2)

Background – About MU

• Division of Student Affairs includes: o Student Lifeo Campus Dining Serviceso Residential Lifeo Disability Centero Counseling Center

o MizzouReco Student & Auxiliary

Serviceso Parent Relationso Mizzou Store

• “Enhancing the Mizzou Student Experience”

Background - The Foundation• The goal: develop or evaluate a program that is

centered around specific learning outcomes so that you have a strong foundation of baseline expectations for learning.

• Developing strong and specific learning outcomes…– creates a purposeful program– appraises the effectiveness of your program – provides direct feedback about whether the students learned

what you intended

Background - The Foundation• Focus on what students gain/learn/aspire to (the ends)

as a result of what we do, NOT what we do (the means)

• Your department may not fit into every domain, and that’s okay!

• This is a tool which will continuously provide feedback of student learning

• This is not just for programs. It can be used on almost any function within your office

WRITING LEARNING OUTCOMES

The 3 M’s• Meaningful: How does the outcome support the

departmental mission or goal?

• Manageable: What is needed to foster the achievement of the outcome? Is the outcome realistic?

• Measurable: How will you know if the outcome is achieved? What is the assessment method?

ABCD Method(it does not have to be sequential)

•Who does the outcome pertain to?Audience/Who

•What do you expect the audience to know/be able to do?Behavior/What

•Under what conditions or circumstances will the learning occur?Condition/How

•How much will be accomplished, how well will the behavior need to be performed, and to what level?

Degree/How much

(Heinich, et al, 1996)

Bloom’s Taxonomy

Participants will be able to identify three university resources related to concerns they may face by the end of the Bridge to MU program.

Audience: Participants

Behavior: identify three university resources related to concerns they may face

Condition: by the end of the Bridge to MU program

Degree: three

Learning Outcomes

• Interpersonal Development– Maintaining Health & Wellness: Mind, Body & Soul

• Practical Competence– Managing Personal Affairs– Collaboration & Team Work

Assessment Methods• Program Pre & Post Evaluations– (direct and indirect)

• Session Pre & Post Evaluations – (What is disability?)

• Scavenger Hunt Checklist– Didn’t provide much profound data.

• Focus Group*

Learning Outcome…

• After completing the “What is Disability” portion of the Bridge to MU program, participants will be able to discuss with mentors and peers to broaden their understanding of disability.

Collaboration & Teamwork

– Evaluation: Pre/post evaluation & snowball discussion

– Learning: Authentic assessment• Measures intellectual accomplishments that are

worthwhile, significant, and meaningful• Activities can include:

– Performance of skills or demonstration of knowledge– Simulations and role plays

Collaboration & Teamwork

Something that

is different and

doesn't really

help you.Lack of

ability,

disadvantage

Physical disability, mental disability, ADHD, dyslexic,

math problems, reading problems.

Being impaired for example, you can use your legs. Your brain is not

developed.

Lack of ability to do certain

tasks.

Collaboration & Teamwork

Having a disability

is not a weakness

or a crutch.

Being disabled isn't a weakness. It can be

a positive thing if you choose to make

it so.People are different in the

ways we learn, get around,

and act. These differences

do not put us in a position

of weakness, rather, a

position of strength of these

"disabilities" give us a different perspective.

Disabilities can be

viewed as a positive

thing, change the

environment, not

the person. They

aren't the problem.

Change the

environment,

not the person.

Learning Outcome…

• After completion of the “Advocacy” session of Bridge to MU, participants will be able to express self-advocacy skills to provide equal access in the classroom if needed during their first year of college.

Managing Personal Affairs

– Evaluation: pre/post evaluation and focus group

– Learning:

• The level of satisfaction declined but in this case it meant that learning occurred, not that the session was inadequate.

Examples of Learning

• Student D – Pre-evaluation: • “I never had a problem not get resolved in high school.”

– Post-evaluation:• “I will talk to people about how to help myself.

However, if it’s a difficult thing to talk about then I get nervous and don’t really want to deal with it.”

Examples of Learning

• Student G – Pre-evaluation: • “I am very independent about my disability, I know

what I need to succeed.”

– Post-evaluation:• “I can be independent but still can be lazy, so I have

some work to do.”

Data

• On a scale of 1-5, 1 being unsatisfactory and 5 being very satisfied, how would you rate your ability to self-advocate? Please explain your selection. – Pre-evaluation: average overall satisfaction was

4.29 out of 5.– Post-evaluation: average overall satisfaction was

4.17 out of 5.• Negative results can equal learning!

Learning Outcome…

• Participants will be able to identify three university resources related to concerns they may face by the end of the Bridge to MU program.

Managing Personal Affairs

– Evaluation: pre/post assessment and focus group– Learning:• One student provide basic knowledge of learning

according to our rubric. Although this may seem disappointing, responses still reflected that learning occurred. – e.g. During the focus group, a student discussed that he/she

utilized the Student Success Center to help with barriers regarding time management. The post-evaluation in regards to this question did not have a response that mentioned time management. We hope this means that although not immediately reflected, learning occurred as he/she knew who to utilize for support.

Focus Group• Graduate assistant led the focus group

• Intern was the notetaker

• Asked a series of six questions, did not try to interpret or clarify the meaning of the questions

• Refrained but did ask a few follow-up questions

• Focus group was recorded. Both split up transcribing. It was a pain, but helpful!

Focus Group• Color coded each student’s response and made notes in the

margins to summarize response

• Highlighted repetitive words/topics, then counted how many times each were brought up

• After listing and noting frequency, together they came up with a list of themes

• Organized into 5 themes: Major, Minor, Ordinary, Expected, Unexpected

Findings – Major Themes• The transition from High School to College was impactful

• Tutoring is a popular accommodation

• Students wish they had better Time Management strategies

• Students would all like early registration as an accommodation

Findings – Major Themes

• Students would like a “Wellness Program” (sleep, health, alcohol, stress)

• Disability Center staff is friendly, helpful, and approachable

• Professors can be difficult to approach/ impersonal

Findings – Minor Themes• The Tour of Campus was a memorable experience

• Students prefer to submit applications online

• The Student Success Center is an important hub to be familiar with

• Multiple Small Groups could increase # of students while retaining intimacy

• TAs are more accessible

Findings – Other Themes

• Social Membership is important to identity: (Sororities, football, wheelchair basketball, FIG)

• The Program could use more marketing / advertisement

• 15 participant maximum de-incentivizes application

Findings – Other Themes“So I think that definitely just a little side thing on how to spend money wisely so you don’t just go spending. Especially if you’re not rich you can’t afford to spend money all the time and I still have to stop myself. Oh I want some more chicken! But no I have to save money on my card. I’m going to be poor if I go everyday just to get chicken.”

• Money Management is an important skill students want to learn

• Gina & Chemistry professor were notable speakers

Program Planning

• Change in program– New topics – money management / sleep– Change in speakers– Even though “3-Part Study Plan” was not a success

there was a real benefit – sign up for tutoring/study plan consultant.

• Change in marketing– “A maximum of 15 participants will be selected!”

• Was a deterrent not an encouragement

Plans for 2014• Changed instructors for “Advocacy in the

Classroom”

• Removed the “3-part study plan” and made it more general

• Added “money management” session

• Changed marketing– Verbiage and look

How does this apply to you?

• Choose a new program…• Don’t reinvent the wheel (i.e., columns of

learning)• Keep it simple– Start with one or two learning outcomes.

Lessons Learned—The Process

• Need 2 people

• Get away from the office!

• Need to devote time (bi-weekly at minimum)

• Define roles and expectations between all involved

Lessons Learned—The Process

• Map out a timeline and set due dates– Block out time in your calendar and advocate for

that time

• Keep all handouts/webinar materials organized and handy.

Lessons Learned—The Process

• Set expectations/parameters

• Utilize a variety of tools available– For us, a dry-erase board was a blessing

Lessons Learned—Evaluations

• Ask questions the exact same way so you get a statistical analysis – Pre/Post Evaluations

• Utilize as many resources that you can – Delegate to grad assistants and interns– Rely on campus colleagues

• An appearance of dissatisfaction does not always equate to a lack-luster session/program

Lessons Learned—Evaluations

• Just because a student did not exceed expectations does not mean the student did not learn.– e.g. Student came in not meeting basic

expectations prior to the program and now has achieved basic understanding. STILL A VICTORY!

Questions & Information

Justin Lozano, M.S. Becca Terry, B.A.Access Advisor Exams Coordinatorlozanoj@missouri.edu terryrs@missouri.edu

University of MissouriDisability Center

S5 Memorial UnionColumbia, MO 65211

573-882-4696VP: 573-234-6662

Website: disabilitycenter.missouri.eduEmail: disabilitycenter@missouri.edu