Learning Outcomes Assessment in Student Services

Learning Outcomes Assessment in Student Services

A Statewide Initiative in Maryland’s Sixteen Community Colleges

2

Context

Maryland’s Community College Governance Structure

• Maryland Higher Education Commission (MHEC)• Maryland Association of Community Colleges

o Liaison with MHECo Government Relationso Community College Financingo Public Relationso Economic Development and Job Trainingo Researcho Education

3

Context

Nine Affinity Groups • Chief Academic Officers

• Chief Student Services Officers

• Chief Administrative Officers

• Chief Planning and Institutional Research Officers

• Continuing Education Officers

• Development Officers

• Public Relations Officers

• Information Technology Officers

• Facility Planning Officers

4

Context

Student Services Affinity Groups• Registrars and Admissions Officers

• Financial Aid

• Intercollegiate Athletics

• Student Life

• Academic Advising/Counseling

• Testing Centers

• Career Counseling

• Tutoring

5

Genesis

• For 10 years, professional organizations have underscored the need to assess student services

• Emphasis on learning outcomes assessment for accreditation

• NASPA Conference –Emphasis on learning outcomes assessment in student services

6

Maryland LOA Project– Started June 2003

Student Learning Assessment Workshop with State Deans/VPs Facilitated by Marilee Bresciani

• Established goals for the project

• Discussed concept of core indicators

• Developed process for developing a statewide assessment plan

7

Project Goals

• Shift focus of assessment from student satisfaction to student learning

• Assist student services staff to learn to design, implement and analyze learning outcomes projects

• Assist staff to improve student services and programs based on learning outcomes data

• Create benchmarks for comparison of findings across institutions throughout the state

8

Process – October 2003

The Year of Assessment for Student Services in Maryland Community Colleges

Workshop• 150 staff attended a workshop facilitated by Dr.

Marilee Bresciani and guided by the Deans/VPs– An overview of learning outcomes assessment– Participants assigned to groups by functional area– Brainstormed core learning values and ways to gather

evidence of learning – Identified one core value to start the process

9

Assist Students to BecomeSelf-Directed Learners

Deans/VPs clarified • Clearly defined “core value” of self-directed learner

• Developed a matrix to distribute to the Affinity Group chairs

• Established what was wanted from Affinity Groups

10

Definition of Self-Directed Learner

The student will:• Define a need or problem and employ effective decision-

making to resolve it

• Plan ahead/set goals

• Acquire knowledge

• Use available resources

• Seek assistance from appropriate people/experts

• Apply critical analysis to consider options

• Evaluate decisions

11

Definition of Self-Directed Learner

Affinity groups focused on:• Outcomes

• Strategies

• Measurements

• Benchmarks

• Feedback loops

12

GOAL: Self-Directed Learner

Learning

Outcome:

Strategies:

Benchmarks:

Measurement of Student Learning:

Feedback Loop:

13

Process – October 2004

• Deans/VPs sponsored a third workshop, another statewide meeting of student services staff with an assessment expert

• Workshop focused on assessment methodology and measurement

14

Examples

• Advising/Counseling Affinity Group

• CCBC Academic Advising Project

• Intercollegiate Athletics Affinity Group

15

GOAL: Self-Directed Learner (Original)

Advising

Outcomes:

To understand academia; understand and navigate through educational policies and procedures.

Advising Strategies: *Provide academic information online, including a glossary of academic and advising terminology and a resource guide; make sure students understand the expectations of college.

Benchmarks: *80% of campuses have the on-line resource. (Based on number of schools who participate in Affinity Group).

Measurement: *Operable site that includes: glossary of terms, resource guide to use of college materials, resource guide to advising tools, list of academic calendar deadlines.

Feedback Loop: *Director of Advising will check participation annually in April and report to the Affinity Group.

16

GOAL: Self-Directed Learner (Revised)

Learning

Outcome:

To understand educational policies and procedures.

Advising Strategies: *Provide academic information online, including a glossary of academic and advising terminology and a resource guide; make sure students understand the expectations of college.

Benchmarks: *80% of students can navigate the policies and procedures; Students score at least 80% on the post test


Pre and post test of understanding of policies and procedures

Feedback Loop: *Students would know correct and incorrect answers immediately; college can make appropriate improvements to the on-line resources based on students’ post test results

17

GOAL: Self-Directed Learner (Original)

Advising Outcomes:

To articulate an educational goal and develop an educational plan.

Advising Strategies: *Recommend students see an adviser at appropriate credit increments (each institution will define “appropriate” and have liberty in the method of contact).

*Develop systems for advisers to create notes on student plans.

Benchmarks: *80% of campuses reach out to students at appropriate increments.

*Institutions will share best practices at October ‘04 meeting (AACC will showcase Datatel Educational Plan as a tool for tracking contacts.)

Measurement: *Directors report on systems and procedures utilized at each institution at October ‘04 meeting.

*Campuses will evaluate best practice and explore appropriate note systems (colleges with same systems could cooperate in system development).

Feedback Loop: * Directors report on systems and procedures utilized at each institution.

* Directors report on practices and procedures utilized at each institution.

18


Learning

Outcome:

Strategies:

Benchmarks:


Feedback Loop:

19

CCBC ExampleAcademic Advising Project

• See Handout –CCBC - Learning Outcomes Project for Academic Advising 2005-06

20


Learning Outcome: Student Athletes will exhibit good sportsmanship

Strategies: *Conduct preseason meetings with teams to reinforce the behaviors student athletes are expected to exhibit

*Conduct preseason meetings with coaches to emphasize the importance of coaches modeling positive behaviors

*Publicize criteria for Student Athlete of the Month (leadership, service, academic performance, etc.)

Benchmarks: *50% decline in total # of unsportsmanlike-like incidents

*50% decline in # of ejections

Measurement: *Compare the reports of Standards and Ethics violations from the previous year

*Numbers of ejections/technical fouls/yellow-red cards

Feedback Loop: *If improvements occur, continue program. If improvements do not occur, survey players and coaches to determine where changes in program are needed.

21

Spring/Summer 2005

• Deans and VPs sponsored a fourth workshop facilitated by assessment experts in April 2005– Assisted affinity groups to finalize statewide projects

– Developed time table for implementation of the project• Affinity groups finalized their work by the end of June and

submitted their plans to the Chairperson of the Deans/VP group by June 30

• Campus plans were developed over the summer and submitted to Chairperson of the Deans/VP groups in September.

22

Transitioning from Statewide Projects to College Plans

• Customized projects to fit mission, programs and services of the individual colleges

• Integrated statewide project into the lexicon and structures for assessment at each institution

23

Montgomery CollegeGermantown Campus

• Small campus in multi-campus college

• Student services not organized like other colleges

• Multiple outcomes projects

• No tracking system

24


Learning Outcome:

Athletics

Student athletes will be in good academic standing

Strategies: *Tutoring in study skills, time management, organization skills, provided weekly

* Grade check will be conducted

* Counselor assigned to athletes

* First Year Experience Course will be required

Benchmarks: * 100% of student athletes will be in good academic standing

Measurement: * The percentage of athletes in good standing will increase

Feedback Loop: * Report on number and percentage of student athletes who are in good academic standing each semester. Track over time.

25


Learning Outcome:

Student Life

Student leaders will identify, develop, and utilize resources and methods to optimize group functioning and motivations in order to better understand their role as leaders and their interactions with community and society

Strategies: * Student participation in one or more of the following activities/programs: clubs, Student Senate, representative on college committees, service learning, leadership workshops…

Benchmarks: * Evaluation of co-curricular portfolio

Measurement: * A matrix is in the development stage

Feedback Loop: * Annual report will be used to evaluate and re-design services

26


Learning Outcome:

Center for International and Multicultural Students

• Students will learn the key academic terminology that will aid them in navigating the class selection and registration process.

• They will also learn the various payment methods available to them, as well as critical deadlines

Strategies: * International Montgomery Advising Program (IMAP) will be required of all international students

Benchmarks: * 80% of students surveyed at the conclusion of the IMAP will indicate an understanding of key terminology, policies and next steps towards registration. Students indicating limited understanding will be called to clarify information.

Measurement: * Student survey

Feedback Loop: * Report at the conclusion of each registration period on survey results and follow-up contact with students

27


Learning Outcome:Student Employment Services

Students will learn what resources and services are available to them through Student Employment Services and how SES can impact their GPA and retention

Strategies: * Produce, update and disseminate marketing materials

* Participate in new student orientation

* Utilize electronic modes of communication

* Conduct classroom presentations in DS + other courses

* Produce newsletter

Benchmarks: * Participation will increase 5% each semester with goal of 60% of students receiving at least one contact

Measurement: * Number of students served in various SES programs, websites, individual sessions, job fairs, workshops

Feedback Loop: * Annual report of student participation

28


Learning Outcome:

Counseling and Advising

• At the conclusion of DS 107, the student will articulate that they have an educational goal and have developed an education plan.

• 2 additional outcomes will be piloted this fall as part of the college-wide outcomes project

Strategies: * Most new students will complete a FYE course such as DS 107 or DS 104

Benchmarks: * At least 80% of the survey or class evaluation responses indicate students have an educational goal and have developed a plan

Measurement: * Survey or class evaluation

Feedback Loop: * Annual report on percentage of students who demonstrated proficiency, to be used for program improvement

29


Learning Outcome:

Tutoring

Students will learn how to apply effective study skills to their coursework

Strategies: * Provide tutoring (individual and group) in study skills to students who are experiencing learning difficulties.

Benchmarks: * 50% of students will improve by one letter grade

Measurement: * Tests, graded assignments, mid-term, and final grades in contact courses

Feedback Loop: * A report on the number of students who receive services and a comparison of their initial grade with the final grade they receive in the targeted class. Results will be used for further improvement of the tutoring program.

30


Learning Outcome:

Student Success Center (Career, Transfer)

• Students will be able to identify at least three careers that match their interests and abilities

• Students will be able to identify at least two potential transfer institutions that match their major and other relevant criteria

Strategies: * Inform students of career/transfer through information tables, displays, web pages, brochures, and other marketing

* More students, especially from the DS courses, will come to the Center to complete assignments

Benchmarks: * 80% of students will indicate that they can name 3 careers or 2 transfer institutions, as appropriate (and why)

Measurement: * A quick survey will be given students as they depart the Center

Feedback Loop: * Results of the survey will be used for program improvement

31


Learning Outcome:

Assessment

Students will learn why placement testing is important and how to be prepared to take a test

Strategies: * Provide thorough pre-test information when appointment is made and when the student checks in for the test

* Build confidence and familiarity by offering tutoring sessions on how to use the computer (for testing) and the mouse.

* Develop a CD for use with headphones before testing

Benchmarks: * Number of retakes will decrease by 40%

Measurement: * Number of retakes decreases

Feedback Loop: Report on number of retakes and ways in which students are being prepared will be submitted annually. Results will be used for further improvement.

32

Montgomery College Example-First Year Experience Program

• See Handout –Montgomery College - Learning Outcomes Plan for First Year Experience Program 2005-06

33

The College is Serious….

• Unit strategic objectives all have “outcomes” “outputs” “and “assessment results” attached

• Strategic plan is the only way to get $$ or positions

• Tracking system is on the way

34

CCBC’s Plan

• Convened student services affinity group representatives across the college and staff from institutional research

• Asked staff to review affinity group plan and submit college projects

• Draft submitted to Deans in August 2005 • Draft submitted to Coordinator of Learning

Outcomes Assessment for the College and College Vice Chancellor for Learning and Student Development

• Plan approved September 2005

35

• See Handout –Harford Community College - Learning Outcomes Plan for Registration Services 2005-06

Harford’s Plan

36

Challenges Encountered

Instructing staff across functions and colleges how to develop learning outcomes assessment projects

• Staff wanted to develop a menu of core indicators rather than focus on one indicator

• Deans/VPs kept groups focused on one indicator as a way of helping staff to learn the outcomes assessment process

37

The two most difficult learning college principles to implement in student services are• assessment of learning • inclusion of students as full partners in

the learning process, i.e. developing students to be self-directed learners.

Research by Dr. Bob Baer (2005)

38

Next Steps

• Colleges are now implementing projects• Colleges will analyze their data over the

summer, refine their projects, and prepare to re-implement them in fall 2006

• Affinity groups will review college results over the summer, benchmark findings and prepare a report for the State Deans/VPs in September 2006

Documents

Learning Outcomes Assessment in Student Services