Page 1 of 5 Office of Institutional Research

Assessment for Student Learning: Course and Program Assessment Board of Trustees Meeting, March 28, 2017

BACKGROUND

Key Performance Indicators (KPI)

Rock Valley College (RVC) has identified KPI to inform the college community about institutional health and development. The College KPI are presented in five categories aligned to Illinois Board of Higher Education (IBHE) Public Agenda Goals, including Educational Attainment, Access, and Success; Affordability; Educational Quality; Accountability; and Addressing Regional Economic Needs. Assessment for student learning is a KPI aligned to the College category of Educational Quality. As a measure of Educational Quality, this category is aligned to the IBHE Public Agenda Goal to raise the number of people with quality postsecondary credentials and to improve transitions along the educational pipeline.

Assessment for Student Learning

Assessment is a continuous and dynamic process of collecting, synthesizing, and interpreting information to aid in continuous improvement efforts and decision making. Assessment for student learning is conducted at the institutional, program or discipline, and course level. Because of the scope and magnitude of the levels of assessment, this KPI Summary Report will focus on the efforts and impact of course and program assessment. Course Assessment Course assessment is designed to measure success across all sections of a particular course (e.g., ENG 101, MTH 099, or SOC 190) and includes all course delivery modalities (e.g., face-to-face, internet, hybrid). As such, course assessment provides for a consistent level of academic rigor and provides faculty with assurances that students can progress to the next level within a specific curriculum or program. Also, since the majority of our students transfer individual courses, if not degrees, course assessment data affirms for students and transfer institutions that our students’ coursework adequately prepares them for further higher education. Program/Discipline Assessment Program/discipline assessment measures student success across a collection of courses that may or may not lead to a credential. A program is a group of courses that culminates in a degree or certificate (e.g., Associate of Applied Science in Automotive Service Technology or Automotive Engine Performance Certificate). A discipline is a collection of courses within an academic department that does not offer a degree or certificate (e.g., Composition and Literature or Mathematics).

Key Performance Indicator (KPI) Report

Details about the assessment process at RVC can be found on the College website through the featured link, Assessment for Student Learning.

Page 2 of 5 Office of Institutional Research

RESULTS Key components of the assessment cycle are presented in Figure 1. These components include review and update of the Master Course Syllabus (MCS) to keep learning outcomes and course content current, creation and implementation of an assessment plan, analysis and documentation of assessment findings, and identification of next steps for continuous improvement. While these key components of assessment have been consistent over time, a couple important procedural changes have occurred since the last KPI Summary Report on course and program assessment. First, the annual assessment timeline moved from a calendar year reporting cycle to an academic year reporting cycle to be consistent with academic operations. Second, Taskstream replaced TracDat as the College’s assessment management system. Taskstream was chosen because of its versatility and ease of use. Figure 2 displays the status of academic year (AY) 2015-2016 course assessment for the first three stages of the assessment cycle. Values represent the percentage of courses scheduled for assessment that had something documented in Taskstream. Courses are expected to be assessed at least once every five years. Figure 2: Percentage of Courses with Progress Documented in Taskstream (AY 2015-2016)

Data in Figure 2 indicate:

MCS were reviewed for less than half of the courses scheduled for assessment. While most courses scheduled for assessment (59%) have documented assessment plans,

only 28% have documented assessment findings.

Figure 3 compares course assessment progress for calendar years 2012 through 2014 and academic year 2015-2016. Percentages represent courses for which a step in the assessment cycle was completed or in progress.

42%

59%

28%

0%

20%

40%

60%

80%

100%

MCS Review Assessment Plan Assessment Findings

Review MCS

Implement

plan

Analyze

findings

Identify next steps

Figure 1: Course Assessment Cycle

Page 3 of 5 Office of Institutional Research

Figure 3: Multi-year Comparison of Course Assessment Progress

Data in Figure 3 indicate:

More MCS were reviewed in calendar year 2014 than any other year. Documentation of assessment plans and findings were strongest in calendar year 2012 and

academic year 2015-2016. For all years, except calendar year 2012, findings are documented at a rate of less than half

of the rate of plan documentation.

These data suggest that course assessment progress is consistent with the focus of faculty development day activities related to assessment. For example, in an attempt to increase documentation for 2012, time was devoted during the January 2013 Faculty Development Day to documenting 2012 plans and findings. This is most likely the reason for the greater percentage of plans and findings documented in 2012 as compared to 2013 and 2014. In January 2014, the development day activity focused on updating MCS to map course objectives to the new institutional student learning outcomes, which likely explains the increase in percentage of MCS reviewed in that year. After a dip in 2013 and 2014, documentation of assessment plans and findings increased for academic year 2015-2016 with the implementation of Taskstream. Many faculty have commented that Taskstream is more user-friendly than the previous assessment management system. Such ease of use has most likely contributed to the increase in documentation. Data provided within this section suggest that, in general, faculty are more likely to make progress on course assessment if provided designated time to work with their colleagues and given access to user-friendly tools. As process improvements increase the level of participation in assessment activities, the College can shift from a focus on compliance to a focus on closing the assessment loop (i.e., using assessment findings to improve student learning). College-wide gains in assessment documentation and use of assessment findings may be further achieved through additional assessment efficiencies. The next section provides exemplars of programs/disciplines that have made progress on assessment by creating efficient processes. EXEMPLARS: IMPROVEMENT THROUGH EFFICIENT ASSESSMENT

Those areas that create efficient assessment processes have been most successful in using assessment findings to improve student learning. Efficient assessment allows for assessment work to be done without creating extra assignments for students or extra grading for instructors. Assessment can become more efficient by extending classroom assessment across sections, using

48%

63%

33%

49%39%

16%

67%

36%

12%

42%

59%

28%

0%

20%

40%

60%

80%

100%

MCS Reviewed Plans Documented Findings Documented

CY 2012

CY 2013

CY 2014

AY 2015-2016

Page 4 of 5 Office of Institutional Research

national standards for local assessment, and aligning assessment activities with student learning outcomes and classroom instruction. Such alignment allows for measures of student learning within the class to also be used for course and program/discipline assessment.

Exemplar 1: Chemistry - Extending Classroom Assessment across Sections

Course assessment can be challenging when multiple sections of a course are taught by different

faculty members. While course objectives are the same across all sections; instruction, activities,

and classroom assessment methods may differ. The faculty teaching Chemistry 130: General

Chemistry II have worked together to turn this challenge into an advantage. The two faculty

teaching the course use a common final exam to determine whether students across sections are

meeting the course objectives. When exam scores were analyzed question by question, the faculty

discovered that students taught by one instructor exceeded expectations on some topics while

students taught by the other instructor exceeded expectations on other topics. The two faculty

members seek to improve student learning across sections by discussing their techniques,

observing one another, and possibly team teaching.

Exemplar 2: Automotive – Using National Standards for Local Assessment

Course and program assessment can seem like additional work when a program also has state or national standards for student learning. The Automotive Program has streamlined its assessment work by aligning course objectives with National Automotive Technicians Education Foundation (NATEF) Task Demonstrations and using NATEF end-of-program exams to assess program outcomes. Table 1 shows how NATEF end-of-program exams align with the Automotive Technician Certificate Program and individual courses.

Table 1: NATEF End-of-program Exams and Automotive Service Technician Courses

NATEF Exams Automotive Technician Certificate Requirements Steering and Suspension ATM 221 Steering and Suspension Brakes ATM 105 Introduction to Brake and Chassis Systems

ATM 114 Brakes Electrical/Electronic Systems ATM 106 Intro to Auto Electrical Systems & Powertrains

ATM 107 Automotive Electronic Fundamentals ATM 223 Automotive Electrical Circuits

Engine Performance ATM 228 Engine Performance I ATM 229 Engine Performance II

Engine Repair ATM 140 Engine Diagnosis and Repair Automatic Transmission ATM 242 Automatic Transmission/Transaxles Manual Transmission ATM Manual Transmission/Transaxles Heating and Air Conditioning ATM 203 Heating and Air Conditioning Systems

Exemplar 3: Nursing – Aligning Objectives, Instruction/Activities, and Assessment

As the Associate Degree Nursing (ADN) Program seeks accreditation from the Accreditation

Commission for Education in Nursing (ACEN), the curriculum has been redesigned with careful

attention to alignment among learning objectives, instruction/activities, and assessment methods.

In addition, courses are grouped and sequenced such that each semester has objectives that

culminate in the program learning outcomes. Figure 4 and Figure 5 exemplify how the ADN

Page 5 of 5 Office of Institutional Research

curriculum has been organized in such a way that facilitates assessment for the improvement of

student learning.

Figure 4: Example of Alignment among Objectives, Instruction/Activities, and Assessment

Figure 5: Example of Sequencing Semester Objectives

FINDINGS AND CONCLUSIONS

While casual conversations suggest that instructors are assessing their courses and programs, data indicate that formal documentation of plans and findings remains inadequate. Limited opportunity to discuss assessment with colleagues seems to be a contributing factor to the low participation suggested in the documentation data. Providing such opportunities for faculty to work together to address and document assessment is important, but may be limited by schedules and other demands on time. However, implementation of user-friendly documentation tools and development of more efficient assessment processes can positively impact assessment efforts and ultimately the documentation of these efforts. As such, the Office of Institutional Research and Effectiveness, along with the Assessment Committee, will continue to provide Taskstream training and help programs/disciplines develop efficient assessment processes.