CONTINUOUS IMPROVEMENT - MACC...Continuous Improvement Report 3 committee members’ knowledge of assessment, and drafting assessment plans for each degree and certificate program

CONTINUOUS IMPROVEMENT

REPORT 2015-2016

TABLE OF CONTENTS Table of Contents 1

Purpose of Assessment 2

History of Assessment at MACC 2

Relationship to Mission and Purposes 6

Measures of Institutional Effectiveness 7 Assessment Activities 2013-2015 10

Professional Development 10

Strategic Planning and Accreditation 10

Assessment of General Education 11

Assessment of Career and Technical Education Programs 11

Assessment of Support Services 11

Other Assessment Activities 12

Budget Summary 12

Entry-Level Assessment of Student Learning 12

Placement Testing 12 Student Advisement 14

Class Profile Report 14

Student Profile 16

Mid-Level Assessment of Student Learning 18

Course Grades 18

Academic Probation 19

Classroom Assessments 19

Comprehensive Final Exams 19

Developmental Education 20

Course Evaluations 22

Assessment of Support Services Surveys 22

Student Retention 23

Exit-Level Assessment of Student Learning 25

CAAP Test 25

Capstone Courses 28

Essential Skills 28

Licensure, Certification, and Exit Exams 29

Degrees Conferred 30

Follow-Up Assessment 31

180-Day Placement Survey 31

Student Transfer 33

Other Assessment Efforts 34

Program Review 34

Employee Satisfaction 34

Conclusion 35

Continuous Improvement Report 2

2015-2016 Continuous Improvement Report

PURPOSE OF ASSESSMENT Accountability is at the forefront of higher education, and assessment is an integral component of

demonstrating institutional effectiveness. Postsecondary institutions are expected to provide a framework to

encourage, cultivate, and nurture student success.

Assessment provides an opportunity for feedback, reflection, and improvement of teaching and learning across

multiple levels, including at the classroom, program, and institution level. The Missouri Assessment Consortium

identifies three major purposes of assessment:

Improvement of student learning and instruction

Achievement of institutional mission

Accountability for achievement of educational goals

Although assessment should be fully integrated into the institution’s entire operational system, including

academic services, administration, student affairs, institutional planning, and resource allocation, the focus of

institutional assessment must be on student learning (Missouri Assessment Consortium, Handbook of

Assessment).

The Higher Learning Commission of the North Central Association of Colleges and Schools has also heralded

the importance of assessment. Assessment is embedded within the criteria for regional accreditation. The

Commission’s Statement on Assessment of Student Learning states that “assessment of student achievement

is essential for each higher learning organization that values its effect on the learning of its students.” The

Higher Learning Commission expects organizations to have structured assessment processes that are

continuous, that involve a variety of institutional constituencies, and that provide meaningful and useful

information to the planning processes as well as to students, faculty, staff and administration. Organizations

should be able to document the relationship between its assessment processes and its improvement of

institutional effectiveness.

HISTORY OF ASSESSMENT AT MACC An Assessment Advisory Committee was named in fall 1997 for the purpose of developing an Institutional

Effectiveness Plan (i.e., to develop processes and procedures to monitor factors affecting how well MACC is

accomplishing its mission). The goal of the committee was to produce a written, comprehensive plan

containing two primary components: an assessment of student learning and an assessment of support

services. In its initial year, the committee published its purpose, goal, guiding principles, and other information.

In 1998-1999, the committee continued building on the work of year one, specifically increasing awareness

campus-wide about assessment, furthering the work of the general education subcommittee, increasing


committee members’ knowledge of assessment, and drafting assessment plans for each degree and certificate

program offered by MACC.

In 1999-2000, the committee focused on identifying strategies and measures to determine accomplishment of

learning outcomes; evaluating, revising, and finalizing assessment plans; developing an administrative

evaluation process; collecting and evaluating data according to assessment plans already adopted; developing

a systematic review of educational programs; and providing conference and workshop opportunities for faculty

and staff.

In 2000-2001, the committee concentrated on assessing general education; collecting and analyzing data;

educating faculty and staff about assessment issues; and using the results of assessment to improve student

learning.

Assessment efforts begun in previous years and continued in 2001-2002 culminated with the re-accreditation

visit by the Higher Learning Commission in March 2002. Staff development opportunities in assessment

practices were offered, and a group of general education faculty studied and experimented with classroom

assessment techniques. The student newspaper featured several articles on assessment and accreditation,

and a student was appointed to the Assessment Advisory Committee. Additionally, the first assessment

stipend was offered to assist faculty with assessment efforts. Course syllabi were revised to include assessment

practices, and the Administrative Evaluation Process was piloted.

In 2002-2003, following a study by two MACC faculty, the Assessment Advisory Committee recommended that

the CAAP (Collegiate Assessment of Academic Proficiency) exam continue as the exam of choice to assess

general education knowledge and skills, but that the administration of the exam be restructured. Additionally,

the Assessment Advisory Committee recommended that the WorkKeys assessment replace the CAAP exam for

career and technical graduates. President’s Council adopted these recommendations for implementation in

2003-2004. An Institutional Effectiveness website was developed, faculty-authored articles were included in

the assessment newsletter, and two student positions were permanently added to the Assessment Advisory

Committee. In addition, the Assessment of Support Services committee implemented an annual faculty/staff

satisfaction survey.

In 2003-2004, the first annual student satisfaction survey was developed and administered, revealing that

students are generally satisfied with the quality of service MACC provides. The faculty evaluation process was

revised and MACC joined the Missouri Consortium for Value-Added Learning. The CAAP general education

Writing and Critical Thinking subtests were embedded into related subjects. MACC students scored above the

national norms on both tests. Students completing AAS degrees participated in WorkKeys assessments. In fall

2003, some career and technical programs began pre-testing student samples using the WorkKeys test. Four

nursing instructors completed a critical thinking project supported by an assessment stipend.

While many assessment activities were continued in 2004-2005, there were new projects as well. MACC

participated in a pilot program to measure value-added learning by giving the Collegiate Learning Assessment

(CLA) to a sampling of students in spring 2005. The Director of Institutional Effectiveness and Planning wrote

and received a grant to support a statewide consortium of community college faculty and staff to study and

make recommendations about developmental education in Missouri. A Language & Literature Professor


received an assessment stipend to purchase software to analyze the relationship between students’ self-

reported information about attendance and study habits to their responses about teaching effectiveness on

faculty evaluations. The 2005-2010 Master Plan was completed and approved by the Board of Trustees.

During the 2005-2006 academic year, technology was utilized to make assessment activities simpler and more

effective. The fourth annual Faculty/Staff Satisfaction Survey was administered during spring 2006 in an online

format. Employees received a link to the survey via an email message. Responses were automatically entered

into a downloadable spreadsheet, which saved hours of staff time in data entry. The paper and pencil ASSET

test was phased out and the College began using Compass, an Internet-based placement exam. With Compass,

students can be placed in courses the same day because results are received immediately. MACC also began

administering the CBASE exam as an exit assessment for all students completing the Associate of Arts in

Teaching (AAT) degree. Traditional assessment activities such as the Assessment in Action newsletter and the

assessment stipend were continued.

In 2006-2007, it was decided that comparing entrance and exit WorkKeys scores did not prove to be an

effective method of measuring value-added learning in the career and technical programs. Pre-testing of

career and technical students was discontinued, and the Dean of Career and Technical Education began

research into other assessment options. The program review processes for general education and career and

technical programs was revised, and an assessment plan was developed for the Medical Laboratory Technician

program. Administration began researching the PEAQ and AQIP accreditation processes in preparation for the

College’s pending accreditation visit in 2012. The 2007-2012 Master Plan was completed and approved by the

Board of Trustees with a renewed focus on assessment activities at the College.

In 2007-2008, examination and revision of the career and technical education plans began in an effort to bring

current departmental practices in line with the goals and objectives of the plans and to determine best

practices for each department. The Coordinator of Early Childhood Education completed an Early Childhood

capstone portfolio project funded by the assessment stipend. In spring 2008, it was decided after campus-

wide discussion through the Assessment Advisory Committee that MACC would continue using the PEAQ

(traditional) accreditation process for the next accreditation cycle. Traditional assessment activities such as the

Assessment in Action newsletter were continued.

In fall 2008, MACC began administering the Noel-Levitz Student Satisfaction Inventory to solicit student

feedback about the quality of instruction provided by the College. Revision of the career and technical

education program assessment plans continued throughout 2008-2009 year. In January 2009, MACC joined

the National Community College Benchmarking Project (NCCBP) in order to better gauge College performance

among peer institutions and determine future standards of achievement. The 2009-2014 Master Plan was

completed and approved by the Board of Trustees. The plan contains the new mission and vision

documentation adopted by the Board in November 2008, and the entire plan was reformatted and rewritten

to better reflect College practices and future plans.

In 2009-2010, an increased focus on College-wide assessment practices led to several changes in traditional

assessment instruments and methods. The Student Satisfaction Survey was completely updated and

redesigned to better reflect current practices and to include the assessment of additional college support

services. Select course sections test-piloted an electronic version of the survey. Additional communications


questions were added to the Staff Satisfaction Survey to gauge the effectiveness of current communications

practices at MACC. Feedback from these questions was distributed to administration and was incorporated

into the College’s Master Planning efforts. The Office of Academic Affairs, in conjunction with MACC English

faculty, reviewed and revised the current developmental English placement score cutoffs.

During 2010-2011, the 2011-2016 Master Plan was completed and approved by the Board of Trustees. For the

first time, stakeholder feedback for the plan was collected online via Survey Monkey. The change in data

collection methods was successful, increasing the number of responses received to over 800 (compared to 354

responses collected for the 2009-2014 Master Plan). Also in 2010-2011, the Missouri Department of Higher

Education (MDHE) began actively developing a plan to implement a Performance-Based funding model for all

Missouri colleges based on requests by the legislature for increased data and accountability.

In fall 2011, the College underwent a comprehensive accreditation review and site visit by the Higher Learning

Commission (HLC). MACC received the Commission’s top rating - full ten-year accreditation with no additional

follow-up visits or reports. Also in 2011-2012, the annual satisfaction survey was delivered entirely online for

the first time, and the employee satisfaction survey was rewritten and reorganized to gather more relevant

data about employee concerns. Better methods were developed to more accurately track retention and

graduation rates, as well as distance education enrollment patterns. Enhanced datasets were distributed

college-wide through appropriate committees for additional discussion and review. The Missouri Department

of Higher Education began implementation of state-wide Performance Funding measures, leading to

significantly increased reporting burdens for the College.

In 2012-2013, an increased call for transparency and accountability at the national and state levels led to

significantly increased reporting requirements for the College. Additional sections on consumer information

were added to the college catalog and the website, and a college-wide compliance committee was formed to

ensure the College is fulfilling its compliance obligations. The student satisfaction survey was updated to

include questions about student activity preferences, and a question asking students to direct their comments

towards a specific site was added to give data recipients more site-specific feedback. Faculty groups met

throughout the year to continue ongoing assessment projects, and several MACC administrators participated

in state-wide assessment initiatives and groups. The 2011-2016 Master Plan was updated and presented to

the Board of Trustees in summer 2013.

In 2013, MACC joined the Complete College America initiative in Missouri. MACC’s Completion Team focused

on identifying strategies that would improve student retention and completion. The committee outlined

initiatives that included multiple measures of placement, academic maps, and redesigned developmental

education courses. It piloted its first initiative, a writing appeal, in spring 2014. Participation in the Complete

College America initiative also included the collection of additional data. In anticipation of the Year 4 HLC

assurance review scheduled in fall 2015, administrators began familiarizing themselves with HLC’s Open

Pathway model for continued accreditation.

Nationally, college completion became one of the top higher education issues in 2014-2015. MACC continued

its completion efforts by piloting four additional Completion Team initiatives (i.e., Math Exit Competency

Assessments, a College Algebra math pilot, Compass workshops, and an English co-req class). MACC ramped

up accreditation efforts by forming a HLC Leadership Team in fall 2014. The team researched, gathered


evidence, and drafted MACC’s Assurance Argument for reaccreditation during 2014-2015. A newly-formed

Strategic Planning Committee spent 2014-2015 redesigning the College’s planning process and drafting the

MACC 2015-2018 Strategic Plan. The scope of the plan was expanded to include a departmental component.

The planning process also involved revising MACC’s mission statement. Missouri Department of Higher

Education requested additional data collection on remediation, specifically considering multiple measures and

alternate pathways. MDHE also explored the possibility of adding a sixth performance funding measure on

graduate outcomes (i.e., employment and transfer). Data collection of a pilot cohort may be required in 2015-

2016.

College completion continued to be a priority in 2015-2016 at both the national and state levels. MACC scaled

up its co-requisite courses, explored multiple measures for reading and math pathways, and promoted the use

of MECAs (Math Exit Competency Assessments) and the writing appeal for alternate placement. The Missouri

Mathematics Pathways Taskforce continued to meet to outline alternatives to algebra for some degrees and

new mathematics pathways. In August 2015, the Board of Trustees approved MACC’s Strategic Plan 2015-

2018. In November, the College submitted its Assurance Argument, the Open Pathway mid-cycle review, and

received notification back from the Commission in January 2016 that it had met the criteria for accreditation.

In May 2016, the College received notification that it had been awarded accreditation by the National Alliance

of Concurrent Enrollment (NACEP).

RELATIONSHIP TO MISSION AND PURPOSES

MACC is dedicated to accomplishing its mission statement. To gauge MACC’s progress, measures of

institutional effectiveness are in place and are reviewed regularly. The tables on the following pages describe

assessment processes and tools that have been implemented to evaluate specific components of MACC’s

mission statement and institutional purposes. The first column lists key areas of MACC’s mission statement

and institutional purposes. The second column identifies specific components of the mission statement and

purposes. The remaining columns list the assessment measure and tool, the responsible person or

department, and the primary use of the results.

7

Measures of Institutional Effectiveness 2015-2016

Mission and Purposes

(What areas are critical to mission fulfillment?)

Specific Components

(What specific components of these areas will be evaluated?)

Indicator

(What will be measured?)

Assessment or Measurement Tool

(How will it be measured?)

Responsibility

(Who directs or Implements the process?)

Action

(How are results reviewed and used?)

Quality of Educational Programs and Services

Career and Technical Programs (AAS, ADN, and Certificates)

Occupational Skills Essential Skills Attainment

Capstone Courses

Certification and Licensure Exams

Post Testing

Student Portfolios

TSA Exam

National Career Readiness Workforce Certificate

Faculty

Program Coordinators

Director of Career and Technical Programs

Testing Services Personnel

Program Advisory Committee Review

Program Reviews

Curriculum Revision

National, State, and Local Benchmarking

General Education Competencies

WorkKeys Assessment (AAS, Cert)

CAAP Exam

Capstone Courses

Compass/CAAP linkage report

ACT/Compass linkage report


Faculty

Office of Academic Affairs

Director of Institutional Effectiveness and Planning

Program Improvement

Curriculum Revision

Teaching/Learning Improvement

Program Reviews

Job Placement Follow-Up Surveys

Placement Rates

Placement Office Program Advisory Committee Review

Program Improvement

Curriculum Revision

State and Local Benchmarking

Program Reviews

Student Retention Retention Rates

Graduation Rates

Completion Team

Retention Committee

Director of Institutional Effectiveness & Planning

Focus Retention Efforts

State and National Benchmarking

Retention Manager

Student Satisfaction Course Evaluations

Follow-Up Surveys

Noel-Levitz Student Satisfaction Inventory

MACC Student Satisfaction Survey

Graduate Satisfaction Survey

Dean of Career and Technical Education

Placement Coordinator

Faculty




Curriculum Revision

Program Reviews

President’s Council

Assessment Committees

Student Success Post-Testing

Grades

Transfer follow-up studies


Faculty

Program & Department Coordinators

Curriculum Revision


Program Value Program Review Process Dean of Career and Technical Education

Program Improvement

Resource Allocation

Program Certification State Agency Review

State and National Certification Exams

Program Directors/Coordinators


Program Improvement

National and State Benchmarking

8



Specific Components

(What specific components of these areas will be evaluated?)

Indicator




Responsibility


Action


Quality of Educational Programs and Services

General Education (AA, AAT, and AS)

General Education Competencies

CAAP Exam

Student Artifacts/Institutional Portfolio

General Education Rubrics

ACT/CAAP Linkage reports

Compass/CAAP Linkage report

MoGEA Exam (AAT)


General Education Assessment Panel

Faculty



Program Improvement

Curriculum Revision


Program Reviews

Transfer Success Transfer Studies

Student Tracking

Transfer Rates

State Reports

Articulation Agreements

Dean of Student Affairs & Enrollment Management

Vice President for Instruction

Dean of Academic Affairs


Program Improvement

Curriculum Revision

Student Support Services Improvement

Program Reviews



Student Satisfaction Survey



Faculty

Dean of Student Affairs and Enrollment Management



Curriculum Revision

National and State Benchmarking

College Assessment Committees


Graduation Rates

Completion Team

Retention Committee

Director of Institutional Effectiveness & Planning

Retention Manager

Program Reviews

Student Academic Gains

Pre/Post Testing

Final Exams

Grades/Grade Distribution

ACT Linkage Reports

Follow-Up Studies

Faculty



Curriculum Revision

Program Value Program Review Process Dean of Academic Affairs

Faculty

Program Improvement

Resource Allocation/Planning

Developmental Education Student Academic Gains

Pre/Post Testing

Grades

Journals

Follow-Up Studies


Faculty

Director of LARC

Developmental Education Coordinators


Curriculum Revision




Student Survey/Comments



Student Satisfaction Survey


Academic Departments


Curriculum Revision



Program Reviews


Graduation Rates

Completion Team

Developmental Education Coordinators


Retention Committee

Retention Manager

Program Review

9



Specific Components

(What specific components of these areas will be

evaluated?)

Indicator




Responsibility


Action


Quality of Support Services for Students

Student Support Services (departments closely aligned with student learning)

User Satisfaction Student Satisfaction Survey

Faculty/Staff Satisfaction Survey

Departmental Self-Studies

Assessment of Support Services Committee

Department

Service Improvement

Resource Allocation

Quality of College Personnel Faculty, Staff, and Administrators Professional Competencies

Faculty

Faculty Portfolios

Administrative Observation and Evaluation

Course Evaluations

Staff Development Participation Supervisors

Supervisor Evaluation

Staff Development Participation Administrators

Administrative Evaluation Surveys

Supervisor Evaluation

Staff Development Participation

President

Vice-President for Instruction



Department Supervisors

Administrative Evaluation Committee


Departmental Improvement

Administrative Improvement

Quality of Collegiate Environment

Student Support Services User Satisfaction Student Satisfaction Survey

Faculty/Staff Satisfaction Survey

Service Area Self-Study

Course Evaluations


Assessment of Support Services Committee

Department Heads

Service Improvement


Quality of Community Partnerships and Cooperative Efforts

Workforce Education Client Satisfaction Certification Rates

Course Evaluation

Entrepreneurship and Business Development Center

Program Improvement

Accessibility Numbers Served

Fiscal Year Report

Entrepreneurship and Business Development Center

Advisory Committee Review

Program Improvement

Fiscal Responsibility Fiscal Year Activity Report Entrepreneurship and Business Development Center

Finance Office

Program Improvement

Resource Allocation

Educational Partnerships Partnerships Articulation Agreements

Dual-Credit Agreements

Clinical Agreements

On-Site 4-year Programs (CMU, Columbia College)

Co-Enrollment Models (MU, TSU)

Vice-President for Instruction



Program Improvement

Curriculum Revision


ASSESSMENT ACTIVITIES 2015-2016 The following narrative highlights assessment accomplishments and activities during the 2015-2016 academic

year.

Professional Development

Three faculty professional development days were held during 2015-2016. General sessions

included the following:

Be That Person (Greg Walker)

A Retention Series

- Our Students, Our Future: Retention and Completion at MACC (Paula Glover)

- MACC’s Retention Manager (Michele McCall)

- Retention Focus Area Discussions – 1) Fostering Academic Success, 2) Creating an

Inclusive Classroom, and 3) Additional Issues and Solutions

Critical Thinking Model (Mike Barrett and LOGOS Project Panel)

Critical Thinking Across the Disciplines Activity (Mike Barrett)

Initiative Updates: Math Pathways (Donna Kessler), LOGOS Project (Cindy Droege and Barbie

Underwood), Strategic Planning (Meghan Holleran), and Accreditation (Paula Glover)

The Vice President for Instruction continued to serve on the statewide Taskforce on College & Career

Readiness created by Missouri Department of Higher Education (MDHE). MDHE distributed a survey

in fall 2015 to gauge how colleges are adhering to the Coordinating Board of Higher Education’s (CBHE)

Principles of Best Practice in Remedial Education.

The Vice President for Instruction initiated a partnership with Moberly High School to create a

bridge math course to reduce the need for remediation at the college level.

The Director of Instructional Technology served the first of a 2-year term on the Missouri Distance

Learning Association (MoDLA) the Board of Directors.

Two full-time English faculty members served on the Midwest Regional Association for Developmental

Education’s (MRADE) Executive Board through October 2105, completing 2-year terms.

Several Math and English faculty were active in Missouri Developmental Education Consortium

(MoDEC).

The Developmental Math Coordinator served on the Missouri Mathematics Pathways Taskforce, a

group charged with exploring alternative math pathways for students.

The Coordinator of Teacher Education was active on Missouri Association for Teacher Education

(MACTE).

The Director of Institutional Effectiveness & Planning served on the Missouri Community College

Association (MCCA) Research Council, a group that meets to ensure consistent reporting at the state

and federal levels.

Strategic Planning and Accreditation

The MACC Strategic Plan 2015-2018 was implemented in fall 2015. Members of the Strategic Planning

Committee met with department heads in the fall to assist with the development of departmental

initiatives that advance the priorities, goals and/or strategies in college plan. College Reporters


submitted progress reports on the goals and strategies assigned to them in December 2015 and May

2016.

MACC’s Assurance Argument was finalized in early fall and submitted to the Higher Learning

Commission in November 2015. In January 2016, the College received the best possible rating from

the HLC. The Final Team Report established that MACC met the criteria for accreditation with no

required monitoring. Four employees attended the HLC Annual Conference in April 2016 to learn more

about the Quality Initiative, the next requirement in Open Pathway.

The Vice President for Instruction began discussions with administrators to outline ideas for MACC’s

Quality Initiative.

In May 2016, the College received notification that it had been awarded accreditation by the National

Alliance of Concurrent Enrollment (NACEP).

Assessment of General Education In fall 2015, Developmental Education and Dual Credit program reviews were presented to the Board.

The General Education Assessment Panel (GEAP) met regularly in 2015-2016 to evaluate the

assessment of student outcomes within courses. An assessment cycle aligns courses and outcomes

and ensures that all outcomes are assessed on a regular basis.

GEAP joined with the LOGOS Project Committee to develop a critical thinking rubric that will be

incorporated into the general education assessment process in 2016-2017.

The LOGOS Project Committee also produced a Reading, Thinking, Writing MACC Handbook as a

resource for all courses. It is available to students at no cost and is posted in the Student Resources

Canvas shell.

Assessment of Career and Technical Education Programs

In fall 2015, Graphic Arts and Associate Degree in Nursing program reviews were presented to the

Board.

WorkKeys was given to Career and Technical Education (CTE) graduates in the summer, fall, and spring.

CTE faculty continued redeveloping program outcomes and revising program assessment plans.

Assessment of Support Services

The annual Noel-Levitz Student Satisfaction Inventory and Noel-Levitz Priorities Survey for Online

Learners were distributed in fall 2015. The College also purchased site-specific results. The results were

shared in President’s Council and with site directors.

The annual Student Assessment of Support Services Survey was administered during the spring of

2016. Almost 700 responses were collected. Results were shared with President’s Council and

distributed to appropriate offices.

The annual Faculty/Staff Assessment of Support Services Survey was administered in spring 2016.

Over 100 responses were received. Results were shared with President’s Council and distributed to

appropriate offices.

A Part-Time Faculty Satisfaction Survey was administered in spring 2016. Almost 100 instructors

completed the survey. Results were shared with President’s Council and with site directors.

Access & ADA Services and Instructional Technology conducted departmental reviews in spring 2016.


Other Assessment Activities

The LOGOS Project Committee coordinated a full-scale One-Read program in 2015-16 using the book

The Immortal Life of Henrietta Lacks by Rebecca Skloot.

Administrative evaluations were conducted in fall 2016. Half of the members of President’s Council

are evaluated each year.

“Pizza with the President” sessions were held at each location during 2015-16 to provide opportunities

for open dialogue between the President and students, faculty, and staff.

BUDGET SUMMARY 2015-2016 The College’s assessment budget and a portion of the budget for the Office of Institutional Effectiveness and

Planning support assessment activities that are not already embedded within other departments’ budgets and

expenses. Faculty and staff attend assessment-related conferences, but these expenses are generally

deducted from individual travel budgets rather than the assessment budget. Below is a summary of

expenditures for the fiscal year 2016.

National Community College Benchmarking Project $ 1,250.00 SurveyMonkey $ 378.00 Conferences/Travel $ 1,972.00 Planning/Assessment Meetings $ 62.00 Copies/Supplies $ 124.00 Printing $ 50.00 Total $ 3,836.00

ENTRY-LEVEL ASSESSMENT OF STUDENT LEARNING As an open admission institution, MACC often finds students enrolling at the College without the necessary

skills to succeed in college-level courses. Through appropriate and effective placement, MACC students are

given the opportunity to enhance their skills and to be challenged at a level equitable to their current ability.

Placement Testing

In 2015-2016, MACC used the Compass, Accuplacer, and ACT exams to place new students into English and

mathematics classes. These tests assess language usage, math, algebra, and reading skills. Placement

assessments enable MACC to place students appropriately and are required of all students who

Are certificate- or degree-seeking, or

Wish to enroll in any math or composition/technical writing course.

Approximately two-thirds of MACC students who take a placement exam are required to take one or more

developmental courses. Figure 1 shows the total enrollment in developmental courses for the past five years

and the percentage of College’s overall total enrollment.


Figure 1: Enrollment in Developmental Courses (Fall 2011-Fall 2015) Fall 2011 Fall 2012 Fall 2013 Fall 2014 Fall 2015 Enrollment in Developmental Courses

1,241

1,261

1,210

1,049

1,111

Percentage of Total Enrollment

22% 22% 21% 19% 23%

In fall 2009, the Office of Academic Affairs in conjunction with MACC English faculty developed new cutoff

score levels for placement in developmental English courses. The new cutoff scores are more consistent with

current ACT/Compass guidelines and will increase student success rates by ensuring that students are better

prepared for the rigors of college work. All students taking a placement test after September 1, 2009, were

evaluated and placed using the updated scoring guidelines.

In fall 2010, the Missouri Community College Association Presidents and Chancellors group charged community

college Chief Academic Officers to determine a set of common placement scores to better streamline student

articulation and to allow for easier data comparison at the state level. These revised scores were adopted by

the Presidents and Chancellors group in spring 2011 and were fully implemented by all Missouri community

colleges in fall 2012.

In spring 2014, MACC began implementing multiple measures to more accurately place students into

coursework. Developmental English faculty created writing prompts for a Writing Appeal. The appeal progress

gives students who believe that they have the necessary writing skills to take the college-level Composition I

course another opportunity to demonstrate their skills. In fall 2014, a multiple measures pilot was also

developed to place students into College Algebra if they scored into a range below the cut core but met a

combination of other measures (e.g., high school GPA). During 2014-2015, developmental math faculty

created Math Exit Competency Assessments (MECA tests) as an additional option beyond the Compass/ACT

exams.

Curriculum Committee approved a requirement in spring 2015 that ensured that more students will complete

the required developmental coursework early in their college career. The requirement stipulates that degree-

and certificate-seeking students required to complete developmental coursework must enroll in the required

developmental courses prior to or immediately following their first six credit hours of enrollment at MACC.

The requirement went into effect in summer 2015.

In spring 2015, Missouri Department of Higher Education adopted statewide placement scores for college-level

English and math courses. MACC’s reading and writing scores were already in alignment with the scores set

by the state, but the new cut scores for math allowed the College to lower the math ACT score by 1 point for

each math class in the sequence. Later in the spring, ACT notified institutions of its plans to phase out all

Compass products (old version and newer web-based version) by the end of 2016. MACC began reviewing

options for a Compass replacement.

In spring 2016, a subcommittee of administrators and faculty recommended replacing the Compass with

Accuplacer. Like the Compass, Accuplacer is an untimed, computerized test. The College began administering

Accuplacer in April 2016.


Student Advisement Students are provided with advisement that focuses on academic achievement upon entry to MACC and

throughout their educational experience. The following advising tools and interventions have been put into

place to ensure consistent and accurate advising:

An advising checklist that is updated each semester

A nursing faculty advising handbook

Academic Planner software from Jenzabar allows students to register for courses online after a plan

has been approved by an advisor

Academic Maps which serve as course sequence guides for students

RegisterBlast allows students to make advising appointments through an online scheduling system

Class Profile Report

ACT’s Class Profile Service Report provides a comprehensive summary of the College’s entering freshman class

and a parallel description of ACT-tested students who were admitted but did not enroll. The fall 2015 Class

Profile Report included 533 ACT-tested students who enrolled at MACC full time. In all, 1,162 students who

took the ACT requested that their scores be sent to MACC. Figure 2 below provides descriptors of the students

who ultimately enrolled at the College in fall 2015.

Figure 2: ACT Fall 2015 Class Profile Report

Students Who Enrolled at MACC Fall 2015

Average ACT Score of MACC incoming full-time freshman 19.9

Most popular planned educational major Health Science & Technologies

Percent of students completing core high school curriculum 65%

Percent of students ranking in top quarter of high school class 27%

Percent reporting minority racial/ethnic background 11%

Percent planning to apply for financial aid 78%

Percent planning to work while attending classes 79%

Percent needing math skills 36% (self-reported)

Percent needing study skills 45% (self-reported)

Percent needing assistance with educational/occupational plans 46% (self-reported)

Percent needing reading skills 22% (self-reported)

Percent needing assistance expressing ideas in writing 16% (self-reported)


Figures 3-6 illustrate a five-year trend in the characteristics of MACC’s ACT-tested freshmen. Figure 3: Student Reported Deficiencies (2011-2015) Figure 4: College Preparation-Core Curriculum &

Top Quarter of Class (2011-2015)

Figure 5: Financial Need--Plan to Apply for Figure 6: Minority Racial/Ethnic Background for Financial Aid and to Work (2011-2015) (2011-2015)

As Figure 7 below indicates, the national average ACT score for all high school graduates who took the exam has stayed relatively consistent. Figure 7: National Average ACT Score for All High School Graduates

2009-2010 2010-2011 2011-2012 2012-1013 2013-2014 2014-2015

21.0

21.1

21.1

20.9

21.0

21.0

Figure 8 illustrates by year the average ACT score of MACC students who submit scores through the ACT Class

Profile Service and the national average ACT score of freshmen enrolling in two-year colleges. The national

data is one year behind the institution-specific data.

0

10

20

30

40

50

2011 2012 2013 2014 2015

Math

Study

Reading

010203040506070

2011 2012 2013 2014 2015

CoreCurr*

TopQtr

0

20

40

60

80

100

2011 2012 2013 2014 2015

FinAid

Work

0

5

10

15

2011 2012 2013 2014 2015

Core is at least 4 yrs. of English, and at least 3 yrs. of Math, Social Studies, and Natural Sciences.


Figure 8: Average ACT Composite Score for MACC Students versus National Average

Student Profile

Each fall, a Student Profile Report is distributed to administration for their review and posted in the Resource

Library that is accessible to all employees. The ethnic/racial composition of MACC students tends to be rather

homogeneous, although the percentage of minority students has increased over the past several years.

(Between 2006 and 2015, the total percentage of White students decreased by 8%). According to fall 2015

census enrollment data, 81% of MACC students are White, 8% are African-American, 3% are Multi-Race, 3%

are Hispanic, 2% are Asian, 2% are Unknown/Other, and less than 1% are Native Americans and Non-Resident

Alien.

Figure 9: Fall 2015 Students by Classification

The distribution of full-time and part-time students has remained relatively level over the past decade. In fall

2005, 53% of MACC students attended part-time. Figure 10 shows that there was just a 1% decrease in part-

time enrollment by fall 2015. Almost three-fifths (58%) of MACC students are female (Figure 11).

18.4

19.4

20.4

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

AVERAGE ACT COMPOSITE SCORE OF STUDENTS ENROLLED AT MACC

AVERAGE NATIONAL ACT COMPOSITE SCORE OF TWO-YEAR COLLEGES

First-Time Freshman

21%

Freshman27%

Sophomore30%

Non-Degree Seeking

5%

Special With Degree

1%

Dual Credit/Enrolled

13%

Visitor3%

As Figure 9 shows, in fall 2015, 48% of MACC students had accumulated less than thirty hours. MACC’s dual-credit and dual-enrolled students made up 13% the student population.


Figure 10: Students by Enrollment Status (Fall 2015) Figure 11: Students by Gender (Fall 2015)

The average age of all fall 2015 students was 22.8. Without dual-credit students, who are usually in the 18 and

under age range, the average age was 23.7. In fall 2015, the largest student categories were those students

aged 18 -19, as shown below in Figure 12. A little less than one-fourth (23%) of MACC’s students are non-

traditional students (aged 25 and above).

Figure 12: Students by Age (Fall 2015)

Under 1812%

18-1933%

20-2431%

25-299%

30-398%

40-494%

50-642%

Over 65/Unknown<1%

Full-Time48%Part-Time

52%

Male42%

Female58%


The average age of MACC college-level students (excluding dual-credit) steadily decreased from 2000-2005,

then began increasing slightly as more non-traditional students enrolled in classes due to negative economic

conditions, and then began decreasing again as the economy improved. This trend is illustrated in Figure 13.

Figure 13: Average Age Trend 1999-2015

The most popular areas of study in fall 2015 were General Studies (1,371 students), Health Science (547), Business (429 students), Education (236 students), STEM Studies (180 students), and Nursing (143).

MID-LEVEL ASSESSMENT OF STUDENT LEARNING Formative evaluation plays an important role at MACC in determining ways to improve the teaching/learning process. Mid-level assessment of student learning occurs at the course level through course grades, individual classroom assessments, final examinations, pre- and post-testing, retention studies, and student evaluations of teaching.

Course Grades Faculty issue final course grades at the end of each semester. These grades are used to indicate student mastery of the objectives of the course and are used to determine grade points students receive for individual courses that are then used in computation of cumulative grade point averages. Faculty are expected to evaluate students frequently enough to get a sound measure of the quality of their work and to give them ample opportunity to improve their grades. Grades are issued at the close of each semester on a letter basis indicating quality of academic work as follows: A Superior Work: excellence in originality of work; accuracy and grasp of content; independence in

completing work; and literate expression of ideas, imagination, and initiative

B Above Average Work: high achievement in knowledge and grasp of work, effective use of knowledge, independence of work, and competence in expression

C Average Work: average attainment in familiarity with course content, participation in class work, evidence of intellectual growth and thinking, and intelligible expression of ideas

D Below Average Work: meager achievement in both quantity and quality of work, difficulty in understanding principles, slow progress, and instructor direction needed to meet requirements

202122232425262728

Age (all) Age (less dual credit)


F Failing Work: failure to achieve the minimum requirements for passing, inaccurate results, faulty reasoning, irresponsible, vague, disorganized, and lack of initiative

I Incomplete Work: unable to complete the required work by the close of the semester; approved only when student has maintained a passing grade prior to the incompletion. No incompletes will be approved for students who have not completed work through three-fourths of a session; all incomplete grades must be completed within the next semester of the regular academic year.

W Withdraw

P Pass: Pass/Fail; the grading policy for some courses Letter grades earned in developmental courses are by designation only and are not computed when figuring

grade point average.

In summer 2008, the grading scale policy was amended to eliminate the Withdraw – Satisfactory (WS) and

Withdraw – Unsatisfactory (WU) options. The single designation of Withdraw (W) for students who drop a

course is preferred by faculty and is consistent with the practices of other postsecondary institutions.

Academic Probation Prior to fall 2010, students who attempted six or more credit hours and fell below a 1.60 grade point average

were placed on academic probation for one semester. Effective fall 2010, the GPA cutoff for academic

probation was raised to 1.75. This change was suggested by MACC’s Retention Committee and allows

administration to identify a greater number of at-risk students and begin intervention measures earlier to

increase student success rates.

Students not showing improvement during a probationary semester may request permission to continue from

the Dean of Academic Affairs or the Dean of Career and Technical Education, as determined by the student’s

major. No student placed on academic probation will be allowed to carry over sixteen hours. Tutoring and

mentoring services are available for students who need help in achieving academic success.

Classroom Assessments Assessment involves both instructors and students in the continuous monitoring of student learning. Individual

classroom assessments are conducted by faculty through various means at the course level. These assessments

provide faculty with useful information on what, how much, and how well students are learning throughout

the semester. Instructors are encouraged to allow for daily or weekly input from students concerning their

learning progress. Weekly participation points for learning progress reports, one-minute assessments, concept

maps, and journaling are some examples of the ways MACC instructors have encouraged feedback from

students. Course portfolios and student exhibits and demonstrations are being utilized in several classes to

provide instructors with an additional assessment of student progress.

Comprehensive Final Examinations Students are given comprehensive course finals at the end of each semester. The course final enables faculty

to assess how students have progressed toward meeting course goals.


Developmental Education MACC’s commitment to open admission is made possible by a strong developmental education program. The

College has established the following developmental courses which concentrate on student writing skills,

reading skills, math skills, and algebra skills:

All students scoring into developmental courses are given a pre- and post-test to determine progress in the

course. Student success is also assessed through journals and surveys. Results of the pre- and post-tests and

surveys are reviewed by administration, faculty, and staff to develop recommendations for curricular

improvement.

In 2011, MACC Math faculty reviewed and redesigned the College’s developmental math coursework based on

the model provided by the National Center for Academic Transformation. Studies have shown that this model,

which allows students greater freedom to complete developmental coursework at their own pace, greatly

increases student success rates. The newly redesigned math courses were offered in fall 2011. The College’s

developmental English course was redesigned during the 2011-2012 academic year.

In spring 2012, three faculty members won the College’s Innovation Award for their work in redesigning the

College’s developmental Math and English courses. The data that has been collected to determine student

success rates in the redesigned courses as compared to the College’s traditionally structured courses shows

encouraging results. In fall 2012, the first sections of MACC’s redesigned developmental English course were

offered to students.

In 2013-2014, MACC’s Completion Team and English and Math faculty explored the development of completion

strategies which included multiple measures and alternate pathways. Figure 14 on the next page outlines the

developmental education initiatives that have been implemented as well as the results of each initiative

through spring 2016.

Language & Literature

LAL020-Transition to Composition I

LAL022-Fundamentals of Writing

LAL023/024-Tech Enhanced Fundamentals of Writing I & II

LAL031-Fundamentals of Critical Reading

LAL032-Reading in Content Area

LAL090-Composition I Lab

Mathematics

MTH010-Fundamentals of Mathematics

MTH011-Fundamentals of Algebra

MTH015-Developmental Technical Mathematics

MTH016-018-Computer Assisted Pre-Algebra I, II, II

MTH019-Computer Assisted Pre-Algebra Supplement

MTH080-Support for Technical Mathematics


Figure 14: Developmental Completion Initiatives

Discipline Initiative Implemented Description Results Through Spring 2016

English Writing Appeal Students can opt for a one-time writing appeal to demonstrate the writing skills necessary for Composition I. The appeal is a proctored writing assignment. English faculty score the writing responses using a rubric. If the writing appeal is granted, students test out the developmental writing course.

85 students completed writing appeals

58 (68%) passed the appeal 28 (48%) have successfully

completed LAL101-Comp I

Compass Prep Workshop Students can enroll in a free workshop that provides test-taking tips and strategies for the Compass Reading and Writing tests. After they complete the workshop, students are given the opportunity to retake the Compass at no cost.

25 students participated Avg. gain of 8.5 points on reading Avg. gain of 28 points on writing 11 tested out of LAL031

7 tested out of LAL022 11 passed LAL101 (10 passed at

MACC, 1 transferred the course to MACC, and 3 withdrew)

Composition I Co-Req Class (LAL090)

In addition to enrolling in Composition I, eligible students participate in a co-requisite lab where foundational reading and writing skills are supported.

44 students participated

39 (89%) passed LAL090 & LAL101 (5 withdrew)

Transition to Composition I (LAL020)

This intersession course is designed to see if a student can bypass developmental courses which would otherwise be required for Composition I. To qualify for enrollment, students must achieve designated reading and writing scores on placement exams. Students passing the course may enroll in Composition I the next semester.

Insufficient enrollment to run the course in summer 2015

Math Math Multiple Measure Alternative (College Algebra Pilot)

Students without the necessary placement score for College Algebra can enroll in the course if they meet other criteria (e.g., recent graduation from high school, GPA, and ACT/Compass/Accuplacer score).

185 students participated

70% passed with a C or higher (compared to 64% of non-MMMA students)

Math Exit Competency Assessments (MECAs)

MECAs are internally-developed competency exams that give students another option beyond the Compass, ACT and Accuplacer to determine which math course they can take. MECA1 = Fundamentals of Math (MTH010) MECA2 = Fundamentals of Algebra (MTH011) MECA3 = Intermediate Algebra (MTH100)

108 MECAs were taken 31 (29%) scores were passing 14 of 34 (41%) passed MECA1 12 of 50 (24%) passed MECA2 5 of 24 (21%) passed MECA3 10 students who passed a MECA

enrolled in and passed the subsequent course

Support for Technical Mathematics (MTH080)

This is a developmental math course which allows students the opportunity to receive just-in-time remediation so that they may complete their college-level math course.

8 students enrolled in the course 7 (88%) passed MTH080 and the

college-level course (1 withdrew)

Two college-level math completion initiatives have also been developed to provide support to students and in completing their required college-level math courses. Figure 15: College-level Completion Initiatives

Discipline Initiative Implemented Description Results Through Spring 2016

Math Transition to College Algebra (MTH020)

This course is intended as an alternative for students who can demonstrate skills they already possess and, therefore, accelerate the math sequence path into MTH140 College Algebra. The course is a quick review of topics usually covered in MTH100 Intermediate Algebra.

Insufficient enrollment to run the course in summer 2015

Support for College Algebra (MTH090)

This course provides co-requisite support and supplemental instruction for students who have met the prerequisites to enroll in MTH100 Intermediate Algebra. Emphasis is placed on experiences that enhance the materials presented in the accompanying MTH140 College Algebra class.

70 students enrolled in MTH090 and a MTH140 co-req

52 (74%) passed MTH090 44 (63%) earned a “D” or higher in

the co-requisite MTH140

The results of these initiatives are being carefully monitored.


Course Evaluations The primary purpose of course evaluations by students is to improve teaching and learning. These evaluations

provide feedback to instructors and administrators concerning student perceptions of the effectiveness of

instructional methods and procedures used in the course. The course evaluation form was revised during 2003-

2004, and a new form was implemented in fall 2004. The form collects demographic information on each

student, so that evaluation results can be interpreted more meaningfully.

All full-time and part-time instructors administer course evaluations each semester. Typically, a student

representative distributes the survey in class, collects the responses, and returns them to the appropriate

dean’s office. Survey responses are tabulated and a summary report is prepared for each course section.

Faculty members receive a typed list of the written comments and the summary form. The deans monitor

survey results to detect any potential issues that should be addressed. Results are also included in the faculty

member’s teaching portfolio.

Assessment of Support Services Surveys

Student Survey

The Assessment of Support Services Committee conducts a comprehensive annual Student Assessment of

Support Services Survey. The survey is delivered completely online. Students access the survey through a link

sent to student email accounts, posted in myMACC, within eCollege, and from a banner announcement on the

website.

In spring 2016, 680 students completed the surveyed, which is a 15% response rate. The following service areas

were evaluated:

Advising, Registration, and Student Activities

Student Financial Services

LARC (Library & Academic Resource Centers)

Facilities

Career and Placement Services

Bookstore

Technology

Media/Public Relations

Survey items cover professionalism and knowledge of staff, convenience and timeliness of services, and

availability and usefulness of information and resources, among other topics that are specific to certain areas.

Students were also asked to give overall satisfaction ratings to service areas, which ranged from a low of 3.15 to

a high of 3.54 on a 4-point scale. Statistical results and student comments were distributed to area supervisors

for use in a variety of quality improvement efforts. Survey results were also discussed in President’s Council.

Figure 16 on the next page shows student satisfaction survey trends from 2008 to 2016.


Departmental Reviews During 2015-2016, the Assessment of Support Services Committee revived departmental reviews/self-studies

and developed a departmental review schedule. Departmental reviews consist of support service departments

conducting self-assessments in which they identify their strengths and weaknesses. A final report is prepared

by an Assessment of Support Services Subcommittee for the department and the President and a meeting is

held with the President to discuss the review. Departments use the results of this process to set goals and

strengthen the services they offer. In spring 2016, Access and ADA Services and Instructional Technology

conducted departmental reviews.

Student Retention MACC defines retention as the return of first-time, full-time, certificate/degree-seeking freshmen from fall to

fall as measured on census date. Students are considered retained even if they attend on a part-time basis.

ACT reports the national retention rate for public two-year community colleges with open admission policies

to be 56% for fall 2014 to fall 2015. MACC’s retention rate for the same period was just slightly higher at 57%.

Figure 17 shows a trend of MACC’s retention as compared to the national retention rate for two-year colleges

with open admission. MACC’s retention rate has improved considerably since 2011.

Figure 17: Fall-to-Fall Retention Rates of First-time, Full-time Degree-Certificate Seeking Students

(2011-2012 to 2014-2015)

0

0.5

1

1.5

2

2.5

3

3.5

4

LowHigh

0%

10%

20%

30%

40%

50%

60%

2011-12 2012-13 2013-14 2014-15

MACC

National

Figure 16: Student Satisfaction Survey Trends (2008-2016)


The College also tracks fall-to-fall retention of all full- and part-time students (excluding high school students),

which gives a much broader perspective than the first-time, full-time cohort addressed above. Figure 18 shows

MACC’s retention rate of all students compared to the national 50th percentile, as reported by National

Community College Benchmarking Project. There had been a downward trend, with the biggest spread

occurring in the FA13 to FA14 rate when the College also experienced a 5% enrollment decline, but the trend

shifted slightly upward in FA14-FA15.

Figure 18: Fall-to-Fall Retention Rates of Total Full- and Part-Time Credit Students

(Fall 2011-Fall 2012 to Fall 2014-Fall 2015)

Evaluating next-term retention (fall to spring) trends helps the College determine not just the most appropriate

type of retention strategies to implement, but also the best timing for them. Because a downward trend had

also been occurring with next-term retention, as Figure 19 indicates, MACC began focusing on more effective

ways to specifically support student success during the fall term.

Figure 19: Next-Term Retention Rates of Total Full- and Part-time Credit Students

(Fall 2011-Spring 2012 to Fall 2014 to Spring 2015)

25%

30%

35%

40%

45%

50%

55%

FA11-FA12 FA12-FA13 FA13-FA14 FA14-FA15

MACC

National 50th Percentile

50%

55%

60%

65%

70%

75%

FA11-SP12 FA12-SP13 FA13-SP14 FA14-SP15

MACC

National 50th Percentile


MACC is committed to student retention and to providing the support students need to achieve their goals. To

this end, the College has implemented a combination of retention initiatives in recent years which include the

following:

Early-alert programs (e.g., Retention Manager)

Intrusive advising

Academic maps (degree program sequence/guide)

Academic Planner (establishes academic plan and allows online enrollment)

SmartThinking online tutoring service

Expanded Library and Academic Resource Center (LARC) services and hours

Attendance policy

Requirement to complete developmental hours prior to or following first six credit hours

Default prevention programs (supported by Default Prevention Grant)

Student Assistance Program

Adjustment to Allied Health admission requirements (required GPA from 2.5 to 2.7)

Multiple measures of placement (Writing Appeal, MECAs, College Algebra math pilot)

Redesigned Math and English Courses (modular and mastery based)

Co-requisite courses

Blended developmental reading and writing instruction

Boot camps

Compass Prep Workshops

Establishment of Department Coordinators (faculty leaders)

Increased role of Lead Instructors (for courses and to observe part-time faculty)

Professional development workshops focused on retention

Setting measurable retention goals in 2015-2018 Strategic Plan

The College monitors the effectiveness of individual measures. Retention has been and will continue to be an institutional priority.

EXIT-LEVEL ASSESSMENT OF STUDENT LEARNING The ability to measure student achievement contributes to improving programs and services. Assessing

outcomes will also help the College respond to accountability requirements from its constituents and other

external agencies.

CAAP Test The CAAP (Collegiate Assessment of Academic Proficiency) test is designed to measure general education

competencies typically attained in the first two years of college. MACC used to require all graduates to take

the CAAP exam; thus, a significant population of students, such as certificate and AAS completers, was being

tested for content which was not the focus of their programs. Thus, MACC graduates typically scored below

the national mean on most subtests.

Beginning in spring 2004, subtests of the CAAP exam were embedded into related general education courses.

The Critical Thinking subtest is administered every year, while the other four subtests – Math, Writing, Reading,


and Science – are used alternately. This change in the administration of the exam more accurately assessed

general education competencies and provided more useful data. Figures 19-23 present the CAAP performance

of MACC students as compared to students nationally.

As shown in Figure 20, student performance initially improved after the test was embedded in courses that

encourage critical thinking in 2004. Since 2004, MACC scores have been very close to the national scores

(within 1%).

As Figure 21 indicates, MACC students have consistently scored very close to the national norm in the writing

section of the CAAP. This test is generally administered in Composition I and II classes.

As illustrated in Figure 22, since the CAAP math subtest was embedded in math-only courses, MACC students

have scored above the national average, with the largest difference (2.4%) occurring in the most recent

administration in 2014.

61.1 61.1 6160.7 60.5 60.5 60.7 60.9 60.9 60.8 60.7 60.7 60.6 60.8 60.6

59.2 59.358.8

59.4

61 60.961.4

60.760.3 60.3

61

60.359.9 60.1

60.7

5858.5

5959.5

6060.5

6161.5

6262.5

63

2000 (

227)

2001 (

292)

2002 (

308)

2003 (

327)

2004*

(310)

2005 (

313)

2006 (

309)

2007 (

259)

2008 (

263)

2009 (

291)

2010 (

255)

2011 (

323)

2013 (

278)

2014 (

331)

2015 (

313)

National MACC

62.7 62.7

62.7

62.6

62.6 62.6

62.5 62.5

62.1

61.5

64.1

62.362.9

62.6

62.2 62.4 63.2

62.6 61.7 61.5

61

62

63

64

65

66

1997 (

192)

1998 (

232)

1999 (

168)

2000 (

227)

2001 (

292)

2002 (

308)

2003 (

327)

2004*

(345)

2008 (

280)

2013 (

322)

National MACC

* Indicates the first year the subtest was embedded. The Writing and Critical Thinking Subtests were given that year. Note: The CAAP was not administered in 2012 or 2016.

Figure 20: CAAP Critical Thinking Scores (2000-2015)

Figure 21: CAAP Writing Scores (1997-2013)


MACC students scored below the national average on the past two administrations (2010 and 2015) of the

CAAP reading test (Figure 23). The reading test is administered in classes that have a focus on reading

comprehension skills. However, it should be noted that the only course that requires a prerequisite reading

score is Composition I. Faculty members have expressed concern about students’ reading skills.

As illustrated in Figure 24, MACC student scores siginificantly improved their performance on the science test

beginning in 2002 when the order the subtests was changed and the science test was given first. In the last

two admnistrations of the test, MACC students scored within 0.5% of the national norm.

56.1 56.2 56.1 56.2 56.3 56.356 56.1 56.2 56

55.7 55.5 55.5

54.8 55

55.7 55.6

58 57.958.4

54

55

56

57

58

59

19

97

(1

92

)

19

98

(2

32

)

19

99

(1

68

)

20

00

(2

27

)

20

01

(2

92

)

20

02

(3

08

)

20

03

(3

27

)

20

05

(2

53

)

20

09

(3

01

)

20

14

(3

44

)

National MACC

61.161.3

61.1

61

61 61.1

60.960.4 60.4

60.1

61.4

59.6

61.5

59.6 58.6

59.960.1 60.3

59.2

58.7

58

59

60

61

62

63

1997 (

192)

1998 (

232)

1999 (

168)

2000 (

227)

2001 (

292)

2002 (

308)

2003 (

327)

2006 (

285)

2010 (

267)

2015 (

296)

National MACC

Figure 22: CAAP Math Scores (1997-2014)

Figure 23: CAAP Reading Scores (1997-2015)


Capstone Courses MACC has added capstone courses to many of the Associate of Applied Science degree programs, including

Business Accounting Technology, Architectural & Mechanical Design, Business & Office Administration,

Computer Information Technology, Early Childhood Education, and Graphic Arts Technology.

Each capstone course requires a culminating project in which the student demonstrates mastery of essential

skills within the program. Results of each capstone course are used in the program’s assessment plan and are

reviewed by faculty, division chairs, the Dean of Career and Technical Education, the program advisory

committee, and various other college-wide assessment committees.

Essential Skills Each Career and Technical Education Program has its own assessment plan, and one portion of that plan is

mastery of Essential Skills. The goal for this assessment component is that 80% of program completers achieve

80% mastery of all Essential Skills in that program area. Previously, under the Federal Perkins III legislation,

results of this assessment were reported annually to the Missouri Department of Elementary and Secondary

Education (DESE). Since the implementation of Perkins IV in 2006, this data is no longer reported to the state

but is still used for internal program assessment. Results are used in each program’s assessment plan and are

reviewed by faculty, division chairs, the Dean of Career and Technical Education, and each program’s advisory

committee. Figure 25 reflects Essential Skill attainment of 2015-2016 Career & Technical Education graduates.

58.8 58.9 58.8

59 58.9 59 58.9

5959.1

59.158.9 58.8

56.9

57.1

55.255.7

59.5

59.6

58.9

58.9

55

56

57

58

59

60

61

1996 (

187)

1997 (

192)

1998 (

232)

1999 (

168)

2000 (

227)

2001 (2

91)

2002 (

308)

2003 (

327)

2007 (

279)

2011 (

314)

National MACC

Figure 24: CAAP Science Scores (1996-2011)


Figure 25: Essential Skills Attainment (2015-2016 AAS Graduates)

Essential Skills Attainment

Program Number of Graduates

Assessed Percent Meeting

Essential Skills Architectural & Mechanical Design/Drafting Design Technology 1 0%

Business Administration: Accounting/Business Accounting Technology 7 100%

Business Administration: Marketing and Management Marketing/Management

9 100%

Business Office Administration/Business and Office Technology 9 78%

Computer Information Technology-Networking and Programming 20 95%

Early Childhood Education 6 100%

Engineering Design & Development-Machine Tool 1 100%

Engineering Systems: Maintenance & Mechatronics Industrial Technology

8 100%

Graphic Arts Technology 9 100%

Law Enforcement Training Center 25 100%

Medical Laboratory Technician 7 100%

Nursing – Associate Degree & Accelerated ADN 42 100%

Nursing – Practical 38 100%

Welding & Metals Technology 1 100%

Licensure, Certification, and Exit Exams

Additional measures of student outcomes in place at MACC include licensure, certification, and exit

examinations. Currently, completers of five career/technical programs take either national or state

competency examinations. Associate Degree of Nursing students take the NCLEX-RN exam and Practical

Nursing students take the NCLEX-PN. Business Office Administration completers take the Office Proficiency

Assessment and Certification (OPAC) exam. Computer Information Technology completers take the Cisco

Certificated Entry Networking Technician (CCENT) certification exam. Completers of the Law Enforcement

Training Certificate take the Missouri State Certification Exam for Basic Peace Officers. Occupational Therapy

Assistant completers take the National Board for Certification in Occupational Therapy (NCBOT) exam.

Completers of Welding and Metals Technology take the American Welding Society (AWS) Level I exam.

Results of these exams are reported in each program’s assessment plan, reviewed by faculty and program

advisory committees, and are used in curriculum revision. Results of these outcome exams are cited in Figure

26 for the 2015-2016 academic year.

Figure 26: Licensure, Certification, Exit Exams (2015-2016) Program Outcomes Assessment Number Tested Pass Rate

Associate Degree in Nursing (Fall 2015 and Spring 2016 Grads)

NCLEX-RN Exam 42 83.33%

Certificate in Practical Nursing (Summer 2015 Grads)

NCLEX-PN Exam 38 100%

Business Office Administration

OPAC Exam 9 89%

Computer Information Technology

CCENT Exam 20 95%

Law Enforcement

Basic Peace Officers (POST) Exam

22 100%

Occupational Therapy Assistant NBCOT Exam 16 100%

Welding & Metals Technology AWS Exam 1 100%


Degrees Conferred The number of degrees and certificates conferred is one indicator of successful student academic achievement

and goal attainment. Figure 27 lists the number of graduates by program over the last five years (2011-2012

to 2015-2016).

Figure 27: Degrees Conferred (2011-2012 to 2015-2016)

2011-2012 2012-2013 2013-2014 2014-2015 2015-2016

Associate Degrees 681 659 750 711 733

Associate of Arts 508 481 549 543 551

Associate of Arts in Teaching 34 41 62 24 41

Associate of Science 8 5 8 10 10

Associate Degree in Nursing 24 36 36 32 23

Accelerated Associate Degree Nursing 8 10 14 25 19

Associate of Applied Science 99 86 81 77 89

Business Accounting Tech 7 7 10 1 0

Business Administration, Accounting 0 0 0 1 6

Business Administration, Management 0 0 0 0 3

Business Administration, Marketing 0 0 0 1 1

Business Office Administration 0 0 0 4 8

Business & Office Tech, Executive 1 8 3 1 0

Business & Office Tech, Legal 0 3 0 0 0

Business & Office Tech, Medical 19 10 5 4 1

Computer Information Tech, Programming 5 2 6 3 10

Computer Information Tech, Networking 8 8 6 9 10

Drafting Design Technology/EDD-Arch & Mech Design 2 3 4 4 1

Early Childhood 8 8 11 10 5

Graphic Arts Technology 11 6 3 10 8

Ind. Electronics/Electricity Tech 1 0 0 0 0

Industrial Tech - Machine Tool 1 1 0 1 1

Industrial Tech – Industrial Maintenance 1 2 2 5 1

Industrial Tech – Mechatronics/ES-Mechatronics 0 0 0 1 6

Industrial Tech, Power Plant Operations 11 2 0 0 0

Marketing/Management 4 3 10 1 4

Medical Laboratory Tech 7 5 5 9 7

Occupational Therapy Assistant 11 17 16 12 16

Welding & Metals Tech 3 1 0 0 1

Certificates 102 84 92 76 70

Accounting 0 1 0 0 1

Business & Office Administration 0 0 0 1 0

Business & Office Tech, Medical 1 0 0 1 0

Computer Information Tech – Networking 1 2 2 0 2

Early Childhood 0 0 1 1 1

Graphic Arts Tech 1 1 0 2 1

Industrial Tech/ES- Maintenance 0 2 1 0 1

Law Enforcement 46 32 39 22 25

Marketing/Management 1 0 0 0 1

Practical Nursing 52 45 49 49 38

Welding & Metals Tech 0 1 0 0 0

Certificates of Specialization 1 0 5 3 1

Computer Support Specialist 0 0 1 0 0

Early Learning Assistant 1 0 2 1 0

Management 0 0 0 0 1

Networking 0 0 0 1 0

Phlebotomy 0 0 0 1 0

Programming 0 0 1 0 0

Systems Operations 0 0 1 0 0


According to IPEDS provisional data on College Navigator, the statewide graduation rate for public community

colleges in Missouri is 23%. MACC’s graduation rate for the fall 2012 cohort was 27%. The graduation rate

reflects the number of students who completed their program of study within three years. The rate includes

full-time certificate-seeking or degree-seeking students who were not previously enrolled in any institution.

Figure 28 shows the total number of students (by race and gender) who enrolled at MACC in fall 2012 and

completed their program by August 31, 2015.

Figure 28: Graduation Rate (Fall 2012 Cohort--Full-time Certificate or Degree-seeking Students) Year 2012

Non-Resident

Alien

Black or African-

American

American Indian or

Alaska Native

Hispanic White Asian Two or

More Races

Native Hawaiian or Other Pacific

Islander

Race/ Ethnicity Unknown

Total Students

Initial Cohort

Men Women

0 2

35 47

0 3

7 13

363 511

6 5

14 21

0 1

0 0

427 602

Total Students

2 82 3 20 874 11 35 1 0 1,028

Completers of Programs

Men Women

0 0

4 3

0 0

2 2

96 159

2 3

1 2

0 0

0 0

105 169

Total Students

0 7 0 4 255 5 3 0 0 274

FOLLOW-UP ASSESSMENT 180-Day Placement Survey

MACC’s Job Placement Office conducts an annual 180-day follow-up study – as required by DESE – for all career

program graduates each year. The basic intent is to report the employment status, continued education

patterns, and salaries of the graduates by program.

Figure 29 indicates the results of the follow-up study for 2014-2015 MACC Career and Technical Education

program graduates. As shown, 89% of MACC graduates were positively placed (ER, CER, and MR). MACC had

an overall placement rate (includes positive placement plus ENR and CENR) of 96%.

Figure 29: Results from 180-Day Follow-up Study for 2014-2015 CTE Graduates Program Grads ER ENR CER CENR MR NP UNK NAP Avg. Wage

Bus Accounting Tech/ Bus Admin

2 2 0 0 0 0 0 0 0

$14.00

Bus Off Tech/Buss Off Admin

11 8 2 0 0 0 0 1 0

$9.19

Comp Info Tech 13 8 2 2 0 0 1 0 0 $16.83

Drafting Design Tech 4 3 0 0 0 0 1 0 0 NA

Early Childhood Ed 12 8 1 2 0 0 0 1 0 $9.67

Graphic Arts Tech 12 3 5 3 0 0 0 1 0 $10.09

Industrial Tech 7 6 1 0 0 0 0 0 0 $11.03

Law Enforcement 22 18 1 2 0 1 0 0 0 NA

Mkt/Mgmt 2 0 1 1 0 0 0 0 0 NA

Med Lab Tech 10 9 0 1 0 0 0 0 0 $15.60

Occup Therapy Asst. 12 9 0 1 0 0 2 0 0 $12.54

ADN 57 50 0 7 0 0 1 0 0 $20.66

PN – Mexico 23 18 0 4 0 0 0 0 0 $12.54

PN – Moberly 26 14 3 9 0 0 0 0 0 $13.98

TOTALS 213 156 16 32 0 1 5 3 0 $16.31* ER Employed, Related ENR Employed, Not Related NAP Not Available for Placement CER Continuing Education, Related CENR Continuing Education, Not Related MR Military, Related NP Not Placed/Unemployed UNK Situation Unknown NA Wage not shared *A total of 49 graduates reported salary information.


Graduate Satisfaction Survey MACC’s graduation application includes a satisfaction survey section with questions that ask graduates to rate 1) the quality of their educational experience, 2) the quality of instruction, and 3) the quality of academic advising. Graduates are also invited to share general comments and are asked the open-ended question, “What would have improved your student experience at MACC?” Figure 30 illustrates 2015-2016 graduates’ average ratings by program on a scale of 1 to 6, with 1 being Very Satisfied and 6 being Very Dissatisfied. Figure 30: Results from Graduate Satisfaction Survey by Program (2015-2016 Graduates)

PROGRAM Quality of

Educational Experience

Quality of Instruction

Quality of Academic Advising

Number of Respondents

Transfer Associate Degrees

Associate of Arts 1.89 1.94 1.97 421

Associate of Arts in Teaching 1.63 1.96 1.85 27

Associate of Science 2.00 1.80 2.20 5

CTE Associate Degrees

Associate Degree in Nursing 1.68 2.05 1.64 22

Accelerated Associate Degree in Nursing

1.67 1.61 1.61 18

Business Administration-Accounting

1.67 1.50 1.50 6

Business Administration- Management

2.00 2.00 2.00 3

Business Office Administration 2.00 1.86 2.57 7

Computer Information Technology, Networking

1.33 1.44 2.11 9

Computer Information Technology, Programming

2.63 2.38 2.75 8

Early Childhood 1.60 1.60 1.80 5

EDD – Arch & Mech Design 4 3 3 1

Graphic Arts Tech 2.00 2.00 2.00 5

Ind Tech, Mechatronics/ Eng Systems, Mechatronics

2.00 1.33 1.67 3

Ind Tech, Machine Tool 1.00 1.00 2.00 1

Marketing/Management 1.50 1.25 1.50 4

Medical Laboratory Technology (MLT)

1.25 1.50 1.25 4

Occupational Therapy Assistant (OTA)

1.50 1.69 1.94 16

Welding & Metals Technology 3 2 2 1

CTE Certificates

Accounting 2 2 2 1

Computer Tech Networking 2 2 1.50 2

Graphic Arts Technology 1.00 1.00 5.00 1

Law Enforcement 1.44 1.50 1.56 18

Marketing/Management 1 1 1 1

Practical Nursing 1.79 1.74 1.89 19

The survey results indicate that the majority of graduates were satisfied with their educational experience, the

quality of instruction, and the quality of advising at MACC. Results of the survey are shared with President’s

Council and are included in each program’s assessment plan.


Student Transfer There are two sources of student transfer data available to MACC – the Statistical Summary published by the

Missouri Department of Higher Education (MDHE), which presents the collective data from all of the state’s

public colleges and universities, and the National Student Clearinghouse service, which is used by the Financial

Aid Office to track student financial aid records. MACC typically uses National Student Clearinghouse data for

all internal and external reporting and comparison purposes as National Student Clearinghouse captures

transfer data on students who transfer to public and private schools across the US whereas MDHE only captures

transfer data on students who transfer to public institutions within the state of Missouri.

According to National Student Clearinghouse data, 61% of MACC graduates from the fall 2013 first-time, full-

time freshmen cohort also transferred on to a four-year institution by August 31, 2016. As Figure 31 illustrates,

the vast majority (94%) of MACC graduates from the cohort chose to transfer to a 4-year institution in Missouri.

MACC’s top two transfer institutions accounted for almost two-thirds (65%) of the 4-year transfers -- University

of Missouri-Columbia (54 students) and Columbia College (34 students).

Figure 31: 4-year Transfers of Graduates from the Fall 2013 First-time, Full-Time Freshman Cohort

Many community college students opt to transfer before completion of their program. Figure 32 shows the

breakdown of the 4-year transfers of non-graduates from the fall 2013 cohort. Although University of

Missouri- Columbia and Columbia College are still the top two transfer institutions, the pie chart illustrates that

they account for approximately two-fifths (42%) of the transfers of non-completers rather than two-thirds. The

4-year transfers of non-completers are spread across 50 separate 4-year institutions, which is more than

double the number of 4-year schools to which the graduates transferred. It is also worth noting that 25% of

non-completers transferred to out-of-state institutions as opposed to just 6% of the completers.

University of Missouri-Columbia

40%

Columbia College

25%

Other Missouri Public

Institutions19%

Other Missouri Private

Institutions10%

Out-of-State Public

Institutions4%

Out-of-State Private Institutions

2%


Figure 32: 4-year Transfers of Non-Graduates from the Fall 2013 First-time, Full-time Freshman Cohort

MACC encourages student transfer by having the following in place:

Having accredited programs

Developing and updating articulation agreements with area schools

Participating in Missouri Transfer Course Library

Developing new partnership agreements with 4-year institutions (e.g., Columbia College and Central

Methodist University offer programs at the Columbia location)

Inviting other institutions to present and provide one-on-one transfer counseling to MACC students

Evaluating the transferability of programs and courses in regular academic program reviews

Realigning programs to improve transfer (e.g., Associate Degree of Science in 2013)

Tracking student transfers

Setting measurable transfer goals in 2015-2018 Strategic Plan

OTHER ASSESSMENT EFFORTS

Program Reviews

In summer 2015, program reviews were conducted for Developmental Education, Dual Credit, Graphic Arts and

the Associate Degree of Nursing. Results were shared during Board of Trustees work sessions in September

and October 2015.

Employee Satisfaction

The Assessment of Support Services Committee distributed its fourteenth Faculty/Staff Assessment of Support

Services Survey during the spring 2016 semester regarding the quality of support services across locations. The

survey was offered online. Employees could access the survey from any computer and complete it in one session

or multiple sessions as their schedules allowed.

University of Missouri-Columbia

28%

Columbia College14%

Other Missouri Public Institutions

15%

Other Missouri Private Institutions

18%

Out-of-State Public Institutions

12%

Out-of-State Private Institutions

13%


In 2012, the Faculty/Staff Assessment of Support Services Survey was split into two parts to prevent survey

fatigue. Departments are surveyed every other year. Twelve areas were evaluated in spring 2016 and included

the following:

A total of 113 faculty and staff completed the Faculty/Staff Assessment of Support Services Survey. This

represented a 42% response rate.

Employees were asked to respond to four statements about each support area by choosing one of the following

options: 1-Strongly Disagree, 2-Disagree, 3-Agree, 4-Strongly Agree. The college-wide average and low-high

ranges for each statement are shown below in Figure 31 that includes four years of comparative data.

Figure 31: Results from Faculty/Staff Assessment of Support Services Survey (2013-2016)

#1 Staff are approachable and

friendly.

#2 Business is generally conducted in

a timely fashion.

#3 Current and accurate information is

provided.

#4 Overall, I am generally satisfied with

the service this department provides.

2013 2014 2015 2016 2013 2014 2015 2016 2013 2014 2015 2016 2013 2014 2015 2016

College-Wide

Average 3.44 3.43 3.47 3.44 3.40 3.35 3.40 3.40 3.43 3.34 3.44 3.42 3.40 3.35 3.43 3.42

Range Low - High

3.24-3.76

3.24-3.61

3.21-3.74

3.09-3.68

3.06-3.75

3.10-3.56

3.02-3.71

3.09-3.61

3.20-3.73

3.07-3.53

3.22-3.71

3.11-3.64

3.15-3.71

3.12-3.50

3.20-3.72

3.14-3.63

The supervisors of each office received the following survey data: individual office results, college-wide results,

an average score for each question, a low/high range for each question, and the typed responses to two open-

ended questions about their departments. Supervisors were encouraged to share the results within their

departments as appropriate and to use the results to develop goals for improvement.

Due to the quality of feedback received on previous surveys, in spring 2008 the “Comments” section was

replaced with two questions of a more focused nature: 1) “What are the strengths of this department?”, and

2) “What changes could this department make to improve services?” It was found that these questions helped

elicit more constructive feedback from survey participants, making it easier for positive departmental change.

CONCLUSION In gauging its progress, Moberly Area Community College demonstrates success in continually moving

assessment forward. The focus for the future is on ensuring completion of the feedback loop for all assessment

processes, creating clear structures for decision-making processes as related to assessment, and using

assessment results to drive improvement in student learning. MACC’s 2015-2018 Strategic Plan emphasizes

the importance the College places on teaching, learning, and continuous improvement. Operationalizing the

new plan is resulting in growth and will keep MACC’s focus squarely on its mission.

1. Activity Center 2. Advanced Technology Center (Mexico) 3. Columbia Higher Education Center 4. Computer Services 5. Financial Aid 6. Hannibal Area Higher Education Center

7. Institutional Development and Alumni Services 8. Instructional Technology 9. Kirksville Higher Education Center 10. LARC (Library and Academic Resource Centers) 11. Security 12. Student Services

Documents

CONTINUOUS IMPROVEMENT - MACC...Continuous Improvement Report 3 committee members’ knowledge of assessment, and drafting assessment plans for each degree and certificate program