NWCCU Focused Interim Report - MSU Billings › assessment › pdf › MSUB_Focused_Int… · NWCCU Focused Interim Report ... (Stage 3), departments are asked to report the results

MONTANA STATE UNIVERSITY BILLINGS

NWCCU Focused Interim Report

May 2010

Table of contents

Introduction

Recommendation 1 Addressing a culture of assessment ..………………………......... 1

General Education Assessment ……………………………………….………………….............. 4

Academic Program Assessment ………………………….……..………….………………..… 6

Recommendation 2 Systematic evaluation of all faculty ………………………….…... 11

Recommendation 3 Review of mission and operations …………………………....…. 13

Recommendation 4 Balancing available resources ………………………………….…… 15

Conclusions and Future Plans ………………………………….……………………………….…… 18

Appendices

Appendix 1 General Education outcomes revisions ..…….…………………….……...… 22

Appendix 2 Stages of Assessment………...………………………………….……………….…… 26

Appendix 3 Examples of Changes to programs or Assessment Processes .….… 32

Appendix 4 Timelines and Accomplishments ……………………………………..………..…. 38

General Education ………………………………………………………………………………. 39

Evaluation of all Faculty …………………………………………………………………….… 40

University Governance ……………………………………………………………………….. 42

Resource Management ………………………………………………………………………. 43

Appendix 5 MSU Billings Mission Review Document…………………………..………..…. 36

Introduction This Focused Interim Report addresses findings following a Northwest Commission on Colleges and Universities (NWCCU) full‐scale accreditation visit to Montana State University Billings in October 2008. The NWCCU review team made four recommendations, asking the university to fully address:

• Implementing a culture of assessment. • Establishing a systematic method for the evaluation of all faculty. • Participating in a comprehensive review of the university’s mission and operations as

part of the Montana University System and the Montana State University family. • Balancing all available resources to match the mission and operations.

The full evaluation and recommendations provided valuable insight and an opportunity to focus on areas where some progress was being made at the university, but was not systematically embraced and/or understood. The university began taking action on the recommendations shortly after the October 2008 visit. By December 2008, MSU Billings had developed a 30‐page corrective action plan, listing action items and timetables for each of the recommendations. During the summer of 2009, a SharePoint website was developed to compile, track and assess each of the corrective action items.* This report summarizes progress made by MSU Billings in addressing NWCCU’s four recommendations.

• The Recommendation 1 report focuses on creating and building a culture of assessment.

• Recommendation 2 report focuses on establishing a University‐wide “systematic and widely communicated procedure for the evaluation of all faculty,” addressing a concern that not all part time faculty and professional staff who teach classes have clear guidelines and systematic departmental evaluations.

• The Recommendation 3 report focuses on the relationship of MSU Billings with and within the Montana University System and the Montana State University family, specifically fully understanding the “distinctive identity” of MSU Billings and its “authority to operate within its assigned mission.”

• The Recommendation 4 report focuses on taking steps “to evaluate and better match resources with mission and operations,” fully recognizing MSU Billings may not have the human or financial resources to offer all the programs and services it currently offers and supports.

The process of addressing these recommendations has been dynamic, inclusive and revealing. Cultural changes are still evolving in many areas and scores of faculty, staff and administrators have been involved in discussing and developing solutions. This process has also has been running parallel with Montana University System mission review processes, discussions of

reform, a new emphasis on two‐year education and the inaugural year of a new president of Montana State University in Bozeman. MSU Billings understands continued improvement does not end here. The actions and initiatives outlined in this report will be a part of the foundation for guiding the university in developing strategic directions in the future, reflective of its mission as a comprehensive urban university. * The SharePoint site includes documentation relating to the NWCCU recommendations. It includes the Full‐Scale Evaluation Committee Report, the Montana State University Billings Corrective Action Plan dated December 31, 2008 and letters from NWCCU relating to the focused visit. It also lists various accreditation tasks and timelines, assessment plans, status reports and results of assessment activities. The site is used to store general and specific materials for ALL future accreditation progress reports and visits, including the College of Business’s AACSB visit in January 2010, the music discipline National Association of Music Schools (NASM) in April 2010, and the College of Education’s OPI/NCATE visit, also in April 2010. Please see the introduction to Appendix A for instructions on accessing the materials.

1 | M S U B I L L I N G S F O C U S E D I N T E R I M R E P O R T / M A Y 2 0 1 0 – d r a f t 4 . 1 3

Recommendations, Responses and Actions RECOMMENDATION 1

“The Committee acknowledges the progress made implementing educational program planning and assessment, but it did not find evidence that this effort is yet truly University‐wide in operation and that it encompasses all of its offerings. The Committee therefore recommends that steps be taken to ensure that all academic programs are completing fully the process of learning outcomes definition, evaluation, analysis of results, and demonstration that, on the basis of such evaluation, curricular and pedagogical changes are made as needed.” (Standard 2.B.1 and Policy 2.2)

I. SUMMARY Assessment of student learning has been, and continues to be, a dynamic process at MSU Billings, resulting in a major cultural shift across the university. Although some academic programs have yet to complete the final stage in the assessment process, significant progress has been made across all programs and services. Recommendation 1 focused on the NWCCU findings with regard to Standard 2.B.1 and Policy 2.2. The primary findings indicated:

• A regular periodic process of assessing student learning outcomes is uneven among programs and should be fully integrated within all programs.

• A formalized system of assessment does not appear to be consistently developed and implemented across all Academic Foundation areas.

Since the October 2008 team visit, MSU Billings can demonstrate significant progress and compliance. Two of the colleges — the College of Business (AACSB candidacy) and the College of Education (NCATE reaccreditation) — have demonstrated a culture of assessment as have other programs which have disciplinary national accreditation. Both COB and COE are preparing for their national accreditation reports in Spring 2010. The NCATE/OPI visit took place April 11‐14. The COB had a successful visit in January 2010 and on April 22, 2010, the AACSB governing board announced officially that MSU Billings had been accredited. The university has adopted a four‐stage process to analyze and track progress in assessing student learning outcomes for degree programs. The process begins with assessment planning (Stage 1), followed by implementation (Stage 2), and concludes with analysis and recommendations for improving student learning (Stage 3). The number of programs in Stage 3 has more than doubled from June 2009 to April 2010. Specifically, assessment activities have grown from 15 degree programs and all initial programs in one college (the College of Education) in June 2009 to 32 degree programs and all initial and advanced programs at the


COE. The university has gone from 22 programs having no assessment plans (Stage 0) in June 2009 to having zero programs at Stage 0 in April 2010. That means all degree programs have completed at least the planning stage (Stage 1) of the assessment process. The NWCCU recommendation on assessment emphasized the unevenness of regular periodic processes for assessing student learning. The completion of assessment plans for each degree program is evidence of a broader culture of assessment at MSU Billings. Faculty have learned about the nature of assessment and have developed plans for courses and degree programs. By the end of spring 2009, 15 programs were at Stage 3, four programs were at Stage 2 and all programs at the COE had developed and implemented assessment plans. Meetings with the remaining departments concluded with outlines for assessment plans —complete with learning objectives of the program, identification of student tasks to assess and assessment methodology, the point in the curriculum when the assessment occurs and the timetable for implementation. Assessment plans have been posted on the SharePoint site as they are created or modified. The implementation stage varies among degree programs. With the completion of the assessment plans, 16 Stage 1 programs were able to conduct assessment in Fall 2009. Fifteen additional program assessments have occurred during spring 2010. Approximately seven programs will be conducting assessment for the first time in Fall 2010 or in Spring 2011, either because the program or assessment process is new or because a capstone course (where most assessment occurs) will not be available until then. During the analysis stage (Stage 3), departments are asked to report the results in a template prepared for each program. The template includes:

• A table that includes the year and semester during which assessment occurred. • The task or activity assessed (and the course it came from, if applicable). • The place where assessment information is stored for future reference. • The results of that assessment. • The knowledge/skills/abilities/areas/topics/situations to improve (identified from the

results). • Recommendations for changes to the program or the assessment process. • Changes already implemented.

As of March 2010, 31 programs that are now in Stage 3 have prepared and submitted assessment reports for posting on the SharePoint site. The assessment process at MSU Billings has been more than an exercise. That process has led to actual and beneficial change, providing more evidence of a culture of assessment. Programmatic alterations include changes in the curriculum, course content, course delivery, and/or teaching techniques to enhance student learning. Reports show several programs have modified or added training experiences, offered a new course or seminar, added content across the curriculum, developed better feedback mechanisms, added online homework, modified the


curriculum, revised course content, offered online courses and developed better assessment methods. (See the attachment for a list of accomplishments in “closing the loop.”) While policy and procedure has changed, there are other indicators of change in the classroom and in programs. There is an emerging general culture of assessment at the university that did not exist before. Indeed, in October 2008, assessment efforts were uneven. The university can now document the assessment progress in the College of Allied Health Professions, College of Arts and Sciences, the General Education* Program and College of Technology, especially in developing a formalized and regular system of assessment. Mechanisms for assessment of learning objectives for programs in these colleges are now in place, or actively being developed, as part of the university‐wide effort to create a culture of assessment of student learning. *General Education was called Academic Foundations and now has returned to the title General Education with the Academic Senate Committee now entitled General Education Committee (GEC) II. PROGRESS SINCE OCTOBER 2008 Organizational Support Objective: Create an infrastructure needed to facilitate the implementation and reporting of assessment activities. Strategies:

1. Improve coordination of assessment activities for academic programs and Academic Foundations/General Education areas.

2. Establish a university‐wide reporting system for assessment results. Accomplishments: 1. On March 25, 2009, the General Education Committee (formerly the Academic

Foundations Committee), which is part of the Academic Senate, was reorganized and expanded to ensure a member of the College of Arts and Sciences is represented for each General Education area. This began the important discussion of General Education assessment and the committee met weekly for the rest of the spring semester and is now is meeting every other week. (See AFC Minutes in SharePoint)

2. In February 2009, an assessment specialist and faculty member was given the

responsibility for coordinating assessment of programs. She has reported directly to the Provost and was reassigned from her teaching responsibilities for Fall 2009 and Spring 2010. This assessment specialist has met with EVERY program and assessed where they were in terms of a comprehensive and coordinated assessment activities starting in October, 2009 to the present. She developed a method to demonstrate progress in the four stages of assessment (from Stage 0 to Stage 3), helped others make progression


along this rubric and developed a university‐wide reporting system. (see Assessment of Academic Programs for descriptions of the stages and the appendix for the status of university programs in SharePoint)

General Education (GE) Assessment Among the notable changes in the committee’s work at the beginning of the 2009‐2010 academic year was a change in the name of the committee from the “Academic Foundations Committee” to the “General Education Committee” (GEC). This change was not merely semantic. The committee noted one of the biggest problems with the previous committee’s work was that it was disconnected from the faculty who taught in the program and as a result, few faculty really understood what they were working toward. Part of that was the name of the committee, laden as it was with philosophical context. Added to that was the unworkably large number of things the committee was asking faculty to measure and track in their classes. So, one of the most fundamental things the committee did was to decide that the program was going to get back to first principles, and the first step of that effort was to re‐envision the committee’s public persona. The name change to the General Education Committee was an effort to demonstrate to faculty the committee was dedicated to seeing to it they felt empowered and integrated into the goals of the program. The second step in this process was to revise the student learning outcomes of the courses in the program. Chief among the concerns of faculty teaching within the program was the sheer number of items the committee asked them to measure and track. These assessment items were chosen by faculty teaching the program from a long list of options devised and provided by the committee. The committee asked faculty to pick six or seven appropriate items. This meant each faculty member teaching in the General Education program had to measure those items for each of the sections they taught in the program. This quickly became an untenable workload demand. The Northwest accreditation team visitors noted this in Fall 2008. Once the General Education Committee was reformulated, it was clear the learning outcomes measured within the program needed to be revised. This effort, begun early in Fall 2009, has been completed and includes new outcomes for all General Education courses. The rapid revision of the learning outcomes was possible principally because the committee conveyed to the faculty they were immediately responsible for the place of their course within the broad scope of the General Education program. It followed, therefore, they were responsible for crafting the new outcomes to replace the outcomes imposed upon the faculty by the previous committee. (See Revised General Education Outcomes, 2010, in SharePoint)


On a course‐by‐course basis, the faculty at MSU Billings constantly assess student outcomes, program effectiveness and student success toward mastery of these outcomes. It is noteworthy that the aggregate average of measures of student success in meeting learning outcome goals in General Education courses was higher in Fall 2009 than at any time since the assessment system was put into place. Moreover, not only was the overall average at a record high (as demonstrated in the chart above) but in seven of 10 categories, percentages of students who met outcomes goals was the highest ever. However, the faculty and GEC recognized applying the aggregated assessment data to make specific changes to specific individual courses is inappropriate. MSU Billings faculty have regularly scored in the 80+ percentile of achievement in the student learning outcomes for their General Education courses, clearly showing how faculty and departments are closing the loop on a regular basis. A second area of accomplishment for the GEC is the completion of a guidance spreadsheet for use by the Advising Center. Following discussions between committee members and Advising Center staff regarding the COMPASS test administered to freshmen, more attention was given to matching competencies measured in tests with courses. It was found that some students who place at a certain level of Math competency, for example, were admitted into courses such as Chemistry that — on the face of them — had no absolute Math prerequisites, but assumed a


higher level of Math competency. The General Education Outcomes Charts (see General Education Outcomes Charts, Fall 2007‐Fall 2009 in SharePoint), particularly the Spring 2009 chart, illustrate the connection between Math competency and success in the Life Sciences. Following lengthy discussions and consultations with faculty in the GE program, the committee devised an advising spreadsheet for use in the Advising Center to more effectively inform students of the competencies expected of them in various courses. A third area of on‐going discussion by the GEC is the place of the First‐Year Experiences (FYEs) in the GE program. Members of the GEC attended the Association for General and Liberal Studies (AGLS) Conference in St. Louis and the First Year Experience Conference in Denver and shared their findings with the committee. The GEC is engaged in conversations about how an FYE can play a role in the GE program, as well as value‐added initiatives such as a new pilot course designed to combine FYE course outcomes with service learning and civic engagement experiences. This pilot program will begin Fall 2010 and has the full support of Academic Senate and the GEC. The committee members recognize reform of FYEs is a very time‐consuming and complex effort, for which quick answers are elusive. As the pilot program gets under way in Fall 2010, the GEC remains open to discussion about including an FYE into the General Education core, although inclusion at this time is not possible based on initial deliberations. Issues under discussion currently include establishing a process whereby courses within the General Education program can be periodically vetted and reviewed for continued inclusion in the program. The committee decided the leaders best positioned to begin that review are the department chairs of disciplines within the program, and this review needs to take place in the fall of a catalog year. Therefore, the committee will continue to work to establish effective protocols for this review process and present it to the academic department chairs as soon as possible. Finally, the GEC is continuing its study on the place of standardized testing in General Education assessment efforts. The committee has urged the provost to maintain communications on progress toward implementation of a standardized, normed, and reliable assessment tool. Timeline (See Appendix 4 on Timelines, Page 29) Academic Program Assessment

Objective: Each academic department will develop and implement an assessment plan for each degree program in the department, including Masters, Bachelors, and Associates degrees and Certificates. Strategies:

1. Conduct an inventory of current assessment activities across departments and programs.


2. Ensure an assessment plan exists or is developed for each program. 3. Report the status of assessment activities in each department. 4. Provide examples of changes made to programs or assessment processes due to assessment data. Accomplishments: Inventory of assessment activities The College Business (COB) assessment specialist serves as the principal assessment coordinator for the AACSB accreditation as well as serving as the lead university assessment specialist for the Provost. Assessment analysis and discussions began at the College of Education (COE) in fall 2005 and continued through spring 2008. Through the efforts of the standing COE Assessment Committee, the redesigned Unit Assessment System was approved during the fall semester of the 2008‐09 academic year and was fully implemented in December 2008. Data from the portfolio system, based on the original 2000 conceptual framework, demonstrated a need to re‐design the conceptual framework and simplify the cumbersome details and number of cells required for candidate completion and data input. A review of the alignment of the course outcomes, student portfolio data, and performance assessments with the conceptual framework indicated to the unit a need to revise and restructure the Conceptual Frameworks. The Initial Conceptual Framework, which was originally heavily based on Interstate New Teacher Assessment and Support Consortium (INTASC) standards, was revised during the fall of 2007 to reflect the specific INTASC principles with modifications to meet Montana state requirements and institution standards. In Spring 2009, the dean of the College of Education (COE) met with the university assessment specialist to evaluate the status of the assessment activities for NCATE accreditation. COE faculty members and staff have been assigned responsibility for this process. Assessment data for the COE has been and was being collected through the summer for their advanced and initial programs. Evaluation of this information took place during a Fall 2009 retreat and the final draft of the report was sent to the NCATE and state team in February 2010. Meetings involving the university assessment specialist, Provost and department chairs in the College of Allied Health Professions (CAHP) and in the College of Arts and Sciences (CAS) were held in Spring 2009 to evaluate the status of assessment activities for all programs in those colleges. The College of Technology (COT) had an outside consultant visitation in May and August 2009 to provide additional guidance towards developing and implementing assessment plans, particularly for programs in the Industry/Computer and Transportation/Business departments. The consultant met with individual faculty members in various program areas. The COT program directors submitted annual reports in May 2009, which included data on assessment to date. Annual reports and assessment activities are combined for programs offering both an


Associate’s degree and a Certificate. Therefore, the assessment plan for an Associate’s degree also applies to the Certificate offered in that area of study. Stand‐alone Certificate programs are reported separately as individual programs. Assessment plans Department chairs and faculty with programs in Stage 0 were assisted in developing assessment plans throughout the summer and fall of 2009. The College of Allied Health Professions, the College of Arts and Sciences, the College of Business and College of Technology have a combined 53 assessment programs to implement. As of the end of January 2010, all programs have developed assessment plans. An assessment plan template was created to assist department chairs and faculty in formalizing assessment plans for Stage 0 programs. The template is also used for compiling assessment plan information in a formalized way for all programs. Each plan includes learning objectives of the program, assessment tasks and measures, assessment points and procedures, and a timetable for implementation.

• Assessment tasks used across programs are nationally‐normed tests and/or course‐embedded student activities which involve direct measures of student learning. Many assessment plans also include indirect measures of assessment by conducting surveys of graduating students, alumni or employers on the learning objectives of the program. Each task is mapped to specific learning objectives.

• Assessment points and procedures include the details of implementing the assessment process. The majority of programs have a capstone experience with students performing tasks that demonstrate achievement of learning objectives. Where assessment plans are completed, decisions have been made regarding when and how knowledge tests and surveys will be administered. Some tests and surveys are paper and pencil types; others are administered online.

• The timetable information indicates when each of the assessment tasks will be administered or information on results will be collected. Some programs have information collected in spring 2009; most will have assessment information in fall 2009 or spring 2010. The timetable also indicates how often assessment information will be collected and analyzed (mostly once per year except for some surveys conducted every other year). The timetable information also designates the person in charge of collecting the information or administering knowledge tests or surveys.

Assessment plans will be accessible via SharePoint to all departments. Status of assessment activities At the beginning of Fall 2009, department chairs received action items for the Fall and Spring semesters to implement assessment plans and report results in preparation for the NWCCU focused visit.

• Department chairs and faculty with programs in Stage 0 were assisted in developing assessment plans.

• Departments with Stage 1 programs are asked to complete assessment plans and begin


gathering assessment information. • Departments with Stage 2 programs are asked to evaluate information and begin a

process for implementing curricular or pedagogical changes if necessary. • Departments with Stage 3 programs are asked to report results of the assessment

process, included changes to the program or assessment process. Departments continue to store assessment documents, data and information in electronic form accessible to everyone in the department. Action items are reported in status documents prepared for each assessment plan and status documents are updated on a continual basis and posted on SharePoint. Departments regularly summarize and report their assessment results. An assessment reporting template has been created for this purpose. These reports are posted on the SharePoint website when completed. Status of assessment planning: At the end of the Spring 2009, approximately one‐half of the 53 programs had undeveloped assessment plans. By the end of January 2010, all programs had completed Stage 1 and have developed assessment plans. Status of assessment results: At the end of the Spring 2009, 14 programs were determined to be in Stage 3. As of March 2010, 17 additional programs have reached Stage 3 and have implemented the assessment plan for some or all of the learning objectives, evaluated the results and made recommendations for changes to the program or the assessment process. Thirty of the Stage 3 programs have prepared and submitted assessment reports for posting on the SharePoint site. In addition, all College of Education assessment reports are posted on the SharePoint site. Fifteen programs are scheduled to collect assessment information during Spring 2010 and seven programs will collect assessment information in Fall 2010 or Spring 2011. Changes made to programs or assessment processes using assessment data Several curricular and programmatic changes were made at the university because of student outcomes assessment. Evaluated and discussed at all academic levels and in all colleges, these changes were made to accomplish a number of goals. For example, a seminar class was added at the College of Education to help enrich the classroom and student experience; online courses were added to the Paramedic Program to help students make better academic progression; systems to provide better student feedback were implemented in the College of Business; and orientation was improved in the Athletic Training internship program to give students and mentors a better understanding of goals and expectations. A full list of the changes in both program/curricular areas and in assessment can be found in Appendix III, Page 28. Brief descriptions of changes to programs, curriculum or content that have been implemented include:

• Modifying or adding training experiences, internships or field experiences (COE, Paramedic, Rad Tech, Masters in Rehab)

• New seminar class (COE) • Additional content across the curriculum for specific topics (COE, COB, Nursing)



• Better feedback to students (COB) • Additional homework online (COB) • Modified curriculum (COE, COMT, Computer Systems, Athletic Training, Masters in

Rehab) • New course (COE, COMT, Paramedic, Computer Systems, Auto Tech) • Revisions in course content (Psychology, Computer Systems, Drafting and Design,

Masters in Rehab) • Online offerings of courses (Paramedic) • Identification of poor performing students and subsequent counseling (Rad Tech) • Better orientation for internships (Athletic Training) • Recommendation for increased curriculum content (Biology) • Recommendation for different assignments (Accounting Tech) • Recommendation for more emphasis of certain skills (Auto Tech) • Recommendation for additional training (Welding, Construction) • Recommendation for overseas study (Spanish)

Brief descriptions to changes to assessment processes include:

• New assessment forms (COE, Paramedic, HHP Teacher Certification) • Improved rubrics (COE, Welding) • Improved processes for rating students (COE) • Better assessment tasks (COB, COMT, Paramedic) • New pre‐test/post‐test process (Art, COMT, Construction) • Recommendations for new method for surveys (CAHP) • Recommendation for modified survey questions (REHA) • Recommendation for a different assessment task (MATH, Associates in REHA) • Recommendation for additional questions to the Career Services survey (Auto Tech) • Recommendation for improved data collection (Diesel Tech, Construction)


“The Committee recommends the University establish a systematic and widely communicated procedure for the evaluation of all faculty, including part‐time instructors, individuals on Letters of Appointment, and others with teaching responsibilities.” (Standard 4.A.5, 4.A.10, and Policy 4.1)

I. SUMMARY MSU Billings faculty members under full‐time contracts (including fixed‐term, lecturers, adjunct, tenure track and tenured faculty) take part in a yearly evaluation process. That includes data and materials to be submitted to department chairs to deans and then to the Vice Chancellor for Academic Affairs and, ultimately, to the Chancellor. Once University Lecturers have taught for two years at MSU Billings, they undergo a comprehensive evaluation every three years. Requirements for all faculty in these positions on the East Campus are detailed in the Collective Bargaining Agreement (CBA) which is reviewed every two years. Faculty classified as half‐time (.50 FTE/AY) to full‐time (1.00 FTE/AY) on the COT campus have a yearly evaluation process, but fall under a different bargaining agreement (VTEM). All tenure track, fixed‐term, and renewable non‐tenure track (RNTTA) faculty are evaluated annually. Tenured faculty are evaluated on a rotation of every three years. This faculty review and evaluation process involves student evaluations, a self‐evaluation tool, classroom observation by the dean, and a review of accomplishments and goals with the dean. The VTEM Union/Management Committee is currently reviewing the evaluation process including discussions concerning the addition of a peer‐review process and requirements for tenure which are not spelled out in the VTEM contract as they are in the senior campus CBA. For part‐ time faculty, the COT launched an evaluation pilot project in Fall 2009 for three part‐time instructors. The pilot was expanded in the Spring 2010 to include all part‐time faculty teaching traditional classroom‐based courses. In addition, two online part‐time faculty members were identified to participate in the Spring 2010 pilot. All COT classroom‐based and online part‐time faculty are expected to participate in the new evaluation process beginning Fall 2010. They will use a model partially adopted from a template designed by staff at Flathead Valley Community College, which received a similar NWCCU recommendation in 2007. (An example of that template can be found in SharePoint) Colleges, departments and programs differ widely on evaluation of part‐time faculty, including faculty and staff on Letter of Appointments (LOA) or Montana University System (MUS) contracts on the Main Campus and those on the COT campus. By Fall 2008, not all programs, departments or colleges had established a clear, annual review process that included more



than the required student assessment of faculty teaching of all courses taught each semester. If student evaluations suggest the instructor is having difficulties or if students complain to department chairs or program heads, then additional evaluation is triggered. Part‐time faculty on each campus have been removed during the semester or have not had their contracts renewed for poor teaching because of the evaluation process. II. PROGRESS SINCE 2008 Each of the colleges and its various programs have now designed a comprehensive procedure to ensure that all faculty — full‐time and part‐time — are evaluated. While these plans differ in details, they all have made progress. Evaluation status is identified in three stages:

• Stage 0: No systematic evaluation procedure except for full time Lecturers, Adjuncts, Fixed Terms, Tenure Track and Tenured faculty.

• Stage 1: Student Assessment of Faculty Teaching (SAFT) done for all faculty members — including those who teach part time —classes and reviewed by department chairs.

• Stage 2: Evaluation plans include peer reviews and self evaluations. Only the College of Arts and Sciences was at Stage 2 in October 2008, but the College of Education and the Academic Support Center were at Stage 1. The COT piloted a Stage 2 plan during the 2009‐2010 academic year. By the end of the spring semester 2010, all colleges except the COT will have launched a full‐scale Stage 2 effort. The COT faculty have developed a part‐time faculty evaluation model piloted by two instructors in Fall 2009. That model will be carried through by 15 instructors during Spring 2010 and will be used throughout the college by Fall 2010. (See chart for a summary, Appendix 2, Page 26 for timelines and accomplishments and the SharePoint for copies of each college plan) College Status October 2008 Fall 2009 Spring 2010 CAS Stage 2 Stage 2 Stage 2

COB Stage 0 Stage 2 Stage 2

COT Stage 0 Stage 2 (4%) Stage 2 (90%)

CAHP Stage 0 Stage 1 (70%) Stage 2

ASC COE

Stage 1

Stage 1

Stage 2

Stage 1

Stage 2

Stage 2

Library Stage 1 Stage 1 Stage 2

Source: College Plans. Note all categories are 100% unless otherwise noted. III. TIMELINE & ACCOMPLISHMENTS TO DATE

(See Appendix 4 Page 31)

Recommendations, Responses and Actions

RECOMMENDATION 3

“In light of the multi‐institutional governance structure currently in place, whereby MSU Billings reports to the Montana Board of Regents (BOR) through MSU Bozeman, the Committee recommends that the Board of Regents and these institutions engage in a comprehensive review of the mission and operation of MSU Billings so that the distinctive identity of the institution is established, that it is clearly communicated to its constituencies, that its authority to operate within its assigned mission is ensured, and that its relationship with other Montana public postsecondary institutions is clarified.” (Standard 1.A.1, 6.A.4, 6.B.5 and 7.A.1)

I. SUMMARY Dr. Waded Cruzado was selected as the new President of Montana State University in Bozeman in October 2009, and her tenure officially began in January 2010. Between the date of her hire and her arrival on duty, she was provided with information that outlined the specific and unique characteristics of MSU Billings. She made her first visit to the university and Billings community in late January 2010. President Cruzado spent the spring of 2010 becoming more familiar with each unit of the MSU subsystem, explaining her vision of the power of a singular university with distinct characteristics and resources in different parts of the state. As a land‐grant university, MSU has the opportunity to reach every citizen in every corner of Montana and MSU Billings will be a part of that larger picture. President Cruzado’s vision of Montana State University as “one university” with unique and separate parts is being discussed and implemented through a new four‐campus leadership council that President Cruzado formed in January 2010. The MSU Billings chancellor and other key university leaders are a part of that council and have active roles in determining the future course of the four‐campus system. All of this will allow substantive discussions to take place with the Office of the Commissioner of Higher Education and the Board of Regents as early as she chooses. Taking place on a parallel track, a working group of the Board of Regents is evaluating the various role and scope of the different units of the Montana University System. A final report and recommendations from that working group is expected to be presented in mid‐2010, which should further clarify the mission and operation of MSU Billings. In the meantime, MSU Billings has undergone a mission review process that helps clarify the university’s position as an urban university. Also taking place alongside all of this is the transition of leadership at MSU Billings that will



follow the retirement of Chancellor Ron Sexton in August 2010. An interim chancellor has been selected and a process will take place in the summer and fall of 2010 — led by MSU President Cruzado with involvement from leaders at MSU Billings and the Billings community— to select a new chancellor. II. PROGRESS SINCE OCTOBER 2008 Action Items and Strategies:

• The Chancellor and President of MSU Billings have had discussions regarding this recommendation and developed communication on the next steps.

• The Chancellor has been working closely with the Commissioner of Higher Education and OCHE staff to prepare a Montana University System perspective on the concerns noted in the recommendation.

• The Chancellor will provide the NWCCU a written response that addresses and communicates clearly a response to Standards 1.A.1, 6.A.4, 6.B.5 and 7.A.1.

TIMELINE & ACCOMPLISHMENTS TO DATE (See Appendix 4, Page 31)


“Acknowledging the University’s admirable commitment to serving the higher educational needs of its region, and, moreover, the commitment of its faculty, staff, administration, and Foundation to meet that mission. The Committee nonetheless observes that resources – both human and financial – are limited, and, particularly given the institution’s dependence on enrollment and the continuing demographic challenges, it recommends that steps be taken to evaluate and better match resources with mission and operations.” (Standards 1.A.4, 1.B.2, 1.B.4, 5.A.2, and 7.B.5)

I. SUMMARY Montana and MSU Billings are not immune to the current recession and the severe pressures the economic downtown has placed on higher education. However, it also presents a new opportunity for the University to wholly assess its institutional mission and values while fully evaluating priorities, resources and programs. State operating support for the Montana University System in the 2009 Montana Legislature was buoyed by one‐time‐only federal stimulus dollars and those dollars are not expected to exist for the coming biennium and may well decline. State leaders – from the Governor’s Office to the state education leaders — have indicated that state operating support is not expected to increase in the coming session. At the same time, the needs and demands of higher education have not subsided. It is not enough to simply agree MSU Billings will be doing more with less. It is imperative the university make proactive, strategic, informed and wise decisions to ensure scarce resources not only match mission and operations, but match the expectations of constituents who rely on a vibrant and effective urban university. Because MSU Billings is a part of a complex state system and Montana State University sub‐system of higher education institutions, some budgetary and planning decisions are made outside of local‐campus control. Embracing that reality, the university has redoubled efforts to be proactive, implementing a strategic planning process covering all divisions and implementing full budgetary review with academic, student affairs and administrative leaders. The strength of this process lies in the depth of the experience, institutional history and the expertise of that leadership. Using this process, the university is reviewing its institutional priorities through the lens of academic outcomes, student services potential, administrative efficiency and community expectations.



In August 2009, the Board of Regents informed the Office of the Commissioner of Higher Education and all units of the Montana University System that $18 million in tuition mitigation funding for FY 2010‐11 would likely not be available for FY2012‐13. In the fall and winter of 2009‐2010, it became clear that the state’s projected revenues for the next fiscal year were dwindling fast and cuts for FY 2010‐11 might be necessary. That was confirmed in January when the Board of Regents agreed to proceed with planning for 2.5 percent system‐wide reductions. As a result, the system (and all institutions including MSU Billings) is planning for those budget reductions. In a large sense, MSU Billings and the University System are ahead of the curve. Because planning began in August to address the “stimulus” shortfall in FY12‐13, different constituencies and scenarios are being addressed to make spending reductions. At the same time, university leaders at all levels remain cognizant of system‐wide initiatives requiring attention and/or input. Some of these are not initiatives of our own making, but could necessitate resources and/or commitments in the future. Those include the College!Now two‐year education initiative, the state’s new virtual community college initiative, the virtual high school academy, dual credit/dual enrollment initiatives, business process redesign, information technology/data initiatives, and continued work toward common‐course numbering and transferability. A robust and evolving planning and review process started in early 2009 continues. As a result of that process, low‐enrolled programs in the College of Technology were closed and two departments in the College of Education were merged into a single, more efficient academic unit. Decisions for future action will be collaborative in nature with input from students, faculty and the Academic Senate along with the Chancellor’s Executive Budget Committee (CEBC). Through university‐wide strategic planning and review, academic, administrative and student affairs divisions will identify hiring and budgetary priorities. Those priorities will be used to position the university to be more efficient administratively, more effective supporting students moving toward graduation and more responsive as an urban university while maintaining a strong commitment to access and excellence. This process will also enable the university to more fully respond to the Board of Regents as it reviews the role, scope and mission of each unit of the Montana University System. That process stared in mid‐2009 and will culminate with recommendations and budgetary decisions later in 2010. While the outcome is far from certain, MSU Billings has addressed the mission review process and developed a document that outlines the university’s unique position in the system. Core values and core themes are being developed for use in assessment/outcomes and also to clearly articulate the university’s mission as an urban university and shape its own future. That document can be found as Appendix V


II. PROGRESS SINCE OCTOBER 2008 Action Items:

A. MSU Billings will establish balanced budgets for the 2009‐2010 and 2010‐2011 fiscal years. The budget and the allocation of the revenues available will be based on a critical analysis and evaluation of the university’s core mission, goals, performance expectations, and future directions.

B. Following record enrollment growth in the 2009‐10 academic year, the university will continue to strategically match resources with needs.

C. The university will ensure that all constituent groups, students, faculty, staff, and community representatives are actively involved and engaged throughout this process. The Academic Senate, Staff Senate, ASMSUB and the Academic Senate Budget Committee, the Chancellor’s Cabinet and the Executive Budget Committee will be directly engaged in the planning, presentation and budgetary processes.

Strategies:

1. Continued updates on legislative and BOR actions. 2. Regular involvement of faculty and deans in CEBC meetings. 3. Faculty involvement/representation on the Academic Senate Budget Committee. 4. Academic Senate Budget Committee involvement once biennial budgets are set. 5. Continued updates and involvement with students and staff.

TIMELINE & ACCOMPLISHMENTS TO DATE (See Appendix 4, Page 31)

Conclusions and Future Plans As with any organization, change is a growth process. That has never been truer than with the last 18 months at MSU Billings. Working on the issues identified in the NWCCU report in October 2008 allowed the administration, faculty, staff and Montana University System to challenge old notions, explore new relationships and stretch abilities. It is a dynamic process that continues to evolve. Using decisions made from assessing the university’s resources, mission, goals and unique position in the state, several significant changes have been made at the university. Some changes are in reaction to different expectations of students, some to take advantage of new technology and some as a result of strategic initiatives and an emerging culture of proactive positive engagement and change. As examples, the university now views assessment and program review as part of its day‐to‐day existence and not simply a once‐in‐a‐decade experience and systems are in place for ensure evaluation of all faculty. Further embracing that culture and using those new systems will be critical for future growth and success. The university has also fully evaluated its mission and resource stewardship in light of new leadership on the state and system level. Moving forward, the university has the following plans: Recommendation 1. A Continue university‐wide assessment support and refinement of measurements While different departments and programs with regional or national accreditation had embraced a culture of assessment as a natural part of its existence, a culture assessment across the board was indeed spotty in October 2008. Assessment plans and reports have been done across the university and adjustments have been made because of those assessments. However, MSU Billings realizes that that it must develop strategies to further encourage faculty commitment to assessment as a continuing, permanent part of the business of education. The assignment of a College of Business faculty member to serve as an on‐site consultant was an important step in making assessment more even across the various academic programs. This faculty member provided further focus to discipline‐specific assessment. With uniform assessment plans across all areas of the university, they will be easier to update and keep current. Each dean is responsible for working with the faculty in each program to emphasize the university’s ongoing commitment to a culture of assessment as well as providing support, suggestions and critiques.



Assessment will be a dynamic and fluid process and everyone at the university will need to be agile enough to adjust to new situations and insights. Eternal vigilance on the part of the college deans, department chairs and all constituencies can help all processes improve and be meaningful. None of this can happen in silos or in a vacuum. An assessment point person may need to be identified to maintain the current momentum into the future. This individual can serve as an assessment consultant, as well as provide guidance to faculty on continued program assessment in addition to overseeing the university‐wide assessment activities. Strategic, data‐driven resource management and decisions will be important considerations in moving this initiative forward. Recommendation 1.B Assign clear responsibility for ongoing assessment, and make assessment activities a part of personnel reviews In each college/unit, responsibility for monitoring and improving assessment has been assigned to specific people — often deans and department chairs or program directors — but the university realizes that a true culture of assessment can occur only when faculty buy into assessment as an integral part of the educational process. Consequently, strategies are being reviewed to involve more faculty in the process. In English, for example, which has a heavy General Education assessment responsibility, reassigned time is being discussed as a way for an individual faculty member to take responsibility for General Education assessment in Composition. Another suggestion is to assign one faculty member from each college as an on‐site consultant for those disciplinary areas and compensate them with reassigned time. Most importantly for the complex and multi‐departmental, even multi‐college General Education courses, faculty who teach those courses and the Academic Senate General Education Committee are taking oversight ownership of the general education assessment process. That process will need to continue for a full culture of assessment to have meaningful effect on program growth and change. Recommendation 2 Assign clear responsibility for ongoing evaluation of all part‐time as well as full‐ time faculty All full time faculty are evaluated under the terms of the two labor agreements (CBA and VTEM). Neither contract covers part‐time instruction. Going forward, each department in each college has a person responsible for evaluating all classroom teachers. It is usually a department chair or a program director. In the case of departments with a large number of


part‐time instructors such as Communication Studies, a faculty member has reassigned time to complete the peer reviews, the review of the syllabus, the self evaluation statement, and the student assessment of faculty teaching (SAFT) numeric scores and comments which is all part of the Stage 2 process. In addition, the Collective Bargaining Agreement (CBA) has been revised to include under department chair duties responsibility for assessment efforts and evaluation of all classroom teaching. All classroom teachers – full and part‐time – sign a contract agreeing to a commitment of continuous quality improvement. Recommendation 3 Clarification of system relationships and governance Each Montana University System (MUS) campus that has had NWCCU reviews in the past two years have had a recommendation similar to the one for MSU Billings. Clarification of the governing process for the Montana University System is actively being pursed on different levels and includes the Montana University System as well as specific systems related to Montana State University (MSU) or the University of Montana (UM). MSU’s new president, Dr. Waded Cruzado, spent the spring of 2010 becoming more familiar with each unit of the MSU subsystem, explaining her vision of the power of a singular university with distinct characteristics and resources in different parts of the state. As a land‐grant university, MSU has the opportunity to reach every citizen in every corner of Montana and MSU Billings will be a part of that larger picture. President Cruzado’s vision of Montana State University as “one university” with unique and separate parts is being discussed and implemented through a new four‐campus leadership council that President Cruzado formed in January 2010. Taking place on a parallel track, a working group of the Board of Regents is evaluating the various role and scope of the different units of the Montana University System. A final report and recommendations from that working group is expected to be presented in mid‐2010, which should further clarify the mission and operation of MSU Billings. In the meantime, MSU Billings has undergone a mission review process that helps clarify the university’s position as an urban university. Recommendation 4 Evaluate and better match resources with mission and operations As with the majority of institutions of higher education across the country in 2010, MSU Billings has to deal with growth and change amid financial constraints. The fact that MSU Billings is in the midst of enrollment growth on many fronts does little to change that reality. The Montana Legislature funds the Montana University System according to a resident student full‐time equivalent model. Because of the nature of the MSU Billings student body and a


growing number of part‐time students, this formula has never been in the university’s best interest. The Board of Regents is exploring alternative funding models but the reality is that the state general fund cannot be expected to fully fund the cost of higher education. Even if Montana, as a state, had surplus funds, state politics are such that higher education is not likely to be high priority in the eyes of the legislators. The university keeps a keen eye on revenue streams, internal and external, as it moves forward. Indeed, for MSU Billings to fully flourish, it will have to assume responsibility for its own fiscal health and stability. Current economic reality dictates that MSU Billings be ever vigilant with resource management at all levels. Budget reductions have already taken place and state support for higher education is unlikely to increase in the 2011 legislative session. Planning is under way for further reductions if state tax revenues continue to decline. MSU Billings has worked strategically and with widespread involvement of administrators, directors, deans, department chairs, faculty, staff and students to maintain a balanced budget and to address a shrinking pool of resources. Resources have been allocated so that students and their ability to complete their degrees in a timely fashion have not been impacted. A robust and meaningful program review process will provide the university with continued insight into which programs are core or essential and which may phased out or put into hibernation. The auxiliaries/athletics planning process, long‐range building planning process and enrollment management planning decisions will play major roles in resource management and strategic decisions


APPENDIX I

GENERAL EDUCATION


GENERAL EDUCATION LEARNING OUTCOMES 2009‐2010 REVISION

CATEGORY I: Global Academic Skills New Outcomes A. Mathematics M 105 Contemporary Mathematics: 3 M 114 Extended Technical Mathematics: 3 M 121 College Algebra: 3 M 131 Math for Elementary Teachers II: 3 M 143 Finite Mathematics: 4 M 151 Precalculus: 5 M 171 Calculus I: 4 STAT 141 Intro to Statistical Concepts: 3 STAT 216 Introduction to Statistics: 4

1. Demonstrate ability to solve problems quantitatively. 2. Solve problems with various mathematical methods of the discipline. 3. Communicate using mathematical terminology.

B. English WRIT 101 College Writing I: 3 WRIT 121 Intro to Technical Writing: 3 WRIT 122 Intro to Business Writing: 3 WRIT 201 College Writing II: 3 WRIT 220 Business & Professional Writing: 3

1. Demonstrate knowledge of and competence in the use of conventional written forms: mechanics, spelling, punctuation, syntax, grammar, etc.

2. Demonstrate ability to apply knowledge of writing strategies.

3. Demonstrate the ability to undertake and accomplish original work in written form.

C. Information Literacy COMT 130 Intro to Public Speaking: 3 LS 125 Research in the Info Age: 3 MIS 150 Info Access & Organization: 3

1.Engage in hands-on research as a process of gathering, assessing, interpreting, and using data from multiple sources to express ideas. 2. Use information effectively to accomplish a specific purpose in oral or written form. 3. Understand the economic, legal, and social issues surrounding the use of information, and access and use information ethically and legally.

CATEGORY II: Natural Sciences A. Life Science BIOL 101 Survey of Biology: 3 BIOL 115 Survey of Biology Lab: 1 BIOL 178 Principles of Biology: 3 BIOL 188 Principles of Biology Lab: 1 SCIN 101 Integrated Sciences I: 3 SCIN 102 Integrated Sciences Lab: 0.5 SCIN 103 Integrated Sciences II: 3 SCIN 104 Integrated Sciences Lab II: 0.5

1. Demonstrate an understanding of knowledge related to the life sciences. 2. Demonstrate the ability to synthesize knowledge from different subject areas concerning the life sciences. 3. Demonstrate the ability to use logical or quantitative approaches to solve problems related to the life sciences.

B. Physical Science CHMY 121 Intro to General Chemistry: 3 CHMY 122 Intro to General Chemistry Lab: 1 CHMY 141 College Chemistry I: 3 CHMY 142 College Chemistry Laboratory I: 1

1. Demonstrate an understanding of knowledge related to the physical sciences. 2. Demonstrate the ability to synthesize knowledge from different subject areas concerning the physical


GEO 101 Introduction to Physical Geology: 3 GEO 102 Intro to Physical Geology Lab: 1 GPHY 111 Introduction to Physical Geography: 3 GPHY 112 Intro to Physical Geography Lab: 1 PHYS 101 Earth Air Fire Water: 3 PHYS 102 Earth Air Fire Water Lab: 1 PHYS 110 College Physics I: 3 PHYS 111 College Physics I Lab: 1 PHYS 201 Introduction to Astronomy: 3 PHYS 203 Introduction to Astronomy Lab: 1 PSSC 101 The Physical World Around Us: 3 PSSC 102 The Physical Word Around Us Lab: 1

sciences. 3. Demonstrate the ability to use logical or quantitative approaches to solve problems related to the physical sciences.

CATEGORY III: Social Sciences BUS 101 Introduction to Business: 3 COMT 109 Human Relations: 3 COMT 110 Interpersonal Communication: 3 ECNS 201 Principles of Microeconomics: 3 ECNS 202 Principles of Macroeconomics: 3 EDF 100 Education and Democracy: 3 GPHY 141 Geography of World Regions: 3 HHP 101 Health Sciences: 3 PSCI 210 Intro to American Government: 3 PSCI 220 Intro to Comparative Gov’t: 3 PSYX 100 Introduction to Psychology: 3 PSYX 231 Human Relations: 3 SOCI 101 Introduction to Sociology: 3 SOCI 201 Social Problems: 3 SOCL 212 Physical Anthropology and Arch’y: 3

1. Demonstrate an ability to identify and solve problems using social science methods. 2. Demonstrate the ability to communicate and analyze data effectively within the disciplines of the social sciences. 3. Demonstrate a basic understanding of the important social themes in the discipline.

CATEGORY IV: History & Cultural Diversity A. History HSTA 101 American History I: 3 HSTA 102 American History II: 3 HSTR 101 Western Civilization I: 3 HSTR 102 Western Civilization II: 3 HSTR 103 Honors Western Civilization I: 3 HSTR 104 Honors Western Civilization II: 3 PSCI 230 Introduction to Int. Relations: 3

1. Demonstrate an ability to use analysis of a variety of types of sources to construct historical knowledge. 2. Demonstrate an ability to organize a variety of historical sources and express them effectively in written form. 3. Demonstrate basic understanding of the historical context of events.

B. Cultural Diversity A&SC 250/SOCI 275 Women, Cult and Soc: 3 ART 131 Global Visual Culture: 3 COMT 160 Intro to Intercultural Communications: 3 GPHY 121 Human Geography: 3 HHP 270 Global Health Issues: 3 LIT 230 World Literature Survey: 3 MUSC 150 Music’s of the World: 3 NAMS 181 Intro to Native American Studies: 3 NAMS 211 Soc Issues of Native Americans: 3 PHIL 105 The Religious Quest: 3 REHA 201 Introduction to Diversity: 3 SOCL 211 Cultural Anthropology: 3 SPNS 150 The Hispanic Tradition: 3

1. Demonstrate an ability to identify and solve problems relating to cultural diversity within the discipline. 2. Demonstrate the ability to communicate and analyze effectively concerning cultural diversity within the discipline 3. Demonstrate a basic understanding of the importance of awareness of cultural diversity within the various disciplines.

CATEGORY V: Arts and Humanities

A. Arts ART 110 Art St Essentials for the Non-Art Maj: 3 ART 142 Introduction to Pottery: 3 ART 161 Introduction to Drawing: 3 COMT 150 Intro to Theatre and Performance: 3 COMT 155 Global Cinema: 3 COMT 250 Introduction to Acting: 3 DSGN 248 Computer Present and Animation: 3 ENGL 204 Fund of Creative Writing: 3 LIT 270 Film & Literature: 3 MUSC 100 Music Appreciation: 3

1. Demonstrate an understanding of insight in the fine arts i.e., the role that insight plays in the face of folly: stereotypes, cliché etc. 2. Demonstrate an awareness of the power of imagination and expression. 3. Demonstrate an appreciation for the fine arts and what that appreciation can mean as a quality of life issue.

B. Humanities ART 132 Art History Survey: 3 HON 181 The Ancient and Medieval Worlds: 3 HON 182 Ren and Modern Worlds: 3 HON 281 Am Intel Heritage (1620-1877): 3 HON 282 Am Intel Heritage (1877-present) : 3 LIT 110 Introduction to Literature: 3 LIT/PHIL 240 The Bible as Literature: 3 PHIL 115 Ethics: 3 PHIL 117 Philosophies of Life: 3

1. Develop an awareness of the role that the humanities play in culture i.e., politics, philosophy, economics, science, math etc. 2. Create an awareness of how to cross-relate/reference humanities based information. 3. Demonstrate an awareness of issues surrounding life, death and morality.



APPENDIX II

STAGES OF ASSESSMENT


The following tables display the list of programs in each stage of the assessment process at four different points in time. The four points in time are June 1, August 28 and October 28, 2009 and March 18, 2010 (chosen arbitrarily). Each point in time is displayed in a separate table. The purpose of these tables is to demonstrate the progress made in developing assessment plans, implementing assessment procedures and evaluating the results of assessment at MSU Billings. The stages are defined below: Stage 0 – Beginning stage. Programs in this stage have not started developing a plan for assessing student learning. Stage 1 – Assessment planning stage. Programs in this stage are developing an assessment plan but have not implemented the plan. Stage 2 – Assessment implementation stage. Programs in this stage have completed Stage 1 and have begun or completed the collection of assessment information but have not begun Stage 3. Stage 3 – Assessment analysis stage. Degree programs in this stage have completed Stage 2 and are in the process or have completed the evaluation of assessment information and made curricular or pedagogical changes as necessary. Many of these programs are accredited by various accrediting agencies and have implemented assessment according to guidelines provided by those agencies. The number of programs in Stage 3 has more than doubled from June, 2009 to March 2010 and all degree programs have completed at least the planning stage (Stage 1) in the assessment process. As of March 2010, 31 programs now in Stage 3 have prepared and submitted assessment reports.






APPENDIX III

EXAMPLES OF CHANGES TO PROGRAMS

OR ASSESSMENT PROCESSES


Examples of Changes to Programs or Assessment Processes Due to Assessment Data (Stage 3 programs) Sources of information: Assessment reports posted on SharePoint. College of Education:

• Summer 2007: Review of initial candidate placement data revealed a need for increased experience with diverse students. A grant was written by Dr. Sharon Hobbs, in partnership with the Friendship House, to support those efforts. Through the grant and collaborative work with the Montana Center on Disabilities, placements were increased, and more focused in nature, with candidate training and specific duties.

• Fall 2007: The COE Graduate Committee discussed revision of the Advanced Conceptual Framework outcomes, formulating six indicators for candidates in advanced programs and candidates seeking initial licensure. The faculty adopted the Master’s Initial and Advanced Conceptual Frameworks, ending the requirements for portfolios. Because of these changes, all of the clinical experience evaluation forms were revised to reflect the new frameworks. The current forms provide better guidance for evaluators and generate useful data.

• Spring 2008‐Fall 2008: Data evaluating teacher candidates was being generated, but was not as useful as needed. The Dispositions Subcommittee began to clarify the dispositions and improve the rubric. Several forms were piloted in various classes, leading to the current dispositions forms, both initial and advanced, and valuable data are now generated.

• Fall 2008: The faculty engages in a process to assess candidates’ ability to demonstrate their effect on student learning. Data from the early version of Evidence of Professional Growth (EPG) form were not useful in evaluating candidates’ skills or their effect on student learning, was cumbersome to use, and both candidates and faculty found the rubric and instructions vague, making the entire instrument invalid. The EPG has been revised and is currently (Fall 2009) being tested for validity and reliability.

• Spring 2009: Based upon a presentation by the Clinical Practice & Skills Committee, the process for rating student teachers was refined to include inter‐rater reliability between university and faculty supervisors.

• Spring 2009: Review of Indian Ed for All (IEFA) data (employer and program completer surveys) revealed that candidates are not familiar with the OPI directive to implement IEFA. In addition, data from the Big Sky Projects showed Native American program completers, who were new teachers in a district, were expected to know and in fact lead the school faculty to initiate IEFA, just because they were themselves, Native American. A faculty member designed and taught an online seminar teaching graduate students in elementary education to effectively implement IEFA. A review from a Native American Studies faculty member, involved in IEFA development, gave the seminar positive comments. Data resulted in all COE faculty participating in OPI IEFA Level I training on‐line during AY 2009‐2010. Lesson plans now include an IEfA section.

• Summer 2009: A structured internship for Graduate students was added to the EDCI 500 beginning curriculum. All paperwork was completed to make the necessary changes to


the program, including changing EDCI 500 to a 2 credit course and adding a 2 credit internship program EDCI 519.

• Fall 2009: Students expressed a need for more field experiences in special education. SPED 420 was determined by faculty to be the course most suitable to include this field experience. Program changes were submitted Spring 2008 and approved to start Fall 2009.

• Fall 2009: The focus in EC 283: Curriculum and Adaptation was changed to include an introduction to informal assessment and its relationship to curriculum delivery. A new course in assessment was piloted in Spring 2010.

• Fall 2009: Content of RD 427 was realigned with program content, goals, objectives and was reexamined against RD 414/415, RD 310.

• Fall 2009: The course content and placement of RD 412/512 was reconsidered in relation to RD 414/514.

• Fall 2009: A discussion regarding candidate assessment led to the initiation of a survey regarding how assessment is taught among the faculty. The results of the survey were enlightening and led to the current review of methods and curriculum class content by the department.

• Fall 2009: The Assessment System was reviewed at the September 2009 faculty meeting. Minor adjustments were agreed upon.

Department of Business Academic Programs (College of Business):

• In 2008, the percentage of students that met or exceeded expectations for written communication skills indicated improvement in student learning was warranted for all traits. To improve written communication skills, a Writing Task Force of three faculty members developed and presented strategies for providing useful feedback to students. The faculty also met with faculty from the English department to discuss strategies for improving instruction in the Business Communications course. The English faculty shared handouts with techniques of good writing and documentation.

• Scores from the Business Critical Thinking Test showed the majority of onsite students scored at or above the adequate level for all critical thinking skills requirements, but with indications that improvement was needed in two areas. For online students the data suggests improvement was needed in three areas. Course‐embedded assessments yielded similar results. The faculty has met to share ideas for enhancing critical thinking to help other faculty develop strategies to enhance critical thinking in their courses.

• A high percentage of onsite and online students met or exceeded expectations for ethical decision‐making, except for one trait which students were not asked to perform. The assignment used for course‐embedded assessment has been modified and is now being assigned in the capstone courses.

• Curriculum mapping of ethics in the business core courses showed little coverage of ethics topics. Additional coverage of ethics has been implemented in the core business classes. The faculty developed a virtual syllabus of ethics topics and identified courses where the topics should be covered. Part‐time faculty teaching core courses are notified to include these topics also.


• Assessment data indicate a number of students did not demonstrate the ability to meet the learning objectives for global awareness. Curriculum mapping revealed little coverage of international topics across the core curriculum. Additional coverage of international business has been implemented in the core business classes. The faculty developed a virtual syllabus of international topics and identified courses where the topics should be covered. Part‐time faculty teaching core courses are notified to include these topics also.

• The scores from the CompXM test suggest our students performed lower in the accounting questions than in other areas, such as production management, strategic management, and finance, compared to other students who took the test. Online homework has been required in accounting courses to increase practice and retention of quantitative skills.

• Assessment tasks for use of technology in decision‐making did not address a business context and not much analysis is being asked of students in the ethics assignment. The technology assessment team created an assignment for the MIS 330 class which addresses both traits in the rubric. The instructor agreed to assign the case this fall as an extra credit assignment.

Department of Health and Human Performance (College of Allied Health Professions):

• The Masters in Athletic Training program: o This fall 2009 the ATEP curriculum was restructured to include a Pharmacology

and Nutrition in HHP and Organization and Administration of AT as well as both a Case Study option for research or an Internship route as well as the elimination of the graduate statistics course. All curriculum changes were approved by the graduate studies committee and went into effect Spring 2010.

Department of Rehabilitation and Human Services (College of Allied Health Professions):

• Beginning fall 2007 semester, HADM 685 Research and Program Evaluation has been adopted as part of the program plan of study core requirements. This course replaces EDF 501 as HADM 685 was determined to be more relevant to student, graduate and employer needs as identified in the CORE accreditation surveys.

• Starting fall 2010 semester the topic of assistive technology will be given more emphasis in the online course offerings of REHA 525 Vocational Placement and Support and REHA 530 Case Management Principles and Plan Development.

• With incorporation of the Counseling Clinic into the practicum and internship experiences, starting fall 2007 semester, students are now provided additional opportunity to develop and apply their counseling skills in a more intensively supervised setting.

The Department of Communications and Theater (College of Arts and Sciences):

• The faculty supervising research projects observed the need for better research skills in the Masters students. To improve students’ research skills in the Masters in Public Relations program, the program now permits more options for taking research courses


from other departments. Students now have an opportunity to take a required research course earlier in the program.

• The Masters students who do not complete either a thesis or a project are now required to take comprehensive exams for assessment purposes. Previously, this track did not have a complete assessment plan.

• The faculty noted a need for more basic skills in the bachelors program. The department has created a new introductory course now required for the bachelor’s programs to enhance basic communications skills and provide baseline data for assessment.

• They also decided on an assessment plan that evaluates students at the beginning of the program. A pre‐test/post‐test assessment process is being initiated in the bachelor’s programs. The pre‐test is administered in the introductory course and the post‐test is administered in the COMT 470 course through a new assignment in the course.

Department of English, Philosophy and Modern Languages (College of Arts and Sciences):

• In the Spanish program: o The only discernable pattern is that students who return after extended time

overseas have a greater command of the language in terms of accent, vocabulary and grammar. The faculty have seen increased enrollment in the overseas programs both in Spain and in Latin America over the last five years. To continue to provide students with greater skills students will continue to be encouraged to study overseas in Spanish speaking countries.

Department of Psychology (College of Arts and Sciences):

• The department has used the outcomes of the capstone course to provide feedback on the lab courses. The assessment of student projects for design, data collection analysis, writing quality and overall interpretation has resulted in several revisions in both the lab content and curriculum of the lower division courses. Changes to the curriculum in research design were implemented this year to include hands‐on labs and individualized instruction. Changes in introduction to behavioral statistics will include broader topics and more assignments. Changes in Cognitive Psychology and Sensation and Perception lab will include design of several studies with analysis.

Department of Nursing, Health and Safety (College of Technology):

• In the Paramedic program: o A new clinical site was added in 2008 to ensure adequate pediatric exposure and

pediatric respiratory experience. o The FI DAP Summative Written Examination has been added to provide a valid

and reliable assessment tool. o Two courses have been added to the Paramedic program: PARA 101 Transition

to Paramedicine Online and PARA 120 Case Studies Online. o Pre‐requisite courses are offered online in the spring semester. o A new tracking document has been developed to improve assessment.


• In the Radiologic Technology program: o Based on employer and graduate surveys, the Radiologic Technology program

has increased student experiences in the operating room and implemented a new rotation plan at the pain clinic.

o A demerit system was instituted at clinical sites in 2008 to identify poor performing students. In 2009 a merit system was added to encourage excellent performance at clinical sites. Faculty are now counseling poor performing students.

• As a result of scores on the ATI tests, the Nursing pathways curriculum has added a Microbiology component to improve ATI test scores in this area.

Department of Industry and Computers (College of Technology):

• In the Computer Systems Technology program: o Based on information from employers a required course in Help Desk Support

has been added to the Computer Systems Technology program. o The Computer Systems Technology program has removed courses to allow for

internships. o The capstone course in the Computer Systems Technology program has

increased the emphasis on oral presentations based on assessment of student oral presentation skills.

• In the Drafting and Design program: o As a result of assessment in the capstone project in the Drafting and Design

program, a recommendation has emerged to change the sequencing for the 2D and 3D content in the first semester. This change will be implemented during the Spring of 2010. Another recommendation is to embed additional 3D content into architectural and civil courses.

o To improve the organization and presentation of projects, a review of the academic foundations communication content is recommended.

Department of Transportation and Business (College of Technology):

• In the Diesel Technology program: o During the assessment of last year’s data, the Diesel Technology faculty

recognized the need for some simple changes in data collection and review that when implemented will improve the efficiency of the program’s assessment procedures.

• In the Automotive Technology program: o The ASE exam preparation course has been approved and is now a requirement

for all students within the Automotive Technology major. The students’ plan of study lists the third semester as the suggested semester for completion, therefore results from Fall 2010 will include the first group of students under the required plan of study.

o The reliability of the ASE exam data will improve due to the required ASE preparation course, allowing for a more valid analysis.


APPENDIX IV

TIMELINES


RECOMMENDATION 1 GENERAL EDUCATION

TIMELINE AND ACCOMPLISHMENTS: 2009‐2010 Fall 2009

• Continue/complete revision of GE outcomes: COMPLETED

• Initiate discussions on GE course prerequisites (re: COMPASS results): COMPLETED

• Initiate effort to devise advising protocols vis‐à‐vis AF prerequisites (above): COMPLETED

• Secure reliable sample of courses for reporting into the database using “old” matrix outcomes: COMPLETED

• Initiate conversation re: course vetting/periodic review: COMPLETED

• Initiate sampling deliberations and devise system for disciplinary rotation: COMPLETED

• Carry out analysis of General Education assessment data Fall 2007‐Spring 2009: COMPLETED

Spring 2010

• Initiate new revised outcomes reporting in database: IN PROGRESS

• Discuss prerequisites (re: COMPASS) and propose schedules/advising protocols: COMPLETED

• Continue study on First‐Year Experiences and GE: IN PROGRESS

• Continue deliberation over course vetting/review protocols, with proposals by semester end: IN PROGRESS

• Carry out analysis of General Education assessment data Fall 2009 and aggregate with previous semesters: COMPLETED

• Report to NWCCU on progress toward GE assessment, with plans for continued revision: COMPLETED



RECOMMENDATION 2 EVALUATION OF ALL FACULTY

TIMELINE AND ACCOMPLISHMENTS: 2008‐2010 Summer 2009

• Have all deans and colleges assess their current evaluation efforts: COMPLETED

• Include in all revised college strategic plans the necessity of annual evaluations of all faculty and staff with teaching responsibilities: COMPLETED

• Initiate discussions on a possible template including student evaluations, self evaluations and departmental evaluations: COMPLETED

• Hold discussions in general on evaluating faculty teaching online courses: COMPLETED

• Have the College of Arts and Sciences dean share what the various departments and programs currently do for annual evaluations: COMPLETED

Fall 2009

• Continue discussions on evaluating online courses: COMPLETED • Have the College of Business share its revised Department Reappointment, Tenure

Committee policies and procedures which includes evaluation of all faculty with teaching responsibilities at a Provost Council meeting: COMPLETED

• Implement all existing and new college and university‐wide evaluation processes developed which includes moving all programs to stage one or stage two: ONGOING

Spring 2010

• Implement all new college and university‐wide evaluation processes not developed in time to use during Fall 2009. The COT will conduct a pilot project partially adopted from a model developed by Flathead Valley Community College: ONGOING

• Discuss what modifications should be made to any/all of these plans once these

evaluations have tried at least for a semester: ONGOING


• Report to NWCCU on the status and progress toward a University‐wide evaluation plan

and each spring or summer assess the plans to see what revisions have been identified.

Other accomplishments • Deans determined that no one template/format would work for all courses, programs,

departments and colleges. • IT fixed the problems with online courses under the new D2L platform which had kept

many students from filling out student evaluations at the end of each online course. IT or eLearning will send reminders to online students on how to access the evaluation near the end of the course.

• All staff members who teach as part of their responsibilities of being part of the

Academic Support Center have been evaluated. The Director evaluates all of ASC staff and a full time faculty member from the College of Arts and Sciences evaluates the Director’s teaching which is included in his/her overall evaluation. (See SharePoint for sample)

• College of Arts and Sciences presented its evaluation models to the Provost Council in the spring of 2009. The College of Business presented its evaluation model to the Provost Council on November 4, 2009. The College of Education and the College of Allied Health presented evaluation models to the Provost Council at the February, 2010 Provost Council meeting. (See SharePoint for models)

• COT faculty has developed a part‐time faculty evaluation model piloted by two faculty

during fall, 2009 and to be piloted by 15 instructors during the spring of 2010. This model was partially adopted from the Flathead Valley Community College part‐time evaluation model. Completed model will be used throughout the college by Fall 2010

• By the end of fall semester 2009, the College of Allied Health had 70% of its academic

classes taught by part‐time faculty who were evaluated. In one program it was 100%, in another it was less than half because the activities courses were not consistently evaluated. By spring, 2010, the entire college will be at Stage 2.

• College of Professional Studies and Life Long Learning evaluates courses based on the

appropriate college evaluation plan and has developed a model to evaluate its first for credit class.

• All colleges and the Library had developed an evaluation process by the end of fall 2009.


.


RECOMMENDATION 3

UNIVERSITY GOVERNANCE TIMELINE AND ACCOMPLISHMENTS: Spring 2010 • Chancellor continues to engage in conversations with the Commissioner and Deputy

Commissioner of OCHE and MSU President Waded Cruzado regarding role, scope and mission of MSU Billings in the Montana University System.

• Discussion with MSU and OCHE officials are taking into consideration the ongoing BOR dialog on mission and scope of all Montana University System (MUS) institutions including MSU Billings. That mission review will be submitted to the Board of Regents in spring 2010. (See Mission Review document in Appendix V)

• Report to NWCCU on the status and progress at the time of the focused review.

Other accomplishments

• Chancellor took part in discussions with Board of Regents working group that is reviewing role, scope and missions of all units of the Montana University System.

• Internal review of mission and vision began in fall 2009.

• MSU Billings Provost discussed the value of undergraduate research with the Provosts of both University of Montana in Missoula and Montana State University in Bozeman. Both agreed that in revising MSU Billings’ mission, it will be important to delineate the importance of research as a major educational tool for undergraduate education.

Action Items for Spring 2010:

1. Begin the discussions with the Commissioner and Deputy Commissioner of OCHE and the new President of MSU.


RECOMMENDATION 4 SUSTAINABILITY

TIMELINE AND ACCOMPLISHMENTS: Spring and Summer 2009

• Initiate discussions on the 2010 budget as information becomes available from the Governor, the State Legislature, the Office of the Commissioner of Higher Education and the Board of Regents: ONGOING

• Deans lead colleges into developing revised strategic plans based on no increase in state funding. All investments must be developed by reallocation: ONGOING

• College strategic plans presented to the Chancellor and Provost and then at the Chancellor’s Cabinet: ONGOING

• Develop a new strategic plan for the Library which has not had a viable plan for many years: COMPLETED

• Begin discussions on possible templates/models for ranking programs based on collegiate core mission and values: ONGOING

• Initiate discussion on ways for all colleges to develop hiring priorities based on collegiate core mission and values: ONGOING

• Student Affairs staff will create departmental goals and divisional strategic initiatives which will be used in the budgeting process: ONGOING

• Administrative services will create departmental goals and divisional strategic initiatives which will be used in the budgeting process: ONGOING

• Academic Affairs will develop an open and inclusive process for further development of comprehensive division plan: ONGOING

• 2010 fiscal year budget is balanced: COMPLETED


Fall 2009

• Continue discussions on the 2011 budget and 2012‐13 budgets as information becomes available from the Governor, the State Legislature, the Office of the Commissioner of Higher Education and the Board of Regents.

• All colleges share with their faculty and staff the final drafts of their strategic plans and then share them with their advisory boards. (including student and community‐based boards)

• Ask the College of Education at the December Provost Council meeting share their modification of the University of West Florida’s model on ranking college programs.

• Continue, and then bring to closure, discussions in colleges on possible templates/models for ranking programs based on collegiate core mission and values. Begin drafting college priorities.

• Continue, and then bring to a closure, discussions in colleges on ways for all colleges to develop hiring priorities based on collegiate core mission and values. Begin drafting college priorities.

• Develop for Student Affairs its hiring priorities for the next 3‐5 years as part of the budgeting process.

• Develop for Administrative Services its hiring priorities for the next 3‐5 years as part of the budgeting process.

• Begin the discussion on Academic Affairs (AA) hiring priorities. AA must wait until the collegiate plans are vetted before developing a comprehensive plan for hiring priorities for the next 3‐5 years.

Spring 2010

• Continue discussions on the 2011 budget and 2012‐13 budgets as information becomes available from the Governor, the State Legislature, the Office of the Commissioner of Higher Education and the Board of Regents.

• Develop a revised mission statement.

• Develop core values and core themes based on MSU Billings’ revised mission statement.

• Develop an Academic Affairs hiring plan for the next 3‐5 years.


• Analyze and assess collegiate ranking of academic programs.

• Report to NWCCU on the status and progress toward matching university resources and

strategic planning. Report to NWCCU on the status and progress of identifying MSU Billings’ mission, core values, and core themes.

Other accomplishments

• Student Affairs has developed departmental goals and divisional strategic initiatives which it can use in the ongoing budget processes. A top priority is bringing in a consultant to focus on enrollment management which has been funded as an investment for AY 2009‐2010.

• The Library has a strategic plan and has reviewed its features with the deans, the Cabinet, and its advisory committees.

• All colleges now have strategic plans that have been vetted to its faculty and staff during back to school, fall, 2009, and vetted to its advisory committees.

• MSU Billings 2010 budget is balanced.

• The College of Education, the College of Arts and Sciences and the College of Technology were able to prepare the required every seven year program reviews for the Board of Regents with using their collegiate strategic plans and annual CQI reports.

• E‐learning and distance education operations have been restructured to better position the university for growth in the future.

• The university has leveraged state and federal dollars for energy efficiency projects.

• The university has entered into a relationship with Noel Levitz to make use of available staffing and resources for more effective recruitment and retention.

•


APPENDIX V

MSU BILLINGS MISSION REVIEW PREPARED FOR THE MONTANA BOARD OF REGENTS

Documents

NWCCU Focused Interim Report - MSU Billings › assessment › pdf › MSUB_Focused_Int… · NWCCU Focused Interim Report ... (Stage 3), departments are asked to report the results