2016-2017 Summative Assessment Report · address enrollment, persistence and completion for the AASCC program: a. Establish a faculty member who will actively promote the program

1

2016-2017 Summative Assessment Report

INTRODUCTION

Participants Involved in Developing the Assessment Plan This document is PBC’s Summative Assessment Report concerning outcomes identified in the 2016-2017 Assessment Plan. It was developed by the Office of Institutional Effectiveness (OIE)through multiple meetings, research of available data, and evaluation of progress toward mission achievement. The 2013-2014 academic year was the baseline year for PBC’s assessment. This report reflects Year 4 annual results, major achievement/challenges and recommended changes to affect outcome achievement. The following groups/individuals participated in this assessment process:

Board of Trustees

Mike Robinson, President

John Osbourn, Chief Academic Officer

Daniel Nicholas, Dean of Students

Maha Snell, Counseling Department Chair

Dee Chadwell, General Studies Department Chair

Rick Booye, Bible and Theology Department Chair

Matt McAuliffe, Program Director Biblical Studies

Mary Neal, Administrative Assistant

Amber Osbourn, Student Development

Terry Pruett, Librarian

Sarah Lee, Counseling Department Faculty

Ana Ramos, Counseling Department Faculty

2

Athena Thompson, Counseling Department Faculty

Victor Borchard, Bible and Theology Department Faculty

B. Analysis of the effectiveness of the previous year’s (2015-16) assessment findings and actions

In terms of effectiveness, PBC accomplished a substantial body of work and changes regarding student learning and institutional effectiveness based on the findings and future actions designated in our 2015-16 assessment process. The following is a summary of actions accomplished in response to our previous year’s assessment:

Findings/Actions #1

Student Course Evaluations - Future data collection will be aggregated and assessed using student input from all classes in the fall and spring semester of the academic year.

Actions: This action was accomplished and student responses were collected for the fall, spring, and summer semesters in the 2016-17 academic year.

Findings/Actions #2

ABHE Bible Exam - We determined that we will continue to issue the exam to future program graduates and monitor for trends. Our ABS department leadership will consider the results of the Old Testament Prophets section and possible causes of the underperformance. In the 2016-17 year, we will identify a group of first-year students that we can test and then re-test at program completion to assess results.

Actions - This action was only partially accomplished. We had 3 students take the Year 1 Bible Freshman exam. All three of these students are currently attending so there is a good possibility we will be able to test them again at completion of their programs. We will continue to increase the numbers of incoming students taking the exam to get better fidelity of the student growth in biblical knowledge throughout their program work at PBC.

Findings/Actions #3

ABS Program - Follow-up on modifying parameters or specific assignment goals to more clearly elicit evidence of successful SLO’s. Follow-up with faculty members teaching specified classes whose assignments were used in our 2015-16 assessment. These follow-ups will serve to add detail and clarity regarding the presentation of curricular content that seemed to be lacking or understated in the initial artifact analysis

3

Actions: The results of the 2016-17 ABS Assessment were provided to a group of faculty members in July to increase awareness of the process and how they can enhance the process. The assessment process was explained and faculty members were able to ask questions about their role in artifact identification for the upcoming academic year.

Findings/Actions #4

AASCC Program - We will strengthen the level of biblical integration with the Christian counseling specific courses. Our goal will be to strengthen or clarify the outcomes to better support biblical integration with the courses by the beginning of the fall 2016 semester. In addition, we will go through a process of identifying the core set of scriptural principles that will distinguish our AASCC graduates in the counseling profession. The department will incorporate input from faculty in the Biblical Studies department to develop and gain approval for these key elements of truth. This process will be completed prior to the beginning of the spring 2016 semester. The program will define student learning outcomes that demonstrate program objective achievement. Our goal is to have this accomplished by the end of the 2016-17 academic year. In addition, the following the actions were identified to address enrollment, persistence and completion for the AASCC program:

a. Establish a faculty member who will actively promote the program to local community. This person will establish contacts with local agencies, employers and churches to enhance awareness of the program.

b. Enhance the value proposition of the program to students through:

1. Expanded articulation agreements with 4-year programs in counseling disciplines.

2. Identification of specific ministry opportunities available upon completion

3. Identification of specific job opportunities available upon completion

4. Creation of an advisory committee that include active employers in for profit and non-profit social services, para-church organizations, and others that can provide input to program design, connect current program participants with service opportunities, and build awareness of the program in our community.

Actions - In response to last year’s findings, we accomplished transforming the AASCC counseling objectives into student learning outcome statements. The new program outcomes are published in the 2017-18 Catalog and Handbook. In addition, we identified a core set of scriptural principles that will distinguish our AASCC graduates in the counseling profession. These were presented to the Academic and Curriculum Standards for review and input. These principles will help to unify our department concerning ethos and will further develop our abilities to integrate biblical principles across our counseling classes. The work now is to disseminate and inform all faculty of these principles in order to realize the benefits.

In response to issues with enrollment and persistence identified in last year assessment, we established a cohort group of AASCC students. These students have been able to take classes together and we coordinate the best schedule for the group to assist them in completing their sequenced counseling courses. Consequently, we were able to run our Christian Counseling I and II courses with good enrollment as all of the cohort took the classes in the Summer and Fall Semesters.

4

We still have not accomplished work toward an articulation agreement with a 4-year program. Advisory committee work and identification of volunteer and paid employment opportunities is still a work in progress.

Findings/Actions #5

Student Support Operations – We determined to accomplish the following actions in 2016-17:

a. Formalize the establishment of an Office of Student Development and assign appropriate staff to oversee the implementation of measures to improve student well-being (academically, personally, spiritually) as well as to make available community resources both within and outside of the college to ensure that students receive adequate care and counsel during their matriculation as PBC students.

b. Re-double efforts to track student withdrawal/exit reasons and to follow up with non-retained students whose reason for leaving is unknown.

c. Sustain student extracurricular involvement by electing effective officers to oversee Student Leadership and initiate the drafting of a Student Government constitution.

d. Implement admissions procedures that provide an open door to those students who are prepared to complete a full program of academic study at PBC while keeping closer track of those who are less prepared. The new provisional admission policy and Academic Worship (study skills) class will be instrumental in this.

e. Begin our recruiting operations earlier to better integrate with the local high school student recruiting season. We will begin working with recruiting events in September of 2016 as opposed to the February-March time-frame that we operated on last year.

f. Our marketing campaign will be better targeted in 2016-17. Radio and print Ads that contained information concerning our scheduled class offerings and registration dates consistently brought the best results. As a result, Ad content and placement will be more focused on our product (need/benefits) as opposed to name recognition and institutional awareness in 2016-17.

g. We will employ a more deliberate strategy for making the home school community aware of the dual-credit opportunities with PBC. Our initial tactic will be to advertise

and deliver early college credit workshops for home school parents and students.

h. Continue to grow our capabilities and agreements to offer dual-credit experiences in local high schools.

i. Provide considerable focus in marketing our Associate of Liberal Arts degree to students who are looking to complete a two-year transfer degree experience.

j. Use a more deliberate strategy of marketing our programs to local churches.

5

Actions -

a. PBC established a Student Development position in January 2017, which was an outgrowth of what was previously the Teacher’s Aide position. The job description entails

oversight of all student success and wellness support functions within the college in conjunction with and under the management of the Dean of Student Affairs. The Office of Student Development is still in a nascent phase as the position itself has seen staff turnover twice since December when it was initially conceptualized. To date, the Office has facilitated the revitalization of the Student Government. This is perhaps its most significant accomplishment this year. In addition, it has also coordinated two chapel services with a de facto campus chaplain, contributed to library access and usability through student orientation and in monitoring circulation, and assisted with the preparation and execution of this year’s new student fall orientation and spring graduation ceremony The Student Development Office has also been instrumental in helping to foster cohesion within the student population. The AASCC program has been revived through building a cohort of students to work through program requirements together and to boost enrollment numbers in individual program courses.

To maintain and foster student participation a monthly E-newsletter is sent out to each student with important information and upcoming events as well as communicating opportunities that they may want to participation in. There is also a weekly announcement sheet that is given to the faculty as a means of communicating pertinent information the student needs for that week as well as any future upcoming events. This means of communication is two-fold; to keep the faculty abreast of current events and future events as well as their ability to inform and encourage the students enhancing community and personal involvement.

Workshops have also been held educating the students in writing papers, identifying learning styles and giving helpful tools to be a successful student. Tutoring is also made available for students in subjects they are needing assistance with.

b. A goal for 2017-18 would involve expanding the duties of a full-time Student Development staff person to include and oversee campus-based student support functions such as peer tutoring, work study opportunities, faculty-led support seminars, etc. An extended goal would involve building the student development staff to include a campus chaplain, an available counselor (a non-faculty member), and a regular recruitment professional. Currently, the Student Development Staff position is not filled but is expected to be renewed before Spring 18 semester. As with last year, exit reasons have been identified for all students enrolled in a specific degree program who have not enrolled in or otherwise completed a class for two consecutive semesters. Out of 34 active program students, 6 (16.2%) withdrew mid-year (4 for personal reasons, 1 for church mission, and 1 for financial reasons) and 1 completed program requirements mid-year. These exit reasons have been recorded in Populi and will be available for reference when needed. Goal for next year: sustain the current method of tracking exit reasons for all students who are unenrolled for two consecutive semesters. Enrollment management staff will have to be especially meticulous as the accuracy of these records may affect federal financial aid. New processes must be developed in order to ensure compliance.

c. Prior to 2017 Student Government was an active and loosely formed body that planned and implemented activities with the goal of creating community. Each semester a new group of people would come together constituting a Student Government group.

6

In January of 2017 a formal student government began to take shape with an invitation for students to participate by submitting their name as candidates in an election process. The offices included: President, Vice-President/Treasurer and Secretary. Eligibility requirements consisted of committing to serve as an elected officer for one year and be in good standing academically and financially. In March, Pacific Bible College held elections with ballots distributed to the students and voting taking place. Candidates were then elected and a Student Government of elected officers was formed. A constitution was written and established by the Student Government and was approved by the Administration. A post mid-term pizza/game night activity was planned and held with a great turnout of students. Student Government meets every other week with the President providing an agenda before each meeting. Future goals of student government: create a student survey to meet the social needs of the students, increase participation at monthly meetings, schedule and plan activities each month to enhance a greater sense of community, volunteer photographer at each of the events, increase the number of chapels from quarterly to monthly, use social media as an outlet to advertise events and keep the students engaged in the activities and community of Pacific Bible College.

d. PBC introduced two new elements into its admission policy for the 2016-2017 academic year: 1) the Wonderlic Basic Skills Test and 2) the Provisional Admission policy. Regarding 1) the BST was introduced to meet the standardized test requirement for those applicants who are unable to provide a valid SAT/ACT score. It has also become a useful tool in evaluating academic preparedness of dual credit high schoolers. The test’s diagnostic tool offers a sketch of each student’s proficiencies in verbal and math skills, while also placing them on a 500-point scale which estimates their proficiency according to average high school grade level (highest being a high school graduate). How PBC uses these test scores is still quite subjective and depends on the student, but in a small number of cases it has allowed us to identify students who will likely need additional academic support upon admission. This is currently the primary purpose, and we do not in principle maintain a minimum required score.

Regarding 2) the Provision Admission policy, this was instituted to allow students who were deemed unprepared for college-level academic work upon submitting their application. It also offers a way in for students whose application files may be incomplete or missing requirement documentation for reasons that cannot be helped by the student. The philosophy behind the provisional policy is that it gives students who demonstrate potential to succeed an opportunity to do so, while also helping them to measure their own abilities and decide on their own how or whether they will thrive in the college environment. Thus far the policy has been used twice to great effect. It will remain in place for 2017-2018.

A goal for the upcoming year should be to sustain the current policy with particular care to enroll students in programs who are likely to be retained and finish the program. A variety of strategies, in light of new compliance measures associated with federal financial aid, should be considered to determine how best to manage each specific program according to its unique enrollment goals and needs. Clearly defining PBC’s philosophy of student success and how we measure this institutionally and within each program, in collaboration with the newly established Student Development and Financial Aid Offices, will help refine the admissions process not only to evaluate student preparedness but also to diagnose student weaknesses earlier and promote adequate support measures with the goal of retaining students throughout their program. PBC should also expand staff involvement in admissions decisions and train multiple staff persons to perform basic admissions transactions and complete new student in-processing.

e. Recruitment operations began this year in October and the high school campaign was completed by early December. We were able to establish personal contact with guidance and career counselors at North Medford, South Medford, Central Medford, Phoenix, Cascade and New Hope Christian high schools. These visits did not generate significant interest among students (New Hope was the only high school that gave us an actual audience and classroom space for an entire period). But we were able to deposit copies of our academic catalog and program brochures and establish points of contact at each school.

7

Other recruiting events that took place this year were primarily between January and April 2017: Careers in Gear (March 16), the PBC-sponsored Biblical Higher Education Summit (March 17), and the OCEANetwork homeschool conference (April 8th).

It is still not clear what the yield will be from these recruitment operations in terms of increased enrollment. It may be that we do not see an increase for at least two years since many of the students at these events are high school juniors or younger. But it has been worth the time spent to lay a groundwork for future recruitment events and strategies based on real experience while also building important bridges into our community.

A goal for next year’s recruitment activities would be to update and significantly improve the aesthetic and marketing appeal of current recruitment materials. Developing a specific marketing strategy and philosophy may help PBC get more out of the current recruitment opportunities that are available to us.

f. This year saw a significant breakthrough for PBC into the Jackson county home school network. 10 new dual credit students were added in the Fall, 6 of whom persisted through the Spring semester. Regular dual credit workshops were offered in November, February, and April (as of this report, June’s has not yet been scheduled). Attendance at these workshops ranged from 0 – 4 interested families. The goal of the workshops is to provide a free service to this particular market so as to inform regarding college preparation and admissions, and to promote the specific benefits of PBC in achieving high school and early college requirements.

PBC will continue to offer regular dual credit workshops in 2017-2018. The challenge for this will be in finding new connections to make within the homeschooling network in order to prevent a growth plateau in the dual credit program. Our recently establish connection with the OCEANetwork, as well as Josephine county home school networks, should be leverage as much as possible in order to continue PBC’s unique offerings in this regard. In addition, PBC must continue to develop and clearly communicate in all recruitment and marketing materials the concrete value proposition for home school or high school students in attending PBC for general studies / ALA courses as opposed to non-Christian or other private college options. PBC will also have to continue to develop its internal financial aid offerings for dual credit high schoolers as internal scholarship funds become more available due to federal financial aid being offered to regular undergraduates.

g. There has been some limited progress in building dual credit experiences with local high schools. PBC to date has not yet looked into working with any public high schools, and one opportunity for offering a psychology class at Cascade Christian this past year fell through. However, as with 2015-2016, we were able to offer a class (SS204) at New Hope Christian School in Grants Pass. This class nearly quadrupled our high school enrollments from January of 2015 (from 3 students that year to 11 students this year). A goal for next year will be to offer a class at both Cascade and New Hope Christian, and to begin to investigate with local public high schools what would be involved in establishing standing agreements to facilitate college-to-high school credit transfers for dual credit students.

h. No measurable progress here apart from increased focus on high/home school students as discussed above. The ALA program currently has 3 non-high school, traditional undergraduates enrolled, making it our smallest program in terms of enrollment. A specific recruitment and marketing strategy for the ALA will be due for the 2017-2018 academic year as federal financial aid will hopefully allow PBC to become a viable option for an increasing number of adult learners as well as recent high school graduates.

i. There is still plenty of room to grow here. PBC was able to offer one remote class section at Ashland Christian Fellowship this past Spring (Th210) which included 11 enrollees (4 credit, 7 audit). The class was offered at a reduced group tuition rate. PBC has also begun developing online versions of our courses required for the Certificate of Christian Ministries. This will make our programs and course offerings more accessible to church staff and others who wish to utilize PBC for self-enrichment or professional development. For

8

2017-2018, PBC should implement a specific marketing strategy for reaching out to churches and offering professional development opportunities or group rate packages through traditional classes on campus, remote course offerings at specific churches, and online courses as they become available.

Findings/Actions #6

President’s Office and Board Governance - We determined to accomplish the following actions in 2016-17:

a. Commit to organizing a full-day board development event in 2016-17 to address the areas of weakness identified. Our plan is to hold this event in June 2017 as the work and input from the Board will be supportive of our institutional assessment which is going through final refinements in the June-July timeframe.

b. Increase communication from the president to the trustees regarding institutional activities and the “pulse” of the PBC community. The goal here is to plan to develop and conduct a standard communication with the board via e-mail on a bi-monthly basis. These communications will be separate from standard notifications concerning quarterly board meetings. A standard template for communication will be one that covers, presidential activities, donor highlights/updates, college activities updates, and other pertinent information relative to the Board’s situational awareness and functioning.

c. Continue to develop trustee recruitment activities by the president and expand efforts to include current trustees. Measurements of success are:

1. Minimum of 6 recruiting visits by the president.

2. Two new recruits identified

3. Board develop a strategy based on gap/needs of current team to strengthen the missional work of the board.

4. This strategy then gets exercised with help of president in recruitment.

d. Have the Board of Trustees take greater ownership of quarterly board meeting agendas and board level projects. The Board will develop and implement a plan for retiring the mortgage on our current facility/property. The 2016-2021 strategic plan sets in place goals for retiring the current mortgage and opening a new campus and the Board will take ownership of creating a plan to accomplish these goals.

e. The Board of Trustees will take on the responsibility of sponsoring and coordinating at least one fundraiser with the goal of raising a minimum of $10,000.

Actions - The Board coordinated and conducted another off-site retreat in the July 2017 timeframe. The purpose of the retreat was to accomplish mandatory annual board task and to enhance areas of weakness identified in the Board of Trustees self-evaluations from last year. Improvements were made in terms of recruiting as the Nominating and Membership Committee was re-established as part of the November 2016 Board Meeting. While the number of interviews of new board candidates was not achieved, one new board member was recruited and established on the Board in 2016. Recruiting efforts are ongoing for new board members and there is greater participation by board members in the process than in previous years. In addition, a special committee was put together to discuss strategies for retiring the mortgage which resulted in the campus building being sold in March of 2017.

9

The Board also accomplished a substantial review of governing documents in response to issues identified as part of the compliance review for the 2016 self-study process. The board handbook, by-laws and constitution were reviewed and revised for congruence and clarity of content.

Findings/Actions #7

Business Office and Development Office - We determined to accomplish the following actions in 2016-17:

a. Increase donor base 10%.

b. Trustees to begin an annual fundraising event.

c. Establish and implement a mortgage retirement plan.

d. Continually monitor these newly established assessment indicators and learn how to use them most effectively and/or change them to other indicators that are more useful.

Actions – We did not increase our donor base by 10% during the 2016-17 academic year but significant progress was made with respect to this aspect of last year’s assessment findings as the college hired its first Chief Development Officer (CDO). The college hired a full-time CDO in June 2017. The CDO is responsible for cultivation of donors and additional revenue streams. The CDO initiated and strategy for fundraising operations and provided an overview to the Board of Trustees at the July 7th board retreat. This key personnel hire will substantially increase our abilities to accomplish the actions designated in last year’s assessment.

The Trustees did sponsor a fundraising event which was held at the Oregon Mountain Coffee company on November 20th, 2016. A special committee was also put together to discuss strategies for retiring the mortgage which resulted in the building being sold in March of 2017.

10

C. 2016-17 Summative Assessment

The following is an overview of our assessment process: PBC determines mission effectiveness by assessing Student Learning and Institutional Effectiveness. We consider the four types of evidence to answer the question,

“How well are our students becoming biblically competent, academic proficient, spiritually cultivated and culturally engaged as servant leaders?” To answer this question, we evaluate one of these outcomes annually. This methodology allows for a complete evaluation of mission outcomes by year 4 and in year 5 to conduct a comprehensive review of the information gained from the first four years to close out the cycle. (The year 5 comprehensive assessment also includes considering any other evidence as designated in the assessment guidance for that year)

11

Assessment Areas/Evidence

PBC’s institution assessment involves evaluation across two areas of emphasis using 4 distinct types of evidence as depicted in the following diagram:

\

12

Areas of Emphasis

Student Learning - The assessment of Student Learning should provide evidence that our students are accomplishing the student related outcomes in our mission statement as well as the designated outcomes associated with their program of study. Evaluation of Student Learning is accomplished by considering the 4 types of evidence as indicators that our students become biblically competent, academically proficient, spiritually cultivated and culturally engaged servant leaders as result of their enrollment in, and subsequent completion of, an academic program at PBC.

13

Institutional Effectiveness - The assessment of Institutional Effectiveness should provide evidence that our student support services and administrative operations are operating efficiently and effectively to accomplish our mission. Evaluation of Institutional Effectiveness is accomplished by considering the 4 types of evidence across two areas of emphasis: 1) Student Support Operations and 2) Administrative Operations. Student support operations are defined as those operations that promote the academic success and holistic experience of students. Student support services consist of advising and advocacy for students and provide resources, referrals, and information across the PBC community. Administrative operations are operations outside of student support that include the President’s office, business office, academic administration, library, and fundraising development/advancement

D. 2016-2017 Student learning Assessment:

The PBC mission outcome that was examined was academic proficiency. We examined student learning by assessing the mission outcome of academic proficiency and by conducting academic program assessment to determine the success of program specific learning. PBC determines that a student is academically proficient if they are:

Indicator 1: Able to read scholarly material with comprehension. Indicator 2: Able to think logically and critically with integration. (“Kingdom Think”) Indicator 3: Able to communicate (writing, speaking) coherently and effectively. Indicator 4: Able to do scholarly research appropriate to undergraduate level.

We evaluated the Student Learning area of emphasis by considering each of the academic proficiency outcomes in light of the four categories of evidence discussed above. We also conducted academic program assessment that evaluated the level of academic proficiency achieved by program students.

Institutional Effectiveness

Student Support Operations

Administrative Operations

14

E. 2016-2017 Institutional Effectiveness Assessment:

We evaluated PBC’s Institutional Effectiveness by considering the following areas using the four categories of evidence:

1. Student Support Operations

Indicator 1: Admission’s Office

Indicator 2: Learning Resources

Indicator 3: Student Personal

2. Administrative Operations

Indicator 1: President’s Office

Indicator 2: Business Office

Indicator 3: Development Office

These two areas of emphasis (Student Learning and Institutional Effectiveness) establish the framework for our assessment. Each area of emphasis has one or

more stated goals supported by relevant, verifiable indicators of achievement. Mission fulfillment is assessed at three levels (area of emphasis, goals, and indicators). Scores are determined by considering the indicators in light of the evidence of level of achievement. A scoring rubric was developed for each indicator and enabled us to determine whether we had “not met”, “partially met”, “met”, or “exceeded expectations” based on the level of performance with regard to the indicator. Once we had the indicator scores, we conducted a qualitative assessment of the validity of that score by assigning an “effectiveness score”. The “effectiveness scores” are determined using an institutional rubric that enables us to have standardized scores across all indicators that can then be averaged to determine the overall performance on a goal. PBC has implemented an effectiveness scoring system based on the following numeric scale and word pictures:

• A score from 0-2 indicates that the college has “not met” expectations. • A score from 3-4 indicates that the college “partially met” expectations. This level of performance is considered acceptable. • A score from 5-8 is representative of very good progress and is deemed to have “met” expectations. • A score of 9-10 indicates that there are conclusive results with evidence that “exceed” expectations.

Once we have average scores for all indicators and goals, we can then determine the average score for each area of emphasis by simply taking the average of the scores

of their respective goals. The OIE met during the June-July 2016 time-frame and determined the average score for indicators, goals, and areas of emphasis as the following:

15

Institutional Assessment Summary Scores

Area of Emphasis: STUDENT LEARNING (Score: 6.69)

AREA OF ASSESSMENT SCORE

GOAL A: Students are Academically Proficient 7.06

Indicator 1: Able to read scholarly material with comprehension 6.88

Indicator 2: Able to think logically and critically with integration. (“Kingdom Think”)

6.90

Indicator 3: Able to communicate (writing, speaking) coherently and effectively.

7.17

Indicator 4: Able to do scholarly research appropriate to undergraduate level.

7.29

GOAL B: Program Specific Learning 6.32

Indicator 1: Associate of Biblical Studies Program Outcomes 6.57

Indicator 2: AAS Christian Counseling Program Outcomes 6.00

Indicator 3: Associates of Liberal Arts 5.71

Indicator 4: ABHE Bible Exam 7.00

16

Area of Emphasis: INSTITUTIONAL EFFECTIVENESS (Score: 6.79)

AREA OF ASSESSMENT SCORE

GOAL C: Student Support Operations 7.03

Indicator 1: Admission’s Office 7.75

Indicator 2: Learning Resources 6.67

Indicator 3: Student Personal 6.67

GOAL D: Administrative Operations 6.55

Indicator 1: President’s Office 7.33

Indicator 2: Business Office 6.00

Indicator 3: Development Office 6.33

17

Pacific Bible College Summative Assessment Report

II. ASSESSMENT OF STUDENT LEARNING

Goal A: Students are Academically Proficient Rationale: This is core component of PBC’s philosophy of student development and values. Defining, developing a self-awareness, and progressively aligning the student’s worldview with the Kingdom and the King are foundational to realization of our mission.

Indicator 1: Able to read scholarly material with comprehension.

Direct/Indirect Measure Success Indicators Assessment Results Effectiveness Score Comments Student Course Evaluations

Mean Score of <4 (expectations not met) Mean Score of 4.0 - 4.3 (expectations partially met) Mean Score of >4.3 <=4.5 (expectations met) Mean Score of >4.5 expectations exceeded)

4.47 – Expectations Met 7 Decrease from Fall to Spring of 4.69 to 4.24

Associate of Biblical Studies program assessment

Mean Score of <3 (expectations not met) Mean Score of 3 but <5 (expectations partially met) Mean Score of 5 but <9 (expectations met) Mean Score of >=9 expectations exceeded)

Expectations Met 7.11

AAS Christian Counseling program assessment



Associate of Liberal Arts Program assessment

Mean Score of <3 (expectations not met) Mean Score of 3 but <5 (expectations partially met) Mean Score of 5 but <9

Expectations met 7.14

18

(expectations met) Mean Score of >=9 expectations exceeded)

Indicator 2: Able to think logically and critically with integration. (“Kingdom Think”)

Direct/Indirect Measure Success Indicators Assessment Results Effectiveness Score Comments Student Course Evaluations Mean Score of <4

(expectations not met) Mean Score of 4.0 - 4.3 (expectations partially met) Mean Score of >4.3 <=4.5 (expectations met) Mean Score of >4.5 expectations exceeded)








Associate of Liberal Arts program assessment



19

Indicator 3: Able to communicate (writing, speaking) coherently and

effectively.









Expectations Met 7




Indicator 4: Able to do scholarly research appropriate to undergraduate level.

20



4.57 -Expectations Exceeded 9 Decrease from Fall to Spring of 4.77 to 4.37










Goal B: Program Specific Learning Rationale: The academic program assessment is critical to the success of our evaluation of student learning. Department Chairs and designated faculty hold the primary responsibility in determining the academic success of our

21

students. Faculty also determine the end of program experiences that provide the best evidence to evaluate student level of outcome achievement.

Indicator 1: Associate of Biblical Studies Program Outcome Achievement

Direct/Indirect Measure Success Indicators Assessment Results Effectiveness Score Comments Associate of Biblical Studies

program assessment Mean Score of <3 (expectations not met) Mean Score of 3 but <5 (expectations partially met) Mean Score of 5 but <9 (expectations met) Mean Score of >=9 expectations exceeded)


Indicator 2: AAS Christian Counseling Program Outcome Achievement

Direct/Indirect Measure Success Indicators Assessment Results Effectiveness Score Comments AAS Christian Counseling

program assessment Mean Score of <3 (expectations not met) Mean Score of 3 but <5 (expectations partially met) Mean Score of 5 but <9 (expectations met) Mean Score of >=9 expectations exceeded)

Expectations met 6

Indicator 3: Associate of Liberal Arts Program Outcome Achievement




Indicator 4: ABHE Bible Exam Results

ABHE Year 2 Bible Exam Mean Scores< ABHE mean 22.8% higher than ABHE – 9 2016-17 results:

22

(New Testament) 2017 Graduates

(expectations not met) Mean Scores = to or > ABHE mean by 10% (expectations partially met) Mean Scores> ABHE mean by 11% -20% (expectations met) Mean Scores> ABHE mean by 20 % (expectations exceeded)

Expectations Exceeded PBC: 73.4% ABHE: 50.5%

2015-16 results: PBC: 71% ABHE: 51.3%

ABHE Year 2 Bible Exam (Old Testament) 2017 graduates

Mean Scores< ABHE mean (expectations not met) Mean Scores = to or > ABHE mean by 10% (expectations partially met) Mean Scores> ABHE mean by 11% -20% (expectations met) Mean Scores> ABHE mean by 20% (expectations exceeded)

6.8% higher than ABHE – Expectations Partially Met

5 2016-17 results: PBC: 69.9% ABHE: 63.1%

2015-16 results:

PBC: 76.8% ABHE: 64.3%

II. ASSESSMENT OF INSTITUTIONAL EFFECTIVENESS

Goal C: Student Support Operations Rationale: Student support operations are defined as those operations that promote the academic success and holistic experience of students. Student support services consist of advising and advocacy for students and provide resources, referrals, and information across the PBC community. They are critical to student retention, persistence, and completion.

Indicator 1: Admissions a. High Quality Applicants b. Rapid Turnaround c. Complete Files

Direct/Indirect Measure Success Indicators Assessment Results Effectiveness Score Comments High School Class Standing

Mean standing in bottom 25% or less (expectations not met) Mean standing below top 50% (expectations partially met) Mean standing of top 50% or greater (expectations met) Mean standing of top 25% or greater (expectations exceeded)

2016-17: Top 31%

Expectation Met

2015-16: Top 34.4 Expectation Met

7

Transfer students’ previous college GPA from previous college

Mean GPA of <2.5 (expectations not met) Mean GPA of 3.0 - 2.5

2016-17: 3.01 Expectations Met

2015-16: 3.75

7

23

(expectations partially met) Mean GPA of <3.5 -3.0 (expectations met) Mean GPA of >=3.5 (expectations exceeded)

Expectations Exceeded

First-time students’ GPA for first semester at PBC

Mean GPA of <2.5 (expectations not met) Mean GPA of 3.0 - 2.5 (expectations partially met) Mean GPA of <3.5 -3.0 (expectations met) Mean GPA of >=3.5 (expectations exceeded)

2016-17: 3.25 Expectations Met

2015-16: 3.23

Expectations Met

7

Processing time from applicant to to enrolled (Dan)

Mean processing time of > 18 days (expectations not met) Mean processing time of 15-18 days (expectations partially met) Mean processing time of 11-14 days (expectations met) Mean processing time of <10 days (expectations exceeded

2016-2017: 9.5 days Expectations exceeded 2015-2016: 16 days Expectations partially met

9

Range was 0-58 days. This reflects the total processing time from application submission to notice of an admission decision. Lengthy waits between submission and decisions was typically due to a delay in required documentation (transcript, references ,etc.) Hence tradition undergraduate applications take longest.

Admission's Conversion rate for Prospects (Dan)

Convert 39% or less of our prospects (expectations not met) Convert 40-49% of our prospects (expectations partially met) Convert 50-59% of our prospects (expectations met) Convert > 60% of our prospects (expectations exceeded)

2016-2017: 64% Expectations exceeded 2015-2016: 78% Expectations exceeded

9 Of 36 new credit applicants, 11 were withdrawn for reasons that are not recorded, but generally were designated as such after prolonged lack of contact or no response from applicant. 1 application is incomplete ( w/ no recent correspondence). 1 application is deferred pending DHS approval for student visas.

Percentage of incomplete files (Dan) % of files incomplete are >20% (expectations not met) % of files incomplete are 16 - 20% (expectations partially met) % of files incomplete are 10 - 15%

2016-2017: 17.5% Expectations partially met 2015-2016: 37% Expectations not Met

5 Progress from last year in that a more conscious effort was made to obtain photo release forms. 3 files were missing key enrollment & liability forms (Matriculation or

24

(expectations met) % of files incomplete are <10% (expectations exceeded)

Early College Credit agreement). No clear indication of proximate cause for these missing files; students no longer enrolled.

Student Admission's Satisfaction Survey

Mean Score of <2.5 (expectations not met) Mean Score of <3.0 - 2.5 (expectations partially met) Mean Score of <3.5 -3.0 (expectations met) Mean Score of >=3.5 expectations exceeded)

2016-2017: 4.5. (see below) Expectations exceeded 2015-2016: 3.18 Expectations Met

9 Due to low response rate in admissions survey, a general Student Services survey was administered at the end of the Fall 2016 semester. The two surveys received 5 and 18 responses, respectively for a 43% response rate.

Student Services Survey

Mean Score of <2.5 (expectations not met) Mean Score of <3.0 - 2.5 (expectations partially met) Mean Score of <3.5 -3.0 (expectations met) Mean Score of >=3.5 expectations exceeded)

2016-2017: 4.35 Expectations exceeded 2015-2016: NO survey done in this year

9 See comments in cell above.

Indicator 2: Learning Resources a. Effective Instruction

Direct/Indirect Measure Success Indicators Assessment Results Effectiveness Score Comments Quality of Instruction (Course evaluations) Measures classroom interaction, management and learning environment

Mean Score of <4 (expectations not met) Mean Score of <4.0 – 4.3 (expectations partially met) Mean Score of <4.3 -4.5 (expectations met) Mean Score of >=4.5 expectations exceeded)

4.67 – Expectations Exceeded 9

Quality of Instruction (Course evaluations) Measures teaching methods, assignments, grading and communication

Mean Score of <4 (expectations not met) Mean Score of <4.0 – 4.3 (expectations partially met) Mean Score of <4.3 -4.5 (expectations met) Mean Score of >=4.5 expectations exceeded)

4.60 – Expectations Exceeded 9

25

Course withdrawal rates for credit students

Rate is <40% (expectations not met) Rate is 21-40% (expectations partially met) Rate is 11 - 20% (expectations met) Rate <=10% (expectations exceeded)

2016-2017: 4.1% Expectations exceeded 2015-2016: 11.43% Expectations Met

9 Total 170 enrollments for 2016-

2017 year; 7 total withdrawals.

Student Persistence: Percentage of any credit student during an academic year who persists from one semester to the next (average all three) Fall to Spring Spring to Summer

Rate is <40% (expectations not met) Rate is 40 -49% (expectations partially met) Rate is 50 - 70% (expectations met) Rate >70% (expectations exceeded)

2016-2017: 53% (see below) Expectations met Fall-Spring: 71% Spring-Summer: 35% Cum. Avg.: 53% 2015-2016: 62.5% Fall to Spring: 78% Spring to Summer: 47% Expectations partially met

7 Lower Spring-Summer persistence fits general pattern over multiple academic years.

Student Completion rates Full-time students: Any full-time, first-time students who began between the start of fall semester and the end of summer semester who complete a program within 150% of their program time. (9 semesters for degree seeking students, 5 semesters for certificate seeking students)

Rate < ABHE mean 10% or greater (expectations not met) Rate < ABHE mean by less than 10% (expectations partially met) Rate = ABHE mean or is > ABHE mean by 10% (expectations met) Rate > ABHE mean by more than 10 % (expectations exceeded)

2016-2017 – 0% Expectations not met 2015-2016 -0% 2014-2015 - 0% 2013-2014 - 0% 2012-2013 - 0% 2011-2012 - 40%

3 (ABHE completion rate: 51.7%, November 2016 Annual Statistical Report) No change. Low FT completion rates are not a strong indicator of student success since vast majority are PT. Total full-time students in cohorts: 2011-12 – 5 students 2012-13 - 6 students 2013-14 - 3 students 2014-15 - 4 students 2015-2016 – 3 students 2016-2017 – 4 students

Student Completion rates Part-time students:

Rate < ABHE mean 10% or greater 2016-2017 – 12.5% (projected, see comments)

3 (ABHE completion rate: 51.7%, November 2016 Annual

26

Any part-time, first-time students who began between the start of fall semester and the end of summer semester who complete a program within 150% of their program time. (12 semesters for degree seeking students, 6 semesters for certificate seeking students)

(expectations not met) Rate < ABHE mean by less than 10% (expectations partially met) Rate = ABHE mean or is > ABHE mean by 10% (expectations met) Rate > ABHE mean by more than 10 % (expectations exceeded)

Expectations not met

2015-2016 - 5% - 2014-2015 - 0%

2013-2014 – 5.3% 2012-2013 – 3.7% 2011-2012 -1.6%

Statistical Report) Reflects % of total program students. ’16-’17 based on projected program completion at end of Summer 2017 semester.

Indicator 3: Student Personal a. Effective student support b. High student retention

Direct/Indirect Measure Success Indicators Assessment Results Effectiveness Score Comments Student Retention (Fall 2015) Any first time credit student who begins in the Fall or Spring semester, and who 1) Re-enrolls subsequent Fall semester, or 2) completes a program in the same academic year.

Rate is <40% (expectations not met) Rate is 40 -49% (expectations partially met) Rate is 50 - 70% (expectations met) Rate >70% (expectations exceeded)

Fall 2016: 43.8% Expectations Partially Met

Fall 2015: 30.8% - Expectations

Not Met

5 -Of new enrollees in 2015-2016 (17 total), 1 completed a certificate in the same year and 7 returned in Fall 2016.

Enrollment Growth Measures total credit enrollment growth from academic year to academic year

Rate is <10% (expectations not met) Rate is 10 -19% (expectations partially met) Rate is >20 - 39% (expectations met) Rate >40% (expectations exceeded)

2016-17: 32% Expectations

met

2015-16 11.6% - Expectations Partially Met

8 2016-17: 544 credits 2015-16: 412 credits 2014-15: 369 credits

Enrollment Growth Measures total Audit enrollment growth from academic year to academic year

Rate is <5% (expectations not met) Rate is 5 -10% (expectations partially met) Rate is >10 - 20% (expectations met) Rate >20% (expectations exceeded)

2016-17: 32% Expectations Exceeded 2015-16: 0% Expectations not Met

9 2016-17: 103 credits 2015-16: 78 credits 2014-15: 78 credits

Annual Student population growth

Rate is <10% 2016-17: 20.8% Expectations Met

7 2016-17: 87 2015-16: 72

27

Based on annual unduplicated Headcount and includes audit and credit

(expectations not met) Rate is 10 -19% (expectations partially met) Rate is 20 - 39% (expectations met) Rate >40% (expectations exceeded)

2015-16: 24% - Expectations Met

2014-15: 58

Fall to Fall Enrollment Comparison (Our goal is to grow to 200 unduplicated enrollments by Fall 2021. This measurement provides a picture of progress toward that goal)

Rate is <25% (expectations not met) Rate is 25 - <34% (expectations partially met) Rate is 34% - 40% (expectations met) Rate >40% (expectations exceeded)

2016-17: 32.5 % Expectations Met 2015-16: 0% Expectations Not Met

7 Fall 2016-17: 53 Fall 2015-16: 40 Fall 2014-15: 41

Library – Average Monthly Circulation (Evaluation of learning resource utilization by the learning community)

Avg Circulation (AC) <=2014-15 AC (expectations not met) AC up to 10%>2014-15 AC (expectations partially met) AC >10% and up to 30%>2014-15 AC (expectations met) AC >30% higher than 2014-15 AC (expectations exceeded)

Average Monthly Circulation 2016-17 – 18.67 Expectations not Met Average Monthly Circulation 2015-16 – 19.75 (9.45% increase) Expectations partially met Average Monthly Circulation 2014-15 – 18

4

Goal D: Administrative Operations

Rationale: Administrative operations are those outside of student support that include the President’s office, business office, academic administration, library, and fundraising development/advancement. They are essential to ensuring institutional integrity in all operational aspects of the college.

Indicator 1: President’s Office and Board Governance a. Effective Leadership b. Vision Casting c. Expanded Constituency

Direct/Indirect Measure Success Indicators Assessment Results Effectiveness Score Coordinating and supporting activities that are maximally beneficial to core

Board of Trustees effectiveness rating: Mean Score of <3.5 (expectations not met)

2016-17: 4.50 – Expectations Met

7

28

internal and external stake holders Mean Score of 3.5 but <4.0 (expectations partially met) Mean Score of 4.0 but <4.5 (expectations met) Mean Score of >=4.5 (expectations exceeded)


Model and communicate regularly institutional mission and activities to stakeholders and the community

Board of Trustees effectiveness rating: Mean Score of <3.5 (expectations not met) Mean Score of 3.5 but <4.0 (expectations partially met) Mean Score of 4.0 but <4.5 (expectations met) Mean Score of >=4.5 (expectations exceeded)

2016-17: 4.33 _Expectations Met

2015-16: 4.17 – Expectations

Met

7

Represent the college to community and business groups, accrediting bodies, national organizations, legislators, alumni, donors, and the media.




Exceeded

7

Donor Development Board of Trustees effectiveness rating: Mean Score of <3.5 (expectations not met) Mean Score of 3.5 but <4.0 (expectations partially met) Mean Score of 4.0 but <4.5 (expectations met) Mean Score of >=4.5 (expectations exceeded)



Exceeded

7

29

Facilitate initiatives such as strategic planning, institutional resource development, new program development, community service projects, partnership development with business, industry, and other educational and governmental agencies and institutions.


2016-17: 4.83 – Expectations Exceeded


Exceeded

9

Annual Board Effectiveness Evaluation

Mean Score of <2.5 (expectations not met) Mean Score of 2.5 but <3.5 (expectations partially met) Mean Score of 3.5 but <4.5 (expectations met) Mean Score of >=4.5 (expectations exceeded)



Met

7

Indicator 2: Business Office a. Financial Stability

Direct/Indirect Measure Success Indicators Assessment Results Effectiveness Score Cash Ratio: Cash ratio is the ratio of cash and cash equivalents of a company to its current liabilities. (Cash + Cash Equivalents/Current Liabilities)

<= 2.0 (expectations not met) >2.0 – 3.0 (expectations partially met) >3.0 but <5.0 (expectations met) >5.0 (expectations exceeded)

2016-17: 1.2 – Expectations Not Met


Exceeded

1

Debt/Equity Ratio- Debt-to-Equity ratio is the ratio of total liabilities of an organization to its equity. (Total Liabilities/Equity).

>= 7.5 (expectations not met) <7.5 – 6.0 (expectations partially met) <=6.0 but >=3.5 (expectations met) <3.5. (expectations exceeded)

2016-17: 0.6 – Expectations Exceeded


Met

9

Working Capital: The working capital indicates whether a company has enough short-term assets to cover its shor- term debt (Current Assets-Current Liabilities).

<= $60K (expectations not met) >$60 - $80K (expectations partially met) >$80K but <$150K (expectations met) >$150K (expectations exceeded)

2016-17: $77,396 – Expectations Partially Met

2015-16: $95,939 – Expectations

Met

4

30

Current Ratio: The current ratio is a liquidity and efficiency ratio that measures an organization's ability to pay off its short-term liabilities with its current assets

< 3.0 (expectations not met) 3.0 – 4.0 (expectations partially met) >4.0 but <6.0 (expectations met) >6.0 (expectations exceeded)

2016-17: 1.2 6.0– Expectations Exceeded


Exceeded

9

Financial Responsibility Composite Score-

< 1.0 (expectations not met) 1.0 – 1.49 (expectations partially met) 1.5 - <2.5 (expectations met) >2.5 (expectations exceeded)



Partially Met

7

Indicator 3: Development Office a. Increased Gift Income

Direct/Indirect Measure Success Indicators Assessment Results Effectiveness Score % increase of gift income 10% increase or less

(expectations not met) 11% - 15% (expectations partially met) >=16% - 30% (expectations met) >30% (expectations exceeded)

22.22% Expectations Met

6

2016-17: 246,480 2015-16: $201,664 2014-15: $213,378

% of operating budget supported by gift income. (Use total expenses before depreciation.)

40% or less (expectations not met) >40% - 50% (expectations partially met) >50% - 70% (expectations met) > 70% (expectations exceeded)

2016-17: 70.0% Expectations Exceeded

2015-16: 70.9% Expectations Exceeded

9

Increase in # of Donors 5% increase or less (expectations not met) >5% - 10% (expectations partially met) >10% - 20% (expectations met) > 20% (expectations exceeded)

6% 2016-17: 71 Donors: 7%

Expectations partially met

2015-16: 66 Donors

4

2016-17: 70 Donors 2015-16: 66 Donors 2014-15: 70 Donors

31

FINDINGS AND FUTURE ACTIONS

Goal A: Students are Academically Proficient

1) Student Course Evaluations – Student course evaluations provide positive indication that PBC students generally believe they are making progress toward achieving academic proficiency in line with PBC’s mission. Student perception scores concerning achievement of three qualities of “Academic Proficiency” were in the “expectations met” range and one quality was rated as exceeding expectations. However, there was a reduction in student perception scores in terms of their achievement of the “Academic Proficiency” outcome of PBC’s mission from Fall to Spring. In researching the student evaluations, the lower spring perception scores could be attributed to a single class. Communication was accomplished with the faculty member and the students concerning the class to consider and implement improvements for future courses.

2) ABS Program Assessment – Overall, Assessment results for the current year seem to indicate that Pacific Bible College is “meeting expectations” as regards the demonstrable achievement of PBC’s mission outcome of Academic Proficiency.

3) AASCC Program Assessment – We determined that the AASCC program had “met expectations” of student learning toward achievement of the PBC mission outcome of

“academically proficient”. The mean scores of assignments were well within the “met expectations” level of mastery based on the evaluation of artifacts. However, course evaluations fell into to the “not met expectations” range concerning the student’s perception of their achievement of “Academic Proficiency”.

The student perception issue was considered. The small sample size of students (7 respondents) allowed for large swings of these perception scores. In this case, it was primarily due to a singular student experience in the spring semester PS201 General Psychology course and a singular student experience in the BI102 New Testament Survey courses. A course syllabi review, indicated that the assignments and course structure for each of these classes should have allowed for good achievement of the PBC mission outcome of academic proficiency. Both classes had a substantial writing assignment that would specifically address all four qualities of academic proficiency. Consensus among reviewers was that the biggest weakness noted in terms of academic proficiency was that APA format was present in student assignments but not strictly adhered to in every circumstance.

4) ALA Program Assessment – Our examination of student artifacts indicated achievement of the PBC mission outcome “Academic Proficiency”. (See ALA results below under

program specific learning)

Goal B: Program Specific Learning

1) ABS Program Assessment –

Evaluation of Student Surveys

32

On the surface, it appears that PBC students were highly satisfied with the quality of their experience at the college and most enthusiastic in their unqualified endorsement of the general curriculum’s ability to exceed expectations as regards the outcomes promised by both the PBC Mission Objectives (Academic Proficiency) and the ABS Program Outcomes (Lead Lovingly). A modicum of caution seems indicated however, given the slightly less uniform endorsement arrived at by the faculty evaluation team (as well as by the ABHE Bible Exam results).

Evaluation of the Student Artifacts:

The artifacts support the conclusion that PBC’s Mission Objective of “Academic Proficiency” is being strongly and consistently met with Assessment Results routinely hovering at or around 7.0 (indicating “Mostly accomplished”). Results were mixed however for the evidence indicating a strong achievement of the ABS Program Outcome “Ability to Lead Lovingly”. Although overall averages for even the lowest scoring “Indicator” (“able to correct and admonish in discipline”) remained just within the range of “Meets Expectations” (at 5.0), scores for individual artifacts often dipped into the “Expectations Partially Met” area. Discussion among the evaluating faculty addressed the inherent difficulties of quantifying certain aspects of spiritual growth and behavior, arriving at the suggestion that the “Indicator” be examined with an eye towards possibly modifying its accessibility to future outcome measurement or else purposing curricular change to more robustly demonstrate success in achieving that outcome. Recommendation: That faculty meetings already scheduled for instructors teaching Theology Dept. curricula be purposefully directed towards examining the difficulties & tension attending this issue with the purpose of identifying mechanisms for assessment which might prove capable of better bridging the tension between subjective goals and quantifiable measurements.

General Conclusions:

Overall, Assessment results for the current year seem to indicate that Pacific Bible College is “meeting expectations” as regards the demonstrable achievement of the learning outcomes promised by its ABS degree program. Although continued discussion on curricular integration and creating common context and vision among faculty (especially with the addition of so many new adjunct members) remains a fruitful area of endeavor, we do not recommend any major changes in current class offerings or curricular content at this time.

Note on Curricular Coherence:

While no major restructuring seems indicated, there was a consensus that continued effort was needed to tie curricular content more consciously and purposefully to our learning outcomes. Towards this end, department heads and program directors will optimize opportunities for faculty to discuss and outline means and methods towards achieving a more integrated approach to curriculum creation and communicating that integration in syllabi, assignments, and pedagogy.

Other Observations:

During the evaluation process, it was noted that additional modifications to the current methods of artifact collection would prove fruitful. Specifically, an emphasis on vetting artifact specimens for completeness prior to collection & storage (digital storage has proved most useful), as would the addition of another type of scoring rubric for less objective artifacts (such as research and reflection papers). On a more abstract level, it was gratifying to examine potentially intractable outcomes such as “Lead Lovingly” and succeed in at least “approximating” an objective quantification for the purposes of assessing curricular efficacy.

2) AASCC Program Assessment –We determined that the AASCC program had “met expectations” of student learning toward achievement of the AASCC program objective of “developing practical counseling strategies through the study of biblical, theological, and psychological foundations”. However, student course evaluations fell into to the “not met expectations” range concerning the student’s perception of their achievement of the AASCC program objective.

33

The student perception issue was considered. The small sample size of students (7 respondents) allowed for large swings of these perception scores. In this case, the lower student perception scores were primarily due to a singular student experience in the spring semester PS201 General Psychology course and a singular student experience in the BI102 New Testament Survey course. Our reviewed of these two courses showed the following: The PS201 course had the assignments and course structure needed to achieve the objective. The self-reflection/research paper very specifically asked the student to discuss how the secular concepts/theories aligned with scriptural truth. In addition, the students were asked to apply these findings to their personal life which provided good opportunity for higher-order thinking skills of evaluation/synthesis. While the BI102 course does not directly address counseling strategies, it does provide a biblical and theological framework that should influence a student’s application of counseling strategies. The final determination was that the student evaluations that pushed the perception scores into the “expectations not met” range were possibly outliers. However, the department chair will review with the faculty member involved with the PS201 course and consider potential actions to better enhance student perception.

Curriculum Coherence:

The curriculum was reviewed and changes were made to the course structure of the AASCC degree in response to ABHE’s changes to Standard 11. The counseling faculty determined to add a science requirement of BI211 General Biology I in the General Education requirements to bring the program into compliance with the changes to Standard 11. The AASCC program objectives were also re-written as student learning outcome statements and these new outcomes will be reflected in the 2017-18 Catalog and Handbook.

Other Observations:

The AASCC program should continue to address, review, and strengthen teaching style of the faculty members with regards to biblical integration. Some considerations that were discussed were the integration of more oral presentations during the course works and the development of a process of that would enable students to complete a final portfolio by the end of their program. The Counseling department will meet to discuss these possible actions before the end of the 2017 Fall Semester.

3) ALA Program Assessment – Our examination of student artifacts indicated achievement of the PBC mission outcome (academic proficiency) and the designated ALA learning outcome with some caveats concerning Christian worldview achievement in the categories of aesthetics and community. We still had more work to do. This work falls into 2 categories:

a. Course content.

i. Improve our ability to design assessment tools/assignments within each course that will more clearly demonstrate the level of accomplishment for each student. We found some samples too brief for adequate assessment, or some assignments more rote in nature than that which would give us a good idea of students’ integrative thought. Department Chair has offered consultation with department members to accomplish this goal.

ii. Next year we will be concentrating on the PBC mission outcome that deals with spiritual cultivation, so, in this more secular liberal arts program, we will need to do an aggressive job of integrating Biblical concepts into those courses and their assessment tools. Some brainstorming sessions will have to occur here. How does one address and evaluate the spiritual integration of a math student? Again, the Department Chair will meet with department faculty and discuss possible options.

b. Assessment procedures.

i. Improve faculty understanding of the importance of assessment tools and artifacts.

34

ii. Rewrite ALA program outcomes in terms that are simplified and more measurable. This will involve development of official definitions for the more abstract terms in the outcomes

iii. Collect more artifacts from more instructors.

iv. Settle on a uniform method of sampling.

Curriculum Coherence:

After the year one review, it was determined to reduce the ALA Science requirement from 12 credits to 8 credits. This change was made due to the logistics involved with running three different science classes a year to meet the requirements of enabling a student to finish in six semesters. In addition, eight semester hours of science is well aligned with the norms of science requirements in two-year programs focused on providing transfer opportunities for students to baccalaureate level programs. This change also allowed the creation of a three-credit elective requirement that will provide students with some flexibility and choice in directing their program courses to completion. Name changes were also accomplished for the Political Thought I and II courses. The new course names better describe the content and should build a higher level of student intrigue and interest.

4) ABHE Bible Exam Results – This year we had two of our program graduates take the ABHE two-year Bible exam to compare their test scores with the ABHE’s mean scores. One of the students had completed our Associate of Biblical Studies and one had completed our Certificate in Christian Ministries. While the performance of PBC graduates regarding their general grasp of New Testament knowledge was indeed laudable (exceeding expectations at 22% above the mean ABHE scores), the display of Old Testament expertise was somewhat less lustrous (only partially meeting expectations at 6.8% above the ABHE mean score). An interesting aspect of this seeming discrepancy appears in the overall breakdown of specific subjects within each of the Testaments. While PBC student performance routinely exceeded ABHE means in almost all subject areas there was a noticeable decline in reference to the Old Testament Prophets, wherein PBC scores fell significantly behind ABHE mean scores (30% vs 52%). A similar, though less severe, gap was also evident for “OT Timeline” (80% vs 82%). Recommendation: That attention to these specific shortfalls be given in the future presentation of curriculum for the BI101 OT Survey and related classes. In addition, we still have not developed an entry cohort of first-year students to take the bible-exam so we can measure progress of these same students at completion. We should continue to develop a process to capture these students on entry so we can measure individual progress.

Goal C: Student Support Operations

1) Admissions: High school class standing for incoming students remained steady with applicants having a mean standing of the top third of their high school class. The average transfer student GPA decreased from 3.75 in 2015-16 to 3.01 in 2016-17. In reviewing the academic performance of transfer students in the 2016-17 academic year, there was not an identifiable negative effect or trend related to the drop in incoming mean GPA. First semester student GPA’s held steady with almost an identical mean GPA to 2015-16.

Processing time for incoming applicants was significantly improved having reduced from 16 days to 9.5 days. Our admissions conversion rate, was slightly less than last year but still exceeded our expectations with greater than 60% of prospects converted to enrolled students. Progress was also realized in complete student files with the percentage of incomplete files being cut by more than 50 % from the rate recorded in 2015-16. We still only partially met expectations with 17.5% of student files being incomplete so we will continue to conduct process improvement to reduce these percentages into the desired range for the meeting expectations standard. Both our Student Admission’s Satisfaction Survey and Student Support Services survey indicated high student satisfaction and we exceeded the expectations we had set regarding these two important surveys.

35

Overall, we saw a substantial improvement regarding direct/indirect measures in the admission’s indicator of successful student support operations. An area of improvement that will be targeted for 2017-18 will be our process for completing student files. Our goal is to lower our incomplete percentage into our “met expectations” range of 10%-15%.

2) Learning Resources

This was another strong year for effective instruction with student course evaluations achieving means scores that exceeded expectations in both categories of quality of instruction. Course withdrawal rates were reduced from 11.43% into an “expectations exceeded” range of 4.1%. We believe this is a good indicator of quality instruction and positive interaction with faculty members. In addition, it could also be an indication of success with our new student intervention process. We incorporated the academic warning system in the Populi college management system into our student intervention work. Populi notifies the Dean of Students immediately during the semester if a student underperforms academically in a single event. This allows us to take intervention measures early on when a student begins to struggle academically. As a result, we worked successfully with students during the year because we were aware of academic issues very early on in their semester of work.

The student persistence rate for the Fall to Spring semester was less than the 2015-16 rate by 7%. The Spring to Summer semester persistence rate was 12% lower than 2015-16. The cumulative average for the year was still in the “met expectations” range but we analyzed the possible causes of our lower persistence rates. One factor that contributed was the growth of the high school age dual-credit students that attended PBC. These students often have very specific courses they are looking to complete to support their home school program or they are part of a high school that requested a specific course for their respective course offerings. Once these requirements are completed, these students often don’t return in the subsequent semester as they have met their individual goals. In addition, this group of students rarely participates in classes over the summer as this is their typical academic break. We had 11 students, who took a dual-credit class with New Hope Christian School in the Spring of 2017 and none of these students took classes over the summer. This was the primary reason for the Spring to Summer persistence rate decrease. The Fall to Spring reduction was much smaller, but can also be attributed partially to the dual-credit students as we had several that participated in the Fall but did not come back in the Spring.

Completion rates for full-time students are still 0%. Our part-time student completion rate increased to 12.5% (from 2015-16’s 5%) which we believe is a result of the significant positive change in persistence rates that began in 2015-16. The uptick in persistence can be directly attributed to the addition of full-time administrative positions that were added on the 2015-16 academic year. We believe completion rates will begin to show increases in 2017-18 and beyond as we are now able to support students with Federal Financial Aid which should bolster the number of full-time students. We will continue to monitor and improve with the goal of exceeding ABHE’s mean completion rate for institutions under their accrediting umbrella.

Student Personal:

1) Student Retention Rates – The college fell short of its stated goal for 50% student retention but had a 13% increase over the 2015-16 rate. Regarding retention, it is to be noted that of 17 first-time students in 2015-2016, one graduated and 7 were retained in Fall 2016-2017. The following trends were observed:

36

In addition, exit reasons have been identified for all students enrolled in a specific degree program who have not enrolled in or otherwise completed a class for two consecutive semesters. Out of 34 active program students, 6 (16.2%) withdrew mid-year (4 for personal reasons, 1 for church mission, and 1 for financial reasons) and 1 completed program requirements mid-year. These exit reasons have been recorded in Populi and will be available for reference when needed. Overall, we saw significant growth in student credit and audit enrollment 32%, annual unduplicated headcount (20.8%) and Fall to Fall enrollment comparison (32.5%). These are all very positive growth trends and provide good evidence of solid Student Personal support. While we met our strategic planning target of 50 (51 actually enrolled) total students enrolled in Fall term of 2016, we will need a substantial growth rate of 31% to achieve our targeted 67 students for the Fall of 2017.

Given the above analysis, the OIE determined the following actions:

a) Continue to improve and resource student advising and support services.

b) Continue efforts to track student withdrawal/exit reasons and to follow up with non-retained students whose reason for leaving is unknown.

c) Follow recruiting/enrollment/retention tactics in the 2017-18 enrollment management plan.

d) Consider and develop better retention strategies for dual-credit students.

f) Hire a new student development staff member for student services. (The position has gone unfilled since June of 2016)

Library Average Monthly Circulation (AMC) – When conducting the research for the AMC calculation for 2016-17 it was discovered that last year’s calculation was incorrect. Therefore, the AMC for 2015-16 should have been 19.75 instead of 28. In light of the new AMC total for 2015-16, the AMC for the library slightly decreased in 2016-17. A major factor that influenced the lack of growth of AMC is the absence of a regular Librarian for the 2016-17 academic year. With the designation of new Librarian in June of this year, we believe 2017-18 will see an increase in student usage of library resources.

37

Goal D: Administration Operations

1) President’s Office and Board Governance – The Trustees provided an assessment of primary functions of the President’s office. Trustees gave four categories of performance high marks with scores ranging from “met” to “exceeded” expectations in all categories of effectiveness. The category of Donor Development was rated as “partially” meeting expectations. Our expectation is that this perception will change next year due to our newly hired Chief Development Officer. His efforts are already creating new donors and an increase in regular monthly donors.

Board Evaluations this year saw a general improvement in the Trustee’s perceptions of their effectiveness. In 2015-16, there were 5 areas of performance that were rated as “partially meeting” expectations but the 2016-17 evaluations had only two areas of performance in this category. It is also interesting that none of the identified areas of performance from last year were repeated on this year’s assessment. Those areas in 2016-17 that had an average rating of “partially meeting” expectations were the following:

a) Fundraising Efforts – Does the board take leadership in PBC’s fundraising efforts through networking, inviting friends to events, and personal giving?

b) Board Size – Is the Board sufficient in size and with qualified members to accomplish its tasks effectively?

The Board of Trustees “overall effectiveness” score was improved from last year and fell into the “expectations met” range. As our OIE team considered areas of improvement, we determined that we need to accomplish the following actions with respect to the President’s Office and Board Governance in 2017-18:

a) Continue to conduct our board development events. The Trustee’s perception of their effectiveness increased. We believe this is partially due to the board retreats that have been conducted over the last two years.

b) Continue to enhance the communication process with the President. This year, we added an Office Manager staff person who has direct responsibility for facilitating communications with the board, organizing the meeting documentation and requirements, and organizing board policies, agendas and meeting notes. The addition of this new staff position should significantly enhance the board function and communications.

2) Business Office and Development Office –We increased revenue by a total of 25%, realizing a 29% increase in tuition revenue and an 22.2% increase in contributions. Our total

revenue was -4.1% of the Strategic Plan, although tuition (-17%) and contributions (-8.7%), fell short. The proceeds of the building sale helped offset the shortfall as our actual

expenses were -2% of projections. The sale of our property allowed us to eliminate all long-term debt and increase the Composite Financial Score to an acceptable level of 1.94.

Donor development showed weakness and led to the hiring of a Chief Development Officer on June 1, 2017. The weaknesses were on number of non-returning donors from the

previous year (20), and the smaller than planned increase in new donors. Further we again missed our fundraising objectives. Transferring this work from the president to the full-

time development officer should improve these weak performance areas. The initial efforts have been to mobilize all the PBC community and establish a new donor drive with a goal

of 200 by calendar year end. If achieved this will position the institution well for cash flow.

38

Realizing federal financial aid eligibility for our students should positively impact our cash position as well. Although premature at the time of this report, initial recruiting

efforts have been bolstered by the availability of Pell Grants. The increase in enrollment should then increase the revenue from this stream. Moving into 2017-18, we have increased

staffing and hence our expenses are budgeted for a 55% increase. We will continue to use the same Business Office Measurement Indicators for financial stability assessment. For

donor development assessment, we will adopt the following indicators:

1. % Increase of revenue from Gift/Contributions/Fundraisers

a. =<50%, Expectations Not met

b. 50%<X<75%, Expectations Partially Met

c. 75%=<X=<100%, Expectations Met

d. >100%, Expectations Exceeded

2. Increase in number of Donors

a. <100, Expectations Not met

b. 100=<X<150, Expectations Partially Met

c. 150=<X<200, Expectations Met

d. =>200, Expectations Exceeded

3. Reduction in % Non-Repeating Donors

a. >20%, Expectations Not met

b. 20%=>X=>12%, Expectations Partially Met

c. 12%>X>4%, Expectations Met

d. =<4%, Expectations Exceeded

39

Summary of Actions identified above to be accomplished in 2017-18

1) Student Course Evaluations – Continue to collect student perception information in the Fall and Spring semesters. The focus for improvement in the 2017-18 academic year will be to increase our respondent’s rate.

Responsible Person(s): CAO and Dean of Students

2) ABHE Bible Exam Results – The Bible and Theology faculty leadership will examine the ABHE Bible Exam Old Testament section and consider where the biblical studies curriculum can be adjusted to address. One of the initiatives, that will be accomplished this year is to add some stand-alone Old Testament course offerings beginning with a Pentateuch course in the 2018 spring semester. We still have not fully-developed an entry cohort of first-year students to take the bible-exam so we can measure progress of these same students at completion. We will continue to develop a process to capture these students on entry so we can measure individual progress.

Responsible Person(s): CAO, Bible and Theology Department Chair & designated Bible/Theology faculty member

3) ABS Program – Faculty meetings already scheduled for instructors teaching Theology Dept. curricula will be leveraged to gain better input on artifacts to identify mechanisms for assessment which will better bridge the tension between subjective goals and quantifiable measurements. Continued effort is needed to tie curricular content more consciously and purposefully to the ABS learning outcomes. Towards this end, department heads and program directors will optimize opportunities for faculty to meet and discuss means and methods towards achieving a more integrated approach to curriculum creation and communicating that integration in syllabi, assignments, and pedagogy.

Responsible Person(s): Bible and Theology Department Chair and Biblical Studies Program Instructors

4) AASCC Program - The AASCC program needs to address, review, and strengthen teaching style of the faculty members with respect to biblical integration. Some considerations that were discussed were the integration of more oral presentations during the course works and the development of a process of that would enable students to complete a final portfolio by the end of their program. The Counseling department will meet to discuss these possible actions before the end of the 2017 Fall Semester.

Responsible Person(s): Counseling Department Chair

5) ALA Program– The General Studies department chair and program faculty will accomplish the following two categories of work concerning the ALA program:

40

a. Course content.

i. Improve the design of assessment tools/assignments within each course to more clearly demonstrate the level of accomplishment for each student. We found some samples too brief for adequate assessment, or some assignments more rote in nature than that which would give us a good idea of students’ integrative thought. Department Chair has offered consultation with department members to accomplish this goal.

ii. Next year we will be concentrating on the PBC mission outcome that deals with spiritual cultivation, so, in this more secular liberal arts program, we will need to do an aggressive job of integrating Biblical concepts into those courses and their assessment tools. Some brainstorming sessions will have to occur here. How does one address and evaluate the spiritual integration of a math student? The Department Chair will meet with department faculty and discuss possible options.

b. Assessment procedures.

i. Improve faculty understanding of the importance of assessment tools and artifacts.

ii. Rewrite ALA program outcomes in terms that are simplified and more measurable. This will involve development of official definitions for the more abstract terms in the outcomes

iii. Collect more artifacts from more instructors.

iv. Settle on a uniform method of sampling.

Responsible Person(s): General Studies Department Chair

6) Student Support Operations - The following actions have been designated for accomplishment in the 2017-18 academic year:

a) Continue to improve and resource student advising and support services.

b) Continue efforts to track student withdrawal/exit reasons and to follow up with non-retained students whose reason for leaving is unknown.

c) Follow recruiting/enrollment/retention tactics in the 2017-18 enrollment management plan.

d) Consider and develop better retention strategies for dual-credit students.

f) Hire a new student development staff member for student services. (The position has gone unfilled since June of 2016)

g) Better integrate library services to increase the AMC from previous academic year.

Responsible Person(s): Dean of Students and Enrollment Services

41

Goal D: Administration Operations

1) President’s Office and Board Governance – As our OIE team considered areas of improvement, we determined that we need to accomplish the following actions with respect to the President’s Office and Board Governance in 2017-18:

a) Continue to conduct our board development events. The Trustee’s perception of their effectiveness increased. We believe this is partially due to the board retreats that have been conducted over the last two years.

b) Continue to enhance the communication process with the President. This year, we added an Office Manager staff person who has direct responsibility for facilitating communications with the board, organizing the meeting documentation and requirements, and organizing board policies, agendas and meeting notes. The addition of this new staff position should significantly enhance the board function and communications.

Responsible Person(s): President, Board Chair, and Office Manager

2) Business Office and Development Office – As our OIE team considered areas of improvement, we determined that we need to accomplish the following actions with respect to

the President’s Office and Board Governance in 2017-18:

a. CDO will implement strategies to:

i. Increase our revenue for Gifts/Contributions/Fundraiser by 75%.

ii. Increase the number of donors to 150.

iii. Reduce the number of non-repeating donors to less than 12%.

Responsible Person(s): Chief Development Officer

Documents

2016-2017 Summative Assessment Report · address enrollment, persistence and completion for the AASCC program: a. Establish a faculty member who will actively promote the program